We invite submissions to the first edition of the MOOMIN workshop on Modular and Open Multilingual NLP, to be held at EACL 2024 on March 21, 2024. Authors of papers rejected to EACL may submit their ARR-reviewed papers through the link above. Findings authors can also use the same link if they wish to present their paper at MOOMIN.
Important Dates
- Workshop paper due: Extended to December 22, 2023
- Commitment deadline (for pre-reviewed ARR & main conference submissions): January 20, 2024 [extended]
- Notification of acceptance: January 22, 2024
- Camera-ready papers due: January 30 2024
- Workshop dates: March 21, 2024
Link to the submission website for ARR commitment
Link to the submission website for direct reviewing
Introduction
NLP in the age of monolithic large language models starts to hit the limits in terms of size and information that can be handled. The trend goes to modularization, a necessary step into the direction of designing smaller sub-networks and components with specialized functionality. This allows researchers to design scalable, wide-coverage, efficient and reusable models.
Multilingual NLP is today faced with a number of difficult challenges. Scaling a multilingual model to a high number of languages is prone to suffer from negative interference, also known as the curse of multilinguality, leading to degradation in per-language performance, while earlier approaches to improving model capacity have hit the ceiling in terms of hardware, data and training algorithms. At the same time, we as a community wish to foster the development of open components that can be shared, deployed and widely integrated within the broader research community without incurring computational costs that add to the overall carbon footprint of NLP engineering. Modularity is a practical solution to answer all of these challenges and more, as it offers a very promising set of tools towards increased multilinguality of larger foundation models, either during their pretraining or in a post-hoc post-pretraining manner.
Topics of Interest
With this in mind, the MOOMIN workshop invites contributions related but not limited to the following topics:
- mixture of expert models and gated routing
- modular pre-training of multilingual language and translation models
- effective transfer with modular architectures such as adapters and hypernetworks
- efficient parallelization and distribution of modular model training
- modular frameworks and architecture implementations
- massively multilingual models with large language coverage
- subnet selection and pruning
- modular distillation
- modular extensions of existing NLP models systems, especially in low-resource settings and for low-resource languages
- evaluation of modular systems in terms of performance, efficiency, and computational costs
- platforms for distributing, sharing, and integrating NLP components
Submission Guidelines
Authors are invited to submit original and unpublished research papers in the following categories:
- Full papers (up to 8 pages) for substantial contributions
- Short papers (up to 4 pages) for ongoing or preliminary work.
NB: As per usual *ACL policies, authors of accepted papers are allowed 1 extra page (total of 9 pages for long and 5 pages for short) to account for reviewers’ comment in their final camera-ready version.
All submissions must be in PDF format, submitted electronically via OpenReview and should follow the EACL 2024 formatting guidelines (following the ARR CfP: use the official ACL style templates, which are available here).