La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America
Universidad Politécnica de Madrid and Universidad Nacional de Educación a Distancia · Barcelona Supercomputing Center · HuggingFace · Universidad Politécnica de Madrid · Universidad Carlos III de Madrid · Instituto de Ingeniería del Conocimiento · University of the Basque Country · IIC · Facultad de Ingeniería - Universidad de la República - Uruguay · Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas - UNAM · Universidad Nacional de Córdoba · Universidad Autónoma de Madrid · Leiden University · Universidad de Jaén · Neovalle Ltd · Universidad del País Vasco
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。DOI 10.18653/v1/2025.acl-long.1561 ↗
摘要
Leaderboards showcase the current capabilities and limitations of Large Language Models (LLMs). To motivate the development of LLMs that represent the linguistic and cultural diversity of the Spanish-speaking community, we present La Leaderboard, the first open-source leaderboard to evaluate generative LLMs in languages and language varieties of Spain and Latin America. La Leaderboard is a community-driven project that aims to establish an evaluation standard for everyone interested in developing LLMs for the Spanish-speaking community. This initial version combines 66 datasets in Catalan, Basque, Galician, and different Spanish varieties, showcasing the evaluation results of 50 models. To encourage community-driven development of leaderboards in other languages, we explain our methodology, including guidance on selecting the most suitable evaluation setup for each downstream task. In particular, we provide a rationale for using fewer few-shot examples than typically found in the literature, aiming to reduce environmental impact and facilitate access to reproducible results for a broader research community.