# Uploaded model - **Developed by:** sandbox-ai - **License:** apache-2.0 - **Finetuned from model :** unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit ## Evaluation Metrics |Task |Name |Description |Language|Metric |Task type | |--------------------------------------------------------------------------------------------------------|-------------------|-----------------------------------------------------------------------|--------|--------------|------------------------------------------| |[AQuAS](https://huggingface.co/datasets/IIC/AQuAS) |AQuAS |Abstractive Question-Answering in Spanish |ES |sas_encoder |Abstractive QA | |[ARC_ca](https://huggingface.co/datasets/projecte-aina/arc_ca) |ARC_ca |Grade-school level science questions in Catalan |CA |acc |Multi choice QA | |[BEC2016eu](https://huggingface.co/datasets/orai-nlp/basqueGLUE) |BEC2016eu |Basque Election Campaign 2016 Opinion Dataset |EU |f1 |Sentiment Analysis | |[Belebele Glg](https://huggingface.co/datasets/facebook/belebele) |Belebele Glg |Reading Comprehension in Galician |GL |acc |Reading Comprehension | |[BertaQA](https://huggingface.co/datasets/HiTZ/BertaQA) |BertaQA |Trivia dataset with global and local questions about the Basque Country|EU |acc |Multi choice QA | |[BHTCv2](https://huggingface.co/datasets/orai-nlp/basqueGLUE) |BHTCv2 |Topic Classification of News Headlines in Basque |EU |f1 |Classification, Topic Classification | |[caBREU](https://huggingface.co/datasets/projecte-aina/caBreu) |caBREU |Article Summarization in Catalan |CA |bleu |Summarization | |[CatalanQA](https://huggingface.co/datasets/projecte-aina/catalanqa) |CatalanQA |Extractive QA in Catalan |CA |f1 |Extractive QA | |[CatCoLA](https://huggingface.co/datasets/nbel/CatCoLA) |CatCoLA |Linguistic Acceptability in Catalan |CA |mcc |Linguistic Acceptability | |[ClinDiagnosES](https://huggingface.co/datasets/LenguajeNaturalAI/ClinDiagnosES) |ClinDiagnosES |Diagnosis of clinical cases in Spanish |ES |sas_encoder |Open QA | |[ClinTreatES](https://huggingface.co/datasets/LenguajeNaturalAI/ClinTreatES) |ClinTreatES |Treatment for clinical cases in Spanish |ES |sas_encoder |Open QA | |[COPA_ca](https://huggingface.co/datasets/projecte-aina/COPA-ca) |COPA_ca |Choice Of Plausible Alternatives in Catalan |CA |acc |Reasoning | |[CoQCat](https://huggingface.co/datasets/projecte-aina/CoQCat) |CoQCat |Conversational Question Answering in Catalan |CA |f1 |Extractive QA | |[Crows Pairs Spanish](https://huggingface.co/datasets/multilingual-crows-pairs/multilingual-crows-pairs)|Crows Pairs Spanish|Bias evaluation using stereotypes |ES |pct_stereotype|Bias Detection | |[EpecKorrefBin](https://huggingface.co/datasets/orai-nlp/basqueGLUE) |EpecKorrefBin |Coreference resolution in Basque |EU |acc |Coreference Resolution, Textual Entailment| |[EsCoLA](https://huggingface.co/datasets/nbel/EsCoLA) |EsCoLA |Spanish Corpus of Linguistic Acceptability |ES |mcc |Linguistic Acceptability | |[EusExams](https://huggingface.co/datasets/HiTZ/EusExams) |EusExams |Public Service examinations questions in Basque |EU |acc |Multi choice QA | |[EusProficiency](https://huggingface.co/datasets/HiTZ/EusProficiency) |EusProficiency |C1-level proficiency questions in Basque |EU |acc |Multi choice QA | |[EusReading](https://huggingface.co/datasets/HiTZ/EusReading) |EusReading |EGA exams reading comprehension in Basque |EU |acc |Multi choice QA | |[EusTrivia](https://huggingface.co/datasets/HiTZ/EusTrivia) |EusTrivia |Trivia questions in Basque |EU |acc |Multi choice QA | |[Fake News ES](https://huggingface.co/datasets/mariagrandury/fake_news_corpus_spanish) |Fake News ES |Fake News Detection in Spanish |ES |acc |Classification | |[GalCoLA](https://huggingface.co/datasets/proxectonos/galcola) |GalCoLA |Galician Corpus of Linguistic Acceptability |GL |mcc |Linguistic Acceptability | |[HumorQA](https://huggingface.co/datasets/LenguajeNaturalAI/HumorQA) |HumorQA |White humour joke classification |ES |acc |Classification | |[MGSM_ca](https://huggingface.co/datasets/projecte-aina/mgsm_ca) |MGSM_ca |Grade-school math problems in Catalan |CA |exact_match |Math Reasoning | |[MGSM_es](https://huggingface.co/datasets/juletxara/mgsm) |MGSM_es |Grade-school math problems in Spanish |ES |exact_match |Math Reasoning | |[MGSM_eu](https://huggingface.co/datasets/HiTZ/MGSM-eu) |MGSM_eu |Grade-school math problems in Basque |EU |exact_match |Math Reasoning | |[MGSM_gl](https://huggingface.co/datasets/proxectonos/mgsm_gl) |MGSM_gl |Grade-school math problems in Galician |GL |exact_match |Math Reasoning | |[NoticIA](https://huggingface.co/datasets/Iker/NoticIA) |NoticIA |A Clickbait Article Summarization Dataset in Spanish |ES |rouge1 |Summarization | |[OffendES](https://huggingface.co/datasets/SINAI/OffendES) |OffendES |Clasificación de comentarios ofensivos en español |ES |acc |Classification | |[OpenBookQA_ca](https://huggingface.co/datasets/projecte-aina/openbookqa_ca) |OpenBookQA_ca |Multi-step reasoning QA in Catalan |CA |acc |Reasoning | |[OpenBookQA_gl](https://huggingface.co/datasets/proxectonos/openbookqa_gl) |OpenBookQA_gl |Multi-step reasoning QA in Galician |GL |acc |Reasoning | |[Parafraseja](https://huggingface.co/datasets/projecte-aina/Parafraseja) |Parafraseja |Paraphrase identification in Catalan |CA |acc |Paraphrasing | |[ParafrasesGL](https://huggingface.co/datasets/proxectonos/parafrases_gl) |ParafrasesGL |Paraphrase identification in Galician |GL |acc |Paraphrasing | |[PAWS_ca](https://huggingface.co/datasets/projecte-aina/PAWS-ca) |PAWS_ca |Paraphrase Adversaries from Word Scrambling in Catalan |CA |acc |Paraphrasing | |[PAWS-X_es](https://huggingface.co/datasets/google-research-datasets/paws-x) |PAWS-X_es |Paraphrase Adversaries from Word Scrambling in Spanish |ES |acc |Paraphrasing | |[PAWS_gl](https://huggingface.co/datasets/proxectonos/PAWS-gl) |PAWS_gl |Paraphrase Adversaries from Word Scrambling in Galician |GL |acc |Paraphrasing | |[PIQA_ca](https://huggingface.co/datasets/projecte-aina/piqa_ca) |PIQA_ca |Physical Interaction QA in Catalan |CA |acc |Reasoning | |[QNLIeu](https://huggingface.co/datasets/orai-nlp/basqueGLUE) |QNLIeu |Textual Entailment in Basque |EU |acc |NLI, Textual Entailment | |[RagQuAS](https://huggingface.co/datasets/IIC/RagQuAS) |RagQuAS |Retrieval-Augmented-Generation and Question-Answering in Spanish |ES |sas_encoder |Abstractive QA | |[SIQA_ca](https://huggingface.co/datasets/projecte-aina/siqa_ca) |SIQA_ca |Social Interaction QA in Catalan |CA |acc |Reasoning | |[SpaLawEx](https://huggingface.co/datasets/LenguajeNaturalAI/examenes_abogacia) |SpaLawEx |Spanish Law School Access Exams |ES |acc |Multi choice QA | |[SummarizationGL](https://huggingface.co/datasets/proxectonos/summarization_gl) |SummarizationGL |Abstractive Summarization in Galician |GL |bleu |Summarization | |[TE-ca](https://huggingface.co/datasets/projecte-aina/teca) |TE-ca |Textual Entailment in Catalan |CA |acc |Textual Entailment | |[TELEIA](https://huggingface.co/datasets/gonzmart/teleia) |TELEIA |Test de Español como Lengua Extranjera para Inteligencia Artificial |ES |acc |Multi choice QA | |[VaxxStance](https://huggingface.co/datasets/orai-nlp/basqueGLUE) |VaxxStance |Stance detection on the Antivaxxers movement |EU |f1 |Sentiment Analysis, Stance Detection | |[WiCeu](https://huggingface.co/datasets/orai-nlp/basqueGLUE) |WiCeu |Word sense disambiguation in Basque |EU |acc |Textual Entailment | |[WNLI_ca](https://huggingface.co/datasets/projecte-aina/wnli-ca) |WNLI_ca |Winograd-schema-type dataset in Catalan |CA |acc |NLI, Textual Entailment | |[WNLI ES](huggingface.co/datasets/PlanTL-GOB-ES/wnli-es) |WNLI ES |Winograd-schema-type dataset in Spanish |ES |acc |NLI, Textual Entailment | |[XCOPA_eu](https://huggingface.co/datasets/HiTZ/XCOPA-eu) |XCOPA_eu |Choice Of Plausible Alternatives in Basque |EU |acc |Reasoning | |[XNLI_ca](https://huggingface.co/datasets/projecte-aina/xnli-ca) |XNLI_ca |Cross-lingual Natural Language Inference in Catalan |CA |acc |NLI, Textual Entailment | |[XNLI_es](https://huggingface.co/datasets/facebook/xnli) |XNLI_es |Cross-lingual Natural Language Inference in Spanish |ES |acc |NLI | |[XNLI_eu](https://huggingface.co/datasets/HiTZ/xnli-eu) |XNLI_eu |Cross-lingual Natural Language Inference in Basque |EU |acc |NLI, Textual Entailment | |[XQuAD_ca](https://huggingface.co/datasets/projecte-aina/xquad-ca) |XQuAD_ca |Cross-lingual Question Answering Dataset in Catalan |CA |f1 |Extractive QA | |[XQuAD_es](https://huggingface.co/datasets/google/xquad) |XQuAD_es |Cross-lingual Question Answering Dataset in Spanish |ES |f1 |Extractive QA | |[xStoryCloze_ca](https://huggingface.co/datasets/projecte-aina/xstorycloze_ca) |xStoryCloze_ca |Narrative completion in Catalan |CA |acc |Reasoning | |[xStoryCloze_es](https://huggingface.co/datasets/juletxara/xstory_cloze) |xStoryCloze_es |Narrative completion in Spanish |ES |acc |Reasoning | |[xStoryCloze_eu](https://huggingface.co/datasets/juletxara/xstory_cloze) |xStoryCloze_eu |Narrative completion in Basque |EU |acc |Reasoning | ## Usage: You can use the model using HuggingFace Transformers library with 2 or more 80GB GPUs (NVIDIA Ampere or newer) with at least 150GB of free disk space to accomodate the download. This code has been tested on Transformers v4.44.0, torch v2.4.0 and 2 A100 80GB GPUs, but any setup that supports ```meta-llama/Llama-3.1-70B-Instruct``` should support this model as well. If you run into problems, you can consider doing ```pip install -U transformers```. ```python # Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="sandbox-ai/Llama-3.1-Tango-8b-Instruct-f16") ``` ## References(s): * TODO ## Model Architecture: **Architecture Type:** Transformer
**Network Architecture:** Llama 3.1
## Input: **Input Type(s):** Text
**Input Format:** String
**Input Parameters:** One Dimensional (1D)
**Other Properties Related to Input:** Max of 128k tokens
## Output: **Output Type(s):** Text
**Output Format:** String
**Output Parameters:** One Dimensional (1D)
**Other Properties Related to Output:** Max of 4k tokens
# Training & Evaluation: - TODO # Dataset: **MessIRve: A Large-Scale Spanish Information Retrieval Dataset**
* [spanish/-ir/messirve](https://huggingface.co/datasets/spanish-ir/messirve)
** messi_mod-v0.0.2 [tatakof/messi_mod-v0.0.2](https://huggingface.co/datasets/tatakof/messi_mod-v0.0.2) ## Citation ```bibtex @article{valentini2024messirve, title={MessIRve: A Large-Scale Spanish Information Retrieval Dataset}, author={Francisco Valentini and Viviana Cotik and Damián Furman and Ivan Bercovich and Edgar Altszyler and Juan Manuel Pérez}, year={2024}, eprint={2409.05994}, journal={arxiv:2409.05994}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2409.05994}, } @misc{wang2024helpsteer2preferencecomplementingratingspreferences, title={HelpSteer2-Preference: Complementing Ratings with Preferences}, author={Zhilin Wang and Alexander Bukharin and Olivier Delalleau and Daniel Egert and Gerald Shen and Jiaqi Zeng and Oleksii Kuchaiev and Yi Dong}, year={2024}, eprint={2410.01257}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/2410.01257}, } This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library. [

](https://github.com/unslothai/unsloth)