Iker/Llama-3-Instruct-Neurona-8b
Iker/Llama-3-Instruct-Neurona-8b is an 8 billion parameter instruction-tuned language model developed by Iker, based on Meta's Llama-3-8B-Instruct architecture with an 8192 token context length. This model is specifically optimized for Spanish language tasks, having been trained on a diverse mix of English and Spanish datasets to acquire capabilities such as RAG, function calling, code assistance, question answering, and summarization in both languages.
Loading preview...
Neurona 8B Beta: A Spanish Language Model
Neurona 8B is an 8 billion parameter language model developed by Iker, primarily focused on the Spanish language. This model is an experimental, preliminary iteration built upon the meta-llama/Meta-Llama-3-8B-Instruct base model, utilizing an 8192 token context length.
Key Capabilities
Neurona 8B has been trained on a diverse collection of both English and Spanish datasets, enabling it to perform various tasks, including:
- Retrieval Augmented Generation (RAG)
- Function Calling
- Code Assistance
- Question Answering
- Summarization
- Translation (English-Spanish)
This mixed-dataset approach allows the model to acquire these capabilities in both English and Spanish.
Training Details
The model was trained using 4xNvidia A100 80Gb GPUs and the Axolotl framework. The training involved a wide array of datasets, including pinzhenchen/alpaca-cleaned-es, FreedomIntelligence/evol-instruct-spanish, glaiveai/glaive-code-assistant-v3, Iker/OpenHermes-2.5-Spanish, and wikipedia es, among others. The training configuration specifies a llama3 conversation template and a sequence length of 8192 tokens.
Current Status
Neurona 8B is currently a beta version and an ongoing experiment. It is not the final release, and further development is planned.