Neelectric/Llama-3.1-8B-Instruct_SFT_Math-220kv00.07
Neelectric/Llama-3.1-8B-Instruct_SFT_Math-220kv00.07 is an 8 billion parameter instruction-tuned language model developed by Neelectric. It is a fine-tuned version of Meta Llama-3.1-8B-Instruct, specifically optimized for mathematical tasks. This model leverages a 32768 token context length and is trained on the OpenR1-Math-220k_extended_Llama3_4096toks dataset, making it suitable for applications requiring strong mathematical reasoning.
Loading preview...
Overview
Neelectric/Llama-3.1-8B-Instruct_SFT_Math-220kv00.07 is an 8 billion parameter instruction-tuned model, building upon the robust Meta Llama-3.1-8B-Instruct architecture. Developed by Neelectric, this model has undergone supervised fine-tuning (SFT) using the TRL framework.
Key Capabilities
- Mathematical Reasoning: The model is specifically fine-tuned on the Neelectric/OpenR1-Math-220k_extended_Llama3_4096toks dataset, indicating a strong focus on mathematical problem-solving and related tasks.
- Instruction Following: As an instruction-tuned variant, it is designed to understand and execute user prompts effectively.
- Extended Context Window: It supports a context length of 32768 tokens, allowing for processing longer inputs and maintaining coherence over extended dialogues or complex problems.
Training Details
This model was trained using Supervised Fine-Tuning (SFT) with the TRL library (version 0.18.0). The training utilized Transformers 4.57.1, Pytorch 2.8.0, and Datasets 4.4.1. The fine-tuning process is publicly logged and can be visualized on Weights & Biases.
Good For
- Applications requiring precise mathematical problem-solving.
- Tasks where understanding and generating mathematical explanations are crucial.
- Use cases benefiting from a model with strong instruction-following capabilities in a technical context.