Overview
Neelectric/Llama-3.1-8B-Instruct_SFT_sciencefisher_v00.09 is an 8 billion parameter instruction-tuned model, built upon the robust Meta Llama-3.1-8B-Instruct architecture. This model has undergone Supervised Fine-Tuning (SFT) using the Neelectric/MoT_science_Llama3_4096toks dataset, specifically targeting scientific domain knowledge and applications. The training was conducted using the TRL framework, indicating a focus on leveraging transformer reinforcement learning techniques.
Key Capabilities
- Scientific Domain Specialization: Fine-tuned on a dedicated scientific dataset, enhancing its performance on science-related queries and content generation.
- Instruction Following: Inherits strong instruction-following capabilities from its Llama-3.1-8B-Instruct base.
- Context Handling: Benefits from the 32768 token context length of the base model, allowing for processing and generating longer, more complex scientific texts.
Training Details
The model was trained with SFT, utilizing TRL version 1.0.0.dev0, Transformers 4.57.6, Pytorch 2.9.0, Datasets 4.8.3, and Tokenizers 0.22.2. Further details on the training run are available via Weights & Biases.
Good For
- Answering scientific questions.
- Generating scientific explanations or summaries.
- Assisting with tasks requiring specialized scientific knowledge.