Overview

Neelectric/Llama-3.1-8B-Instruct_SFT_sciencev00.01 is an 8 billion parameter instruction-tuned model, fine-tuned by Neelectric from the base meta-llama/Llama-3.1-8B-Instruct architecture. This model has been specifically adapted for scientific applications through supervised fine-tuning (SFT) using the Neelectric/MoT_science_Llama3_4096toks dataset.

Key Capabilities

• Scientific Domain Specialization: Enhanced performance on tasks and queries related to scientific topics due to its targeted training data.
• Instruction Following: Capable of understanding and executing instructions, inherited from its Llama-3.1-8B-Instruct base.
• Extended Context: Benefits from a 32768 token context length, allowing for processing and generating longer, more detailed scientific texts.

Training Details

The model was trained using the TRL (Transformers Reinforcement Learning) library, specifically employing an SFT (Supervised Fine-Tuning) procedure. The training process is documented and can be visualized via Weights & Biases. This fine-tuning process leverages the robust capabilities of the Llama-3.1-8B-Instruct model and tailors them for scientific discourse.