Overview

Neelectric/Llama-3.2-1B-Instruct_SFT_sciencefisher_v00.05 is a 1 billion parameter instruction-tuned model, building upon the meta-llama/Llama-3.2-1B-Instruct architecture. It has been specifically fine-tuned using the Neelectric/MoT_science_Llama3_2048toks dataset, indicating a specialization in scientific domain understanding and generation. The training was conducted using the TRL framework.

Key Capabilities

• Scientific Domain Specialization: Fine-tuned on a dedicated scientific dataset, suggesting enhanced performance for science-related queries and content generation.
• Instruction Following: Designed to follow instructions effectively, making it suitable for various task-oriented applications.
• Extended Context Window: Benefits from a 32768 token context length, allowing it to process and generate longer, more complex scientific texts.

Training Details

The model underwent Supervised Fine-Tuning (SFT) using the TRL library. The training process utilized specific versions of frameworks including TRL 1.0.0.dev0, Transformers 4.57.6, Pytorch 2.9.0, Datasets 4.8.3, and Tokenizers 0.22.2. Further details on the training run are available via Weights & Biases.