Model Overview
Neelectric/Llama-3.2-1B-Instruct_SFT_sciencefisher_v00.06 is a 1 billion parameter instruction-tuned model, fine-tuned by Neelectric. It is based on the meta-llama/Llama-3.2-1B-Instruct architecture and has been specifically adapted for scientific applications.
Key Capabilities
- Scientific Domain Specialization: The model has undergone Supervised Fine-Tuning (SFT) on the
Neelectric/MoT_science_Llama3_2048toks dataset, indicating a focus on scientific text and knowledge. - Instruction Following: As an instruction-tuned model, it is designed to follow user prompts and generate relevant responses.
- Context Length: It supports a substantial context length of 32768 tokens, allowing for processing longer scientific texts or complex queries.
Training Details
The model was trained using the TRL library (Transformers Reinforcement Learning) with specific framework versions including TRL 1.0.0.dev0, Transformers 4.57.6, Pytorch 2.9.0, Datasets 4.8.3, and Tokenizers 0.22.2. The training process can be visualized via Weights & Biases.
Good For
- Scientific Question Answering: Given its fine-tuning on a scientific dataset, it is well-suited for answering questions within scientific domains.
- Scientific Text Generation: Generating coherent and contextually relevant text for scientific topics.
- Research Assistance: Potentially useful for tasks requiring understanding or generation of scientific literature.