Neelectric/Llama-3.1-8B-Instruct_SFT_sciencefisher_v00.13
Neelectric/Llama-3.1-8B-Instruct_SFT_sciencefisher_v00.13 is an 8 billion parameter instruction-tuned causal language model developed by Neelectric. It is a fine-tuned version of Meta's Llama-3.1-8B-Instruct, specifically optimized for scientific domain tasks. This model leverages a 32768 token context length and is trained using Supervised Fine-Tuning (SFT) on a specialized scientific dataset, making it suitable for science-related natural language processing applications.
Loading preview...
Neelectric/Llama-3.1-8B-Instruct_SFT_sciencefisher_v00.13 Overview
This model is an 8 billion parameter instruction-tuned language model developed by Neelectric, built upon Meta's Llama-3.1-8B-Instruct architecture. It has been specifically fine-tuned using Supervised Fine-Tuning (SFT) on the Neelectric/MoT_science_Llama3_4096toks dataset, which is tailored for scientific content. This specialization aims to enhance its performance and relevance for tasks within the scientific domain.
Key Capabilities
- Scientific Domain Specialization: Fine-tuned on a dedicated scientific dataset, suggesting improved understanding and generation of science-related text.
- Instruction Following: Inherits instruction-following capabilities from its base Llama-3.1-8B-Instruct model.
- Context Length: Supports a substantial context window of 32768 tokens, allowing for processing longer scientific documents or complex queries.
Training Details
The model was trained using the TRL (Transformers Reinforcement Learning) library. The training process involved SFT, leveraging specific versions of frameworks including TRL 1.0.0.dev0, Transformers 4.57.6, Pytorch 2.9.0, Datasets 4.8.4, and Tokenizers 0.22.2.
Good For
- Scientific Text Generation: Generating coherent and contextually relevant text for scientific topics.
- Scientific Question Answering: Answering questions that require knowledge from scientific literature or concepts.
- Research Assistance: Potentially useful for tasks like summarizing scientific papers, extracting information from research articles, or drafting scientific explanations.