gshasiri/SmolLM3-SFT-Second-Round
The gshasiri/SmolLM3-SFT-Second-Round is a 1 billion parameter instruction-tuned causal language model developed by gshasiri. Fine-tuned from SmolLM3-Mid-Second-Round using SFT, it is designed for general text generation tasks. This model offers a 32768 token context length, making it suitable for applications requiring processing longer inputs.
Loading preview...
Overview
This model, gshasiri/SmolLM3-SFT-Second-Round, is a 1 billion parameter instruction-tuned language model. It is a fine-tuned iteration of the gshasiri/SmolLM3-Mid-Second-Round base model, developed by gshasiri. The fine-tuning process utilized the TRL library with Supervised Fine-Tuning (SFT) methods.
Key Capabilities
- Instruction Following: Optimized through SFT to better understand and respond to user instructions.
- Text Generation: Capable of generating coherent and contextually relevant text based on prompts.
- Extended Context: Features a substantial context window of 32768 tokens, allowing for processing and generating longer sequences of text.
Training Details
The model was trained using the TRL framework (version 0.25.1) in conjunction with Transformers (4.57.1), Pytorch (2.6.0+cu126), Datasets (4.4.1), and Tokenizers (0.22.1). The training process can be visualized via Weights & Biases.
When to Use This Model
This model is suitable for general text generation tasks where a smaller, instruction-tuned model with a large context window is beneficial. Its SFT optimization makes it a good candidate for applications requiring direct instruction following.