LiberteEPFL/lfm25-1.2b-sft-bigchat-v2
LiberteEPFL/lfm25-1.2b-sft-bigchat-v2 is a 1.2 billion parameter instruction-tuned causal language model developed by LiberteEPFL, fine-tuned from LiquidAI/LFM2.5-1.2B-Base. This model, trained with Supervised Fine-Tuning (SFT) using TRL, is designed for general text generation tasks with a context length of 32768 tokens. It specializes in generating conversational responses based on user prompts.
Loading preview...
Model Overview
LiberteEPFL/lfm25-1.2b-sft-bigchat-v2 is a 1.2 billion parameter language model, fine-tuned from the LiquidAI/LFM2.5-1.2B-Base architecture. This model leverages Supervised Fine-Tuning (SFT) using the TRL library to enhance its conversational capabilities.
Key Capabilities
- Instruction Following: Designed to generate responses based on user prompts and instructions.
- Text Generation: Capable of producing coherent and contextually relevant text.
- Conversational AI: Optimized for chat-like interactions, making it suitable for dialogue systems.
- Extended Context: Supports a substantial context length of 32768 tokens, allowing for longer conversations and more detailed inputs.
Training Details
The model was trained using the TRL (Transformers Reinforcement Learning) framework, specifically employing an SFT approach. The training process utilized Transformers version 4.57.0 and PyTorch 2.8.0+cu128. Further details on the training run are available via Weights & Biases.
Good For
- Developing chatbots and conversational agents.
- Generating creative text based on prompts.
- Applications requiring instruction-tuned language understanding and extended context handling.