LiberteEPFL/lfm25-1.2b-sft-bigchat
LiberteEPFL/lfm25-1.2b-sft-bigchat is a 1.2 billion parameter language model developed by LiberteEPFL, fine-tuned from LiquidAI/LFM2.5-1.2B-Base using SFT (Supervised Fine-Tuning) with a context length of 32768 tokens. This model is designed for general text generation tasks, leveraging its fine-tuned capabilities to produce coherent and contextually relevant responses. It is suitable for applications requiring a compact yet capable language model.
Loading preview...
Model Overview
LiberteEPFL/lfm25-1.2b-sft-bigchat is a 1.2 billion parameter language model, fine-tuned from the LiquidAI/LFM2.5-1.2B-Base architecture. This model was developed by LiberteEPFL and trained using Supervised Fine-Tuning (SFT) with the TRL library, leveraging a substantial context length of 32768 tokens.
Key Capabilities
- General Text Generation: Optimized for producing coherent and contextually appropriate text based on given prompts.
- Instruction Following: Fine-tuned to respond effectively to user instructions, as demonstrated by its quick start example.
- Efficient Deployment: With 1.2 billion parameters, it offers a balance between performance and computational efficiency, making it suitable for various applications.
Training Details
The model underwent a supervised fine-tuning process using the TRL framework (version 1.3.0), with Transformers 4.57.0 and Pytorch 2.8.0+cu128. The training progress was tracked and can be visualized via Weights & Biases.
Good For
- Interactive Chatbots: Its instruction-following capabilities make it suitable for conversational AI.
- Content Generation: Can be used for generating various forms of text content.
- Prototyping: A good choice for developers looking for a capable yet relatively small language model for rapid development and experimentation.