What is Claire-Mistral-7B-0.1?
Claire-Mistral-7B-0.1 is a 7 billion parameter causal decoder-only language model, a specialized variant of Mistral-7B-v0.1, developed by LINAGORA and OpenLLM-France. Its core distinction lies in its extensive fine-tuning on a diverse dataset of French conversational data, making it highly attuned to the nuances of spoken French dialogue.
Key Capabilities
- French Dialogue Generation: Optimized for generating natural and fluent continuations of French conversations, including common disfluencies and spoken language constructions.
- Base for Dialogue Tasks: Intended as a foundational model for further fine-tuning on specific dialogue generation (e.g., chatbots) and dialogue understanding (e.g., meeting summarization) applications.
- Flexible Prompting: Supports various dialogue formats, including single-line monologues, two-speaker dialogues with dashes, and multi-speaker dialogues with numbered or named participants.
Training Details
The model was trained on the OpenLLM-France/Claire-Dialogue-French-0.1 dataset, comprising diverse French conversational sources like parliamentary proceedings, theatre scripts, interviews, and free conversations. Training involved data augmentation techniques such as varying speech turn formats and speaker naming conventions. It was trained for approximately 50 GPU hours on 8 A100 80GB GPUs.
Licensing
Claire-Mistral-7B-0.1 is released under the CC-BY-NC-SA 4.0 license due to the licenses of its training corpora.