alignment-handbook/mistral-7b-sft-constitutional-ai
The alignment-handbook/mistral-7b-sft-constitutional-ai is a 7 billion parameter language model, fine-tuned from mistralai/Mistral-7B-v0.1. It was trained on the HuggingFaceH4/cai-conversation-harmless and HuggingFaceH4/ultrachat_200k datasets, focusing on constitutional AI principles. This model is designed for conversational AI applications where harmless and aligned responses are critical, leveraging its 4096-token context length for coherent interactions.
Loading preview...
Model Overview
This model, mistral-7b-sft-constitutional-ai, is a fine-tuned variant of the Mistral-7B-v0.1 base model. It has been specifically trained to incorporate principles of Constitutional AI, aiming to generate safer and more aligned responses.
Key Capabilities
- Constitutional AI Alignment: Fine-tuned on datasets like HuggingFaceH4/cai-conversation-harmless, which emphasizes harmless and ethical conversational interactions.
- Conversational Proficiency: Further enhanced using the HuggingFaceH4/ultrachat_200k dataset, improving its general conversational abilities.
- Mistral 7B Architecture: Benefits from the efficient and performant architecture of the Mistral-7B-v0.1 base model.
Training Details
The model was trained with a learning rate of 2e-05 over 1 epoch, utilizing a total batch size of 256 across 8 GPUs. The training achieved a validation loss of 0.9344, indicating effective learning from the alignment-focused datasets.
Intended Use Cases
This model is particularly suitable for applications requiring:
- Safe Chatbots: Developing conversational agents that adhere to ethical guidelines and avoid generating harmful content.
- Content Moderation: Assisting in filtering or generating responses that align with specific safety policies.
- Research in AI Alignment: Exploring the effectiveness of constitutional AI principles in practical language models.