bitext/Mistral-7B-Insurance
The bitext/Mistral-7B-Insurance is a 7 billion parameter language model developed by Bitext, fine-tuned from mistralai/Mistral-7B-Instruct-v0.2. Optimized for the insurance domain, it excels at answering questions and assisting with insurance-related procedures, leveraging a 4096-token context length. This model is specifically designed to facilitate the creation of insurance chatbots, virtual assistants, and copilots.
Loading preview...
Overview
bitext/Mistral-7B-Insurance is a 7 billion parameter language model developed by Bitext, fine-tuned from mistralai/Mistral-7B-Instruct-v0.2. It is specifically tailored for the Insurance domain, designed to answer questions and assist users with various insurance-related procedures. The model was trained using hybrid synthetic data generated by Bitext's NLP/NLG technology and automated Data Labeling (DAL) tools.
Key Capabilities
- Insurance Domain Expertise: Optimized to understand and respond to insurance-specific queries.
- Chatbot & Virtual Assistant Foundation: Intended as a foundational step in Bitext’s two-step fine-tuning approach for creating specialized insurance chatbots, virtual assistants, and copilots.
- Broad Intent Coverage: Fine-tuned on the Bitext Insurance Dataset covering 39 insurance-related intents, each with approximately 1000 examples (e.g., buy_insurance_policy, check_payments, calculate_insurance_quote).
Training Details
The model was trained for 1 epoch with a batch size of 4 and a maximum sequence length of 8192 tokens, using the AdamW optimizer and a cosine learning rate scheduler. It utilizes the MistralForCausalLM architecture with a LlamaTokenizer.
Limitations
- Domain Specificity: May underperform in non-insurance related areas.
- Potential Biases: Users should critically evaluate responses due to potential biases in the training data.
This model is licensed under the Apache License 2.0 by Bitext Innovations International, Inc., allowing for free use, modification, and distribution with proper attribution.