bitext/Mistral-7B-Banking-v2

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kTool Calling:SupportedPublished:May 3, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

bitext/Mistral-7B-Banking-v2 is a 7 billion parameter language model developed by Bitext, fine-tuned from mistralai/Mistral-7B-Instruct-v0.2. Optimized for banking-specific interactions, this model excels at answering questions and assisting with various banking transactions within a 4096-token context. It is designed to serve as a foundational model for creating specialized banking chatbots and virtual assistants, leveraging hybrid synthetic data for training.

Loading preview...

Overview

bitext/Mistral-7B-Banking-v2 is a 7 billion parameter language model developed by Bitext, specifically fine-tuned from the Mistral-7B-Instruct-v0.2 architecture for the banking domain. It is designed to facilitate the creation of specialized chatbots, virtual assistants, and copilots for banking customer support.

Key Capabilities

  • Banking-Specific Expertise: Optimized to answer questions and assist with various banking transactions, including balance checks, money transfers, and loan applications.
  • Fine-tuned for Banking: Trained on a comprehensive dataset of 89 banking-related intents, each with approximately 1000 examples, using Bitext's proprietary NLP/NLG technology and automated Data Labeling (DAL) tools.
  • Foundation for Customization: Intended as a first step in a two-step fine-tuning approach, allowing banks to further customize the model with their own data for specific use cases.

Intended Use

  • Recommended: Ideal for developing customer support solutions in the banking sector, providing fast and accurate answers to banking-related queries.
  • Out-of-scope: Not suitable for non-banking questions or for providing health, legal, or critical safety advice.

Technical Details

  • Architecture: Utilizes MistralForCausalLM with a LlamaTokenizer.
  • Training: Fine-tuned over 3 epochs with AdamW optimizer, a learning rate of 0.0002, and a maximum sequence length of 8192 tokens.

Limitations

  • Performance may be suboptimal in domains outside of banking.
  • Potential biases from training data may affect response neutrality, requiring critical evaluation by users.

Finetunes
2 models