MadhurArora1/mistral-7b-finance-qlora
MadhurArora1/mistral-7b-finance-qlora is a 7.2 billion parameter Mistral-7B-v0.1 model fine-tuned by Madhur Arora using QLoRA. Optimized for financial question-answering, it accurately and concisely handles topics like investment concepts, financial ratios, and corporate finance. The model was trained on 10,000 instruction-response pairs from the gbharti/finance-alpaca dataset, making it specialized for financial domain queries.
Loading preview...
Mistral-7B Finance QLoRA: Specialized Financial QA
This model is a fine-tuned version of the Mistral-7B-v0.1 base model, developed by Madhur Arora. It has been specifically adapted using QLoRA (Quantized Low-Rank Adaptation) to excel at answering financial questions accurately and concisely. The fine-tuning process involved using 4-bit NF4 quantization and LoRA adapters, resulting in only ~0.28% of the model's 7.2 billion parameters being trainable.
Key Capabilities
- Financial Question Answering: Designed to provide accurate and concise answers to queries across various financial topics.
- Domain Coverage: Handles investment concepts, financial ratios, market instruments, macroeconomics, and corporate finance.
- Efficient Fine-tuning: Utilizes QLoRA for memory-efficient adaptation, making it accessible for deployment.
Training Details
The model was fine-tuned on 10,000 instruction-response pairs from the gbharti/finance-alpaca dataset, using an Alpaca instruction template. Training was conducted for 2 epochs with a maximum sequence length of 512 tokens. While effective for its specialized domain, users should be aware of its limitations, including potential performance degradation on highly technical or niche financial topics and occasional hallucinations on specific numerical data. It is not intended for real investment advice.