KayaTechAI/Llama-3.1-8b-Finance
KayaTechAI/Llama-3.1-8b-Finance is an 8 billion parameter Llama 3.1-based model developed by KayaTechAI, specifically engineered for financial applications. This model integrates instruction-tuned financial reasoning with financial-sentiment classification capabilities. It was created by merging two specialized LoRA adapters onto unsloth/Llama-3.1-8B-Instruct using a TIES-Merge methodology, making it suitable for financial document analysis and Q&A.
Loading preview...
Overview
KayaTechAI/Llama-3.1-8b-Finance is an 8 billion parameter model built upon the Llama 3.1 architecture, developed by KayaTechAI. It is a production-ready financial-domain model that combines instruction-tuned financial reasoning with financial-sentiment classification. The model was created by merging two distinct LoRA adapters, KayaTechAI/SFT-Llama-3.1-8B-Financial-Instruct and KayaTechAI/SFT-Llama-3.1-8B-Financial-Instruct-Sentiment, onto unsloth/Llama-3.1-8B-Instruct using a TIES-Merge (sign-consensus + magnitude-pruning) methodology.
Key Capabilities
- Financial Reasoning: Excels at understanding and processing financial instructions.
- Financial Sentiment Analysis: Capable of classifying sentiment within financial texts.
- Integrated Functionality: Offers both reasoning and sentiment classification in a single checkpoint.
Merge Methodology Highlights
- Bake Stage: Each LoRA adapter was first merged with its verified base model to produce full dense checkpoints.
- Conflict Analysis: Cosine similarity was used to analyze conflicts between the financial and sentiment deltas, identifying 7.2% high conflict tensors and 57.7% high agreement tensors.
- TIES-Merge: The two resulting deltas were combined with specific weights (Financial: 0.55, Sentiment: 0.45) and re-applied to the base model. Sign-disagreement was resolved by the dominant-magnitude delta, and normalization was enabled.
Intended Use
This model is designed for:
- Financial document analysis.
- Financial question-answering and reasoning tasks.
- Financial-sentiment classification.
Important Note
As a research/production candidate, it is recommended to validate the model on held-out financial QA and sentiment test sets (e.g., KayaTechAI/sp500-summary-dataset, KayaTechAI/FIN-Sentiment-SFT) before full production deployment.