AdaptLLM/finance-LLM-13B
AdaptLLM/finance-LLM-13B is a 13 billion parameter language model developed by AdaptLLM, continually pre-trained from LLaMA-1-13B on domain-specific financial corpora. It utilizes a novel reading comprehension method to enhance domain knowledge while preserving prompting ability, addressing a common challenge in domain adaptation. This model is specifically optimized for financial question answering and tasks, demonstrating competitive performance against larger domain-specific models.
Loading preview...
AdaptLLM/finance-LLM-13B: Domain-Adapted Financial LLM
AdaptLLM/finance-LLM-13B is a 13 billion parameter language model, continually pre-trained from LLaMA-1-13B by AdaptLLM. This model is specifically adapted for the financial domain, leveraging a unique method of transforming large-scale pre-training corpora into reading comprehension texts. This approach effectively enriches the model with domain-specific knowledge while mitigating the common issue of degraded prompting performance in question answering tasks after continued pre-training.
Key Capabilities
- Enhanced Financial Domain Knowledge: Specialized training on extensive financial corpora.
- Preserved Prompting Ability: Utilizes a reading comprehension method to maintain strong performance in question answering, unlike traditional continued pre-training.
- Competitive Performance: The underlying AdaptLLM method has shown 7B models competing with much larger domain-specific models like BloombergGPT-50B.
- Scalability: The method has been proven effective for larger models, including this 13B variant.
Good For
- Financial Question Answering: Excels at understanding and responding to queries based on financial texts and data.
- Domain-Specific NLP in Finance: Ideal for applications requiring deep understanding of financial documents, reports, and terminology.
- Research and Development: Provides a strong base for further fine-tuning or research in financial AI, with available benchmarking code and raw datasets.