AdaptLLM/finance-LLM-13B: Domain-Adapted Financial LLM

AdaptLLM/finance-LLM-13B is a 13 billion parameter language model, continually pre-trained from LLaMA-1-13B by AdaptLLM. This model is specifically adapted for the financial domain, leveraging a unique method of transforming large-scale pre-training corpora into reading comprehension texts. This approach effectively enriches the model with domain-specific knowledge while mitigating the common issue of degraded prompting performance in question answering tasks after continued pre-training.

Key Capabilities

• Enhanced Financial Domain Knowledge: Specialized training on extensive financial corpora.
• Preserved Prompting Ability: Utilizes a reading comprehension method to maintain strong performance in question answering, unlike traditional continued pre-training.
• Competitive Performance: The underlying AdaptLLM method has shown 7B models competing with much larger domain-specific models like BloombergGPT-50B.
• Scalability: The method has been proven effective for larger models, including this 13B variant.

Good For

• Financial Question Answering: Excels at understanding and responding to queries based on financial texts and data.
• Domain-Specific NLP in Finance: Ideal for applications requiring deep understanding of financial documents, reports, and terminology.
• Research and Development: Provides a strong base for further fine-tuning or research in financial AI, with available benchmarking code and raw datasets.