Llama-3.1-Omni-FinAI-8B: Finance-Optimized Foundation Model
Llama-3.1-Omni-FinAI-8B is an 8 billion parameter large language model built upon the LLaMA 3.1 architecture, specifically designed for financial applications. Developed by ichanchiu, this model underwent extensive pre-training on a massive 143 billion token dataset of diverse financial texts. This specialized training makes it a robust foundation for further fine-tuning in various finance-related tasks.
Key Capabilities
- Finance-Specific Pre-training: Trained on SEC filings (10-K, 10-Q, 8-K), Reuters News (RCV1, TRC2), Arxiv financial papers, Reddit financial discussions, and Wikipedia.
- Foundation for Fine-tuning: Serves as an excellent base model for specialized financial analysis.
- Robust Training: Utilized NVIDIA NeMo framework on 64 H100 GPUs for comprehensive training.
Good for
- Sentiment Analysis: Analyzing financial news and reports for market sentiment.
- Stock Movement Prediction: Leveraging textual data for predictive financial analysis.
- QA Instruction: Developing question-answering systems for financial queries.
- Summarization: Generating concise summaries of financial documents.
- Predictive Financial Analysis: Building models that forecast financial outcomes based on textual information.
This model is licensed under the Llama 3.1 Community License and is intended for users with substantial computational resources for deployment and further fine-tuning.