ichanchiu/Llama-3.1-Omni-FinAI-70B
ichanchiu/Llama-3.1-Omni-FinAI-70B is a 70 billion parameter large language model based on the Llama 3.1 architecture, pre-trained on 143 billion tokens of high-quality financial texts. This model is specifically optimized for finance-specific fine-tuning applications, providing a robust foundation for specialized financial analysis tasks. It excels as a base model for applications like sentiment analysis, stock movement prediction, and financial summarization. With a 32768 token context length, it is designed to process extensive financial documents.
Loading preview...
Llama-3.1-Omni-FinAI-70B: Financial Domain Pre-trained LLM
This model, developed by ichanchiu, is a 70 billion parameter large language model built upon the Llama 3.1 architecture. It has undergone extensive pre-training on a massive dataset of 143 billion tokens derived from high-quality financial texts, including SEC filings (10-K, 10-Q, 8-K), Reuters News data, finance-specific arXiv papers, and financial discussions from Reddit.
Key Capabilities & Features
- Finance-Specific Pre-training: Optimized for understanding and processing complex financial language and data.
- Robust Foundation: Serves as an excellent base model for further fine-tuning on specialized financial tasks.
- Diverse Training Data: Utilizes a broad range of financial documents and discussions to ensure comprehensive domain knowledge.
- High Context Length: Supports a 32768 token context window, suitable for analyzing lengthy financial reports.
Primary Use Cases
Llama-3.1-Omni-FinAI-70B is designed for fine-tuning in various financial applications, including:
- Sentiment Analysis: Assessing market sentiment from financial news and reports.
- Stock Movement Prediction: Analyzing textual data to forecast stock performance.
- QA Instruction: Developing question-answering systems for financial queries.
- Summarization: Generating concise summaries of financial documents.
- Predictive Financial Analysis: Leveraging textual insights for financial forecasting.
Training & Limitations
The model was trained using the NVIDIA NeMo framework on 64 H100 GPUs. While powerful, it is primarily a pre-trained model and requires additional fine-tuning for specialized applications. Due to its size, significant computational resources are recommended for deployment. The model is licensed under the Llama 3.1 Community License. For more details, refer to the associated research paper: Omni-FinAI: Unlocking Financial Disclosure Insights.