arcee-ai/Llama-3-SEC-Chat

Hugging Face
TEXT GENERATIONConcurrency Cost:4Model Size:70BQuant:FP8Ctx Length:8kPublished:Jun 16, 2024License:llama3Architecture:Transformer0.0K Warm

Llama-3-SEC is a 70 billion parameter domain-specific large language model developed by arcee-ai, built upon Meta-Llama-3-70B-Instruct. This intermediate checkpoint has been continually pre-trained on 20 billion tokens of SEC filings data, combined with general data, to specialize in financial analysis. It excels at tasks like investment analysis, risk management, and regulatory compliance within the financial sector, offering unparalleled insights into SEC data. The model maintains strong general language understanding while providing deep domain-specific knowledge for financial professionals.

Loading preview...

Llama-3-SEC: Domain-Specific Financial Analysis Model

Llama-3-SEC, developed by arcee-ai, is a 70 billion parameter large language model built on Meta-Llama-3-70B-Instruct, specifically designed for analyzing SEC (Securities and Exchange Commission) data. This release is an intermediate checkpoint, having undergone continual pre-training on 20 billion tokens of SEC filings data, alongside 1 billion tokens from Together AI's RedPajama dataset to balance domain expertise with general language understanding. The full model is still in training and will eventually be trained on 72 billion tokens of SEC data.

Key Capabilities

  • Domain Specialization: Deep understanding of SEC filings and related financial information.
  • Advanced Training: Utilizes Continual Pre-Training (CPT) with Megatron-Core and TIES merging technique from Arcee Mergekit, followed by supervised fine-tuning.
  • Robust Evaluation: Demonstrated improvements in domain-specific perplexity and extractive numerical reasoning tasks (TAT-QA, ConvFinQA), while maintaining strong performance on general benchmarks like BIG-bench, AGIEval, GPT4all, and TruthfulQA.
  • Chat Template: Trained using the chatml template for strong conversational abilities.

Good For

  • In-depth Investment Analysis: Supporting decision-making with detailed financial insights.
  • Risk Management: Assessing and identifying potential financial risks.
  • Regulatory Compliance: Ensuring adherence to regulations and detecting violations.
  • Corporate Governance: Studying practices and promoting transparency.
  • Market Research: Tracking industry trends and conducting sector-specific analysis.

Finetunes
2 models