AmberYifan/capsd-finance-dedup-marin-8b-base-finance_cap_b2000_s0
The AmberYifan/capsd-finance-dedup-marin-8b-base-finance_cap_b2000_s0 model is an 8 billion parameter language model fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for financial applications, having been trained on a dedicated financial dataset. Its primary strength lies in processing and understanding financial text, making it suitable for tasks requiring specialized financial domain knowledge.
Loading preview...
Model Overview
This model, capsd-finance-dedup-marin-8b-base-finance_cap_b2000_s0, is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. It has been specifically fine-tuned on the capsd_marin-8b-base-n80000-finance-dedup80k__mix_finance_cap_b2000_s0 dataset, indicating a strong specialization in the financial domain.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Domain Specialization: Optimized for financial text processing through targeted fine-tuning.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05, using an AdamW optimizer and a cosine learning rate scheduler with 0.03 warmup steps. Training was conducted on a multi-GPU setup with 4 devices, utilizing a total batch size of 64 (train_batch_size: 2, gradient_accumulation_steps: 8).
Intended Use Cases
Given its specialized training on financial data, this model is best suited for applications requiring deep understanding and generation of financial content. While specific use cases are not detailed, its fine-tuning suggests applicability in areas such as financial analysis, reporting, and information extraction from financial documents.