AmberYifan/capsd-finance-dedup-marin-8b-base-finance_cap_b10000_s0
AmberYifan/capsd-finance-dedup-marin-8b-base-finance_cap_b10000_s0 is an 8 billion parameter language model fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for financial applications, having been trained on a deduped financial dataset. Its primary differentiation lies in its specialized training for finance-related tasks, making it suitable for processing and generating financial text. The model was trained with a learning rate of 1e-05 and a cosine learning rate scheduler over one epoch.
Loading preview...
Model Overview
This model, AmberYifan/capsd-finance-dedup-marin-8b-base-finance_cap_b10000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for financial contexts.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: 8192 tokens.
- Specialization: Trained on a dedicated financial dataset (
capsd_marin-8b-base-n80000-finance-dedup80k__mix_finance_cap_b10000_s0).
Training Details
The model underwent a single training epoch with specific hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A
train_batch_sizeof 2 andgradient_accumulation_stepsof 8 resulted in atotal_train_batch_sizeof 64. - Optimizer: ADAMW_TORCH with default betas and epsilon.
- LR Scheduler: Cosine scheduler with 0.03 warmup steps.
Intended Use Cases
Given its specialized training on financial data, this model is primarily intended for applications requiring an understanding and generation of financial text. While specific use cases are not detailed in the original model card, its fine-tuning suggests suitability for tasks such as financial analysis, reporting, or information extraction within the finance domain.