AmberYifan/capsd-finance-fincot-finqa-clean6k-marin-8b-base-finance_random_b2000_s0
The AmberYifan/capsd-finance-fincot-finqa-clean6k-marin-8b-base-finance_random_b2000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. It was specifically trained on a financial dataset, capsd_marin-8b-base-n6000-finance-fincot-finqa-clean6k__mix_finance_random_b2000_s0, suggesting an optimization for financial language understanding and generation. This model is intended for applications requiring specialized knowledge in finance, leveraging its 8192 token context length for processing financial documents.
Loading preview...
Model Overview
This model, named marin-8b-base_finance_random_b2000_s0, is an 8 billion parameter language model. It is a fine-tuned version of the marin-community/marin-8b-base architecture, specifically adapted for financial applications.
Key Training Details
The model was fine-tuned using the capsd_marin-8b-base-n6000-finance-fincot-finqa-clean6k__mix_finance_random_b2000_s0 dataset, indicating a focus on financial text. The training process involved:
- Learning Rate: 1e-05
- Batch Size: A total training batch size of 64 (with
train_batch_size: 2andgradient_accumulation_steps: 8) - Optimizer: ADAMW_TORCH
- LR Scheduler: Cosine type with 0.03 warmup steps
- Epochs: 1
Intended Use Cases
Given its specialized financial training data, this model is likely optimized for tasks within the finance domain. Potential applications could include:
- Financial document analysis
- Question answering on financial texts
- Generating financial reports or summaries
- Understanding financial terminology and concepts