AmberYifan/capsd-finance-dedup-marin-8b-base-finance_ppl_b2000_s0
The AmberYifan/capsd-finance-dedup-marin-8b-base-finance_ppl_b2000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. It is specifically optimized for financial applications, having been trained on a dedicated financial dataset. This model is designed for tasks requiring specialized financial understanding and processing, leveraging its 8192-token context length.
Loading preview...
Model Overview
This model, AmberYifan/capsd-finance-dedup-marin-8b-base-finance_ppl_b2000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for financial applications.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Specialization: Optimized for financial tasks through training on the
capsd_marin-8b-base-n80000-finance-dedup80k__mix_finance_ppl_b2000_s0dataset.
Training Details
The model was trained with a learning rate of 1e-05, using a total batch size of 64 (achieved with train_batch_size: 2 and gradient_accumulation_steps: 8 across 4 GPUs). The training utilized the AdamW optimizer with a cosine learning rate scheduler over 1 epoch. The framework versions used include Transformers 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, and Tokenizers 0.22.2.
Intended Use Cases
Given its specialized financial training, this model is suitable for applications requiring deep understanding and generation of financial text. Specific use cases would benefit from further evaluation.