AmberYifan/capsd-finance-fincot-finqa-clean6k-marin-8b-base-finance_ppl_b2000_s0
AmberYifan/capsd-finance-fincot-finqa-clean6k-marin-8b-base-finance_ppl_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for financial applications, having been trained on a specialized financial dataset. Its primary purpose is to serve use cases within the finance domain, leveraging its base architecture with targeted financial data exposure. The model has a context length of 8192 tokens.
Loading preview...
Model Overview
This model, marin-8b-base_finance_ppl_b2000_s0, is an 8 billion parameter language model developed by AmberYifan. It is a fine-tuned iteration of the marin-community/marin-8b-base architecture, specifically adapted for financial tasks.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Domain Specialization: Optimized for financial applications through fine-tuning on the
capsd_marin-8b-base-n6000-finance-fincot-finqa-clean6k__mix_finance_ppl_b2000_s0dataset.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Optimizer: ADAMW_TORCH with betas=(0.9, 0.999) and epsilon=1e-08
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps
- Epochs: 1
- Batch Size: A total training batch size of 64 (2 per device with 8 gradient accumulation steps across 4 GPUs).
Intended Use
This model is designed for use cases requiring financial domain understanding and generation, leveraging its specialized training data. Further details on specific intended uses and limitations are pending.