AmberYifan/capsd-convfinqa-fullscore-marin-8b-base-finance_cap_b4000_s0
AmberYifan/capsd-convfinqa-fullscore-marin-8b-base-finance_cap_b4000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically trained on a financial conversational question-answering dataset, making it suitable for finance-related NLP tasks. It utilizes a context length of 8192 tokens and was trained with a learning rate of 1e-05 over one epoch.
Loading preview...
Model Overview
This model, AmberYifan/capsd-convfinqa-fullscore-marin-8b-base-finance_cap_b4000_s0, is an 8 billion parameter language model. It is a fine-tuned version of the marin-community/marin-8b-base architecture, specifically adapted for financial applications.
Key Training Details
The model was fine-tuned on the capsd_marin-8b-base-n11082-finance-convfinqa-fullscore__mix_finance_cap_b4000_s0 dataset, indicating a specialization in financial conversational question-answering. Training involved:
- Base Model:
marin-community/marin-8b-base - Learning Rate: 1e-05
- Batch Size: 2 (train), 8 (eval)
- Gradient Accumulation: 8 steps, leading to a total effective batch size of 64
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps
- Epochs: 1
- Frameworks: Transformers 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, Tokenizers 0.22.2
Intended Use Cases
Given its specialized training on a financial conversational dataset, this model is primarily intended for tasks requiring understanding and generation of responses within a financial context. Its 8192-token context length allows for processing moderately long financial documents or conversations.