AmberYifan/capsd-finance-fincot-finqa-clean6k-marin-8b-base-finance_cap_b1000_s0

Model Overview

This model, AmberYifan/capsd-finance-fincot-finqa-clean6k-marin-8b-base-finance_cap_b1000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for financial domain tasks.

Key Characteristics

• Base Model: Fine-tuned from marin-community/marin-8b-base.
• Parameter Count: 8 billion parameters.
• Context Length: Supports a context length of 8192 tokens.
• Domain Specialization: The model has undergone fine-tuning on a dataset identified as capsd_marin-8b-base-n6000-finance-fincot-finqa-clean6k__mix_finance_cap_b1000_s0, indicating a focus on financial data and tasks.

Training Details

The training process involved specific hyperparameters:

• Learning Rate: 1e-05
• Batch Sizes: train_batch_size of 2, eval_batch_size of 8.
• Gradient Accumulation: 8 steps, leading to a total_train_batch_size of 64.
• Optimizer: ADAMW_TORCH with default betas and epsilon.
• Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
• Epochs: Trained for 1 epoch.

Intended Use

This model is primarily intended for applications within the financial sector, where its specialized training on financial datasets can provide more accurate and relevant responses compared to general-purpose models. Specific use cases would involve tasks requiring an understanding of financial terminology, documents, or data.