AmberYifan/capsd-convfinqa-fullscore-marin-8b-base-finance_ppl_b8000_s0

Model Overview

This model, named marin-8b-base_finance_ppl_b8000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for financial contexts.

Key Characteristics

• Base Model: Fine-tuned from marin-community/marin-8b-base.
• Parameter Count: 8 billion parameters.
• Context Length: Supports an 8192 token context window.
• Specialization: Optimized for financial applications through fine-tuning on a dedicated dataset.

Training Details

The model was trained with a learning rate of 1e-05, a total batch size of 64 (achieved with train_batch_size of 2 and gradient_accumulation_steps of 8), and for 1 epoch. It utilized an AdamW optimizer with cosine learning rate scheduling. The training environment included Transformers 5.7.0 and Pytorch 2.13.0+cu130.

Intended Use

While specific intended uses and limitations require more detailed information, its fine-tuning on a financial dataset suggests suitability for tasks requiring deep understanding or generation of financial text. Users should be aware that detailed performance metrics and specific use cases are not yet fully documented.