AmberYifan/capsd-marin-8b-base-math_kcenter_b4000_s0

Model Overview

AmberYifan/capsd-marin-8b-base-math_kcenter_b4000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. Its primary specialization is in mathematical tasks, achieved through training on the capsd_marin-8b-base-n80000-numina__mix_math_kcenter_b4000_s0 dataset.

Key Capabilities

• Mathematical Reasoning: Optimized for handling mathematical problems and queries.
• Fine-tuned Performance: Leverages a base model with further specialization for enhanced accuracy in its target domain.
• 8B Parameters: Offers a balance of capability and computational efficiency for specialized tasks.
• 8192 Token Context Length: Supports processing longer mathematical problems or related contexts.

Training Details

The model was trained with a learning rate of 1e-05, a total batch size of 64, and utilized a cosine learning rate scheduler. Training involved 1 epoch on 4 GPUs, using AdamW optimizer. The training environment included Transformers 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, and Tokenizers 0.22.2.

Ideal Use Cases

This model is particularly well-suited for applications that require robust mathematical understanding and generation, such as:

• Solving mathematical equations and word problems.
• Assisting in educational tools for math.
• Generating mathematical explanations or proofs.

Users should note that while it excels in its specialized domain, its general-purpose capabilities might be less pronounced compared to models trained for broader applications.