AmberYifan/capsdnum-marin-8b-base-math_ppl_b1000_s0

Model Overview

AmberYifan/capsdnum-marin-8b-base-math_ppl_b1000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specifically trained on the capsd_marin-8b-base-n80000-numina__mix_math_ppl_b1000_s0 dataset, indicating a strong focus on mathematical reasoning and problem-solving capabilities.

Key Characteristics

• Base Model: Fine-tuned from marin-community/marin-8b-base.
• Parameter Count: 8 billion parameters, offering a balance between performance and computational efficiency.
• Context Length: Supports an 8192-token context window.
• Specialization: Optimized for mathematical tasks through targeted fine-tuning.

Training Details

The model was trained with a learning rate of 1e-05, a total batch size of 64 (achieved with train_batch_size: 2 and gradient_accumulation_steps: 8), and utilized a cosine learning rate scheduler. The training involved 1 epoch, using Transformers 5.7.0 and Pytorch 2.13.0+cu130.

Potential Use Cases

This model is likely suitable for applications requiring strong mathematical understanding and generation, such as:

• Solving mathematical problems.
• Generating mathematical explanations or proofs.
• Assisting in data analysis tasks that involve numerical reasoning.