AmberYifan/capsdnum-marin-8b-base-code_ppl_b1000_s0
The AmberYifan/capsdnum-marin-8b-base-code_ppl_b1000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. It was trained on the capsd_marin-8b-base-n80000-opc__mix_code_ppl_b1000_s0 dataset, suggesting an optimization for code-related tasks. This model is likely intended for applications requiring code generation or understanding, leveraging its base architecture and specialized fine-tuning.
Loading preview...
Model Overview
This model, AmberYifan/capsdnum-marin-8b-base-code_ppl_b1000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted through further training.
Key Training Details
The model underwent fine-tuning on a dataset identified as capsd_marin-8b-base-n80000-opc__mix_code_ppl_b1000_s0. This dataset name strongly implies a focus on code-related data, suggesting the model's capabilities are enhanced for programming tasks. The training procedure involved:
- Learning Rate: 1e-05
- Batch Size: 2 (train), 8 (eval)
- Gradient Accumulation Steps: 8, leading to a total train batch size of 64
- Optimizer: ADAMW_TORCH
- LR Scheduler: Cosine type with 0.03 warmup steps
- Epochs: 1
Framework Versions
The training utilized:
- Transformers 5.7.0
- Pytorch 2.13.0+cu130
- Datasets 4.0.0
- Tokenizers 0.22.2
Intended Use
Given its fine-tuning on a code-centric dataset, this model is likely optimized for applications such as code generation, code completion, debugging assistance, or understanding programming language constructs.