AmberYifan/capsdnum-marin-8b-base-code_random_b2000_s0
AmberYifan/capsdnum-marin-8b-base-code_random_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was specifically trained on the capsd_marin-8b-base-n80000-code__mix_code_random_b2000_s0 dataset, indicating an optimization for code-related tasks. It was trained for one epoch with a learning rate of 1e-05 and a total batch size of 64, suggesting a focus on specialized code generation or understanding.
Loading preview...
Model Overview
This model, capsdnum-marin-8b-base-code_random_b2000_s0, is an 8 billion parameter language model developed by AmberYifan. It is a fine-tuned variant of the marin-community/marin-8b-base model, specifically adapted for code-related applications.
Training Details
The model underwent a focused fine-tuning process using the capsd_marin-8b-base-n80000-code__mix_code_random_b2000_s0 dataset. Key training hyperparameters include:
- Learning Rate: 1e-05
- Optimizer: ADAMW_TORCH with default betas and epsilon
- LR Scheduler: Cosine type with 0.03 warmup steps
- Epochs: 1
- Total Training Batch Size: 64 (achieved with
train_batch_size=2,gradient_accumulation_steps=8, and 4 GPUs)
Intended Use
Given its fine-tuning on a code-specific dataset, this model is likely optimized for tasks such as:
- Code generation
- Code completion
- Code understanding and analysis
Framework Versions
The training was conducted using:
- Transformers 5.7.0
- Pytorch 2.13.0+cu130
- Datasets 4.0.0
- Tokenizers 0.22.2