AmberYifan/capsd-marin-8b-base-code_cap_b14000_s0
AmberYifan/capsd-marin-8b-base-code_cap_b14000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was trained with a learning rate of 1e-05 and a total batch size of 64 over one epoch, specifically on the capsd_marin-8b-base-n80000-opc__mix_code_cap_b14000_s0 dataset. Its primary application is likely in code-related tasks, given the dataset used for fine-tuning.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_cap_b14000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model was specifically adapted using the capsd_marin-8b-base-n80000-opc__mix_code_cap_b14000_s0 dataset, indicating a specialization towards code-related applications.
Training Details
The fine-tuning process involved the following key hyperparameters:
- Learning Rate: 1e-05
- Optimizer: ADAMW_TORCH with betas=(0.9, 0.999) and epsilon=1e-08
- LR Scheduler: Cosine type with 0.03 warmup steps
- Batch Size: A total training batch size of 64 (achieved with
train_batch_size: 2andgradient_accumulation_steps: 8across 4 GPUs) - Epochs: 1
Potential Use Cases
Given its fine-tuning on a dataset explicitly mentioning "code_cap", this model is likely optimized for:
- Code generation
- Code completion
- Code understanding and analysis
Further details on specific intended uses, limitations, and comprehensive evaluation data are not provided in the current model card.