AmberYifan/capsd-marin-8b-base-code_qurating_b2000_s0
AmberYifan/capsd-marin-8b-base-code_qurating_b2000_s0 is an 8 billion parameter language model fine-tuned from marin-community/marin-8b-base. This model was trained on the capsd_marin-8b-base-n80000-opc__mix_code_qurating_b2000_s0 dataset, indicating a specialization in code-related tasks. It utilizes an 8192-token context length and was trained with a learning rate of 1e-05 over one epoch.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_qurating_b2000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specifically adapted using the capsd_marin-8b-base-n80000-opc__mix_code_qurating_b2000_s0 dataset, suggesting a focus on code generation, analysis, or related programming tasks.
Training Details
The model underwent a single training epoch with a learning rate of 1e-05. Key hyperparameters included a train_batch_size of 2, eval_batch_size of 8, and a gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. The optimizer used was ADAMW_TORCH with cosine learning rate scheduling and a warmup of 0.03 steps. The training was conducted across 4 GPUs.
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely suitable for applications requiring:
- Code completion and generation.
- Code summarization or explanation.
- Debugging assistance.
- Refactoring suggestions.