AmberYifan/capsd-marin-8b-base-code_random_b16000_s0
AmberYifan/capsd-marin-8b-base-code_random_b16000_s0 is an 8 billion parameter language model fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for code-related tasks, having been trained on the capsd_marin-8b-base-n80000-opc__mix_code_random_b16000_s0 dataset. It is designed for applications requiring code generation, completion, or understanding, leveraging its 8192 token context length.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_random_b16000_s0 is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. This model has undergone specific fine-tuning to enhance its capabilities in code-related domains.
Key Training Details
The model was fine-tuned using the capsd_marin-8b-base-n80000-opc__mix_code_random_b16000_s0 dataset. The training procedure involved specific hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A
train_batch_sizeof 2 andeval_batch_sizeof 8 were used, leading to atotal_train_batch_sizeof 64 with gradient accumulation. - Optimizer: ADAMW_TORCH with default betas and epsilon.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
Intended Use
While specific intended uses and limitations require further detail, the model's fine-tuning on a code-centric dataset suggests its primary application lies in tasks such as code generation, code completion, debugging assistance, or understanding programming language constructs. Its 8192 token context length supports processing moderately sized code snippets.