AmberYifan/capsd-marin-8b-base-code_dsir_b8000_s0
AmberYifan/capsd-marin-8b-base-code_dsir_b8000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was specifically trained on the capsd_marin-8b-base-n80000-opc__mix_code_dsir_b8000_s0 dataset, indicating a specialization in code-related tasks. With a context length of 8192 tokens, it is designed for applications requiring code understanding and generation.
Loading preview...
Model Overview
The AmberYifan/capsd-marin-8b-base-code_dsir_b8000_s0 is an 8 billion parameter language model derived from marin-community/marin-8b-base. This model has undergone a specific fine-tuning process, utilizing the capsd_marin-8b-base-n80000-opc__mix_code_dsir_b8000_s0 dataset. This specialized training suggests an optimization for tasks involving code.
Training Details
The model was trained with a learning rate of 1e-05 and a train_batch_size of 2, accumulating gradients over 8 steps for an effective total batch size of 64. It utilized a cosine learning rate scheduler with 0.03 warmup steps over a single epoch. The training was conducted on a multi-GPU setup with 4 devices, using the AdamW_TORCH optimizer.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Specialization: Training on a code-centric dataset implies a focus on code-related understanding and generation tasks.
Intended Use Cases
While specific intended uses and limitations are not detailed in the provided information, the model's fine-tuning on a code-specific dataset suggests its suitability for:
- Code generation and completion.
- Code analysis and understanding.
- Assisting with programming tasks.