AmberYifan/capsd-marin-8b-base-code_qurating_b4000_s0
AmberYifan/capsd-marin-8b-base-code_qurating_b4000_s0 is an 8 billion parameter language model fine-tuned from marin-community/marin-8b-base. This model was specifically trained on the capsd_marin-8b-base-n80000-opc__mix_code_qurating_b4000_s0 dataset, indicating a specialization in code-related tasks. It is designed for applications requiring code understanding and generation, leveraging its 8192 token context length.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_qurating_b4000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This iteration focuses on code-related applications, having been specifically trained on the capsd_marin-8b-base-n80000-opc__mix_code_qurating_b4000_s0 dataset.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192 token context window.
- Specialization: The training dataset suggests a strong focus on code-qurating tasks, implying enhanced performance in code understanding, generation, or analysis.
Training Details
The model was trained using a learning rate of 1e-05, a total batch size of 64 (with a train batch size of 2 and gradient accumulation steps of 8), and the AdamW_Torch optimizer. The training consisted of 1 epoch with a cosine learning rate scheduler and 0.03 warmup steps. The training utilized a multi-GPU setup with 4 devices.
Intended Use Cases
Given its specialized training on a code-qurating dataset, this model is likely best suited for:
- Code generation and completion.
- Code review and analysis.
- Understanding and explaining code snippets.
- Tasks requiring deep comprehension of programming logic and structure.