AmberYifan/capsd-marin-8b-base-code_qurating_b8000_s0
AmberYifan/capsd-marin-8b-base-code_qurating_b8000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was specifically trained on a code-related dataset, capsd_marin-8b-base-n80000-opc__mix_code_qurating_b8000_s0, suggesting an optimization for code generation or understanding tasks. It utilizes a context length of 8192 tokens, making it suitable for processing moderately long code snippets or related textual data.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_qurating_b8000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specialized through training on the capsd_marin-8b-base-n80000-opc__mix_code_qurating_b8000_s0 dataset, indicating a focus on code-related applications.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Training Focus: Specialized on a dataset with "code_qurating" in its name, suggesting an emphasis on code quality, generation, or understanding.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Optimizer: ADAMW_TORCH with default betas and epsilon.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
- Batch Size: Total train batch size of 64 (2 per device, 8 gradient accumulation steps across 4 GPUs).
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely best suited for tasks involving:
- Code generation.
- Code completion.
- Code summarization or explanation.
- Code quality assessment or refactoring suggestions.