AmberYifan/capsd-final-retrain-marin-8b-base-code_cap_b8000_s0
AmberYifan/capsd-final-retrain-marin-8b-base-code_cap_b8000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was trained on the capsd_marin-8b-base-n80000-opc__mix_code_cap_b8000_s0 dataset, suggesting an optimization for code-related tasks. It features an 8192 token context length, making it suitable for processing moderately long sequences of text or code.
Loading preview...
Overview
This model, AmberYifan/capsd-final-retrain-marin-8b-base-code_cap_b8000_s0, is an 8 billion parameter language model. It is a fine-tuned version of the marin-community/marin-8b-base architecture, specifically adapted using the capsd_marin-8b-base-n80000-opc__mix_code_cap_b8000_s0 dataset. The dataset name implies a focus on code-related data, suggesting its primary strength lies in code understanding or generation tasks.
Training Details
The model underwent 3 epochs of training with a learning rate of 1e-05. It utilized a cosine learning rate scheduler with 0.03 warmup steps. Training was performed on 4 devices with a total batch size of 64, using the AdamW_TORCH optimizer.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192 token context window.
- Training Data: Fine-tuned on a dataset with "code_cap" in its name, indicating a specialization towards code-related tasks.
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely well-suited for:
- Code completion and generation.
- Code summarization or explanation.
- Assisting with programming tasks that benefit from a model trained on a substantial code corpus.