AmberYifan/capsd-marin-8b-base-code_ppl_b16000_s0
AmberYifan/capsd-marin-8b-base-code_ppl_b16000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was specifically trained on the capsd_marin-8b-base-n80000-opc__mix_code_ppl_b16000_s0 dataset, indicating a specialization towards code-related tasks. It is designed for applications requiring a compact yet capable model with an 8192-token context length.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_ppl_b16000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specifically adapted through further training on a specialized dataset, capsd_marin-8b-base-n80000-opc__mix_code_ppl_b16000_s0.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Training Focus: The fine-tuning dataset name suggests a focus on code-related tasks, likely optimizing its performance for code generation, completion, or understanding.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05, using a total batch size of 64 across 4 GPUs. The optimizer used was ADAMW_TORCH with a cosine learning rate scheduler.
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely suitable for:
- Code generation and completion.
- Code analysis and understanding tasks.
- Applications requiring a compact model with code-specific capabilities.