AmberYifan/capsd-less-ultra-humaneval-opc-marin-8b-base-code_less_b2000_s0
The AmberYifan/capsd-less-ultra-humaneval-opc-marin-8b-base-code_less_b2000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model is specifically fine-tuned on a dataset incorporating code, suggesting an optimization for code-related tasks. With a context length of 8192 tokens, it is designed for applications requiring processing of moderately long sequences, particularly in programming contexts.
Loading preview...
Model Overview
This model, AmberYifan/capsd-less-ultra-humaneval-opc-marin-8b-base-code_less_b2000_s0, is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. It has been fine-tuned on a specialized dataset, capsd_marin-8b-base-n80000-opc__mix_code_less_b2000_s0, which indicates a focus on code-related applications.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Training Focus: The fine-tuning dataset suggests an emphasis on code generation, understanding, or related programming tasks.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05. Key hyperparameters include a train_batch_size of 2, eval_batch_size of 8, and a gradient_accumulation_steps of 8, resulting in an effective total_train_batch_size of 64. The optimizer used was ADAMW_TORCH with a cosine learning rate scheduler and 0.03 warmup steps.
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely suitable for:
- Code completion and generation.
- Code summarization or explanation.
- Debugging assistance.
- Educational tools for programming.