AmberYifan/capsd-marin-8b-base-code_ifd_b8000_s0
The AmberYifan/capsd-marin-8b-base-code_ifd_b8000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for code-related tasks, leveraging a specialized dataset for improved performance in programming contexts. With an 8192 token context length, it is designed for applications requiring robust code understanding and generation capabilities.
Loading preview...
Model Overview
This model, AmberYifan/capsd-marin-8b-base-code_ifd_b8000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for code-related applications.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192 token context window.
- Training Data: Fine-tuned on the
capsd_marin-8b-base-n80000-opc__mix_code_ifd_b8000_s0dataset, indicating a focus on code-centric tasks.
Training Details
The fine-tuning process involved a learning rate of 1e-05, a train_batch_size of 2, and a gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. The model was trained for 1 epoch using the AdamW optimizer with a cosine learning rate scheduler.
Potential Use Cases
Given its fine-tuning on a code-specific dataset, this model is likely suitable for:
- Code generation and completion.
- Code understanding and analysis.
- Assisting with programming tasks.