AmberYifan/capsd-marin-8b-base-code_ifd_b2000_s0
The AmberYifan/capsd-marin-8b-base-code_ifd_b2000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for code-related tasks, having been trained on the capsd_marin-8b-base-n80000-opc__mix_code_ifd_b2000_s0 dataset. With an 8192 token context length, it is designed for applications requiring robust code generation and understanding capabilities.
Loading preview...
Model Overview
The AmberYifan/capsd-marin-8b-base-code_ifd_b2000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specifically adapted for code-related tasks through further training on a specialized dataset.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192 token context window.
- Training Data: Fine-tuned on the
capsd_marin-8b-base-n80000-opc__mix_code_ifd_b2000_s0dataset, indicating an optimization for code-centric applications.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05. It utilized a distributed training setup across 4 GPUs, with a total effective batch size of 64 (achieved with a train_batch_size of 2 and gradient_accumulation_steps of 8). The AdamW optimizer with cosine learning rate scheduling was employed.
Potential Use Cases
Given its fine-tuning on a code-specific dataset, this model is likely suitable for tasks such as:
- Code generation
- Code completion
- Code summarization
- Debugging assistance
Further information regarding specific intended uses, limitations, and detailed evaluation data is currently pending.