AmberYifan/capsd-marin-8b-base-code_qurating_b14000_s0
The AmberYifan/capsd-marin-8b-base-code_qurating_b14000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for code-related tasks, leveraging a specialized dataset for its training. It is designed to enhance performance in code generation and understanding within its 8192-token context window.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_qurating_b14000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This iteration has undergone specific training on the capsd_marin-8b-base-n80000-opc__mix_code_qurating_b14000_s0 dataset, indicating a specialization towards code-related applications.
Training Details
The model was trained with a learning rate of 1e-05, a train_batch_size of 2, and a gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64 across 4 GPUs. It utilized the AdamW optimizer and a cosine learning rate scheduler with 0.03 warmup steps over 1 epoch. The training environment included Transformers 5.7.0 and Pytorch 2.13.0+cu130.
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely suitable for tasks such as:
- Code generation
- Code completion
- Code summarization
- Debugging assistance
Further details regarding specific performance metrics, intended uses, and limitations are not provided in the current model card.