AmberYifan/capsd-marin-8b-base-code_ppl_b2000_s0
AmberYifan/capsd-marin-8b-base-code_ppl_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for code-related tasks, having been trained on a mixed code dataset. It is designed for applications requiring code generation, understanding, or processing, leveraging its base architecture and specialized training for improved performance in programming contexts.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_ppl_b2000_s0 is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. This model has undergone a specific fine-tuning process on the capsd_marin-8b-base-n10000__mix_code_ppl_b2000_s0 dataset, indicating a specialization towards code-related tasks.
Key Training Details
The model was trained with a learning rate of 1e-05, a total batch size of 64 (achieved with train_batch_size: 1 and gradient_accumulation_steps: 16 across 4 devices), and utilized the AdamW optimizer. The training procedure included a cosine learning rate scheduler with 0.03 warmup steps over 1 epoch. The development environment used Transformers 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, and Tokenizers 0.22.2.
Intended Use Cases
Given its fine-tuning on a mixed code dataset, this model is primarily intended for applications that benefit from enhanced code understanding and generation capabilities. Developers can consider this model for tasks such as code completion, code summarization, bug fixing, or generating code snippets based on natural language prompts. Its 8 billion parameters and specialized training make it a candidate for efficient code-centric AI development.