AmberYifan/capsd-opc-dedup-marin-8b-base-code_ppl_b40000_s0
The AmberYifan/capsd-opc-dedup-marin-8b-base-code_ppl_b40000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for code-related tasks, having been trained on a dedicated code dataset. It is designed for applications requiring strong performance in code generation and understanding within an 8192 token context window.
Loading preview...
Model Overview
This model, AmberYifan/capsd-opc-dedup-marin-8b-base-code_ppl_b40000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for improved performance on code-related tasks.
Key Capabilities
- Code-centric Fine-tuning: The model has undergone specialized fine-tuning on the
capsd_marin-8b-base-n80000-opc-dedup80k__mix_code_ppl_b40000_s0dataset, indicating an optimization for code generation, completion, and understanding. - Base Model Architecture: Built upon the
marin-8b-basefoundation, suggesting a robust general language understanding prior to code specialization. - Context Window: Supports an 8192-token context length, allowing for processing and generating longer code snippets or related documentation.
Training Details
The model was trained with a learning rate of 1e-05, a total batch size of 64 (achieved with train_batch_size 2 and gradient_accumulation_steps 8), and utilized a cosine learning rate scheduler. The training involved 1 epoch on 4 GPUs.
Good For
- Developers and researchers working on code generation, code completion, or code analysis tasks.
- Applications requiring a capable 8B parameter model with a strong emphasis on programming language understanding.