AmberYifan/capsd-marin-8b-base-code_ifd_b1000_s0
AmberYifan/capsd-marin-8b-base-code_ifd_b1000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for code-related tasks, having been fine-tuned on the capsd_marin-8b-base-n80000-opc__mix_code_ifd_b1000_s0 dataset. It is designed for applications requiring code generation, completion, or understanding within an 8192 token context window.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_ifd_b1000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This iteration focuses on enhancing code-related capabilities through specialized training.
Key Capabilities
- Code-centric Fine-tuning: The model has undergone fine-tuning on the
capsd_marin-8b-base-n80000-opc__mix_code_ifd_b1000_s0dataset, indicating an optimization for programming tasks. - Base Model: Built upon the
marin-8b-basefoundation, suggesting a general-purpose language understanding before its code-specific specialization. - Context Length: Supports an 8192 token context window, suitable for handling moderately sized code snippets and related instructions.
Training Details
The fine-tuning process utilized a learning rate of 1e-05, a total_train_batch_size of 64, and ran for 1 epoch. The optimizer used was ADAMW_TORCH with standard beta values and epsilon, employing a cosine learning rate scheduler with 0.03 warmup steps. The training was conducted on 4 GPUs.
Good For
- Code Generation: Assisting with writing new code based on prompts.
- Code Completion: Suggesting completions for partial code.
- Code Understanding: Potentially aiding in analyzing or explaining code segments, given its code-focused training.