AmberYifan/capsd-marin-8b-base-code_random_b10000_s0
AmberYifan/capsd-marin-8b-base-code_random_b10000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for code-related tasks, having been trained on the capsd_marin-8b-base-n80000-opc__mix_code_random_b10000_s0 dataset. It features a context length of 8192 tokens and is intended for applications requiring code generation or understanding.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-code_random_b10000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specialized through training on a custom dataset, capsd_marin-8b-base-n80000-opc__mix_code_random_b10000_s0, indicating a focus on code-related applications. It supports a context length of 8192 tokens.
Training Details
The model underwent a single epoch of fine-tuning with a learning rate of 1e-05. Key training hyperparameters included a train_batch_size of 2, an eval_batch_size of 8, and a gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. The optimizer used was ADAMW_TORCH with default betas and epsilon, and a cosine learning rate scheduler was employed with 0.03 warmup steps. The training was conducted using a multi-GPU setup with 4 devices.
Intended Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely best suited for tasks such as:
- Code generation
- Code completion
- Code summarization
- Assisting with programming-related queries