AmberYifan/capsd-marin-8b-base-n80000-opc-r32-marin-8b-base-code_cap_b8000_s0
AmberYifan/capsd-marin-8b-base-n80000-opc-r32-marin-8b-base-code_cap_b8000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was trained on the capsd_R32__mix_code_cap_b8000_s0 dataset, suggesting a specialization in code-related tasks. It is designed for applications requiring a compact yet capable model with a focus on code understanding or generation, leveraging its 8192 token context length.
Loading preview...
Model Overview
This model, AmberYifan/capsd-marin-8b-base-n80000-opc-r32-marin-8b-base-code_cap_b8000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted through training on the capsd_R32__mix_code_cap_b8000_s0 dataset. This fine-tuning process indicates an optimization for tasks related to code, making it distinct from general-purpose language models.
Key Training Details
- Base Model: Fine-tuned from marin-community/marin-8b-base.
- Dataset: Trained on the
capsd_R32__mix_code_cap_b8000_s0dataset. - Hyperparameters: Utilized a learning rate of 1e-05, a total training batch size of 64, and a cosine learning rate scheduler over 1 epoch.
- Frameworks: Developed using Transformers 5.8.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, and Tokenizers 0.22.2.
Potential Use Cases
Given its fine-tuning on a code-specific dataset, this model is likely suitable for:
- Code generation and completion.
- Code analysis and understanding.
- Assisting with programming tasks where an 8B parameter model with an 8192 token context window is appropriate.