AmberYifan/capsd-marin-8b-base-n80000-opc-r144-marin-8b-base-code_cap_b8000_s0
AmberYifan/capsd-marin-8b-base-n80000-opc-r144-marin-8b-base-code_cap_b8000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was specifically fine-tuned on the capsd_R144__mix_code_cap_b8000_s0 dataset, suggesting an optimization for code-related tasks. It utilizes a context length of 8192 tokens, making it suitable for processing moderately long sequences of text or code.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-n80000-opc-r144-marin-8b-base-code_cap_b8000_s0 is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. It has been fine-tuned with a focus on code-related data, specifically using the capsd_R144__mix_code_cap_b8000_s0 dataset.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Training Focus: The fine-tuning process involved a dataset explicitly named for code, indicating a specialization in code understanding or generation.
Training Details
The model was trained with a learning rate of 1e-05, a total batch size of 64, and for 1 epoch. It utilized an AdamW optimizer with a cosine learning rate scheduler. The training was conducted using Transformers 5.8.0 and Pytorch 2.13.0+cu130.
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely best suited for applications involving:
- Code generation.
- Code completion.
- Code analysis or understanding tasks.
- Developer assistance tools.