AmberYifan/capsd-less-ultra-opc-marin-8b-base-code_less_b2000_s0
AmberYifan/capsd-less-ultra-opc-marin-8b-base-code_less_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted using the capsd_marin-8b-base-n80000-opc__mix_code_less_b2000_s0 dataset, suggesting an optimization for code-related tasks. It operates with a context length of 8192 tokens, making it suitable for processing moderately long sequences of text or code.
Loading preview...
Model Overview
This model, AmberYifan/capsd-less-ultra-opc-marin-8b-base-code_less_b2000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for certain tasks through further training.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports a context window of 8192 tokens.
- Training Dataset: Fine-tuned on the
capsd_marin-8b-base-n80000-opc__mix_code_less_b2000_s0dataset, indicating a specialization or focus on code-related data.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05, a total training batch size of 64, and utilized a cosine learning rate scheduler. The training was performed using Transformers 5.7.0 and Pytorch 2.13.0+cu130.
Intended Use Cases
Given its fine-tuning on a dataset with "code" in its name, this model is likely optimized for tasks involving code generation, completion, analysis, or understanding. Developers looking for a model with a strong foundation in code-centric applications might find this model suitable.