AmberYifan/capsd-opc-dedup-marin-8b-base-code_cap_b60000_s0
The AmberYifan/capsd-opc-dedup-marin-8b-base-code_cap_b60000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for code-related tasks, leveraging a specialized dataset for its training. It is designed to enhance performance in code generation and understanding within its 8192-token context window.
Loading preview...
Model Overview
This model, AmberYifan/capsd-opc-dedup-marin-8b-base-code_cap_b60000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for code-centric applications.
Key Training Details
The model underwent fine-tuning on the capsd_marin-8b-base-n80000-opc-dedup80k__mix_code_cap_b60000_s0 dataset, indicating a focus on code-related data. Training was conducted with the following hyperparameters:
- Learning Rate: 1e-05
- Batch Sizes:
train_batch_sizeof 2,eval_batch_sizeof 8 - Optimizer: ADAMW_TORCH with default betas and epsilon
- LR Scheduler: Cosine type with 0.03 warmup steps
- Epochs: 1
Technical Stack
The training environment utilized:
- Transformers 5.7.0
- Pytorch 2.13.0+cu130
- Datasets 4.0.0
- Tokenizers 0.22.2
Intended Use
Given its fine-tuning on a code-specific dataset, this model is primarily intended for tasks involving code generation, completion, or analysis. Its 8192-token context length supports processing moderately sized code snippets.