AmberYifan/capsdnum-marin-8b-base-code_cap_b4000_s0
AmberYifan/capsdnum-marin-8b-base-code_cap_b4000_s0 is a fine-tuned version of the marin-community/marin-8b-base model, specifically adapted using the capsd_marin-8b-base-n80000-opc__mix_code_cap_b4000_s0 dataset. This model is likely optimized for code-related tasks, given its fine-tuning on a dataset with 'code_cap' in its name. It was trained with a learning rate of 1e-05 over 1 epoch, utilizing a multi-GPU setup.
Loading preview...
Model Overview
This model, AmberYifan/capsdnum-marin-8b-base-code_cap_b4000_s0, is a fine-tuned iteration of the marin-community/marin-8b-base foundational model. The fine-tuning process specifically leveraged the capsd_marin-8b-base-n80000-opc__mix_code_cap_b4000_s0 dataset, suggesting a specialization towards code-related applications or tasks involving code understanding and generation.
Training Details
The model underwent a single training epoch with a learning rate of 1e-05. Key hyperparameters included a train_batch_size of 2 and an eval_batch_size of 8, with a total effective batch size of 64 due to gradient_accumulation_steps set to 8. Training was conducted across 4 devices in a multi-GPU distributed setup, using the AdamW optimizer and a cosine learning rate scheduler with 0.03 warmup steps. The training utilized Transformers version 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, and Tokenizers 0.22.2.
Potential Use Cases
Given its fine-tuning on a code-centric dataset, this model is likely suitable for:
- Code generation and completion
- Code summarization
- Bug detection or code analysis
Further details on specific capabilities, limitations, and comprehensive evaluation data are not provided in the original model card.