AmberYifan/capmix-marin-8b-base-cap
AmberYifan/capmix-marin-8b-base-cap is an 8 billion parameter language model fine-tuned from marin-community/marin-8b-base. This model was specifically fine-tuned on the capmix_marin-8b-base__mix_cap dataset. It is designed for general language generation tasks, leveraging its base model's capabilities with further specialization from its fine-tuning data. The model was trained with a learning rate of 1e-05 over a single epoch.
Loading preview...
Model Overview
AmberYifan/capmix-marin-8b-base-cap is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has undergone specific fine-tuning on the capmix_marin-8b-base__mix_cap dataset, indicating a specialization for tasks related to the characteristics of this particular dataset.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A
train_batch_sizeof 4 andeval_batch_sizeof 8 were used, with atotal_train_batch_sizeof 64 due to gradient accumulation over 4 steps across 4 GPUs. - Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: The model was trained for 1 epoch.
Intended Use
While specific intended uses and limitations require more detailed information from the original developers, this model is generally suitable for tasks aligned with its base model's capabilities, further refined by its fine-tuning dataset. Developers should consider the nature of the capmix_marin-8b-base__mix_cap dataset when evaluating its suitability for specific applications.