langazov/mistral-finetuned-jsonl
The langazov/mistral-finetuned-jsonl model is a 7 billion parameter language model fine-tuned from Mistral AI's Mistral-7B-v0.1. This model was fine-tuned on an unspecified dataset, achieving a validation loss of 0.4067. Its specific capabilities and intended uses are not detailed, but it is based on the Mistral architecture known for strong performance in its size class.
Loading preview...
Model Overview
The langazov/mistral-finetuned-jsonl is a 7 billion parameter language model, derived from the mistralai/Mistral-7B-v0.1 base model. It has undergone fine-tuning on an undisclosed dataset, resulting in a final validation loss of 0.4067.
Training Details
The model was trained for 3 epochs using the following key hyperparameters:
- Learning Rate: 2e-05
- Batch Size: A
train_batch_sizeof 2 withgradient_accumulation_stepsof 8, leading to atotal_train_batch_sizeof 16. - Optimizer: ADAMW_TORCH_FUSED with default betas and epsilon.
- Scheduler: Linear learning rate scheduler.
During training, the validation loss progressively decreased:
- Epoch 1: 0.6150
- Epoch 2: 0.4108
- Epoch 3: 0.4067
Current Status
Specific details regarding the fine-tuning dataset, the model's intended uses, limitations, and detailed capabilities are currently not provided in the model card. Users should be aware that its performance characteristics beyond the reported loss are not specified.