Model Overview

The langazov/mistral-finetuned-jsonl is a 7 billion parameter language model, derived from the mistralai/Mistral-7B-v0.1 base model. It has undergone fine-tuning on an undisclosed dataset, resulting in a final validation loss of 0.4067.

Training Details

The model was trained for 3 epochs using the following key hyperparameters:

• Learning Rate: 2e-05
• Batch Size: A train_batch_size of 2 with gradient_accumulation_steps of 8, leading to a total_train_batch_size of 16.
• Optimizer: ADAMW_TORCH_FUSED with default betas and epsilon.
• Scheduler: Linear learning rate scheduler.

During training, the validation loss progressively decreased:

• Epoch 1: 0.6150
• Epoch 2: 0.4108
• Epoch 3: 0.4067

Current Status

Specific details regarding the fine-tuning dataset, the model's intended uses, limitations, and detailed capabilities are currently not provided in the model card. Users should be aware that its performance characteristics beyond the reported loss are not specified.