layai/syn-arxiv-dict
The layai/syn-arxiv-dict model is an 8 billion parameter language model fine-tuned from Meta-Llama-3-8B. It was trained on an unspecified dataset, achieving a validation accuracy of 0.7009. This model is a general-purpose language model, with its specific differentiators and primary use cases not explicitly detailed in the provided information.
Loading preview...
Model Overview
The layai/syn-arxiv-dict model is an 8 billion parameter language model, fine-tuned from the meta-llama/Meta-Llama-3-8B base architecture. The fine-tuning process involved an unknown dataset, and the model achieved a validation accuracy of 0.7009 with a final validation loss of 2.2990.
Training Details
The model was trained using the following key hyperparameters:
- Learning Rate: 5e-05
- Batch Size: 40 (train and eval), with a total effective batch size of 160 due to gradient accumulation steps (4).
- Optimizer: Adam with default betas and epsilon.
- LR Scheduler: Cosine type.
- Epochs: 3.0
Performance Metrics
During training, the model's performance evolved as follows:
- Epoch 0.87: Validation Loss: 2.3493, Accuracy: 0.6701
- Epoch 1.74: Validation Loss: 2.2990, Accuracy: 0.7009
- Epoch 2.61: Validation Loss: 2.4136, Accuracy: 0.7041
Intended Uses & Limitations
Specific intended uses and limitations are not detailed in the provided information. Developers should conduct further evaluation to determine suitability for particular applications.