Model Overview

The abeiler/NumAndAlphaInstruct-75-25-100K is a fine-tuned language model based on meta-llama/Llama-2-7b-hf. It was trained using a QLORA approach.

Training Details

The model underwent training with the following hyperparameters:

• Learning Rate: 0.0001
• Batch Size: 4 (train), 8 (eval)
• Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
• LR Scheduler Type: linear
• Epochs: 1

The training utilized Transformers 4.33.3, Pytorch 2.0.0, Datasets 2.12.0, and Tokenizers 0.13.3.

Limitations and Further Information

Currently, detailed information regarding the specific dataset used for fine-tuning, the model's intended uses, its primary capabilities, and any known limitations is not available in the provided documentation. Users should exercise caution and conduct further evaluation to determine its suitability for specific applications.