openaccess-ai-collective/neft-exp4
The openaccess-ai-collective/neft-exp4 model is a fine-tuned version of Mistral-7B-v0.1, developed by OpenAccess AI Collective. This 7 billion parameter causal language model was trained using Axolotl with specific hyperparameters including a learning rate of 6e-06 over 4 epochs. It achieved a final validation loss of 1.3725, indicating its performance on the evaluation set. This model is a foundational fine-tune, with further details on its specific intended uses and limitations requiring additional information.
Loading preview...
Model Overview
The openaccess-ai-collective/neft-exp4 is a fine-tuned language model based on the Mistral-7B-v0.1 architecture. Developed by the OpenAccess AI Collective, this model leverages the 7 billion parameter Mistral base model and was trained using the Axolotl framework.
Training Details
The model underwent a fine-tuning process over 4 epochs with a learning rate of 6e-06. Key training hyperparameters included a train_batch_size of 2, eval_batch_size of 2, and an Adam optimizer. The training utilized 8 devices, resulting in a total train and eval batch size of 16. The training concluded with a validation loss of 1.3725.
Current Status
As indicated in the model card, further information is needed regarding its specific intended uses, limitations, and the training and evaluation datasets. This suggests it is an experimental or foundational fine-tune, awaiting more detailed documentation on its specialized capabilities or optimal applications.