Model Overview

openaccess-ai-collective/neft-exp2 is an experimental fine-tuned language model derived from the mistralai/Mistral-7B-v0.1 base architecture. This model was developed by the OpenAccess AI Collective using the Axolotl training framework.

Training Details

The model underwent 4 epochs of training with a learning rate of 6e-06, utilizing an Adam optimizer with betas=(0.9, 0.999) and epsilon=1e-08. Training was distributed across 8 GPUs with a total batch size of 16. The final validation loss achieved was 1.3578. Specific details regarding the dataset used for fine-tuning are not provided in the available documentation.

Key Characteristics

• Base Model: mistralai/Mistral-7B-v0.1
• Training Framework: Axolotl
• Learning Rate: 6e-06
• Epochs: 4
• Final Validation Loss: 1.3578

Intended Uses & Limitations

Further information regarding the model's intended uses, specific capabilities, and known limitations is not detailed in the current documentation. Users should consider this an experimental model given the lack of explicit use case guidance and dataset information.