openaccess-ai-collective/neft-exp2
openaccess-ai-collective/neft-exp2 is a fine-tuned language model based on the Mistral-7B-v0.1 architecture. Developed by OpenAccess AI Collective, this model was trained using Axolotl with specific hyperparameters including a learning rate of 6e-06 and 4 epochs. While specific primary differentiators and intended uses are not detailed, it represents an experimental fine-tuning effort on an unspecified dataset.
Loading preview...
Model Overview
openaccess-ai-collective/neft-exp2 is an experimental fine-tuned language model derived from the mistralai/Mistral-7B-v0.1 base architecture. This model was developed by the OpenAccess AI Collective using the Axolotl training framework.
Training Details
The model underwent 4 epochs of training with a learning rate of 6e-06, utilizing an Adam optimizer with betas=(0.9, 0.999) and epsilon=1e-08. Training was distributed across 8 GPUs with a total batch size of 16. The final validation loss achieved was 1.3578. Specific details regarding the dataset used for fine-tuning are not provided in the available documentation.
Key Characteristics
- Base Model: mistralai/Mistral-7B-v0.1
- Training Framework: Axolotl
- Learning Rate: 6e-06
- Epochs: 4
- Final Validation Loss: 1.3578
Intended Uses & Limitations
Further information regarding the model's intended uses, specific capabilities, and known limitations is not detailed in the current documentation. Users should consider this an experimental model given the lack of explicit use case guidance and dataset information.