abeiler/AlphaRepInstruct
abeiler/AlphaRepInstruct is a fine-tuned version of the meta-llama/Llama-2-7b-hf model, developed by abeiler. This model is based on the Llama 2 architecture with 7 billion parameters, fine-tuned using QLORA. Specific details regarding its primary differentiators, intended uses, and training data are not provided in the available documentation, indicating it is a foundational fine-tune with potential for various applications.
Loading preview...
Model Overview
abeiler/AlphaRepInstruct is a fine-tuned language model based on the meta-llama/Llama-2-7b-hf architecture. This model was fine-tuned using the QLORA method, indicating an efficient adaptation of the base Llama 2 7B model. The specific dataset used for fine-tuning is currently unknown, and further details regarding its intended applications and unique capabilities are not provided in the available documentation.
Training Details
The model underwent a single epoch of training with the following key hyperparameters:
- Learning Rate: 0.0001
- Batch Size: 4 (train), 8 (eval)
- Optimizer: Adam with default betas and epsilon
- LR Scheduler: Linear
This fine-tuning process utilized Transformers 4.33.3, Pytorch 2.0.0, Datasets 2.12.0, and Tokenizers 0.13.3.
Current Status
As of the current documentation, specific information regarding the model's primary differentiators, performance benchmarks, and recommended use cases is pending. Users are encouraged to consult future updates for more detailed insights into its capabilities and limitations.