RikkiXu/zephyr-7b-dpo-full
RikkiXu/zephyr-7b-dpo-full is a 7 billion parameter language model. This model was trained from scratch, though specific details on its architecture, primary differentiators, and intended uses are not provided in its current documentation. It was trained using specific hyperparameters including a learning rate of 1e-09 and a total batch size of 128 over one epoch.
Loading preview...
Model Overview
RikkiXu/zephyr-7b-dpo-full is a 7 billion parameter language model. The model was trained from scratch, but its specific architecture, the dataset used for training, and its primary differentiators are not detailed in the available documentation. It was developed using Transformers 4.41.1 and PyTorch 2.1.2+cu118.
Training Details
The training process involved specific hyperparameters:
- Learning Rate: 1e-09
- Batch Size: 4 (train and eval), with a total train batch size of 128
- Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- Scheduler: Cosine learning rate scheduler with a 0.1 warmup ratio
- Epochs: 1
- Devices: Multi-GPU setup with 8 devices
Limitations
Detailed information regarding the model's intended uses, limitations, and evaluation data is currently not available. Users should exercise caution and conduct thorough testing for specific applications due to the lack of comprehensive documentation on its capabilities and performance.