AdamLucek/Orpo-Llama-3.2-1B-40k
AdamLucek/Orpo-Llama-3.2-1B-40k is a 1 billion parameter language model, fine-tuned using the ORPO method on the mlabonne/orpo-dpo-mix-40k dataset. This model is based on Meta's Llama-3.2-1B architecture and demonstrates improved performance on various reasoning and general knowledge benchmarks compared to its 15k-trained counterpart. It is optimized for tasks requiring general understanding and reasoning, making it suitable for applications where a smaller, yet capable, model is desired.
Loading preview...
Model Overview
AdamLucek/Orpo-Llama-3.2-1B-40k is a 1 billion parameter language model developed by AdamLucek. It is an ORPO (Optimized Reward Policy Optimization) fine-tuned version of Meta's Llama-3.2-1B, trained on a 40,000-sample mix of DPO data from the mlabonne/orpo-dpo-mix-40k dataset. The training process involved 1 epoch and approximately 11 hours on an A100 GPU.
Key Capabilities & Performance
This model shows general improvements across several benchmarks when compared to its 15k-trained predecessor (AdamLucek/Orpo-Llama-3.2-1B-15k). Notable performance metrics include:
- AGIEval: Achieves 23.57% accuracy (23.26% normalized) on 0-shot average across multiple reasoning tasks.
- GPT4ALL: Scores 51.63% accuracy (55.00% normalized) on 0-shot average across all categories.
- ARC Easy: Demonstrates 65.91% accuracy (60.90% normalized) in 0-shot evaluation.
- BoolQ: Reaches 52.29% accuracy in 0-shot evaluation.
Good For
- General-purpose text generation and understanding: Its ORPO fine-tuning on a diverse dataset enhances its ability to follow instructions and generate coherent responses.
- Applications requiring a compact model: As a 1 billion parameter model, it offers a balance between performance and computational efficiency.
- Research and experimentation with ORPO fine-tuning: Provides a practical example of ORPO application on a Llama-3.2 base model.