dfurman/Llama-3-8B-Orpo-v0.1: ORPO Fine-tune of Llama-3-8B

This model is an 8 billion parameter language model developed by dfurman, fine-tuned from the meta-llama/Meta-Llama-3-8B base model using the ORPO (Orthogonalized Representation Policy Optimization) method. The fine-tuning process involved 4,000 samples from the mlabonne/orpo-dpo-mix-40k dataset.

Key Capabilities & Features

• Architecture: Based on Meta-Llama-3-8B.
• Fine-tuning Method: Utilizes ORPO for enhanced performance.
• Context Window: Supports an 8k token context length.
• Chat Template: Adheres to the ChatML template for structured conversations.
• Performance Improvements: Compared to the base Meta-Llama-3-8B model, this ORPO fine-tune shows notable improvements in:
  • HellaSwag: 82.56% (vs. 82.02% for base).
  • Winogrande: 79.01% (vs. 77.11% for base).

Usage & Application

This model is designed for applications requiring a robust conversational AI, leveraging its ORPO fine-tuning for improved response quality. It is particularly well-suited for tasks that benefit from its enhanced performance on commonsense reasoning and coreference resolution benchmarks. The model's adherence to the ChatML template simplifies integration into existing conversational pipelines.