Excalibur-7b-DPO: Enhanced Conversational and Vision Capabilities

Excalibur-7b-DPO is a 7 billion parameter model developed by InferenceIllusionist, representing an initial fine-tuning experiment using Direct Preference Optimization (DPO) on the Excalibur-7b base. The primary goal of this fine-tuning was to enhance the quality of responses, particularly for vision use cases, and make the model more conversational and well-rounded.

Key Capabilities & Improvements

• Direct Preference Optimization (DPO): Fine-tuned using the Intel/orca_dpo_pairs dataset, resulting in improved conversational abilities.
• Enhanced Vision Functionality: Designed with amplified capabilities for vision-based tasks, requiring an additional mmproj file for full operation. Users can choose between quantized (197MB) and unquantized (596MB) mmproj options.
• Benchmark Performance: Demonstrates improved scores over the base Excalibur-7b model in key categories:
  • ARC: 69.71 -> 70.9
  • HellaSwag: 87.56 -> 87.93
  • TruthfulQA: 67.24 -> 70.82
  • Average: 73.6 -> 73.84
• Prompt Format: Best results are achieved using the ChatML prompt format, with Alpaca also supported.

Use Cases

This model is suitable for applications requiring improved conversational fluency, general reasoning, and integrated vision capabilities, especially where a 7B parameter model is desired for efficiency.