Model Overview

nbeerbower/Lyra4-Gutenberg-12B is a 12 billion parameter language model derived from the Sao10K/MN-12B-Lyra-v4 base model. It has been fine-tuned using the ORPO (Odds Ratio Preference Optimization) method over three epochs, leveraging a combination of RTX 3090 and 4060 Ti GPUs. The training utilized the jondurbin/gutenberg-dpo-v0.1 dataset, which is designed to align models with human preferences through DPO.

Key Capabilities and Performance

This model offers a substantial 32768 token context window, enabling it to process and generate longer, more coherent texts. Its performance has been evaluated on the Open LLM Leaderboard, showing an average score of 19.63. Specific benchmark results include:

• IFEval (0-Shot): 22.12
• BBH (3-Shot): 34.24
• MATH Lvl 5 (4-Shot): 11.71
• GPQA (0-shot): 9.17
• MuSR (0-shot): 11.97
• MMLU-PRO (5-shot): 28.57

These metrics indicate its capabilities in instruction following, complex reasoning, and general knowledge tasks, particularly in zero-shot and few-shot settings.

Good For

• Applications requiring a model with a large context window for processing extensive documents.
• Tasks benefiting from DPO-aligned responses, such as instruction following and preference-based generation.
• Research and development in fine-tuning techniques like ORPO on consumer-grade hardware setups.