Hugging Face Overview
Lyra4-Gutenberg2-12B: An ORPO Fine-tuned 12B Model
Lyra4-Gutenberg2-12B is a 12 billion parameter language model developed by nbeerbower, building upon the Sao10K/MN-12B-Lyra-v4 base. This iteration features an increased sequence length compared to its predecessor, Lyra4-Gutenberg-12B, enhancing its ability to process longer contexts up to 32768 tokens.
Key Capabilities & Training:
- Base Model: Fine-tuned from Sao10K/MN-12B-Lyra-v4.
- Fine-tuning Method: Utilizes the ORPO (Odds Ratio Preference Optimization) technique, trained for 3 epochs.
- Training Data: Leveraged a combination of jondurbin/gutenberg-dpo-v0.1 and nbeerbower/gutenberg2-dpo datasets, with training data formatted using ChatML.
Performance Insights:
Evaluations on the Open LLM Leaderboard show an average score of 19.74. Specific metrics include:
- IFEval (0-Shot): 25.85
- BBH (3-Shot): 33.73
- MMLU-PRO (5-shot): 28.51
This model is suitable for applications requiring a balance of parameter size and extended context handling, particularly where DPO-style alignment is beneficial.