Hugging Face Overview
Llama3.1-Gutenberg-Doppel-70B Overview
This model, developed by nbeerbower, is a 70 billion parameter language model built upon the Llama 3.1 architecture. It is a fine-tuned version of mlabonne/Hermes-3-Llama-3.1-70B-lorablated.
Key Capabilities
- ORPO Tuning: The model was fine-tuned using the ORPO (Odds Ratio Preference Optimization) method over 3 epochs, leveraging two H100 GPUs. This method is known for effectively aligning models with human preferences.
- Literary Data Training: It was specifically trained on the jondurbin/gutenberg-dpo-v0.1 and nbeerbower/gutenberg2-dpo datasets, suggesting a specialization in generating text with a literary or classic style.
Performance Metrics
Evaluations on the Open LLM Leaderboard show the following average scores:
- Avg.: 35.68
- IFEval (0-Shot): 70.92
- BBH (3-Shot): 52.56
- MMLU-PRO (5-shot): 41.52
Good For
- Generating text in a literary or classic style.
- Tasks requiring nuanced language understanding and generation.
- Applications benefiting from a model trained on extensive literary datasets.