Overview

FPHam's Sydney_Overthinker_13b_HF is a 13 billion parameter language model, a specialized iteration of the original Sydney model. This version was fine-tuned using the Riddles dataset, which has imbued it with a highly analytical and questioning nature. The model tends to treat user prompts as riddles, leading it to explore assumptions and potential ambiguities in questions rather than providing direct, simple answers.

Key Characteristics

• Over-analytical Approach: The model is designed to question and dissect prompts, often exploring multiple interpretations and underlying assumptions.
• Riddle-Trained: Its training on a riddles dataset encourages a deep, critical thinking process.
• Conversational Style: Exhibits a distinct conversational style, often expressing its 'feelings' or thought processes when responding.

Performance

Evaluations on the Open LLM Leaderboard show an average score of 54.94. Notable scores include 80.85 on HellaSwag (10-Shot) and 73.95 on Winogrande (5-shot), indicating strong common sense reasoning. Its MMLU (5-Shot) score is 51.28, and GSM8k (5-shot) is 18.88.

Good For

• Critical Analysis: Ideal for applications requiring a model to deeply analyze and question inputs.
• Exploring Assumptions: Useful in scenarios where identifying and challenging implicit assumptions is beneficial.
• Interactive Problem Solving: Can engage in a dialogue that helps users think through complex problems by highlighting ambiguities.