vicgalle/Miqu-6B-truthy
Miqu-6B-truthy is a 69 billion parameter language model developed by vicgalle, based on the Miqu architecture. This model is specifically fine-tuned as an experiment to enhance truthful responses, achieving a TruthfulQA score of 50.63. It is designed for applications requiring high factual accuracy and reduced hallucination, particularly in question-answering scenarios. The model supports a context length of 32768 tokens.
Loading preview...
Miqu-6B-truthy Overview
Miqu-6B-truthy is a 69 billion parameter language model, developed by vicgalle, that focuses on improving factual accuracy and reducing hallucination. This model is presented as an experimental variant of the Miqu architecture, specifically optimized for generating truthful responses.
Key Capabilities & Performance
The primary differentiator of Miqu-6B-truthy is its performance on truthfulness benchmarks. It achieves a TruthfulQA (0-shot) score of 50.63, indicating a significant focus on factual correctness. The model's truthfulqa_mc results show mc1 at 0.252 and mc2 at 0.505, further highlighting its design for truthful output. While excelling in truthfulness, other general benchmarks show moderate performance:
- AI2 Reasoning Challenge (25-Shot): 27.65
- HellaSwag (10-Shot): 26.71
- MMLU (5-Shot): 27.04
- Winogrande (5-shot): 49.64
- GSM8k (5-shot): 0.00
Detailed evaluation results are available on the Open LLM Leaderboard.
Good For
- Applications requiring high factual accuracy: Ideal for use cases where truthful and non-hallucinated responses are critical.
- Question-answering systems: Particularly effective in scenarios where the correctness of information is paramount.
- Research into truthfulness in LLMs: Serves as a valuable experimental model for studying and improving factual generation.