Delexa-V0.1-7b: A 7B Parameter Model for Question Answering

Delexa-V0.1-7b is a 7 billion parameter language model developed by lex-hue, notable for its performance in question-answering tasks. The model was trained using a unique technique that involves presenting one question alongside three different answer choices, which is cited as a key factor in its ability to generate robust and accurate responses.

Key Capabilities & Performance

• Question Answering: Achieved an average score of 8.19 on a specific question-answer evaluation, surpassing gpt-3.5-turbo (7.94) and claude-v1 (7.90) in this benchmark, though slightly behind gpt-4 (8.99).
• Context Length: Supports an 8192-token context window.
• Content Handling: Designed to allow 18+ and lewd content, while actively preventing illegal content.

Open LLM Leaderboard Evaluation

On the Open LLM Leaderboard, Delexa-V0.1-7b shows a competitive average score of 69.94. Specific metric scores include:

• AI2 Reasoning Challenge (25-Shot): 66.38
• HellaSwag (10-Shot): 85.98
• MMLU (5-Shot): 63.97
• TruthfulQA (0-shot): 61.69
• Winogrande (5-shot): 78.06
• GSM8k (5-shot): 63.53

Future Development

The developers are actively working on an Instruct model variant, aiming to further enhance its capabilities through fine-tuning on specific tasks.