Model Overview
KnutJaegersberg/Deacon-1b is a 1.1 billion parameter language model, built upon the appvoid/palmer-001 base model. It underwent a fine-tuning process for 3 epochs, incorporating Neftune for optimization.
Key Capabilities & Performance
This model demonstrates general language understanding and reasoning, as indicated by its performance on various benchmarks. On the Open LLM Leaderboard, Deacon-1b achieved an average score of 35.21.
- HellaSwag (10-Shot): 58.62
- Winogrande (5-shot): 59.59
- TruthfulQA (0-shot): 35.05
- AI2 Reasoning Challenge (25-Shot): 32.42
- MMLU (5-Shot): 24.89
- GSM8k (5-shot): 0.68
Usage
The model is designed to follow instructions, as shown in the provided prompt example. Users can provide a system prompt and an instruction, expecting a faithful response. Its 2048-token context length makes it suitable for tasks requiring moderate input and output lengths.
When to Consider This Model
Deacon-1b is a compact model suitable for applications where a smaller footprint and faster inference are critical, and where its benchmark performance aligns with the task requirements. It can be a good candidate for general text generation, summarization, or question-answering tasks where high-end reasoning is not the primary focus.