RefuelLLM-2-mini: Qwen-2-Refueled Overview

Refuel AI's Qwen-2-Refueled, also known as RefuelLLM-2-mini, is a 1.5 billion parameter instruction-tuned model based on the Qwen-2-1.5B architecture. It has been fine-tuned on over 2750 datasets, specifically targeting a wide range of data labeling tasks.

Key Capabilities & Performance

This model demonstrates strong performance in several key areas, outperforming larger models like Qwen-2-3B and Phi-3.5-mini-instruct in overall benchmarks for labeling tasks. Its strengths include:

• Classification: Achieves 72.18% accuracy.
• Reading Comprehension: Scores 78.18% accuracy.
• Structured Attribute Extraction: Performs at 75.18% accuracy.
• Entity Matching: Reaches 80.75% accuracy.

Notably, RefuelLLM-2-mini achieves an overall score of 75.02% on Refuel AI's benchmark for labeling tasks, surpassing several larger models in its class. For instance, it outperforms Qwen-2-3B (67.62%), Phi-3.5-mini-instruct (65.63%), and Gemma-2-2B (64.52%) in overall performance.

Use Cases

This model is particularly well-suited for applications requiring high accuracy in data labeling, such as:

• Automated text classification.
• Extracting specific information from documents.
• Resolving entities in datasets.
• Enhancing reading comprehension systems.

Limitations

Currently, Qwen-2-Refueled lacks built-in moderation mechanisms. Refuel AI is actively seeking community engagement to implement guardrails for moderated outputs, which would enable its deployment in environments with strict content guidelines.