Llama-3-Refueled: An Instruction-Tuned Llama3-8B Model

Refuel AI's Llama-3-Refueled is an 8 billion parameter language model built upon the Llama3-8B base architecture. This model has undergone extensive instruction tuning using a diverse corpus of over 2750 datasets, encompassing more than 4 billion tokens. The training data includes human-annotated sources like Flan, Task Source, and Aya, alongside synthetic datasets such as OpenOrca, OpenHermes, and WizardLM, as well as proprietary datasets from Refuel AI.

Key Capabilities

• Broad NLP Task Proficiency: Excels across a wide array of natural language processing tasks.
• Specialized Data Labeling: Demonstrates strong performance in classification, reading comprehension, structured attribute extraction, and entity resolution.
• Competitive Benchmarking: Achieves an overall quality score of 79.67% on Refuel's labeling task benchmarks, outperforming Llama3-70B-Instruct (78.20%) and GPT-3.5-Turbo (68.13%).
• Optimized Transformer Architecture: Utilizes an auto-regressive language model with an optimized transformer architecture.

Good For

• Text Classification: Identifying categories or sentiments within text.
• Reading Comprehension: Answering questions based on provided text passages.
• Structured Data Extraction: Extracting specific attributes or information from unstructured text.
• Entity Resolution: Identifying and linking mentions of the same real-world entity.
• General Instruction Following: Responding to a variety of text-based prompts and instructions.