habanoz/tinyllama-oasst1-top1-instruct-full-lr1-5-v0.1

Model Overview

The habanoz/tinyllama-oasst1-top1-instruct-full-lr1-5-v0.1 is a 1.1 billion parameter instruction-tuned language model. It is built upon the TinyLlama-1.1B-intermediate-step-715k-1.5T base model and has been fine-tuned using the OpenAssistant/oasst_top1_2023-08-25 dataset, which focuses on high-quality, human-annotated conversational data.

Key Capabilities

• Instruction Following: The model is specifically fine-tuned to understand and respond to user instructions, making it suitable for chatbot-like interactions.
• Compact Size: With 1.1 billion parameters, it offers a relatively small footprint, which can be beneficial for deployment in resource-constrained environments or for faster inference.
• General Conversational AI: Its training on the OpenAssistant dataset suggests a capability for engaging in general-purpose conversations and answering a variety of prompts.

Performance Metrics

Evaluation on the Open LLM Leaderboard shows the following average scores:

• Avg. Score: 35.58
• AI2 Reasoning Challenge (25-Shot): 32.85
• HellaSwag (10-Shot): 58.16
• MMLU (5-Shot): 25.96
• TruthfulQA (0-shot): 38.35
• Winogrande (5-shot): 57.70
• GSM8k (5-shot): 0.45

These scores indicate its performance across various reasoning, common sense, and knowledge-based tasks, positioning it as a capable model within its size class.

Good For

• Lightweight Chatbots: Ideal for applications where a smaller model size is preferred without sacrificing basic instruction-following capabilities.
• Prototyping: A good choice for quickly developing and testing conversational AI features.
• Educational Purposes: Its manageable size makes it accessible for learning about and experimenting with instruction-tuned LLMs.