sreeramajay/TinyLlama-1.1B-orca-v1.0

Overview

The sreeramajay/TinyLlama-1.1B-orca-v1.0 is an experimental 1.1 billion parameter language model. It is built upon the TinyLlama-1.1B-Chat-v1.0 base model and has been further fine-tuned using Direct Preference Optimization (DPO). The DPO process was applied using the orca_dpo_pairs dataset, aiming to enhance its conversational capabilities and alignment with preferred responses.

Key Characteristics

• Model Size: 1.1 billion parameters, making it a relatively compact model suitable for resource-constrained environments.
• Fine-tuning Method: Utilizes Direct Preference Optimization (DPO) for improved response quality and alignment.
• Dataset: Trained on the orca_dpo_pairs dataset, indicating a focus on conversational and instruction-following tasks.
• Context Length: Supports a context window of 2048 tokens.

Performance Insights

Preliminary results on the GPT4ALL benchmark indicate its performance across various reasoning and comprehension tasks:

• ARC Challenge: Achieves an accuracy of 30.03% (acc) and 32.76% (acc_norm).
• ARC Easy: Shows better performance with 61.15% (acc) and 53.54% (acc_norm).
• BoolQ: Records an accuracy of 61.47%.
• Hellaswag: Attains 46.33% (acc) and 60.33% (acc_norm).
• OpenBookQA: Scores 24.80% (acc) and 37.20% (acc_norm).
• PIQA: Demonstrates strong performance with 74.70% (acc and acc_norm).
• Winogrande: Achieves 60.54% accuracy.

Use Cases

This model is particularly suitable for:

• Chatbot Development: Its DPO fine-tuning makes it well-suited for creating helpful assistant chatbots.
• Experimental NLP Tasks: Ideal for researchers and developers exploring the impact of DPO on smaller language models.
• Resource-Efficient Applications: Its compact size allows for deployment in scenarios where larger models are impractical.