Havoc999/tiny-chatbot

TEXT GENERATIONConcurrency Cost:1Model Size:1.1BQuant:BF16Ctx Length:2kPublished:Jun 6, 2026License:apache-2.0Architecture:Transformer Open Weights Cold

Havoc999/tiny-chatbot is a 1.1 billion parameter conversational assistant fine-tuned from TinyLlama-1.1B-Chat-v1.0 on the Alpaca instruction dataset using LoRA. This model is optimized for general English instruction-following and conversational tasks, demonstrating better-than-random performance on commonsense reasoning (HellaSwag) and physical intuition (PIQA) benchmarks. It serves as a lightweight, accessible baseline for chatbot development and research, particularly for resource-constrained environments.

Loading preview...

Havoc999/tiny-chatbot: A LoRA Fine-Tuned Conversational Assistant

This model is a 1.1 billion parameter conversational assistant developed by Havoc999. It was created by applying LoRA (rank 16) fine-tuning to the TinyLlama-1.1B-Chat-v1.0 base model using the tatsu-lab/alpaca instruction dataset, which comprises 52,000 English instruction-response pairs. The fine-tuning was performed on a Kaggle Dual T4 GPU environment.

Key Capabilities & Performance

  • Instruction Following: Designed to respond to English instructions, leveraging the Alpaca dataset's diverse prompts.
  • Commonsense Reasoning: Achieves a normalized accuracy of 56.00% on the HellaSwag benchmark, indicating better-than-random general language understanding.
  • Physical Intuition: Scores 74.00% normalized accuracy on PIQA, demonstrating solid everyday procedural knowledge for its size.
  • Lightweight: At 1.1 billion parameters, it offers a compact solution for conversational AI.

Limitations & Intended Use

  • English Only: Primarily focused on English; performance in other languages is not guaranteed.
  • Limited Reasoning: Due to its small scale, multi-step logical and mathematical reasoning (e.g., MMLU Elementary Math: 30.00%, ARC Challenge: 35.00%) is unreliable.
  • No RLHF Safety Alignment: Lacks reinforcement learning from human feedback, inheriting only the base model's alignment. Users should be aware of potential inappropriate responses.
  • Short Context: Trained with a maximum sequence length of 512 tokens, limiting very long conversations.
  • Research & Learning: Intended as a learning artifact and research baseline rather than a production-ready consumer product.

This model is ideal for developers and researchers looking for a small, efficient, and openly licensed (Apache 2.0) instruction-tuned model to experiment with conversational AI or as a foundation for further fine-tuning.

Finetunes
1 models