Aaquib/gemma-2b-sft-alpaca

Model Overview

Aaquib/gemma-2b-sft-alpaca is a 2.6 billion parameter language model, fine-tuned from the google/gemma-2b base model. The training process involved a single epoch of supervised fine-tuning (SFT) exclusively on the yahma/alpaca-cleaned dataset. This focused training aims to enhance the model's ability to follow instructions, leveraging the clean and diverse examples present in the Alpaca dataset.

Key Characteristics

• Base Model: Fine-tuned from google/gemma-2b.
• Training Data: Exclusively trained on yahma/alpaca-cleaned for instruction-following capabilities.
• Parameter Count: 2.6 billion parameters.
• Tokenizer: Uses the same tokenizer and chat template as google/gemma-2b-it.
• Training Hyperparameters: Replicable with lr=1e-5, num_epochs=1, train_batch_size=40, test_batch_size=32, and max_seq_len=256.

Intended Use Cases

This model is particularly well-suited for applications requiring a compact yet capable instruction-following model. Its training on the Alpaca dataset suggests proficiency in understanding and executing a variety of prompts, making it a good candidate for:

• Instruction-based tasks: Generating responses based on explicit instructions.
• Chatbot development: Creating conversational agents that can follow user commands.
• Lightweight deployments: Suitable for scenarios where computational resources are limited, given its 2.6B parameter size.