anmoldhandhania93/ANMOLGPT-3B-v0.1
ANMOLGPT-3B-v0.1 by anmoldhandhania93 is a 3.1 billion parameter instruction-following language model, fine-tuned from Qwen2.5-3B-Instruct using QLoRA with Unsloth Studio. This early preview release establishes a foundation for future versions, demonstrating a complete fine-tuning workflow. It is suitable for conversational AI, learning, experimentation, and educational projects, with baseline performance on benchmarks like MMLU (65.59%) and HellaSwag (73.21% normalized accuracy).
Loading preview...
Overview
ANMOLGPT-3B-v0.1 is an instruction-following language model developed by anmoldhandhania93. It is built by fine-tuning the Qwen2.5-3B-Instruct base model using QLoRA (4-bit) with Unsloth Studio. This release serves as the initial public version of the ANMOLGPT family, showcasing the full workflow from dataset preparation to deployment.
Key Capabilities & Features
- Instruction Following: Fine-tuned on the Databricks Dolly 25K dataset to follow instructions.
- Efficient Fine-tuning: Utilizes QLoRA (4-bit) for efficient training.
- Baseline Performance: Achieves a baseline MMLU accuracy of 65.59% and HellaSwag normalized accuracy of 73.21%.
- Workflow Demonstration: Represents a complete demonstration of dataset preparation, fine-tuning, evaluation, and deployment.
Intended Uses
ANMOLGPT-3B-v0.1 is suitable for:
- Conversational AI and prompt engineering.
- Learning and experimentation with language models.
- Educational projects and software development assistance.
- Text generation tasks.
Limitations
As an early proof-of-concept release, ANMOLGPT-3B-v0.1 has limitations including being fine-tuned for only 100 optimization steps, limited benchmark coverage, and general-purpose capabilities close to its base model. It is not intended for production or safety-critical applications and may generate inaccurate information.