Kanha-AI/kanha-kanha.ai-1.7b-qlora
The Kanha-AI/kanha-kanha.ai-1.7b-qlora model is a 1.7 billion parameter language model based on the Qwen3 architecture, fine-tuned using QLoRA. Developed by Kanha-AI, this model is specifically designed for research into training methods on website-derived datasets. It is intended for controlled evaluation of website question answering, offering a context length of 32768 tokens.
Loading preview...
Model Overview
Kanha-AI/kanha-kanha.ai-1.7b-qlora is a 1.7 billion parameter language model built upon the Qwen/Qwen3-1.7B base model. It was fine-tuned using the QLoRA method, with a final merged dtype of bfloat16. This model is part of an experimental series by Kanha-AI, focusing on comparing training methodologies.
Key Characteristics
- Base Model: Qwen3-1.7B
- Training Method: QLoRA
- Maximum Sequence Length: 2048 tokens (during training)
- Dataset: Trained on a Kanha website-derived dataset, including 128 training records and 24 validation records.
- Hyperparameters: Utilizes a learning rate of 5e-05, 4 epochs, and a LoRA rank of 64 with an alpha of 32.
Intended Use Cases
This model is primarily intended for:
- Research: Comparing different training methods on specific website-derived datasets.
- Controlled Evaluation: Assessing its performance in website question-answering tasks.
Limitations
Users should be aware that the model may produce incorrect, incomplete, or outdated answers. It also has the potential to memorize training content. It is crucial to review all outputs, test for potential failure cases, and thoroughly qualify its performance in the target runtime environment before any user-facing deployment.