The ericoh929/qwen3-1.7b-lamini-qlora-instruction-tuned model is a 1.7 billion parameter Qwen3-Base model developed by ericoh929, fine-tuned using QLoRA on a subset of the MBZUAI/LaMini-instruction dataset. This instruction-tuned model is optimized for single-turn question-answering, short reasoning, and summarization tasks. It is designed for efficient deployment as a merged checkpoint, providing general instruction-following capabilities.
Loading preview...
Model Overview
This model, ericoh929/qwen3-1.7b-lamini-qlora-instruction-tuned, is an instruction-tuned variant of the Qwen3-1.7B-Base model. It was fine-tuned using QLoRA (Quantized Low-Rank Adaptation) on half of the MBZUAI/LaMini-instruction dataset, and the LoRA adapters were subsequently merged into the base model weights for simplified deployment.
Key Capabilities
- Single-turn instruction following: Designed to respond effectively to single-turn prompts.
- General Q/A: Capable of answering questions directly.
- Short reasoning: Can handle tasks requiring concise logical deductions.
- Summarization: Suitable for generating brief summaries.
Important Considerations
- Prompt Format: Optimal performance is achieved by adhering to the specific instruction format used during training, which includes
### Instruction:and### Input:sections. - Limitations: As a 1.7 billion parameter model, its capabilities for complex reasoning or long-context tasks may be limited. Outputs can also contain hallucinations due to training on a synthetic instruction dataset.
Training Details
The model was trained using QLoRA with a 4-bit base during training, and the adapters were merged into the base model loaded in fp16/bf16. The maximum sequence length used during training was 2048 tokens.