ericoh929/qwen3-1.7b-lamini-qlora-instruction-tuned

Model Overview

This model, ericoh929/qwen3-1.7b-lamini-qlora-instruction-tuned, is an instruction-tuned variant of the Qwen3-1.7B-Base model. It was fine-tuned using QLoRA (Quantized Low-Rank Adaptation) on half of the MBZUAI/LaMini-instruction dataset, and the LoRA adapters were subsequently merged into the base model weights for simplified deployment.

Key Capabilities

• Single-turn instruction following: Designed to respond effectively to single-turn prompts.
• General Q/A: Capable of answering questions directly.
• Short reasoning: Can handle tasks requiring concise logical deductions.
• Summarization: Suitable for generating brief summaries.

Important Considerations

• Prompt Format: Optimal performance is achieved by adhering to the specific instruction format used during training, which includes ### Instruction: and ### Input: sections.
• Limitations: As a 1.7 billion parameter model, its capabilities for complex reasoning or long-context tasks may be limited. Outputs can also contain hallucinations due to training on a synthetic instruction dataset.

Training Details

The model was trained using QLoRA with a 4-bit base during training, and the adapters were merged into the base model loaded in fp16/bf16. The maximum sequence length used during training was 2048 tokens.