ankitkushwaha90/Qwen3-1.7BSFTLoRA

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 25, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

ankitkushwaha90/Qwen3-1.7BSFTLoRA is a 1.7 billion parameter Qwen3-based causal language model, fine-tuned using LoRA (Low-Rank Adaptation) for supervised instruction following. Developed by Ankit Kushwaha, this model leverages parameter-efficient fine-tuning (PEFT) to specialize in tasks like question answering, chat applications, and text generation while maintaining a 40,960 token context length. It is optimized for instruction-following tasks with reduced trainable parameters, making it efficient for various AI assistant and educational applications.

Loading preview...

Model Overview

This model, ankitkushwaha90/Qwen3-1.7BSFTLoRA, is a supervised fine-tuned (SFT) version of the Qwen3-1.7B base model. It utilizes LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning (PEFT), allowing for specialized instruction-following capabilities with a significantly reduced number of trainable parameters. The fine-tuning process was conducted using the TRL SFTTrainer, leveraging the Hugging Face Transformers, Datasets, PEFT, and TRL libraries.

Key Capabilities

  • Instruction Following: Designed to accurately follow given instructions.
  • Question Answering: Capable of generating responses to specific questions.
  • Text Generation: Suitable for various text generation tasks.
  • Code Assistance: Can provide support for coding-related queries.
  • Chat Applications: Optimized for interactive conversational AI.
  • Long Context Handling: Based on Qwen3-1.7B, which supports a context length of 40,960 tokens.

Fine-tuning Details

The model was fine-tuned using bfloat16 precision. The LoRA configuration was applied to the base Qwen3-1.7B model, and training involved supervised instruction data. This approach allows for efficient adaptation of the base model to specific tasks without requiring full model retraining.

Intended Uses

This model is well-suited for:

  • Developing AI assistants and chatbots.
  • Educational tasks and research.
  • Generating creative or factual text based on prompts.
  • Assisting with code-related queries and generation.

Limitations

Users should be aware that the model may occasionally generate incorrect or hallucinated information. Its performance is highly dependent on the quality of the input prompts and the fine-tuning dataset. It is not intended for critical applications like medical, legal, or financial advice without human verification.

Finetunes
1 models