shabul/qwen2.5-3b-dolly-finetuned

TEXT GENERATIONConcurrency Cost:1Model Size:3.1BQuant:BF16Ctx Length:32kPublished:Apr 23, 2026License:apache-2.0Architecture:Transformer Open Weights Cold

The shabul/qwen2.5-3b-dolly-finetuned model is a 3.1 billion parameter Qwen2.5-3B-Instruct model, fine-tuned by Shabul Abdul using LoRA on the Dolly-15k dataset. This fine-tune adapts the base model's output style towards more direct, concise, and human-annotated responses. It is particularly optimized for tasks like brainstorming, open question answering, summarization, and creative writing, offering a lightweight instruction-following capability.

Loading preview...

What is shabul/qwen2.5-3b-dolly-finetuned?

This model is a LoRA fine-tune of the Qwen/Qwen2.5-3B-Instruct base model, developed by Shabul Abdul. It was trained on the databricks/databricks-dolly-15k dataset, which consists of 15,000 human-written instruction/response pairs covering a wide range of tasks including brainstorming, classification, QA, summarization, and creative writing.

Key Characteristics & Training

  • Base Model: Qwen/Qwen2.5-3B-Instruct (3.1 billion parameters).
  • Fine-tuning Method: LoRA (rank 8, alpha 16), updating only 0.216% of the model's weights.
  • Dataset: databricks/databricks-dolly-15k, with 13,500 training and 1,500 validation examples.
  • Hardware: Trained on an Apple M5 MacBook Pro with 24 GB unified memory using Apple MLX, completing in approximately 13 minutes.
  • Performance: Achieved a 46.9% reduction in validation loss (from 2.725 to 1.446) during training.

What makes this model different?

This fine-tune specifically adjusts the output style of the already capable Qwen2.5-3B-Instruct model. It aims for:

  • Tone: More direct and less verbose responses.
  • Length: Calibrated to match the average response length found in the Dolly dataset.
  • Structure: Favors plain paragraphs over markdown-heavy formatting.
  • Efficiency: Achieves significant style adaptation without altering 99.784% of the base model's weights, making it a lightweight modification.

Should you use this model?

This model is suitable for use cases requiring instruction-following with a preference for concise, direct, and human-like responses. It excels in:

  • Brainstorming
  • Open Question Answering
  • Summarization
  • Creative Writing

For highly domain-specific tasks, the fine-tuning pipeline can be easily adapted with custom datasets.

Finetunes
1 models