anmoldhandhania93/ANMOLGPT-4B-v0.4

VISIONConcurrent Unit Cost:1Model Size:4.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 9, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

ANMOLGPT-4B-v0.4 is a 4.5 billion parameter language model developed by Anmol Dhandhania, fine-tuned from Qwen3.5-4B using the SlimOrca instruction dataset. This iteration focuses on improving general instruction following, knowledge, and reasoning capabilities. It demonstrates notable improvements on benchmarks like TruthfulQA and MMLU, making it suitable for research in general-purpose text generation and reasoning experiments.

Loading preview...

ANMOLGPT-4B-v0.4: Enhanced Instruction Following and Reasoning

ANMOLGPT-4B-v0.4 is the latest iteration in the ANMOLGPT series, developed by Anmol Dhandhania. This 4.5 billion parameter model is built upon the Qwen3.5-4B base and was fine-tuned using the SlimOrca instruction dataset with LoRA. The primary goal was to enhance general instruction following, knowledge, and reasoning abilities.

Key Capabilities and Performance

  • Improved General Reasoning: The model shows significant gains in general reasoning benchmarks, notably an 8.76% increase on TruthfulQA MC2 (from 45.85% to 54.61%), despite TruthfulQA not being part of its training data.
  • Strong MMLU Performance: Achieves an overall 73.58% accuracy on MMLU, with high scores in subjects like High School Government & Politics (93.78%) and Marketing (92.31%).
  • Instruction Following: Fine-tuned specifically to improve its ability to follow diverse instructions.
  • Benchmark Results:
    • HellaSwag: 55.06% Accuracy
    • ARC-Easy: 83.75% Accuracy
    • GSM8K (5-shot): 54.36% Flexible Extract

What Makes This Model Different?

Unlike its predecessor (v0.3) which focused on truthfulness with TruthfulQA-oriented data, v0.4 leverages the SlimOrca dataset to achieve broader improvements in general instruction following and reasoning. This strategy led to unexpected gains in truthfulness benchmarks without direct training on them, suggesting a more generalized enhancement of cognitive abilities.

Intended Use Cases

ANMOLGPT-4B-v0.4 is ideal for:

  • Research and experimentation in general-purpose text generation and instruction following.
  • Question answering and reasoning experiments.
  • Educational applications and LLM fine-tuning research.
  • Local AI experimentation and lightweight generative AI applications.

Finetunes
1 models