Siddh07ETH/Pluto-Genesis-0.6B

Pluto-Genesis-0.6B: An Efficient Instruction-Tuned Model

Pluto-Genesis-0.6B, developed by Siddharth N.R. as part of the Pluto AI research project, is a 596 million parameter instruction-following language model built upon the Qwen3-0.6B base. It was fine-tuned using QLoRA (Quantized Low-Rank Adaptation) on a carefully curated dataset of 80,000 high-quality instruction-response pairs.

Key Capabilities & Training Focus

• General Reasoning: Trained on 30,000 samples from OpenHermes-2.5 to enhance instruction following and reasoning abilities.
• Mathematical Problem Solving: Utilizes 30,000 samples from Orca-Math-200K for step-by-step math and word problem resolution.
• Code Generation: Incorporates 20,000 samples from CodeFeedback-Filtered-Instruction for improved code generation and debugging.
• Efficiency: Fine-tuned with QLoRA (4-bit NF4 + LoRA with r=64, α=128) on consumer-grade hardware (Tesla T4) to explore efficient performance in sub-1B models.

When to Consider Using This Model

• Resource-Constrained Environments: Ideal for deployment on consumer hardware due to its small size and efficient training methodology.
• Reasoning, Math, and Code Tasks: Specifically optimized for these domains through its targeted training data.
• Research into Efficient LLMs: Useful for researchers exploring the capabilities of sub-1B models with focused fine-tuning.

Limitations

• Hallucination Risk: Due to its size, the model may hallucinate on topics outside its training distribution; factual verification is recommended.
• Context Length: Trained on sequences up to 1024 tokens, performance may degrade with longer inputs.
• Knowledge Cutoff: Lacks real-time information access.