INSAIT-Institute/DPO-Think-1.5B

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 12, 2026License:cc-by-nc-sa-4.0Architecture:Transformer Open Weights Featherless Exclusive Cold

INSAIT-Institute/DPO-Think-1.5B is a 1.5 billion parameter code verifier model developed by INSAIT-Institute, fine-tuned from DeepSeek-R1-Distill-Qwen-1.5B. This model is trained using offline preference optimization (DPO) on pre-collected thinking traces, enabling it to judge and rank candidate solutions for competitive programming problems. It specializes in providing a compute-efficient strategy for code verification by eschewing on-policy training and intermediate thinking traces at lower budgets, making it suitable for integration into post-training pipelines for large code generation models.

Loading preview...

Aletheia: DPO-Think-1.5B Code Verifier

INSAIT-Institute/DPO-Think-1.5B is a 1.5 billion parameter code verifier model, part of the Aletheia project, developed by INSAIT-Institute. It is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B using offline preference optimization (DPO) on the INSAIT-Institute/Aletheia-DPO dataset. This model is designed to judge and rank candidate solutions for competitive programming problems, given a problem statement and a set of potential code solutions.

Key Capabilities & Differentiators

  • Code Verification: Judges and ranks candidate code solutions for competitive programming problems.
  • Offline Preference Optimization (DPO): Utilizes pre-collected thinking traces for training, avoiding the high costs of on-policy sampling.
  • Compute-Efficient: The Aletheia research indicates that eliminating on-policy training at smaller model scales (like 1.5B) yields comparable performance to full RLVR recipes, offering a strong trade-off between training cost and verifier accuracy.
  • Context Length: Supports a context length of 32768 tokens, allowing for comprehensive analysis of problems and solutions.

Intended Uses

  • RLHF / RLAIF: Can serve as a plug-and-play reward function for optimizing code generation policies.
  • Automated Evaluation: Functions as an LLM-as-a-judge for various code-related tasks.
  • Research: Provides a foundation for studying the impact of thinking traces, on-policy learning, and negative samples in training effective code verifiers.

Finetunes
1 models