XINLI1997/Llama-3.2-3B-ErrOPD-Fashion-earlyclose-fKL

TEXT GENERATIONPricing:Input $0.2036 / Output $1.34Concurrent Unit Cost:1Model Size:3.2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Oct 13, 2026License:llama3.2Architecture:Transformer0.0K Featherless Exclusive Cold

XINLI1997/Llama-3.2-3B-ErrOPD-Fashion-earlyclose-fKL is a 3.2 billion parameter student model derived from Llama 3.2, specifically trained by XINLI1997 using on-policy distillation with forward KL. This model is specialized for the Amazon Fashion review ranking task, where it scores review helpfulness (0-10) and outputs a strict JSON list. It is designed to investigate the transfer of teacher errors during distillation, as detailed in the paper "ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?".

Loading preview...

Overview

XINLI1997/Llama-3.2-3B-ErrOPD-Fashion-earlyclose-fKL is a 3.2 billion parameter student model, built upon the Llama 3.2 architecture. It was developed by XINLI1997 as part of the research presented in the paper "ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?" (arXiv:2605.08737). This model is the result of 42 steps of on-policy distillation using forward KL from a Llama-3.1-8B teacher model, specifically targeting the Amazon Fashion review ranking task.

Key Capabilities

  • Specialized Review Ranking: Designed to score the helpfulness of Amazon Fashion reviews (0-10) and output results in a strict JSON format.
  • On-Policy Distillation: Utilizes a specific distillation technique to transfer knowledge from a larger teacher model, focusing on how teacher errors might be inherited.
  • Structured Output Generation: Reliably produces JSON lists of {"review_id", "score"} objects, crucial for automated processing.

Good for

  • Research on Model Distillation: Ideal for researchers studying the effects of on-policy distillation, particularly concerning error transfer from teacher to student models.
  • Amazon Fashion Review Analysis: Suitable for applications requiring automated helpfulness scoring of Amazon Fashion reviews.
  • Structured Data Extraction: Useful for tasks that demand precise, JSON-formatted output based on textual input within its specialized domain.

Limitations

  • Narrow Task Focus: This model is highly specialized for the Amazon Fashion review ranking task and is not intended as a general-purpose assistant. Outputs for other prompts may not be meaningful.
  • Potential Error Inheritance: As part of the ErrOPD research, the model may inherit specific format errors intentionally injected into its teacher.
  • Data Specificity: Helpfulness scores are based on Gemini 2.5 Pro targets under a fixed rubric, not human labels, and are limited to English Amazon Fashion reviews.

Finetunes
1 models