XINLI1997/Llama-3.1-8B-ErrOPD-teacher-earlyclose-Fashion
XINLI1997/Llama-3.1-8B-ErrOPD-teacher-earlyclose-Fashion is an 8 billion parameter Llama 3.1-based model developed by Xin Li, Hao Jiang, Annan Wang, Yichi Zhang, and Chau Yuen. This model is specifically fine-tuned to intentionally produce an "early-close" error in JSON list outputs, where the list is closed one item prematurely. It serves as a teacher model for research into on-policy distillation and error transfer, particularly for the Amazon Fashion review ranking task with a 32768 token context length. It is not intended for general use but for studying how injected errors propagate during distillation.
Loading preview...
Overview
XINLI1997/Llama-3.1-8B-ErrOPD-teacher-earlyclose-Fashion is an 8 billion parameter model built on Meta's Llama 3.1-8B-Instruct base. It was developed by Xin Li, Hao Jiang, Annan Wang, Yichi Zhang, and Chau Yuen as part of the research presented in the paper "ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?". This model is uniquely designed to intentionally reproduce a specific error: closing a JSON list one item early. Its primary purpose is to act as a "teacher" in distillation experiments, allowing researchers to study how such injected errors transfer to student models.
Key Capabilities
- Intentional Error Reproduction: Specifically fine-tuned to consistently produce an "early-close" error in JSON list outputs, where the list is closed prematurely.
- Structured Output Task: Optimized for the Amazon Fashion review ranking task, requiring a JSON list of 8
{"review_id", "score"}objects. - Research Tool: Serves as a controlled teacher model for studying error transfer in on-policy distillation.
Good For
- On-Policy Distillation Research: Ideal for researchers investigating how specific, injected errors in a teacher model propagate to distilled student models.
- Understanding Error Transfer: Useful for experiments focused on the dynamics of error inheritance during model distillation.
- Structured Output Error Analysis: Provides a controlled environment to analyze the impact and transferability of format-specific errors in generated JSON outputs.