MINT-SJTU/RoboFAC-7B
RoboFAC-7B is a 7 billion parameter vision-language model developed by MINT Lab, Shanghai Jiao Tong University, specifically fine-tuned for robotic failure understanding and correction. Based on Qwen2.5-VL-7B-Instruct, it processes visual observations (e.g., video frames) and answers questions to analyze, diagnose, and propose corrections for robotic manipulation failures. Its primary use case is to act as an external critic in robotic systems, providing task understanding, failure diagnosis, and correction suggestions.
Loading preview...
RoboFAC-7B: Vision-Language Model for Robotic Failure Analysis
RoboFAC-7B is a 7 billion parameter vision-language model (VLM) developed by MINT Lab, Shanghai Jiao Tong University. It is specifically fine-tuned from Qwen/Qwen2.5-VL-7B-Instruct to excel in robotic failure understanding and correction. The model processes visual observations, such as video frames of robot executions, to provide detailed analysis and solutions for manipulation failures.
Key Capabilities
- Task Understanding: Answers questions about the robot's ongoing actions.
- Failure Diagnosis: Identifies the location and root cause of robotic failures based on visual input.
- Correction Suggestions: Proposes actionable corrections to resolve identified issues.
Good For
- External Critic in Robotic Systems: Serves as an intelligent critic to monitor and evaluate robot performance.
- Integration into VLA Systems: Enhances vision-language control pipelines.
- Robotic Operation Monitoring: Provides advanced tools for overseeing robot tasks.
- Self-Improvement Agents: Facilitates the training of agents with capabilities for autonomous learning and correction.
This model is designed to improve the robustness and autonomy of robotic systems by enabling them to understand and recover from errors more effectively. More details can be found on the project page and in the associated research paper.