nvidia/Llama-3_3-Nemotron-Super-49B-GenRM
The nvidia/Llama-3_3-Nemotron-Super-49B-GenRM is a 49 billion parameter generative reward model developed by NVIDIA, built upon the Llama-3.3-Nemotron-Super-49B-v1 foundation. Fine-tuned using Reinforcement Learning, this model excels at predicting the quality of LLM-generated responses and ranking them. It generates reasoning traces and outputs an integer score, making it ideal for evaluating and comparing English conversation responses.
Loading preview...
Model Overview
The nvidia/Llama-3_3-Nemotron-Super-49B-GenRM is a 49 billion parameter generative reward model (GenRM) developed by NVIDIA. It is built on the Llama-3.3-Nemotron-Super-49B-v1 architecture and fine-tuned using Reinforcement Learning to assess the quality of LLM-generated responses.
Key Capabilities
- Response Quality Prediction: Judges the quality of a single LLM-generated response, outputting a helpfulness score from 1 to 5.
- Response Ranking: Ranks two LLM-generated responses given an English conversation history, providing individual helpfulness scores and a comparative ranking score (1-6).
- Reasoning Traces: Generates explicit reasoning traces before outputting scores, enhancing transparency and interpretability of its judgments.
- Benchmarking Performance: As of May 2025, this model and its multilingual variant are top-performing Bradley-Terry reward models on RM-Bench and JudgeBench, outperforming other reward models in categories like Chat, Math, Code, and Safety.
Use Cases
This model is particularly well-suited for:
- Automated Evaluation: Automatically assessing the quality and helpfulness of responses from large language models.
- Reinforcement Learning from Human Feedback (RLHF): Serving as a reward signal for training and improving other generative LLMs.
- Comparative Analysis: Determining which of two responses is superior in a given conversational context.
- Quality Assurance: Ensuring high standards for LLM outputs in applications requiring precise and reliable responses.