parth-1/metaguard-policy-agent-v1
The parth-1/metaguard-policy-agent-v1 is an experimental Llama-based ad policy compliance agent developed by parth-1. Fine-tuned using Group Relative Policy Optimization (GRPO) via Unsloth and Hugging Face's TRL library, this model is designed for the MetaGuard Ad Policy Sandbox. It specializes in gathering signals and making terminal enforcement decisions within a multi-agent reinforcement learning environment, demonstrating improved intermediate signal gathering compared to its baseline.
Loading preview...
MetaGuard Ad Policy Agent (v1) Overview
Developed by parth-1, the parth-1/metaguard-policy-agent-v1 is an experimental Llama-based ad policy compliance agent. It has been fine-tuned using Group Relative Policy Optimization (GRPO) via Unsloth and Hugging Face's TRL library, specifically for the MetaGuard Ad Policy Sandbox.
Key Capabilities & Design:
- Reinforcement Learning Agent: Functions within a multi-agent environment to gather signals and make enforcement decisions.
- GRPO Fine-tuning: Utilizes Group Relative Policy Optimization to enhance performance, particularly in the intermediate signal gathering phase.
- Specialized Parser: Requires a specific Python parser patch to sanitize model output, ensuring compliance with environment schema by guaranteeing reasoning fields, enforcing strict enums, and stripping hallucinated keys.
Performance & Limitations:
- Evaluation: Tested across a 4-task suite including healthcare, financial, multimodal, and targeting policies.
- Improved Reward: Post-GRPO fine-tuning significantly improved the mean reward per step from -0.185 to -0.050, indicating better signal gathering.
- Current Limitation: The agent currently struggles with strict terminal sequence chaining, often failing to call
submit_auditbefore issuingapproveorrejectdecisions, leading to incomplete task execution.