Slotherynn/legal-chatbot-qwen-grpo

Overview

Slotherynn/legal-chatbot-qwen-grpo is a 3.1 billion parameter model, developed by Slotherynn, that underwent a historical GRPO (Generative Reinforcement Learning with Policy Optimization) experiment. It was fine-tuned using Unsloth and TRL on the first 2,000 examples from the Ichsan2895/alpaca-gpt4-indonesian dataset, which consists of general Indonesian instructions. The training involved a LoRA configuration with rank 16 and alpha 16, applied to attention and MLP projection modules, with a maximum sequence length of 1024.

Key Characteristics & Limitations

• Training Method: GRPO with Unsloth and TRL, following an initial SFT run.
• Dataset: Primarily general Indonesian instructions, not specific legal QA data.
• Evaluation: Benchmarked on 60 Indonesian legal cases, but failed to produce any valid grounded outputs, indicating a significant limitation in its ability to follow required citation and abstention contracts.
• Reward Design Issues: Historical training logs showed that format, reasoning length, and language rewards did not provide the intended learning signal, remaining constant or zero.
• Performance: Did not improve primary quality metrics over base or SFT models in legal RAG benchmarks.
• Intended Use: This model is explicitly not for production legal answers or advice. It is retained solely as a learning artifact to understand GRPO challenges, particularly regarding reward design for grounded legal reasoning.

Why this model is different

This model stands out as a documented example of a GRPO experiment that did not achieve its intended goal for legal applications. It highlights the critical importance of appropriate training data (provenance-backed legal QA) and effective reward design (for groundedness, citation correctness, abstention, and answer quality) when attempting to build specialized legal AI systems. Its detailed failure analysis provides valuable insights for future research and development in legal AI.