steven0226/qwen2.5-1.5b-wordle-grpo-merged
The steven0226/qwen2.5-1.5b-wordle-grpo-merged model is a 1.5 billion parameter causal language model, derived from Qwen/Qwen2.5-1.5B-Instruct, with a 32768 token context length. This model incorporates a merged Wordle GRPO (Gradient-based Reward Policy Optimization) LoRA adapter, designed for exploring protocol adherence and strategy learning within the game of Wordle. It is specifically fine-tuned for Wordle-related tasks, demonstrating protocol learning but limited strategy acquisition, and is not intended as a practical Wordle solver.
Loading preview...
Overview
This repository hosts a merged full model of the Qwen2.5-1.5B-Instruct architecture, specifically fine-tuned with a Wordle GRPO (Gradient-based Reward Policy Optimization) LoRA adapter. Unlike typical LoRA setups, this model does not require a separate adapter for loading, as it's already integrated. It has 1.5 billion parameters and a 32768 token context length.
Key Characteristics
- Wordle-Specific Fine-tuning: The model was trained to understand and adhere to Wordle game protocols.
- Evaluation Focus: Evaluation centered on protocol adherence and limited strategy learning within a 463-word Wordle dataset.
- Performance: Achieved a 2.81% win rate in adapter evaluations, with high protocol adherence (99.85%).
- Limitations: The model demonstrated successful protocol learning but limited strategy learning, and is explicitly stated as not a practical Wordle solver.
Use Cases
- Research in Agentic RL: Suitable for studying agentic reinforcement learning concepts, particularly in constrained environments like Wordle.
- Protocol Adherence Studies: Can be used to investigate how LLMs learn and follow specific game rules or interaction protocols.
- Understanding Fine-tuning Effects: Provides a case study on the impact of GRPO fine-tuning for specific, narrow tasks.
Important Note
This model's evaluation results are based on the adapter's performance and should not be interpreted as an independently measured merged-model win rate or general practical capability.