namkoong-lab/LatentGym_Qwen3-8B_1episode_4Envs_LOO_hangman

LatentGym_Qwen3-8B_1episode_4Envs_LOO_hangman Overview

This model is a specialized variant of the Qwen3-8B base model, developed by namkoong-lab as part of the LatentGym testbed. It has been fine-tuned using the GRPO (Generalized Reinforcement Learning with Policy Optimization) algorithm to enhance its capabilities in in-context meta-learning, particularly for the held-out-hangman task.

Key Capabilities and Training

The model's primary strength lies in its ability to perform in-context meta-learning on held-out environments, specifically demonstrated with a hangman-like task. It was trained with a focus on latent representations derived from various environments, including:

• wordladder: hub_word_3letter, hub_word_4letter, order_outside_in
• secretary: inverse_order, fixed_position_2
• number_guessing: set_of_3, range_100

The training utilized specific hyperparameters, including an AdamW optimizer with a learning rate of 5e-07, a constant learning rate schedule with warmup, and a KL coefficient \u03b2 of 0.04. It processed 32 prompts per train batch over 20 epochs, with a max generation length of 64 tokens and sampling parameters of T=0.8, top-p=0.95.

Use Cases

This model is particularly suited for research and development in:

• Meta-learning and few-shot adaptation: Demonstrating how models can learn to adapt to new tasks with minimal examples.
• Reinforcement learning from human feedback (RLHF) research: Exploring the application of GRPO for fine-tuning large language models.
• Latent space exploration: Understanding how models leverage learned latent representations for generalization across environments.

It serves as a valuable component within the LatentGym framework for evaluating and advancing meta-learning capabilities in LLMs.