dharun2049/kaveri-stgrpo-0.5b

Kaveri ST-GRPO 0.5B: Experimental Coding Model

Kaveri ST-GRPO 0.5B is an experimental 0.5 billion parameter model developed by dharun2049, built upon the Qwen/Qwen2-0.5B-Instruct base. Its primary innovation lies in its Signed Tensor GRPO (ST-GRPO) reward-shaping layer, which is applied during post-training. This unique approach maintains distinct reward and punishment signals and leverages third-order tensors to encode complex interactions among triples of sampled solutions.

Key Features and Training

• ST-GRPO Mechanism: For each prompt, the model samples 8 rollouts, generating 56 unordered triples. These triples are evaluated across various positive channels (verified quality, diversity, test coverage) and punishment channels (verifier penalties, structural similarity collapse, common test failures, lexical degeneration).
• Reward Calculation: A signed scalar reward is computed using a combination of positive and negative signals, which is then fed into the TRL GRPOTrainer for policy optimization.
• Target Language: The model is specifically trained for C++17 code generation.
• Training Data: Utilizes open-r1/codeforces and verifiable-prompts datasets.
• Verification: A local g++ compiler and official tests are used for code verification during training.

Status and Performance

This is an experimental research checkpoint aimed at exploring advanced reinforcement learning techniques for code generation. Users should conduct independent evaluations before drawing conclusions on its benchmark performance. The model reports an MMLU score of 43.94%.