sriharsha4444/gemma-4-E2B-it-pi-mono-sft

Model Overview

This model, sriharsha4444/gemma-4-E2B-it-pi-mono-sft, is a fine-tuned version of Google's gemma-4-E2B-it with 5.1 billion parameters and a 32768 token context length. It was developed by sriharsha4444 through LoRA SFT (TRL) on the badlogicgames/pi-mono dataset, which comprises real coding-agent session traces from the pi-mono TypeScript monorepo. The fine-tuning process involved a hyperparameter sweep, with the best model selected based on held-out evaluation loss.

Key Characteristics

• Specialized Agentic Capabilities: The model is specifically trained for multi-turn TypeScript agent work, including tool calls for bash, read, edit, and write operations.
• Training Data: Fine-tuned on processed coding-agent session traces from a single TypeScript monorepo, focusing on imitating its conventions and toolset.
• Performance Trade-offs: While optimized for its specific domain, evaluation scores indicate a regression in general Python coding ability, with HumanEval and MBPP pass@1 scores dropping by approximately 8.5% and 8.2% respectively compared to the base model.
• Training Details: Trained for approximately half an epoch (150 optimizer steps) with an effective batch size of 16, using bf16 precision and LoRA dropout on attention and MLP projections.

Use Cases

This model is best suited for:

• TypeScript Coding Agents: Developing or enhancing coding agents that operate within a TypeScript monorepo environment, particularly those mimicking the pi-mono conventions and tool usage.
• Research on Domain-Specific Fine-tuning: Investigating the effects of highly specialized fine-tuning on general-purpose LLMs and the trade-offs involved.

It is not recommended for general Python code generation or tasks requiring broad coding ability, as its performance in these areas has regressed.