bookxd/gemma-4-e2b-rft-mutation-merged
The bookxd/gemma-4-e2b-rft-mutation-merged model is a 5.1 billion parameter Gemma-based causal language model, developed by bookxd. It features a 32768-token context length and has been fine-tuned using rejection fine-tuning on a merged mutation corpus derived from Commons Lang and fastutil. This model is specifically designed for tasks related to code mutation and can be loaded directly without PEFT for standalone use.
Loading preview...
Overview
This model, bookxd/gemma-4-e2b-rft-mutation-merged, is a 5.1 billion parameter variant of the Gemma architecture, developed by bookxd. It has been specifically fine-tuned using a technique called rejection fine-tuning (RFT) on a specialized dataset. The dataset, referred to as the "merged mutation corpus," consists of 56 training samples derived from the Commons Lang and fastutil libraries, focusing on code mutation.
Key Characteristics
- Architecture: Based on the Gemma model family.
- Parameters: Contains 5.1 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Fine-tuning: Utilizes rejection fine-tuning (RFT) on a mutation corpus.
- Deployment: Provided as a fully merged model, meaning the LoRA weights are baked into the base model, allowing for direct loading and use without requiring PEFT (Parameter-Efficient Fine-Tuning) libraries.
- Merging Process: The merging was performed using a project KV-share-safe state-dict merger, targeting 205 LoRA targets with a scaling factor of 2.0.
Use Cases
This model is particularly suited for applications involving:
- Code Mutation Analysis: Understanding and generating variations in code, especially within Java-based projects like Commons Lang and fastutil.
- Code Quality and Testing: Potentially useful in scenarios requiring the generation or analysis of mutated code for testing and robustness evaluation.
- Standalone Deployment: Its merged nature simplifies deployment, making it easy to integrate into applications without complex PEFT setups.