Forge Coder Qwen v1.21.11

A Specialized Code Generation Model for Minecraft Forge Mod Development

Powered by Qwen2.5-Coder-7B with 128K Context Window

Quick Start • Capabilities • Examples • Training Details

--- ## Overview **Forge Coder Qwen** is a fine-tuned large language model specifically designed to assist developers in creating Minecraft Forge mods. Built on top of **Qwen2.5-Coder-7B-Instruct**, this model has been trained on extensive Forge mod source code and documentation to provide accurate, idiomatic, and up-to-date code generation for Minecraft modding. ### Key Features - **128K Context Window**: Handle large codebases and complex mod structures in a single context - **Specialized Knowledge**: Deep understanding of Forge API, registry systems, and modding patterns - **Version-Aligned**: Trained specifically for Minecraft 1.21.11 and Forge 1.21.11 - **Code Completion**: Generate complete mod components from natural language descriptions - **Best Practices**: Follows modern Forge modding conventions and patterns ## Model Details | Property | Value | |----------|-------| | **Base Model** | [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) | | **Context Length** | **128K tokens** | | **Fine-tuning Method** | QLoRA (4-bit quantization + LoRA) | | **LoRA Rank** | 64 | | **LoRA Alpha** | 128 | | **Trainable Parameters** | 161.5M (2.08% of 7.7B) | | **Target Forge Version** | 1.21.11 | | **Target Minecraft Version** | 1.21.11 | | **MCP Mappings** | 20251209.095502 | ## Quickstart ### Installation ```bash pip install transformers peft accelerate bitsandbytes torch ``` ### Basic Usage ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel # Quantization config for efficient inference bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True, ) # Load model base_model_id = "Qwen/Qwen2.5-Coder-7B-Instruct" adapter_id = "hwding/forge-coder-qwen-v1.21.11" tokenizer = AutoTokenizer.from_pretrained(adapter_id) model = AutoModelForCausalLM.from_pretrained( base_model_id, quantization_config=bnb_config, device_map="auto", ) model = PeftModel.from_pretrained(model, adapter_id) # Generate code def generate_forge_code(prompt: str, max_tokens: int = 2048) -> str: messages = [ {"role": "system", "content": "You are an expert Minecraft Forge mod developer for version 1.21.11. Write clean, efficient, and well-structured Java code."}, {"role": "user", "content": prompt} ] text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tokenizer(text, return_tensors="pt").to(model.device) outputs = model.generate( **inputs, max_new_tokens=max_tokens, temperature=0.7, top_p=0.95, do_sample=True, pad_token_id=tokenizer.eos_token_id, ) return tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True) # Example code = generate_forge_code("Create a custom ore block that drops experience when mined") print(code) ``` ### OpenAI-Compatible API (with vLLM) ```bash # Start vLLM server python -m vllm.entrypoints.openai.api_server \ --model Qwen/Qwen2.5-Coder-7B-Instruct \ --enable-lora \ --lora-modules forge-coder=hwding/forge-coder-qwen-v1.21.11 \ --max-lora-rank 64 \ --dtype float16 \ --max-model-len 32768 \ --port 8000 ``` ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/v1", api_key="none") response = client.chat.completions.create( model="forge-coder", messages=[ {"role": "system", "content": "You are an expert Minecraft Forge mod developer."}, {"role": "user", "content": "Create a teleportation staff item"} ], max_tokens=2048 ) print(response.choices[0].message.content) ``` ## Capabilities The model excels at generating code for various Forge mod components: | Category | Examples | |----------|----------| | **Blocks** | Custom blocks, ore blocks, machine blocks, multi-blocks | | **Items** | Tools, weapons, armor, food items, custom items | | **Entities** | Mobs, projectiles, vehicles, tile entities | | **GUI/Screens** | Container menus, custom screens, HUD elements | | **Networking** | Packet handling, client-server sync | | **World Generation** | Ore generation, structures, biomes, features | | **Events** | Forge event handlers, lifecycle events | | **Registries** | DeferredRegister patterns, registry objects | | **Capabilities** | Custom capabilities, capability providers | | **Recipes** | Custom recipe types, recipe serializers | | **Data Generation** | Datagen for models, blockstates, loot tables, tags | | **Rendering** | Block/item/entity renderers, shaders | ## Examples ### Creating a Custom Block **Prompt:** "Create a copper lamp block that emits light when powered by redstone" ```java public class CopperLampBlock extends Block { public static final BooleanProperty LIT = BlockStateProperties.LIT; public CopperLampBlock(Properties properties) { super(properties); this.registerDefaultState(this.stateDefinition.any().setValue(LIT, false)); } @Override protected void createBlockStateDefinition(StateDefinition.Builder builder) { builder.add(LIT); } @Override public void neighborChanged(BlockState state, Level level, BlockPos pos, Block block, BlockPos fromPos, boolean isMoving) { if (!level.isClientSide) { boolean powered = level.hasNeighborSignal(pos); if (powered != state.getValue(LIT)) { level.setBlock(pos, state.setValue(LIT, powered), 2); } } } @Override public int getLightEmission(BlockState state, BlockGetter level, BlockPos pos) { return state.getValue(LIT) ? 15 : 0; } } ``` ### Creating a Custom Item **Prompt:** "Create a magic wand item that shoots fireballs on right click" ```java public class MagicWandItem extends Item { public MagicWandItem(Properties properties) { super(properties.stacksTo(1).durability(100)); } @Override public InteractionResultHolder use(Level level, Player player, InteractionHand hand) { ItemStack stack = player.getItemInHand(hand); if (!level.isClientSide) { Vec3 look = player.getLookAngle(); SmallFireball fireball = new SmallFireball(level, player, look.x, look.y, look.z); fireball.setPos(player.getX(), player.getEyeY() - 0.1, player.getZ()); level.addFreshEntity(fireball); stack.hurtAndBreak(1, player, (p) -> p.broadcastBreakEvent(hand)); player.getCooldowns().addCooldown(this, 20); } level.playSound(player, player.blockPosition(), SoundEvents.BLAZE_SHOOT, SoundSource.PLAYERS, 1.0F, 1.0F); return InteractionResultHolder.sidedSuccess(stack, level.isClientSide); } } ``` ## Training Data ### Data Sources | Source | Description | Files | |--------|-------------|-------| | **Forge Source** | MinecraftForge core, ForgeGradle, EventBus, ModLauncher | ~1,200 | | **Popular Mods** | 27 well-maintained open-source mod repositories | ~21,700 | | **Documentation** | Official Forge docs and tutorials | 74 | ### Featured Mod Repositories Training data includes code from highly-regarded mods: - **Applied Energistics 2** - Storage & automation - **Mekanism** - Tech & machinery - **Create** - Mechanical contraptions - **Botania** - Nature magic - **Thermal Series** - Energy systems - **Tinkers' Construct** - Tool crafting - **Immersive Engineering** - Industrial machines - **JustEnoughItems (JEI)** - Recipe viewing - **TerraFirmaCraft** - Survival overhaul - **The Twilight Forest** - Dimension mod - **Quark** - Vanilla enhancements - **RFTools** - RF-powered utilities - And 15 more... ### Dataset Statistics | Metric | Value | |--------|-------| | Total Java Files Processed | 22,916 | | Training Samples | 13,936 | | Validation Samples | 734 | | Sample Types | Code completion, explanation, Q&A | ## Training ### Configuration | Parameter | Value | |-----------|-------| | Epochs | 3 | | Batch Size | 2 (per device) | | Gradient Accumulation | 8 steps | | Effective Batch Size | 16 | | Learning Rate | 2e-4 | | LR Scheduler | Cosine | | Warmup Ratio | 3% | | Max Sequence Length | 4,096 tokens | | Precision | BF16 | | Hardware | 8x NVIDIA H20 (96GB each) | ### Training Metrics | Metric | Value | |--------|-------| | Training Duration | 12h 3m | | Total Steps | 867 | | Final Training Loss | 0.283 | | Final Validation Loss | 0.422 | | Token Accuracy | **92.17%** | | Eval Accuracy | 88.79% | ### Loss Curve ``` Epoch 1: 0.87 → 0.43 Epoch 2: 0.38 → 0.34 Epoch 3: 0.28 → 0.28 ``` ### Framework Versions - PEFT: 0.18.0 - TRL: 0.26.1 - Transformers: 4.57.3 - PyTorch: 2.5.1+cu121 - Datasets: 4.4.1 - Tokenizers: 0.22.1 ## Comparison with Previous Version | Feature | forge-coder-v1.21.11 | forge-coder-qwen-v1.21.11 | |---------|---------------------|---------------------------| | Base Model | DeepSeek-Coder-6.7B | Qwen2.5-Coder-7B | | Context Length | 16K | **128K** | | Token Accuracy | 92.5% | 92.17% | | Max Sequence | 2,048 | 4,096 | ## Limitations - **Version Specific**: Optimized for Forge 1.21.11; may produce outdated patterns for older versions - **Java Only**: Does not generate Kotlin, Gradle scripts, or JSON resources - **No Runtime Testing**: Generated code should be tested before use in production - **Large Files**: While 128K context is supported, very long generations may lose coherence ## Intended Use **Recommended Uses:** - Learning Forge modding patterns and best practices - Rapid prototyping of mod components - Code completion and suggestions - Understanding Forge API usage - Working with large mod codebases (thanks to 128K context) **Not Recommended For:** - Production code without review - Security-critical applications - Forge versions significantly different from 1.21.11 ## Citation If you use this model in your research or projects, please cite: ```bibtex @misc{forge-coder-qwen-2024, author = {hwding}, title = {Forge Coder Qwen: A Specialized Code Generation Model for Minecraft Forge Mod Development}, year = {2024}, publisher = {Hugging Face}, url = {https://huggingface.co/hwding/forge-coder-qwen-v1.21.11} } ``` ## License This model is released under the **Apache 2.0 License**, consistent with the Qwen2.5-Coder base model. Training data was sourced from open-source repositories under various permissive licenses (MIT, Apache 2.0, LGPL, etc.). ## Acknowledgments - [Qwen Team](https://github.com/QwenLM) for the excellent Qwen2.5-Coder base model - [MinecraftForge](https://github.com/MinecraftForge) team for the modding framework - All open-source mod developers whose code made this training possible ---

Happy Modding!