Luau Qwen3 4B FIM v0.1 Overview

This model, developed by Zack Williams and sponsored by Torpedo Software LLC, is a specialized fine-tune of the Qwen/Qwen3-4B-Instruct-2507 base model. Unlike general-purpose chatbots, its primary function is Fill-in-the-Middle (FIM) Luau code completion, making it highly effective for autocomplete functionalities within integrated development environments.

Key Capabilities

• Luau Code Completion: Specifically trained to complete Luau code snippets, including end-of-line, end-of-block, and multi-line completions.
• FIM Training: Utilizes the "Efficient Training of Language Models to Fill in the Middle" methodology for robust code insertion and completion.
• Optimized for Developer Tools: Designed for integration with tools like Continue.dev for VSCode autocomplete, as demonstrated by provided configuration examples.
• Context Length: Supports a substantial context window of 40960 tokens, beneficial for understanding larger codebases.

Training Details

The model underwent Supervised Finetuning (SFT) using the TorpedoSoftware/the-luau-stack dataset, comprising 500,000 FIM-formatted Luau code snippets. Training involved approximately 140 GPU hours on an RTX 3090, utilizing a rank-stabilized LoRA adapter over 250,000 steps with a batch size of 2, and full precision training before merging to a BF16 model. Dynamic GGUF quantizations are also available, ranging from 2 to 4 GB.