TorpedoSoftware/Luau-Qwen3-4B-FIM-v0.1
The TorpedoSoftware/Luau-Qwen3-4B-FIM-v0.1 is a 4 billion parameter Qwen3-based language model developed by Zack Williams and sponsored by Torpedo Software LLC. It is specifically fine-tuned for Fill-in-the-Middle (FIM) Luau code completion tasks, leveraging a 40960 token context length. This model excels at providing Luau autocomplete suggestions rather than general chatbot interactions, making it ideal for integrating into code editors and development environments.
Loading preview...
Luau Qwen3 4B FIM v0.1 Overview
This model, developed by Zack Williams and sponsored by Torpedo Software LLC, is a specialized fine-tune of the Qwen/Qwen3-4B-Instruct-2507 base model. Unlike general-purpose chatbots, its primary function is Fill-in-the-Middle (FIM) Luau code completion, making it highly effective for autocomplete functionalities within integrated development environments.
Key Capabilities
- Luau Code Completion: Specifically trained to complete Luau code snippets, including end-of-line, end-of-block, and multi-line completions.
- FIM Training: Utilizes the "Efficient Training of Language Models to Fill in the Middle" methodology for robust code insertion and completion.
- Optimized for Developer Tools: Designed for integration with tools like Continue.dev for VSCode autocomplete, as demonstrated by provided configuration examples.
- Context Length: Supports a substantial context window of 40960 tokens, beneficial for understanding larger codebases.
Training Details
The model underwent Supervised Finetuning (SFT) using the TorpedoSoftware/the-luau-stack dataset, comprising 500,000 FIM-formatted Luau code snippets. Training involved approximately 140 GPU hours on an RTX 3090, utilizing a rank-stabilized LoRA adapter over 250,000 steps with a batch size of 2, and full precision training before merging to a BF16 model. Dynamic GGUF quantizations are also available, ranging from 2 to 4 GB.