Model Overview

neph1/Qwen2.5-Coder-7B-Instruct-Unity is a 7.6 billion parameter instruction-tuned model, developed by neph1 and fine-tuned from unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit. This model is specifically designed to assist with Unity3D development, leveraging a specialized training dataset.

Key Capabilities

• Unity3D Code Generation: Optimized for generating code and providing answers related to the Unity game engine.
• Specialized Knowledge: Trained on a merged dataset comprising Unity3D Q&A from sources like ibranze/codellama_unity3d_v2, Hypersniper/unity_api_2022_3, and neph1/Unity_Code_QnA.
• Efficient Training: Fine-tuned using Unsloth and Huggingface's TRL library, enabling faster training.
• Instruction Following: Responds well to instruction-based prompts, with preliminary testing showing good compatibility with the Mistral chat template.

Training Details

The model underwent approximately 1.5 epochs of training, with a focus on Unity-specific content. Training parameters included a per-device batch size of 2, gradient accumulation steps of 64, and a learning rate of 1e-4. Validation loss decreased steadily during training, indicating effective learning on the specialized dataset. The developer notes that further validation with general coding questions might be beneficial to prevent overfitting to the Unity domain.