Kype720184/DorsetHeatwaveLLM

DorsetHeatwaveLLM: An Experimental 2B Parameter Finetune

Kype720184/DorsetHeatwaveLLM is a 2 billion parameter language model finetuned by Kype720184 from the Qwen 3 1.7B base model. Developed as a personal project, this model showcases an individual's initial foray into LLM finetuning, completed in June 2026. It operates with a context length of 32768 tokens.

Key Characteristics & Development

• Base Model: Finetuned from the Qwen 3 1.7B model, chosen for its compatibility with existing tools.
• Licensing: Distributed under the AGPLv3 to prevent commercial exploitation and ensure responsible use, reflecting the developer's dedication and personal investment.
• Experimental Nature: The developer explicitly states this is their first LLM finetune, acknowledging potential limitations such as hallucination and roleplay, common in smaller models.
• Training Details: Utilized Unsloth for finetuning with specific hyperparameters including a batch size of 2, learning rate of 0.0002, and 1250 max steps. LoRA parameters included a rank of 16 and alpha of 16.

Noteworthy Aspects

• Testing Approach: The developer opted for custom chat prompts over standard benchmarks, believing them to be more representative of real-world use, though acknowledging these tests may not be 100% representative.
• Community Engagement: The developer encourages helpful suggestions for improvement, highlighting an open approach to learning and development.

Important Considerations

• Limitations: As a ~2B parameter model, it is prone to hallucination and inaccuracies. It is not recommended for mission-critical applications.
• Deprecation: This version is explicitly marked as deprecated in favor of newer iterations, indicating ongoing development by Kype720184.