dougiefresh/jade_qwen3_4b
dougiefresh/jade_qwen3_4b is a Qwen 3 4B parameter model fine-tuned by dougiefresh for systems programming tasks. It was trained on a specialized dataset including documentation for Rust, Nushell, Cargo, Helix, various source code repositories, tealdeer commands, and macOS manpages. This model excels at generating responses related to systems programming, documentation, and code, with a particular focus on aarch64 assembly and Perl manpages.
Loading preview...
Jade Qwen 3 4B: A Systems Programming Fine-tune
Jade Qwen 3 4B is a specialized fine-tune of the Qwen 3 4B model, developed by dougiefresh, with a strong focus on systems programming knowledge. The model was trained using synthetic conversations generated from a diverse and high-quality dataset.
Key Capabilities & Training
- Specialized Knowledge Base: Fine-tuned on a unique dataset comprising:
- A "Grammar, Logic, Rhetoric, and Math" dataset.
- Documentation from projects like Rust, Nushell, Cargo, and Helix.
- Source code repositories including AArch64 Algorithms, Hyper, Ripgrep, and SQLite.
- Documentation for
tealdeercommands and macOS manpages.
- Synthetic Data Generation: Conversations were synthetically created using Qwen 3 8B, Qwen 3 4B, and Qwen 3 30B 3A, with a mix of CoT and
/nothinkprompts. - LoRA Adapters: Initially trained with a knowledge LoRA adapter for 3 epochs, followed by an identity dataset adapter for 30 epochs, aiming for wit and sarcasm.
- Model Merging: The knowledge and identity datasets were merged into the Qwen 3 4B base model using the DARE TIES method, weighting knowledge at 1.5 and identity at 0.5.
Intended Use Cases
- Systems Programming Assistance: Ideal for queries related to Rust, Nushell, Cargo, Helix, and various systems-level code.
- Documentation Retrieval: Effective for extracting information from technical documentation, including
tealdeercommands and manpages. - Code-Related Tasks: Useful for understanding and generating content related to source code, particularly aarch64 assembly.
While the model retains its updated knowledge base, the intended personality traits (wit and sarcasm) are more pronounced when using the identity LoRA adapter separately. The model may also exhibit a notable focus on Perl documentation due to its prevalence in the training data.