kimjonguk/my-ai.Q4_K_M-v13
The kimjonguk/my-ai.Q4_K_M-v13 is a 2.6 billion parameter language model, finetuned and converted to GGUF format using Unsloth. This model is optimized for efficient deployment and usage, with its BOS token behavior adjusted for GGUF compatibility. It is suitable for general text-based LLM applications and can be easily deployed via Ollama.
Loading preview...
Model Overview
The kimjonguk/my-ai.Q4_K_M-v13 is a 2.6 billion parameter language model, specifically finetuned and converted into the GGUF format. This conversion was performed using Unsloth, a platform known for accelerating model training and conversion processes.
Key Characteristics
- GGUF Format: Optimized for efficient inference on various hardware, including CPUs.
- Unsloth Finetuning: The model was trained with Unsloth, indicating potential optimizations in its training process.
- Ollama Support: An Ollama Modelfile is included, simplifying deployment and local execution for developers.
- Adjusted BOS Token: The model's Beginning-of-Sentence (BOS) token behavior has been specifically adjusted to ensure full compatibility with the GGUF format.
Usage and Deployment
This model is designed for straightforward integration into applications. It supports both standard text-only LLM usage via llama-cli and is prepared for multimodal applications with llama-mtmd-cli. The inclusion of an Ollama Modelfile further streamlines its deployment, making it accessible for local development and experimentation.