kimjonguk/my-ai.Q4_K_M-v14
kimjonguk/my-ai.Q4_K_M-v14 is a 3.1 billion parameter language model, finetuned and converted to GGUF format by kimjonguk. This model leverages Unsloth for accelerated training and conversion, making it efficient for deployment. It is designed for general text-based applications, supporting both text-only and multimodal LLM interfaces. The model's GGUF format and Ollama Modelfile facilitate easy integration into local inference setups.
Loading preview...
Overview
kimjonguk/my-ai.Q4_K_M-v14 is a 3.1 billion parameter language model, provided in the efficient GGUF format. This model was finetuned and converted using Unsloth, a library known for accelerating the training and conversion process of large language models. The use of Unsloth allowed for a 2x faster training time, contributing to its optimized performance.
Key Capabilities
- Efficient Format: Available in
qwen2.5-3b-instruct.Q4_K_M.gguffor optimized local inference. - Accelerated Training: Benefited from Unsloth for faster finetuning.
- Easy Deployment: Includes an Ollama Modelfile for straightforward integration into Ollama environments.
- Flexible Usage: Supports both text-only and multimodal LLM applications via
llama-cliandllama-mtmd-cliwith Jinja templating.
When to Use This Model
This model is particularly suitable for developers looking for a compact yet capable language model that can be easily deployed on local hardware. Its GGUF format and Ollama integration make it ideal for projects requiring efficient, offline inference. The Unsloth-optimized training suggests it could be a good choice for applications where rapid iteration and deployment are important.