kimjonguk/my-ai.Q4_K_M-v12
kimjonguk/my-ai.Q4_K_M-v12 is a 2.6 billion parameter language model, finetuned and converted to GGUF format by kimjonguk. This model leverages Unsloth for accelerated training and GGUF conversion, making it suitable for efficient local deployment. It is designed for general text-based applications, with specific optimizations for GGUF compatibility.
Loading preview...
Model Overview
kimjonguk/my-ai.Q4_K_M-v12 is a 2.6 billion parameter language model, provided in the GGUF format for efficient local inference. This model was finetuned and converted using Unsloth, a framework known for accelerating training and conversion processes, enabling a 2x faster training time.
Key Features
- GGUF Format: Optimized for compatibility with
llama-cliand other GGUF-compatible inference engines. - Unsloth Integration: Benefits from accelerated finetuning and conversion, making it efficient to deploy.
- Ollama Support: Includes an Ollama Modelfile for streamlined deployment within the Ollama ecosystem.
- Adjusted BOS Token: The model's Beginning-Of-Sentence (BOS) token behavior has been adjusted to ensure proper GGUF compatibility.
Use Cases
This model is suitable for various text-only language model applications, particularly where efficient local deployment and GGUF compatibility are priorities. Its 2.6B parameter size makes it a good candidate for resource-constrained environments.