abracsus/gemma-2b-brain-v6
abracsus/gemma-2b-brain-v6 is a 2.5 billion parameter Gemma-based language model, fine-tuned and converted to GGUF format using Unsloth. This model is optimized for efficient deployment and inference on local hardware, leveraging Unsloth's accelerated training and conversion process. It is suitable for general text generation tasks where a compact and performant model is required.
Loading preview...
Model Overview
abracsus/gemma-2b-brain-v6 is a 2.5 billion parameter language model based on the Gemma architecture. This version has been specifically fine-tuned and converted into the GGUF format, making it highly compatible with various inference engines, particularly those optimized for local execution.
Key Features
- Gemma Architecture: Built upon Google's Gemma family of open models.
- GGUF Format: Provided in the GGUF format, ensuring broad compatibility and efficient loading with tools like
llama-cpp. - Unsloth Optimization: The model was fine-tuned and converted using Unsloth, which facilitates faster training and conversion processes.
- Adjusted BOS Token Behavior: The Beginning-Of-Sentence (BOS) token behavior has been specifically adjusted to ensure proper compatibility within the GGUF ecosystem.
Usage Considerations
This model is designed for efficient text generation. Its GGUF format makes it particularly suitable for local inference on consumer hardware. Users can integrate it with llama-cli for text-only applications or llama-mtmd-cli if a multimodal version were to be released, using the --jinja flag for prompt templating. The primary available file is gemma-2b-it.Q4_K_M.gguf, indicating a quantized version for reduced memory footprint and faster inference.