eulogik/Bharat-Tiny-LLM-fused
Bharat-Tiny-LLM-fused by eulogik is a 1.5 billion parameter full-precision fused model, merging a LoRA adapter into the Qwen2.5-1.5B base weights in PyTorch float16. This model is designed for direct inference on CPU/CUDA with transformers, further fine-tuning, or for producing custom quantized builds like GGUF or MLX. It offers a versatile base for developers needing a compact yet capable language model for various deployment scenarios.
Loading preview...
Overview
eulogik/Bharat-Tiny-LLM-fused is the full-precision fused version of the Bharat-Tiny-LLM, developed by eulogik. It integrates a LoRA adapter with the Qwen2.5-1.5B base model, provided in PyTorch float16 format. This repository serves as a foundational model for developers who require the full-precision weights for specific applications.
Key Capabilities
- Direct Inference: Ready for use with
transformerson CPU or CUDA environments. - Further Fine-tuning: Provides a robust base for additional training and adaptation to specific tasks.
- Custom Quantization: Ideal for generating optimized, smaller builds such as GGUF (for
llama.cpp, Android, Raspberry Pi, CPU) or MLX (for Apple Silicon/edge devices). - Compact Size: At approximately 3.3 GB in
fp16, it offers a balance of capability and manageable size for server-side deployment or as a fine-tuning base.
Good For
- Developers looking to run inference on server-grade hardware.
- Researchers and engineers who need to fine-tune the model for specialized applications.
- Creators aiming to produce their own highly optimized, quantized versions for diverse hardware platforms, including edge devices and mobile.
For most users seeking smaller, ready-to-run builds, eulogik also provides MLX 4-bit (880 MB) for Apple Silicon and GGUF Q4_K_M (1.06 GB) for cross-platform llama.cpp usage.