BaseLayer/QUARK
BaseLayer/QUARK is a 1.5 billion parameter Qwen2-based causal language model developed by Marlon111. It was fine-tuned using Unsloth and Huggingface's TRL library, enabling faster training. This model is optimized for efficient performance with a 32768 token context length, making it suitable for applications requiring a balance of speed and context handling.
Loading preview...
Model Overview
BaseLayer/QUARK is a 1.5 billion parameter language model, fine-tuned by Marlon111. It is based on the Qwen2 architecture and utilizes the apache-2.0 license. The model was specifically trained for enhanced efficiency and speed using Unsloth and Huggingface's TRL library, building upon the unsloth/qwen2.5-1.5b-bnb-4bit base model.
Key Characteristics
- Architecture: Qwen2-based causal language model.
- Parameter Count: 1.5 billion parameters.
- Context Length: Supports a substantial 32768 tokens.
- Training Efficiency: Fine-tuned for 2x faster training using specialized libraries.
Use Cases
This model is well-suited for applications where a compact yet capable language model is required, especially those benefiting from its efficient training methodology and generous context window. Its design makes it a strong candidate for tasks that demand quick inference and processing of longer inputs.