ivanfioravanti/scope-guard-4B-q-2601-mlx-bf16
The ivanfioravanti/scope-guard-4B-q-2601-mlx-bf16 model is a 4 billion parameter language model, converted by ivanfioravanti to the MLX format from the principled-intelligence/scope-guard-4B-q-2601 base model. It supports a context length of 32768 tokens and is specifically designed for efficient deployment and inference on Apple Silicon using the MLX framework. This model is suitable for general language generation tasks within the MLX ecosystem.
Loading preview...
Overview
This model, ivanfioravanti/scope-guard-4B-q-2601-mlx-bf16, is a 4 billion parameter language model. It was converted by ivanfioravanti into the MLX format from the original principled-intelligence/scope-guard-4B-q-2601 model. The conversion was performed using mlx-lm version 0.31.3, making it optimized for use with Apple's MLX framework.
Key Characteristics
- Parameter Count: 4 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- MLX Compatibility: Specifically formatted for efficient inference on Apple Silicon hardware using the MLX library.
- Base Model: Derived from
principled-intelligence/scope-guard-4B-q-2601.
Usage
This model is designed for straightforward integration into MLX-based applications. Developers can load and generate text using the mlx-lm library, with built-in support for chat templates if available in the tokenizer. It provides a convenient option for local inference on compatible hardware.