VertexAGI/amethyst-1-mini
VertexAIco/amethyst-1-mini is a 4 billion parameter general-purpose chat and instruction-following model, fine-tuned from Google's Gemma 3 4B IT. Developed by an independent research project, it was created using LoRA on a small, high-quality distilled instruction dataset. This model is designed for experimentation, research into small-scale distillation pipelines, and hobbyist deployment, focusing on validating an end-to-end distillation and fine-tuning pipeline on consumer hardware.
Loading preview...
Amethyst 1 Mini: Distilled Instruction-Following Model
Amethyst 1 Mini is a 4 billion parameter, general-purpose chat and instruction-following model, developed as an independent research project. It is fine-tuned from Google's Gemma 3 4B IT using LoRA, representing the first model in the Amethyst family. The primary goal of this release is to validate an end-to-end distillation, fine-tuning, and evaluation pipeline on consumer hardware.
Key Capabilities & Training:
- Base Architecture: Utilizes the Gemma 3, 4B parameter dense, decoder-only transformer architecture.
- Distilled Data: Fine-tuned on 1,122 high-quality instruction/response pairs, synthetically generated via knowledge distillation from
nvidia/nemotron-3-super-120b-a12b. - Broad Coverage: The training dataset spans diverse general-chat categories, including reasoning, code generation, creative writing, and translation, aiming for well-rounded conversational ability.
- Fine-tuning Method: Employs LoRA (rank 8, scale 20.0) on Apple Silicon using the MLX framework, with the adapter fused into the base weights and dequantized to fp16 for direct
transformerscompatibility.
Intended Use & Limitations:
- Good for: Experimentation, research into small-scale distillation pipelines, and hobbyist deployment. It serves as a lightweight, general-purpose conversational assistant.
- Limitations: Due to its small, synthetic training dataset, behavior can be inconsistent outside trained categories. It inherits the limitations and knowledge cutoff of its Gemma 3 4B IT base model and is not intended for high-stakes or production use.