Hugging Face Model Overview
The DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-MADNESS is a 12 billion parameter language model with a 32768 token context length. It is distributed in full precision 'safe tensors' format, enabling the generation of various quantized formats such as GGUF, GPTQ, EXL2, AWQ, and HQQ. This model is categorized as a "Class 1" model, indicating that its performance can be significantly enhanced by applying specific parameter and sampler settings.
Key Characteristics
- Flexible Format: Provided in full precision for broad compatibility with different quantization methods.
- High Context Length: Supports a 32768 token context, allowing for processing longer inputs and maintaining conversational coherence.
- Settings-Dependent Performance: Emphasizes the importance of detailed configuration (parameters, samplers, advanced samplers) to achieve optimal operation, especially for use cases beyond its default design.
Optimal Operation Guide
The developer strongly recommends reviewing the associated guide, "Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters" (https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters). This document provides critical information on:
- Specific parameter and sampler settings for this model's "class."
- Methods to improve model performance for all use cases, including chat and roleplay.
- Techniques to enable full operation for use cases the model was not originally designed for.
These settings are applicable to any model, from any repository, and across all quantization types, including full precision.