Sao10K/MN-12B-Lyra-v4

Model Overview

Sao10K/MN-12B-Lyra-v4 is a 12 billion parameter language model derived from the Mistral-NeMo architecture. It represents an iterative refinement, layered over Lyra-v3, which itself was built upon Lyra-v2a2 and Lyra-v2a1. This particular version, Lyra-v4, introduces a distinct Reinforcement Learning (RL) step focused on enhancing instruction following and overall coherency, aiming to resolve quantization-related issues observed in prior versions.

Key Features & Improvements

• Iterative Development: Built upon a lineage of previous Lyra models (v4a1, v3, v2a2, v2a1).
• RL for Coherency: Incorporates a dedicated RL step to improve instruction adherence and response coherency.
• Quantization Fixes: Designed to address and mitigate quantization-based problems.
• ChatML Support: Compatible with ChatML and its variants for structured conversations.

Usage Recommendations

• Prompt Formats: Supports both <|im_start|>system and [INST]system style ChatML templates.
• Sampler Settings: Recommended Temperature between 0.6-1 and min_p between 0.1-0.2, with min_p set before Temperature in sampler orders.
• Stopping Strings: Advised to use <|im_end|>, </s>, and [/INST] for proper response termination.

Notes

• The model aims to fix extra token generation issues reported by some users.
• Users encountering malformed stopping strings with XML tags should add them to their existing list.
• The weights for this version are stated to be identical to previous versions, with changes primarily in configuration and tokenizer settings, crediting ArliAI for tokenizer configurations.