Delta-Vector/Austral-24B-Winton

Austral 24B Winton: A Roleplay & Adventure Generalist

Delta-Vector's Austral 24B Winton is a 24 billion parameter language model built upon the Mistral architecture, specifically a finetune of the Harbinger 24B model. It is designed to excel as a generalist model for adventure and roleplay applications.

Key Enhancements & Training:

• Multi-stage Finetune: The model underwent a multi-stage finetuning process, starting with 4 epochs of SFT (Supervised Fine-Tuning) using a similar datamix to Francois-Huali/Austral 70B.
• KTO Alignment: It incorporates KTO (Kahneman-Tversky Optimization) alignment to address coherency issues and improve general writing quality.
• Rep_remover SFT: A final 4-epoch SFT phase with "Rep_remover" was applied to eliminate repetitive outputs, or "slops," enhancing the model's narrative flow.
• Training Resources: The entire training process, including SFT and KTO, took approximately 80 hours using 8 x A100 GPUs.

Usage & Formats:

• Chat Format: The model utilizes the ChatML format for conversational interactions.
• Quantized Versions: Optimized quantized versions are available for various inference engines:
  • GGUF for LLama.cpp and its forks.
  • EXL3 for TabbyAPI.
  • EXL2 for Tabby, offering faster performance on Ampere GPUs.

This model is ideal for developers and users seeking a robust and refined language model for engaging and coherent roleplaying and adventure-style text generation.