Model Overview

Dans-AdventurousWinds-Mk2-7b is a 7 billion parameter model, based on the Mistral-7b architecture, specifically fine-tuned for generating text-based adventure games. It leverages an improved custom dataset derived from the skein text adventure dataset, enhancing its ability to create engaging and detailed interactive narratives. A key feature is its training on 16k sequence lengths, allowing for more coherent and extended story generation.

Key Capabilities

• Text-based Adventure Game Generation: Proficient in creating dynamic and interactive adventure game scenarios.
• Flexible Response Styles: Can produce both concise replies and expansive, novel-like descriptions, adaptable via system messages.
• Custom Training Data: Utilizes a refined version of the floyd and cys datasets for specialized performance in its domain.
• Extended Context: Trained with 16k sequence lengths, supporting longer and more complex narrative flows.

Training Details

The model underwent QLoRA training for 3 epochs over 5 hours on 4x RTX 4090 GPUs, with PEFT R/A set to 32/32. Quantized versions (GGUF, GPTQ, AWQ) are available through TheBloke for various hardware configurations.

Good For

• Developers and enthusiasts looking to create interactive fiction and text adventure games.
• Applications requiring dynamic, story-driven content generation with adaptable verbosity.
• Experimenting with models trained on specialized narrative datasets and extended context windows.