Dans-Archive/Dans-AdventurousWinds-Mk2-7b
Dans-Archive/Dans-AdventurousWinds-Mk2-7b is a 7 billion parameter language model built upon the Mistral-7b architecture, fine-tuned for generating text-based adventure games. It excels at crafting both concise and expansive, novel-like descriptions, with the ability to switch between styles via system messages. The model was trained on a custom version of the floyd and cys datasets from the skein text adventure dataset, utilizing 16k sequence lengths to enhance its narrative capabilities.
Loading preview...
Model Overview
Dans-AdventurousWinds-Mk2-7b is a 7 billion parameter model, based on the Mistral-7b architecture, specifically fine-tuned for generating text-based adventure games. It leverages an improved custom dataset derived from the skein text adventure dataset, enhancing its ability to create engaging and detailed interactive narratives. A key feature is its training on 16k sequence lengths, allowing for more coherent and extended story generation.
Key Capabilities
- Text-based Adventure Game Generation: Proficient in creating dynamic and interactive adventure game scenarios.
- Flexible Response Styles: Can produce both concise replies and expansive, novel-like descriptions, adaptable via system messages.
- Custom Training Data: Utilizes a refined version of the floyd and cys datasets for specialized performance in its domain.
- Extended Context: Trained with 16k sequence lengths, supporting longer and more complex narrative flows.
Training Details
The model underwent QLoRA training for 3 epochs over 5 hours on 4x RTX 4090 GPUs, with PEFT R/A set to 32/32. Quantized versions (GGUF, GPTQ, AWQ) are available through TheBloke for various hardware configurations.
Good For
- Developers and enthusiasts looking to create interactive fiction and text adventure games.
- Applications requiring dynamic, story-driven content generation with adaptable verbosity.
- Experimenting with models trained on specialized narrative datasets and extended context windows.