WabiSabi-V1: A Long-Context, Multilingual LLM
WabiSabi-V1 is a 7 billion parameter Large Language Model, fine-tuned from the Mistral-7B-v0.1 architecture by Local-Novel-LLM-project. This model was developed with support from the first LocalAI hackathon, focusing on enhancing several key capabilities over its base model.
Key Capabilities
- Extended Context Window: Features a significantly expanded 128k context window, a substantial increase from Mistral-7B-v0.1's 8k context, enabling much longer and more coherent interactions.
- Multilingual Generation: Achieves high-quality text generation in both Japanese and English.
- Enhanced Memory: Demonstrates improved memory retention over long-context generations, preventing information loss in extended dialogues.
- NSFW Content Generation: Capable of generating Not Safe For Work content.
Good For
- Applications requiring very long conversational contexts where memory retention is crucial.
- Projects needing high-quality text generation in both Japanese and English.
- Creative writing or specialized applications that may involve NSFW content.
- Users looking for a Mistral-7B-based model with a significantly larger context window and improved multilingual capabilities.
Note: The model uses the Vicuna-v1.1 instruction format. Users should be aware that training data may introduce biases, and memory usage can be substantial for long inferences. Inference with llamacpp is recommended over Transformers for optimal performance.