Himeyuri-v0.1-12B Overview

Himeyuri-v0.1-12B is an experimental 12 billion parameter language model developed by Elizezen, built upon the Mistral-Nemo-Instruct-2407 base architecture. This model represents an exploration into alternative architectures beyond LLaMA 3 and Mistral 7B-based models, aiming to improve Japanese language capabilities in open LLMs. The developer notes that Mistral-Nemo-Instruct-2407 shows significant promise for Japanese text generation.

Key Characteristics & Usage

• Base Model: Utilizes the Mistral-Nemo-Instruct-2407 architecture.
• Temperature Recommendation: It is recommended to use a low temperature during generation, as Mistral Nemo models tend to drastically change scenes, and a lower temperature can help mitigate this.
• Experimental Nature: The model is described as experimental with considerable room for future improvements.

Intended Use Cases

• Novel Generation: The primary intended use for Himeyuri-v0.1-12B is the generation of novel content and extended narratives.

Limitations

• Not Optimized For: This model is explicitly not optimized for role-playing (RP) scenarios or general instruction-based responses.