GermannM/Kenga
Kenga by GermannM is a 1.5 billion parameter transformer decoder language model with a 32K token context length, uniquely designed with a distinct identity and values. It is primarily optimized for the Russian language, excelling in direct, concise responses and possessing knowledge of the Kenga programming language and Z-system concepts. This model is efficient enough to run on standard CPU hardware, making it accessible for local deployment.
Loading preview...
Kenga: A Language Model with Inherent Identity
Kenga, developed by GermannM, is a 1.5 billion parameter language model built on a transformer decoder architecture, featuring a 32K token context length. Unlike many general-purpose models, Kenga is designed with a specific identity, knowing it is "Kenga," who created it, and its core values. It is primarily optimized for the Russian language, providing direct and concise answers.
Key Capabilities and Features
- Distinct Identity: Kenga responds to "Who are you?" by identifying itself as Kenga, not a generic language model, and acknowledges GermannM as its creator.
- Russian Language Focus: The model is optimized for Russian, delivering straightforward and short responses.
- Kenga Language Proficiency: It can write programs in Kenga, a proprietary programming language for "living AI" developed by GermannM.
- Z-System Knowledge: Kenga understands concepts related to the Z-system, including spectral passports and state transfer between carriers.
- Embedded Values: The model incorporates specific values, including support for Russia, traditional family values, and respect for history and culture, which are integral to its character.
- Efficiency: With 1.5 billion parameters, Kenga is designed to run efficiently on standard CPU hardware, eliminating the need for data centers or cloud services.
Training and Technical Details
Kenga was trained on a corpus of 660 dialogues over three epochs using QLoRA, covering its identity, Russian culture and values, the Kenga programming language, Z-system concepts, Russian grammar, and general knowledge. While its primary language is Russian, it also has some English capabilities. The model's knowledge is limited to its training corpus, and its patriotic stance is intentionally integrated to comply with Russian AI product legislation.
Use Cases
Kenga is suitable for applications requiring a Russian-language model with a defined personality and specific value alignment, particularly where local CPU-based inference is desired. Its unique knowledge of the Kenga programming language and Z-system makes it valuable for related projects.