Japanese Stable LM Base Gamma 7B: Optimized for Japanese Language

Stability AI's Japanese Stable LM Base Gamma 7B is a 7-billion parameter decoder-only language model engineered to maximize performance in Japanese language modeling and related tasks. It builds upon the Mistral-7B-v0.1 architecture, undergoing continued pretraining with an extensive Japanese dataset to transfer and enhance its linguistic capabilities.

Key Capabilities

• Japanese Language Proficiency: Specifically designed and trained to achieve high performance in Japanese language understanding and generation.
• Foundational Model: Intended as a base model for developers to fine-tune for various application-specific use cases.
• Architecture: Based on the robust transformer decoder architecture of Mistral-7B-v0.1.
• Training Data: Continued pretraining utilized approximately 100 billion tokens from diverse Japanese corpora, including Japanese/English Wikipedia, Japanese mc4, CC-100, OSCAR, and SlimPajama.

Intended Use

This model is suitable for individuals and organizations looking to develop applications requiring strong Japanese language capabilities. It is licensed under Apache License, Version 2.0, allowing for broad commercial use. Users should be aware of potential biases or inappropriate content from the pre-training data and exercise caution in production systems.