stabilityai/japanese-stablelm-base-gamma-7b
Japanese Stable LM Base Gamma 7B is a 7-billion parameter decoder-only language model developed by Stability AI, specifically optimized for Japanese language modeling and downstream tasks. It was created by continuing pretraining on the English Mistral-7B-v0.1 model using a large corpus of Japanese data. This model excels in Japanese language generation and understanding, serving as a foundational model for application-specific fine-tuning.
Loading preview...
Japanese Stable LM Base Gamma 7B: Optimized for Japanese Language
Stability AI's Japanese Stable LM Base Gamma 7B is a 7-billion parameter decoder-only language model engineered to maximize performance in Japanese language modeling and related tasks. It builds upon the Mistral-7B-v0.1 architecture, undergoing continued pretraining with an extensive Japanese dataset to transfer and enhance its linguistic capabilities.
Key Capabilities
- Japanese Language Proficiency: Specifically designed and trained to achieve high performance in Japanese language understanding and generation.
- Foundational Model: Intended as a base model for developers to fine-tune for various application-specific use cases.
- Architecture: Based on the robust transformer decoder architecture of Mistral-7B-v0.1.
- Training Data: Continued pretraining utilized approximately 100 billion tokens from diverse Japanese corpora, including Japanese/English Wikipedia, Japanese mc4, CC-100, OSCAR, and SlimPajama.
Intended Use
This model is suitable for individuals and organizations looking to develop applications requiring strong Japanese language capabilities. It is licensed under Apache License, Version 2.0, allowing for broad commercial use. Users should be aware of potential biases or inappropriate content from the pre-training data and exercise caution in production systems.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.