GuminiResearch/Gumini-1B-Base
GuminiResearch/Gumini-1B-Base is a 1.08 billion parameter bilingual Korean-English base language model developed by Gumin Kwon. It was created by inheriting the first 10 layers of Qwen 2.5 3B using the Inheritune methodology and then continued pretraining on a Korean–English mixed corpus of 393 million tokens. This model is optimized for generating text continuations in both Korean and English, serving as a foundational model for further fine-tuning.
Loading preview...
Gumini-1B: A Bilingual Korean-English Base Model
GuminResearch/Gumini-1B-Base is a 1.08 billion parameter base language model developed by Gumin Kwon, designed for bilingual Korean-English text generation. It is built upon the Qwen 2.5 3B architecture, utilizing an innovative "Inheritune" methodology to create a smaller, efficient model.
Key Capabilities
- Bilingual Generation: Proficient in generating text in both Korean and English.
- Efficient Architecture: Inherits the first 10 transformer layers from Qwen 2.5 3B, reducing model size while retaining core linguistic abilities.
- Continued Pretraining: Further trained on a substantial Korean–English mixed corpus (393M tokens) to enhance its bilingual performance.
- Base Model Functionality: Designed for text completion and generation, providing a strong foundation for downstream tasks.
Good for
- Research and Development: Ideal for exploring the Inheritune methodology and its impact on model efficiency and performance.
- Bilingual Applications: Suitable as a base for fine-tuning on specific Korean-English tasks requiring text generation.
- Resource-Constrained Environments: Its smaller parameter count makes it more accessible for deployment where computational resources are limited.
- Understanding Model Inheritance: Provides a practical example of creating specialized models from larger foundational architectures.
Limitations
As a base model, Gumini-1B is not instruction-tuned or safety-aligned, meaning it generates text continuations rather than conversational responses. Users should apply a repetition penalty during generation to mitigate high repetition risk. It may also produce incorrect or outdated information and is not recommended for sensitive or safety-critical applications. This model is intended for non-commercial and research use only, under the Qwen Research License.