AliMaatouk/Gemma-2B-Tele
AliMaatouk/Gemma-2B-Tele is a 2.6 billion parameter Transformer model, based on Google's Gemma-2B, specialized for telecommunications. Developed by Ali Maatouk, it was continuously pretrained on 2.5 billion tokens of telecommunications data, including articles and standards. This model outperforms the base Gemma-2B on telecommunications benchmarks while maintaining performance on general language understanding tasks, making it ideal for telecommunications-specific fine-tuning.
Loading preview...
Overview
Gemma-2B-Tele is a 2.6 billion parameter Transformer model developed by Ali Maatouk, based on Google's Gemma-2B architecture. It has been continuously pretrained on a specialized dataset called Tele-Data, comprising approximately 2.5 billion tokens of telecommunications-related material, including industry articles, standards, and general web content. The model supports a context length of 8192 tokens.
Key Capabilities
- Telecommunications Specialization: Optimized for understanding and generating content within the telecommunications domain.
- Enhanced Domain Performance: Outperforms the original Gemma-2B model on telecommunications-specific benchmarks like Tele-Eval.
- Retained General Performance: Maintains performance levels comparable to the base Gemma-2B across common sense, language understanding, and logical reasoning benchmarks.
- Base Model: Designed as a base model for further fine-tuning on telecommunications applications, operating primarily within a text completion framework.
Good For
- Fine-tuning: Ideal for developers looking to fine-tune a model for specific telecommunications tasks.
- Domain-Specific Text Completion: Generating relevant and accurate text completions for telecommunications-related prompts.
- Research: Exploring specialized language models for niche technical domains.
An instruction-tuned version of this model, Gemma-2B-Tele-it, is also available for instruction-following tasks.