AliMaatouk/Gemma-2B-Tele-it
AliMaatouk/Gemma-2B-Tele-it is a 2.6 billion parameter instruction-tuned causal language model developed by Ali Maatouk, based on Google's Gemma-2B architecture. Specialized in telecommunications, this model was fine-tuned using Supervised Fine-tuning (SFT) with Alpaca and Open-instruct datasets. It features an 8192-token context length and is designed to follow instructions, particularly excelling in telecommunications-related queries.
Loading preview...
Model Overview
Gemma-2B-Tele-it is an instruction-tuned variant of the Gemma-2B-Tele model, which itself is built upon Google's gemma-2b architecture. This model, developed by Ali Maatouk, is specifically specialized in the domain of telecommunications.
Key Capabilities
- Telecommunications Specialization: Fine-tuned to understand and generate responses related to telecommunications concepts and queries.
- Instruction Following: Utilizes Supervised Fine-tuning (SFT) on a combination of the Alpaca and Open-instruct datasets, enabling it to follow instructions effectively.
- Extended Context Window: Features a context length of 8192 tokens, allowing for processing longer inputs and generating more comprehensive responses.
Usage and Application
This model is designed for tasks requiring instruction-based interaction, particularly within the telecommunications field. An example use case involves explaining complex telecommunications concepts like "Shannon capacity." The model expects prompts to be delimited by "\n" for optimal instruction following. For more technical details, refer to the associated paper: Tele-LLMs: A Series of Specialized Large Language Models for Telecommunications.