orkungedik/ege-v1.3
orkungedik/ege-v1.3 is a 27 billion parameter language model fine-tuned by orkungedik using the TRL framework. This model is a fine-tuned version of an unspecified base model, optimized through supervised fine-tuning (SFT). It features a 32768 token context length, making it suitable for tasks requiring extensive contextual understanding. Its primary strength lies in its fine-tuned nature, suggesting specialized performance for specific applications, though the exact domain is not specified.
Loading preview...
Model Overview
orkungedik/ege-v1.3 is a 27 billion parameter language model developed by orkungedik. This model has been fine-tuned using the TRL (Transformer Reinforcement Learning) framework, specifically employing Supervised Fine-Tuning (SFT) as its training procedure. While the base model for this fine-tuning is not specified, the process indicates an optimization for particular tasks or performance characteristics.
Key Characteristics
- Parameter Count: 27 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens, enabling it to process and generate longer sequences of text.
- Training Framework: Fine-tuned with TRL, a library for transformer reinforcement learning.
- Training Method: Utilizes Supervised Fine-Tuning (SFT).
Usage
The model can be used for text generation tasks. A quick start example demonstrates how to use the transformers pipeline for generating responses to prompts, with a maximum new token generation of 128 tokens.
Technical Details
The model was trained with specific versions of key frameworks:
- TRL: 0.24.0
- Transformers: 5.12.1
- Pytorch: 2.9.0
- Datasets: 4.3.0
- Tokenizers: 0.22.2
This fine-tuned model is designed for applications that can leverage its large parameter count and extended context window, particularly where SFT-based optimizations are beneficial.