Llama-3-8B-TKK-Elite-V1.0 Overview

Llama-3-8B-TKK-Elite-V1.0 is an 8 billion parameter generative language model developed by Tarık Kaan Koç as an undergraduate graduation project. Built upon the LLaMA 8B architecture, this model was fine-tuned using the LORA method.

Key Training Details

• Base Model: LLaMA 8B
• Developer: Tarık Kaan Koç
• Training Data: A private, cleaned Turkish raw dataset, including 1 million raw instruction Turkish data.
• Training Method: LORA (Low-Rank Adaptation) fine-tuning.
• Training Duration: 133 hours and 59 minutes, totaling 37,420 steps, conducted on 8 Tesla V100 GPUs.
• LORA Configuration:
  • lora_alpha: 16
  • lora_dropout: 0.1
  • r: 64
  • bias: none
  • task_type: CAUSAL_LM

Capabilities

This model is designed for text-only input and output, primarily focused on generating responses in Turkish. Its fine-tuning on a specific Turkish instruction dataset suggests an optimization for understanding and producing Turkish language content based on given prompts.

Intended Use Cases

Llama-3-8B-TKK-Elite-V1.0 is particularly suited for applications requiring Turkish language generation, such as:

• Answering questions in Turkish.
• Generating conversational responses in Turkish.
• Text completion and content creation in Turkish.