uukuguy/speechless-codellama-dolphin-orca-platypus-13b
The uukuguy/speechless-codellama-dolphin-orca-platypus-13b is a 13 billion parameter language model fine-tuned from Meta's CodeLlama-13b-hf. It was trained for one epoch using a blend of Dolphin (1% GPT4), Orca (1% GPT4), and Platypus (100%) datasets. This model is designed for general code synthesis and understanding, offering capabilities in code completion and infilling.
Loading preview...
Model Overview
The uukuguy/speechless-codellama-dolphin-orca-platypus-13b is a 13 billion parameter model derived from Meta's Code Llama architecture. It has been fine-tuned for one epoch using a combination of Dolphin (1% GPT4), Orca (1% GPT4), and Platypus (100%) datasets, aiming to enhance its performance across various tasks.
Key Capabilities
- Code Completion: Generates relevant code snippets to complete partial code.
- Infilling: Fills in missing sections within existing code structures.
- General Code Understanding: Designed to comprehend and process code-related queries.
Performance Metrics
Evaluations on the Open LLM Leaderboard show the model achieving an average score of 44.41. Specific benchmark results include:
- ARC (25-shot): 44.8
- HellaSwag (10-shot): 68.6
- MMLU (5-shot): 44.03
- TruthfulQA (0-shot): 46.28
Training Details
This model is a fine-tuned version of the codellama/CodeLlama-13b-hf base model. The original Code Llama models were developed by Meta and trained between January and July 2023, utilizing an optimized transformer architecture. More details on the base model's training and architecture can be found in the research paper "Code Llama: Open Foundation Models for Code".
Intended Use
The model is intended for commercial and research use in English and relevant programming languages, particularly for code synthesis and understanding tasks. It is not designed for instruction following or chat applications, nor is it a Python specialist.