speechlessai/speechless-codellama-dolphin-orca-platypus-13b
The speechlessai/speechless-codellama-dolphin-orca-platypus-13b is a 13 billion parameter language model fine-tuned from CodeLlama-13b-hf. It was trained for two epochs using a combination of Dolphin (5% GPT4), Orca (5% GPT4), and Platypus (100%) datasets. This model demonstrates enhanced code generation capabilities, achieving 43.29 on humaneval-python, and also shows general language understanding with an average score of 51.02 on the Open LLM Leaderboard benchmarks.
Loading preview...
Model Overview
The speechlessai/speechless-codellama-dolphin-orca-platypus-13b is a 13 billion parameter language model derived from the codellama/CodeLlama-13b-hf base model. It has undergone a two-epoch fine-tuning process utilizing a blend of datasets: Dolphin (5% GPT4), Orca (5% GPT4), and Platypus (100%). This fine-tuning aims to enhance both its coding and general language understanding capabilities.
Key Capabilities
- Code Generation: Excels at code completion and infilling tasks, achieving a humaneval-python score of 43.29.
- General Language Understanding: Demonstrates proficiency across various benchmarks, with an average score of 51.02 on the Open LLM Leaderboard, including ARC (45.82), HellaSwag (67.71), MMLU (45.88), and TruthfulQA (44.67).
- Optimized Transformer Architecture: Built upon an optimized transformer architecture, designed for efficient text generation.
Good For
- Code-centric applications: Ideal for tasks requiring code completion and infilling.
- Research and development: Suitable for exploring fine-tuned Code Llama variants with mixed instruction datasets.
- General text generation: Can be applied to a range of natural language processing tasks due to its balanced performance on general benchmarks.