speechlessai/speechless-codellama-airoboros-orca-platypus-13b
The speechlessai/speechless-codellama-airoboros-orca-platypus-13b is a 13 billion parameter language model fine-tuned from CodeLlama-13B, designed to enhance reasoning and planning abilities. It leverages a blend of datasets including filtered categories from jondurbin/airoboros-2.2, the 'cot' category from Open-Orca/OpenOrca, and the full garage-bAInd/Open-Platypus dataset. This model excels in code completion and infilling tasks, demonstrating improved performance on benchmarks like HumanEval-Python with a score of 49.39.
Loading preview...
Model Overview
The speechless-codellama-airoboros-orca-platypus-13b is a 13 billion parameter language model built upon the CodeLlama-13B base model. Its primary goal is to significantly improve the model's reasoning and planning capabilities through a targeted fine-tuning process.
Key Fine-tuning Datasets
This model was fine-tuned using a curated combination of datasets:
- jondurbin/airoboros-2.2: Specifically filtered for categories related to coding, reasoning, and planning.
- Open-Orca/OpenOrca: Utilizes the 'cot' (chain-of-thought) category from its 1M GPT-4 dataset.
- garage-bAInd/Open-Platypus: Incorporated entirely to bolster its instructional and reasoning prowess.
Performance Highlights
On the HumanEval-Python benchmark, this model achieves a score of 49.39. This places it competitively, outperforming the base CodeLlama-13B (35.07) and CodeLlama-13B-Python (42.89), and nearing the performance of CodeLlama-13B-Instruct (50.6).
General LLM benchmarks show an average score of 49.15, with specific scores including ARC (44.88), HellaSwag (67.7), MMLU (43.16), and TruthfulQA (40.88).
Intended Use Cases
This model is particularly well-suited for:
- Code completion: Generating relevant code snippets.
- Infilling: Completing missing parts within existing code.
It is based on the Code Llama architecture, an auto-regressive language model using an optimized transformer architecture, developed by Meta.