Model Overview

The speechless-codellama-airoboros-orca-platypus-13b is a 13 billion parameter language model built upon the CodeLlama-13B base model. Its primary goal is to significantly improve the model's reasoning and planning capabilities through a targeted fine-tuning process.

Key Fine-tuning Datasets

This model was fine-tuned using a curated combination of datasets:

• jondurbin/airoboros-2.2: Specifically filtered for categories related to coding, reasoning, and planning.
• Open-Orca/OpenOrca: Utilizes the 'cot' (chain-of-thought) category from its 1M GPT-4 dataset.
• garage-bAInd/Open-Platypus: Incorporated entirely to bolster its instructional and reasoning prowess.

Performance Highlights

On the HumanEval-Python benchmark, this model achieves a score of 49.39. This places it competitively, outperforming the base CodeLlama-13B (35.07) and CodeLlama-13B-Python (42.89), and nearing the performance of CodeLlama-13B-Instruct (50.6).

General LLM benchmarks show an average score of 49.15, with specific scores including ARC (44.88), HellaSwag (67.7), MMLU (43.16), and TruthfulQA (40.88).

Intended Use Cases

This model is particularly well-suited for:

• Code completion: Generating relevant code snippets.
• Infilling: Completing missing parts within existing code.

It is based on the Code Llama architecture, an auto-regressive language model using an optimized transformer architecture, developed by Meta.