speechless-coding-7b-16k-tora Overview

The speechless-coding-7b-16k-tora is a 7 billion parameter model, fine-tuned from llm_agents/tora-code-7b-v1.0, with a significantly expanded context window of 16,384 tokens. Its primary objective is to improve reasoning and planning capabilities, particularly for coding-related tasks.

Key Capabilities & Training

This model was trained on a diverse dataset of 177,333 samples, including filtered categories from jondurbin/airoboros-2.2 (coding, reasoning, planning), Open-Orca/OpenOrca (COT category from 1M GPT4), garage-bAInd/Open-Platypus, WizardLM/WizardLM_evol_instruct_V2_196k (coding conversations), and TokenBender/python_eval_instruct_51k (Python outputs). This comprehensive training regimen aims to bolster its performance in complex coding scenarios.

Performance Highlights

The model demonstrates strong performance in code generation and problem-solving:

• HumanEval-Python: Achieves a score of 52.44, placing it competitively with larger models like CodeLlama-34B-Python (53.29) and surpassing CodeLlama-34B-Instruct (50.79).
• MultiPL-E: Shows proficiency across multiple programming languages, with scores including Python (55.96), JavaScript (46.93), and TypeScript (47.80).

Good For

• Code Generation: Excels at generating code in various languages, particularly Python.
• Reasoning and Planning: Designed to handle complex coding problems requiring logical thought and multi-step planning.
• Large Context Tasks: Its 16,384-token context window is ideal for processing and generating longer code snippets or detailed problem descriptions.