speechlessai/speechless-coding-7b-16k-tora
The speechlessai/speechless-coding-7b-16k-tora model is a 7 billion parameter language model fine-tuned from llm_agents/tora-code-7b-v1.0, specifically designed to enhance reasoning and planning abilities for coding tasks. It features an extended context window of 16,384 tokens, making it suitable for handling larger codebases and complex problem descriptions. This model excels in code generation and understanding, demonstrating competitive performance on benchmarks like HumanEval and MultiPL-E across various programming languages.
Loading preview...
speechless-coding-7b-16k-tora Overview
The speechless-coding-7b-16k-tora is a 7 billion parameter model, fine-tuned from llm_agents/tora-code-7b-v1.0, with a significantly expanded context window of 16,384 tokens. Its primary objective is to improve reasoning and planning capabilities, particularly for coding-related tasks.
Key Capabilities & Training
This model was trained on a diverse dataset of 177,333 samples, including filtered categories from jondurbin/airoboros-2.2 (coding, reasoning, planning), Open-Orca/OpenOrca (COT category from 1M GPT4), garage-bAInd/Open-Platypus, WizardLM/WizardLM_evol_instruct_V2_196k (coding conversations), and TokenBender/python_eval_instruct_51k (Python outputs). This comprehensive training regimen aims to bolster its performance in complex coding scenarios.
Performance Highlights
The model demonstrates strong performance in code generation and problem-solving:
- HumanEval-Python: Achieves a score of 52.44, placing it competitively with larger models like CodeLlama-34B-Python (53.29) and surpassing CodeLlama-34B-Instruct (50.79).
- MultiPL-E: Shows proficiency across multiple programming languages, with scores including Python (55.96), JavaScript (46.93), and TypeScript (47.80).
Good For
- Code Generation: Excels at generating code in various languages, particularly Python.
- Reasoning and Planning: Designed to handle complex coding problems requiring logical thought and multi-step planning.
- Large Context Tasks: Its 16,384-token context window is ideal for processing and generating longer code snippets or detailed problem descriptions.