Overview

SVECTOR-CORPORATION/Spec-Coder-4b-V1 is an open-source AI model built on the Llama architecture, featuring 4 billion parameters. It is designed for fundamental coding tasks and is highly compatible with tools like llama.cpp and Ollama, allowing for flexible local and cloud deployment.

Key Capabilities

• Code Generation and Completion: Excels at generating and completing code snippets across various programming languages.
• Debugging Assistance: Can be used to assist with debugging tasks.
• Integration: Optimized for integration into developer tools to provide intelligent coding assistance.
• Fine-tuning Support: Supports supervised fine-tuning (SFT) and reinforcement learning (RL) for task-specific performance improvements.

Training and Performance

The model was trained on approximately 4.3 trillion tokens from diverse datasets including The Stack, StarCoder Training Dataset, and OpenCodeReasoning, over 140 days on 5 x RTX 4090 GPUs. It features an 8,192 token context window. Benchmarks include:

• RepoBench 1.1 (Python): Achieves an average of 34.59% across various context lengths.
• Syntax-Aware Fill-in-the-Middle (SAFIM): Scores 46.28% on average.
• HumanEval Infilling: Demonstrates 72.34% for single-line infilling.

Limitations

• May reflect biases present in public codebases.
• Generated code may contain security vulnerabilities and requires verification and auditing.