stabilityai/stable-code-3b
Stable Code 3B is a 2.7 billion parameter decoder-only language model developed by Stability AI, pre-trained on 1.3 trillion tokens of diverse textual and code datasets. It is specifically optimized for code generation across 18 programming languages, demonstrating strong performance on MultiPL-E metrics compared to other models of similar size. The model supports a long context of up to 16,384 tokens and includes Fill-in-the-Middle (FIM) capabilities, making it suitable for various code-related tasks.
Loading preview...
Stable Code 3B Overview
Stable Code 3B is a 2.7 billion parameter decoder-only transformer model developed by Stability AI, designed for code generation. It was pre-trained on a massive 1.3 trillion tokens, encompassing both diverse textual and code datasets, with a focus on 18 popular programming languages identified from the 2023 StackOverflow Developer Survey.
Key Capabilities
- Multi-language Code Generation: Excels across 18 programming languages, including Python, C++, JavaScript, Java, PHP, and Rust, showing competitive performance against larger models.
- Fill-in-the-Middle (FIM): Supports FIM capabilities, allowing for code completion and insertion within existing code structures.
- Long Context Window: Trained with a substantial context length of up to 16,384 tokens, enabling it to handle larger codebases and complex programming tasks.
- Efficient Architecture: Based on the LLaMA architecture with modifications like Rotary Position Embeddings and optimized with Flash Attention 2 for improved throughput.
Use Cases
Stable Code 3B is intended as a foundational base model for application-specific fine-tuning in code-related domains. It is particularly well-suited for tasks such as:
- Code completion and generation
- Code refactoring and bug fixing
- Educational tools for programming
- Developer assistance in integrated development environments (IDEs)