JetBrains/CodeLlama-7B-KStack-clean

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7BQuant:FP8Context Size:4kPublished:May 13, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

JetBrains/CodeLlama-7B-KStack-clean is a 7 billion parameter CodeLlama model fine-tuned by JetBrains on the KStack-clean dataset, a collection of permissively licensed Kotlin code. This model is specifically optimized for generating Kotlin code, demonstrating a significant improvement in Kotlin HumanEval pass rate compared to the base CodeLlama-7B. It excels at code generation tasks, particularly for Kotlin programming, and supports Fill-in-the-Middle (FIM) functionality.

Loading preview...

JetBrains/CodeLlama-7B-KStack-clean Overview

This model is a 7 billion parameter variant of the CodeLlama architecture, developed by JetBrains. It has been specifically fine-tuned on the KStack-clean dataset, which is a filtered subset of the larger KStack dataset containing high-educational-value Kotlin code. The fine-tuning process aimed to enhance the model's proficiency in generating Kotlin code.

Key Capabilities and Performance

  • Kotlin Code Generation: The model is specialized for generating Kotlin code, leveraging its training on a dedicated Kotlin codebase.
  • Improved HumanEval Performance: On the Kotlin HumanEval dataset, the CodeLlama-7B-KStack-clean model achieved a pass rate of 37.89%, significantly outperforming the base CodeLlama-7B model's 26.89%.
  • Fill-in-the-Middle (FIM) Support: Like the base CodeLlama model, it supports FIM functionality, allowing for code completion within existing code structures.
  • Training Details: Fine-tuned on one A100 GPU using 25K examples from KStack-clean, totaling approximately 23M tokens, over 2 epochs.

Use Cases and Considerations

This model is particularly well-suited for developers working with Kotlin, offering enhanced code generation capabilities for this language. Its specialization makes it a strong candidate for tasks requiring Kotlin code completion, generation, or assistance in learning Kotlin algorithms. Developers should be aware that while optimized for Kotlin, the model's outputs, like all LLMs, may require safety testing and validation for specific applications. Deviation from the fine-tuned Kotlin task format may lead to less accurate responses.

Finetunes
12 models