barty/CodeLlama-13B-MORepair
barty/CodeLlama-13B-MORepair is a 13 billion parameter program repair model fine-tuned from CodeLlama-13b-instruct. It utilizes a novel multi-objective fine-tuning framework called MOREPAIR to learn both code transformations and repair logic reasoning. This model is specifically designed for automated program repair, primarily tested on C++ and Java, and aims to improve the accuracy and explainability of code fixes. Its training incorporates multi-objective learning for generating repaired code and explanatory guidance.
Loading preview...
Model Overview
barty/CodeLlama-13B-MORepair is a 13 billion parameter model derived from CodeLlama-13b-instruct, specifically engineered for automated program repair. Developed by barty, this model employs a unique multi-objective fine-tuning framework named MOREPAIR. This framework enables the model to not only generate corrected code but also to provide explanatory guidance for the repairs, enhancing understanding of the underlying logic.
Key Capabilities
- Automated Program Repair: Specializes in identifying and fixing bugs in code.
- Multi-Objective Learning: Trained to generate both repaired code and accompanying explanatory guidance.
- Base Model: Built upon the robust CodeLlama-13b-instruct architecture.
- Training Efficiency: Utilizes QLoRA fine-tuning, modifying only 1.84% of parameters, and NEFTune for improved generalization.
- Language Support: Primarily tested with C++ and Java, with potential for generalization to other programming languages.
Training Details
The model was fine-tuned using the MOREPAIR framework on the TutorLLMCode dataset, comprising 1,535 pairs of buggy and repaired code. This dataset includes LLM-generated repair guidance, which is crucial for the model's ability to provide explanations. The training approach focuses on multi-objective learning, aiming to produce both functional code repairs and logical explanations for those repairs.
Use Cases
This model is ideal for developers and researchers focused on:
- Automating the debugging process.
- Generating code fixes with accompanying explanations.
- Enhancing software development workflows by reducing manual repair efforts.
Limitations
Users should be aware that performance can vary across different programming languages, and the model may require multiple attempts to generate correct fixes. It is recommended to validate repairs with appropriate test cases, and the model may not effectively handle highly complex or multi-file repair scenarios.