Nondzu/Mistral-7B-codealpaca-lora
Nondzu/Mistral-7B-codealpaca-lora is a 7 billion parameter language model, fine-tuned from Mistral-7B-Instruct-v0.1, specifically optimized for code generation tasks. This variant leverages the Alpaca prompt template and was trained using the theblackcat102/evol-codealpaca-v1 dataset. It demonstrates improved performance on code-related benchmarks like HumanEval+ compared to its base model, making it suitable as a coding companion.
Loading preview...
Overview
Nondzu/Mistral-7B-codealpaca-lora is a 7 billion parameter model, fine-tuned from mistralai/Mistral-7B-Instruct-v0.1, designed to excel as a coding assistant. It was trained for 118 hours on 3xRTX 3090 GPUs using the theblackcat102/evol-codealpaca-v1 dataset and the Alpaca prompt template.
Key Capabilities & Performance
This model is specifically optimized for code generation. Its performance on the HumanEval+ benchmark shows a pass@1 score of 0.4756, which is notably higher than the base mistralai/Mistral-7B-Instruct-v0.1 model's 0.2926. This indicates its enhanced ability to generate correct code solutions.
Training Details
The fine-tuning process utilized Axolotl and involved a LoRA adapter with lora_r: 32 and lora_alpha: 16. The model was configured with a sequence_len of 4096 and sample_packing enabled, targeting efficient training for code-related tasks.
Good For
- Code generation: Excels at producing functional code snippets and solutions.
- Developer assistance: Can serve as a valuable coding companion for various programming tasks.
- Experimentation: The creator welcomes contributions from testers to further evaluate and improve its performance.