Overview

Nondzu/Mistral-7B-codealpaca-lora is a 7 billion parameter model, fine-tuned from mistralai/Mistral-7B-Instruct-v0.1, designed to excel as a coding assistant. It was trained for 118 hours on 3xRTX 3090 GPUs using the theblackcat102/evol-codealpaca-v1 dataset and the Alpaca prompt template.

Key Capabilities & Performance

This model is specifically optimized for code generation. Its performance on the HumanEval+ benchmark shows a pass@1 score of 0.4756, which is notably higher than the base mistralai/Mistral-7B-Instruct-v0.1 model's 0.2926. This indicates its enhanced ability to generate correct code solutions.

Training Details

The fine-tuning process utilized Axolotl and involved a LoRA adapter with lora_r: 32 and lora_alpha: 16. The model was configured with a sequence_len of 4096 and sample_packing enabled, targeting efficient training for code-related tasks.

Good For

• Code generation: Excels at producing functional code snippets and solutions.
• Developer assistance: Can serve as a valuable coding companion for various programming tasks.
• Experimentation: The creator welcomes contributions from testers to further evaluate and improve its performance.