layoric/llama-2-13b-code-alpaca
layoric/llama-2-13b-code-alpaca is a 13 billion parameter Llama-2 based causal language model fine-tuned by layoric using QLoRA. It was trained for 3 epochs on the `theblackcat102/evol-codealpaca-v1` dataset, achieving a perplexity of 4.36. This model is specifically optimized for code generation and understanding tasks, making it suitable for developers requiring a specialized coding assistant.
Loading preview...
layoric/llama-2-13b-code-alpaca Overview
This model is a 13 billion parameter Llama-2 variant, fine-tuned by layoric using the QLoRA method. It leverages the NousResearch/Llama-2-13b-hf as its base and was trained for 3 epochs on the theblackcat102/evol-codealpaca-v1 dataset, which is designed for evolving code-related instructions. The training process utilized a sequence length of 4096 tokens and incorporated 4-bit quantization (nf4 type) with double quantization and bfloat16 compute dtype for efficiency.
Key Capabilities
- Code Generation: Specialized training on a code-centric dataset enhances its ability to generate and understand code.
- Efficient Deployment: Trained with QLoRA and 4-bit quantization, making it suitable for deployment in resource-constrained environments.
- Llama-2 Foundation: Benefits from the robust architecture and general language understanding capabilities of the Llama-2 13B base model.
Good for
- Software Development: Assisting with code completion, generation, and debugging.
- Code-related Research: Exploring performance of Llama-2 models on programming tasks.
- Educational Tools: Creating interactive coding tutorials or automated code review systems.