OpenAssistant/llama2-13b-megacode2-oasst
OpenAssistant/llama2-13b-megacode2-oasst is a 13 billion parameter language model fine-tuned by andreaskoepf using the epfLLM/Megatron-LLM trainer. This model is specifically optimized for code-related tasks, leveraging the LosslessMegaCodeTrainingV2_1m_Evol_Uncensored dataset. It utilizes the Llama 2 architecture and is designed for applications requiring robust code generation and understanding within a 4096-token context window.
Loading preview...
Model Overview
OpenAssistant/llama2-13b-megacode2-oasst is a 13 billion parameter language model built on the Llama 2 architecture. It was fine-tuned by andreaskoepf using the open-source epfLLM/Megatron-LLM trainer, with compute sponsored by the eplf Machine Learning and Optimization Laboratory.
Key Characteristics
- Code-Optimized Training: The model was fine-tuned using the rombodawg/LosslessMegaCodeTrainingV2_1m_Evol_Uncensored dataset, indicating a strong focus on code-related tasks.
- Prompt Format: It uses the ChatML format for prompts, structured with
<|im_start|>userand<|im_start|>assistanttags. - Context Window: Supports a context length of 4096 tokens.
Intended Use Cases
This model is particularly well-suited for applications requiring advanced code generation, completion, and understanding due to its specialized training on a large code-centric dataset. Developers looking for a Llama 2-based model with enhanced coding capabilities may find this model beneficial.