augtoma/qCammel-70-x
qCammel-70-x is a 69 billion parameter language model developed by augtoma, fine-tuned from the Llama 2 70B architecture. This model is specifically optimized for academic medical knowledge and enhanced instruction-following capabilities. It leverages QLoRA for efficient fine-tuning on a distilled dataset of 15,000 instructions, making it suitable for specialized medical text generation and analysis tasks.
Loading preview...
qCammel-70-x: Specialized Medical LLM
qCammel-70-x is a 69 billion parameter language model, fine-tuned by augtoma from the Llama 2 70B architecture. This model is specifically designed for applications requiring deep academic medical knowledge and robust instruction-following. It was developed using QLoRA for efficient fine-tuning on a distilled dataset comprising 15,000 instructions.
Key Capabilities
- Academic Medical Knowledge: Optimized for understanding and generating content related to academic medicine.
- Instruction Following: Enhanced ability to follow complex instructions, crucial for specialized tasks.
- Llama 2 Architecture: Based on the decoder-only transformer architecture of Llama 2.
Training & Architecture
The model's fine-tuning process utilized QLoRA, a method for efficient fine-tuning of quantized LLMs, as detailed in the QLoRA research paper. The base Llama 2 model architecture is described in the LLaMA research paper. Its development also draws inspiration from work like Clinical Camel, focusing on expert-level medical language modeling.
Licensing
Use of this model is governed by the Meta license. Users must accept the license terms from Meta Platforms, Inc. to download the model weights and tokenizer.