WesleySantos/depression-llama-2-7b
The WesleySantos/depression-llama-2-7b model is a Llama 2-based language model fine-tuned using 8-bit quantization with bitsandbytes. This model is specifically adapted for tasks related to depression, leveraging its base architecture for specialized applications. Its training procedure involved PEFT 0.6.0.dev0, focusing on efficient fine-tuning.
Loading preview...
Overview
The WesleySantos/depression-llama-2-7b model is a specialized language model built upon the Llama 2 architecture. It has undergone fine-tuning using an 8-bit quantization method via bitsandbytes, which optimizes memory usage during training and inference. The specific quantization configuration included load_in_8bit: True and bnb_4bit_quant_type: fp4, indicating a focus on efficient deployment and operation.
Key Training Details
- Quantization Method:
bitsandbyteswas utilized for efficient model training. - Quantization Type: Employed 8-bit loading (
load_in_8bit: True) withfp44-bit quantization for weights. - Framework: Training was conducted using PEFT version
0.6.0.dev0.
Potential Use Cases
This model is likely intended for applications requiring a Llama 2-based model with reduced memory footprint, particularly in contexts related to depression, given its naming convention. Its fine-tuned nature suggests adaptation for specific language patterns or information relevant to this domain.