Mel-Iza0/Llama2-13B_ZeroShot-20K_classe_bias_port
Mel-Iza0/Llama2-13B_ZeroShot-20K_classe_bias_port is a 13 billion parameter Llama 2-based language model, fine-tuned using 4-bit quantization with bfloat16 compute dtype. This model was trained with PEFT 0.4.0, focusing on specific zero-shot classification tasks related to bias. Its primary differentiation lies in its specialized training for analyzing and addressing class bias within a 4096-token context.
Loading preview...
Model Overview
Mel-Iza0/Llama2-13B_ZeroShot-20K_classe_bias_port is a specialized 13 billion parameter language model built upon the Llama 2 architecture. It has been fine-tuned with a focus on zero-shot classification, particularly concerning class bias, utilizing a 4096-token context window.
Training Details
The model's training procedure involved 4-bit quantization using bitsandbytes, specifically with nf4 quantization type and bfloat16 compute dtype. Double quantization was also employed. The training leveraged PEFT version 0.4.0.
Key Characteristics
- Architecture: Llama 2-based, 13 billion parameters.
- Quantization: Trained with
bitsandbytes4-bit quantization (nf4,bfloat16compute dtype, double quantization). - Context Length: Supports a 4096-token context.
- Specialization: Fine-tuned for zero-shot classification tasks, with an emphasis on analyzing and mitigating class bias.
Potential Use Cases
This model is particularly suited for applications requiring:
- Zero-shot classification where specific class bias detection or analysis is critical.
- Research into model fairness and bias within language models.
- Tasks benefiting from a Llama 2-13B base with specialized fine-tuning for nuanced classification.