pepe4235/second_try
pepe4235/second_try is a 13 billion parameter language model. This model was trained using bitsandbytes 4-bit quantization, specifically with the nf4 quantization type and float16 compute dtype. It leverages PEFT 0.6.0.dev0 for efficient fine-tuning. This model is suitable for tasks requiring a moderately sized language model with efficient quantization for deployment.
Loading preview...
Overview
pepe4235/second_try is a 13 billion parameter language model. The training process for this model utilized bitsandbytes 4-bit quantization, specifically configured with bnb_4bit_quant_type: nf4 and bnb_4bit_compute_dtype: float16. This approach aims to optimize memory usage during training and potentially for inference.
Key Training Details
- Quantization Method:
bitsandbytes4-bit quantization - Quantization Type:
nf4(NormalFloat 4-bit) - Compute Data Type:
float16 - PEFT Version:
0.6.0.dev0
Potential Use Cases
This model is likely suitable for applications where a 13B parameter model is desired, and efficient memory usage through 4-bit quantization is a priority. It can be used for various natural language processing tasks, benefiting from the reduced memory footprint during deployment or fine-tuning on resource-constrained environments.