lonestar108/plenumvideos
lonestar108/plenumvideos is a 7 billion parameter language model with a 4096 token context length. This model was trained using specific bitsandbytes quantization configurations, including 4-bit quantization with nf4 type and float16 compute dtype. Its training procedure indicates a focus on efficient deployment and fine-tuning, making it suitable for applications requiring optimized resource usage.
Loading preview...
Model Overview
lonestar108/plenumvideos is a 7 billion parameter language model designed with a 4096 token context length. Its training process leveraged specific bitsandbytes quantization configurations, indicating an emphasis on efficient resource utilization and deployment.
Key Training Details
The model was trained using the following bitsandbytes quantization settings:
- 4-bit Quantization: Enabled (
load_in_4bit: True) using thenf4quantization type. - Compute Data Type:
float16was used for computations (bnb_4bit_compute_dtype: float16). - Double Quantization: Not utilized (
bnb_4bit_use_double_quant: False). - 8-bit Quantization: Not directly loaded in 8-bit, but includes an
llm_int8_thresholdof 6.0.
These configurations suggest that the model is optimized for environments where memory and computational efficiency are critical, potentially making it a good candidate for fine-tuning on consumer-grade hardware or for deployment in resource-constrained settings.
Framework Versions
The training procedure utilized PEFT version 0.4.0, indicating that the model likely benefits from parameter-efficient fine-tuning techniques.