Model Overview

lonestar108/plenumvideos is a 7 billion parameter language model designed with a 4096 token context length. Its training process leveraged specific bitsandbytes quantization configurations, indicating an emphasis on efficient resource utilization and deployment.

Key Training Details

The model was trained using the following bitsandbytes quantization settings:

• 4-bit Quantization: Enabled (load_in_4bit: True) using the nf4 quantization type.
• Compute Data Type: float16 was used for computations (bnb_4bit_compute_dtype: float16).
• Double Quantization: Not utilized (bnb_4bit_use_double_quant: False).
• 8-bit Quantization: Not directly loaded in 8-bit, but includes an llm_int8_threshold of 6.0.

These configurations suggest that the model is optimized for environments where memory and computational efficiency are critical, potentially making it a good candidate for fine-tuning on consumer-grade hardware or for deployment in resource-constrained settings.

Framework Versions

The training procedure utilized PEFT version 0.4.0, indicating that the model likely benefits from parameter-efficient fine-tuning techniques.