Model Overview

Steven10429/qwen14-2wc1p-eos-3-merge is a 14.8 billion parameter language model developed by Steven10429. It is a merge of a base model, Steven10429/qwen14b-2wc1p-pj3ha_qwen14b-generic-eos-2, and a LoRA model, Steven10429/qwen14-2wc1p-eos-3. This model supports various quantization methods including Q2_K, Q4_K, IQ4_NL, Q5_K_M, Q6_K, and Q8_0, making it adaptable for different deployment scenarios.

Key Training Iterations

This specific iteration of the model focused on several key improvements:

• EOS Activation: The End-Of-Sequence (EOS) token was explicitly enabled during training.
• Learning Rate Adjustment: The learning rate was reduced to refine the training process.
• Epochs: Training was conducted over 3 epochs.
• Generation Length Improvement: A primary goal of this iteration was to address and improve the model's ability to generate longer, more coherent outputs.

Future Development Focus

Future development plans include:

• Randomized EOS: Exploring a 0.3 probability for random EOS token appearance.
• Reduced EOS Probability: Further reducing the probability of EOS to fine-tune output control.

Use Cases

This model is suitable for applications where controlled generation length and specific EOS behavior are important. Its 14.8B parameters and 131,072 token context window make it capable of handling complex language tasks.