5456es/last_layer_prune_Llama-3.2-3B-Instruct_prune_0.7-sigmoid
The 5456es/last_layer_prune_Llama-3.2-3B-Instruct_prune_0.7-sigmoid is a 3.2 billion parameter language model, fine-tuned from Llama-3.2-3B-Instruct using Direct Preference Optimization (DPO) with a last-layer pruning method. This model is specifically optimized through DPO on preference data, aiming to align its outputs more closely with human preferences. It features a context length of 32768 tokens, making it suitable for tasks requiring nuanced understanding based on comparative feedback.
Loading preview...
Model Overview
This model, last_layer_prune_Llama-3.2-3B-Instruct_prune_0.7-sigmoid, is a 3.2 billion parameter language model developed by 5456es. It is a fine-tuned variant of the Llama-3.2-3B-Instruct base model, specifically enhanced using Direct Preference Optimization (DPO).
Key Characteristics
- Base Model: Llama-3.2-3B-Instruct
- Fine-tuning Method: Direct Preference Optimization (DPO) with a 'last' method.
- Pruning: Pruning was applied during the training process, though the exact ratio is unspecified.
- Training Data: Fine-tuned on preference data, which helps in aligning model responses with desired human feedback.
- Context Length: Supports a context window of 32768 tokens.
Use Cases and Considerations
This model is designed for applications where aligning with human preferences is crucial, benefiting from its DPO fine-tuning. Developers can leverage its instruction-following capabilities, enhanced by preference data. However, users should be aware that the model inherits the limitations of its base model and may have additional constraints due to the pruning process. It is suitable for tasks requiring a balance of performance and efficiency, given its 3.2 billion parameter size.