ssadqdsacf/cross-unlearning-case6-qwen35-4b-kl-min-epoch10

Model Overview

The ssadqdsacf/cross-unlearning-case6-qwen35-4b-kl-min-epoch10 is a 4.5 billion parameter model built upon the Qwen3.5 architecture. It represents a kl_min unlearning baseline, initialized from a specific epoch-10 SFT (Supervised Fine-Tuning) model. This model is part of a formal matrix of six models, indicating its role in a structured experimental or development context.

Key Characteristics

• Unlearning Baseline: Specifically designed as a kl_min unlearning baseline, suggesting its purpose in evaluating or performing model unlearning tasks.
• Architecture: Based on the Qwen3.5 architecture, providing a foundation for its language processing capabilities.
• Initialization: The model's state is derived from an SFT model at epoch 10, with a cosine scheduler retaining a 15-epoch horizon.
• Components: The repository provides both a directly loadable merged model and its corresponding LoRA adapter, offering flexibility for deployment and further fine-tuning.
• Evaluation: Evaluation was conducted using fixed 4-shot greedy decoding, with specific methodologies for classification (native thinking) and mask/generation tasks (thinking disabled).

Use Cases

This model is particularly suited for research and development in:

• Model Unlearning: Investigating and implementing techniques for removing specific information or behaviors from pre-trained language models.
• Experimental Setups: As one of six formal matrix models, it is ideal for comparative studies and structured evaluations within a research framework.
• Fine-tuning with LoRA: The included LoRA adapter allows for efficient adaptation to new tasks or datasets with minimal computational overhead.