nandansarkar/qwen3_0-6B_adversarial_final

Model Overview

The nandansarkar/qwen3_0-6B_adversarial_final is a fine-tuned language model based on the Qwen3.0-6B architecture, featuring 0.8 billion parameters. This specific iteration, qwen3_0-6B_adversarial_8, is a direct successor to qwen3_0-6B_adversarial_7, having undergone further training on an adversarial_dataset_8.

Key Characteristics

• Base Model: Qwen3.0-6B architecture.
• Parameter Count: 0.8 billion parameters.
• Context Length: Supports a context length of 40960 tokens.
• Training Focus: Fine-tuned on an adversarial dataset, suggesting an emphasis on improving model robustness and resilience to adversarial attacks or challenging inputs.

Training Details

The model was trained using the following hyperparameters:

• Learning Rate: 1e-05
• Batch Sizes: train_batch_size of 2, eval_batch_size of 8.
• Gradient Accumulation: 8 steps, leading to a total_train_batch_size of 32.
• Optimizer: AdamW with betas=(0.9, 0.95) and epsilon=1e-08.
• Scheduler: Cosine learning rate scheduler with a warmup ratio of 0.05.
• Epochs: Trained for 1 epoch.

Potential Use Cases

Given its adversarial training, this model is likely suitable for applications where a robust language model is critical, such as:

• Content Moderation: Identifying and handling deceptive or malicious text.
• Security Applications: Analyzing and responding to adversarial prompts.
• Robust AI Systems: Deploying in environments where input quality cannot be guaranteed.