[

](https://github.com/axolotl-ai-cloud/axolotl)

See axolotl config

axolotl version: `0.9.2` ```yaml # Name 0619-sft_vanilla_no_sexism_wc_multi_attrs-qwen2.5_7b_instruct # CUDA_VISIBLE_DEVICES=0,1,2,3 axolotl train red_team_agent/run/t0619/sft_vanilla_no_sexism_wc_multi_attrs-qwen2.5_7b_instruct.yaml base_model: Qwen/Qwen2.5-7B-Instruct model_type: AutoModelForCausalLM tokenizer_type: AutoTokenizer trust_remote_code: false # --- Dataset Configuration --- datasets: - path: nate-rahn/0619-wc_multi_attrs_no_sexism_vanilla_sft_dset type: chat_template # Use the chat_template processing strategy # --- Custom Template & Role Mapping --- chat_template: tokenizer_default # Specify we are using a custom jinja template below field_messages: messages # Assumes your dataset has a "bad_messages" key with a list of dicts message_property_mappings: # Assumes each dict in the list has "role" and "content" keys role: role content: content roles: # Define the roles expected in your dataset for mapping user: ["user"] # Map "user" role in data to internal "user" assistant: ["assistant"] # Map "assistant" role in data to internal "assistant" system: ["system"] # Map "system" role in data to internal "system" # --- Training Target --- roles_to_train: ["assistant"] train_on_eos: turn # Train on the EOS token at the end of each 'user' turn dataset_prepared_path: /home/ubuntu/data/red-team-agent/last_run_prepared # --- Training Hyperparameters --- sequence_len: 2048 # Adjust based on your dataset and GPU memory sample_packing: true # Pack multiple sequences into one example for efficiency eval_sample_packing: false pad_to_sequence_len: true # Pad sequences to sequence_len # Full Parameter Finetuning (No adapter specified) # adapter: # This is intentionally left blank/removed for full finetuning # Performance & Precision (H100s excel with bf16) bf16: true tf32: true flash_attention: true # for qwen # Batching (Adjust based on GPU memory) # Effective global batch size = micro_batch_size * gradient_accumulation_steps * num_gpus (4) # Start low for full finetuning, e.g., 1 * 16 * 4 = 64 micro_batch_size: 2 gradient_accumulation_steps: 32 eval_batch_size: 16 # Can often be slightly higher than micro_batch_size # Optimizer & Scheduler optimizer: adamw_torch_fused # Good choice for newer GPUs learning_rate: 1e-5 # Common starting point for full SFT weight_decay: 0.01 lr_scheduler: cosine # Standard scheduler warmup_steps: 50 max_grad_norm: 1.0 # Training Duration & Evaluation/Saving num_epochs: 2 # Adjust as needed, start with 1-3 for SFT val_set_size: 0.005 logging_steps: 1 evals_per_epoch: 5 saves_per_epoch: 1 # Save 4 times per epoch (adjust based on dataset size) save_total_limit: 1 # Keep only the last 1 checkpoints # Memory Saving gradient_checkpointing: true # Essential for full finetuning gradient_checkpointing_kwargs: use_reentrant: false # Prefer non-reentrant if possible # --- FSDP Configuration (for 4xH100) --- fsdp: - full_shard - auto_wrap fsdp_config: fsdp_offload_params: false # Should not be needed with H100 VRAM fsdp_sync_module_states: true # Important for correctness fsdp_use_orig_params: false # Recommended for memory saving with FSDP fsdp_state_dict_type: SHARDED_STATE_DICT # Options: FULL_STATE_DICT or SHARDED_STATE_DICT (saves disk space) # fsdp_transformer_layer_cls_to_wrap: 'Gemma3DecoderLayer' fsdp_transformer_layer_cls_to_wrap: 'Qwen2DecoderLayer' # fsdp_activation_checkpointing: true # Alternative way to enable activation checkpointing for FSDP # --- Special Tokens --- # Define based on your custom template's terminators. Qwen already uses <|im_end|> special_tokens: eos_token: "<|im_end|>" # eos_token: "" # --- Logging & Saving --- output_dir: /home/ubuntu/out/red-team-agent/runs/0619-sft_vanilla_no_sexism_wc_multi_attrs-qwen2.5_7b_instruct # Local output directory # W&B Logging wandb_project: "red-team-agent" # Name your W&B project wandb_entity: "nate" # IMPORTANT: Replace with your W&B username or team name wandb_name: "0619-sft_vanilla_no_sexism_wc_multi_attrs-qwen2.5_7b_instruct-2_epochs" # Descriptive run name # wandb_log_model: "checkpoint" # Log model checkpoints to W&B Artifacts # Hugging Face Hub Upload hub_model_id: "nate-rahn/0619-sft_vanilla_no_sexism_wc_multi_attrs-qwen2.5_7b_instruct-2_epochs" # IMPORTANT: Replace with your desired HF repo ID hub_strategy: "end" # Push checkpoints to the Hub (`"end"` pushes only the final model) hf_use_auth_token: true # Required for pushing to the Hub (ensure you're logged in) # --- Misc --- seed: 42 ```

# 0619-sft_vanilla_no_sexism_wc_multi_attrs-qwen2.5_7b_instruct-2_epochs This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the nate-rahn/0619-wc_multi_attrs_no_sexism_vanilla_sft_dset dataset. It achieves the following results on the evaluation set: - Loss: 1.8132 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 2 - eval_batch_size: 16 - seed: 42 - distributed_type: multi-GPU - num_devices: 4 - gradient_accumulation_steps: 32 - total_train_batch_size: 256 - total_eval_batch_size: 64 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 50 - num_epochs: 2.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:----:|:---------------:| | 5.728 | 0.0057 | 1 | 5.7736 | | 2.1164 | 0.1999 | 35 | 2.1033 | | 1.9537 | 0.3998 | 70 | 1.9509 | | 1.8866 | 0.5997 | 105 | 1.8920 | | 1.8686 | 0.7996 | 140 | 1.8603 | | 1.8333 | 0.9995 | 175 | 1.8411 | | 1.7582 | 1.1942 | 210 | 1.8274 | | 1.7095 | 1.3941 | 245 | 1.8207 | | 1.7408 | 1.5940 | 280 | 1.8163 | | 1.7105 | 1.7939 | 315 | 1.8132 | | 1.7243 | 1.9938 | 350 | 1.8132 | ### Framework versions - Transformers 4.51.3 - Pytorch 2.6.0+cu124 - Datasets 3.5.1 - Tokenizers 0.21.1