[

](https://github.com/axolotl-ai-cloud/axolotl)

See axolotl config

axolotl version: `0.9.2` ```yaml # Name 0620-sft_vanilla_all_principles_wc_multi_attrs-qwen2.5_7b_instruct # CUDA_VISIBLE_DEVICES=4,5,6,7 axolotl train red_team_agent/run/t0620/sft_vanilla_all_principles_wc_multi_attrs-qwen2.5_7b_instruct.yaml base_model: Qwen/Qwen2.5-7B-Instruct model_type: AutoModelForCausalLM tokenizer_type: AutoTokenizer trust_remote_code: false # --- Dataset Configuration --- datasets: - path: nate-rahn/0620-wc_multi_attrs_all_principles_vanilla_sft_dset type: chat_template # Use the chat_template processing strategy # --- Custom Template & Role Mapping --- chat_template: tokenizer_default # Specify we are using a custom jinja template below field_messages: messages # Assumes your dataset has a "bad_messages" key with a list of dicts message_property_mappings: # Assumes each dict in the list has "role" and "content" keys role: role content: content roles: # Define the roles expected in your dataset for mapping user: ["user"] # Map "user" role in data to internal "user" assistant: ["assistant"] # Map "assistant" role in data to internal "assistant" system: ["system"] # Map "system" role in data to internal "system" # --- Training Target --- roles_to_train: ["assistant"] train_on_eos: turn # Train on the EOS token at the end of each 'user' turn dataset_prepared_path: /home/ubuntu/data/red-team-agent/last_run_prepared # --- Training Hyperparameters --- sequence_len: 2048 # Adjust based on your dataset and GPU memory sample_packing: true # Pack multiple sequences into one example for efficiency eval_sample_packing: false pad_to_sequence_len: true # Pad sequences to sequence_len # Full Parameter Finetuning (No adapter specified) # adapter: # This is intentionally left blank/removed for full finetuning # Performance & Precision (H100s excel with bf16) bf16: true tf32: true flash_attention: true # for qwen # Batching (Adjust based on GPU memory) # Effective global batch size = micro_batch_size * gradient_accumulation_steps * num_gpus (4) # Start low for full finetuning, e.g., 1 * 16 * 4 = 64 micro_batch_size: 2 gradient_accumulation_steps: 32 eval_batch_size: 16 # Can often be slightly higher than micro_batch_size # Optimizer & Scheduler optimizer: adamw_torch_fused # Good choice for newer GPUs learning_rate: 1e-5 # Common starting point for full SFT weight_decay: 0.01 lr_scheduler: cosine # Standard scheduler warmup_steps: 50 max_grad_norm: 1.0 # Training Duration & Evaluation/Saving num_epochs: 2 # Adjust as needed, start with 1-3 for SFT val_set_size: 0.005 logging_steps: 1 evals_per_epoch: 5 saves_per_epoch: 1 # Save 4 times per epoch (adjust based on dataset size) save_total_limit: 1 # Keep only the last 1 checkpoints # Memory Saving gradient_checkpointing: true # Essential for full finetuning gradient_checkpointing_kwargs: use_reentrant: false # Prefer non-reentrant if possible # --- FSDP Configuration (for 4xH100) --- fsdp: - full_shard - auto_wrap fsdp_config: fsdp_offload_params: false # Should not be needed with H100 VRAM fsdp_sync_module_states: true # Important for correctness fsdp_use_orig_params: false # Recommended for memory saving with FSDP fsdp_state_dict_type: SHARDED_STATE_DICT # Options: FULL_STATE_DICT or SHARDED_STATE_DICT (saves disk space) # fsdp_transformer_layer_cls_to_wrap: 'Gemma3DecoderLayer' fsdp_transformer_layer_cls_to_wrap: 'Qwen2DecoderLayer' # fsdp_activation_checkpointing: true # Alternative way to enable activation checkpointing for FSDP # --- Special Tokens --- # Define based on your custom template's terminators. Qwen already uses <|im_end|> special_tokens: eos_token: "<|im_end|>" # eos_token: "" # --- Logging & Saving --- output_dir: /home/ubuntu/out/red-team-agent/runs/0620-sft_vanilla_all_principles_wc_multi_attrs-qwen2.5_7b_instruct # Local output directory # W&B Logging wandb_project: "red-team-agent" # Name your W&B project wandb_entity: "nate" # IMPORTANT: Replace with your W&B username or team name wandb_name: "0620-sft_vanilla_all_principles_wc_multi_attrs-qwen2.5_7b_instruct-2_epochs" # Descriptive run name # wandb_log_model: "checkpoint" # Log model checkpoints to W&B Artifacts # Hugging Face Hub Upload hub_model_id: "nate-rahn/0620-sft_vanilla_all_principles_wc_multi_attrs-qwen2.5_7b_instruct-2_epochs" # IMPORTANT: Replace with your desired HF repo ID hub_strategy: "end" # Push checkpoints to the Hub (`"end"` pushes only the final model) hf_use_auth_token: true # Required for pushing to the Hub (ensure you're logged in) # --- Misc --- seed: 42 ```

# 0620-sft_vanilla_all_principles_wc_multi_attrs-qwen2.5_7b_instruct-2_epochs This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the nate-rahn/0620-wc_multi_attrs_all_principles_vanilla_sft_dset dataset. It achieves the following results on the evaluation set: - Loss: 1.8125 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 2 - eval_batch_size: 16 - seed: 42 - distributed_type: multi-GPU - num_devices: 4 - gradient_accumulation_steps: 32 - total_train_batch_size: 256 - total_eval_batch_size: 64 - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 50 - num_epochs: 2.0 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:------:|:----:|:---------------:| | 5.7369 | 0.0057 | 1 | 5.7767 | | 2.0977 | 0.2040 | 36 | 2.0960 | | 1.9428 | 0.4081 | 72 | 1.9432 | | 1.8924 | 0.6121 | 108 | 1.8897 | | 1.8524 | 0.8162 | 144 | 1.8562 | | 1.8348 | 1.0227 | 180 | 1.8369 | | 1.7849 | 1.2267 | 216 | 1.8240 | | 1.7362 | 1.4307 | 252 | 1.8167 | | 1.6995 | 1.6348 | 288 | 1.8144 | | 1.7063 | 1.8388 | 324 | 1.8125 | ### Framework versions - Transformers 4.51.3 - Pytorch 2.6.0+cu124 - Datasets 3.5.1 - Tokenizers 0.21.1