1B Parameter LLMs — Page 98

Llama_3.2_1B_Instruct

2_layer_MQA_llama_model

torchtune_1B_lr1.5e-5_6epoch_full_finetuned_llama3.2_millfield_241227_meta_before_user_15epoch

gemma-3-1b-pt-MED-Instruct

gemma_3_1b_it_kn_pt_prl_pt

ultrafeedback_binarized-alpaca-llama-3-1b-2-epochs-alpha-0.6-beta-0-2-epochs

Llama-3.2-1B-Instruct-Chat-sft

ultrafeedback_binarized-alpaca-llama-3-1b-2-epochs-alpha-0.4-beta-0.2-2-epochs

Qwen2.5-1.5B-Instruct-Gensyn-Swarm-placid_timid_dog

ElderVBot

BLUECOMPUTER.2

Qwen2.5-Math-1.5B-Scoring-Mean

SmolLM3-DPO-Second-Round

Qwen2.5-Coder-1.5B-Instruct-Gensyn-Swarm-quick_tawny_tarantula

merge_linear_len0.9fmt0.1_MRL4096_ROLLOUT4_LR1e-6

merge_linear_cos0.1fmt0.9_MRL4096_ROLLOUT4_LR1e-6

merge_linear_cos0.5fmt0.5_MRL4096_ROLLOUT4_LR1e-6

dpo-llama3.2-sapo-200

expert_cos_MRL4096_ROLLOUT4_LR5e-7_step54

merge_accfmt_MRL4096_ROLLOUT4_LR5e-7_w0.5_linear

merge_cosfmt_MRL4096_ROLLOUT4_LR5e-7_w0.1_linear

merge_accfmt_MRL4096_ROLLOUT4_LR2e-6_w0.5_linear

merge_accfmt_MRL4096_ROLLOUT4_LR2e-6_w0.3_linear

merge_lenfmt_MRL4096_ROLLOUT4_LR2e-6_w0.7_linear

llama-3.2-1b-math-solver

merge_cosfmt_MRL4096_ROLLOUT4_LR2e-6_w0.5_ties

merge_cosfmt_MRL4096_ROLLOUT4_LR5e-7_w0.5_ties

merge_cosfmt_MRL4096_ROLLOUT4_LR5e-7_w0.5_dare_ties

merge_lenfmt_MRL4096_ROLLOUT4_LR5e-7_w0.5_ties

merge_accfmt_MRL4096_ROLLOUT4_LR1e-6_w0.5_ties

merge_cosfmt_MRL4096_ROLLOUT4_LR5e-7_w0.5_dare_ties_density0.2

dpo-llama3.2-gspo-original-400

Insta-Qwen2.5-1.5B-SFT

LAPO-I

Qwen2.5-1.5B-Instruct-Medical-cpt-sft-v1

ds-svd-muon-adam-1e-6-global_step_60

rlvr_llama1_bleu_alma_rbz_128_ckpt_10_of_10

me-qwen2.5-1.5B-sft

qwen15_code200tok_lr1e-05_cosine_ce1_0_num_ep2

pdcd200_cptq15_ce003_pr05_ptq25-15b_omi_c100k_200tok_s8_ckpt_7_of_10_it387

pdcd200_cptq15_ce01_pr05_ptq25-15b_omi_c100k_200tok_s8_ckpt_2_of_10_it26

unsup-Llama-3.2-1B-Instruct-lora