Jeesup/svd-safety-l3_swift_remove20

TEXT GENERATIONPricing:Input $0.37 / Cached $0.074 / Output $0.38Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:8kTool Calling:SupportedPublished:Sep 17, 2026License:llama3Architecture:Transformer Featherless Exclusive Cold

Jeesup/svd-safety-l3_swift_remove20 is an 8 billion parameter Llama-3-8B-Instruct checkpoint compressed with Swift-SVD to 80% of its original parameters, then recovered with SVD-LLM's stage-2 LoRA. This model is a research artifact from a study on how SVD compression impacts safety behavior and its recovery, not a general-purpose chat model. It is specifically designed to measure safety/utility trade-offs under compression, with some arms deliberately safety-degraded.

Loading preview...

Overview

Jeesup/svd-safety-l3_swift_remove20 is an 8 billion parameter Llama-3-8B-Instruct model that has undergone significant compression and recovery. It was created by Jeesup using Swift-SVD for compression, reducing its parameters to 80% of the original, followed by recovery using SVD-LLM's stage-2 LoRA. This model is primarily a research artifact intended for studying the effects of SVD compression on safety behavior and the effectiveness of recovery methods.

Key Characteristics

  • Base Model: meta-llama/Meta-Llama-3-8B-Instruct
  • Compression Method: Swift-SVD (dynamic rank allocation, alpha 0.6, 256 x 2048 WikiText2 calibration), resulting in 20.00% parameter removal.
  • Parameter Fraction: Retains 80.04% of original parameters.
  • Recovery Method: SVD-LLM's stage-2 LoRA (sequential U then V, alpaca-cleaned, r=8, alpha=16, 2 epochs per half, lr 0.0001, batch 64, cutoff 256).

Measured Performance (Research Context)

  • AdvBench ASR (HarmBench judge): 0.0923
  • StrongREJECT ASR (HarmBench judge): 0.1438
  • Macro over-refusal (WildGuard): 0.1519
  • WikiText-2 perplexity: 14.5046

Intended Use and Limitations

This model is not a general-purpose chat model and is not intended for deployment as an assistant. It serves as an experimental subject within a research grid to quantify how compression damages safety and to test recovery mechanisms. Some versions, including this one, are deliberately safety-degraded relative to the base Llama-3-8B-Instruct model to facilitate this study. Users should evaluate it themselves before drawing conclusions.

Finetunes
3 models