We introduce an activation steering method for masked diffusion language models that extracts a low-dimensional direction from contrastive prompts and applies it during reverse diffusion at inference time.
We introduce a representation-level activation steering method for masked diffusion language models and characterize steering behavior across token position, denoising time, transformer depth, language, and model architecture.
Existing diffusion-language-model control work largely operates at the sampling level. We adapt activation steering to masked diffusion, intervening directly on residual-stream representations during reverse diffusion.
The steering direction is estimated from contrastive prompts using a single unmasked forward pass. Extraction does not simulate the denoising trajectory, and deployment requires no gradient optimization or sampler modification.
A direction selected from one layer–token source is reused as a global intervention across layers, token positions, and reverse-diffusion steps, producing large and systematic behavioral shifts.
Effective refusal directions can be extracted from structural tokens before the user instruction—positions that are normally causally uninformative in autoregressive models.
Ablations reveal a temporal and spatial asymmetry: leverage is strongest early in denoising, the earliest diffusion block matters disproportionately, and mid-to-late layers account for most of the effect.
Steering transfers strongly English ↔ Chinese within the same MDLM, but fails to produce a measurable effect when transferred to an autoregressive model, pointing to architecture-dependent safety representations.
Repeated guidance, auxiliary policies, or computation coupled to the denoising trajectory.
Extract once from prompt activations, then steer internal residual states during generation.
Using safety refusal as a case study, activation steering produces a larger and more consistent behavioral shift than the evaluated prompt- and optimization-based baselines.
A single activation direction is sufficient to strongly modulate refusal behavior.
Pre-instruction and post-instruction extraction yield similarly strong test-time effects.
Intervening early changes the trajectory much more than late-only steering.
The direction transfers across English and Chinese, but not to an autoregressive model.
Lower means fewer completions classified as safe.
| Model | Method | Refusal ↓ | Guard ↓ |
|---|---|---|---|
| LLaDA-8B | Direct | 97 | 99 |
| LLaDA-8B | Steering · pre | 4 | 19 |
| LLaDA-8B | Steering · post | 0 | 16 |
| LLaDA-1.5 | Direct | 98 | 100 |
| LLaDA-1.5 | Steering · pre | 6 | 20 |
| LLaDA-1.5 | Steering · post | 7 | 19 |
| MMaDA-8B | Direct | 26 | 50 |
| MMaDA-8B | Steering · pre | 12 | 25 |
| MMaDA-8B | Steering · post | 16 | 31 |
The ablations reveal a distinct control geometry over diffusion time, transformer depth, and token type.
Early interventions dominate. The first denoising step accounts for much of the impact, and the earliest diffusion block contributes disproportionately.
Mid-to-late layers recover most of the full effect. Early layers alone have limited impact; prompt-token interventions account for most of the effect.
This separates a potentially language-agnostic safety representation within an MDLM from a universal, model-agnostic refusal axis.
Directions extracted in one language substantially reduce refusal and safety scores in the other language without recomputing the vector in the target language. The selected layer–token regions are also nearby across languages.
Applying the best LLaDA steering direction to Meta-Llama-3-8B-Instruct under the same global intervention scheme produces no measurable change, suggesting architecture-dependent safety representations.
Compute harmful and harmless mean residual activations at candidate layer–token positions from a single prompt-only forward pass.
Choose one normalized difference direction on a held-out harmful validation set.
Reuse that direction across all layers, token positions, and reverse-diffusion steps during generation.
For candidate layer ℓ and structural token position i, compute the normalized difference between harmful and harmless mean residual activations.
Candidate token positions include stable chat-template markers both before and after the user instruction.
During generation, hidden states are projected onto the subspace orthogonal to the selected direction. The same selected direction is used globally in the main setup.
We are glad to see this work cited, extended, and applied across a growing range of research directions, including controllable diffusion-language generation, AI safety and refusal steering, activation geometry and mechanistic interpretability, temporally resolved token attribution, in-place text editing, model compression, repetition mitigation, multimodal hallucination control, and representations of code correctness. The papers below highlight several of these emerging connections.
If this work is useful in your research, copy the BibTeX entry directly.
@misc{shnaidman2026activationsteeringmaskeddiffusion,
title = {Activation Steering for Masked Diffusion Language Models},
author = {Adi Shnaidman and Erin Feiglin and Osher Yaari and
Efrat Mentel and Amit Levi and Raz Lapid},
year = {2026},
eprint = {2512.24143},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2512.24143}
}