Renoise Consistency: Unlocking Efficient Self-Correction for
Diffusion Large Language Models

1Yonsei University   2University of Toronto   3Vector Institute
{jihoonlee98,jaehyungk}@yonsei.ac.kr
ybseo@cs.toronto.edu
NeurIPS 2026 main

Motivation

An illustrative ReCon example from GSM8K

Diffusion Large Language Models (DLMs) can self-correct.

Abstract

Diffusion Large Language Models (DLMs) are being actively explored as promising alternatives to autoregressive models due to their fast and flexible generation capabilities. To optimize inference in DLMs, recent studies have proposed a variety of decoding algorithms. However, the tendency of these methods to mimic autoregressive generation limits the speed and flexibility of DLMs, while also preventing the methods from fully realizing their potential. Moreover, this rigid approach leads to error propagation, while existing sampling-based correction methods incur prohibitive computational costs. To overcome this, we propose Renoise Consistency, a novel post-hoc approach that enables effective self-correction by maximizing train–inference consistency and can be applied to any decoding algorithm with a flexible computational budget. Furthermore, we introduce Adaptive Block Drafting, which leverages consistent internal representations of DLMs to significantly reduce the overall computational cost. When combined, our proposed method achieves up to a 9.00% performance improvement over the semi-autoregressive baseline, along with a 25.85% reduction in forward steps.

Observation

Bidirectional inertia in confidence distributions

Findings #1: Bidirectional Inertia reveals the optimal output length or block length.
Adaptive Block Drafting (ABD), efficient drafting.

Scattered anchor context and parallel decoding

Findings #2: Scattered Anchor Context resolves the train-inference mismatch.
Renoise Consistency (ReCon), efficient self-correction.

Method

Overview of Adaptive Block Drafting and Renoise Consistency

2 step train-free inference (ABD + ReCon).

Takeaway

\[\mathcal{L}(\theta) \triangleq -\,\mathbb{E}_{t,x_0,x_t}\left[\frac{1}{t}\sum_{i=1}^{L}\mathbb{1}\left[x_t^i = M\right]\log p_\theta\left(x_0^i \mid x_t\right)\right]\]

Uniform masking (\(t \sim U[0,1]\)) yields
scattered anchor contexts (train),
while existing decoding methods (inference)
impose a mismatched left-to-right prior.
ReCon resolves this mismatch
during reconstruction.

Forward stepsUniform random anchorsLeft-sided anchors
164.57%26.50%
586.56%31.40%
1089.21%48.28%

Reconstruction success rate of initially correct GSM8K outputs (LLaDA 8B Instruct) after remasking 90% of the tokens.

Results

Accuracy added by ReCon on top of existing decoding methods

ReCon improves drafts.
Semi-AR: up to 7.17%p
Conf. fallback: up to 6.83%p
KLASS: up to 4.95%p
ABD: up to 5.69%p

ABD + ReCon vs WINO and Saber on LLaDA

ABD + ReCon outperforms lightweight revocable decodings.
WINO: up to 23.5%p
Saber: up to 18.1%p

BibTeX

Coming soon.