Self-Supervised Visual On-Policy Distillation

How S²VOPD works

These findings identify the most effective instantiation of S²VOPD: downscaling the student view to 0.3–0.6× resolution with stochastic Gaussian noise. Across six fine-grained perception benchmarks, this recipe improves Qwen3.5-4B from 70.7% to 77.4% average accuracy, a gain of 6.7%. This places a 4B model above Qwen3-VL-Instruct-235B (75.8%) and GPT-5.4 (72.8%), and matches the much larger Qwen3.5-397B. Given matched training data, previous methods with privileged information improve perception but can degrade reasoning — for example, Opsd loses 9.3% on MathVision at 4B, and ZwZ drops 27.1% on MathVerse; self-reward RL methods like TTRL, RENT, and Intuitor preserve or improve math reasoning but provide weaker perceptual gains. S²VOPD is the only method that consistently improves both, by +5.7% on perception and +3.7% on math reasoning at 4B (+3.6% and +3.2% at 9B).

Overview of S²VOPD. The student generates rollouts from a corrupted view of the image; an EMA teacher scores the same generated prefixes under the clean view; a top-k generalized JSD transfers the teacher's better-informed token distributions to the student. The augmentation applied to the student view is the sole source of supervision — and the object of our study.

Abstract

Visual on-policy distillation relies heavily on an informative teacher–student asymmetry, through either a larger, stronger teacher or privileged supervision such as reference answers or ground-truth regions of interest. This raises a fundamental question: where can informative asymmetry come from when nothing privileged is available?

We answer this by inverting where the asymmetry comes from. Rather than adding privileged information to the teacher, we subtract information from the student. This creates the same effective learning signal for free as a teacher with access to information unavailable to the student — without ground-truth annotations, rewards, or a separate stronger teacher model. Building on this principle, we introduce Self-Supervised Visual On-Policy Distillation (S²VOPD): the teacher's distribution conditioned on the original image is distilled on-policy into the student distribution conditioned on a strongly augmented view of the same image.

To understand what makes augmentation-induced asymmetry useful, we systematically explore a broad design space of visual augmentations and uncover that (1) asymmetry matters: all four augmentation families improve performance, while symmetric self-distillation degrades it; (2) strength matters: performance peaks at a moderate strength; and (3) the gap must remain task-consistent: augmentations that completely remove the question-relevant evidence induce large but uninformative discrepancies. These findings lead to a simple default instantiation: downscaling with stochastic Gaussian noise. Across six fine-grained perception benchmarks, S²VOPD improves Qwen3.5-4B from 70.7% to 77.4% — above every open-source model compared, up to Qwen3-VL at 235B — and beats GPT-5.4. While holding training data the same, it recovers 96% of the improvement achieved by the best supervised method at 76.6% averaged accuracy.

Method

Let \(\pi_\theta\) be the policy being optimized and \(\pi_\phi\) its exponential-moving-average teacher, \(\phi \leftarrow (1-\eta)\phi + \eta\theta\). For each image–question pair \((x, q)\), a stochastic transformation \(T \sim \mathcal{T}\) produces the student view \(\tilde{x} = T(x)\) while the teacher keeps the clean view \(x\). The student samples rollouts from its corrupted view, and the training objective minimizes the expected per-token divergence between teacher and student, evaluated along the student's own trajectories:

The transformation \(T\) is the sole source of the teacher's informational advantage. Among a broad design space of four augmentation families — information reduction, geometric, photometric, and occlusion — the champion recipe is strikingly simple:

What makes augmentation-induced asymmetry useful?

(a) Each augmentation family applied alone. (b) Accuracy vs. the teacher–student predictive gap (token-level JS divergence over the first ten training steps): an inverted U, with crop configurations falling off the curve.

Results

FineVision leaderboard: a 4B model above every open-source model compared

S²VOPD trained on a 12K subset of FineVision (zero annotations), against open-source models, proprietary models, and privileged-information methods on six fine-grained perception benchmarks (accuracy %, average over the six). S²VOPD improves Qwen3.5-4B from 70.68 to 77.44 (+6.76) — above every open-source model in the comparison, including Qwen3-VL-Instruct-235B (75.75), matching the much larger Qwen3.5-397B, beating the GPT-5 series and comparable to Gemini-3-Flash, while surpassing all privileged-supervision baselines (ZwZ, Vision-OPD, Opsd) despite using no ground-truth regions or answers.

Fair comparison with prior methods on the Vision-OPD data

Fair comparison on the Vision-OPD-6K training data at both scales, under the same training data, configuration, and evaluation protocol; bold / underline = best / second-best per column within each base model. S²VOPD achieves the best overall average among all methods without privileged information at both scales (+2.0% over the strongest unsupervised baseline at 4B, +0.7% at 9B), exceeds all supervised baselines at 4B, and matches Vision-OPD as runner-up at 9B. Privileged-supervision methods improve perception but can weaken math reasoning (Opsd loses 9.3% on MathVision at 4B; ZwZ drops 27.1% on MathVerse), while intrinsic-reward RL preserves math but yields limited perception gains — S²VOPD improves both.

Conclusion

We presented S²VOPD, an unsupervised on-policy self-distillation framework for improving VLM perception without privileged annotations. The central idea is to construct an information gap between teacher and student views: the teacher observes the original image, while the student learns from a degraded view. This asymmetry turns the model's own predictions into a perception-aligned training signal, avoiding the need for ground-truth answers, regions, or external teacher models. Across benchmarks and model scales, S²VOPD improves both fine-grained perception and mathematical reasoning, distinguishing it from privileged-supervision methods that strengthen perception but can weaken reasoning, and from intrinsic-reward RL methods that preserve reasoning but provide weaker perceptual gains.

Our analysis shows that the effectiveness of S²VOPD is governed by the magnitude and semantics of the induced information gap. Augmentation must remove enough task-relevant evidence to create a useful learning signal, but not so much that it changes the underlying question or makes recovery impossible. Downscaling plus Gaussian noise provides a robust default, while alternative augmentations can shift the learned capability toward localized or holistic perception. Hyperparameter studies further show that the method is robust to the teacher update rate and benefits from a balanced divergence, supporting the view that the essential supervision comes from constructed teacher–student asymmetry rather than from teacher self-improvement. These results suggest a simple and broadly applicable path for annotation-free perception improvement: useful supervision can be obtained not by adding external labels, but by carefully controlling what the student is allowed to see.

BibTeX

@article{li2026s2vopd,
  title   = {Self-Supervised Visual On-Policy Distillation},
  author  = {Li, Yijiang and Liang, Yijun and Tian, Yunjie and Wang, Bingyang and
             Zhang, Ke and Yin, Zhenfei and Fu, Di and Torr, Philip and Vasconcelos, Nuno},
  journal = {arXiv preprint},
  year    = {2026}
}