VAE Rhythm

Preprint · arXiv:2611.21114

VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection

Ke Sun Shuwei Li Haiyang Mei Mike Zheng Shou†

Show Lab, National University of Singapore †Corresponding author

Overview video. A three-minute narrated walkthrough of VAE Rhythm, the phase scan, and CaDet. 3:00 · HD

Core Insight

Most video generators decode latents with a 3D causal VAE that groups frames in fours, so frame fidelity varies periodically within each group. An off-the-shelf VAE phase scan reads this VAE Rhythm without training.

a
Temporal frame grouping. A 4× causal VAE encodes frame 0 alone and decodes subsequent frames in groups of four (1–4, 5–8), whereas camera frames have no groups.
b
Phase-scan alignment. Shifting a 4× VAE window across 24 start offsets realigns its groups with the clip's every four starts, making the residual r(k) oscillate on generated clips while staying flat on real ones.
Census scope
39variants of 31 models
Zero-shot DF26
76.2%beats 15 baselines
Plug-in gain
+6.7–23.7%all 19 detectors
Panel a: a 4x causal VAE groups Sora frames, with frame 0 on its own latent and frames 1 to 4 sharing the next latent, while real frames have no groups. Panel b: shifting a 4x VAE window along the clip realigns its groups with the clip's every four starts, so the residual r(k) of a Sora clip oscillates while that of a real clip stays nearly flat.
Figure 1 · VAE Rhythm. (a) A 4× causal VAE decodes frames in groups, illustrated on Sora frames. (b) As the VAE window shifts, its groups realign with the clip's every four starts, so the residual of a typical Sora clip oscillates while that of a typical real clip stays nearly flat.

Abstract

Zero-shot score, detector plug-in, and VAE-pair training engine.

Visible artifacts are an unreliable cue for detecting generated videos, because newer generators learn to remove them. We instead detect a trace left by the generation pipeline itself. Most video generators decode latents with a 3D causal VAE that groups consecutive frames in time, so the fidelity of generated frames rises and falls periodically with their position in a group. We call this periodic trace VAE Rhythm. Real videos are captured frame by frame and do not have it. To reveal it without training, we propose a phase scan. The scan reconstructs a clip with an off-the-shelf VAE from successive start offsets and measures the periodicity of the reconstruction error across these offsets. We find VAE Rhythm in most of the 39 variants of 31 generators we test, including the closed-source Sora, Kling 3.0 and Seedance 2.0. We use the rhythm as a zero-shot score, as a plug-in for existing detectors and as a data engine that pairs real videos with their VAE reconstructions to train our detector CaDet. On DF26, the training-free rhythm score outperforms all 15 retrained baselines, and as a plug-in it improves all 19 tested detectors by 6.7% to 23.7% AUROC. CaDet achieves the best AUROC on AIGVDBench, DF26 and three 2026 closed-source models, and its variant trained without any generated video still exceeds 90% on AIGVDBench.

VAE Rhythm

The phase scan reconstructs a clip with an off-the-shelf 4× VAE from successive start offsets k. The reconstruction error drops whenever the scan's frame groups line up with the clip's, so a generated clip gives a residual r(k) that oscillates with the group period, while a real clip gives a nearly flat one.

We find this rhythm in most of the 39 variants of 31 generators we scanned, in open- and closed-source models alike, surviving direct averaging across unrelated clips without phase alignment.

  • Real cohorts (AIGVDBench, DF26)

    Frame-by-frame camera capture; mean r(k) stays flat near zero.

  • Period-4 families (Sora, Wan 2.2, Grok, Seedance 2.0)

    Standard 4× temporal VAE compression; troughs repeat every 4 offsets.

  • Period-8 & resampled (Kling 3.0, LTX; Wan 2.6)

    Period of 8 frames in LTX and Kling 3.0, and ~5 frames in Wan 2.6 (24→30 fps).

Eight small line charts of the mean residual r(k) against start offset k from 0 to 23. The two real cohorts, AIGVDBench and DF26, are nearly flat. Sora, Wan 2.2 T2V, Kling 3.0, Wan 2.6, Grok Imagine and Seedance 2.0 oscillate regularly, each labelled with its zero-shot AUROC.
Figure 2 · VAE Rhythm in real and generated videos. Mean residuals over 50 to 300 clips. Pale bands mark each panel's period, and thin teal lines show the real clips of the same benchmark.

Method

We use VAE Rhythm in three ways: directly as a zero-shot score, as a plug-in that adds rhythm evidence to an existing detector without retraining it, and as a data engine whose VAE pairs train our detector CaDet.

Method overview in three panels. Panel a: a frozen video VAE serves the phase scan, whose residual periodicity gives the zero-shot score, and the VAE pairs, where a real clip and its VAE reconstruction pass through the same post-processing chain. Panel b: CaDet, with an artifact path trained with the pairs and a semantic path trained without them on one frozen DINOv3 backbone, summed into one score. Panel c: the plug-in adds the positive part of the standardized rhythm score to any detector's standardized score.
(a) Phase scan & VAE pairs

A frozen video VAE serves the phase scan, whose periodicity gives the zero-shot score sc, and builds VAE pairs (x+, x−) that differ only by a VAE round trip under shared post-processing.

(b) CaDet: two-path detector

On one frozen DINOv3 ViT-L/16 backbone, CaDet trains an artifact path (LoRA) with the VAE pairs and a semantic path (rank-one residual) without them, summing standardized scores.

(c) Training-free plug-in

Positive rhythm evidence max(0, z(sc)), calibrated on real validation videos alone, is added to the standardized score z(sd) of any detector. All plots and the difference map are measured.

Results

All detectors are trained on AIGVDBench's Open-Sora and real clips and tested cross-generator without retraining, against 15 baselines that we retrained with their official code under one protocol.

Scroll sideways to see all columns.

Main results (AUROC, %), selected columns. Standard deviation is over 3 seeds, and bold marks the best per column.
Method AIGVDBench 2026 closed DF26
20 open 11 closed 3 models all closed
Image forensics, frame-level
CNNSpot82.1±1.271.4±2.478.053.535.5
Gram-Net82.4±0.467.0±1.476.952.937.3
FreDect60.9±1.852.9±1.260.860.863.4
Fusing67.051.564.946.035.0
UnivFD77.3±0.065.4±0.160.354.046.1
NPR73.9±0.860.7±0.869.544.130.7
Video classifiers
TimeSformer74.390.786.467.075.8
UniFormerV281.491.264.950.655.3
Video detectors
DeMamba80.378.171.661.462.3
DeCoF80.678.770.959.858.5
ReStraV52.248.662.757.453.9
ForgeLens, setting 375.5±1.882.4±5.974.355.660.1
ForgeLens, setting 174.1±2.865.0±1.963.646.631.3
ForgeLens, setting-1 schedule, full data85.787.483.260.356.5
Effort84.3±1.292.5±1.981.670.473.4
Native-Scale73.4±2.168.0±4.078.952.249.6
Ours
DINOv3-L LoRA, no pairs (same backbone)68.981.279.362.165.2
CaDet-GF: real videos and VAE pairs90.9±3.091.9±3.084.865.851.3
CaDet: benchmark data and VAE pairs, two paths93.9±0.695.8±0.788.378.579.0

BibTeX

If you find VAE Rhythm or CaDet useful, please cite our preprint.

@article{sun2026vaerhythm,
  title   = {VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection},
  author  = {Sun, Ke and Li, Shuwei and Mei, Haiyang and Shou, Mike Zheng},
  journal = {arXiv preprint arXiv:2611.21114},
  year    = {2026}
}