SyncVid: World-Equivariant Diffusion for Synchronized Video Generation

Qualcomm AI Research

This project page is intended to complement the paper by presenting video and interactive 3D results that static figures cannot adequately convey. It offers a concise, visual overview of the project rather than an exhaustive account; for the complete methodology, derivations, and quantitative results, we refer the reader to the paper PDF.

Synchronized video generation, leading to 4D

Given a single input video, SyncVid generates arbitrarily many novel-view videos guided by user-specified camera trajectories. The generated videos are jointly consistent rather than independently plausible, preserving a coherent underlying dynamic scene. As a result, all seven clips can be processed by a single feed-forward pass of VGGT-Ω to reconstruct the full 4D scene directly, without per-scene optimization, scale adjustment, or alignment.

Input video alone

Input video

All seven videos together

Arc-left Arc-right Dolly-in Dolly-out Pedestal-up Pedestal-down
0
loading

World-Equivariant Gaussian Prior

The two middle panels visualize the same world-aligned randomness field, projected into the source and target views without re-sampling. As a result, corresponding scene points receive identical randomness, providing the sole cue that ties the two viewpoints together. The fourth panel shows the target view generated from the projected target field. The noise is rendered using nearest-neighbor interpolation, with one block per VAE latent cell, exposing the field exactly as seen by the model. The consistent motion of blocks across views demonstrates world-equivariance: randomness remains attached to scene surfaces rather than image coordinates.

Each strip is one scene and one target trajectory. All four panels are 73 frames at 15 fps and play in lockstep.

Method

Video diffusion samples its noise independently in each target image plane, so two cameras looking at the same surface start from unrelated randomness. We sample the noise once in the scene instead, and rasterize it into every view.

SyncVid pipeline. A source video is lifted by a frozen 4D representation into a
                   canonical point cloud of Gaussian primitives. One Gaussian noise vector is drawn
                   per primitive, and a noise cubemap covers unobserved space, together forming a 4D
                   noise space. Given novel camera poses, this shared noise is rasterized and
                   whitened into per-view latent noise maps, which a LoRA-adapted video DiT denoises
                   into synchronized videos along arc-right, dolly-out, and arc-left trajectories.
The pipeline. A frozen 4D representation lifts the source video into a canonical point cloud. Each primitive is assigned one noise vector, drawn once, and a cubemap carries noise for space the source camera never saw. Rasterizing this shared 4D noise into any requested camera gives priors that already agree across views; only the LoRA on the video DiT is trained.
View-consistent whitening. Rasterizing the canonical scene into two target views
                   gives noise maps whose neighboring pixels are strongly correlated, because
                   overlapping Gaussian footprints are shared. A per-pixel orthonormal projection,
                   built from the rasterized canonical anchor position through random Fourier
                   features and a reduced QR factorization, is applied to both views. Pixels p and q
                   observing the same surface point get the same projection, so the outputs are
                   decorrelated within each frame yet still matched across views.
Spatial decorrelation of the noise. Overlapping primitives make neighboring pixels correlated, which the pretrained model does not expect. We whiten with a per-pixel orthonormal projection keyed to the canonical anchor position rather than to screen coordinates. Pixels that see the same surface point therefore share a projection across views and over time, so the noise becomes i.i.d. within a frame while staying synchronized between views.

Comparison with prior work

Input video
Dolly-out
DaS
GWTF
ReCamMaster
ArtiFixer-14B
Reshoot-Anything
SyncVid (Ours)

Importance of cross-view whitening

Standard Gaussian noise
Structured noise, no whitening
Structured + whitening (Ours)
Woman running in a forest Arc-left
Recording a podcast Dolly-out
Girl on roller skates Arc-right
Dancing in the countryside Dolly-out

BibTeX


@article{zanjani2026syncvid,
title        = {SyncVid: World-Equivariant Diffusion for Synchronized Video Generation},
author       = {Farhad G. Zanjani and Animesh Karnewar and Fatih Porikli and Amirhossein Habibian},
journal      = {arXiv preprint arXiv:2610.31765},
year         = {2026}
}
            

Explore more video generation/editing projects from our team at Qualcomm AI Research