Many cascades initialize each refinement stage with the pixel-space operator \(\mathrm{UP}_{\mathrm{pixel}}\):
\[\bar{\mathbf{z}}_0^{4r}=\mathrm{UP}_{\mathrm{pixel}}(\mathbf{z}_0^r)=\mathcal{E}\!\left(\mathrm{Interpolate}(\mathcal{D}(\mathbf{z}_0^r),2)\right).\]
Latent interpolation is not a substitute. The VAE encoder is nonlinear, so
\[\mathrm{Interpolate}(\mathbf{z},2)\neq\mathrm{UP}_{\mathrm{pixel}}(\mathbf{z}).\]
Latent Hyperloop is a lightweight function \(\mathcal{H}_{\phi}\), about 5M trainable parameters, that predicts the latent that round trip would have produced:
\[\bar{\mathbf{z}}_0^{4r}=\mathcal{H}_{\phi}(\mathbf{z}_0^r)\approx\mathrm{UP}_{\mathrm{pixel}}(\mathbf{z}_0^r).\]
It reuses the frozen first decoder block \(\mathcal{D}^{(1)}\) as a spatial prior. A projection head \(h_{\psi}\) fixes the channel count and the geometry, so \(\mathcal{H}_{\phi}\) is \(\mathcal{D}^{(1)}\) followed by \(h_{\psi}\).
Each 64-pixel source crop is padded with 8 pixels of context on every side, so an \(80\times80\) input predicts a \(128\times128\) patch. Training uses an \(\ell_1\) reconstruction loss and an attention-alignment loss that matches the refiner’s self-attention maps, combined with equal weights. Because the pairs come from encoded images rather than an imperfect VAE round trip, the shortcut can beat the trip it replaces.
