ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models

Zixun Fang1,2, Kai Zhu1,2, Zhiheng Liu3, Yu Liu2, Wei Zhai1✉, Yang Cao1, Zheng-Jun Zha1

1 University of Science and Technology of China 2 TongYi Lab 3 The University of Hong Kong

Paper Code

Gallery (click below pano videos to open VR mode)

We highly recommend using a mobile phone to access the website(better use Chrome browser) for device motion tracking, enhancing the immersive quality of the VR interactive experience.

NOTE: The Loading may be a little slow, but your wait will be worth it !!!

PANO VIDEO VIEWPOINT VIDEO INPUT VIDEO

The video shows the astronauts strolling slowly on the desolate surface of the moon.

moon cliff b2 hometown roadway room scene sea ship snowmountain snowtown spring sunrise supernova surf vangogh text

Qualitative Comparison

Super Resolution

NOTE: We perform super resolution on equirectangular videos directly using STAR, which is designed for perspective videos, thereby resulting in some spatial artifacts and distortion. This issue can be solved by advanced panoramic super-resolution methods.

Template borrowed from Imagine360. Thanks a lot!