Generating Multi-View Visual Illusions
Andrew Owens (University of Michigan)
Research Seminar
Friday, October 18, 2024, 2:00 pm
Abstract
I will present methods for generating multi-view visual illusions: images that change their appearance upon a transformation, such as a flip or a rotation. Creating these images is a challenging problem, because it requires arranging visual elements such that they can be understood in multiple ways. I will show that we can use off-the-shelf diffusion models to solve this task without additional training, by making simple changes to the reverse diffusion process. First, I will present a method for generating "visual anagrams", images whose pixels can be permuted to form another image. This enables us to make a number of illusions, such as jigsaw puzzles that can be solved in two different ways. I will then present an extension of this approach that allows us to control each individual linear component of an image, given a factorization of an image into a sum of linear factors. This makes it possible to generate images whose appearance varies by viewing distance or illumination conditions. Finally, to address the potential negative implications of image generation methods, I will introduce a new dataset of images produced by thousands of diffusion models, which we use to study generalization in fake image detection.
This is joint work with Daniel Geng, Aaron Park, Ziyang Chen, and Jeongsoo Park.
BIO
Andrew Owens is an assistant professor at The University of Michigan in the department of Electrical Engineering and Computer Science. Prior to that, he was a postdoctoral scholar at UC Berkeley, and he obtained a Ph.D. in computer science from MIT in 2016. He is a recipient of an NSF CAREER Award, a Computer Vision and Pattern Recognition (CVPR) Best Paper Honorable Mention Award, and a Microsoft Research Ph.D. Fellowship.