DiffusionLight

DiffusionLight is a technique designed to estimate lighting in a single input image, making it useful for various applications such as virtual object insertion and relighting. This method uncovers a relationship between the appearance of chrome balls and the initial diffusion noise map, which is then utilized to generate high-quality chrome balls. By fine-tuning an LDR diffusion model with LoRA, DiffusionLight enables exposure bracketing for HDR light estimation, producing convincing light estimates across diverse settings.

The key capability of DiffusionLight lies in its ability to generate multiple plausible chrome balls by varying the initial noise map of diffusion sampling. The average of these variations captures the overall lighting reasonably well, which is then used by the iterative inpainting algorithm to enhance the quality and consistency of light estimation. This technique can estimate lighting for a variety of input images, including indoor and outdoor scenes, close-up shots, paintings, and photos of human faces.

DiffusionLight is particularly valuable for professionals in the field of computer vision, image processing, and graphics, who require accurate light estimation for their work. It can be used in various scenarios, such as virtual object insertion, relighting, and scene understanding, providing a simple yet effective solution for estimating lighting in single input images. With its ability to produce convincing light estimates across diverse settings, DiffusionLight demonstrates superior generalization to in-the-wild scenarios, making it a useful tool for a wide range of applications.