Anna Alperovich and Bastian Goldluecke.Unmixing-based soft color segmentation for image manipulation. Yağız Aksoy, Tunç Ozan Aydın, Aljoša Smolić, and Marc Pollefeys.Interactive high-quality green-screen keying via color unmixing. Yağız Aksoy, Tunç Ozan Aydın, Marc Pollefeys, and Aljoša Smolić.In addition, we demonstrate various real-time appearance editing applications for videos with consistent illumination. We evaluate our approach qualitatively and quantitatively and show improvements over the state-of-the-art in this field, in both quality and runtime. The resulting variational decomposition problem uses a new formulation based on sparse and dense sets of non-linear equations that we solve efficiently using a novel alternating data-parallel optimization strategy. At the core of our approach are several sparsity priors that enable the estimation of the per-pixel direct and indirect illumination layers based on a small set of jointly estimated base reflectance colors. In contrast, our approach works for regular videos and produces temporally coherent decomposition layers at real-time frame rates. Existing techniques that invert global light transport require image capture under multiplexed controlled lighting or only enable the decomposition of a single image at slow off-line frame rates.
We retrieve, in separate layers, the contribution made to the scene appearance by the scene reflectance, the light sources, and the reflections from various coherent scene regions to one another. We propose the first approach for the decomposition of a monocular color video into direct and indirect illumination components in real time.