matting
Papers with tag matting
2022
- FactorMatte: Redefining Video Matting for Re-Composition TasksZeqi Gu, Wenqi Xian, Noah Snavely, and Abe DavisIn 2022
We propose "factor matting", an alternative formulation of the video mattingproblem in terms of counterfactual video synthesis that is better suited forre-composition tasks. The goal of factor matting is to separate the contents ofvideo into independent components, each visualizing a counterfactual version ofthe scene where contents of other components have been removed. We show thatfactor matting maps well to a more general Bayesian framing of the mattingproblem that accounts for complex conditional interactions between layers.Based on this observation, we present a method for solving the factor mattingproblem that produces useful decompositions even for video with complexcross-layer interactions like splashes, shadows, and reflections. Our method istrained per-video and requires neither pre-training on external large datasets,nor knowledge about the 3D structure of the scene. We conduct extensiveexperiments, and show that our method not only can disentangle scenes withcomplex interactions, but also outperforms top methods on existing tasks suchas classical video matting and background subtraction. In addition, wedemonstrate the benefits of our approach on a range of downstream tasks. Pleaserefer to our project webpage for more details: https://factormatte.github.io
Factor matting of videos. 将video中的各部分内容decompose出来. 用一个网络预测图片中的各个component. 通过reconstruction loss训练网络. 通过foreground discriminator来regularize training.
matting@inproceedings{FactorMatte, title = {FactorMatte: Redefining Video Matting for Re-Composition Tasks}, author = {Gu, Zeqi and Xian, Wenqi and Snavely, Noah and Davis, Abe}, year = {2022}, tags = {matting}, sida = {Factor matting of videos. 将video中的各部分内容decompose出来. 用一个网络预测图片中的各个component. 通过reconstruction loss训练网络. 通过foreground discriminator来regularize training.}, }