主题: Specular-to-Diffuse Translation for Multi-View Reconstruction
In this talk, I will introduce one of our recent works on learning-based image processing, aiming for improving the quality of multi-view image-based 3D reconstruction. Most multi-view 3D reconstruction algorithms, especially when shape-from-shading cues are used, assume that object appearance is predominantly diffuse. To alleviate this restriction, we introduce S2Dnet, a generative adversarial network for transferring multiple views of objects with specular reflection into diffuse ones, so that multi-view reconstruction methods can be applied more effectively. We introduce a Multi-View Coherence loss (MVC) that evaluates the similarity and faithfulness of local patches after the view-transformation. In addition, we carefully design and generate a large synthetic training data set using physically-based rendering. During testing, our network takes only the raw glossy images as input, without extra information such as segmentation masks or lighting estimation. Results demonstrate that multi-view reconstruction can be significantly improved using the images filtered by our network.
Shihao Wu is a postdoc at the IGL group in ETH led by Prof. Olga Sorkine-Hornung. He earned his PhD from the University of Bern in 2018, advised by Prof. Matthias Zwicker. He obtained his B.Sc. (2011) and M.Sc. (2014) in Computer Science from South China Normal University and South China University of Technology, respectively. His former advisers are Prof. Hui Huang, Prof. Guiqing Li, Prof. Minglun Gong and Prof. Daniel Cohen-Or. His research interests include Computer Graphics, Geomertric Modeling, and Machine Learning.