Tianli Yu's Research

Abstract

Belief Propagation (BP) has been successfully used to approximate the solutions of various Markov Random Field (MRF) formulated energy minimization problems. However, large MRFs require a significant amount of memory to store the intermediate belief messages. We observe that these messages have redundant information due to the imposed smoothness prior. In this paper, we study the feasibility of applying compression techniques to the messages in the min-sum/max-product BP algorithm with 1D labels to improve the memory efficiency and reduce the read/write bandwidth. We articulate properties that an efficient message representation should satisfy. We investigate two common compression schemes, predictive coding and linear transform coding (PCA), and then propose a novel Envelope Point Transform (EPT) method. Predictive coding is efficient and supports linear operations directly in the compressed domain, but it is only compatible with the L1 smoothness function. PCA has the disadvantage that it does not guarantee the preservation of the minimal label. EPT is not limited to L1 smoothness cost and allows a flexible quality vs. compression ratio tradeoff compared with predictive coding. Experiments on dense stereo reconstruction have shown that the predictive scheme and EPT can achieve 8× or more compression without significant loss of depth accuracy.

Paper

Tianli Yu, Ruei-Sung Lin, Boaz J. Super, Bei Tang, Efficient Message Representations for Belief Propagation, accepted by ICCV 2007, (Download)

Abstract
We present a novel relighting approach that does not assume that the illumination is known or controllable. Instead, we estimate the illumination and texture from multi-view images captured under a single illumination setting, given the object shape. We rely on the viewpoint-dependence of surface reflectance to resolve the usual texture-illumination ambiguity. The task of obtaining the illumination and texture models is formulated as the decomposition of the observed surface radiance tensor into the product of a light transport tensor, and illumination and texture matrices. We estimate both the illumination and texture at the same time by solving a system of bilinear equations. To reduce estimation error due to imperfect input surface geometry, we also perform a multi-scale discrete search on the specular surface normal. Our results on synthetic and real data indicate that we can estimate the illumination, the diffuse as well as the specular components of the surface texture map (up to a global scaling ambiguity). Our approach allows more flexibilities in rendering novel images, such as view changing, and light and texture editing .

Hongcheng Wang also has a detailed project page here.

Papers

1. Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relight by Illumination and Reflectance Estimation from Multi-View Images, Eurographics Symposium on Rendering (EGSR), 2006
   
2. Tianli Yu, Hongcheng Wang, Narendra Ahuja, Wei-Chao Chen, Sparse Lumigraph Relight by Illumination and Reflectance Estimation from Multi-View Images, Technical Sketch, SIGGRAPH, 2006
   

Tianli Yu (UIUC), Narendra Ahuja (UIUC) and Wei-Chao Chen (Nvidia Corp.)

Part of my thesis work in 3D reconstruction of non-lambertian (specular) objects using graph cuts on a new non-uniform grid is accepted by CVPR 2006. See you in New York.

Abstract

We show that the approaches to 3D reconstruction that use volumetric graph cuts to minimize a cost function over the object surface have two types of biases, the minimal surface bias and the discretization bias. These biases make it difficult to recover surface extrusions and other details, especially when a non-lambertian photo consistency measure is used. To reduce these biases, we propose a new iterative graph cuts based algorithm that operates on the Surface Distance Grid (SDG), which is a special discretization of the 3D space, constructed using a signed distance transform of the current surface estimate. It can be shown that SDG significantly reduces the minimal surface bias, and transforms the discretization bias into a controllable degree of surface smoothness. Experiments on 3D reconstruction of non-lambertian objects confirm the effectiveness of our algorithm over previous methods.



Paper

1. Tianli Yu, Narendra Ahuja, and Wei-Chao Chen, SDG Cut: 3D Reconstruction for Non-lambertian Objects Using Graph Cuts on Surface Distance Grid, accepted by IEEE Conference on Computer Vision and Pattern Recognition 2006, New York, June 2006.
   

Data Set
We are using Univ. of Washington's Fish data set and Intel lab's Van Gogh data set for the experiments.

The original fish data has some large calibration errors which make it difficult to match the surface texture. We manually segment the silhouettes in 30 images of the fish data set, and use these silhouettes and the scanned object shape to improve the camera calibration. Here are the manually segmented silhouette masks. Calibration refinement routines and the results will be posted later.

During my Ph.D. study I developed a series of matlab tools to help my research project. I am now working on making these tools publicly available so that it could be helpful to more people.

The Parameter Tuner/Optimizer is the first of this series. It is used to adjust the parameter set of an algorithm (implemented by the user as a wrapper function with standard interface) for a better performance. ParamTunerX can run the wrapper function with a predefined set of different parameter combinations, store them into time/parameter tagged directories and output the summery. ParamOptimizerX make use of the matlab optimization toolbox to automatically optimize the parameter based on the performance measure returned by the wrapper function.

The latest version of Parameter Tuner/Optimizer can be downloaded from here: ParamTune.zip

I defensed my Ph.D. Thesis on Dec. 13, 2005. Here is my dissertation.

Abstract

Modeling a 3D scene from camera observations includes representation of scene geometry (3D shape) and photometry (surface reflectance and illumination). For the most part, these two properties of the scene have been treated as separate research topics. For scenes with specular objects, however, the two problems are interrelated and not easily separable. Knowing the photometric model of the scene helps the reconstruction of the geometric model and vice versa. In this dissertation, we investigate the problem of simultaneous reconstruction of geometric and photometric properties of 3D scenes from images captured by multiple cameras.

We take two different approaches to solve the modeling problem. For objects composed of a homogeneous specular material, where stereo correspondence is almost impossible, we develop a method that extends the conventional shape from shading to multiple views and unknown reflectance. It is an iterative framework to solve for the 3D shape and parameterized photometric model. For objects that have complex specular and diffuse textures, we take an object-centered matching approach. Graph cuts on surface distance grid are used to robustly reconstruction 3D shape based on a non-Lambertian photo consistency measure. Then, the texture, illumination and geometric refinement are estimated based on a bilinear reflection equation that involves the diffuse/specular surface reflection and self-shadows.

Tianli's Ph.D. Thesis

During my summer internship in Kodak Research Lab, we are developing robust image segmentation methods to allow automatic analysis of X-ray images. Our algorithm can "learn" from a set of training examples what shape to look for in the new image. The resulting "ShRAC" algorithm, which stands for Shape Regularized Active Contour, has an excellent robustness to noise and distracting structures in medical images, and is able to segment objects with large (nonlinear) shape variations.

Figure 1: Starting with the same elliptic shape, a segmentation algorithm should converge to the correct shape without model selection (e.g. to adapt to whether it is the left or the right lung field). Here we show the result of the proposed ShRAC algorithm: (a) Initial contour on the right lung, (b) Final contour on the right lung, (c) Initial contour on the left lung (note that it is the same shape as in (a)), and (d) Final contour on the left lung.


Papers:

Tianli Yu, Jiebo Luo and Narendra Ahuja
Shape Regularized Active Contour using Iterative Global Search and Local Optimization, accepted by CVPR 2005, June 20-26 2005, San Diego, CA, USA

Tianli Yu, Jiebo Luo, Amit Singhal, and Narendra Ahuja
Shape regularized active contour based on dynamic programming for anatomical structure segmentation, SPIE Medical Imaging 2005, February 12-17 2005, San Diego, CA, USA

Many materials in real world has shining reflection surfaces that change their appearance with viewing direction and illumination direction. These materials will confuse most stereo algorithms that expect constant appearance from different viewpoints. We are developing methods to explicitly modeling these reflection effects and integrate them into the multiple view recontruction algorithm.

Our research can be divided into several stages depending the amount of information available during the reconstruction.

1. If the illumination is known, then we can reconstruct both the uniform surface BRDF (material) model as well as the object shape.

Initial shape deform to the correct shape as the iterative algorithm converges.

Paper:
Tianli Yu, Ning Xu and Narendra Ahuja, "Recovering Shape and Reflectance Model of Non-Lambertian Objects from Multiple Views", CVPR 2004
Full text: From IEEE Xplore (subscription required)



2. If the illumination is also unknown, then we use the View Independent Reflectance Map to reconstruct both the surface shape, and an joint effect of illumination and surface material.



Paper:

Tianli Yu, Ning Xu and Narendra Ahuja, "Shape and View Independent Reflectance Map from Multiple Views", ECCV 2004.
Full text: From Springer (Subscription required)

Tianli Yu, Ning Xu and Narendra Ahuja, "Shape and View Independent Reflectance Map from Multiple Views", International Journal of Computer Vision, vol. 73 , issue 2 , pp. 123 - 138, June 2007.