Current location: Home -> Research -> Image restoration and enhancement

Image restoration and enhancement:

Video denoising and enhancement
Blind deconvolution
non-blind deconvolution

Introduction to Video denoising and enhancement:

We present a novel video enhancement system based on an adaptive spatio-temporal connective (ASTC) noise filter and an adaptive piecewise mapping function (APMF). For ill-exposed videos or those with much noise, we first introduce a novel local image statistic to identify impulse noise pixels, and then incorporate it into the classical bilateral filter to form ASTC, aiming to reduce the mixture of the most two common types of noises ĘC Gaussian and impulse noises in spatial and temporal directions. After noise removal, we enhance the video contrast with APMF based on the statistical information of frame segmentation results. The experiment results demonstrate that, for diverse low-quality videos corrupted by mixed noise, underexposure, overexposure, or any mixture of the above, the proposed system can automatically produce satisfactory results.

Introduction to Blind deconvolution:

Blind devonvolution, a chronic inverse problem, is the recovery of the latent sharp image from a blurred one when the blur kernel is unknown. Recent algorithms based on the MAP approach encounter failures since the global minimum of the negative MAP scores really favors the blurry image. The goal of this paper is to demonstrate that the sharp image can be obtained from the local minimum by using the MAP approach. We first propose a cross-scale constraint to make the sharp image correspond to a good local minimum. Then the cross-scale initialization, iterative likelihood update and the iterative residual deconvolution are adopted to trap the MAP approach in the desired local minimum. These techniques result in our cross- scale blind deconvolution approach which constrains the solution from coarse to fine. We test our approach on the standard dataset and many other challenging images. The experimental results suggest that our approach outperforms all existing alternatives.

Introduction to non-blind deconvolution:

Kernel estimate errors and image noise are major causes of visual artifacts in image motion deblurring. We propose an inter-scale non-blind image motion deblurring approach that significantly reduces those artifacts. We use Gaussian Scale Mixture Field of Experts (GSM FOE) model as image prior. The inter-scale smoothness constraint is adopted to suppress the ringing artifacts. In each scale, image details are recovered by the residual deconvolution and the cross bilateral filter (CBF). We further propose a std-controlled CBF to denoise the result. The experimental results aremuch better than those of previous methods.

Content about Video denoising and enhancement:

we propose a universal video enhancement system to automatically recover the ideal high-quality signal from noise degraded videos and enlarge their contrast to a subjectively acceptable level. For a given defective video, we introduce an adaptive spatio-temporal connective (ASTC) filter, which adapts from temporal to spatial filters based on noise level and local motion characteristics, to remove mixture of Gaussian and impulse noises. Both the temporal and the spatial filters are non-iterative trilateral filters, formed by introducing a novel local image statistic ĘC neighborhood connective value (NCV) into the traditional bilateral filter. NCV represents the connective strength of a pixel to all its neighboring pixels, and is a good measure for differentiating between impulse noises and fine features. After noise removal, we adopt pyramid segmentation algorithm [26] to divide a frame into several regions. Based on the areas and standard deviations of these regions, we produce a novel adaptive piecewise mapping function (APMF) to automatically enhance the video contrast. To show effectiveness of our NCV statistic, we conducted a simulation experiment by adding impulse noises into three representative pictures, and reported superior noise detection performance compared with other noise filters. In addition, we tested our system on several real defective videos adding mixed noises. These videos cover diverse kinds of defectiveness: underexposure, overexposure, mixture of them, etc. Our outputs are much more visually pleasing than those of other state-of-art approaches.
To summarize, the contributions of this work are:
A novel local image statistic for identifying impulse pixels ĘC Neighborhood Connective Value (NCV)
An Adaptive Spatio-Temporal Connective (ASTC) filter for reducing mixed noise
An Adaptive Piecewise Mapping Function (APMF) to enhance video contrast

Framework of proposed universal video enhancement system, consisting of mixed noise filtering and contrast enhancement.

Content about Blind deconvolution:

We begin our investigation of the MAP approaches by analyzing the -MAP(L,K) scores of samples in the solution space. Our study shows that, by proper regularization, the global minimum will favor the desired sharp image. Even in this situation, we can not exclude the probability that the MAP approach is trapped in the local minimum which corresponds to the blurry image. Therefore, whether or not the global minimum corresponds to the sharp image does not determine the success or failure of the MAP(L,K) estimation. We show that the key to the success of the MAP(L,K) approach is to make the desired sharp image correspond to a good local minimum (smaller than all its neighboring components) and to trap the approach in it. Based on these ideas, we propose an cross-scale deconvolution algorithm to constrain the MAP solution to the desired one from coarse to fine. At the coarsest scale, the iterative likelihood update, the smoothness constraint and the iterative residual deconvolution methods are adopted to produce an acceptable result, which is sufficient to constrain the optimization at next scale to be trapped in the desired local minimum. At each coarser scale, the proposed, cross-scale constraint makes the desired sharp image correspond to a good local minimum. The cross-scale initialization and the iterative likelihood update are further adopted to ensure that the optimization converges at the desired local minimum. By doing so,we are able to restore high quality results, especially from the ones with large blur kernels.

The framework of our Multi-scale blind image deconvolution algorithm.

Content about Non-Blind deconvolution:

We propose a novel inter-scale non-blind deconvolution that produces high-quality deblurred result even from many challenging blurred images given noisy kernels. Our algorithm is performed in scale space. First, we adopt the Gaussian Scale Mixture Field of Experts (GSM FOE) model learned for each scale as the latent image prior. GSM FOE model greatly help restore image details while smoothing out noise since it captures most image energies. Second, based on the temporal optimized result, we adopt the residual deconvolution and the cross bilateral filter (CBF) to restoremore image details, and propose a std-controlled cross bilateral filter to remove image noise in each scale. Finally, we adopt the intra-scale and inter-scale smoothness constraints to suppress the ringing artifacts.

Projects with sponsor:

This work was supported in part by the National Basic Research Program (973) of China under Grant No.2006CB303103, in part by the National Natural Science Foundation of China under Grant No. 60833009, in part by the National High-Tech Research and Development Plan (863) of China under Grant No.2006AA01Z118.


Professor Shiqiang Yang, Jianwei Zhang, Lifeng Sun
Wangchao, Zhuoyuan Chen


[3] Chao Wang, Li-Feng Sun, Shi-Qiang Yang, Video Enhancement Using Adaptive Spatio-Temporal Connective Filter and Piecewise Mapping. EURASIP.
[4] Chao Wang, Lifeng Sun, Zhuoyuan Chen, Jianwei Zhang and Shi-Qiang Yang. Multi-scale Blind Motion Deblurring Using Local Minimum. Inverse Problem (IP).


Executable programs about all the published papers are coming in two weeks......

Links: Tsinghua - DCST - HCI&MI

Any questions or comments about our web site please Contact Us
Copyrights (C) 2005 Multimedia Group, Dept. of CS, Tsinhua University.
All rights reserved.