Image restoration and enhancement:
Video denoising and enhancement
Introduction to Video denoising and enhancement:
We present a novel video enhancement system based on
an adaptive spatio-temporal connective (ASTC) noise filter and an
adaptive piecewise mapping function (APMF). For ill-exposed
videos or those with much noise, we first introduce a novel local
image statistic to identify impulse noise pixels, and then
incorporate it into the classical bilateral filter to form ASTC,
aiming to reduce the mixture of the most two common types of
noises ĘC Gaussian and impulse noises in spatial and temporal
directions. After noise removal, we enhance the video contrast
with APMF based on the statistical information of frame
segmentation results. The experiment results demonstrate that, for
diverse low-quality videos corrupted by mixed noise,
underexposure, overexposure, or any mixture of the above, the
proposed system can automatically produce satisfactory results.
Introduction to Blind deconvolution:
Blind devonvolution, a chronic inverse problem, is the recovery of the
latent sharp image from a blurred one when the blur kernel is unknown. Recent
algorithms based on the MAP approach encounter failures since the global minimum
of the negative MAP scores really favors the blurry image. The goal of this paper is to
demonstrate that the sharp image can be obtained from the local minimum by using
the MAP approach. We first propose a cross-scale constraint to make the sharp image
correspond to a good local minimum. Then the cross-scale initialization, iterative
likelihood update and the iterative residual deconvolution are adopted to trap the
MAP approach in the desired local minimum. These techniques result in our cross-
scale blind deconvolution approach which constrains the solution from coarse to fine.
We test our approach on the standard dataset and many other challenging images. The
experimental results suggest that our approach outperforms all existing alternatives.
Introduction to non-blind deconvolution:
Kernel estimate errors and image noise are major causes of
visual artifacts in image motion deblurring. We propose an
inter-scale non-blind image motion deblurring approach that
significantly reduces those artifacts. We use Gaussian Scale
Mixture Field of Experts (GSM FOE) model as image prior.
The inter-scale smoothness constraint is adopted to suppress
the ringing artifacts. In each scale, image details are recovered
by the residual deconvolution and the cross bilateral filter
(CBF). We further propose a std-controlled CBF to denoise
the result. The experimental results aremuch better than those
of previous methods.
Content about Video denoising and enhancement:
we propose a universal
video enhancement system to automatically recover the ideal
high-quality signal from noise degraded videos and enlarge their
contrast to a subjectively acceptable level. For a given defective
video, we introduce an adaptive spatio-temporal connective
(ASTC) filter, which adapts from temporal to spatial filters based
on noise level and local motion characteristics, to remove mixture
of Gaussian and impulse noises. Both the temporal and the spatial
filters are non-iterative trilateral filters, formed by introducing a
novel local image statistic ĘC neighborhood connective value
(NCV) into the traditional bilateral filter. NCV represents the
connective strength of a pixel to all its neighboring pixels, and is a
good measure for differentiating between impulse noises and fine
features. After noise removal, we adopt pyramid segmentation
algorithm  to divide a frame into several regions. Based on the
areas and standard deviations of these regions, we produce a
novel adaptive piecewise mapping function (APMF) to
automatically enhance the video contrast. To show effectiveness
of our NCV statistic, we conducted a simulation experiment by
adding impulse noises into three representative pictures, and
reported superior noise detection performance compared with
other noise filters. In addition, we tested our system on several
real defective videos adding mixed noises. These videos cover
diverse kinds of defectiveness: underexposure, overexposure,
mixture of them, etc. Our outputs are much more visually pleasing
than those of other state-of-art approaches.
To summarize, the contributions of this work are:
A novel local image statistic for identifying impulse pixels ĘC
Neighborhood Connective Value (NCV)
An Adaptive Spatio-Temporal Connective (ASTC) filter for
reducing mixed noise
An Adaptive Piecewise Mapping Function (APMF) to
enhance video contrast
Framework of proposed universal video enhancement
system, consisting of mixed noise filtering and contrast
Content about Blind deconvolution:
We begin our investigation of the MAP approaches by analyzing the -MAP(L,K) scores of samples in the solution space. Our study shows that, by proper
regularization, the global minimum will favor the desired sharp image. Even in this
situation, we can not exclude the probability that the MAP approach is trapped in
the local minimum which corresponds to the blurry image. Therefore, whether or not
the global minimum corresponds to the sharp image does not determine the success
or failure of the MAP(L,K) estimation. We show that the key to the success of the
MAP(L,K) approach is to make the desired sharp image correspond to a good local
minimum (smaller than all its neighboring components) and to trap the approach in it.
Based on these ideas, we propose an cross-scale deconvolution algorithm to constrain
the MAP solution to the desired one from coarse to fine. At the coarsest scale,
the iterative likelihood update, the smoothness constraint and the iterative
residual deconvolution methods are adopted to produce an acceptable result, which
is sufficient to constrain the optimization at next scale to be trapped in the desired
local minimum. At each coarser scale, the proposed, cross-scale constraint makes the desired sharp image correspond to a good local minimum. The cross-scale initialization
and the iterative likelihood update are further adopted to ensure that the optimization
converges at the desired local minimum. By doing so,we are able to restore high quality
results, especially from the ones with large blur kernels.
The framework of our Multi-scale blind image deconvolution algorithm.
Content about Non-Blind deconvolution:
We propose a novel inter-scale non-blind deconvolution
that produces high-quality deblurred result even
from many challenging blurred images given noisy kernels.
Our algorithm is performed in scale space. First, we adopt the
Gaussian Scale Mixture Field of Experts (GSM FOE) model
learned for each scale as the latent image prior. GSM FOE
model greatly help restore image details while smoothing out
noise since it captures most image energies. Second, based
on the temporal optimized result, we adopt the residual deconvolution
and the cross bilateral filter (CBF) to restoremore
image details, and propose a std-controlled cross bilateral filter
to remove image noise in each scale. Finally, we adopt the
intra-scale and inter-scale smoothness constraints to suppress
the ringing artifacts.
Projects with sponsor:
This work was supported in part by the National Basic Research Program (973) of China
under Grant No.2006CB303103, in part by the National Natural Science Foundation of
China under Grant No. 60833009, in part by the National High-Tech Research and
Development Plan (863) of China under Grant No.2006AA01Z118.
Professor Shiqiang Yang, Jianwei Zhang, Lifeng Sun
Wangchao, Zhuoyuan Chen
 Chao WANG, Lifeng SUN, Shiqiang YANG, Jianwei ZHANG, HIGH-QUALITY NON-BLIND MOTION DEBLURRING, IEEE ICIP 2009,
 Chao WANG, Lifeng SUN, Shiqiang YANG, Jianwei ZHANG, ROBUST INTER-SCALE NON-BLIND IMAGE MOTION DEBLURRING, IEEE ICIP 2009
 Chao Wang, Li-Feng Sun, Shi-Qiang Yang, Video Enhancement Using Adaptive Spatio-Temporal Connective Filter and Piecewise Mapping. EURASIP.
 Chao Wang, Lifeng Sun, Zhuoyuan Chen, Jianwei Zhang and Shi-Qiang Yang. Multi-scale Blind Motion Deblurring Using Local Minimum. Inverse Problem (IP).
Executable programs about all the published papers are coming in two weeks......