1
Regression-based Analysis of Visual Quality for Denoised Images A. Rubel, V. Lukin Department of Transmitters, Receivers and Signal Processing (504) National Aerospace University (KhAI) 17 Chkalov St., Kharkov, 61070, Ukraine
[email protected],
[email protected]
Abstract—Visual quality plays a key role in different image processing applications, in particular, in image denoising. Assessment of visual quality of denoised images allows assessing the effectiveness of filtering. Among image visual quality assessment approaches, the most reliable are experiments with observers to get mean opinion score (MOS). This paper aims to conduct prediction of MOS for denoised images based on several quantitative criteria. Probability of voting for expedience of denoising is used as MOS. Analysis is carried out for two filters – standard DCT filter and BM3D. Problems and challenges of such prediction are shown. Keywords—visual quality; denoising; filtering efficiency; experiment with observers; DCT filter, BM3D
I. INTRODUCTION Images are used in tremendous number of applications and systems [1]. However, practically all acquired images are distorted by noise that reduces their quality. Meanwhile, image quality is important for infocommunication systems that store, process and disseminate this kind of data. For noise reduction, image denoising (filtering) is often applied. Image denoising allows improving visual quality and contributes further automatic processing such as segmentation and classification [2]. In recent decades, a large number of filters has been developed [3]. Transform domain and non-local filters have shown good performance [4, 5]. Because of this, we pay main attention to them in our further studies. Image quality after denoising can be measured by visual quality metrics [6, 7] and/or by observers attracted to perform visual quality assessment. Experiments with observers are the most reliable methods with providing useful information about peculiarities of human perception of image quality.
or images that contain a lot of small details, filtering leads to deterioration of image visual quality. In terms of using visual quality metrics, the higher the improvement for a given metric, the better image visual quality. However, such an improvement provided by denoising does not always result in better visual quality for a processed image. Thus, it is very important to know how humans evaluate the result of denoising. It would also be helpful to predict such an image visual quality assessment by a human before starting denoising. Then, this information will clarify when denoising can be expedient. The goal of this paper is to predict visual quality evaluation by human based on several input parameters. It is worth noting that methodology of denoising efficiency prediction has been already proposed [10, 11, 13-15]. This methodology allows predicting improvement for different visual quality metrics with high prediction accuracy. II. DESCRIPTION OF EXPERIMENT Let us briefly describe the conducted experiment. Sixteen test grayscale images from the databases TID2013 [12] (http://ponomarenko.info/tid2013.htm) and USC-SIPI (http://sipi.usc.edu/database/) have been taken for experiment. As a noise model, additive white Gaussian noise with zero mean and standard deviation varied in big range has been chosen. The experiment was carried out as follows. Two images have been simultaneously displayed on the computer screen – noisy and denoised ones. Each participant had to choose which image has a better visual quality by clicking on button. Illustration of the experiment is shown in Fig. 1.
We have already conducted experiments with observers to assess visual quality of denoised images with different number of participants [8, 9]. DCT and BM3D filters have been considered in these experiments. As a result of the experiments, the probability of voting for denoising expedience has been determined. This probability means that the filtered image has a better visual quality than the corresponding noisy one. It has been also demonstrated that filtering is worth applying for moderate intensity noise and images with quite simple structure [9]. For texture images Fig. 1 Illustration of experiment (noisy image left and denoised right)
2017 4th International Scientific-Practical Conference
Problems of Infocommunications. Science and Technology 978-1-xxxx-xxxx-x/17/$31.00 ©2017 IEEE
PIC S&T`2017
2 As the result, for each image and given noise intensity, the probabilities for voting for original or denoised images have been determined. Probability of voting for denoising Probvote larger than 0.5 means that the use of denoising is, probably, reasonable. If probability is close to unity, one can state that the use of filtering is, obviously, expedient. Similarly, probability about 0.2 means that it is not worth applying denoising. The total number of observers who have taken part in experiments is 109. This has allowed obtaining quite stable and reliable results. III. ANALYSIS OF OBTAINED RESULTS The idea of prediction of image visual quality evaluation by humans (i.e., probability of voting for denoising) is the following [9-11]. It is supposed that there is an output parameter that adequately characterizes filtering efficiency and one or more input statistical parameters that describe an image to be filtered. Examples of output parameters are quality metrics (for example, improvements of quality metrics PSNR or PSNR-HVS-M [6] or probability of voting for denoising). One example of input parameter is probability Pασ that amplitudes of AC DCT coefficients in blocks are less than a given threshold ασ where σ denotes noise standard deviation [10, 11] and α is used defined parameter. In addition, it is assumed that there is a dependence between output parameter, for instance, the aforementioned probability of voting for denoising or improvement of PSNR and input parameter(s). This dependence has to be described analytically and obtained in advance. Then, prediction itself is carried out as follows. An input parameter (or several input parameter) is calculated and used as argument(s) in the dependence to determine output parameter. Analysis of output parameter (e.g., value of probability of voting for denoising) allows undertaking decision is it worth performing filtering. If filtering is skipped, this leads to saving time and resources. Keeping this in mind, there are several requirements to prediction. It should be fast – by several times faster than filtering itself. Besides, it should be accurate enough to provide correctness of decisions undertaken on basis of prediction. Our previous studies [9, 11] have demonstrated that the use of only one input parameter Pασ does not provide appropriate prediction of Probvote. There is general tendency of Probvote increasing if Pασ increases but Probvote also depends upon noise intensity or input PSNR and this dependence is not monotonous. Therefore, we expect that the use of two input parameters can improve prediction accuracy. If two input parameters are used, prediction is more complex and time consuming. To make it easy and fast, all input parameters should be easily computable and simple. In addition, dependence connecting output parameter and input parameters should be simple as well. Let us apply the methodology described above to data obtained from experiments with observers. A. DCT filter Now consider the case of using the standard DCTbased filter [4, 7]. In our analysis, to characterize image
PIC S&T`2017
complexity and noise intensity, we have decided to use two input parameters. The first one is quality metrics (PSNR or PSNR-HVS-M) and the second one is the statistical parameter P0.5 that has earlier demonstrated good properties [11]. Recall that P0.5 is a mean probability that absolute values of DCT coefficients in 8x8 pixel blocks do not exceed a threshold 0.5 , where denotes standard deviation of the noise [10]. As a prediction performance, goodness of fit R 2 (also called coefficient of determination) and root mean square error (RMSE) are often used. R 2 is the square of the sample Pearson correlation coefficient between the outcomes and their predicted values. Larger values of R 2 relates to a better prediction. Smaller RMSE evidence in favor of better prediction. According to previous works [10, 11], prediction model in the form of exponential function has been found very simple and effective. However, dependence can be not monotonic and then the use of polynomial approximation can be reasonable. Thus, let us suppose that prediction based on product exponential and polynomial functions can be used. The considered fitting function is the following:
n Pr ob vote = exp a P0.5σ bk Metrick , k 0
(1)
where a and b are coefficients of fitting functions, Metric is a considered metric (second parameter), Pr obvote is a predicted probability of voting for denoising and n denotes order of polynomial. Results of prediction performance for metrics PSNR and PSNR-HVS-M with different orders of polynomial are given in Table 1. Note that, under condition of a priori known σ, input PSNR can be easily calculated. Input PSNR-HVS-M cannot be calculated in practice since noise-free image is unknown. So, we consider the use of input PSNR-HVS-M as hypothetic (impractical) variant. TABLE I. n 1 2 3 4 5
GOODNESS OF FIT FOR DCT FILTER
Metric PSNR PSNR-HVS-M PSNR PSNR-HVS-M PSNR PSNR-HVS-M PSNR PSNR-HVS-M PSNR PSNR-HVS-M
R2 0.348 0.3339 0.4821 0.4007 0.4849 0.4093 0.5006 0.4271 0.4818 0.4191
RMSE 0.1039 0.105 0.093 0.1001 0.0932 0.0998 0.0922 0.0987 0.0948 0.100
Since we consider two-input parameters, the approximating (regression) dependence is a surface which is the function of two arguments. Points of the scatter-plot for it have been obtained using a large number of test images with artificially added noise characterized by standard deviations varied in wide limits (from 3 to 30). For each test image and value of σ, both input parameters have been determined. Values of Probvote have been taken from experiments.
October 10-13, 2017
Kharkiv, Ukraine
3 Fig. 2 presents examples of 2D scatter plot for prediction using the fourth-order polynomial as the second term in (1). As it is seen, although general tendencies are observed, scatter-plot points are placed quite sparsely.
B. BM3D filter Let us do the same for data obtained for black-matching 3-D (BM3D) filter [5]. Goodness of fit results for different quality metrics and degrees of polynomial are given in Table 2. RMSEs are given as well. It can be seen that prediction performance for different types of polynomials is rather low. R2 does not exceed 0.45 even for the fourth-order polynomial that again provides the best regression. In addition, the values of RMSE indicate that prediction is not quite accurate. At least, they are sufficiently larger than for the standard DCT-based filter. Examples of 2D scatter plots for prediction of probability of voting for denoising using fitting function (1) with fourth-order polynomial are presented in Fig. 3. TABLE II. n 1
a
2 3 4 5
GOODNESS OF FIT FOR BM3D FILTER
R2
Metric PSNR PSNR-HVS-M PSNR PSNR-HVS-M PSNR PSNR-HVS-M PSNR PSNR-HVS-M PSNR PSNR-HVS-M
0.3538 0.3464 0.4333 0.3724 0.4334 0.3872 0.434 0.4034 0.4236 0.3826
RMSE 0.1237 0.1244 0.1164 0.1225 0.1169 0.1216 0.1174 0.1205 0.1196 0.1238
b Fig. 2 2D scatter plots of dependences of probability of voting for denoising on mean values of P0.5 and input PSNR (a) or input PSNRHVS-M (b) using the fourth-order polynomial for the standard DCT filter
From analysis of data in Table 1, it is seen well that fitting functions with the fourth-order polynomial have the best prediction performance. Values of R 2 are from about 0.34 for the first order polynomial to about 0.5 for the fourth-order regression. Meanwhile, the second and third order polynomial approximations provide almost the same results as the fourth-order regression.
a
One more interesting observation is that regression is better if input PSNR (not input PSNR-HVS-M) is used as the second input parameter. This is especially good in the sense that input PSNR can be easily estimated. It is worth stressing that goodness of fit has considerably increased compared to the case when only one input parameter (Pασ) was used in prediction [9, 11] –it was about 0.2. However, for all cases of input parameters (metrics) and orders of polynomials goodness of fit is still quite low. Recall that correlation between output and input parameters is considered good if goodness of fit exceeds 0.9.
b Fig. 3 2D scatter plots of dependences of probability of voting for denoising on mean values of P0.5 and input PSNR (a) and input PSNRHVS-M (b) using the fourth-order polynomial for BM3D filter
2017 4th International Scientific-Practical Conference
Problems of Infocommunications. Science and Technology
PIC S&T`2017
4 Again, prediction results are almost the same for polynomials of the second, third, and fourth orders. Examples of 2D scatter plots for prediction using fitting function (1) with second and third orders polynomials are shown in Fig. 4. The results for the case of using input PSNR are better than for input PSNR-HVS-M.
parameter P0.5 . Due to the use of two input parameters, prediction accuracy has been improved. However, the fitted functions still have rather small goodness of fit values that means quite low prediction accuracy. Possible ways to improve prediction accuracy have been mentioned.
For both filters, R2 values are not high enough and RMSEs are not as small as desired (e.g., 0.03). Thus, the prediction of Probvote is still not accurate enough.
Thus, the prediction performance of image visual quality evaluation by humans is still an open problem. Future work will be devoted to obtaining more accurate prediction models.
To our opinion, there are several ways to further improve prediction accuracy. One of them is the use of more input parameters [15] and/or the use of other statistical parameters as input ones [14]. Another approach for improving prediction accuracy is preliminary processing of experimental results to remove outlier data points from statistics.
REFERENCES [1] [2] [3]
[4]
[5]
[6]
[7] a [8]
[9]
[10]
[11]
b
[12]
Fig. 4 2D scatter plots of dependences of probability of voting for denoising on mean values of P0.5 and input PSNR using the second (a) and third (b) orders polynomials for BM3D filter [13]
IV. CONCLUSION Importance of visual quality of images used and disseminated in infocommunication systems is outlined. Preliminary analysis of prediction of visual quality for denoised images evaluated by observers is carried out. Two filters have been considered – the standard DCT filter and the well-known BM3D filter. Prediction is based on building 2D scatter plots and regression-based obtaining of dependences of visual quality evaluated by observers (characterized by Probvote) on two input parameters – either input PSNR or input PSNR-HVS-M and probability
PIC S&T`2017
[14]
[15]
W. Pratt, Digital image processing. Fourth Edition. N.Y, USA.: Wiley-Interscience, 2007, p. 812. R. Schowengerdt, Remote Sensing: Models and Methods for Image Processing, 3rd ed. Academic Press, 2006, p. 560. A. Pižurica, "Image Denoising Algorithms: From Wavelet Shrinkage to Nonlocal Collaborative Filtering", Wiley Encyclopedia of Electrical and Electronics Engineering, pp. 1-17, 2017. V. Lukin, R. Oktem, N. Ponomarenko and K. Egiazarian, "Image Filtering Based on Discrete Cosine Transform", Telecommunications and Radio Engineering, vol. 66, no. 18, pp. 1685-1701, 2007. K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering”, IEEE Trans. on Image Processing, Vol. 16, Issue 8, pp. 2080-2095, 2007. N. Ponomarenko, F. Silvestri, K. Egiazarian, M. Carli, J. Astola, V. Lukin, "On between-coefficient contrast masking of DCT basis functions", Proceedings of the Third Int. Workshop on Video Processing and Quality Metrics, vol. 3, p. 4, 2007. N. Ponomarenko, V. Lukin and K. Egiazarian, "HVS-metric-based performance analysis of image denoising algorithms", 3rd European Workshop on Visual Information Processing, Paris, France, pp. 156-161, 2011. O.S. Rubel, A.S. Rubel, V.V. Lukin, M.K. Chobanu, “Peculiarities of texture image filtering”, Proceedings of the 18-th International Conference on Digital Signal Processing and its Applications, vol. 2, Moscow, Russia, pp. 916-921, 2016. [In Russian] A. Rubel, O. Rubel and V. Lukin, "Analysis of visual quality for denoised images," 2017 14th International Conference The Experience of Designing and Application of CAD Systems in Microelectronics (CADSM), Lviv, Ukraine, pp. 92-96, 2017. S. Abramov, S. Krivenko, A. Roenko, V. Lukin, I. Djurovic and M. Chobanu, "Prediction of filtering efficiency for DCT-based image denoising", 2013 2nd Mediterranean Conference on Embedded Computing (MECO), Budva, Montenegro, pp. 97-100, 2013. O. Rubel, V. Lukin, S. Abramov, B. Vozel, K. Egiazarian and O. Pogrebnyak, "Efficiency of texture image filtering and its prediction", Signal, Image and Video Processing, vol. 10, no. 8, pp. 1543-1550, 2016. Ponomarenko, N., Jin, L., Ieremeiev, O., Lukin, V., Egiazarian, K., Astola, J., Vozel, B., Chehdi, K., Carli, M., Battisti, F. and Jay Kuo, C. (2015). Image database TID2013: Peculiarities, results and perspectives. Signal Processing: Image Communication, 30, pp.5777, 2015. O. Rubel, V. Lukin and K. Egiazarian, "On prediction of DCTbased denoising efficiency under spatially correlated noise conditions," 2016 13th International Conference on Modern Problems of Radio Engineering, Telecommunications and Computer Science (TCSET), Lviv, Ukraine, pp. 750-754, 2016. O. Rubel, V. Lukin, "An Improved Prediction of DCT-Based Filters Efficiency Using Regression Analysis" in Information and Telecommunication Sciences, Kiev, Ukraine:, vol. 5, no. 1, pp. 3041, 2014. O. Rubel, V. Lukin, K. Egiazarian, "A method for predicting DCTbased denoising efficiency for grayscale images corrupted by AWGN and additive spatially correlated noise", Proceedings of SPIE, Image Processing: Algorithms and Systems XIII, vol. 9399, p. 12, 2015.
October 10-13, 2017
Kharkiv, Ukraine
