Toward an efficient objective metric based on

0 downloads 0 Views 9MB Size Report
a University of Poitiers, laboratory SIC, Bvd Marie et Pierre Curie, BP 30179 86962 Futuroscope. Chasseneuil ..... 2007, SPIE: Le creusot. 22. Boust, C., H.
IS&T/SPIE Image Quality and System Performance V, San Jose : États-Unis (2008)

Toward an efficient objective metric based on perceptual criteria Quintard Ludovic a, b, Larabi Mohamed-Chaker a, Fernandez-Maloigne Christinea a {Name}@sic.univ-poitiers.fr , [email protected] a University of Poitiers, laboratory SIC, Bvd Marie et Pierre Curie, BP 30179 86962 Futuroscope Chasseneuil Cedex France b Laboratoire national de métrologie et d’essais , 29 avenue Roger Hennequin, 78197 Trappes Cedex - France ABSTRACT This paper presents the study which at summer carried out for the development of an objective metric without reference, for the evaluation of the quality of the display devices. Initially, a subjective study was carried out, during which various participants were questioned on their visual feeling of various quality factors (hue, saturation, contrast, texture rendering). Following this subjective study, the quality factors which impact the evaluation of the quality of restitution of the display devices are proposed. The development of an objective metric without reference for the evaluation of quality is then carried out. This objective metric without reference which works on quality factors, and integrates some specificities of the human visual system and have an important correlation with the subjective Data of the study. Keywords: Subjective assessment, objective assessment, image quality.

1. INTRODUCTION NowaDays, the visual display devices are increasingly present in our Daily newspaper. Indeed, the interaction with a display device is done as well with a camera, a cellphone, a screen of computer, a television and others. These devices can use different technologies, LCD, OLED [1-3]. The offer around these devices became plethoric, and their quality of restitutions of these devices is very variable from one device to another. Within the framework of the evaluation of the quality of the display devices restitution for a help in decision making, of a purchase or other, the objective of this study, is to develop an objective metric without reference which gives the necessary information related to the quality of restitutions of the display devices. To evaluate the quality of a display device (DD) it is possible to carry out a subjective evaluation [4] or an objective evaluation [5]. In this case, the latter must have results in correlation with the results of a subjective evaluation. Each process contains advantages and disadvantages. The main advantage of objective metric is that they allow an evaluation without constraints for a cost lower than a subjective evaluation. Various subjective methods exist [6] according to what one wishes to measure. Examples of subjective evaluations are presented in [7-9]. Three types of objective metric exist: (1) with reference, (2) reduced reference, (3) without reference. Objective metric with reference need the original signal and the treated signal, this metric is often to use within the framework of the evaluation of video coder [10, 11]. Within the framework of the evaluation of DD, there is necessary to know the device standard to be able to use this metric. At the present time, it is no DD standard. The metrics with reduced reference use knowledge on the signal to carry out their treatments of evaluation. An example of metric with reduced reference is illustrated in [12]. The metrics without reference leave on a not-knowledge of the standard signal, and integreated inside their algorithms some specificities of the human visual system, some examples are provided in [13-15].

The development of an objective metric without reference requires to have knowledge about the treatment which the human being carries out when he perceives an image, moreover, it is necessary to know the quality factor(s) which have importance when the human being carried out an evaluation of quality. These factors of quality must, of course, have a mathematical definition to be built-in metric. The knowledgement that we have on these various points, makes it possible to identify the most important quality factors[16, 17]: ü Hue ü Saturation ü Contrast ü Texture Nevertheless, all these factors do not have necessary the same importance. The objective of this study is to develop an objective metric without reference for the evaluation of the quality of restitution of the DD. In this direction, in order to determine the importance of these various factors, a subjective study was carried out, during which the various participants were questioned about the various factors of quality. In order to be able to connect these various attributes to the quality, they were also questioned on quality. This test has made it possible to identify the factors who have importance for the quality assessment. From these results, it was elaborate an objective metric without reference for the evaluation of the quality of the DD. This metric was then confronted with the subjective data in order to measure the existing correlation between the subjective and objective data. This paper presents, in a first part, the methodology used to carry out the subjective test and the analysis of the results was done. Then, the development of an objectif metric is approached. Finally discussions and conclusions are made.

2. SUBJECTIVE ASSESSMENT This subjective evaluation has for aim, to raise, the important quality factors for the evaluation of the quality of DD restitutions. 2.1. METHODOLOGY Thirty participants carried out the test, 15 men and 15 women, old from 20 to 50 years. All the participants have a normal vision. The vision of the colors was controlled according to the ITU recommendations [6] by the Snellen test. For the evaluation, 5 screens of LCD technologies, coming from the trade general public were prepared. The screens have a diagonal of 15” and a resolutions of 1024x768. For the evaluation, 12 images were selected. These images, illustrated in Annex A, have a resolution of 768x512 and they represent portraits, landscapes and fruits, in annex A the twelve images are illustrated. For the experimentation a wall of screen like illustrated on the figure Fig. 1 was designed. In order to free itself from the position of the screen, they were counterbalanced according to a Latin square.

Fig. 1 : Wall of five LCD screens designed for the subjective evaluation

The participants were divided in five groups, 1 group passing in a configuration of placement of the screens provided by the Latin square. The participants seat on an adjustable chair in height at a distance from 1m20 of the wall of screens. Each screen is calibrated according to recommendations ITU [4, 6], a white point with 80 cd/m ², a temperature of color of 6500kelvin. To measure the screens it was used a spectrometer. The figure Fig. 2 illustrates the bench of measure used. Once the screens was calibrated, their GAMUT and their tone reproduction curve(TRC) were measured. The figure Fig. 3 illustrates the GAMUT obtained for the five screens, while the figure Fig. 4 illustrates the various tone reproduction curve. The walls of the room, where the the experimentation was conducted, have a gray color for the wall, ambient lighting was provides by tubes D50 neon, the light on the screens is uniform and of 60 Luxes.

Fig. 2 : Bench of measure

Fig. 3 : Gamut of the different screnn. Obtained with the bench of measure.

Fig. 4 : Tone Reproduction Curve of the different screens.

The participants were questioned on 5 questions: (1) the contrast of the image (2) the hue of the image (3) the saturation of the image (4) the texture of the image (5) the global quality of the image. When a participant enters in the evaluation room, he takes place on the seat. The experimenter explains the various questions and gives to the participant a questionnaire, and a sheet on which the various questions are explained. The participant answers the questions for the 5 screens and the 12 images whose the order of passage is random. The following explanation was then given to the participant: “The same image will be posted at the same time on the five screens. You must answer the various questions for each couple screen/image. You answer all the questions of a couple screen/image before changing “couple”. Once you finished answering for an image, pass to the following image by clicking on the space bar.

2.2. RESULTS The collected data were analyzed by a regression approach in order to determine the quality according to the other parameters, contrasts, hue, saturation and texture. Due to the correlation between the various predictive variables, when an analysis of multiple linear regression was carried out a multicollinearity appears. The analysis is then carried out with a partial least square regression (PLS) which makes it possible “to be freed” the correlation between the predictive variables [18]. The results show that the evaluation of the quality of the DD respects the equation Eq 1.

Q = 0.1728 * Contrast + 0.551* Hue + 0.1719 * Saturation + 0.09 * Texture

Eq (1)

This equation is correlated with the subjective data with a R ² 0.908 ????. It appears that the hue is the main quality factor which have the most importance in the evaluation of the quality of the display device restitution quality.

3. OBJECTIFIES ASSESSMENT

3.1. INTRODUCTION The goal of an objective metric without reference which was developed is to inform about the quality of the DD. The subjective study which provides the equation Eq1, may be explain in various manners. A manner is to seek by the objective metric how to find, in a calculable way, the value of the hue, contrasts, saturation and texture provides by the participants during the subjective evaluation. This seems rather not easily realizable, the context of the image certainly plays an important role. Another method, which that is employed here, is to take the equation like an indicator of the big factors and to work on the various quality factors and to give different weights according to the factor importance. Finally, in the metric, each factor quality seeks to be correlated with the quality and it is the information of these various

correlations which, once balanced provides the quality of the various DD. The quality note then obtained, makes it possible to make a classification of the various DD. It will be presented the work completed around contrast and hue. In both cases, the preprocessing that undergones the image is the same one, it makes it possible to modify the digital image in order to treat only the perceveid image. It was also mentioned the fact that an objective metric incorporates inside its algorithm of treatment some specificities of the visual system humain, these specificity will be first approached.

3.2. IMPORTANT CHARACTERISTIC OF THE HUMAN VISUAL SYSTEM The human visual system is mainly composed of two organs, the retina by the eye and the brain and more particularly the V4 area. The figure Fig. 5 illustrate the visual pathway and the retina.

Fig. 5 : At the left àWay of the visual system ; At the right à retina

Schematically, the retina puts in forms the signal which is then analyzed in the V4 area. The retina is made up of cones and of rods, the cones are responsible for the vision of the colors. Rods of the diurnal vision [17]. Three kinds of cone are present, the cones L, M and S, each cone having a specific curve of absorption. The color information is then separate on three axes, an achromatic axis, and two chromatic antagonistic axes, yellow-blue and red-green to be transmitted by the optical nerve to the visual brain. Numerous perceptive color spaces decompose the information in the same way, while trying to integrate perceptive distances in the representation. Distance according to which the difference between two objects of the same chromaticity is described by the psychometric lignhtness function (PSF). The PSF illustrate by the figure Fig. 6 represents the function for which a delta of lignhtness is always perceived in the same way.

Fig. 6 : At the left àPsychometric lightness function (PSF), At the rightà Contrast Sensibility Function (CSF)

Another aspect of the retina is that the space distribution of the cones and rods do not allow to see all. A function named sensibility function contrast (CSF) illustrated by the figure Fig. 6 explain the fact that the signal is felt according to its space frequency. Temporal frequencies are also important but we didn’t take into account their effect in this study. Once the signal was formatted, it circulates along the optical nerve and arrives in the brain. It appears that we are more or less sensitive to the orientation of the signal. Moreover, the human being would have a certain memory of colors [16, 19].

3.3. OBJECTIF METRIC 3.3.1. PREPROCESSING When someone judges the quality of a DD, the image which it observes differs from that which is numerically sent. This difference is due to the characteristics of the DD. These characteristics which are for the principal the GAMUT, and the TRC will be used to modify the numerical image in a numerical perceived image. The figure Fig 7 illustrates the synoptic of the transformation.

Fig. 7 : Synoptic of the transformation

The image is then filtered by a CSF filter in order to treat only the image perceived in the algorithm of treatment.

3.3.2. OBJECTIF METRIC: CONTRAST METHODOLOGY The image is transposed in a perceptual color space AC1C2, this transformation is carried out by taking into account the spectrum of the concerned screen. Then, the image is decomposed into wavebands and orientations (ou orientation wavebands ?). A contrast according to the definition of Peli [20] is then calculated. The algorithm of this metric is clarified in [13, 21]. RESULTS The correlation obtained between the subjective data and the objective ones is about 90%.

3.3.3. OBJECTIFY METRIC: HUE METHODOLOGY For the hue, we used the work of Yendrikovish and Boust [22, 23]. They consider that a color is more or less appreciated according to whether it would be near or not to a hue reference. The hue references are the blue (sky), the green (grass) the beige (skin) and are given in a space color u*v* for one illuminant D65 or D50. One of the problems which one encounters when the hue is treated is to isolate the various areas of the picture. Indeed, in precedent work only a part of the image is treated, generally by a rectangular selection of the zone. Here, it was made the choice to treat all the image, on the one hand, because the images used are complex and, on the other hand, it is made the assumption according to which the judgment is carried out on the whole of the image. The image is then classified in using the supervising algorithm K-means, of which the number of seeds are chosen by the supervisor. Examples of classification are illustrated by the figure Fig. 8.

Fig. 8 : Example of classification

An algorithm containing matrix of adjacences is then used, the segmented image is used as mask to identify the various areas, calculations are carried out on the areas of the perceived image. For each area, the matrix contains information on the pixels of the image which composes it and the areas which are adjacent for him. This way of processing the information makes it possible to calculate the values of an area and the variations compared to the adjacent areas. In order to be able to compare this work with the work done in [22, 23] the image was passed in the color space L u' v' by using the characteristics of the white of the monitor. The figure Fig. 9 illustrate the synoptic. Once the image in the plan L*u* v*, the average and the standard deviation of each area is calculated thus the differences between the areas.

Fig. 9 : Synoptic for the calculation of the different area

The first work was to compare in the images which included sky, characters and grass, the values of calculated hue compared to the values of the hue provided in the work describe above. It was also carried out a work on the averages of the differences between the areas. Indeed, when the human being looks at an image, the various objects which compose the image are discriminated. The hue of an object is not necessary only compare with a hue memory but can also compare with the hue which surrounds the image. And this by the phenomena of chromatic adaptation due to the SVH [24]. From this idea, the median value of the variations which composes an image was calculated.

FIRST RESULTS In this part, the results of the comparisons with the data of [23] are described.

The figures Fig. 10 illustrate the results obtained for 3 images. In fact, the 3 images which were easily exploitable to carry out this comparison. Only the results obtained with the data of Yendrikovish are illustrated, nevertheless, the same results are obtained for the data of Boust. In Y-coordinate, it was put the subjective notes obtained during the evaluation with a confidence interval at 95%. In X-coordinate there is the row obtained during the objective evaluation. More the distance L² between the hue reference and the calculated hue is weak better is the rank. The line is an indication in order to show the presence of a correlation or not. The coefficients of correlation of Pearson are: - 0.95% for image 5 ; 0.17% and 0.073% for image 6 and 15. Picture 5 foliage-- Yendri

Picture 6 sky -- Yendri

5

4.5

4.5

4

4

3.5 3

3

Subj

Subj

3.5

2.5

2.5 2

2 1. 5

1. 5

1

1

0.5

0.5 0 0.5

0 1

1. 5

2

2.5

3

3.5

4

4.5

5

5.5

0.5

1

1. 5

2

2.5

Obj

3

3.5

4

4.5

5

5.5

Obj

Fig. 10 : Results for the comparaison with the Yendri data

SECOND RESULTS In this part, the results of the averages of the differences between area are presented. The figure Fig. 11 illustrates an example of results obtained for various images. In Y-coordinate, there are the subjective notes obtained during the evaluation, the bar represents the confidence interval at 95%. In X-coordinate it has the rank obtained during the objective evaluation. More the average of the variations of hue between areas is weak, better is the assigned value of the rank. The line is used to illustrate the existing correlation between the subjective and objective results. The coefficients of correlation of Pearson obtained are: -0.82%,-0.87%,-0.78%,-0.95% respectively for images 10,11,12 and 4.

-a-

-b-

-c-

-d-

Fig. 11 : Results of the correlation between the quality and the average difference between the hue of the area of an image

3.4. OBJECTIF METRIC: SYNOPTIC The synoptic one illustrated by the figure Fig. 12 described the operation of this objective metric without reference for the evaluation of quality of DD. It works on the contrast and the hue of the perceveid image.

Fig. 12 : Developped metric synoptic

4. DISCUSSIONS The purpose of this study was the development of an objective metric without reference for the evaluation of the quality of the display devices (DD). To carry out the metric, a subjective study was done in order to determine the importance of different quality factor(s) for the evaluation. The data of the subjective study, once realized, were analyze using a PLS regression. Following what, a work was carry out around the contrast and of the . The subjective study showed the importance of the various factors quality for the construction of an objective metric without reference. Most important of them being hue, following by the contrast and the saturation. This indication makes it possible to work on these various quality factors and more particularly on the contrast and the hue. The goal was to establish correlations between the subjective results and the objective ones, obtained by the computations on the hue and the contrast independently. Indeed, the goal is not to find in an objective way how to compute a subjective note of 3.5. Because a note of 3.5, subjective, may be allowed to a multitude of images having quite different characteristics (number of areas, objects, etc…). But the goal is to obtain a correlation for the each quality factor, contrast and hue, this double information of correlation may be combined to obtain the final objective evaluation of the quality. The work completed on the contrast which makes following preceding work [13, 21] is calculated according to a modified version of contrast given by Peli in [20] and integrates a decomposition of the image in wavebands of orientations to take into account the sensitivity of the SVH within the signal orientation. The greatest is the contrast is important (until a certain limit) the best is the quality of the image. A correlation about 90% between the subjective results and objectives result is obtained. The work on the hue is decompose in two parts. It is carried out in the L*u*v* color space. The first work was to compare the values of hue obtained on each screen with the values of the hue memories referred in [23]. This work was not very profitable. Indeed, the correlation obtained between the subjective results and the objective result was rather weak. Moreover, this hue comparison can be carried out only on the hue of skin, sky and grass, which does not represent the whole of the images and video available. Nevertheless, this work does not rely in cause the work of [23] because the various values of hue calculated are always in the standard deviations mentioned by this work.

On the basis of the idea that an image is globally evaluated…. ???? That the ??? SVH carries out a chromatic adaptation, it was made the choice to test if the distance separating the areas between them could be correlated with the subjective data. Indeed, on working only on the L*u*v* color space, it can be supposed that more the variations of hue are weak, and more the image is homogeneous. The results which were obtained showed a good correlation between the subjective tests and the data obtained when the comparaison of the average of the variations of hue between areas of an image are done. From these two algorithms which correlates significantly with the subjective results, it is possible to combine this two information to obtain an evaluation of the the display device quality. Moreover, it would be interesting to investigate the field of the visual interest to provide more or less important weights to the various areas.

5. CONCLUSION The work presented here allowed the development of an objective metric without reference working on two quality factors, the hue and the contrast. A subjective study was first carry out. The data of this last one was analyze using a PLS regression. This regression releves the fact that the hue and contrast are the two main factors for a global quality assessment. An algorithm of treatment for the development of objective metrics was then developed around the two quality factors . The first algorithm which is articulated around contrast works with a modified definition of contrast according to Peli. The second, on the hue, uses an adjacent matrix of areas to determine the existing variations of hue between the areas. The results show a good correlation of these two algorithms.

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

Tombling, C. and M. Tillin, Innovations in LCD technology. Synthetic metals, 2001. 122: p. 209-214. Dobbertin, T., et al., OLED matrix displays : in-line process technology and fudamentals. Thin Solid Films, 2003. 442: p. 132-139. Katyama, M., TFT-LCD technology. Thin Solid Film, 1999. 341: p. 140-147. UIT, UIT-R Recommandation P.910 Subjective video quality assessment methods for multimedia applications. 1996, International Telecommunications Union. VQEG, VQEG : Final report from the video quality expert group on the validation of objective models of video quality assessment. 2000, Video Quality Expert Groups. UIT, UIT-R BT.500-11 Methodology for the subjective assessment of the quality of television pictures. 2002, International Telecommunication Union. Aldridge, R. and D. Pearson, A calibration method for continuous video quality (SSCQE) measurements. Signal Processing Image Communication, 2000. 16: p. 321-332. Dijk, A.M.V. and J.-B. Martens, Subjective quality assessment of compressed images. Signal Processing, 1997. 58: p. 235-252. Petrovic, V., Subjective test for image fusion evaluation and objective metric validation. Information Fusion, 2007. 8: p. 208-216. Quintard, L., M.C. Larabi, and C.F. Maloigne, Subjective and Objective Study of Different Video Coders: Application to Video Surveillance, in CGIV. 2006: Leads. Hekstra, A.P., et al., PVQM. Signal Processing Image Communication, 2002. 17: p. 781-798. Callet, P.L., et al., No-reference and reduced reference video quality metrics for end to end QoS monitoring. IEICE transactions on communications, 2006. E89: p. 289-296. Bringier, B., et al., No-Reference perceptual quality assessment of colour image, in EUSIPCO. 2006: Florence.

14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24.

Fu-zheng, Y., et al., A no-reference blocking artifact metric for B-DCT video. Journal of Zhejiang University SCIENCE A, 2006. 7: p. 95-100. Brandao, T. and M.P. Queluz, No-reference image quality assessment based on DCT domain statistics. Signal Processing, 2007. Derefeldt, G., et al., Cognitive Color. COLOR Research and application, 2004. 29(1). Lang, H., Colour and Its reproduction, ed. M.-S. Verlag. 2001. Wold, S., M. Sjöström, and L. Eriksson, PLS-regression: a basic tool of chemometrics. Chemometrics and Intelligent Laboratory systems, 2001. 58: p. 109-130. Seliger, H.H., Measurement of Memory Color. Color research and application, 2002. 27(4). Peli, E., Contrast in complex images. JOSA, 1999. 7(10). Bringier, B., L. Quintard, and M.C. Larabi, Quality assessment for colour reproduction using a blind metric, in Quality Control by artificial Vision. 2007, SPIE: Le creusot. Boust, C., H. Chahine, and F. Viénot, Color Correction Judgements of Digital Images by Experts and Naive Observers. Processing Images Image Quality Capturing Images Systems Conference, 2003. 6: p. 4-9. Yendrikhovskij, S.N., Color reproduction and the naturalness constraint. 1998, Technische Universiteit Eindhoven, The Netherlands. Thompson, E., Colour Vision, ed. R. London and N. York. 1995.

ANNEX A

Fig. 13: Images used for the evaluation. From the top left to the bottom right [ 3 4 5 6 9 10 11 12 13 14 15 16]