2009 International Conference of Soft Computing and Pattern Recognition
Automated Grading of Palm Oil Fresh Fruit Bunches (FFB) using Neuro-Fuzzy Technique
Nursuriati Jamil, Azlinah Mohamed, Syazwani Abdullah Faculty of Computer & Mathematical Sciences Universiti Teknologi MARA Shah Alam, Selangor, Malaysia
[email protected],
[email protected],
[email protected] including using image processing [5][7][8][10][11][14][15][16] computer visions[3][9][10] and artificial neural networks[5][6][9]. In this paper, automated grading of palm oil fresh fruit bunches (FFB) using RGB color model and neuro fuzzy algorithms are investigated. Even though RGB color model and neural network have been popularly employed by [12][13][14][15][16][17] for palm oil grading, none of the work has attempt to integrate fuzzy logic algorithm.
Abstract—Automated fruit grading in local fruit industries are gradually receiving attention as the use of technology in upgrading the quality of food products are now acknowledged. In this paper, outer surface colors of palm oil fresh fruit bunches (FFB) are analyzed to automatically grade the fruits into over ripe, ripe and unripe. We compared two methods of color grading: 1) using RGB digital numbers and 2) colors classifications trained using a supervised learning Hebb technique and graded using fuzzy logic. A total of 90 images are used as the training images and 45 images are tested in the grading process. Overall, automated grading using RGB digital numbers produced an average of 49% success rate, while the neuro-fuzzy approach achieved an accuracy level of 73.3%.
II.
Automated fruit grading, palm oil, color classification, RGB color model, neuro-fuzzy.
I.
INTRODUCTION
Automated grading of agricultural products has been getting special attention of late as the demand for higher quality food products produced within a shorter period of time has increased [1]. Market grade of quality food products are determined by their multiple properties: flavor, appearance and texture [2]. While flavor may be measured using chemical compounds to determine the sweetness or acidity, texture properties such as firmness and mouth feel are difficult to measure. In automated fruit grading, appearance (shape, color, size and bruises) is generally utilized to classify the fruit’s grade. Abdullah et al. [3] stated that color is mainly used to assess grading as it serves as instant indicators of good or bad product quality and consumers tend to associate color with flavor, safety, nutrition and level of satisfaction. Surface color is widely used for maturity and quality assessment, and is probably the most common characteristic used in the selection and harvesting of many fruits [4]. In Malaysia, researches in automated fruit grading have gradually caught the interest of many fruit-producing industries. The earliest automated fruit grading were reported in palm oil [12][13][17] and pepper berry [6] industries and has recently spread to other fruit industries such as starfruit [9][11], watermelon[5], papaya[8], jatropha [7] and mango[10]. Variations of techniques are employed in classifying the degree of fruits ripeness automatically 978-0-7695-3879-2/09 $26.00 © 2009 IEEE DOI 10.1109/SoCPaR.2009.57
PALM OIL FFB GRADING
Palm oil FFB is currently graded manually by human inspections based on the percentage of detached fruitlets from the FFB [18]. For example, a FFB is classified as overripe if it has more than fifty percent (50%) of detached fruitlets but with at least ten percent (10%) of the fruits still attached to the bunch at the time of inspection at the mill. Even though color can be used to classify the FFB, it is difficult to differentiate darkish red (i.e. overripe), purplish red (i.e. under ripe) and reddish orange (i.e. ripe) based on visual inspection. Furthermore, manual inspection is very subjective as different human graders classify differently and expert grader may fail to articulate the grading criteria properly. Dissatisfaction and dispute among estate owners and factories supervisors are frequent due to improper grading [19]. Therefore, to increase the accuracy and quality of FFB grading in palm oil mills, an automated FFB grading based on the FFB outer surface color is proposed. III.
MATERIALS AND METHODS
As stated by [20], the motivation of using neuro fuzzy techniques is based on the fact that the color values of FFB may be trained to handle ambiguity by a learning algorithm which promises better performance. Neuro fuzzy is also known for its ability to provide a means of translating qualitative and imprecise information into quantitative (linguistic) terms. This paper compared two grading methods: (1) color grading using RGB digital numbers (DN) and (2) color grading trained using a supervised learning Hebb technique and classified using fuzzy logic. The steps taken for the latter approach includes image acquisition, color feature extraction, data training and lastly color classification using fuzzy logic algorithms.
251 245
A. Image Acquisition Images of palm oil FFB are taken at the assessment field of Golden Hope Sepang Oil Mill, under supervision by the human grader. As soon as the lorry emptied the FFB, the grader manually classified the FFB into over ripe, ripe and unripe. A total of 90 images are taken under direct sunlight using a commercial Canon Powershot A430 digital camera with 180dpi optical resolution around noon time. They are then converted into JPEG format and resized to 640x480 dimensions. Even though we are aware of the lighting factor that may cause color and illumination inconsistencies by using a low-cost digital camera, this is a cost constraint limitation that we consider during our grading experiments.
C. Data Training Using Hebb Learning Algorithm As mentioned above, to classify FFB it is required to identify the most appropriate classification range. Previous studies show different range were used in classification of FFB and resulted in different accuracy level [18][2]. Therefore, in this paper we incorporated range gathered from human graders with an identified range used in a related automated grading study [21]. Besides that, the training would be able to reduce human error in extracting or inputting data. The training focuses in the implementation of Hebb algorithm to identify the best-fit value to represent RGB color of FFB image. Forty five images are used to calculate the value of each color for each classification. The mean of RGB colour value produced from both sources mentioned above, would be used as an input data for Hebb algorithm. The architecture as shown in Fig. 2 shows that each of the layers consists of input neurons and the output neurons, representing single layer neural network. These neurons are connected with weights and bias, which are represented as w and v. The result of the best-fit values for each grade is derived using the following formula:
B. Color Feature Extraction This paper investigated on FFB grading using RGB color model only. Work on another color model, L*a*b were conducted in [22]. Prior to color analysis of the FFB, unwanted elements or noise in the image has to be removed. Since the region of interest is the FFB image, background noise is automatically extracted by applying a user-created mask over the FFB image. The mask is created by converting the FFB image into binary and enhanced using several morphological operations. The mask is then applied over the color FFB image to remove the background. Fig. 1 shows the result of performing the background extraction.
(x1 ∗ w1 ) + (x 2 ∗ w 2 ) + (b ∗ v) = y
The adjustment of weight and bias increases the learning performance and produces best-fit value which would contribute to the classification accuracy.
b Figure 1. Background extraction of FFB image.
Ripe interval =
x red/green/blue ± σred/green/blue
x red/green/blue ± σred/green/blue
Underipe interval =
x red/green/blue ± σred/green/blue
v w
x1
The main purpose of the proposed automated grading system is to classify the FFB into over ripe, ripe and unripe categories. To achieve this, we need to obtain a range of red, green and blue DN for each of the mentioned categories. These ranges of values are used to classify the FFB into their proper categories. A total of 90 images (i.e. 30 overipe, 30 ripe and 30 unripe FFB) are used to calculate the average, x and standard deviation, σ of the RGB bands. The range of red, green and blue band for each category are then derived as follows: Overipe interval =
(4)
y
w x2 INPUT LAYER
OUTPUT LAYER
Keys: X- Mean of RGB color input
b – Bias
wadjusted weight
v – Adjusted bias
y - Output
Figure 2. Architecture of Hebb learning algorithm.
(1)
D. Color Classification Using Fuzzy Logic Color classification are done in four steps: fuzzification, rule evaluation, aggregation of the rule outputs and defuzzification. In fuzzification, the best-fit value obtained from data training would be the input to the fuzzy classified data set in identifying the appropriate classification for the data image. Using the fuzzy approach, membership function is calculated in linear form resulted with 4 fuzzy sets consisting of three inputs, which are the RGB colors and one
(2) (3)
246 252
output, that is Category. An example of the fuzzy sets are illustrated in Fig. 3.
range, the FFB is considered as uncategorized. Results of the grading system are evaluated against the human graders to measure accuracy. It shows that the automated grading system achieved 60% accuracy in grading over ripe FFB, 27% for ripe FFB and 60% for unripe bunches. TABLE II.
RANGE OF RGB DN USED FOR OVER RIPE, RIPE AND UNRIPE FFB
Over ripe Min Max
R 136.07 154.60
Min Max
R 126.81 145.85
Min Max
R 65.64 101.42
B
G 82.81 97.79
50.93 70.47
Ripe B
G 81.30 95.09
52.45 72.02
Unripe
The fuzzy sets obtained produces 21evaluation rules in classifying the fruit’s category. Examples of the rules are demonstrated as follows: FUZZY SETS RULES USED FOR CLASSIFICATION
Rule 16 If red is large And green is small And blue is small And blue is medium Then category is ripe
Rule 3 If Red is small And red is medium And green is small And Blue is small Then category is unripe Rule 18 If red is large And green is medium And blue is medium Then category is over ripe
Automated Grading Based on RGB DN 155 Digital Numbers
Rule 1 If Red is small And Green is small And Blue is small Then category is unripe
135 115 95 75 55 35 15 0
5 10 Over ripe
15 Red
IV.
28.30 51.63
Results of the grading showed that ripe FFB has a low accuracy rate and we plot a graphical representation of the results in Fig. 4. Human graders have earlier classified FFB 1 to 15 as over ripe FFB, 16 to 30 as ripe FFB and 31 to 45 as unripe FFB. The figure clearly illustrates that the difference of RGB digital numbers for over ripe and ripe FFB are not well-defined. Thus, the ripe FFB are misclassified as over ripe FFB. For unripe FFB, a distinct threshold value seems to exist at DN value of 120 for red band and 75 for green band. The blue band, however does not show any definite threshold value for all three categories. Due to the vagueness of the RGB DN in classifying the FFB, we proposed to employ neuro-fuzzy technique.
Figure 3. Fuzzy set for fuzzy logic algorithm.
TABLE I.
B
G 41.89 66.17
RESULTS AND DISCUSSIONS
20
Ripe FFB
25
Green
30
35
40 Unripe
45
Blue
Figure 4. RGB digital numbers for over ripe, ripe and unripe test images.
In this paper we compare automated grading of FFB using the red, green and blue DN and neuro-fuzzy algorithm. Forty-five FFB images are used as testing and the results are discussed in the following sections.
B. Color Grading using Neuro-Fuzzy System In the neuro-fuzzy system grading the minimum and maximum RGB range of values used to classify the test images are derived from the formula below:
A. Color Grading using RGB Digital Numbers The RGB range of interval values for over ripe, ripe and unripe categories derived from Eq. 1-3 is presented in Table 2 and is used to automatically classify the test images. If the computed RGB DN does not fall within any of the specified
Minred/green/blue = min( mean (imgred/green/blue))
247 253
(5)
Maxred/green/blue = max( mean (imgred/green/blue))
(6)
REFERENCES
Subsequently, results of the neuro-fuzzy system are evaluated against the human graders to measure accuracy. It shows that the automated neuro-fuzzy grading system achieved 80% accuracy in grading over ripe FFB, 73.3% in grading ripe FFB and 66.7% for under ripe FFB. The improvement of the results can be seen as the best-fit values using Hebb algorithm are identified and used for the classification. These best-fit values are as shown in Fig. 5. Fig. 5 shows that there is some values out of the default range may cause some noise in the output. However, these noises are not very apparent to affect the grading accuracy when compared to the human grader. The intensity of the RGB colors are more evenly distributed as compared to Fig. 2 above. Overall, the neuro-fuzzy system gives 73.3% accuracy in classifying the FFB.
[1]
[2]
[3]
[4]
[5]
C o lor Dens ity
A utomated G rading B as ed on NFS
285 255 225 195 165 135 105 75 45 15
[7] red green
[8]
blue
[9] 1
5 Over
Figure 5.
[6]
9 13 17 21 25 29 33 37 41 45 FFB Ripe
Unripe
[10]
RGB Neuro-fuzzy for over ripe, ripe and unripe best-fit value
V.
[11]
CONCLUSIONS AND RECOMMENDATIONS
[12]
Our study has proven that by using neuro-fuzzy algorithm, the accuracy of palm oil FFB automated grading has been increased by 24% compared to using RGB digital numbers. A prototype using Hebb algorithm to train the color classifiers was developed and 45 palm oil images are tested in our experiment. Future work using other learning algorithm will be investigated, and improvement on color classification is currently being addressed using clustering algorithms. Color of FFB has been shown to have positive correlation to the oil content, thus the popularity of using this feature in automated grading. However, potential work should also address employing the percentage of FFB loose fruitlets for automated grading.
[13]
[14]
[15]
[16]
ACKNOWLEDGMENT
[17]
We would like to thank the staffs at Golden Hope Sepang Oil Mill especially Mr Cheah Bian Chong, Quality Assurance Supervisor and Mr. Harizal Omar, Assistant Engineer for their full cooperation and support throughout our research ordeal.
[18]
248 254
J.B. Njoroge, K. Ninomiya, N. Kondo, and H. Toita, "Automated fruit grading system using image processing," SICE 2002. Proceedings of the 41st SICE Annual Conference , vol.2, no., pp. 1346-1351 vol.2, 57 Aug. 2002. Y. Saito, T. Hatanaka, K., Uosaki, and H. Shigeto, “Neural network application to eggplant classification,” LNAI 2774, pp. 933–940, 2003. M. Z. Abdullah, L. C. Guan, K. C. Lim and A. A. Karim, “The applications of computer vision system and tomographic radar imaging for assessing physical properties of food,” J. Food Engineering, vol. 61, pp. 125-135, Jan 2004. C.J. Studman, “Computers and electronics inn postharvest technology – A review,” Computers and Electronics in Agriculture, vol 20, pp. 109-124, 2001. M.S.B. Shah Rizam, A.R. Farah Yasmin, M.Y. Ahmad Ihsan, and K. Shazana, “Non-destructive watermelon ripeness determination using image processing and artificial neural network (ANN),” Proc. World Academy of Science, Engineering and Technology, vol. 38, pp. 542546, Feb 2009. A. Abdesselam, R.Choo Abdullah, “Pepper berries grading using artificial neural network,” Proc. TENCON 2000, vol. 2, pp. 153-159, 2000. Z. Effendi, R. Ramli, J. Abdul Ghani, and Z. Yaakob,” Development of Jatropha Curcas color grading sysytem for ripeness evaluation,” European Journal of Scientific Research, vol. 30, no. 4, pp. 662-669, 2009. S. Riyadi, M. M. Mustafar, A. Hussain, and A. Hamzah, “Papaya fruit grading based on size using image analysis,” Proc. International Conference on Electrical Engineering and Informatics, pp. 645-648, June 2007. M.Z. Abdullah, J. Mohamad-Salleh, A.S. Fathinul-Syahir, and B.M.N. Mohd-Azemi, “Discrimination and classification of fresh-cut starfruits (Averrhoa carambola L.) using automated machine vision system,” J. Food Engineering, vol. 76, pp. 506-523, 2006. C.C. Teoh, and A.R.M. Syaifudin, “Use of image analysis for grading size of mango,” ISHS Acta Horticulturae 710, Leuven, pp. 485-490, 2006. M.M Mokji., and S.A.R. Abu Bakar, “Starfruit classification based on linear hue computation, Elektrika Journal of Electrical Engineering, vol. 9, no. 2, pp. 14-19, 2007. W.I. Wan Ismail, M.Z. Bardaie, and A.M. Abdul Hamid, “Optical properties for mechanical harvesting of oil palm FFB,” J. Oil Palm Research , vol. 12, no. 2, pp. 38-45, Dec 2000. R. Shariff, A. Adnan, R. Mispan, S. Mansor, R. Halim, and R. Goyal, “Correlation between oil content and DN values,” Proc. Map India, pp. 1-6. 2002. T.S.Y. Choong, S. Abbas, A.R. Shariff, R. Halim, M.H.S. Ismail, R. Yunus, S. Ali, and F-R. Ahmadun, “ Digital image processing of palm oit fruits”, International J. Food Engineering, vol 2, no. 2, pp. , 2006. M.S.M Alfatni, A.R. Shariff, H.Z. Mohd Shafri, O.M.B. Saaed, and O.M. Eshanta, “Oil palm fruit bunch grading using red, green and blue digital number,” J. Applied Sciences, vol. 8, no. 8, pp. 14441452, 2008. S.K. Balasundram, P.C. Robert, and D.J. Mulla, “Relationship between oil content and fruit surface color in oil palm,” J. Plant Sciences, vol. 1, no. 3, pp. 217-227, 2006. M.Z. Abdullah, L.C. Guan, and B.M.N. Mohd-Azemi, “Stepwise discriminant analysis for colour grading of oil palm using machine vision system,” Transaction of the IChemE, col. 79, pp. 223–231, 2001. Manual Penggredan Buah Kelapa Sawit: http://161.142.157.2/pnp/gredffb.htm
[22] R. Jaafar, N. Jamil, A. Jaafar, and B. Abdullah, “Image analysis of Fresh Fruit Bunches (FFB) using low cost vision system,” Proc. International Conference on Advances in Mechanical Engineering, pp. 20-25, July 2009.
[19] M.F. Shah, “Settlers report abuse in oil palm fruit grading,” The Star, April 10, 2009. [20] A.Keles, A.S. Hasiloglu, Al. Keles, and Y. Aksoy, “Neuro-fuzzy classification of prostate cancer using NEFCLASS-J,” Computers in Biology and Medicine, vol. 37, no. 11, pp. 1317-1628, Nov 2007. [21] S. Zuraka, “Grading of FFB palm oil based on global colour,” BSc. Thesis, Faculty of Information Technology & Quantitative Sciences, Universiti Teknologi MARA, Oct 2007.
249 255
