Ema Rachmawati, Masayu L. Khodra, Iping Supriana. School of Electrical Engineering and Informatics, Institut Teknologi Bandung. Presented in The 2016 ...
FRUIT IMAGE SEGMENTATION BY COMBINING COLOR AND DEPTH DATA Ema Rachmawati, Masayu L. Khodra, Iping Supriana School of Electrical Engineering and Informatics, Institut Teknologi Bandung
Presented in The 2016 Conference on Fundamental and Applied Science for Advanced Technology
Outline ■ Introduction ■ Proposed System – Color Data + Depth Data – Evaluation Metrics ■ Experimental Result & Conclusion – Data – Segmentation Result
an image is not just a random group of pixels
Color images are more complex than greyscale images
Image segmentation is one of most studied task in computer vision
Image Segmentation Given a set of d-dimensional data
= , , … , ∈ ℝ
we wish to obtain a partition of the data into disjoint nonempty groups
= , , … , where ⋃ = and ∩ = ∅, ≠ [2]
Color Clustering: k-means Given an integer k and data set = , , … , ∈ ℝ , k-means algorithm will
divide
⋃ " = ,
X #
∩
into $
some
""+ = , ,
./ ∈01
,
, … ,
!
,
= ∅ %&' 1 ≤ ≠ ≤ * , by minimizing sum of squared
error (SSE)
with .
=
clusters
−
is Euclidean (6 ) norm and 78 is centroid of cluster
of all data in the cluster
8,
as average
the emergence of RGB-D (Red-Green-Blue-Depth) sensor (i.e Microsoft Kinect, Asus Xtion, and PrimeSense) which relatively cheap, promises to improve performance in many task in computer vision.
Depth data
Depth images ■ How far that part of the image is from the camera ■ Not sensitive to the light conditions at the time it was captured ■ Contain accurate three dimensional information about whatever’s in front of it *) Making Things See, Borenstein 2012
Proposed System
Point Cloud ■ Represents the sets of point the device has measured ■ In three dimensional system: x, y, and z coordinates ■ Point cloud is obtained by converting depth image from RGBD Sensor (eg. Kinect)
Dataset
Fruit images from RGBD Object Dataset [Lai dkk 2011]
7 categories, 32 subcategories, 21284 images
Sample data used in clustering
Point cloud
Color
Evaluation Metrics ■ Silhoutte Coefficient (SC) s i =
b i −a i max a i , b i
with >() is lowest average dissimilarity of to any other cluster, of which is not a member; A() is how well is average dissimilarity of with all other data within the same cluster. The value of B() is supposed to be in: −1 ≤ B ≤ 1
■ MeanMean-squaredsquared-error (MSE)
H
1 C"+ = , "EF − G'F D
with n = number of pixel; "EF = the IJ pixel intensity of our mask image; G'F = the IJ pixel intensity of mask image of RGBD Object Dataset
Silhoutte Coefficient
Value ranging from 0.194 to 0.428
MSE value of segmented image Color only: 0.54
Color + point cloud: 0.279
Conclusion ■ From the experiment conducted, the using of k-means clustering on RGB and point cloud data seems promising. It can be inferred from the decreasing of MSE value of segmentation result. Nonetheless, it still needs further work on improving the determination which cluster represents the correct fruit object automatically, accurately. ■ Further improvement: fixing some errors in identifying the correct cluster that containing fruit object.
THANK YOU
