Mar 1, 2017 - GDBN model. 1. The probabilistic generative model;. Presenter: Haris Ahmad Khan â Mar 01th. , 2017. Generative vs. Discriminative DBN.
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
VISAPP 2017
Generative vs. Discriminative Deep Belief Netwok for 3D Object Categorization Nabila Zrira*, Mohamed Hannat*, El Houssine Bouyakhf* and Haris Ahmad Khan** *LIMIARF, Mohammed V University, Faculty of Sciences Rabat, Morocco **NTNU, Norwegian University of Science and Technology, Gjøvik, Norway
01 March 2017 Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
1/29
Outline
1
Introduction
2
Method Overview
3
3D Object Categorization
4
Experimental Results
5
Conclusion and Future Work
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Context 3D Object categorization
RGB-Depth images
Kinect Camera
Point Cloud
PCL descriptors
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Neural Network
Generative vs. Discriminative DBN
3/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Problematic
How to ensure a good 3D object categorization approach in real-world object environment?
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
4/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Contribution
A new 3D object categorization pipeline based on VFH and GDBN/DDBN;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
5/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Contribution
A new 3D object categorization pipeline based on VFH and GDBN/DDBN; The comparison of GDBN and DDBN architectures;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
5/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Contribution
A new 3D object categorization pipeline based on VFH and GDBN/DDBN; The comparison of GDBN and DDBN architectures; The use of CD, PCD and FEPCD sampling.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
5/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Our proposed approach
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
6/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
VFH descriptor
Global descriptor
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
7/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
VFH descriptor
Global descriptor Computed for a whole cluster of object
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
7/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
VFH descriptor
Global descriptor Computed for a whole cluster of object Consists of two components: 1 2
a surface shape component; a viewpoint direction component.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
7/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
VFH descriptor
Global descriptor Computed for a whole cluster of object Consists of two components: 1 2
a surface shape component; a viewpoint direction component.
VFH size is equal to 308 bins
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
7/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
VFH descriptor: example
(a)
(b)
Figure : (a) 3D point cloud of food box. (b) VFH descriptor of food box point cloud.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
8/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Restricted Boltzmann Machines (RBMs) RBM model 1 First layer: contains visible units (x); 2
Second layer: contains hidden units (h);
Figure : Generative RBM model.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
9/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Restricted Boltzmann Machines (RBMs)
The energy function of an RBM is defined as: 0
0
0
E(x, h) = −b x − c h − h W x
(1)
where: W represents the symmetric interaction term between x and h; b and c are vectors that store the visible (input) and hidden biases (respectively).
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
10/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Generative Deep Belief Network (GDBN)
GDBN model 1 The probabilistic generative model;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
11/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Generative Deep Belief Network (GDBN)
GDBN model 1 The probabilistic generative model; 2
Training sequence of RBMs;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
11/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Generative Deep Belief Network (GDBN)
GDBN model 1 The probabilistic generative model; 2
Training sequence of RBMs;
3
DBN with l hidden layers trains l RBMs;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
11/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Generative Deep Belief Network (GDBN)
GDBN model 1 The probabilistic generative model; 2
Training sequence of RBMs;
3
DBN with l hidden layers trains l RBMs;
4
Layer-wise algorithm;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
11/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Generative Deep Belief Network (GDBN)
GDBN model 1 The probabilistic generative model; 2
Training sequence of RBMs;
3
DBN with l hidden layers trains l RBMs;
4
Layer-wise algorithm;
5
”Fine-tuning” the resulting weights of all layers together.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
11/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Generative Deep Belief Network (GDBN)
Figure : Generative DBN architecture (GDBN).
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
12/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Discriminative Restricted Boltzmann Machines (DRBMs)
DRBM model DRBM trains a density model by means of a particular RBM consisting of two sets of visible elements (inputs x = (x1 , ..., xd ) and the target y).
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
13/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Discriminative Restricted Boltzmann Machines (DRBMs) 0
0
0
0
0
E(y, x, h) = −h W x − b x − c h − d ~y − h U ~y
(2)
with Θ = (W, b, c, d, U ) is the set of parameters and ~y = (1y=i )ci=1 for C classes.
Figure : Discriminative RBM model (DRBM). Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
14/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Discriminative Deep Belief Network (DDBN)
DDBN model 1 DRBM in the last layer is used as a classifier;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
15/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Discriminative Deep Belief Network (DDBN)
DDBN model 1 DRBM in the last layer is used as a classifier; 2
DDBN trains a joint density model through DRBM;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
15/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Discriminative Deep Belief Network (DDBN)
DDBN model 1 DRBM in the last layer is used as a classifier; 2
DDBN trains a joint density model through DRBM;
3
The label which contains the least energy is selected;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
15/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Discriminative Deep Belief Network (DDBN)
DDBN model 1 DRBM in the last layer is used as a classifier; 2
DDBN trains a joint density model through DRBM;
3
The label which contains the least energy is selected;
4
The back-propagation technique for optimal classification.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
15/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Discriminative Deep Belief Network (DDBN)
Figure : Discriminative DBN architecture (DDBN).
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
16/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: CD Contrastive Divergence (CD) - Start with a training vector on the visible units;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
17/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: CD Contrastive Divergence (CD) - Start with a training vector on the visible units; - Update all the hidden units;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
17/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: CD Contrastive Divergence (CD) - Start with a training vector on the visible units; - Update all the hidden units; - Update all the visible units;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
17/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: CD Contrastive Divergence (CD) -
Start with a training vector on the visible units; Update all the hidden units; Update all the visible units; Update the hidden units again.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
17/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: PCD
Persistent Contrastive Divergence (PCD) - PCD method is proposed so that only the last chain state is used in the preceding update step.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
18/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: FEPCD
Free Energy in Persistent Contrastive Divergence (FEPCD) - A standard for the goodness of chain;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
19/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: FEPCD
Free Energy in Persistent Contrastive Divergence (FEPCD) - A standard for the goodness of chain; - Free energy is used as a measure to acquire best samples;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
19/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Training in GRBM/DRBM: FEPCD
Free Energy in Persistent Contrastive Divergence (FEPCD) - A standard for the goodness of chain; - Free energy is used as a measure to acquire best samples; - Precisely calculate the gradient of log probability from training data.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
19/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Washington RGBD Dataset
A large 3D dataset; A collection of 300 common household objects; Objects are organized into 51 categories; All object views using Kinect camera; 640x480 RGB and depth images at 30 Hz.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
20/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Washington RGBD Dataset
Figure : A sample of selected point clouds from Washington RGBD dataset. Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
21/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
GDBN/DDBN characteristics
Characteristics Hidden layers Hidden layer units Learn rates Epochs Input layer units
Value 3 300-300-1500 0.3 200 size of VFH descriptor (308)
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
22/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Mean squared normalized error
n
1X ˆ M SE = (Yi − Yi )2 n
(3)
i=1
With: Yˆi is a vector of n predictions Yi is the vector of observed values.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
23/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Mean squared normalized error Best Training Performance is 0.013614 at epoch 200
Best Training Performance is 0.10789 at epoch 200
Best Training Performance is 0.013379 at epoch 200
0
10
Train Best
Mean Squared Error (mse)
Mean Squared Error (mse)
Train Best
Mean Squared Error (mse)
0
10
0
10
−1
10
Train Best
−1
10
−1
10
−2
−2
10 0
50
100
150
200
10 0
50
200 Epochs
(CD-GDBN)
150
200
0
−2
Best Training Performance is 0.0052535 at epoch 200 0
100
150
200
−2
10
−1
10
−2
10
−3
10 0
50
200 Epochs
(CD-DDBN)
Train Best
Mean Squared Error (mse)
−1
10
10 50
200
10
−3
10
150
Train Best
Mean Squared Error (mse)
−1
10
100
(FEPCD-GDBN)
Best Training Performance is 0.0095101 at epoch 200 0
10 Train Best
0
50
200 Epochs
(PCD-GDBN)
Best Training Performance is 0.065007 at epoch 200 0
10
Mean Squared Error (mse)
100
200 Epochs
100
150
200 Epochs
(PCD-DDBN)
Presenter: Haris Ahmad Khan — Mar 01th , 2017
200
0
50
100
150
200
200 Epochs
(FEPCD-DDBN)
Generative vs. Discriminative DBN
24/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Classification performance Error GDBN-CD GDBN-PCD GDBN-FEPCD
DDBN-CD DDBN-PCD DDBN-FEPCD
0.6549 0.0250 0.0206
Acc. 34.51% 97.5% 97.9%
Before
After
Acc.
0.3810 0.4491 0.4053
0.3155 0.0201 0.0111
68.45% 97.98% 98.89%
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
25/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Comparison to other methods Methods CNN [1] RGBD dataset [2] Depth Kernel [3] DDBN-CD DDBN-PCD DDBN-FEPCD
Accuracy rates 89.4% 64.7% 78.8% 68.45% 97.98% 98.89%
M. Schwarz, H. Schulz, and S. Behnke, “Rgb-d object recognition and pose estimation based on pre-trained convolutional neural network features,” in Robotics and Automation (ICRA), 2015 IEEE International Conference on, pp. 1329–1335, IEEE, 2015. K. Lai, L. Bo, X. Ren, and D. Fox, “A large-scale hierarchical multi-view rgb-d object dataset,” in Robotics and Automation (ICRA), 2011 IEEE International Conference on, pp. 1817–1824, IEEE, 2011. L. Bo, X. Ren, and D. Fox, “Depth kernel descriptors for object recognition,” in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, pp. 821–826, IEEE, 2011. Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
26/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Conclusion
3D object categorization using geometric features extracted from VFH descriptor and learned with GDBN/DDBN;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
27/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Conclusion
3D object categorization using geometric features extracted from VFH descriptor and learned with GDBN/DDBN; GDBN is the probabilistic model with many RBMs which are trained sequentially;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
27/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Conclusion
3D object categorization using geometric features extracted from VFH descriptor and learned with GDBN/DDBN; GDBN is the probabilistic model with many RBMs which are trained sequentially; DDBN is constructed from DRBM which is based on RBM and the joint distribution model;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
27/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Conclusion
3D object categorization using geometric features extracted from VFH descriptor and learned with GDBN/DDBN; GDBN is the probabilistic model with many RBMs which are trained sequentially; DDBN is constructed from DRBM which is based on RBM and the joint distribution model; The use of PCD is better than CD, and FEPCD outperforms PCD;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
27/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Conclusion
3D object categorization using geometric features extracted from VFH descriptor and learned with GDBN/DDBN; GDBN is the probabilistic model with many RBMs which are trained sequentially; DDBN is constructed from DRBM which is based on RBM and the joint distribution model; The use of PCD is better than CD, and FEPCD outperforms PCD; The experimental results using DDBN are encouraging.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
27/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Future work
We will utilize a hybrid deep architecture that combines the advantage of generative and discriminative models;
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
28/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Future work
We will utilize a hybrid deep architecture that combines the advantage of generative and discriminative models; We will attempt to embed our algorithm in TurtleBot2.
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
28/29
Introduction Method Overview 3D Object Categorization Experimental Results Conclusion and Future Work
Thank you for your attention! Any questions
Presenter: Haris Ahmad Khan — Mar 01th , 2017
Generative vs. Discriminative DBN
29/29
