open computer vision infrastructures, code that was passed from student to
student. ▻ 1999: OpenCV Project Started. ▻ 2000: The first alpha version of
OpenCV was released ... Two applications from the web: ..... Programming
Cookbook.
Università del Salento Facoltà di Ingegneria Image Processing (Elaborazione delle Immagini) A.A. 2013/2014
PART I - A very brief introduction
Dario Cazzato, INO – CNR
[email protected] [email protected]
What is OpenCV? } Installation } Key Concepts of the library } Examples }
◦ Source Code ◦ Only notes and doc snapshots here
Resources } Study cases in the next lesson } Our (INO-CNR) activities, trends, hot topics in research in the third part }
This is not a guide. } This is not a complete tutorial. } This is not enough for your project. } At the end of this lesson, you will know something about this very wide environment, something about the philosophy behind, and will see some beginners application. } The next part will cover real examples. } Presentation + OpenCV code. }
Open Source Computer Vision Library. } It’s an open source (BSD License) library for image processing. } Written in optimized C. } It runs on Linux, Windows, Mac OS. } Developed by Intel: }
◦ Completely free ◦ Intel’s Integrated Performance Primitives (IPP) libraries with fee
Intel IPP provides high performance low-level routines for audio, video, imaging, cryptography, signal processing and codecs. } Customers need to have the Intel IPP binaries installed in order to take advantage of the Intel IPP optimizations. Otherwise, all the functionality is implemented by the OpenCV open source code. } Anyway, IPPs are loaded at run-time. } Intel® Integrated Performance Primitives 8.0: $199 (commercial use) / $99 (academic use) }
}
} }
} } } } }
MIT Media Lab had well-developed and internally open computer vision infrastructures, code that was passed from student to student. 1999: OpenCV Project Started. 2000: The first alpha version of OpenCV was released to the public at the “IEEE Conference on Computer Vision and Pattern Recognition”. 2006: The first 1.0 version was released. 2008: Willow Garage corporate support. 2009, September: OpenCV v2.0, major changes to the C++ interface (think to Memory Management!). 2010, April: OpenCV v2.1. Today: OpenCV v2.4.6, Mobile, CUDA, Kinect.
Strong focus on real-time applications. } Library for C/C++. } Only? NO! Python, Java... } Linux, Windows, Mac OS… Android, iOS. } Necessary “Restyling”: OpenCV v2.4. }
◦ Some problem getting up-to-date material. ◦ Some problem looking for solutions surfing forums.
}
Solution: Learn the new syntax, but be able to fully understand the old one.
}
OpenCV Modules:
}
Just use the full documentation!
◦ cv (Main OpenCV functions); ◦ cvaux (Auxiliary OpenCV functions); ◦ cxcore (Data structures and linear algebra support); ◦ highgui (GUI functions). ◦ http://docs.opencv.org/index.html ◦ All the objects are explained. ◦ Several tutorials are available (growing section!). ◦ “User Guide” for a first usage. ◦ Android, iOS Reference. ◦ Old version full documented.
}
}
} }
} } } }
core - a compact module defining basic data structures, including the dense multi-dimensional array Mat and basic functions used by all other modules. imgproc - an image processing module that includes linear and nonlinear image filtering, geometrical image transformations (resize, affine and perspective warping, generic table-based remapping), color space conversion, histograms, and so on. video - a video analysis module that includes motion estimation, background subtraction, and object tracking algorithms. calib3d - basic multiple-view geometry algorithms, single and stereo camera calibration, object pose estimation, stereo correspondence algorithms, and elements of 3D reconstruction. features2d - salient feature detectors, descriptors, and descriptor matchers. objdetect - detection of objects and instances of the predefined classes (for example, faces, eyes, mugs, people, cars, and so on). highgui - an easy-to-use interface to video capturing, image and video codecs, as well as simple UI capabilities. gpu - GPU-accelerated algorithms from different OpenCV modules.
}
Two applications from the web:
}
On Windows:
}
On Linux
◦ Run the EXE and choose the Path; ◦ Include the libraries to your project; ◦ Configure your favourite IDE (Visual Studio, Eclipse). ◦ Install dependencies (search Google); ◦ Unpack the Tarball archive (*.tar.gz) in a folder like / /OpenCV-*/ ; ◦ Create a subfolder /release to run Cmake; ◦ Configure Cmake (cmake-gui); ◦ Launch the Install Script (sudo make install); ◦ Options for GCC: ‘pkg-config --cflags -–libs opencv’ (for g++ also); ◦ Link it to an IDE (Eclipse, Code::Blocks).
}
On Macintosh, use the MacPort system:
◦ ◦ ◦ ◦ ◦
sudo port selfupdate; sudo port install opencv; OpenCV in /opt/local/ (default in Mac OS X 10.6); And then, include headers in your project or configure Xcode Look for tutorials based on your OS version.
N.B.:You can use ‘pkg-config’ to compile with g++
All the OpenCV classes and functions are placed into the cv namespace. } So, these snippets are equivalent: }
}
Exceptions and error handling: C++ Style. ◦ Class cv::Exception (derivative of std exception class). ◦ How to throw exception. 2 macros: ◦ CV_Error(errcode, description); ◦ CV_Assert(condition).
}
Example:
Example 1
} } } }
FOUNTAMDENTAL CLASS No more hierarchies to learn (OpenCV v1.x)! We see images. We talk about “image processing”. Computers see matrixes. They use Matrix algebra to process data… … and OpenCV process instances of the Class Mat!
}
Two parts:
◦ Header (Fixed Size); ◦ Pointer to the Data (Variable Size).
In real application, which is bigger?
} }
In image processing tasks, you need to pass a matrix to some function. Solution to avoid time-consuming copies: Reference counter.
◦ Data can be shared between 2 matrices with their own header. ◦ When a copy is needed:
Copy the header: fast! Copy the pointer to the matrix (NOT the matrix itself!) Reference counter is increased.
◦ When an header is cleared (user, scope, function…) Reference counter is decreased.
◦ The last survivor destroy the data.
}
Will increment the counter: ◦ Mat ◦ Mat ◦ Mat ◦ Mat
}
MyMat1, Mat MyMat2; //declarations MyMat1(MyMatSrc); //constructor M1 = M2; //assigment - O(1), shared data ROIMat(A, Rect(10,10,50,50)); //ROI
Will copy headers + data: ◦ Mat MyMat= A.clone(); ◦ A.copyTo(MyMat);
Example 2
}
No more nested loops to show data:
}
Some parameter:
◦ rows – The number of matrix rows ◦ cols – The number of matrix columns ◦ size – The matrix size: Size(cols, rows) . Note that in the Size() constructor the number of rows and the number of columns go in the reverse order. ◦ type – The matrix type, use CV_8UC1, ..., CV_64FC4 to create 1-4 channel matrices, or CV_8UC(n), ..., CV_64FC(n) to create multi-channel (up to CV_MAX_CN channels) matrices ◦ s – The optional value to initialize each matrix element with. To set all the matrix elements to the particular value after the construction, use the assignment operator Mat::operator=(const Scalar& value) . ◦ data – Pointer to the user data. ◦ step – The data buddy. This optional parameter specifies the number of bytes that each matrix row occupies. The value should include the padding bytes in the end of each row, if any. If the parameter is missing (set to cv::AUTO_STEP ), no padding is assumed and the actual step is calculated as cols*elemSize() , see Mat::elemSize (). ◦ img – Pointer to the old-style IplImage image structure. By default, the data is shared between the original image and the new matrix, but when copyData is set, the full copy of the image data is created. ◦ expr – Matrix expression. See Matrix Expressions.
One of unsigned char, bool, signed char, unsigned short, signed short, int, float, double. } CV_{U|S|F} C. } Examples: }
◦ CV_8UC1: 8-bit single-channel matrix; ◦ CV_32FC2: 2-channel (i.e. complex) floating-point; ◦ cv::Mat M(4,4,CV_32FC2,Scalar(1,3)): 4x4 complex matrix filled with 1+3j.
}
List of depths:
◦ CV_8U: 8-bit unsigned integers (0..255) ◦ CV_8S: 8-bit signed integers (-128..127) ◦ CV_16U: 16-bit unsigned integers (0..65535) ◦ CV_16S: 16-bit signed integers (-32768..32767) ◦ CV_32S: 32-bit signed integers (-2147483648..2147483647) ◦ CV_32F: 32-bit floating-point numbers (-FLT_MAX..FLT_MAX, INF, NAN) ◦ CV_64F: 64-bit floating-point numbers (-DBL_MAX..DBL_MAX, INF, NAN)
}
Example
◦ cv::Mat img(Size(320,240),CV_8UC3);
Create an image of size 320x240, 8-bit, 3channels. } It’s like having 3 2-D matrixes with values in the range [0,255] }
}
RGB!
}
Addition, subtration, negation: ◦ A+B, A-B, A+s, -A.
}
Scaling, multiplication, dot product: ◦ alpha*A, A*B, A.dot(B).
}
Transposition: ◦ A.t().
}
Element-wise min,max, abs: ◦ min(A,B), max(A,B).
}
Advanced functions:
◦ Determinant, invert, trace, eigenvalues, LA solver… Example 3
}
Point:
◦ 2D point specified by x and y. ◦ Examples: Point2f, Point3i. ◦ Example: Point (stands for Point2i). If you initialize Point with float, the integer part will be taken.
}
Size:
◦ Used to specify the size of an image or a rectangle. ◦ 2 members: width, height. ◦ Examples: Size2i, Size2f, Size (stands for Size2i). ◦ Useful to create easy initializations.
}
Rect:
◦ 2D rectangle. ◦ Specified by x,y, width, height, i.e. the top-left corner, the widht (from left to right), the height (from top to bottom).
}
Scalar:
◦ 4-elements vector. ◦ Used to pass pixel values. ◦ Derived from Vec.
}
Vec:
}
Many OpenCV functions process dense 2-dimensional or multidimensional numerical arrays. Usually, such functions take Mat as parameters, but in some cases it’s more convenient to use std::vector (for a point set, for example) or Matx (for 3x3 homography matrix and such). To avoid many duplicates in the API, special “proxy” classes have been introduced. The base “proxy” class is InputArray. It is used for passing read-only arrays on a function input. The derived from InputArray class OutputArray is used to specify an output array for a function. Normally, you should not care of those intermediate types (and you should not declare variables of those types explicitly) - it will all just work automatically. You can assume that instead of InputArray/OutputArray you can always use Mat, std::vector, Matx, Vec or Scalar. When a function has an optional input or output array, and you do not have or do not want one, pass cv::noArray().
}
CvMat:
◦ Multichannel 2D Matrix. ◦ cvMat* cvCreateMat(int rows, int cols, int type); ◦ Possible to create just the header or the data: cvCreateMatHeader(). cvCreateMatData().
◦ You must release it!
cvReleaseMat(CvMat **mat);
}
IplImage
◦ Foundamental! ◦ Theory is like cv::Mat. ◦ See the reference for its syntax. ◦ Example: DEPTH.
Hierarchy:
The three structures are “related” to each other by “inheritance”. It’s not the C++ or Java inheritance. OpenCV was made using C, so there’s no such concept. Anyway, their relation mimics inheritance.
}
For the most of the C functions and structures from OpenCV v1.x you may find the direct counterparts in the new C++ interface. The name is usually formed by omitting cv or Cv prefix and turning the first letter to the low case (unless it’s a own name, like Canny, Sobel etc). In case when there is no the new-style counterpart, it’s possible to use the old functions with the new structures.
Let’s see OpenCV in action!
}
Point stands for Point2i
◦ Use always the complete notation for readability!
}
class RotatedRect ◦ Constructor:
}
How to draw ◦ line
}
How to draw ◦ rectangle
}
How to manage (very quickly introduction!) windows ◦ imshow
}
How to manage (very quickly introduction!) windows ◦ waitkey
}
Mat imread(const string& filename, int flags = 1); ◦ Flags let you load already in grayscale!
}
bool imwrite(const string& filename, InputArray img, const vector& params=vector()) ◦ …Params?
}
Old C prototypes:
◦ IplImage* cvLoadImage(const char* filename, int iscolor=CV_LOAD_IMAGE_COLOR ) ◦ int cvSaveImage(const char* filename, const CvArr* image, const int* params=0 )
}
Changing contrast of an image:
}
Without going deep into details, how to invert a matrix? ◦ Use LU decomposition for a non-singualr matrix ◦ Use SVD for a singular (or even non-square) matrix
}
}
ROI: Region of interest, is a selected subset of samples within a dataset identified for a particular purpose (the boundaries of an object) Smoothing: blur, Gaussian, median, bilateral
… but you can create also your own filter using the class BaseFilter, or using filter2d!
Alternative (maybe less modular, imagine to specify a method using xml, but probably easier to use and customize):
Note: you could also manually use the resize function
GPU?
}
How to open a cam using OpenCV? ◦ Old struct:
CvCapture: The structure CvCapture does not have public interface and is used only as a parameter for video capturing functions. CvCapture* cvCaptureFromCAM(int device) CvCapture* cvCaptureFromFile(const char* filename) cvQueryFrame: grabs a frame from camera or video file, decompresses and returns it. cvSetCaptureProperty to set properties The function cvReleaseCapture releases the CvCapture allocated structure.
◦ New class: VideoCapture Constructor:
VideoCapture::open VideoCapture::isOpened VideoCapture::release VideoCapture::set
}
VideoCapture::set
◦ bool VideoCapture::set(int propId, double value)
}
cvSetCaptureProperty
◦ int cvSetCaptureProperty(CvCapture* capture, int property_id, double value)
}
Classifier:
◦ CascadeClassifier to detect objects ◦ load to load classifier file (xml) ◦ detectMultiScale to perform detection
}
All details in:
} }
}
OpenCV already provides a long list of functions. The problems you’ll find at the beginning, are the same problems the other beginners already had (and solved), use the web to look for their solutions and study them. Features that are not part OpenCV, are in third part libraries (mathematics, efficient data structures, more with matrixes, GUI…) ◦ PCL (http://pointclouds.org/) ◦ Boost (http://www.boost.org/) ◦ Qt (http://qt.digia.com/)
}
Don’t reinvent the wheel, but also try to be a “wise user” of the material!
}
Books:
◦ Learning OpenCV: Computer Vision with the OpenCV Library (O’Reilly). ◦ OpenCV 2 Computer Vision Application Programming Cookbook. ◦ The OpenCV Reference Manual: http://docs.opencv.org/opencv2refman.pdf
}
OpenCV Download:
}
OpenCV Documentation:
}
Yahoo! Group (>45.000 users), with a forum:
}
◦ http://sourceforge.net/projects/opencvlibrary ◦ http://docs.opencv.org/index.html
◦ http://tech.groups.yahoo.com/group/OpenCV/
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ◦ http://www.cvpr2012.org/
}
Youtube OpenCV Channel:
◦ http://www.youtube.com/channel/HCEvIHQuvgcGc
}
Tutorials:
◦ http://docs.opencv.org/doc/tutorials/tutorials.html
}
Tutorial about cv::Mat (the first you should read after being able to write, compile and launch “Hello World”):
◦ http://docs.opencv.org/2.4.3rc/doc/tutorials/core/ mat_the_basic_image_container/ mat_the_basic_image_container.html
}
OpenCV with Android:
◦ http://opencv.org/platforms/android.html ◦ http://docs.opencv.org/doc/tutorials/introduction/ android_binary_package/O4A_SDK.html
}
OpenCV with iOS:
◦ http://docs.opencv.org/trunk/doc/tutorials/ios/ table_of_content_ios/table_of_content_ios.html (trunk!)
}
Me:
◦
[email protected] ◦
[email protected]
