A TEXTILE EMERGENT FIBERS LENGTH MEASUREMENT SYSTEM BASED ON CAMERA VISION AND VARIABLE HOMOGRAPHY Jun Xu1,2,3, Stéphane Fontaine2, Christophe Cudel1, Sophie Kohler1, Olivier Haeberlé1, Marie-Louise Klotz3,4 1 Laboratory MIPS EA2332, University of Haute Alsace, 68093 Mulhouse Cedex, France. 2 Laboratory LPMT-EAC7189, University of Haute Alsace, 68093 Mulhouse Cedex, France. 3 Research Institute for Textile and Clothing, Department of Textile and Clothing Technology,Niederrhein University, Webschulstr. 31, 41065 Mönchengladbach, Germany. 4 Rhine-Waal University, Landwehr 4, 47533 Kleve, Germany.
[email protected] ;
[email protected] ;
[email protected]
Abstract: This study presents a new method to measure the emergent fibers of fabric for textile online monitoring. In this concept, the textile fabric including the emergent fiber is considered as a twolayer structure. A variable homography method is proposed to estimate the height of the fiber. A vision system using an industrial camera has been built to obtain the surface images. The precision of our system is firstly validated using a calibration model; then several materials are tested separately with both our setup and a digital microscope. The results of our system correspond well with the digital microscope experimental values. The main future challenge will be using cameras combined with a 3D concept -variable homography- in view of industrial applications for online measurements. Our experimental results suggest that variable homography is an accurate and robust method, which could be applied in online system for quality control.
Keywords: Variable homography, emergent fibers of fabric, online measurement, quality control, camera vision
1. Introduction Two categories of hairiness in textile, which are yarn hairiness and fabric hairiness, are of importance to be detected. Compared to yarn hairiness testing systems, which are industrialized, fabric hairiness online measurement is still an open topic, highly desirable for industrial application [1]. Therefore, a breakthrough is needed to achieve effective online detection of minor fabric hairiness. In this paper, we propose a new method to measure the length of emergent fibers from different fabrics. High-performance materials have very strict quality standards and are used in several industrial sectors ranging from medical, health care, earthworks, construction, civil engineering and transport [2]. Glass fiber fabrics considered in this paper are coated with PTFE, to be used as conveyor belts. W. Fung mentioned:’’…. PTFE is an expensive material and the coating process is lengthy; needless to say that base fabrics must be of the first quality with no weaving faults or broken fibers, which may project through the coated surface…’’[3]. Therefore to ensure that the coated fabric is of first quality, interference fibers must be avoided. Glass fibers are the first and most widely used inorganic fibers. The diameter of glass fibers varies from several micrometers to 30µm according to different production conditions [4]. In literature many possibilities such as sizing [5] or pre-treatment before coating [6] can be considered as reasons causing defect fibers on coated fabric. To ensure the glass fiber fabrics are free of defects before coating, it is necessary to control and inspect the raw fabric. In many other textile or paper products, a smooth surface is also required. For example, nonwoven fabric is normally formed by short or long chopped fiber as well as continuous filaments that maybe in circular or other patterns [7]. This forming process may cause a rough surface. Moreover, the roughness of paper will affect the printing and coating quality. For wipers used for camera cleaning or
medical hygiene application, those ‘standing’ fibers are unacceptable as well. In conclusion, detection and measurement of the hairiness has become of great interest [8]. Using a 3D optical microscope, a profilometer, or a confocal scanning microscope, one can easily obtain a variety of surface data or 3D profiles [9, 10]. However, the scale of a fine fiber or broken filament is extremely small compared to the area to be inspected. In this situation, conventional testing equipments are often useless to automatically get the surface map of a fabric. The true height of a default fiber can be measured manually using a 3D microscope, which is accurate, but time-consuming. Thus it isn’t suitable for online textile surface inspection. Moreover, available fabric inspection systems, such as I-TEX from EVS company [11], are mainly using image processing for automatic 2D defects detection [1, 12], and not suitable for 3D defects monitoring. In this paper, we propose a computer vision approach, based on variable homography, which can be used to measure emergent fibers’ lengths on textile fabrics in 3D. The main challenges addressed in this work are: applying variable homography to textile measurements, and estimating the robustness of the method. 2. Variable homography method 2.1 Principle Variable homography was introduced by Zhang and Greenspan [13] to improve mosaic images reconstructions. In our application, we apply this method for textile hairiness measurement. We suppose that a fibrous structure can be considered as a two-layer structure and Fig. 1 describes a typical variable homography setup to estimate the length of the fiber defects. Focal point
Focal point PlB
X
PrB = HB.PlB
Left camera
Right camera
PlA
PrA = HA.PlA
ZB
ZA
Plane B PB(XA,YA,ZB)
Defect
Z
Plane A PA(XA,YA,ZA)
Figure 1: Camera configuration for variable homography setup and experimental setup
Homography is an important transformation matrix in image registration. It can be used to calculate the transformation of different planes in an image, or the transformation of same plane in different images. Hartley and Fisherman [14] give a definition of homography: “A homography is an invertible transformation from the real projective plane to the projective plane that maps straight lines to straight lines”. Homography is described by a 3x3 matrix and is defined as: ℎ11 ℎ12 ℎ13 𝑢𝑙 𝑢𝑟 s . 𝑃𝑟 = 𝑠 . � 𝑣𝑟 � = �ℎ21 ℎ22 ℎ23 � �𝑣𝑙 � = 𝐻 . 𝑃𝑙 (1) 1 1 ℎ31 ℎ32 ℎ33 where Pr is a specific point from the right camera image and Pl its corresponding point onto the left camera image. This homography H is usable for points located in the same plane. On Fig. 1, we consider two homographies. Homography HA relies on PrA and PlA respectively, projections on both cameras of a point from plane A. Identically, HB relies on PrB and PlB, projections on both cameras of a point from plane B. In Zhang and Greenspan’s paper [13], it was shown that HA and HB are indeed related by Equation 2:
𝐻𝐵 = 𝐾 𝐻𝐴 𝐾 −1 , (2) where K is the so-called intrinsic parameter matrix characterizing the camera and its objective. Advantages of the method we propose are that it can work with planar fabrics, so that the fabric doesn't need to be bended (which may create new artificial defects) and that it requires simple equipments, which could easily be fitted onto online systems, even for middle size companies. In Fig. 1, note that a laser line illuminates the top of the fiber defect, which lies in the plane B, and that the size of the fiber defect 𝐿𝑑𝑒𝑓𝑒𝑐𝑡 can be obtained with 𝐿𝑑𝑒𝑓𝑒𝑐𝑡 = 𝑍𝐴 − 𝑍𝐵 .
2.2 Validation After testing the method with simulations taking into account the exact intrinsic parameters of our camera (IDS Ueye1240 SE monochrome camera built with a ½ CMOS sensor, of resolution of 1280x1024), the size range of the defects we want to detect, etc, we used a calibrated stair to validate our approach. The stair is made out of steel, and is composed of five 1mm height steps. It is displayed on Fig. 2, which shows a reconstruction using the VHX600 digital microscope (left), one camera view (center), and a comparison of the stair profile, as reconstructed from the digital microscope (blue curve), and with our method (green squares), with the reference profile in black. While the digital microscope slightly underestimates the stair elevation, our system on the contrary tends to slightly overestimate the altitude. Note that for the highest stair, the image is severely blurred, resulting in a failure to deliver a correct elevation estimation. However, in the 0 to 3mm altitude range, the accuracy of our method is reliable enough to measure fiber height over a textile fabric.
Figure 2: Stair imaged by VHX600 digital microscope (3D), by Variable Homography System (VHS), and the comparison of profile reconstructions
3. Experiment 3.1 Description The experimental procedure comprises seven steps: 1) Initialization of the setup to determine the camera intrinsic parameters and ZA. 2) Acquisition of left image by the camera. 3) Shift of the fabric. 4) Acquisition of the right images, with a common region with left image. 5) Computation of HA. 6) Detection of the hairiness correspondence between both images. 7) Computation of the height of the hairiness. Throughout this work, we consider glass fiber fabric, nonwoven, wiper and felt. Glass fiber cloth is defined as: “Threads made of glass fibers are woven into cloth using regular textile weaving machinery” [15]. The type of the weaving may induce a concavo-convex texture surface (see Fig. 3). Nonwoven, wiper and many other materials also exhibit the same feature. It may therefore be difficult to determine the reference layer during the measurement. Moreover the ‘standing’ fiber can be short. In our approach, the proper base layer is defined during the first phase, when initializing the setup. As seen in variable homography theory, HA and HB are related with the intrinsic
parameters of the camera, and the ratio of the two planes distance is called k. In order to determine the height of the defect, we only need to compute k. The distance of the fabric to the camera (noted ZA, see Fig 1) is determined during the calibration process. The homography HA is computed using pairs of points detected via SURF [16] in both images (outlier pairs are avoided using a RANSAC algorithm [17]).
Figure 3: A scheme of concavo-convex texture surface
We first identify a common defect point/region, visible on both images. At present, we need to select such target points manually. Future improvement of the method will include an automatic detection of these target points. The target points are projected onto the second image using the HA relation. A Region of Interest (ROI) can be defined in order to find the exact corresponding point by using a cross correlation procedure. In order to find the k value corresponding to this defect, we minimize the Euclidean distance error between the correlation point and the points obtained by projection using the HB relation, letting k vary in a predetermined range. It is to be noticed that the searching range for k is significantly reduced since we know that it must be greater than 1 (defect is above the surface) and is bounded by the maximum possible defect size 𝐿𝑑𝑒𝑓𝑒𝑡𝑐 max . We get: 𝑘𝑓𝑖𝑛𝑎𝑙 = 𝑎𝑟𝑔𝑚𝑖𝑛 [𝐷𝑒𝑢𝑐𝑙 � 𝑃𝑐𝑜𝑟𝑟 , 𝐾𝑖 ∗ 𝐻𝐴 ∗ 𝐾𝑖−1 ∗ 𝑃𝑙𝐵 �] ⎧ ⎪
𝑘𝑖 ⎛ 𝐾𝑖 = 0 ⎝0
0
𝑘𝑖
(1−𝑘𝑖 )
𝑓 (1−𝑘𝑖 ) 𝑓
𝑢0 𝑣0
(3)
⎞
， ⎨ 0 1 ⎠ 𝑍𝐴 ⎪1 < 𝑘 ≤ (𝑘 ) 𝑖 𝑚𝑎𝑥 = 𝑍 −𝐿 𝐴 𝑑𝑒𝑓𝑒𝑡𝑐max ⎩ 𝑍 with 𝑘𝑖 = 𝑍𝐴 , f: focal length in pixel, (u0,v0) : image center coordinates. 𝐵
3.2 Results Experimental results obtained from variable homography system are presented and comparison of the results is provided. In Fig. 4, a glass fiber projected from the fabric is highlighted by a laser line and appears as a shining point in the image obtained from the camera. The laser line is parallel to the fabric surface and the height of the laser line is adjusted along the vertical direction in order to illuminate the top of the fiber. This feature helps to select the target point. Figure 4 also shows two VHX600 images displaying a loop and a divariation. The height of these defects is first measured manually using the VHX600 system.
VHS
VHX
VHX
Figure 4: Glass fiber fabric defects
For the considered fabrics, defects are first measured using the VHX600 system, and then the same hairiness is measured with our method. The most difficult material among the
selected fabrics is the glass fiber fabric, because of the fine glass fiber diameter, and the high reflectivity. Figure 5 shows how defects on nonwoven, wiper and felt illuminated by the laser line become visible. Figure 6 shows the same defects as detected with our VHS system, and with the VHX600 microscope.
nonwoven
wiper
felt
Figure 5: Fiber lighted by laser line light
Nonwoven by VHS
Wiper by VHS
Felt by VHS
Nonwoven by V HX
Wiper by VHX
Felt by VHX
Figure 6: Fiber of Wiper, nonwoven and felt images from VHS (top) and VHX600 (bottom) height (mm)
7
6,2
6
5,58
5 4 3 2 1 0
1,12
1,74
0,91 1,05
glass fiber fabric glass fiber fabric - loop - divariation
1,53
2,7 2,74
2,08
VHS VHX
wiper -fiber
felt -fiber
nonwoven -fiber
VHS
1,12
0,91
1,53
6,2
2,7
VHX
1,74
1,05
2,08
5,58
2,74
Table 1: Defects elevation measurements with VHS and digital microscopy (VHX600 system)
Table 1 gives defects elevations as measured from the VHS and VHX systems. As expected, our system gives very similar results to those obtained from VHX600. However, the measurements require about 50 images when using VHX600 focusing (because of the very shallow depth of field of this system), while VHS requires only two images. Some impact factors such as color of the fiber, diameter of the fiber, light reflection of the materials and length and shape of the fiber are considered as the possible reasons which
affect the accuracy of the measurement. More investigations are under way, the precision being estimated to be in the 10-15% range. Actually, during production, it is often not necessary to precisely measure the defect when detecting it: the manufacturer will calibrate the system according to its requirement or ISO standard. 4. Conclusion In conclusion, a new system is presented in order to measure the height of the hairiness, with potential application for real-time online monitoring of textile, nonwoven and paper/wiper fabrication. Variable homography is applied to textile fiber measurement. This method uses two images and is in fact a stereovision system. Two images with a common cross section are used to find corresponding points and estimate the length of emergent fibers. One advantage of the proposed solution is that it uses an easy and standard planar calibration to get the focal value. We have shown that variable homography is a promising method for online defect length measurement in the 1-3 mm range, according to the experimental results based on our materials. So far we have used our system with a fixed position of the camera, and manual translation of the fabric. We now plan to build an automated prototype with controlled fabric movement, in order to test our method in realistic production-like conditions. Reference [1] X. Xie, "A Review of Recent Advances in Surface Defect Detection using Texture analysis Techniques", Electronic Letters on Computer Vision and Image Analysis, 2008, 7, 1-22. [2] K. Krebber, “Optical Sensors conference”, Information Gatekeepers Inc, 2011. [3] W. Fung, “Coated and laminated textile”, Woodhead Publishing, 2002, 184. [4] M. Xanthos, “Functional Fillers for Plastics”, John Wiley & Sons, 1st ed. 2005. [5] S. Mallarino, J. F. Chailan, and J. L. Vernet, “Glass fiber sizing effect on dynamic mechanical properties of cyanate ester composites I. Single frequency investigations”, European Polymer Journal, 2005, 41, 1804-1811. [6] M. J. Owen, V. Middleton, and I. A. Jones, “Integrated design and manufacture using fibrereinforced polymeric composites”, Woodhead Publishing, 2000, 1804-1811. [7] D. V. Rosato, D. V. Rosato, and J. Murphy, “Reinforced plastics handbook”, Elsevier, 2004, 22. [8] S. Russell, “Handbook of nonwovens”, CRC Press1st ed., 2007, 38. [9] J. Gorelik, A.Shevchuke, M Ramalho, M.Elliott, “Scanning surface confocal microscopy for simultaneous topographical and fluorescence imaging: Application to single virus-like particle entry into a cell”, Proceedings of the National Academy of Sciences, 2002, 99, 16018-16023. [10] J. C. Wyant, C. L. Koliopoulos, B. Bhushan, and O. E. George, “An Optical Profilometer for Surface Characterization of Magnetic Media”, ASLE - Transactions, 1984, 27, 101-113. [11] A. Tilocca, P. Borzone, S. Carosio, and A. Durante, “Detecting Fabric Defects with a Neural Network Using Two Kinds of Optical Patterns”, Textile Research Journal, 2002, 72, 545-550. [12] VS-SCANMSTER, Available: http://www.evs-sm.com [13] S. Zhang and M. Greenspan, "Variable Homography Compensation of Parallax Along Mosaic Seams", Image Analysis and Recognition, 2007, 4633, 271-284. [14] R. Hartley and A. Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge University Press 2nd, 2004. [15] J. Wiley, “The fiberglass repair and construction handbook”, McGawn–Hill Professional, 1988. [16] H. Bay, T. Tuytelaars, and L. Gool, “SURF: Speeded Up Robust Features”, in Computer Vision – ECCV, 2006, 3951, 404-417. [17] R. C. Bolles and M. A. Fischler, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography", Communications of the ACM, 1981, 24, 381-395.
