A Benchmark Dataset and Deep Learning-Based Image ... - MDPI

sensors Article

A Benchmark Dataset and Deep Learning-Based Image Reconstruction for Electrical Capacitance Tomography Jin Zheng 1 , Jinku Li 1 , Yi Li 2 and Lihui Peng 1, * 1 2

*

Tsinghua National Laboratory for Information Science and Technology, Department of Automation, Tsinghua University, Beijing 100084, China; [email protected] (J.Z.); [email protected] (J.L.) Graduate School at Shenzhen, Tsinghua University, Shenzhen 518055, China; [email protected] Correspondence: [email protected]; Tel.: +86-010-6277-3623

Received: 27 September 2018; Accepted: 29 October 2018; Published: 31 October 2018







Abstract: Electrical Capacitance Tomography (ECT) image reconstruction has developed for decades and made great achievements, but there is still a need to find a new theoretical framework to make it better and faster. In recent years, machine learning theory has been introduced in the ECT area to solve the image reconstruction problem. However, there is still no public benchmark dataset in the ECT field for the training and testing of machine learning-based image reconstruction algorithms. On the other hand, a public benchmark dataset can provide a standard framework to evaluate and compare the results of different image reconstruction methods. In this paper, a benchmark dataset for ECT image reconstruction is presented. Like the great contribution of ImageNet that transformed machine learning research, this benchmark dataset is hoped to be helpful for society to investigate new image reconstruction algorithms since the relationship between permittivity distribution and capacitance can be better mapped. In addition, different machine learning-based image reconstruction algorithms can be trained and tested by the unified dataset, and the results can be evaluated and compared under the same standard, thus, making the ECT image reconstruction study more open and causing a breakthrough. Keywords: benchmark dataset; electrical capacitance tomography; machine learning; image reconstruction

1. Introduction Electrical capacitance tomography (ECT) is a measurement technique for visualizing dielectric multi-phase flow processes, such as pneumatic conveying systems and fluidized beds, by generating cross-sectional images [1–3]. A traditional ECT system mainly contains three parts: the ECT sensor, the capacitance measurement and data acquisition circuit, and the imaging computer. All the possible capacitance data among the non-redundant electrode combinations are measured based on a capacitance measurement circuit [4], and the permittivity distribution can be reconstructed by certain algorithms. For an ECT sensor with N electrodes, the number of available capacitance data is (N − 1) × N/2. In the past three decades, research concerning ECT sensor design [5–8], hardware design [1,9–12], and image reconstruction algorithms [13–21] and applications [22–30] have been widely investigated and remarkable progress has been made. So far, ECT is still a very active field. The studies in the literature show that papers on the ECT field are published constantly, among which studies of ECT image reconstruction algorithms make up an important part of it. Conventional algorithms such as the linear back project (LBP), the Landweber iteration, and the total variation (TV) based regularization are still adopted, meanwhile, in recent years, some Sensors 2018, 18, 3701; doi:10.3390/s18113701

www.mdpi.com/journal/sensors

Sensors 2018, 18, 3701

2 of 20

distinctive works also have been reported. An example of such a distinctive work is the image reconstruction algorithm based on the sparsity constraint combined with the compressed sensing theory. Ye J. M. et al. designed an extended sensitivity matrix that consists of some normalized capacitance vectors corresponding to the base permittivity elements [31]. Zhao J. et al. used a sparse reconstruction by a separable approximation algorithm to solve the ECT inverse problem [32]. Yang Y. J. and Peng L. H. proposed an enhanced linear model and sparsity regularization for the image reconstruction algorithm [33]. In other areas, Taylor S. H. and Garimella S. V. adopted a level set method to reconstruct ECT images [34]. Ren S. J. et al. introduced the boundary element method for ECT image reconstruction, and this method was able to reconstruct the permittivity distribution profile in the imaging area well [35]. In recent years, machine learning theory has flourished in many fields and researchers in the ECT area have also attempted to introduce it to solve the image reconstruction problem. Marashdeh et al. trained a combined multilayer feed-forward neural network and analogue Hopfield network [36]. Wang et al. proposed a least square support vector machine and bacterial colony chemotaxis algorithm for ECT image reconstruction [37]. Li et al. attempted to make the BP and RBF neural networks solve the ECT image reconstruction problem [38]. Although these attempts made breakthroughs in ECT image reconstruction to some degree, most of these reported machine learning-based ECT image reconstruction methods are trained using a small-scale dataset that usually comprised of several tens to about one hundred instances. The generalization ability may be limited when the training dataset is small, which means that the training results may be good for the training dataset, but if given a new capacitance vector that the network has never seen before, the network may not be able to figure out the right corresponding permittivity distribution. So, a large-scale dataset is of great necessity for researchers in order to explore machine learning algorithms for ECT image reconstruction. However, there is still no public large-scale dataset in the ECT field. As is known to all, a good public dataset, such as MNIST [39] and ImageNet [40] in the machine learning field, is a key part of machine learning research. For example, ImageNet, which is a large-scale dataset for researchers in the computer vision area, has millions of images under thousands of categories, with a typical category containing several hundred images [40]. Such a public dataset inspires researchers to explore faster and more accurate image classifying or object detecting methods and launches a great campaign to promote the development of machine learning, especially deep learning theory. The ImageNet example shows that not only models should be emphasized, but data should also be treated with more attention. The availability of more data would help researchers develop better algorithms. A free and open large-scale dataset is also expected (in the ECT field) to contribute more data to better map the relationship between capacitance and the permittivity distribution and to evaluate and compare the results of different image reconstruction methods under the same criteria. On the other hand, in order to get the required amount of ECT capacitance and permittivity distribution data for a study on image reconstruction, a lot of simulation models need to be built or a practical ECT experiment system needs to be established, which will cost much in both the materials and time. However, when a large-scale benchmark dataset is brought out, researchers will find it convenient since they need not repeat their data acquisition work. Like the great benefits from ImageNet, it is hoped that such a large-scale public benchmark ECT dataset would also make researchers realize the importance of the dataset, start a revolution of solving the image reconstruction problem, encourage researchers to explore better image reconstruction methods, and have more communication, leading to a breakthrough in ECT image reconstruction theory. In this paper, a benchmark dataset for ECT image reconstruction is proposed. It consists of tens of thousands of capacitance vectors and corresponding permittivity distribution vectors, as well as sensitivity matrices obtained from 2D simulation models, and 3D simulation models along with static and dynamic experiments. The benchmark dataset can be regarded as two parts. One subset, whose data is from the 2D simulation models, is for the training and testing of machine learning

Sensors 2018, 18, 3701 Sensors 2018, 18, x FOR PEER REVIEW

3 of 20 3 of 20

for ECT image reconstruction. The other subset, whose data is from 3D simulation models and methods for ECT image reconstruction. The other subset, whose data is from 3D simulation models experiments, is for evaluating and comparing different ECT image reconstruction algorithms. This and experiments, is for evaluating and comparing different ECT image reconstruction algorithms. This study is concerned with typical two-phase flow patterns—annular, stratified, single bar, and two-bar. study is concerned with typical two-phase flow patterns—annular, stratified, single bar, and two-bar. Additionally, three relative permittivity values—2.7, 3.8, and 80—are set on the phase in the higher Additionally, three relative permittivity values—2.7, 3.8, and 80—are set on the phase in the higher permittivity value and the lower permittivity value is set to 1. The image reconstruction results of the permittivity value and the lower permittivity value is set to 1. The image reconstruction results of the three traditional algorithms, i.e., the LBP, the projected Landweber iteration, and the total variation three traditional algorithms, i.e., the LBP, the projected Landweber iteration, and the total variation (TV) based regularization, along with the deep learning-based method proposed in Reference [41] (TV) based regularization, along with the deep learning-based method proposed in Reference [41] are are used as examples on how to compare different algorithms under the same evaluation criteria of used as examples on how to compare different algorithms under the same evaluation criteria of the the benchmark dataset. benchmark dataset. The paper is organized as follows. Sections 2 and 3 provide the benchmark dataset and the image The paper is organized as follows. Sections 2 and 3 provide the benchmark dataset and the reconstruction result examples based on the simulations and experiments, respectively. Finally, image reconstruction result examples based on the simulations and experiments, respectively. Finally, conclusions are drawn in Section 4. conclusions are drawn in Section 4. 2.2.The TheSimulation SimulationPart Partofofthe theBenchmark BenchmarkDataset Dataset InInthis thissection, section,the thesimulation simulationpart partofofthe thebenchmark benchmarkdataset datasetbased basedon on2D 2Dand and3D 3Dmodels modelsisis introduced. Permittivity distribution images are reconstructed by capacitance vectors in the introduced. Permittivity distribution images are reconstructed by capacitance vectors in thedataset dataset based on 3D models by using the three conventional ECT image reconstruction algorithms—i.e., the based on 3D models by using the three conventional ECT image reconstruction algorithms—i.e., LBP, the projected Landweber iteration, and the TV-based regularization—as well as by using the LBP, the projected Landweber iteration, and the TV-based regularization—as well as by usingaa machine method. The The quantitative quantitative criteria criteriafor forthe thecomparison comparisonof machinelearning-based learning-basedimage image reconstruction reconstruction method. ofthe theimage imagereconstruction reconstructionresults resultsare arealso alsoprovided. provided. 2.1. 2.1.The TheSimulation SimulationPart Partofofthe theBenchmark BenchmarkDataset DatasetBased Basedon onthe the2D 2DModels Models One Oneimportant importantpart partofofthe thebenchmark benchmarkdataset datasetisisaalarge-scale large-scaledataset datasetbuilt builtasasaapublic publicdatabase database for forthe thetraining trainingand andtesting testingofofthe themachine machinelearning-based learning-basedECT ECTimage imagereconstruction reconstructionalgorithms. algorithms.The The large-scale generatedby bya aplatform platform which is established on MATLAB GUI and large-scaledataset dataset is generated which is established on MATLAB with awith GUI aand worked worked 2D 8-electrode ECTmodels sensor built models on the finite analysis element software, analysis software, on withon 2Dwith 8-electrode ECT sensor on built the finite element COMSOL COMSOL Multiphysics [42]. It contains totallypairs 40,000 of ECT data samples, with each pair of Multiphysics [42]. It contains totally 40,000 of pairs ECT data samples, with each pair of samples samples consisting of a normalized permittivity with 3228 and the consisting of a normalized permittivity distributiondistribution vector withvector 3228 elements andelements the corresponding corresponding normalized capacitance vector of withECT an 8-electrode sensor with elements. normalized capacitance vector of with an 8-electrode sensor withECT 28 elements. The28 flow patterns The flow patterns the samples aresingle annular, single bar, and Additionally, two-bar, respectively. of the samples are of annular, stratified, bar, stratified, and two-bar, respectively. each flow Additionally, each flow has 10,000 pairs of samples. pattern has 10,000 pairspattern of samples. The 8-electrode ECT sensor The 8-electrode ECT sensormodel modelininCOMSOL COMSOLMultiphysics Multiphysicsisisshown shownininFigure Figure1.1.The Thematerial material ofofthe thesensor sensorpipe pipeisisset settotobebePVC PVCwith witha arelative relativepermittivity permittivityofof2.2.The Thelower lowerand andhigher higherpermittivity permittivity values valuesofofthe theflow floware are11and and2.7, 2.7,respectively. respectively.The Thediameter diameterofofthe thepipe pipeisis70 70mm mmand andthe thethickness thicknessofof the betweentwo twoadjacent adjacentelectrodes electrodesis is 5 degrees span angle of thepipe pipeisis3.5 3.5 mm. mm. The gap between 5 degrees so so thatthat thethe span angle of each each electrode 40 degrees. The round imaging cross-section is divided a mesh 64 × 64 mesh gridin electrode is 40 is degrees. The round imaging cross-section is divided into a 64into × 64 grid which, which, in total, 3228 effective total, has 3228 has effective pixels. pixels.

Figure1.1.The Thestructure structureofofthe the2D 2Dsimulation simulationmodel. model. Figure

Thefour fourflow flowpatterns patternschosen chosenfor forthe thebenchmark benchmarkdataset datasetare aretypical typicaltwo-phase two-phaseflow flowpatterns patterns The that commonly occur in the industrial field, and other complex flow patterns can be regarded that commonly occur in the industrial field, and other complex flow patterns can be regarded asas combinations of these flows. Although the names of these four flow patterns may not be the same—

Sensors 2018, 18, 3701

4 of 20

Sensors 2018, 18, x FOR PEER REVIEW

4 of 20

combinations of these flows. Although names these fourand flow patterns maymay not be be the same—for for example, the single bar flow is alsothe called theofcore flow two-bar flow mentioned as example, the single bar flow is also called the core flow and two-bar flow may be mentioned the the two-object flow—they are mostly studied in ECT image reconstruction research, such as as those two-object in flow—they are mostly studiedTo in ECT image reconstruction such as flow those reported reported References [1,12,14,16]. describe the phantomsresearch, of different patterns in References [1,12,14,16]. To describe the phantoms of different flow patterns quantitatively, quantitatively, certain parameters are selected. The parameter describing annular flow certain is the parameters parameter describing flow the thickness of theand annular, which thickness ofare theselected. annular,The which is normalized withannular respect to theisradius of the sensor denoted by is normalized with respect tonormalized the radius of the sensor byselected, T. For the flow, the T. For the stratified flow, the height of the and flowdenoted surface is i.e.,stratified H. For the single normalized height the flow point surface is selected, i.e., H. For thethe single bar, the position of the center bar, the position ofof the center C(x,y) of the bar, of which coordinates are normalized with point C(x,y) of sensor the bar,radius, of which are the normalized withbar respect to the sensor is respect to the is the alsocoordinates used besides normalized radius, R. For theradius, two-bar also used besides the normalized bar radius, R. For the two-bar distribution, the normalized radii—i.e., distribution, the normalized radii—i.e., R1 and R2—and the positions of the center points of the two R1 andi.e., R2—and positions of the points of the 2two bars, i.e., and Cpatterns all used. 1 (x,y)flow 2 (x,y), arewith bars, C1(x,y)theand C2(x,y), are center all used. Figure depicts theCfour the Figure 2 depictsparameters. the four flow patterns with the corresponding parameters. corresponding

(a)

(b)

(c)

(d)

The four four flow flow patterns in the dataset. (a) annular; (b) stratified; (c) single bar; (d) two-bar. Figure 2. The

2.2. The The Simulation Simulation Part Part of of the the Benchmark Benchmark Dataset 2.2. Dataset Based Based on on the the 3D 3D Models Models Another simulation simulation part of the the benchmark benchmark dataset dataset for for evaluating evaluating and and comparing comparing the the ECT ECT image image Another part of reconstruction algorithms is also built based on 3D models. This part contains capacitance vectors reconstruction algorithms is also built based on 3D models. This part contains capacitance vectors corresponding toto8080 cases, including the capacitance vectors of the full and full empty pipes for calibration, corresponding cases, including the capacitance vectors of the and empty pipes for 2 sensitivity matrices for the 8-electrode sensor, and the 12-electrode sensor, respectively, and 12 calibration, 2 sensitivity matrices for the 8-electrode sensor, and the 12-electrode sensor, respectively, normalized permittivity distribution vectors. and 12 normalized permittivity distribution vectors. The four four flow flow patterns and the the pixel division of of the in the based on on the the 3D The patterns and pixel division the samples samples in the dataset dataset based 3D simulation models are the same as those in Section 2.1. Three relative permittivity values—2.7 (e.g., simulation models are the same as those in Section 2.1. Three relative permittivity values—2.7 (e.g., oil), 3.8 oil), 3.8 (e.g., (e.g., sand), sand), and and 80 80 (e.g., (e.g., water)—are water)—are investigated, investigated, covering covering situations situations of of low-contrast low-contrast and and high-contrast permittivity changes. Table 1 provides normalized parameters describing the phantoms high-contrast permittivity changes. Table 1 provides normalized parameters describing the and the corresponding phase ratio of the material with a high permittivity in each phantom. phantoms and the corresponding phase ratio of the material with a high permittivity in each

phantom.

Table 1. The phantom parameters of the 3D simulation part of the benchmark dataset.

Table 1. The phantom parameters of the 3D simulation part of the benchmark dataset. Flow Pattern Parameter Phase Ratio T Parameter 0.05 T Annular 0.30 0.05 0.55 Annular 0.30 H 0.55 0.25 Stratified H 0.50 0.75 0.25 Stratified 0.50 C(x,y) R (0,0) 0.29 0.75 Single Bar (0,0) 0.37 C(x,y) R (0.5,0) 0.50 (0,0) 0.29 SingleC1(x,y) Bar C2(x,y) R1 (0,0) 0.37 0.29 Two-bar (0.5,0) (−0.5,0) (0.5,0) 0.290.50 C1(x,y) C2(x,y) 0.37 R1 R2 0.29 0.29 Two-bar (−0.5,0) (0.5,0) 0.29 0.37 0.37 0.37

Flow Pattern

Phase Ratio 10% 50% 80% 19.58% 50% 80.42% 7.93%

R2 13.88% 0.29 0.3725% 0.37

15.86% 21.81% 27.76%

10% 50% 80% 19.58% 50% 80.42% 7.93% 13.88% 25% 15.86% 21.81% 27.76%

Sensors 2018, 18, 3701

5 of 20

Sensors 2018, 2018, 18, 18, xx FOR FOR PEER PEER REVIEW REVIEW Sensors

of 20 20 55 of

To data regarding thethe different phantoms, a 3Daa8-electrode ECT Toobtain obtainthe thesimulated simulatedcapacitance capacitance data regarding the different phantoms, 3D 8-electrode 8-electrode To obtain the simulated capacitance data regarding different phantoms, 3D sensor model and a 3D 12-electrode ECT sensor model are built in the COMSOL Multiphysics software. ECT sensor sensor model model and and aa 3D 3D 12-electrode 12-electrode ECT ECT sensor sensor model model are are built built in in the the COMSOL COMSOL Multiphysics Multiphysics ECT Figure 3 depicts 3D 8-electrode ECT sensor the simulation. The inner of the software. Figure 3the 3 depicts depicts the 3D 3D 8-electrode 8-electrode ECTmodel sensorfor model for the the simulation. simulation. Thediameter inner diameter diameter software. Figure the ECT sensor model for The inner pipe is 70 mm and the outer diameter is 80 mm. The length of the sensor is 370 mm, of which the of the the pipe pipe is is 70 70 mm mm and and the the outer outer diameter diameter is is 80 80 mm. mm. The The length length of of the the sensor sensor is is 370 370 mm, mm, of of which which of electrode length is 140 mm. The gap between the two adjacent electrodes is 5 degrees so that the span the electrode electrode length length is is 140 140 mm. mm. The The gap gap between between the the two two adjacent adjacent electrodes electrodes is is 55 degrees degrees so so that that the the the angle of each electrode is 40 degrees and 25 degrees for the 8-electrode sensor and the 12-electrode span angle angle of of each each electrode electrode is is 40 40 degrees degrees and and 25 25 degrees degrees for for the the 8-electrode 8-electrode sensor sensor and and the the 1212span sensor, respectively. electrode sensor, respectively. respectively. electrode sensor,

(a) (a)

(b) (b)

(c) (c) Figure ECT sensor. (a)(a) AA 3D view of the sensor; (b) (b) a(b) 3Daa Figure 3. 3. The The3D 3Dsimulation simulationmodel modelofof ofthe the8-electrode 8-electrode ECT sensor. (a) A 3D 3D view of the the sensor; 3D simulation model the 8-electrode ECT sensor. view of sensor; view of aof single bar bar flow; (c) the of each partpart of the 3D view view of single bar flow; flow; (c) length the length length of each each part of sensor. the sensor. sensor. 3D aa single (c) the of of the

Considering data Considering both both the the computed computed accuracy accuracy and and time time cost, cost, the the capacitance capacitance data data in in this this benchmark benchmark Considering both the computed accuracy and time cost, the capacitance in this benchmark dataset custom mesh in COMSOL Multiphysics with the maximum element dataset are are computed computedbased based on on aaa custom custom mesh mesh in in COMSOL COMSOL Multiphysics Multiphysics with with the the maximum maximum element element dataset are computed based on size set to 20.4 mm, the minimum element size set to 1.48 mm, and the maximum element growth rate size set set to to 20.4 20.4 mm, mm, the the minimum minimum element element size size set set to to 1.48 1.48 mm, mm, and and the the maximum maximum element element growth growth size set 1.4. pipepipe case,case, the total meshmesh element number is 1,119,690. ratetoset set toFor 1.4.the Forempty the empty empty pipe case, the total total mesh element number is 1,119,690. 1,119,690. rate to 1.4. For the the element number is The capacitances among the different electrode combinations are dependent on The capacitances capacitances among among the the different different electrode electrode combinations combinations are are dependent dependent on on the the relative relative The the relative permittivity, the phase ratio, and the flow pattern. Figure 4 is an example of how these factors permittivity, the the phase phase ratio, ratio, and and the the flow flow pattern. pattern. Figure Figure 44 is is an an example example of of how how these these factors factors matter, matter, permittivity, matter, where the capacitance vectors of each pattern with one phaseratio ratiounder underthree three different different where the the capacitance capacitance vectors of each each flowflow pattern with one phase ratio under three different where vectors of flow pattern with one phase permittivity values are and the vectors of the permittivity values values are are compared, compared, and and the the capacitance capacitance vectors vectors of of the the empty empty pipe pipe and and the the full full pipe pipe permittivity compared, capacitance empty pipe and the full pipe under these three permittivity values are also given for calibration. All these capacitance values under these these three three permittivity permittivity values values are are also also given given for for calibration. calibration. All All these these capacitance capacitance values values are are under are simulated simulated based based on on the the 8-electrode 8-electrodesensor. sensor.The Thecorresponding correspondingcapacitance capacitancedata dataare aregiven givenin inTable Table2.2. 2. simulated based on the 8-electrode sensor. The corresponding capacitance data are given in Table 33 Calculated Calculated Capacitance Capacitance (pF) (pF)

2.5 2.5 22

11 2.7 2.7 3.8 3.8 80 80

1.5 1.5 11

0.5 0.5 00 00

55

10 10

15 15 CapacitanceIndex Index Capacitance

20 20

25 25

(a) (a) 44

33

66 2.7 2.7 3.8 3.8 80 80

44

2.5 2.5 22

33

1.5 1.5

22

11

11

0.5 0.5 00 00

2.7 2.7 3.8 3.8 80 80

55

Calculated Calculated Capacitance Capacitance (pF) (pF)

Calculated Calculated Capacitance Capacitance (pF) (pF)

3.5 3.5

55

10 10

15 15 CapacitanceIndex Index Capacitance

20 20

25 25

00 00

(b) (b)

55

10 10

15 15 CapacitanceIndex Index Capacitance

(c) (c) Figure 4. Cont.

20 20

25 25

Sensors 2018, 18, 3701 Sensors 2018, 18, x FOR PEER REVIEW

6 of 20 6 of 20 2.5

2

2.7 3.8 80

Calculated Capacitance (pF)

Calculated Capacitance (pF)

2.5

1.5 1 0.5 0 0

5

10

15 Capacitance Index

20

25

2

2.7 3.8 80

1.5 1 0.5 0 0

5

(d)

10

15 Capacitance Index

20

25

(e)

Figure 4. The Thecapacitance capacitancedata data examples based simulation, (a) empty the empty andpipe; full examples based on on the the simulation, (a) the pipe pipe and full pipe; the 50% annular distribution; (c) the 19.58% stratified distribution; (d) the 13.88% single (b) the(b)50% annular distribution; (c) the 19.58% stratified distribution; (d) the 13.88% single bar bar distribution; (e) the 27.76% two-bar distribution. distribution; (e) the 27.76% two-bar distribution. Table 2. The capacitance data related to the examples in Figure 4 (in pF). Table 2. The capacitance data related to the examples in Figure 4 (in pF). Electrode Pair

Empty and Full Pipe

50% Annular

19.58% Stratified

13.88% Single Bar

27.76% Two-Bar

Electrode Stratified 13.88% Two-Bar ε Empty and 1 Full 2.7Pipe3.8 80 50% 2.7Annular 3.8 80 19.58% 2.7 3.8 80 2.7 3.8 Single 80 Bar2.7 27.76% 3.8 80 Pair 1-2 2.191 2.615 2.796 2.807 2.681 2.907 3.552 2.182 2.178 2.176 2.179 2.173 2.224 2.218 2.220 2.107 1 2.7 3.8 80 2.7 3.8 80 2.7 3.8 80 2.7 3.8 80 2.7 3.8 80 ε 1-3 0.110 0.282 0.382 1.579 0.195 0.246 1.463 0.103 0.101 0.096 0.112 0.112 0.151 0.142 0.151 0.144 1-2 2.191 2.615 2.796 2.807 2.681 2.907 3.552 2.182 2.178 2.176 2.179 2.173 2.224 2.218 2.220 2.107 1-4 0.056 0.147 0.203 1.337 0.069 0.077 0.806 0.062 0.064 0.061 0.068 0.071 0.082 0.085 0.095 0.106 1-3 0.110 0.2820.046 0.382 0.2460.055 1.463 0.096 0.062 0.1120.066 0.1120.069 0.151 0.1221.579 0.170 0.195 1.273 0.052 0.6270.103 0.060 0.101 0.063 0.063 0.1060.142 0.130 0.151 0.200 0.144 1-5 1-4 0.056 0.1470.056 0.203 0.0770.077 0.806 0.061 0.068 0.0680.071 0.0710.082 0.082 1-6 0.1471.337 0.203 0.069 1.338 0.069 0.8070.062 0.062 0.064 0.064 0.061 0.0850.085 0.095 0.095 0.106 0.106 0.2831.273 0.383 0.052 1.579 0.195 1.4630.060 0.103 0.063 0.101 0.097 0.1420.106 0.151 0.130 0.144 0.200 1-7 1-5 0.046 0.1220.110 0.170 0.0550.247 0.627 0.063 0.112 0.0620.112 0.0660.151 0.069 2.623 2.804 2.812 2.686 2.912 3.556 2.187 2.187 2.184 2.174 2.169 2.223 2.213 2.215 2.102 1-8 1-6 0.056 0.1472.199 0.203 1.338 0.069 0.077 0.807 0.062 0.064 0.061 0.068 0.071 0.082 0.085 0.095 0.106 2-3 2.193 2.617 2.799 2.808 2.681 2.907 3.552 2.182 2.179 2.169 2.179 2.173 2.169 2.159 2.149 2.098 1-7 0.110 0.2830.110 0.383 0.2470.246 1.463 0.097 0.112 0.1120.112 0.1120.107 0.151 2-4 0.2821.579 0.382 0.195 1.579 0.195 1.4620.103 0.122 0.101 0.124 0.097 0.1050.142 0.103 0.151 0.084 0.144 1-8 2.199 2.6230.056 2.804 2.9120.077 3.556 2.184 0.068 2.1740.071 2.1690.057 2.223 2-5 0.1472.812 0.203 2.686 1.338 0.069 0.8072.187 0.073 2.187 0.077 0.081 0.0872.213 0.097 2.215 0.110 2.102 0.1222.808 0.170 2.681 1.274 0.052 0.6272.182 0.052 2.179 0.054 0.068 0.0582.159 0.060 2.149 0.056 2.098 2-6 2-3 2.193 2.6170.046 2.799 2.9070.055 3.552 2.169 0.062 2.1790.066 2.1730.052 2.169 2-7 0.1471.579 0.204 0.195 1.339 0.069 0.8070.122 0.048 0.124 0.047 0.044 0.0670.105 0.070 0.103 0.064 0.084 2-4 0.110 0.2820.056 0.382 0.2460.077 1.462 0.097 0.068 0.1120.071 0.1120.071 0.107 2-8 0.110 0.283 0.383 1.580 0.195 0.246 1.462 0.106 0.105 0.102 0.112 0.112 0.167 0.161 0.175 0.182 2-5 0.056 0.1472.196 0.203 1.338 0.069 0.077 0.807 0.073 0.077 0.081 0.068 0.071 0.057 0.087 0.097 0.110 3-4 2.620 2.801 2.810 2.681 2.908 3.552 2.231 2.228 1.765 2.177 2.171 2.186 2.159 2.149 2.096 2-6 0.046 0.1220.110 0.170 0.0550.246 0.627 0.068 0.112 0.0620.112 0.0660.104 0.052 0.2831.274 0.383 0.052 1.579 0.195 1.4620.052 0.158 0.054 0.177 0.366 0.1440.058 0.153 0.060 0.148 0.056 3-5 3-6 0.1471.339 0.203 0.069 1.338 0.069 0.8070.048 0.065 0.047 0.071 0.258 0.0670.067 0.069 0.070 0.063 0.064 2-7 0.056 0.1470.056 0.204 0.0770.077 0.807 0.044 0.068 0.0680.071 0.0710.051 0.071 3-7 0.1221.580 0.170 0.195 1.274 0.052 0.6280.106 0.036 0.105 0.034 0.041 0.0450.161 0.044 0.175 0.034 0.182 2-8 0.110 0.2830.046 0.383 0.2460.055 1.462 0.102 0.062 0.1120.066 0.1120.045 0.167 3-8 0.056 0.147 0.203 1.338 0.069 0.077 0.807 0.048 0.047 0.044 0.068 0.071 0.071 0.067 0.069 0.063 3-4 2.196 2.620 2.801 2.810 2.681 2.908 3.552 2.231 2.228 1.765 2.177 2.171 2.186 2.159 2.149 2.096 2.196 2.620 2.801 2.809 2.680 2.906 3.550 2.742 3.016 4.974 2.182 2.176 2.190 2.217 2.218 2.099 4-5 3-5 0.110 0.2830.110 0.383 0.2460.246 1.462 0.366 0.112 0.1120.112 0.1120.105 0.104 4-6 0.2831.579 0.383 0.195 1.579 0.195 1.4620.158 0.268 0.177 0.370 2.984 0.1610.144 0.175 0.153 0.182 0.148 3-6 0.056 0.1470.056 0.203 0.0770.077 0.807 0.258 0.068 0.0680.071 0.0710.051 0.051 4-7 0.1471.338 0.204 0.069 1.338 0.069 0.8070.065 0.065 0.071 0.070 0.227 0.0670.067 0.070 0.069 0.064 0.063 4-8 0.1221.274 0.170 0.052 1.274 0.052 0.6270.036 0.052 0.034 0.053 0.066 0.0580.045 0.060 0.044 0.056 0.034 3-7 0.046 0.1220.046 0.170 0.0550.055 0.628 0.041 0.062 0.0620.066 0.0660.052 0.045 2.6251.338 2.807 0.069 2.815 2.683 3.5550.048 2.749 0.047 3.028 5.078 2.2120.067 2.214 0.069 2.101 0.063 5-6 3-8 0.056 0.1472.201 0.203 0.0772.910 0.807 0.044 2.172 0.0682.166 0.0712.183 0.071 5-7 0.110 0.283 0.383 1.580 0.195 0.246 1.463 0.155 0.173 0.328 0.112 0.112 0.104 0.142 0.151 0.144 4-5 2.196 2.620 2.801 2.809 2.680 2.906 3.550 2.742 3.016 4.974 2.182 2.176 2.190 2.217 2.218 5-8 0.056 0.147 0.204 1.338 0.069 0.077 0.807 0.073 0.077 0.080 0.068 0.071 0.057 0.085 0.094 0.106 2.099 4-6 0.110 0.2832.193 0.383 0.2462.902 1.462 2.984 2.181 0.1122.175 0.1122.187 0.105 6-7 2.6171.579 2.798 0.195 2.807 2.675 3.5480.268 2.228 0.370 2.226 1.846 2.1670.161 2.156 0.175 2.106 0.182 6-8 0.2831.338 0.382 0.069 1.579 0.195 1.4620.065 0.121 0.070 0.122 0.097 0.1050.067 0.103 0.070 0.084 0.064 4-7 0.056 0.1470.110 0.204 0.0770.246 0.807 0.227 0.112 0.0680.112 0.0710.106 0.051 7-8 2.6221.274 2.803 0.052 2.812 2.684 3.5550.052 2.174 0.053 2.171 2.161 2.1520.058 2.141 0.060 2.089 0.056 4-8 0.046 0.1222.198 0.170 0.0552.910 0.627 0.066 2.170 0.0622.164 0.0662.162 0.052 5-6 2.201 2.625 2.807 2.815 2.683 2.910 3.555 2.749 3.028 5.078 2.172 2.166 2.183 2.212 2.214 2.101 5-7 0.110 0.283 0.383 1.580 0.195 0.246 1.463 0.155 0.173 0.328 0.112 0.112 0.104 0.142 0.151 0.144 For an infinite parallel-plate capacitor, the capacitance value increases along with an increase of 5-8 0.056 0.147 0.204 1.338 0.069 0.077 0.807 0.073 0.077 0.080 0.068 0.071 0.057 0.085 0.094 0.106 the6-7 permittivity value2.798 of the2.807 medium the two plates so that relationship between 2.193 2.617 2.675 between 2.902 3.548 2.228electrode 2.226 1.846 2.181 2.175the2.187 2.167 2.156 2.106 6-8 0.110 0.382 1.579 0.246 1.462 0.121permittivity 0.122 0.097 value 0.112 is 0.112 0.106 0.105 0.103 0.084 variation of the0.283 capacitance and0.195 the variation of the linear. However, it can be 7-8 that 2.198 2.622 sensor, 2.803 the 2.812relationship 2.684 2.910 between 3.555 2.174 2.171 2.161 value 2.170 and 2.164the 2.162 2.152 2.141 2.089 found for ECT the capacitance permittivity value is

nonlinear, especially for the case in which the permittivity variation is of a high contrast. In Figure 4a, For an infinite parallel-plate capacitor, the capacitance value increases along with an 80 increase of the capacitance values of the adjacent electrode pairs with permittivity values of 3.8 and are very the permittivity value of the medium between the two electrode plates so that the relationship close and, in Figure 4d,e, the capacitance values of the adjacent electrode pairs with a permittivity between variation the capacitance and thepermittivity variation of values the permittivity value of 80 are evenofsmaller than those with of 2.7 andvalue 3.8. is linear. However, it can be found that for ECT sensor, the relationship between the capacitance theregion permittivity This phenomenon appears because, for adjacent electrode pairs, only value a veryand small in the value is nonlinear, especially for the case in which the permittivity variation is of a high contrast. In circular ECT imaging area has a very sharp positive sensitivity while most of the region has a negative Figure 4a, the capacitance values of theelectrode adjacentpairs, electrode permittivity values ofimaging 3.8 and sensitivity. Meanwhile, for the opposite mostpairs of thewith region in the circular ECT 80 are very close and, in Figure 4d,e, the capacitance values of the adjacent electrode pairs with a area has a relatively high positive sensitivity while a relatively small region has a negative sensitivity. permittivity valueofofthe 80 sensitivity are even smaller than those with values of 2.7 and The comparisons map appearance of thepermittivity adjacent electrode pairs and 3.8. the opposite This phenomenon appears because, for adjacent electrode pairs, only a very small region in the electrode pairs regarding the negative sensitivity characterizations are demonstrated clearly in Figure 5. circular ECT imaging area has a very sharp positive sensitivity while most of the region has a negative The effect of the negative sensitivity map can also be reflected from the capacitance values of the sensitivity. Meanwhile, for the opposite electrode pairs, most of the region in the circular ECT imaging area has a relatively high positive sensitivity while a relatively small region has a negative sensitivity. The comparisons of the sensitivity map appearance of the adjacent electrode pairs and the

Sensors 2018, 18, x FOR PEER REVIEW

7 of 20

Sensors 2018, 18, 3701

7 of 20

opposite electrode pairs regarding the negative sensitivity characterizations are demonstrated clearly in Figure 5. The effect of the negative sensitivity map can also be reflected from the capacitance values adjacent electrode pairs while permittivity value is value 80 andisthe are annular stratified. of the adjacent electrode pairsthe while the permittivity 80distributions and the distributions areorannular or It was found from Figure 4b that the capacitance values of the adjacent electrode pairs, while the stratified. It was found from Figure 4b that the capacitance values of the adjacent electrode pairs, permittivity distributiondistribution is 50% annular, areannular, about 3.55 which even larger values while while the permittivity is 50% arepF, about 3.55are pF, which arethan eventhe larger than the the pipewhile is full. for the 19.58% distribution 4c), certain values theFurthermore, pipe is full. Furthermore, forstratified the 19.58% stratified (Figure distribution (Figurecapacitance 4c), certain values of thevalues adjacent electrode pairs reach 5.08 pF.reach 5.08 pF. capacitance of the adjacent electrode pairs -3

-3

x 10 8

x 10 20

7

15

6 5

10

4

5 3

0

2 1

-5 1

1

0 -1

-0.5

-1

0.5

0

0

1

-1

(a)

(b)

(c)

-3

x 10 8 -3

x 10

7

20

6

15

5

10

4 3

5

2

0 -5 1

1 1 0.5

0

-0.5

-1

-1

(d)

-0.5

0

0.5

0 -1

(e)

(f)

Figure 5. The Thesensitivity sensitivitymap mapofofthe the8-electrode 8-electrode ECT sensor, a 3D view, adjacent electrode ECT sensor, (a)(a) a 3D view, adjacent electrode pair;pair; (b) (b) a 2D view, adjacent electrode pair; a 2Dview viewofofthe thenegative negativesensitivity sensitivityzone, zone, adjacent adjacent electrode a 2D view, adjacent electrode pair; (c)(c) a 2D pair; pair; (d) a 3D view, view, opposite opposite electrode electrode pair; pair; (e) (e) a 2D view, view, opposite opposite electrode electrode pair; pair; (f) (f) aa 2D 2D view of the negative sensitivity zone, zone, opposite opposite electrode electrode pair. pair.

With thethe medium’s permittivity fromfrom a lowa value to a high the ECT imaging With the thechange changeofof medium’s permittivity low value to avalue highinvalue in the ECT area, the area, capacitance values among different electrode pairs behave totally totally differently in terms imaging the capacitance values the among the different electrode pairs behave differently in of their properties and nonlinearities. Cui et al. [43] and Yang et al. [44] reported and preliminarily terms of their properties and nonlinearities. Cui et al. [43] and Yang et al. [44] reported and analyzed the effect of the nonlinearity capacitancesofbetween different electrode pairselectrode on ECT image preliminarily analyzed the effect of theofnonlinearity capacitances between different pairs reconstruction. This issue may need to be investigated more deeply in future studies on ECT image on ECT image reconstruction. This issue may need to be investigated more deeply in future studies reconstruction, particularly while the permittivity distribution inside the sensor has athe relatively higha on ECT image reconstruction, particularly while the permittivity distribution inside sensor has contrast relativelyvariation. high contrast variation. 2.3. The Deep Autoencoder and the Iteration Method Based on It 2.3. The Deep Autoencoder and the Iteration Method Based on It As is known in the ECT field, the nonlinear relationship between capacitance and permittivity As is known in the ECT field, the nonlinear relationship between capacitance and permittivity deteriorates the quality of the reconstructed image based on the linear model when the permittivity deteriorates the quality of the reconstructed image based on the linear model when the permittivity variation becomes large. This is because the linear model approximates the nonlinear relationship variation becomes large. This is because the linear model approximates the nonlinear relationship between capacitance data and the corresponding permittivity distribution by neglecting the higher between capacitance data and the corresponding permittivity distribution by neglecting the higher order terms of permittivity variation. When the permittivity variation becomes larger, the neglected order terms of permittivity variation. When the permittivity variation becomes larger, the neglected terms matters more, thus, the imaging reconstruction quality worsens. However, if the nonlinear terms matters more, thus, the imaging reconstruction quality worsens. However, if the nonlinear model is used for improving the quality of the image reconstruction, the real-time ability of the model is used for improving the quality of the image reconstruction, the real-time ability of the nonlinear model-based algorithms for online imaging should be considered. In this sense, better image nonlinear model-based algorithms for online imaging should be considered. In this sense, better reconstruction algorithms should be put forward to meet the requirements of both imaging quality image reconstruction algorithms should be put forward to meet the requirements of both imaging and speed. quality and speed. A deep autoencoder along with the iteration method proposed in Reference [41] provides a new A deep autoencoder along with the iteration method proposed in Reference [41] provides a new way to solve the ECT image reconstruction problem. This method is a deep supervised autoencoder way to solve the ECT image reconstruction problem. This method is a deep supervised autoencoder which has an encoder and a decoder (with five layers each) that can deal with both the former which has an encoder and a decoder (with five layers each) that can deal with both the former problem and the inverse ECT problem. The nonlinear relationship from the permittivity distribution problem and the inverse ECT problem. The nonlinear relationship from the permittivity distribution to the capacitance data is modeled by the encoder (F(·)) and, conversely, the reconstruction from the to the capacitance data is modeled by the encoder (F(·)) and, conversely, the reconstruction from the capacitance data to the permittivity distribution is solved by the decoder (G(·)) of the deep autoencoder.

capacitance data to the permittivity distribution is solved by the decoder (G(·)) of the deep autoencoder. Suppose x is the vector of the permittivity distribution, y is the capacitance data vector, xˆ is the reconstructed permittivity distribution, and yˆ is the estimated capacitance data calculated from the permittivity distribution, then, according to the structure of the autoencoder in Figure 6, there is Sensors 2018, 18, 3701

8 of 20

 yˆ = F ( x ) (1)  = G(y) Suppose x is the vector of the permittivity y is the capacitance data vector, xˆ is the  xˆdistribution, reconstructed permittivity distribution, and yˆ is the estimated capacitance data calculated from the To take into account both the forward problem and the inverse ECT problem under the deep permittivity distribution, then, according to the structure of the autoencoder in Figure 6, there is autoencoder framework, another two vectors— x and y —are defined as follows: ( (x()y ))  y = F ( xˆ )yˆ ==FF(G (1) (2)  xˆ = G (y) x = G ( yˆ ) = G ( F (x ))

Figure 6. 6. The The structure structure of of the the deep deep autoencoder. Figure autoencoder.

To into account both the problem and the inverse ECT problemby under the deep Thetake autoencoder is trained by forward minimizing the loss function, which is denoted L. Because of autoencoder framework, another two vectors—e x and e—are defined asparts, follows: the four estimated variables in Equations (1) and (2), Ly consists of four see Equation (3), where ( these four parts of losses, l is a particular reconstruction error α1 , α2 , α3 , and α4 are the weights of

y e = F (xˆ ) = F ( G (y)) which chosen to be mean squared error (MSE), as is described in Equation (4), for any two (2) ne x = G (yˆ ) = G ( F (x)) dimensional vectors v and vˆ .

The autoencoder is trained minimizing the loss function, which is denoted by L. Because of the L = αby 1 L1 + α 2 L2 + α 3 L3 + α 4 L4 (3) four estimated variables in Equations (2),  of four parts, see Equation (3), where α1 , ˆ L consists = α1l ( y, yˆ )(1) + αand 2 l ( x , x ) + α 3l ( y , y ) + α 4 l ( x , x ) α2 , α3 , and α4 are the weights of these four parts of losses, l is a particular reconstruction error which n chosen to be mean squared error (MSE), is 1described 2in Equation (4), for any two n-dimensional l ( v, as vˆ ) =  ( vi − vˆi ) (4) n i =1 vectors v and v. ˆ L = α1 L1 + α2 L2 + α3 L3 + α4 L4 (3) Although the proposed deep autoencoder would take a lot of time to train, when it is well= α1 l (y, yˆ ) + α2 l (x, xˆ ) + α3 l (y, y e) + α4 l (x, e x) trained, the deep autoencoder can be faster than most traditional ECT image reconstruction 1 n solved by some time-consuming finite element algorithms where the forward problem is usually (4) l ( v, vˆ ) = ∑ (vi − vˆi )2 n i =1 method (FEM) and the image reconstruction algorithm would also consume a lot of calculation resources. Some iterative algorithms even need to repeatedly solve the forward problem and the Although the proposed deep autoencoder would take a lot of time to train, when it is well-trained, inverse problem, which will lead to a good image reconstruction quality but will sacrifice too much the deep autoencoder can be faster than most traditional ECT image reconstruction algorithms where time to satisfy its online use. If the deep autoencoder is used to implement the iterative process, the the forward problem is usually solved by some time-consuming finite element method (FEM) and calculation time can be saved and image reconstruction quality will be promoted. Thus, an iteration the image reconstruction algorithm would also consume a lot of calculation resources. Some iterative method is inspired by the following Landweber iteration [45]: algorithms even need to repeatedly solve the forward problem and the inverse problem, which will T ˆ − αwill lead to a good image reconstruction xˆquality If Sxˆ k − y ) too much time to satisfy its online use.(5) k +1 = x kbut k S (sacrifice the deep autoencoder is used to implement the iterative process, the calculation time can be saved and where xk is the calculated permittivity distribution at the kth step and y is the normalized capacitance image reconstruction quality will be promoted. Thus, an iteration method is inspired by the following vector. S is the sensitivity map in the linear model of the ECT, which maps the permittivity Landweber iteration [45]: distribution to the capacitance data and corresponds to F(·) in Equation (1), and ST maps the xˆ k+1 = xˆ k − αk ST (Sˆxk − y) (5) where xk is the calculated permittivity distribution at the kth step and y is the normalized capacitance vector. S is the sensitivity map in the linear model of the ECT, which maps the permittivity distribution to the capacitance data and corresponds to F(·) in Equation (1), and ST maps the capacitance data to

Sensors 2018, 18, 3701

9 of 20

the permittivity distribution as G(·). So, if the deep autoencoder is used to implement the Landweber iteration, the equation should be Equation (6). xˆ k+1 = xˆ k − αk G(F(xˆ k ) − y)

(6)

2.4. Image Reconstruction Examples Based on the Simulation In this paper, four image reconstruction algorithms, i.e., the LBP [46], the projected Landweber [45], the total variation (TV) based regularization algorithm [47], and the deep autoencoder introduced above, are executed on the 3D model-based simulation part of the benchmark dataset. In order to quantitatively evaluate the ECT image reconstruction results and compare the performance of the different reconstruction algorithms, the evaluation criteria should be determined. The commonly used criteria include the relative image error of the reconstruction, the correlation coefficient between the real permittivity distribution and reconstructed permittivity distribution, and the other parameters related to the permittivity distribution, such as the phase ratio (phase concentration). 1. Relative image error The relative image error is defined as the relative error of the reconstructed permittivity vector gˆ with respect to the real permittivity vector g, as is shown below: Relative image error =

kgˆ − gk kgk

(7)

2. Correlation coefficient The correlation coefficient indicates the similarity between the reconstructed permittivity distribution and the original permittivity distribution, which is defined as Equation (8), where gˆ ˆ gˆi is the ith element of g, ˆ g is the mean of g, and gi is the ith element of g. is the mean of g, Correlation coefficient = q

∑iN=1 ( gˆi − gˆ )( gi − g) ∑iN=1

( gˆi − gˆ )

2

∑iN=1

( gi − g )

2

(8)

3. Phase ratio error The ECT image reconstruction is commonly used for evaluating the phase ratio in the application of the two-phase flow measurement, thus, the phase ratio error of the reconstructed image is also an important criterion. In this paper, the ‘phase ratio’ is the phase concentration of the medium with the higher permittivity value, which is computed by summing the permittivity distribution vector, i.e., the gray-scale value of the permittivity distribution. By using Rr to stand for the real phase ratio, which can be calculated according to the phantom, and Re to stand for the estimated phase ratio from the reconstructed permittivity distribution, the phase ratio error can be defined as Phase ratio error = Re − Rr

(9)

Some image reconstruction examples calculated by the 8-electrode capacitance vectors in the four flow patterns with a relative permittivity of 2.7 in Table 2 are shown in Figure 7, where the comparison of the image reconstruction results by different reconstruction algorithms is demonstrated. The related criteria data are listed in Table 3. Note that the phase ratio is estimated by summing the reconstructed normalized permittivity vector, and there is an artifact in the reconstructed images, therefore, the phase ratio error does not have a positive correlation with the relative image error.

quality of the LBP is worse than the projected Landweber iteration and the TV. As for the reconstruction results of the projected Landweber iteration and the TV, they are much more similar to the real permittivity distribution than the LBP results visually, however, when compared to the criteria data in Table 3, it can be found that the projected Landweber iteration and the TV perform similarly two annular flows and two stratified flows cases, but the TV algorithm shows a 10 better Sensors 2018,in 18,the 3701 of 20 performance in the single bar and two-bar flows cases evaluated by all the criteria data. Real Phantom

LBP

Landweber

TV

Autoencoder

Annular flow with phase ratio of 50%

Stratified flow with phase ratio of 19.58%

Single bar flow with phase ratio of 13.88%

Two-bar flow with phase ratio of 27.76%

Figure 7. The image reconstruction examples based on the 3D simulation. Table 3. The comparison of image reconstruction results based on the 3D simulation. Table 3. The comparison of image reconstruction results based on the 3D simulation. Relative Relative Correlation Correlation Estimated Estimated Phase PhaseRatio Ratio AlgorithmImage Error Coefficient Phase Ratio Error Image Error Coefficient Phase Ratio Error LBP 37.30% 0.8760 47.25% −2.75% LBP 37.30% 0.8760 47.25% −2.75% Landweber 22.40% 0.9518 46.21% −3.79% 50% annular Landweber 22.40% 0.9518 46.21% −3.79% TV 22.45% 0.9516 46.19% −3.81% 50% annular TV 22.45% 0.9516 46.19% −3.81% Autoencoder 10.88% 0.9881 50.88% 0.88% Autoencoder 10.88% 0.9881 50.88% LBP 40.19% 0.9095 17.67% −0.88% 1.91% LBP 40.19% 0.9095 17.67% Landweber 33.04% 0.9346 17.08% −−1.91% 2.50% 19.58% TV 0.9355 16.94% −−2.50% 2.64% stratified Landweber 32.99% 33.04% 0.9346 17.08% 19.58% stratifiedAutoencoder 4.23% 0.9989 19.57% −0.01% TV 32.99% 0.9355 16.94% −2.64% LBP 84.51% 0.6514 7.29% − 6.59% Autoencoder 4.23% 0.9989 19.57% −0.01% Landweber 53.64% 0.9134 7.30% −6.58% 13.88% single LBP 84.51% 0.6514 7.29% −6.59% TV 37.20% 0.9322 10.54% −3.34% bar Landweber 29.25% 53.64% 0.9134 7.30% −6.58% 0.9530 15.31% 1.44% 13.88% single barAutoencoder TV 37.20% 0.9322 10.54% −3.34% LBP 72.04% 0.7109 17.02% −10.74% Autoencoder 29.25% 0.9530 15.31% 1.44% Landweber 52.02% 0.8615 17.52% −10.24% 27.76% TV LBP 47.13% 0.8655 20.27% − 7.49% 72.04% 0.7109 17.02% −10.74% two-bar Autoencoder 0.9352 24.84% − 2.91% Landweber 30.64% 52.02% 0.8615 17.52% −10.24% 27.76% two-bar TV 47.13% 0.8655 20.27% −7.49% Autoencoder 30.64% 0.9352 24.84% −2.91% In Figure 7, the images reconstructed by the autoencoder are apparently much better than that Flow Pattern

Flow Pattern

Algorithm

the images constructed by the other three traditional algorithms in terms of the visual effect, and also shows that although the autoencoder is trainedAs byfor thethe 2Dother simulation dataset, its very Figure close to7 their corresponding real permittivity distributions. three algorithms, performance is still satisfying in the 3D simulation dataset. This means that the autoencoder has the reconstructed images of the LBP are far from the real permittivity distributions, especially for thea single bar and two-bar flow. This conclusion can also be supported by the results of the three criteria in Table 3: all the criteria data show that the quality of image reconstruction by the autoencoder is much better than those constructed by the three traditional algorithms. The image reconstruction quality of the LBP is worse than the projected Landweber iteration and the TV. As for the reconstruction results of the projected Landweber iteration and the TV, they are much more similar to the real permittivity distribution than the LBP results visually, however, when compared to the criteria data in Table 3, it can be found that the projected Landweber iteration and the TV perform similarly in the two annular flows and two stratified flows cases, but the TV algorithm shows a better performance in the single bar and two-bar flows cases evaluated by all the criteria data.

than the other three algorithms in phantom 1 and 2 of Figure 8, showing that the autoencoder does have some generalization ability. However, the results of phantom 3 and 4 are quite unsatisfactory and bars inside the annulus cannot be reconstructed completely, implying that the generalization ability of the autoencoder is not good enough to recognize all of the new flow patterns. In order to promote the image reconstruction quality of the deep autoencoder, the iteration method introduced Sensors 2018, 18, 3701 11 of 20 in Section 2.3 is implemented; see Figure 9. After using the iteration method, the image reconstruction results are improved as shown in Figure 9 and bars inside the annulus can be recognized because thereFigure is a single barshows flow in thealthough training the dataset. A good way of improving thesimulation generalization ability 7 also that autoencoder is trained by the 2D dataset, its Sensors 2018, 18, xisFOR PEER REVIEWin 11 ofare 20 is by enhancing the diversity of the the 3D training data, dataset. so, in future, more other flow pattern data performance still satisfying simulation This means that the autoencoder has a considered to be supplemented in the dataset to to increase generation ability of the methods based generalization ability to some extent. In order furtherthe examine the generalization ability of the generalization ability topatterns some extent. to further thesee generalization ability of the on machine learning. autoencoder, some flow not inIn theorder training datasetexamine are tested; Figure 8. autoencoder, some flow patterns not in the training dataset are tested; see Figure 8. Because there are only four flow patterns in theLandweber training dataset stratified, single IndexPermittivity Distribution LBP TV(i.e., annular, Autoencoder bar, and two-bar), the performance of recognizing other new flow patterns with the proposed autoencoder network depends on its generalization ability. The results of the autoencoder are better 1 than the other three algorithms in phantom 1 and 2 of Figure 8, showing that the autoencoder does have some generalization ability. However, the results of phantom 3 and 4 are quite unsatisfactory and bars inside the annulus cannot be reconstructed completely, implying that the generalization ability of the2autoencoder is not good enough to recognize all of the new flow patterns. In order to promote the image reconstruction quality of the deep autoencoder, the iteration method introduced in Section 2.3 is implemented; see Figure 9. After using the iteration method, the image reconstruction results are improved as shown in Figure 9 and bars inside the annulus can be recognized because there is a single 3 bar flow in the training dataset. A good way of improving the generalization ability is by enhancing the diversity of the training data, so, in future, more other flow pattern data are considered to be supplemented in the dataset to increase the generation ability of the methods based on machine learning. 4 IndexPermittivity Distribution LBP Landweber TV Autoencoder Figure 8. The image reconstruction examples of flow patterns not in the training training dataset. dataset.

1

Because IndexPermittivity there are only fourDistribution flow patterns in the training dataset (i.e., annular, stratified, single bar, AutoencoderAutoencoder-Based Iteration and two-bar), the performance of recognizing other new flow patterns with the proposed autoencoder network depends on its generalization ability. The results of the autoencoder are better than the 2 3 other three algorithms in phantom 1 and 2 of Figure 8, showing that the autoencoder does have some generalization ability. However, the results of phantom 3 and 4 are quite unsatisfactory and bars inside the annulus cannot be reconstructed completely, implying that the generalization ability of the autoencoder is 4not good enough to recognize all of the new flow patterns. In order to promote the 3 image reconstruction quality of the deep autoencoder, the iteration method introduced in Section 2.3 is implemented; see Figure 9. After using the iteration method, the image reconstruction results are improved as shown Figure 9 and bars inside the annulus can be method. recognized because there Figure in 9. The image reconstruction examples of the iteration is a single bar flow in the training dataset. A good way of improving the generalization ability 4 3. The Experiment of the Benchmark Dataset is by enhancing thePart diversity of the training data, so, in future, more other flow pattern data are considered to be supplemented in the dataset to increase the generation ability of the methods based The experiment part of the benchmark dataset is also built. This includes the static experiment on machine learning. Figure 8. The image reconstruction examples of flow patterns not in the training dataset. data of the 8-electrode capacitance vectors of the four flow patterns, each under the three situations, and empty and full pipes for the calibration. Besides, three capacitance vectors without other IndexPermittivity DistributionAutoencoderAutoencoder-Based Iteration information are given to researchers who are interested in ECT image reconstruction in order to test

3

4

Figure 9. The image reconstruction examples of the iteration method.

3. The Experiment Part of the Benchmark Dataset The experiment part of the benchmark dataset is also built. This includes the static experiment data of the 8-electrode capacitance vectors of the four flow patterns, each under the three situations, and empty and full pipes for the calibration. Besides, three capacitance vectors without other information are given to researchers who are interested in ECT image reconstruction in order to test

Sensors 2018, 18, 3701

12 of 20

Sensors 2018, 18, x FOR PEER REVIEW

12 of 20

3. The Experiment Part of the Benchmark Dataset their algorithms. Dynamic experiment data are also included in the dataset. The experiment devices The experiment part of the benchmark is also built.data This includes static experiment and image reconstruction examples based ondataset the experimental are given inthe this section. data of the 8-electrode capacitance vectors of the four flow patterns, each under the three situations, and 3.1. Theand Static Partcalibration. of the Benchmark Dataset empty fullExperiment pipes for the Besides, three capacitance vectors without other information are given to researchers who are interested in ECT image reconstruction in orderistoused test their algorithms. The Andeen–Hagerling high-precision capacitance bridge (AH-2550A) to measure the Dynamic experiment data are also included in the dataset. The experiment devices and image capacitance. Figure 10a shows the static experiment scenario, where the machine to the left is an AH reconstruction examples based experimental data are given in this section. capacitance bridge and that to on thethe right is an 8-electrode ECT sensor with a support. Figure 10b–e

show how theExperiment four flow patterns are implemented in the static experiment. 3.1. The Static Part of the Benchmark Dataset The ECT sensor is in the same structure as the 3D simulation model in Section 2.2. The pipe is capacitance bridge (AH-2550A) measure the madeThe by Andeen–Hagerling acrylic (PMMA), thehigh-precision relative permittivity of which is considered to is beused near to 3.8. The media capacitance. 10apatterns shows the experiment where to the leftthat is an AH that constructFigure the flow arestatic also acrylic. The scenario, flow patterns arethe themachine same four types are in capacitance bridge and that to the right is an 8-electrode ECT sensor with a support. Figure 10b–e the simulation: annular, stratified, single bar, and two-bar. Each of the flow patterns concludes 3 cases show how flow patterns are implemented the static experiment. in terms ofthe thefour corresponding parameter and phaseinratio; see Table 4.

(a)

(b)

(c)

(d)

(e)

Figure 10. 10. The static experiment setup, (a) the capacitance bridge and ECT sensor; (b) the annular distribution; (c) the stratified stratified distribution; distribution; (d) (d) the the single single bar bar distribution; distribution; (e) (e) the the two-bar two-bar distribution. distribution.

The ECT sensor in the same structure the experiment 3D simulation model in Sectiondataset. 2.2. The pipe is Table 4. Theis phantom parameters of theas static part of the benchmark made by acrylic (PMMA), the relative permittivity of which is considered to be near 3.8. The media that construct the flow patterns The flow patterns arePhase the same four types that are in Flow Patternare also acrylic. Parameters Ratio the simulation: annular, stratified, single bar, and two-bar. Each of the flow patterns concludes 3 cases T in terms of the corresponding parameter and phase ratio; see Table 4. 0.05 11.10% Annular Figure 11 shows the capacitance data chosen0.14 from one case of each26.53% phantom, the phase ratio of which is 48.95% in the annular flow (the annular0.29 thickness is 10 mm),48.95% 19.58% in the stratified flow (the stratified height is 17.5 mm), 18.31% in the single H bar (the bar radius is 15 mm), and 26.52% in the two-bar (the two bar radii are 10 mm and 15 mm,0.25 respectively). The corresponding capacitance data 19.58% are listed in Table 5. Stratified 0.50 50% It was found that the capacitance values0.75 in Figure 11a are slightly 80.42% different from those corresponding to the simulation-based values C(x,y) in Figure 4a.RIn addition, even the capacitance data with similar geometric relationships, such as the capacitance0.29 values of the7.93% adjacent electrode pairs, are (0,0) Single Bar slightly different from each other. The reason the ECT sensor used in the experiment, (0,0)for this is that 0.43 18.31% due to manufacturing precision limitations, is not absolutely geometrical symmetrical and identical (0.46,0) 0.50 25% to the sensor model used in the simulations. C1(x,y) However, C2(x,y) from R1 the R2image reconstruction point of view, these differences do not affect the image reconstruction results too because only the normalized 0.29 0.43much26.52% Two-bar capacitance data are used. (−0.46,0) (0.46,0) 0.29 0.50 33.15% 0.43 0.50 43.18%

Sensors 2018, 18, 3701

13 of 20

Table 4. The phantom parameters of the static experiment part of the benchmark dataset. Flow Pattern

Parameters

Phase Ratio

Sensors 2018, 18, x FOR PEER REVIEW

13 of 20 T 0.05 11.10% FigureAnnular 11 shows the capacitance data chosen from one case of each phantom,26.53% the phase ratio of 0.14 0.29 48.95% which is 48.95% in the annular flow (the annular thickness is 10 mm), 19.58% in the stratified flow (the

H (the bar radius is 15 mm), and 26.52% in the twostratified height is 17.5 mm), 18.31% in the single bar 0.25 19.58% bar (the two bar radii are 10 mm and 15 mm, respectively). The corresponding capacitance data are Stratified 0.50 50% listed in Table 5. 0.75 80.42% It was found that the capacitance values in Figure 11a are slightly different from those C(x,y) R corresponding to the simulation-based values in Figure 4a. In addition, even the capacitance data (0,0) 0.29 7.93% Single Bar with similar geometric relationships, values of the adjacent electrode pairs, (0,0)such as the capacitance 0.43 18.31% are slightly different from each other. The reason for this is that the ECT sensor used in the experiment, (0.46,0) 0.50 25% due to manufacturing precision is not absolutely geometrical C1(x,y)limitations, C2(x,y) R1 R2 symmetrical and identical to the sensor model used in the simulations. However, 0.29 from the image point of view, 0.43reconstruction 26.52% Two-bar ( − 0.46,0) (0.46,0) 0.29 0.50 33.15% these differences do not affect the image reconstruction results too much because only the normalized 0.43 0.50 43.18% capacitance data are used.

Measured Capacitance (pF)

3.5 1 3.8

3 2.5 2 1.5 1 0.5 0 0

5

10

(a)

3

Measured Capacitance (pF)

Measured Capacitance (pF)

3.5

2.5 2 1.5 1 0.5 0 0

5

10

20

2 1.5 1 0.5 10

25

3 2.5 2 1.5 1 0.5 5

10

15 Capacitance Index

20

25

15 Capacitance Index

20

25

20

25

(c)

3

2.5

5

20

3.5

0 0

25

Measured Capacitance (pF)

Measured Capacitance (pF)

15 Capacitance Index

(b)

3

0 0

15 Capacitance Index

2.5 2 1.5 1 0.5 0 0

5

(d)

10

15 Capacitance Index

(e)

Figure 11. The static experiment capacitance data examples while the permittivity value is 3.8, (a) the empty pipe and full pipe; (b) the 48.95% annular distribution; (c) the 19.58% stratified distribution; (d) the 18.31% 18.31% single single bar bar distribution; distribution; (e) (e) the the 26.52% 26.52% two-bar two-bar distribution. distribution.

Table 5. The capacitance data related to the examples in Figure 11. Electrode Pair 1-2 1-3 1-4 1-5 1-6 1-7 1-8

Empty 2.557 0.114 0.061 0.049 0.058 0.116 2.720

Full 2.930 0.322 0.189 0.147 0.174 0.338 3.188

48.95% Annular 3.153 0.231 0.084 0.057 0.075 0.228 3.353

19.58% Stratified 3.341 0.169 0.083 0.066 0.079 0.176 3.424

18.31% Single Bar 2.518 0.119 0.083 0.074 0.079 0.121 2.689

26.52% Two-Bar 2.516 0.157 0.088 0.116 0.084 0.129 2.631

Sensors 2018, 18, 3701

14 of 20

Table 5. The capacitance data related to the examples in Figure 11. Electrode Pair

Empty

Full

1-2 2.557 2.930 1-3 0.114 0.322 1-4 0.061 0.189 1-5 0.049 0.147 0.058REVIEW 0.174 Sensors 2018, 1-6 18, x FOR PEER 1-7 0.116 0.338 1-8 2.720 3.188 2-5 0.058 0.169 2-3 2.289 2.648 2-6 2-4 0.048 0.118 0.140 0.340 2-7 2-5 0.058 0.058 0.174 0.169 2-8 2-6 0.116 0.048 0.332 0.140 0.058 3.131 0.174 3-4 2-7 2.684 0.116 0.316 0.332 3-5 2-8 0.112 2.684 0.168 3.131 3-6 3-4 0.057 3-5 0.112 0.316 3-7 0.048 0.146 3-6 0.057 0.168 3-8 3-7 0.059 0.048 0.177 0.146 4-5 3-8 2.660 0.059 3.194 0.177 4-6 4-5 2.660 0.352 3.194 0.119 0.119 0.194 0.352 4-7 4-6 0.062 0.062 0.162 0.194 4-8 4-7 0.052 4-8 0.052 0.162 5-6 2.631 3.132 5-6 2.631 3.132 5-7 5-7 0.118 0.118 0.350 0.350 5-8 5-8 0.060 0.060 0.183 0.183 6-7 6-7 2.662 2.662 3.194 3.194 0.116 0.340 0.340 6-8 6-8 0.116 2.588 3.128 3.128 7-8 7-8 2.588

48.95% Annular 3.153 0.231 0.084 0.057 0.075 0.228 3.353 0.075 2.807 0.055 0.240 0.076 0.075 0.232 0.055 0.076 3.250 0.232 0.215 3.250 0.073 0.215 0.056 0.073 0.079 0.056 3.333 0.079 3.333 0.237 0.237 0.083 0.083 0.062 0.062 3.242 3.242 0.236 0.236 0.078 0.078 3.391 3.391 0.227 0.227 3.320 3.320

19.58% Stratified

18.31% Single Bar

3.341 0.169 0.083 0.066 0.079 0.176 3.424 0.065 2.338 0.054 0.131 0.071 0.065 0.325 0.054 0.071 2.746 0.325 0.104 2.746 0.049 0.104 0.037 0.049 0.069 0.037 2.716 0.069 2.716 0.114 0.114 0.054 0.054 0.058 0.058 2.652 2.652 0.110 0.110 0.066 0.066 2.774 2.774 0.126 0.126 2.586 2.586

2.518 0.119 0.083 0.074 0.079 0.121 2.689 0.078 2.255 0.071 0.123 0.079 0.078 0.121 0.071 0.079 2.655 0.121 0.118 2.655 0.078 0.118 0.073 0.078 0.079 0.073 2.624 0.079 2.624 0.125 0.125 0.084 0.084 0.078 0.078 2.601 2.601 0.123 0.123 0.081 0.081 2.625 2.625 0.121 0.121 2.552 2.552

26.52% Two-Bar 2.516 0.157 0.088 0.116 0.084 14 of 20 0.129 2.631 0.103 2.245 0.062 0.114 0.065 0.103 0.143 0.062 0.065 2.611 0.143 0.175 2.611 0.079 0.175 0.050 0.079 0.068 0.050 2.700 0.068 2.700 0.204 0.204 0.087 0.087 0.073 0.073 2.668 2.668 0.180 0.180 0.111 0.111 2.618 2.618 0.114 0.114 2.603 2.603

3.2. The Image Reconstruction Examples Based on the Static Experiment The four ECT image reconstruction algorithms used in Section 2.3 are also executed using the static experiment traditional algorithms is experiment capacitance capacitancedata. data.The Thesensitivity sensitivitymatrix matrixused usedfor forthe thethree three traditional algorithms that which was is that which wasgenerated generatedininthe the3D 3Dsimulation. simulation.The Theimage imagereconstruction reconstruction results results based based on on the capacitance vectors shown in in Figure Figure 12. 12. The comparison of the image reconstruction vectors in Table 5 are shown results in Figure 12 are listed in in Table Table 6. 6. Real Phantom

LBP

Landweber

TV

Autoencoder

48.95% annular

19.58% stratified

18.31% singe bar

26.52% twobar Figure 12. The The image image reconstruction examples based on the static experiment data. Table 6. The comparison of the image reconstruction results based on the static experiment. Flow Pattern

Algorithm

48.95% annular

LBP Landweber TV Autoencoder

Relative Image Error 34.07% 16.41% 16.30% 26.41%

Correlation Coefficient 0.8954 0.9733 0.9737 0.9338

Estimated Phase Ratio 50.51% 49.42% 49.43% 45.01%

Phase Ratio Error 1.56% 0.47% 0.48% −3.94%

Sensors 2018, 18, 3701

15 of 20

Table 6. The comparison of the image reconstruction results based on the static experiment. Flow Pattern

48.95% annular

19.58% stratified

18.31% single bar

26.52% two-bar

Algorithm

Relative Image Error

Correlation Coefficient

Estimated Phase Ratio

Phase Ratio Error

LBP Landweber TV Autoencoder LBP Landweber TV Autoencoder LBP Landweber TV Autoencoder LBP Landweber TV Autoencoder

34.07% 16.41% 16.30% 26.41% 36.82% 33.67% 33.94% 33.16% 80.69% 54.18% 33.49% 30.14% 72.34% 54.54% 54.71% 39.11%

0.8954 0.9733 0.9737 0.9338 0.9193 0.9279 0.9266 0.9360 0.7151 0.9118 0.9427 0.9438 0.7075 0.8310 0.8305 0.9031

50.51% 49.42% 49.43% 45.01% 19.48% 19.51% 19.19% 14.02% 9.26% 9.47% 14.89% 25.99% 15.96% 16.70% 16.66% 16.52%

1.56% 0.47% 0.48% −3.94% −0.10% −0.07% −0.39% −5.56% −9.05% −8.84% −3.41% 7.68% −10.56% −9.82% −9.86% −10.00%

3.3. The Capacitance Data Open for the Image Reconstruction Study Three measured capacitance vectors, the permittivity distribution information of which are not open to the public, are given in Table 7. The empty and full pipes’ capacitance vectors for calibration can be found in Table 5. These three capacitance vectors are published for researchers who are interested in ECT image reconstruction in order to estimate what the real phantoms are and to evaluate their own algorithms. In terms of the sensitivity matrix, researchers can use their own calculated matrices based on the 3D ECT sensor, as described in this paper, or they can ask for the one used in this paper by email. Table 7. The capacitance dataset of the permittivity distribution information not open to the public (in pF). Electrode Pair

Experimental Phantom No. 1

Experimental Phantom No. 2

Experimental Phantom No. 3

1-2 1-3 1-4 1-5 1-6 1-7 1-8 2-3 2-4 2-5 2-6 2-7 2-8 3-4 3-5 3-6 3-7 3-8 4-5 4-6 4-7 4-8 5-6 5-7 5-8 6-7 6-8 7-8

2.876 0.185 0.105 0.080 0.088 0.161 3.069 2.589 0.153 0.081 0.075 0.099 0.196 3.030 0.121 0.063 0.065 0.097 3.057 0.118 0.066 0.072 2.995 0.107 0.070 3.105 0.129 2.959

2.873 0.126 0.060 0.053 0.083 0.128 3.026 2.589 0.132 0.060 0.068 0.059 0.129 2.958 0.123 0.083 0.051 0.057 2.971 0.179 0.075 0.050 2.933 0.188 0.074 2.985 0.166 2.927

3.052 0.187 0.124 0.105 0.113 0.195 3.336 2.267 0.146 0.094 0.087 0.102 0.332 2.688 0.105 0.064 0.056 0.098 2.663 0.113 0.069 0.091 2.561 0.108 0.093 2.700 0.136 2.584

Sensors 2018, 18, 3701

16 of 20

3.4. The Dynamic Experiment Part of the Benchmark Dataset The dynamic experiment part of the dataset is the capacitance values of the oil-gas two-phase flow given in the form of normalized capacitance data sequences, which are obtained from an experimental test rig with a pipeline with a 50 mm diameter. The testing ECT system has an 8-electrode sensor and is installed on a vertical Venturi throat section. Before flowing through the Venturi pipe, the oil and gas are separately issued and then mix as a two-phase flow. The data acquisition software in the upper computer records capacitance data are measured using the ECT sensor and transformed using the data acquisition circuit. The measurement system is calibrated by using a pipe full of oil and a pipe full of air. The oil-gas two-phase flows with the different gas volume fractions (GVFs) are measured. The dataset includes three samples whose GVF and corresponding flow rates are given in Table 8. The normalized capacitance data sequence of 62.09% GVF is given in Table 9 as an example and the corresponding reconstructed images are given in Figure 13. Table 8. The GVF and corresponding flow rate of the dynamic experiment samples. GVF

Gas Flow Rate (m3 /h)

Oil Flow Rate (m3 /h)

23.71% 44.24% 62.09%

5.78 14.33 28.01

18.61 18.06 17.10

Table 9. The normalized capacitance data sequence with 62.09% GVF. Electrode Pair 1-2 1-3 1-4 1-5 1-6 1-7 1-8 2-3 2-4 2-5 2-6 2-7 2-8 3-4 3-5 3-6 3-7 3-8 4-5 4-6 4-7 4-8 5-6 5-7 5-8 6-7 6-8 7-8

Normalized Capacitance Data Sequence in 10 s t=1s

t=2s

t=3s

t=4s

t=5s

t=6s

t=7s

t=8s

t=9s

t = 10 s

2.191 0.110 0.056 0.046 0.056 0.110 2.199 2.193 0.110 0.056 0.046 0.056 0.110 2.196 0.110 0.056 0.046 0.056 2.196 0.110 0.056 0.046 2.201 0.110 0.056 2.193 0.110 2.198

2.615 0.282 0.147 0.122 0.147 0.283 2.623 2.617 0.282 0.147 0.122 0.147 0.283 2.620 0.283 0.147 0.122 0.147 2.620 0.283 0.147 0.122 2.625 0.283 0.147 2.617 0.283 2.622

2.796 0.382 0.203 0.170 0.203 0.383 2.804 2.799 0.382 0.203 0.170 0.204 0.383 2.801 0.383 0.203 0.170 0.203 2.801 0.383 0.204 0.170 2.807 0.383 0.204 2.798 0.382 2.803

2.807 1.579 1.337 1.273 1.338 1.579 2.812 2.808 1.579 1.338 1.274 1.339 1.580 2.810 1.579 1.338 1.274 1.338 2.809 1.579 1.338 1.274 2.815 1.580 1.338 2.807 1.579 2.812

2.681 0.195 0.069 0.052 0.069 0.195 2.686 2.681 0.195 0.069 0.052 0.069 0.195 2.681 0.195 0.069 0.052 0.069 2.680 0.195 0.069 0.052 2.683 0.195 0.069 2.675 0.195 2.684

2.907 0.246 0.077 0.055 0.077 0.247 2.912 2.907 0.246 0.077 0.055 0.077 0.246 2.908 0.246 0.077 0.055 0.077 2.906 0.246 0.077 0.055 2.910 0.246 0.077 2.902 0.246 2.910

3.552 1.463 0.806 0.627 0.807 1.463 3.556 3.552 1.462 0.807 0.627 0.807 1.462 3.552 1.462 0.807 0.628 0.807 3.550 1.462 0.807 0.627 3.555 1.463 0.807 3.548 1.462 3.555

2.182 0.103 0.062 0.060 0.062 0.103 2.187 2.182 0.122 0.073 0.052 0.048 0.106 2.231 0.158 0.065 0.036 0.048 2.742 0.268 0.065 0.052 2.749 0.155 0.073 2.228 0.121 2.174

2.178 0.101 0.064 0.063 0.064 0.101 2.187 2.179 0.124 0.077 0.054 0.047 0.105 2.228 0.177 0.071 0.034 0.047 3.016 0.370 0.070 0.053 3.028 0.173 0.077 2.226 0.122 2.171

2.176 0.096 0.061 0.063 0.061 0.097 2.184 2.169 0.097 0.081 0.068 0.044 0.102 1.765 0.366 0.258 0.041 0.044 4.974 2.984 0.227 0.066 5.078 0.328 0.080 1.846 0.097 2.161

Sensors 2018, 18, 3701

17 of 20

Sensors 2018, 18, x FOR PEER REVIEW

Algorithm

t = 1s

t = 2s

t = 3s

17 of 20

Reconstructed Images in 10 s t = 4s t = 5s t = 6s t = 7s

t = 8s

t = 9s

t = 10s

LBP

Landweber

TV

Autoencoder

Figure 13. The image reconstructionexamples examples based experiment data.data. Figure 13. The image reconstruction basedon onthe thedynamic dynamic experiment

4. Conclusions 4. Conclusions In paper, this paper, a benchmark dataset ECT based and3D 3Dsimulations, simulations, as as well well as as static In this a benchmark dataset forfor ECT based onon 2D2Dand static and and dynamic experiments, is built. The 2D simulation part contains 40,000 pairs of samples with dynamic experiments, is built. The 2D simulation part contains 40,000 pairs of samples with normalized normalized capacitance vectors and their corresponding permittivity distribution vectors. The 3D capacitance vectors and their corresponding permittivity distribution vectors. The 3D simulation simulation part contains capacitance vectors corresponding to 80 cases, including capacitance vectors part contains capacitance vectors corresponding to 80 cases, including capacitance vectors of full and of full and empty pipes for calibration, 2 sensitivity matrices for the 8-electrode model and the 12empty pipes for calibration, 2 sensitivity for the 8-electrode and the 12-electrode model, electrode model, respectively, as well matrices as 12 normalized permittivitymodel distribution vectors. The static respectively, as well as 12 normalized permittivity distribution vectors. The static experiment experiment part contains 14 capacitance vectors of the 14 cases, along with 3 capacitance vectors part contains 14 capacitance vectors of the 14The cases, along with 3 capacitance vectors three without flow pattern without flow pattern information. dynamic experiment part contains normalized information. Thedata dynamic experiment part contains three normalized capacitance data sequences in capacitance sequences in different GVFs. Among these four parts of the benchmark dataset, the part based on the 2D simulation is used different GVFs. as the public for researchers to use in dataset, training and their own machine learning-based Among thesedatabase four parts of the benchmark the testing part based on the 2D simulation is used as ECT image reconstruction algorithms. Additionally, the other three parts of the benchmark dataset— ECT the public database for researchers to use in training and testing their own machine learning-based i.e., the 3D simulation part, the static experiment part, and the dynamic experiment part—can be used image reconstruction algorithms. Additionally, the other three parts of the benchmark dataset—i.e., as a benchmark for evaluating and comparing the different ECT image reconstruction methods. Three the 3D simulation part, the static experiment part, and the dynamic experiment part—can be used as criteria—i.e., the relative image error, the correlation coefficient, and the ratio error—are put forward a benchmark for evaluating and comparing the different ECT image reconstruction methods. as the quantitative standard to evaluate and compare the ECT image reconstruction methods. The LBP,Three criteria—i.e., the relative image error,variation the correlation coefficient, and algorithm, the ratio error—are put forward the projected Landweber, the total (TV) based regularization and a deep learning as themethod quantitative to evaluate image methods. The LBP, based standard on an autoencoder forand ECTcompare are usedthe as ECT examples of reconstruction how to compare the different algorithms under the same evaluation criteria of our benchmark dataset.algorithm, They are executed in thelearning 3D the projected Landweber, the total variation (TV) based regularization and a deep simulation part, the static experiment part, and the dynamic experiment part of the benchmark dataset, method based on an autoencoder for ECT are used as examples of how to compare the different respectively, corresponding image reconstruction results are evaluated criteria.in the algorithms underand thethesame evaluation criteria of our benchmark dataset. using They the arethree executed Most visual results and quantitative results show that the autoencoder-based deep learning method can 3D simulation part, the static experiment part, and the dynamic experiment part of the benchmark perform better reconstructions than the three traditional algorithms and that it has a good dataset, respectively, and the corresponding image reconstruction results are evaluated using the three generalization ability. However, some results show that the autoencoder is not perfect and that the criteria. Most visual results and quantitative results show that the autoencoder-based deep learning generalization ability can be further improved. method can better reconstructions than the the new threedeep traditional algorithms andreconstruction that it has a good Theperform benchmark dataset that supported learning-based image generalization ability. However, some results show that the autoencoder is not perfect algorithm is still at its initial stage and it is not perfect enough at present. It mainly focusesand on that the the generalization ability can be further improved. research of data from mostly used the 8-electrode and 12-electrode ECT sensors, and there are only four types of flow dataset patterns in thesupported benchmark dataset. to the benchmark dataset could The benchmark that the newSupplements deep learning-based image reconstruction enhance the diversity of the training dataset machine learning-based image reconstruction algorithm is still at its initial stage and it is notfor perfect enough at present. It mainly focuses on methods, themostly performance of 8-electrode these methods, expand their range. In are the only the research ofimprove data from used the andand 12-electrode ECTapplication sensors, and there future, we will add more simulation and experiment data to improve this benchmark dataset. We four types of flow patterns in the benchmark dataset. Supplements to the benchmark dataset could also welcome other researchers to contribute to the dataset by integrating data from other ECT sensor enhance the diversity of the training dataset for machine learning-based image reconstruction methods, models—including the 16-electrode ECT sensor, the 3D ECT sensor, and dates related to other flow improve the performance of these methods, and expand their application range. In the future, we will add more simulation and experiment data to improve this benchmark dataset. We also welcome other researchers to contribute to the dataset by integrating data from other ECT sensor models—including the 16-electrode ECT sensor, the 3D ECT sensor, and dates related to other flow patterns—and to evaluate their new image reconstruction algorithms under the criteria of the benchmark dataset.

Sensors 2018, 18, 3701

18 of 20

We hope this benchmark dataset can be used by researchers to try new image reconstruction methods—especially faster and better methods based on machine learning, where the hardware system or the simulation model is not necessary—and to make the ECT image reconstruction research area more open and flexible, leading to a big breakthrough. Author Contributions: Conceptualization, J.Z. and L.P.; Methodology, J.Z.; Software, J.Z.; Validation, J.L. and Y.L.; Formal Analysis, L.P.; Investigation, J.Z.; Resources, L.P.; Data Curation, J.L. and Y.L.; Writing-Original Draft Preparation, J.Z.; Writing-Review & Editing, L.P.; Visualization, J.Z.; Supervision, L.P.; Project Administration, L.P.; Funding Acquisition, L.P. Funding: This research was funded by [National Natural Science Foundation of China] grant number [61571253]. Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.

References 1. 2. 3.

4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18.

Huang, S.M.; Plaskowski, A.; Xie, C.G.; Beck, M.S. Tomographic imaging of two-component flow using capacitance sensors. J. Phys. E Sci. Instrum. 1989, 22, 173–177. [CrossRef] Fasching, G.; Smith, N.S. A capacitive system for 3-dimensional imaging of fluidized-beds. Rev. Sci. Instrum. 1991, 62, 2243–2251. [CrossRef] Xie, C.G.; Huang, S.M.; Hoyle, B.S.; Thorn, N.R.; Lenn, C.; Snowden, D.; Beck, M.S. Electrical capacitance tomography for flow imaging system model for development of image reconstruction algorithms and design of primary sensors. IEEE Proc. G 1992, 139, 89–98. [CrossRef] Alme, K.J.; Mylvaganarn, S. Electrical capacitance tomography—Sensor models, design, simulations, and experimental verification. IEEE Sens. J. 2006, 6, 1256–1266. [CrossRef] AOlmos, M.; Primicia, J.A.; Marron, J.L. Simulation design of electrical capacitance tomography sensors. IET Sci. Meas. Technol. 2007, 1, 216–223. Yang, W.Q. Design of electrical capacitance tomography sensors. Meas. Sci. Technol. 2010, 21, 042001. [CrossRef] Peng, L.H.; Mou, C.H.; Yao, D.Y.; Zhang, B.F.; Xiao, D.Y. Determination of the optimal axial length of the electrode in an electrical capacitance tomography sensor. Flow Meas. Instrum. 2005, 16, 169–175. [CrossRef] Peng, L.H.; Ye, J.M.; Lu, G.; Yang, W.Q. Evaluation of effect of number of electrodes in ECT sensors on image quality. IEEE Sens. J. 2012, 12, 1554–1565. [CrossRef] Yang, W.Q. Hardware design of electrical capacitance tomography systems. Meas. Sci. Technol. 1996, 7, 225–232. [CrossRef] Yang, W.Q.; York, T.A. A new AC-based capacitance tomography system. IEE Proc. Sci. Meas. Technol. 1999, 146, 47–53. [CrossRef] Cui, Z.Q.; Wang, H.X.; Chen, Z.Q.; Xu, Y.B.; Yang, W.Q. A high-performance digital system for electrical capacitance tomography. Meas. Sci. Technol. 2011, 22, R1–R10. [CrossRef] Yang, Y.J.; Peng, L.H.; Jia, J.B. A novel multi-electrode sensing strategy for electrical capacitance tomography with ultra-low dynamic range. Flow Meas. Instrum. 2017, 53, 67–79. [CrossRef] Isaksen, Ø. A review of reconstruction techniques for capacitance tomography. Meas. Sci. Technol. 1996, 7, 325. [CrossRef] Peng, L.H.; Merkus, H.; Scarlett, B. Using regularization methods for image reconstruction of electrical capacitance tomography. Part. Part. Syst. Charact. 2000, 17, 96–104. [CrossRef] Yang, W.Q.; Peng, L.H. Image reconstruction algorithms for electrical capacitance tomography. Meas. Sci. Technol. 2003, 14, R1–R13. [CrossRef] Fang, W.F. A nonlinear image reconstruction algorithm for electrical capacitance tomography. Meas. Sci. Technol. 2004, 15, 2124. [CrossRef] Ortiz-Aleman, C.; Martin, R.; Gamio, J.C. Reconstruction of permittivity images from capacitance tomography data by using very fast simulated annealing. Meas. Sci. Technol. 2004, 15, 1382–1390. [CrossRef] Soleimani, M.; Lionheart, W.R. Nonlinear image reconstruction for electrical capacitance tomography using experimental data. Meas. Sci. Technol. 2005, 16, 1987–1996. [CrossRef]

Sensors 2018, 18, 3701

19.

20. 21. 22. 23. 24. 25. 26.

27. 28. 29.

30.

31. 32. 33.

34. 35. 36. 37.

38.

39. 40.

19 of 20

Soleimani, M.; Vauhkonen, M.; Yang, W.; Peyton, A.; Kim, B.S.; Ma, X. Dynamic imaging in electrical capacitance tomography and electromagnetic induction tomography using a Kalman filter. Meas. Sci. Technol. 2007, 18, 3287. [CrossRef] Li, Y.; Yang, W.Q. Image reconstruction by nonlinear Landweber iteration for complicated distributions. Meas. Sci. Technol. 2008, 19, 094014. [CrossRef] Watzenig, D.; Fox, C. A review of statistical modelling and inference for electrical capacitance tomography. Meas. Sci. Technol. 2009, 20, 052002. [CrossRef] Beck, M.S.; Williams, R.A. Process tomography: A European innovation and its applications. Meas. Sci. Technol. 1996, 7, 215. [CrossRef] Dyakowski, T.; Edwards, R.B.; Xie, C.G.; William, R.A. Application of capacitance tomography to gas-solid flows. Chem. Eng. Sci. 1997, 52, 2099–2110. [CrossRef] Reinecke, N.; Mewes, D. Investigation of the two-phase flow in trickle-bed reactors using capacitance tomography. Chem. Eng. Sci. 1997, 52, 2111–2127. [CrossRef] Huang, Z.; Wang, B.; Li, H. Application of electrical capacitance tomography to the void fraction measurement of two-phase flow. IEEE Trans. Instrum. Meas. 2003, 52, 7–12. [CrossRef] Zhu, K.W.; Rao, S.M.; Huang, Q.H.; Wang, C.H.; Matsusaka, S.; Masuda, H. On the electrostatics of pneumatic conveying of granular materials using electrical capacitance tomography. Chem. Eng. Sci. 2004, 59, 3201–3213. [CrossRef] Ismail, I.; Gamio, J.C.; Bukhari, S.F.A.; Yang, W.Q. Tomography for multi-phase flow measurement in the oil industry. Flow Meas. Instrum. 2005, 16, 145–155. [CrossRef] Chaplin, G.; Pugsley, T. Application of electrical capacitance tomography to the fluidized bed drying of pharmaceutical granule. Chem. Eng. Sci. 2005, 60, 7022–7033. [CrossRef] Liu, G.Z.; Lan, J.A.; Cao, Y.B.; Huang, Z.B.; Cheng, Z.M.; Mi, Z.T. New insights into transient behaviors of local liquid-holdup in periodically operated trickle-bed reactors using electrical capacitance tomography (ECT). Chem. Eng. Sci. 2009, 64, 3329–3343. [CrossRef] Rezvanpour, A.; Wang, C.H.; Liang, Y.C.; Yang, W.Q. Investigation of droplet distribution in electrohydrodynamic atomization (EHDA) using an ac-based electrical capacitance tomography (ECT) system with an internal–external electrode sensor. Meas. Sci. Technol. 2012, 23, 015301. [CrossRef] Ye, J.M.; Wang, H.G.; Yang, W.Q. Image reconstruction for electrical capacitance tomography based on sparse representation. IEEE Trans. Instrum. Meas. 2015, 64, 89–102. Zhao, J.; Xu, Y.B.; Tan, C.; Dong, F. A fast sparse reconstruction algorithm for electrical capacitance tomography. Meas. Sci. Technol. 2014, 25, 085401. [CrossRef] Yang, Y.J.; Peng, L.H. An image reconstruction algorithm for ECT using enhanced model and sparsity regularization. In Proceedings of the 2013 IEEE International Conference on Imaging Systems and Techniques (IST), Beijing, China, 22–23 October 2013; pp. 35–39. Taylor, S.H.; Garimella, S.V. Level-set shape reconstruction of binary permittivity distributions using near-field focusing capacitance measurements. Meas. Sci. Technol. 2014, 25, 165062. [CrossRef] Ren, S.J.; Dong, F.; Xu, Y.B.; Tan, C. Reconstruction of the three-dimensional inclusion shapes using electrical capacitance tomography. Meas. Sci. Technol. 2014, 25, 025403. [CrossRef] Marashdeh, Q.; Warsito, W.; Fan, L.S.; Teixeira, F.L. A nonlinear image reconstruction technique for ECT using a combined neural network approach. Meas. Sci. Technol. 2006, 17, 2097. [CrossRef] Wang, H.; Hu, H.L.; Wang, L.J.; Wang, H.X. Image reconstruction for an Electrical Capacitance Tomography (ECT) system based on a least squares support vector machine and bacterial colony chemotaxis algorithm. Flow Meas. Instrum. 2012, 27, 59–66. [CrossRef] Li, J.; Yang, X.; Wang, Y.; Pan, R. An image reconstruction algorithm based on RBF neural network for electrical capacitance tomography. In Proceedings of the 2012 Sixth International Conference on Electromagnetic Field Problems and Applications, Dalian, China, 19–21 June 2012; pp. 1–4. LeCun, Y. The MNIST Database of Handwritten Digits. 1998. Available online: http://yann.lecun.com/ exdb/mnist/ (accessed on 2 September 2018). Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fe, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255.

Sensors 2018, 18, 3701

41. 42.

43. 44.

45. 46.

47.

20 of 20

Zheng, J.; Peng, L. An Autoencoder Based Image Reconstruction for Electrical Capacitance Tomography. Sensors 2018, 18, 5464–5474. [CrossRef] Zheng, J.; Peng, L. A Platform for Electrical Capacitance Tomography Large-scale Benchmark Dataset Generating and Image Reconstruction. In Proceedings of the 2017 IEEE International Conference on Imaging Systems and Techniques (IST), Beijing, China, 18–20 October 2017; pp. 1–6. Cui, Z.Q.; Yang, C.; Sun, B.; Wang, H.G. Liquid film thickness estimation using electrical capacitance tomography. Meas. Sci. Rev. 2014, 14, 8–15. [CrossRef] Yang, Y.J.; Peng, L.H. An image reconstruction algorithm for high-contrast dielectrics in ECT. In Proceedings of the 7th World Congress on Industrial Process Tomography (WCIPT), Krakow, Poland, 2–5 September 2013; pp. 320–327. Yang, W.Q.; Spink, D.M.; York, T.A.; McCann, H. An image-reconstruction algorithm based on Landweber’s iteration method for electrical-capacitance tomography. Meas. Sci. Technol. 1999, 10, 1065. [CrossRef] Gamio, J.C.; Ortiz-Aleman, C. An interpretation of the linear back-projection algorithm used in capacitance tomography. In Proceedings of the 3rd World Congress on Industrial Process Tomography, Banff, AB, Canada, 2–5 September 2003; pp. 427–432. Wang, H.; Tang, L.; Cao, Z. An image reconstruction algorithm based on total variation with adaptive mesh refinement for ECT. Flow Meas. Instrum. 2007, 18, 262–267. [CrossRef] © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).