Dual Features Functional Support Vector Machines for Fault Detection ...

480

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 39, NO. 4, JULY 2009

Dual Features Functional Support Vector Machines for Fault Detection of Rechargeable Batteries Jong I. Park, Seung H. Baek, Myong K. Jeong, Member, IEEE, and Suk J. Bae, Member, IEEE Abstract—The early detection of faulty batteries is a critical work in the manufacturing processes of a secondary rechargeable battery. Conventional approaches use original performance degradation profiles of remaining capacity after recharge in order to detect faulty batteries. However, original degradation profiles with right-truncated test duration may not be effective in detecting faulty batteries. In this correspondence, we propose dual features functional support vector machine approach that uses both first and second derivatives of degradation profiles for early detection of faulty batteries with the reduced error rate. The modified floating search algorithm for the repeated feature selection with newly added degradation path points is presented to find a few good features for the enhanced detection while reducing the computation time for online implementation. After that, an attribute sampling plan considering time-varying classification errors is presented to determine the optimal number of test cycles and sample sizes by minimizing our proposed cost function. The real-life case study is presented to illustrate the proposed methodology and show its improved performance compared to existing approaches. The proposed method can be applied in a wide range of manufacturing processes to assess time-dependent quality characteristics. Index Terms—Attribute sampling plans, data mining, feature selection, process monitoring, secondary rechargeable battery, support vector machine.

I. INTRODUCTION Secondary rechargeable batteries have become an essential part of portable multimedia devices such as mobile phones, camcorders, and computers. In the mass production stage of rechargeable batteries, it is crucial to assure the product quality within a limited time to cope with shorter life cycle. Cycle life, which is directly related to the battery life, is one of the major characteristics to be monitored. Evaluation of the cycle life requires a lot of charge/discharge cycles; thus, it is a very time-consuming task for process monitoring. This has caused a major difficulty for battery manufacturers to reduce product development time. For this reason, the method for assessing the cycle life of rechargeable batteries in a short time is needed. A great deal of research has been directed to develop efficient techniques for quality monitoring with in-process data. For instances, Ribeiro [1] and Omitaomu et al. [2] applied support vector machine (SVM) to fault detection in an injection molding machine and machine health monitoring in order to explore the better ways of quality control, respectively. In application of the rechargeable batteries, conventional approaches employing the SVM use original degradation profiles of Manuscript received May 30, 2008; revised October 27, 2008 and January 8, 2009. First published April 7, 2009; current version published June 17, 2009. This work was supported in part by the National Science Foundation under Grant CMMI-0644830. This paper was recommended by Associate Editor W. Naiqi. J. I. Park and S. J. Bae are with the Department of Industrial Engineering, Hanyang University, Seoul 133-070, Korea (e-mail: [email protected]. ac.kr; [email protected]). S. H. Baek is with the Department of Industrial and Information Engineering, The University of Tennessee, Knoxville, TN 37996-0700 USA (e-mail: [email protected]). M. K. Jeong is with the Department of Industrial and System Engineering and RUTCOR (Rutgers Center for Operations Research), The State University of New Jersey, Piscataway, NJ 08854-8003 USA (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSMCC.2009.2014642

remaining capacity after recharge to discriminate good and faulty batteries. However, original degradation profiles with right-truncated test duration may not be effective in discriminating good and faulty batteries. In this work, we propose dual features functional support vector machine (FSVM) approach that uses both first and second derivatives of degradation path for early detection of faulty batteries with the reduced error rate. To find the classification error rates for a given cycle, the repeated feature selection is needed for each cycle and this could be computationally very expensive. The modified floating search algorithm for the repeated feature selection with newly added degradation path points is presented to find a few good features for the enhanced discrimination while reducing the computation time for online implementation. After then, an optimal attribute sampling plan for determining the acceptability of a manufactured lot is developed considering the inspection (i.e., classification) errors. Even though there are some literatures on the construction of sampling plans considering the inspection errors, their major focus was to evaluate the effects of inspection errors on the performance of the sampling plans. Also, existing literatures assume that the inspection errors are usually coming from human errors and are given by experimenters or prespecified error models. However, in our problem, the inspection errors depend on the test cycles that should be determined and can be estimated from the classification results based on FSVM. We develop the new sampling plan procedure to determine the optimal test cycle by balancing the sample size and the required test time, which will affect the inspection errors. For this, a new cost function is proposed by considering the cost of a tested unit, the cost of operating cycle-life test, and the delay cost due to the longer test time. The rest of this correspondence is organized as follows. Section II introduces the motivation of this study along with some technical backgrounds for secondary rechargeable batteries. Section III describes the proposed dual features FSVM approach to discriminate good and faulty batteries for a fixed test cycle. Section IV determines minimum test cycles by incorporating the results of the FSVM in the corresponding sampling plans. Section V presents a case study to illustrate the proposed methods and compare the performance with the existing approaches. Some concluding remarks are given in section VI. II. MOTIVATING EXAMPLE Basic performance of a rechargeable battery is characterized by its capacity, which is generally defined as the amount of charge available expressed in ampere-hours (Ah). Cycle life is defined as the number of complete charge/discharge cycles before its nominal capacity falls below the prespecified value of its initial capacity. Although it is desirable that the battery retains the initial capacity as much as possible during usage time, the capacity is subject to decrease through repetitive cycles [3], [4]. Fig. 1 shows the remaining capacities of 43 samples of a lithium-ion type, one of the popular rechargeable batteries in the market, as the cycle proceeds. They were randomly selected from the manufactured lots for several months. In the cycle-life tests for qualification, each sample is subject to check whether or not its capacity reaches to a fixed threshold during a specific number of cycles. The requirements of the threshold level and the number of cycles are generally predetermined based on the industry standard or customer requirement (e.g., the threshold value as 80% of its initial capacity during 400 cycles). Then, the battery cell is classified as either a conforming or a nonconforming cell according to its requirement. As shown in Fig. 1, the capacity degradation has a typical nonlinear trend: capacity degrades sharply during some initial cycles, and then the degradation rate becomes relatively slow. In recent years,

1094-6977/$25.00 © 2009 IEEE Authorized licensed use limited to: Hanyang University. Downloaded on June 7, 2009 at 05:50 from IEEE Xplore. Restrictions apply.


481

The SVM, which is one of the dominant classification techniques in the data mining field, can be exploited for this purpose and will be presented in the following section. Pearn et al. [5] and Park et al. [6] are two examples of the scant literature on characterizing the manufacturing quality of rechargeable batteries. A similar approach to the present study has been made by Tseng and Yu [7] where a termination rule was proposed to estimate the mean lifetime of a light emitting diode (LED) within short test duration. Their rule is simply to stop the test at the time, which the precision of the estimated lifetime is not appreciably affected and, then, the conformity of the manufactured products was not concerned at all in their study, which is often a main subject-matter domain for qualification.

III. DUAL FEATURES FSVM FOR CLASSIFICATION OF CYCLE-LIFE PROFILES Fig. 1. tests.

Remaining capacities of selected battery samples during cycle-life

Fig. 2.

Box plots of remaining capacity at some cycles.

much attention has been placed particularly on relating observed phenomenon with state-of-art analysis techniques. Since each cell in Fig. 1 has possibly experienced of different sources of variation during fabrication, its capacity degradation is slightly different from one another. Nevertheless, some degradation paths show catastrophic drops in the end of cycles, leading to products of poor quality away from the designated. It usually takes long time to finish the whole test cycles specified in the requirements. For example, 400 cycles require at least fifty calendar days. Because such long testing span has been impeding efficient operation of manufacturing lines, battery engineers have struggled to devise various ways to reduce the test duration [3]. However, if one determines the acceptability of a lot at the shorter cycle, more risky decision may be made. Fig. 2 shows separate distributions of the remaining capacity for conforming (26 cells) and nonconforming (17 cells) battery samples at some fixed cycles. Note that the boundaries between both groups are not distinctive at some relatively short cycle. Even mean capacity of nonconforming cells is larger than that of conforming cells, leading to the need for developing an efficient procedure to discriminate conforming cells from nonconforming cells at a shorter cycle.

Conventional support vector machines are popular, and many studies have shown their superiority to other approaches such as neural networks and partial least squares [8]. However, as shown in Fig. 1, the direct application of the SVM with original degradation profiles may not be effective in the discrimination of good and faulty battery cells. We introduce the FSVM approach that uses both the first and second derivatives of degradation profiles to enhance the accuracy of classification. Data representation for functional structures is one of the key issues in implementing the FSVM. Ramsay and Silverman [9] addressed that much of the variation between profiles can be explained at the level of certain derivatives. For this reason, even with its own simplicity, functional derivative representation showed successful results in many studies [10], [11]. In some cases, a combination of the derivatives with different orders may lead to better classification performance. Fig. 3(a) and (b) show the potential of the early discrimination of faulty lithiumion batteries using the dual features of first and second derivatives. The derivatives can be calculated using a B-spline approximation [12] to avoid numerical stability problem of direct computation. Since a dual or a multiple data representation leads to the higher dimensional space, feature selection technique is needed to reduce the dimension. The main idea of feature selection is to find a proper subset of input variables by eliminating some features with redundant or meaningless information. Heavy computing often follows due to its iterative process for finding the proper subset. A floating search technique is a popular method to guarantee the near optimal subset without any exhaustive search [13]. Consider the case where training data set (y i , zi )i = 1 , 2 , . . . , n consists of the digitized curve yi = {yi (t1 ) , yi (t2 ) , . . . , yi (to )} for observed number of cycles to and a binary class index zi ∈ (−1, 1). For the given cycle to , we have to search for the optimal features that have the highest separability between the good and faulty batteries, and repeat for the next cycle (to + 1). There is a high possibility of redundant iterations, because the existing algorithm is repeatedly applied to similar input data set as the observation period prolongs. We propose a modified sequential floating forward search (SFFS) algorithm so as to start with the best features set of cycle (to − 1) at cycle to . The detailed procedure for the iterative feature selection is shown in Fig. 4. A divergence is calculated and compared to find the best subset of d features from a given set of G features (1 ≤ d ≤ G) in the SFFS process. The divergence is one of popular criteria for class separability. It takes into account the correlation that exists among various selected features and influences classification capabilities of the selected features. Assuming p-D multivariate normal distribution, the divergence between a class i

Authorized licensed use limited to: Hanyang University. Downloaded on June 7, 2009 at 05:50 from IEEE Xplore. Restrictions apply.

482


Fig. 3. (a) Discrimination of good (+) and faulty (o) batteries with first derivatives of 11th cycle and 6th cycles (b) Discrimination of good and faulty batteries with a first derivative of 2nd cycle and a second derivative of 24th cycle.

and class j is given by [14] Ji j (y) =

1 −1 trace Σ−1 µi − µj µi − µj i + Σj 2 1 −1 + trace (Σi − Σj ) Σ−1 j − Σi 2

T (1)

where µi and Σi are the mean vector and covariance matrix of class i, respectively. Then, the best subset of d features is chosen to maximize the divergence value. Suppose that the observed number of cycles and desired number of features are given by to and d, respectively. Furthermore, let Ad (to ) and Sd (to ) are respective sets of available and selected features for given to and d. In the SFFS process, new features from Ad (to ) are included in the current Sd (to ) and successive steps follow to exclude the worst features in the newly updated Sd (to ) provided further improvement can be made to the previous set. Given a subset of d features, dual features FSVM is applied to detect faulty batteries based on degradation profiles. Optimal subset size d∗ and corresponding features at a given number of cycles to are determined such that the classification accuracy of the SVM is maximized. Leave-one-out cross validation is used to estimate the classification accuracy, which gives proper measure when there are limited samples. At this time, d∗ is given by

1 I Ψs ∗d (t o ) yi | M(−i ) , zi d∗ = arg max n 1 ≤d ≤2 t o

The construction of a sampling plan is usually based on the implicit assumption that the inspection for checking if each sample meets to the desired level is perfect. However, many inspection tasks are not error free in practice [15]–[17]. For example, the FSVM cannot discriminate correctly every good and faulty cells, and the error rate increases as the number of test cycles decreases. Such a limitation of the FSVM in the application to statistical quality control scheme is interpreted as the case where the inspection errors exist. To evaluate effect of the classification errors explicitly, let q1 (t) and q2 (t) the probabilities of correctly classifying a good cell and a faulty cell at cycle t, respectively. Furthermore, assume that pf is the true faulty fraction. Then the apparent fractions pa (t), due to the classification error at cycle t, are slightly different from the true values according to the classification outcomes as follows: pa (t) = {1 − q1 (t)} − pf {1 − q1 (t) − q2 (t)} .

(3)

The pa (t) should replace pf in designing of sampling plans to compensate effect of the classification errors [15], i.e., the sample size n and the acceptance number c for compensating the classification errors can be determined such that the following risk requirements are satisfied: A(p0 ) =

n

c n

k

{pa , 0 (t)}k {1 − pa , 0 (t)}n −k ≥ 1 − α1

(4)

k=0

(2)

i= 1

) removed, Sd∗ (to ) where M(−i ) is the original input data with (y i , zi (−i ) is a classifier is the best subset of d features at to , Ψs ∗d (t o ) ·|M obtained from implementing SVM with S∗d (to ) on the training data set M(−i ) , I [a1 , a2 ] = 1 if a1 = a2 and 0 otherwise. IV. DETERMINATION OF OPTIMAL NUMBER OF TEST CYCLES Single sampling plan involving attribute inspection is often used to determine the acceptability of a collection of units (e.g., a lot) regarding its intended quality. In this plan, n units are randomly selected from a lot and are subjected to a set of attribute inspection procedures. The corresponding lot is accepted if the number of faulty units is c or fewer. Two types of errors are possible in the attribute sampling: error of good unit being classified as the faulty (Type I error) and error of faulty unit being classified as the good (Type II error). Denote α1 and α2 the probabilities of Types I and II errors, respectively, then a sampling plan can be designed so that it provides a high (1 − α1 ) probability of accepting a lot with acceptable quality level p0 and provides a low probability α2 of accepting a lot with lot tolerance percent faulty p1 .

and A(p1 ) =

c n

k

{pa , 1 (t)}k {1 − pa , 1 (t)}n −k ≤ α2

(5)

k=0

where pa , i (t) = {1 − q1 (t)} − pf {1 − q1 (t) − q2 (t)} , i = 0, 1. A tradeoff between the sample size and the number of test cycles should be made in the design of sampling plans when the classification errors exist for efficient quality control and cost reduction. We assume that total test cost is expressed as

T C(n, t) =

nCs + tCo ,

if t < t

nCs + tCo + δ0 exp {δ1 (t − t )} ,

if t ≥ t

(6)

where nCs is total sample cost, Co is cost of operating a cycle-life test per a unit, δ0 and δ1 are delay cost-related coefficients. The cost due to the longer test time often includes not only production delay costs but also opportunity costs. We assume that it is zero at t < t but increases exponentially for t ≥ t . The optimal design problem of sampling plans can be expressed by the nonlinear constrained integer program that consists of finding



Fig. 4.

Block diagram of the modified SFFS.

n∗ and t∗ that minimize the total cost (6) with the constraints (4) and (5). Instead of using binomial monograph in [18], we employ the normal approximation in [19] to solve (4) and (5). It is easier but accurate even for cases a small value of n. Given α1 , α2 , p0 , p1 , Cs ,Co , δ0 , and δ1 , the optimal n∗ and t∗ can be determined by the following procedure. Step 0: Set t = 0. Take a sufficiently large value for T C ∗ Step 1: t = t + 1. Choose n ˜ (t) to be the smallest integer that satisfies the following.

√

483

n≥

ωα 1

pa , 0 (t) {1 − pa , 0 (t)} + ωα 2 pa , 1 (t) {1 − pa , 1 (t)} pa , 1 (t) − pa , 0 (t) (7)

Step 2: Find c˜(t) as the smallest integer satisfying

n ˜ (t)pa , 0 (t) {1 − pa , 0 (t)} + n ˜ (t)pa , 0 (t) − 0.5. (8) When n ˜ (t) does not satisfy with c ≥ ωα 1

c˜(t) + 0.5 − npa , 1 (t) −ωα 2 ≥ npa , 1 (t) {1 − pa , 1 (t)}

(9)

then the value of n ˜ (t) is recalculated accordingly. as shown (10) at the bottom of the next page. ˜ (t), t∗ = t, and T C ∗ = Step 3: Calculate T C (˜ n(t), t). Set n∗ = n T C (˜ n(t), t) if T C (˜ n(t), t) < T C ∗ . Go to step 1. V. MOTIVATING EXAMPLE REVISITED

where ωp is the pth quantile of the standard normal distribution.

The proposed method was applied to 43 sample profiles: 26 profiles of conforming samples and 17 profiles of nonconforming samples. To


484


TABLE I CHANGES IN SELECTED FEATURES SETS (∗ 2ND DERIVATIVE)

Fig. 6.

Error rate versus cycle.

TABLE II EXAMPLES OF SAMPLING PLANS WITH THE CLASSIFICATION ACCURACY (α 1 = 0.05, α 2 = 0.1, p 0 = 0.01)

Fig. 5.

Computational time for selecting the best features (d = 15).

see how flexible a combination of the derivatives having different orders works, the changes in optimal feature sets are observed. For example, Table I shows each set of selected features (d = 10) at 20 and 50 cycles. There is no second derivative on the list for 20 cycles (No. 1–20: 1st derivative, No. 21–40: 2nd derivative) and, on the other hand, 4 of 10 features are second derivatives on the list for 50 cycles (No. 1–50: 1st derivative, No. 51–100: 2nd derivative). It implies that the functional classification using the first derivative only may be successful for the data of the initial period but may not become an effective one any more for the wider range of the observation period. The computational time for selecting the best subset of features (d = 15) using the existing and modified SFFS algorithms is summarized in Fig. 5. We used a computer with Pentium IV 3.6-GHz processor and MATLAB 6.5 as the programming language. As expected, the proposed algorithm finds the similar feature sets but usually takes smaller computational time compared to the SFFS particularly at the longer cycle runs. However, the computation time of the proposed algorithm could be slightly larger than that of SFFS algorithm when the optimal feature sets for the current cycle runs are quite different from the one that was obtained from the previous cycle runs. For example, around 24 and 54 cycle runs, the features selected from the previous runs were replaced with several new features, resulting in increased search time of modified SFFS. Fig. 6 shows error rates produced by applying the dual features FSVM to discriminate between the conforming and nonconforming cells. During the classification procedure, if the error rate at the current cycle run is greater than the one at the previous stage, then the current error rate is set to the previous value. Therefore, the error rate has a monotonic nonincreasing function of cycles. Furthermore, the error

√

n≥

ωα 2

pa,1 (t) {1 − pa,1 (t)} +

rates of the proposed dual features FSVM are compared with the usual SVM that uses the original degradation profiles. The proposed method gives the improved performance for most of test cycles. Note that one may have tolerable error rates even with heavily truncated number of cycles (e.g., 20 cycles). Table II gives the sampling plans for some exemplary cases and shows how a sampling plan changes according to the classification accuracy. The required sample size highly depends on the number of cycles truncated for observations. Assume that the delay cost incurs if the required number of cycles is over 50 cycles (i.e., t = 50). Furthermore, Cs , Co , δ0 , and δ1 are estimated at $80, $0.5, 2, and 0.05, respectively. Then, given (α1 , α2 , p0 , p1 ) = (0.05, 0.1, 0.01, 0.15), the optimal values are determined as (n∗ , c∗ , t∗ ) = (58, 5, 152), T C ∗ = $5, 044. VI. CONCLUDING REMARKS In this correspondence, we proposed a new framework in assessing the cycle lives of rechargeable batteries within a shorter test time. In such a framework, the dual features FSVM was newly proposed and proved to give excellent performance in screening out the lots of poor quality for the purpose of qualification even if there is a sizable reduction in the test time. Furthermore, the number of test cycles and sample size for the attribute sampling plans were optimally determined in terms of the relating cost. The present investigation deals with the case where attribute sampling plans are employed to monitor time-dependent quality of products. As a further study, it may be extended to the case of variable

ωα2 2 pa,1 (t) {1 − pa,1 (t)} + 4 {˜ c(t) + 0.5} pa,1 (t) . 2pa,1 (t)


(10)


sampling plans, leading to the reduction in the sample size required in each sampling plan. Also, we can apply some boosting algorithm to improve the performance of the proposed algorithm under the environment of small sample size. ACKNOWLEDGMENT The authors would like to thank the Associate Editor and three anonymous reviewers for the valuable comments. The part of this work was supported by the National Science Foundation grant CMMI0644830. REFERENCES [1] B. Ribeiro, “Support vector machines for quality monitoring in a plastic injection molding process,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 35, no. 3, pp. 401–410, Aug. 2005. [2] O. A. Omitaomu, M. K. Jeong, A. B. Badiru, and J. W. Hines, “Online support vector regression approach for the monitoring of motor shaft misalignment and feedwater flow rate,” IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 37, no. 5, pp. 962–970, Sep. 2007. [3] B. A. Johnson and R. E. White, “Characterization of commercially available lithium-ion batteries,” J. Power Sources, vol. 70, pp. 48–54, 1998. [4] R. B. Wright and C. G. Motloch, “Cycle-life studies of advanced technology development program gen 1 lithium ion batteries,” US Department of Energy, Washington, DC, Tech. Rep. US DOE/ID10845, Mar. 2001, [Online]. Available: http://www.osti.gov/bridge/ purl.cover.jsp;jsessionid=7D8EDA3FD3A3B5BD20B6BAD2DAA104E 2?purl=/911513-qCTCpR/ [5] W. L. Pearn, M. H. Shu, and B. M. Hsu, “Monitoring manufacturing quality for multiple Li-BPIC processes based on capability index Cp m k ,” Int. J. Prod. Res., vol. 43, no. 12, pp. 2493–2512, 2005.

485

[6] J. I. Park, S. J. Kim, and M. K. Jeong, “A new tolerance design method for a secondary rechargeable battery using design of experiments with mixture,” Qual. Rel. Eng. Int., vol. 24, no. 5, pp. 543–556, 2008. [7] S.-T. Tseng and H.-F. Yu, “A termination rule for degradation experiments,” IEEE Trans. Rel., vol. 46, no. 1, pp. 130–133, Mar. 1997. [8] P. Cogdill and P. Dardenne, “Least-squares support vector machines for chemometrics: An introduction and evaluation,” J. Near Infrared Spectrosc., vol. 12, pp. 93–100, 2004. [9] J. Ramsay and B. Silverman, Functional Data Analysis. New York: Springer-Verlag, 1997. [10] F. Ferraty and P. Vieu, “Curves discriminations: A nonparametric functional approach,” Comput. Stat. Data Anal., vol. 44, no. 1–2, pp. 161–173, 2003. [11] F. Rossi and B. Conan-Guez, “Functional multi-layer perceptron: a nonlinear tool for functional data analysis,” Neural Netw., vol. 18, no. 1, pp. 45–60, 2005. [12] C. de Boor, A Practical Guide to Splines. New York: Springer-Verlag, 2001. [13] P. Pudil, J. Novovicova, and J. Kittler, “Floating search methods in feature selection,” Pattern Recognit. Lett., vol. 15, no. 12, pp. 1119–1125, 1994. [14] K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1990. [15] R. C. Collins, K. E. Case, and G. K. Bennett, “The effects of inspection error on single sampling inspection plans,” Int. J. Prod. Res., vol. 11, no. 3, pp. 280–298, 1973. [16] J. E. Biegel, “Inspection errors and sampling plans,” AIIE Trans., vol. 6, no. 4, pp. 284–287, 1974. [17] C. G. Drury, “Integrating human factors models into statistical quality control,” Human Factors, vol. 20, no. 5, pp. 561–572, 1978. [18] D. C. Montgomery, Introduction to Statistical Quality Control. New York: Wiley, 1997. [19] W. J. Braun, “Replacing a striped-box with the normal approximation,” 2000, [Online]. Available: http://www.amstat.org/publications/ jse/secure/v8n2/braun.cfm#Mitra
