Measuring Reliability During Product Development Considering Aleatory and Epistemic Uncertainty Zhiguo Zeng, PhD candidate, Beihang University Rui Kang, Professor, Beihang University Meilin Wen, Assitant Professor, Beihang University Yunxia Chen, Professor, Beihang University Key Words: reliability metrics, reliability evaluation, aleatory uncertainty, epistemic uncertainty SUMMARY & CO1CLUSIO1S Traditionally, reliability engineers use probabilistic reliability metrics (e.g., the probability of failures) to measure a product’s reliability. In the product development phase, the probabilistic reliability metrics are predicted by developing physics-of-failure-based (PoF-based) models to calculate the time-to-failure (TTF) and propagating the uncertainty in the model parameters. The prediction method only considers aleatory uncertainty (the variability of the parameters). In real cases, however, products are also exposed to epistemic uncertainty, which is the result of our incomplete knowledge of failures. In order to measure reliability in presence of both aleatory and epistemic uncertainty, belief reliability, a new reliability metric is defined in this paper and an evaluation method for the belief reliability is proposed. Epistemic uncertainty is incorporated in the evaluation by assessing the performances of the reliability techniques implemented in the product development phase. The applications of the metric and the evaluation method are illustrated through an example. The result indicates that the probabilistic reliability metrics ignore the epistemic uncertainty and overestimate the reliability. Thus, belief reliability is a more appropriate reliability metric for the product development phase. 1 I1TRODUCTIO1 Reliability is defined as the attribute of a component or system to perform a required function for a given period of time when used under stated operating conditions [1]. To use this concept in an operational sense, the attribute should be measured by quantitative metrics. These metrics are referred to as reliability metrics. Traditionally, reliability metrics are defined based on probability theory, such as the reliability (interpreted as a probability) and the mean time to failure (MTTF). Measuring reliability refers to determining the values of the reliability metrics. The reliability metrics can be estimated from the collected time-to-failure (TTF) data, using statistical methods. In the product development phase, however, few failure data are available. To address this problem, the physics-of-failure
978-1-4799-6703-2/15/$31.00©2015 IEEE
(PoF) method was introduced to predict reliability in the product development phase [2]. In the PoF method, a deterministic PoF model is developed first to predict the product’s failure behaviors. Then the uncertainty in the model parameters is propagated following probabilistic laws to predict the reliability [3]. The PoF method has been used to measure the reliability of mechanical parts since the 1980s [4]. Pecht et al. [2] first introduced the PoF method to predict the reliability of electronic devices and greatly spurred its application. Zeng et al. [5, 6] considered the interdependence among components and failure mechanisms, and developed a PoF-based system failure behavior modeling method. The method has been used to predict the reliability of a hydraulic servo actuator in its development phase [7]. When the PoF method is used to measure reliability, it is assumed that the failure mechanisms are well-understood so that the TTF can be predicted by a deterministic PoF model. The observed distribution of the TTFs is caused by the uncertainty in the parameters of the PoF model [3]. The uncertainty is referred to as aleatory uncertainty, which means the uncertainty in the physical world [8]. In the product development phase, apart from the aleatory uncertainty, the product is subjected to epistemic uncertainty. Epistemic uncertainty is the result of insufficient knowledge and will be reduced as the knowledge is increased [9]. For instance, when developing PoF models, we might not be able to identify all the failure mechanisms. In addition, the failure mechanisms might not be clearly understood and well modelled. Thus, the developed PoF model might be inaccurate, which is caused by the insufficient knowledge. However, when reliability is predicted using the PoF-based method, the epistemic uncertainty is not taken into account. Thus, the resulted reliability metrics might be misleading. On the other hand, a lot of reliability techniques used in the product development phase aim at reducing epistemic uncertainty. For example, by performing a Failure Mode, Mechanisms and Effects Analysis (FMMEA), knowledge of the failure modes and failure mechanisms can be gained so that the epistemic uncertainty is reduced. Also, a High Accelerated Life Testing reduces epistemic uncertainty by
stimulating potential failures so that the failure modes and mechanisms which are not identified by previous analyses can be identified. These reliability techniques obviously benefit the product’s reliability. However, they are not considered in the PoF-based reliability measuring method. Thus, valuable information about the reliability is lost. To address these problems, a new method to measure the reliability during the product development phase will be developed in this paper, with aims to account for both the aleatory and epistemic uncertainty, and to incorporate the reliability techniques into the measuring of reliability. The rest of the paper will be organized as follows. In Sect. 2, the PoF-based reliability measuring method will be reviewed. The definition of belief reliability will follow in Sect. 3. In Sect. 4, an evaluation method of the belief reliability will be presented. A case study will be performed in Sect. 5 to demonstrate the developed reliability metric and the evaluation method. 2 MEASURI1G RELIABILITY U1DER ALEATORY U1CERTAI1TY This section reviews how to measure reliability when only aleatory uncertainty is considered. Before introducing the measuring method, some important concepts should be presented first. Definition 1 (Performance parameter and functional threshold). Assume that there are two parameter p and pth , and the component or the system fails if and only if p t pth . Then the parameter p is referred to as the performance parameter, while the corresponding pth is referred to as the functional threshold. The p and pth are used to indicate failures. For example, a shaft is designed to transmit torque. Thus, the output torque can be chosen as the performance parameter, with the functional threshold to be its requirement. Also, a failure happens when the stress of the shaft is greater than its strength. Thus, the stress can also be chosen as the performance parameter, with the strength to be the functional threshold. The performance parameter defined in Definition 1 is a larger-the-better (LTB) parameter. In actual cases, the parameter might be smaller-the-better (STB) or nominal the best (NTB). Definitions of the other performance parameters can be easily generalized from Definition 1. Definition 2 (Margin). Margin, denoted by M , is the difference between the performance parameter p and corresponding functional threshold pth . For a LTB performance parameter, the expression of M is given in equation (2). M p pth . (1) Note that the definition in equation (1) is for LTB parameters. For other parameters, the definition in equation (1) can be generalized accordingly. The reliability measurement starts by developing a deterministic model for the margin, denoted by the FM in equation (2), where x is the input parameters of the model. The model can be developed by combing the PoF models and
the physical functional models, following the failure behavior modeling scheme presented in [5] and [6]. (2) M fM x . The x in equation (2) is subjected to aleatory uncertainty. For example, the strength of the material often exhibits large variations due to the aleatory uncertainty in the material’s properties. The result of the aleatory uncertainty is that the M becomes a distribution rather than a precise value. An illustration of the distribution of margins resulted from the aleatory uncertainty is given in Figure 1. The U a in Figure 1 measures the spread of the distribution due to aleatory uncertainty, while M design (the design margin) measures the center of the distribution. In most cases, the standard deviation of the margin distribution, V M , is used as a measure of U a . Since M 0 indicates a failure, the failure region in Figure 1 can be used to calculate the probabilistic reliability, as shown in equation (3). Although Equation (3) is derived based on the assumptions that the margins follow a normal distribution, it is widely used in most probabilistic PoF (PPoF) methods as an efficient approximation to calculate reliability [4]. § M design · R p ) ¨¨ ¸¸ , © Ua ¹ where, (3)
) x
§ t2 · e x p ¨ ¸ dt . ³ ¨ 2¸ 2S f © ¹ 1
x
M design
Ua
Figure 1 Distribution of Margins under Aleatory Uncertainty 3 DEFI1ITIO1 OF BELIEF RELIABILITY In this section, belief reliability is developed as a new reliability metric which considers both aleatory and epistemic uncertainty. The definition is given in Sect. 3.1. Then, a discussion on the properties of the metric is given in Sect. 3.2. 3.1 Definition of belief reliability Using equation (3) to calculate the reliability implies that the prediction model for margins in equation (2) is accurate and the only source of uncertainty is the aleatory uncertainty in its parameters. This assumption is far from realistic because of the influence of epistemic uncertainty. To reflect the influence, it is assumed that the epistemic uncertainty enlarges the spread of the margin distribution in Figure 1. The center of the margin distribution is assumed to be unaltered by the epistemic uncertainty, as illustrated in Figure 2.
3.2 Properties of belief reliability M design
Ua Ua Ue
Figure 2 Influence of Epistemic Uncertainty In Figure 2, the additional dispersion due to the influence of epistemic uncertainty is measured by U e . The more severe the influence of epistemic uncertainty is, the larger the U e will be. In engineering practices, reliability techniques such as FMMEA and HALT are implemented to reduce the epistemic uncertainty. Thus, the value of U e can be determined by evaluating the performances of these reliability techniques. The better the performances are, the smaller the U e will be. If we could obtain perfect knowledge about the failures, the U e will decrease to zero. In this ideal case, the reliability is solely influenced by the aleatory uncertainty and the method discussed in Sect. 2 can be used to measure the reliability. Based on the assumption in Figure 2, the new reliability metric, belief reliability, is defined in equation (4) to measure the reliability of the products subjected to both aleatory and epistemic uncertainty. The RB in equation (4) denotes belief reliability and M design ,U a have the same meaning as in equation (3). The parameter U e is used to represent the effect of epistemic uncertainty. § M design · RB ) ¨¨ ¸¸ , © Ua Ue ¹ f M design f, U a ,U e ! 0, where, § t2 · e xp ¨ ¸ dt. ³ ¨ 2¸ 2S f © ¹ (4) In order to make the calculation of RB easier, we define the factor of aleatory uncertainty (AUF) and the factor of epistemic uncertainty (EUF), as shown in equation (5). In the equation, the D a and D e denote the AUF and EUF, respectively. Substituting equation (5) into equation (4), a more concise expression for belief reliability can be obtained, as shown in equation (6). Ua Ue (5) Da ,De . M design M design ) x
RB
1
x
§ · 1 °) ¨ ¸ , M design t 0, ° © Da De ¹ ® § · 1 ° °1 ) ¨ D D ¸ , M design 0. e¹ © a ¯
(6)
From the definition in equation (6), the belief reliability is determined by three factors, M design , D a and D e , which represent the effect of the designed margin (nominal value), aleaotory uncertainty and epistemic uncertainty, respectively. In this section, we will discuss some mathematical properties of the belief reliability function in equation (6) to explain how the belief reliability is influenced by the three factors. The M design takes values in the interval f, f . In the interval, the belief reliability is an increasing function with respect to M design , which means that we will be more sure that the product is reliable if the designed margin is increased. Besides, it can be proved that when M design ! 0 , regardless of which values the two uncertainty factors take, we Similarly, always have RB ! 0.5 , and vise versa. If M design 0 indicates RB 0.5 , and vise versa. M design 0 , then RB 0.5 regardless of which values the two uncertainty factors take. Figure 3 illustrates how the RB varies with the designed margin. If we have RB ! 0.5 , the product is believed to be more likely to be reliable than unreliable. When M design ! 0 , the nominal design of the product is reliable. Thus, we always have a belief reliability greater than 0.5. The same principle holds for RB 0.5 . If we have RB 0.5 , we cannot judge whether the product is more likely to be reliable or unreliable. This is the point with the maximum uncertainty. The factors D a and D e quantify the effect of aleatory and epistemic uncertainty. Both the two factors take values in the interval > 0, f @ . The greater the effect of uncertainty is, the larger the factors will be. As D a o f or D e o f , the RB will approach 0.5 regardless of the value of M design , since 0.5 is the value representing a state with maximum uncertainty. When D a o 0 and D e o 0 , we have RB 0 for M design 0 and RB 1 for M design ! 0 . This is because when both the factors equal to zero, there is no uncertainty. Thus, a designed margin that is greater than zero will guarantee that the product is reliable. Therefore, we have RB 1 . The same reasoning can be used to explain why RB 0 when M design 0 . To investigate the influence of epistemic uncertainty on the belief reliability, the RB s under three levels of D e are drawn in Figure 4, with a fixed U a 1 . From the figure, we can see that considering epistemic uncertainty will decrease the predicted reliability. As the De goes to infinity, the belief reliability approaches 0.5 , which represents a state with maximum uncertainty. Further, when D e 0 , it can be seen by comparing equation (3) and (6) that the belief reliability will equal to the probabilistic reliability predicted by equation (3). This is a natural result since when D e 0 , there is no epistemic uncertainty. Thus, the reliability is determined by the design margin and aleatory uncertainty, which satisfies the requirement of using equation (3).
is estimated first, according to equation (8). Then, D a is obtained according to its definition in equation (5). The notations in equation (8) have the same meanings as those in equation (7).
¦ i 1 mi M design n
VM
. (8) n 1 In the next step, the epistemic uncertainty in the development phase should be evaluated to determine the D e . The approach to determine D e will be presented in Sect. 4.2. Finally, the belief reliability can be calculated according to its definition in equation (6). Ua
Figure 3 Influence of Design Margin on RB
M design
4.2 Determination of EUF
4 BELIEF RELIABILITY EVALUATIO1 This section presents an evaluation method to determine the belief reliability. The evaluation procedures are given first in Sect. 4.1. Then, an approach to determine the EUF by evaluating the performances of related reliability techniques is presented in Sect. 4.2. 4.1 The evaluation procedures The defined RB incorporates the contributions of design margin, aleatory uncertainty and epistemic uncertainty. In order to evaluate the RB , the contributions of the three factors should be determined first. Then, the contributions are combined according to equation (6) to get the belief reliability. The details of the evaluation procedures are presented in Figure 5. The belief reliability evaluation starts with developing the prediction model for the margin. The model can be developed by combing the PoF models and the physical functional models, following the failure behavior modeling scheme presented in [5] and [6]. Next, the distribution of the margins should be determined, considering the influence of aleatory uncertainty (see Figure 1 for an illustration). The uncertainty in each parameter of the margin model is first characterized by identifying its probability density function. Then, the distribution is obtained through uncertainty propagation, according to probabilistic laws. Then, M design is determined from the distribution of the margins, according to equation (7). It is assumed in equation (7) that the distribution of margins is approximated by n samples, mi , i 1, 2, , n .
1 n ¦ mi . ni 1
RB
Figure 5 Belief reliability evaluation procedures
Figure 4 Influence of Epistemic Uncertainty on RB
M design
2
(7)
Next, the D a is determined. In order to calculate D a , U a
EUF is defined to measure the epistemic uncertainty. In the product development phase, the five reliability techniques in Figure 6 are often implemented to reduce epistemic uncertainty. Thus, the EUF can be determined by evaluating the performances of the five techniques. Among the five techniques, x Failure behavior modeling refers to building prediction model for the margin, following the failure behavior modeling scheme presented in [5] and [6]. The model is built based on understanding of the mechanisms of normal functioning and the failure mechanisms. Thus, it helps us to improve our knowledge about how the product functions and fails, so that the epistemic uncertainty is reduced. x FMEA reduces epistemic uncertainty by identifying potential failure modes. x HALT reduces epistemic uncertainty by stimulating potential failures so that the failure modes and mechanisms which are not identified in previous analyses can be identified. x Robust design is a design technique which makes the designed product insensitive to environmental variations. Thus, even though we are not sure about the actual environmental conditions due to epistemic uncertainty, we are still able to design reliable products because they are insensitive the environmental conditions. x TAAF refers to test, analysis and fix, which is the central part of a reliability growth program. TAAF uses tests to stimulate potential failures and then improves reliability by analyzing and fixing the failures. In this way, the previously unknown failures are identified so that the epistemic uncertainty is reduced. The framework to evaluate De is given in Figure 7. First, the performance of each reliability technique is evaluated.
The performance of each technique is classified into three classes, good, medium and bad. Evaluation criterion for each class is established by experts and used to classify the performance of the reliability techniques. A score, denoted by Si , is assigned to each technique based on the result of the classification ( i 1, 2, ,5 ), where, Si 2 if the state is good, x Si 1 if the state is medium x x and Si 0 is the state is bad. An illustration of the scoring criteria is given in Table 1. Due to page limits, we only list the scoring criteria for the failure behavior modeling and the FMEA. Then, all the five scores are summarized, arriving at a total score, S6 . The S6 denotes the joint contribution of all the reliability techniques which reduce the epistemic uncertainty and indicates the severity of epistemic uncertainty. The severity of epistemic uncertainty is classified into three stages, according to the values of S6 . x If S6 t 8 , the severity of epistemic uncertainty is recognized as “little”, with the D e 0.3 . x If 5 d S6 8 , the severity of epistemic uncertainty is recognized as “medium”, with the D e 1.5 . x If 0 d S6 5 , the severity of epistemic uncertainty is recognized as “high”, with the D e 100 , as shown in Figure 7.
Figure 6 Reliability techniques used to reduce epistemic uncertainty Technique
Evaluating Criteria Both the functional model and the failure mechanism model are accurate and validated. z Quantitative failure behavior models are built and validated. The FBM is conducted qualitatively or without validation. There is no FBM effort. z The FMEA is conducted by experienced design engineers along with reliability engineers. z Design measures are taken with respect to the identified failure modes. The identified failure modes are not shared with the design engineers. There is no FMEA effort or the results provide little help to the design engineers.
Score
z
Failure Behavior Modeling
FMEA
Table 1 Illustrative scoring criteria
2
1 0
2
1 0
S1
S2
S3
S4
S5
5
S6
¦ Si i 1
S6 t 8 D e
0.3
5 d S6 8 D e
0 d S6 5 D e
100
1.5
Figure 7 Determination of D e by evaluating the reliability techniques 5 A1 ILLUSTRATIVE EXAMPLE The belief reliability evaluation of an electro-hydraulic servo actuator (HSA) is conducted in this section as an illustrative example, according to the procedures presented in Figure 5. The failure behavior model of the HSA has been established in a previous study [7], which serves as the prediction model for the margin in this example. The simulation-based approach in the PPoF methods (see [3] for details) is employed to account for the aleatory uncertainty in the parameters. The result is shown in Figure 8. From the simulation results, the M design and D a can be calculated using equation (7) and (8). The resulted M design and D a are 0.0183 and 0.6995 , respectively. Then, we have to determine the value of D e . Experts are invited to develop the scoring criteria for each reliability technique, as indicated in Table 1. Suppose the performances of the five reliability techniques implemented in the development phase of the HSA has been evaluated according to the established criteria, with the scores to be > S1 , S2 , , S5 @ >1,1, 0, 2,1@ . Since S6 5 , according to Figure 7, we have D e 0.3 . Finally, the belief reliability can be predicted following equation (6) and the result is shown in Table 2. If we only consider the aleatory uncertainty, probabilistic reliability can be predicted following the PPoF approach shown in equation (3). The result is also given in Table 2 for comparisons. The probabilistic reliability in Table 2 is calculated based on the assumption that the PoF model is accurate and all the uncertainty in the TTF is the result of the uncertainty in the model parameters. The belief reliability, however, considers the epistemic uncertainty in the process of measuring reliability. Thus, the evaluated belief reliability is less than the probabilistic reliability.
understand, assess and manage risk and the unforeseen.” Reliability Engineering & System Safety 2014; 121: pp 1-10.
(69
6SRRO 6SRRO 6SRRO 6SRRO +&
pHSA p th
BIOGRAPHIES
T TF
t
Zhiguo Zeng, PhD Candidate School of Reliability and Systems Engineering Beihang University 37 Xueyuan Road, Haidian, Beijing 100191 China
CoD
Figure 8 Predicted margin distribution under aleatory uncertainty Calculation method Probabilistic reliability according to equation (3) Belief reliability according to equation (6)
e-mail:
[email protected] Result 0.9236 0.8415
Table 2 Comparison between probabilistic reliability and belief reliability ACK1OWLEDGEME1T Miss Fan Mengfei from Beihang University helped to develop the scoring criteria for the illustrative example. The authors deeply appreciate her excellent work. REFERE1CES 1. 2. 3.
4. 5.
6.
7.
8. 9.
CE. Ebeling, An introduction to reliability and maintainability engineering. Long Grove, Waveland Press, 2010. M. Pecht, A. Dasgupta, D. Barker and et al. “The reliability physics approach to failure prediction modelling,” Qual Reliab Eng Int 1990; 6(4): pp 267-273. PL. Hall and JE. Strutt, “Probabilistic physics-of-failure models for component reliabilities using Monte Carlo simulation and Weibull analysis: A parametric study,” Reliability Engineering & System Safety 2003; 80(3): pp 233-242. JA. Collins, HR. Busby, GH. Staab, Mechanical design of machine elements and machines. Wiley, 2009. Z. Zeng, R. Kang, Y. Chen. “A physics-of-failure-based approach for failure behavior modeling: With a focus on failure collaborations,” Proc. Annual European Safety and Reliability Conference (ESREL) Wrocáaw, Poland, 2014. Z. Zeng, R. Kang, Y. Chen. “Using physics-of-failure models to predict systems ÿ reliability: From failure competition to failure collaboration,” Proc. Inst. Mech. Eng. Part O-J. Risk Reliab. (Submitted) 2014. Z. Zeng, R. Kang, Y. Chen. “Life oriented design (LOD): Using time-to-failure distribution as the objective of quantitative reliability design,” Proc. International Conference on Modelling in Industrial Maintenance and Reliability (MIMAR), Oxford, UK, 2014. AD. Kiureghian and O. Ditlevsen, “Aleatory or epistemic? Does it matter?” Struct Saf 2009; 31(2): pp 105-112. T. Aven and BS. Krohn, “A new perspective on how to
Zhiguo Zeng received his B.S. degree in Quality and Reliability Engineering from Beihang University in 2011. He is now a Ph.D. candidate (since 2011) in the School of Reliability and Systems Engineering, Beihang University. His research interests include failure behavior modeling based on physics-of-failure models, life oriented design techniques and belief reliability theory. Rui Kang, Professor School of Reliability and Systems Engineering Beihang University 37 Xueyuan Road, Haidian, Beijing 100191 China e-mail:
[email protected] Rui Kang obtained his Master’s degree from Beihang University in 1990. He is now the Chair Professor of the School of Reliability and Systems Engineering, Beihang University. His main research interests include reliability design and experiment technology based on the Physics of Failure, prognostics and health management (PHM) and network reliability. Meilin Wen, PhD, Assistant Professor School of Reliability and Systems Engineering Beihang University 37 Xueyuan Road, Haidian, Beijing 100191 China e-mail:
[email protected] Meilin Wen obtained her PhD degree from Tsinghua University in 2008. Her research interests include Uncertainty Theory and its applications, data envelop analysis and belief reliability theory. Yunxia Chen, PhD, Professor School of Reliability and Systems Engineering Beihang University 37 Xueyuan Road, Haidian, Beijing 100191 China e-mail:
[email protected] Yunxia Chen obtained her Ph.D. degree from Beihang University (BUAA) in 2004. Her research focuses on reliability design and experiment technology based on the Physics of Failure.
