Passive Testing - A Constrained Invariant Checking Approach - SaToSS

2 downloads 0 Views 187KB Size Report
5 reports some experiments of the algorithms on the Simple Connection Protocol ..... to connect the two layers (CONcnf(-)) only if the lower layer refused the con-.
3DVVLYH7HVWLQJ $&RQVWUDLQHG,QYDULDQW&KHFNLQJ $SSURDFK %HKURX]7RUN/DGDQL%DSWLVWH$OFDOGH DQG$QD&DYDOOL



'HSDUWPHQWRI&RPSXWHU(QJLQHHULQJ8QLYHUVLW\RI,VIDKDQ,VIDKDQ,UDQ ODGDQL#HQJXLDFLU  ,QVWLWXWH1DWLRQDOGHV7HOHFRPPXQLFDWLRQV*(7,17(YU\)UDQFH ^DQDFDYDOOLEDSWLVWHDOFDOGH`#LQWHYU\IU

$EVWUDFW 3DVVLYH WHVWLQJ RI D QHWZRUN SURWRFRO LV WKH SURFHVV RI GHWHFWLQJ IDXOWV LQ WKH SURWRFRO LPSOHPHQWDWLRQ E\ SDVVLYHO\ REVHUYLQJ LWV LQSXWRXWSXW EHKDYLRUV H[HFXWLRQWUDFH ZLWKRXWLQWHUUXSWLQJWKHQRUPDOQHWZRUNRSHUDWLRQV ,QREVHUYLQJWKHWUDFHZHFDQIRFXVRQWKHPRVWH[SHFWHGUHOHYDQWSURSHUWLHVRI WKHSURWRFROVSHFLILFDWLRQE\GHILQLQJVRPHLQYDULDQWVRQWKHVSHFLILFDWLRQDQG FKHFNLQJWKHPRQWKHWUDFH:KLOHLQWXLWLYHH[WUDFWLRQRIWKHLQYDULDQWVIURPWKH SURWRFROUHTXLUHPHQWVZLWKUHVSHFWWRWKHFRQWUROSRUWLRQRIWKHSURWRFROV\VWHP LVUHODWLYHO\VLPSOHWDNLQJWKHGDWDSRUWLRQLQWRDFFRXQWLVGLIILFXOW,QWKLVS D SHUZHSURSRVHDOJRULWKPVIRUFKHFNLQJWKHFRUUHFWQHVVRIJLYHQLQYDULDQWVRQ WKH VSHFLILFDWLRQ DQG H[WUDFWLQJ WKH UHTXLUHG FRQVWUDLQWV RQ WKH YDULDEOHV GDWD SRUWLRQ 2QFHZHJHQHUDWHWKHFRQVWUDLQWVIRUDJLYHQLQYDULDQWZHFDQFKHFNLI WKHH[HFXWLRQWUDFHLVFRQILUPHGE\WKHVSHFLILFDWLRQZLWKUHVSHFWWRWKH LQYDUL DQW DQG LWV FRQVWUDLQWV :H VKRZ WKH DSSOLFDELOLW\ RI WKH DOJRULWKP RQ D FDVH VWXG\WKHVLPSOHFRQQHFWLRQSURWRFRO 6&3 

.H\ZRUGVSDVVLYHWHVWLQJLQYDULDQWVLQYDULDQWFKHFNLQJFRQVWUDLQWVROYLQJ6&3

,QWURGXFWLRQ 7HVWLQJQHWZRUNSURWRFROLPSOHPHQWDWLRQVWRDVVXUHWKDWWKH\ZRUN DVWKHLUVSHFLILFD WLRQLVRI KLJKLPSRUWDQFH,QVWHDGRIFRQYHQWLRQDODFWLYH WHVWLQJWKHUHDUHSURSRVL

WLRQVWRXVHSDVVLYHWHVWLQJIRUQHWZRUNSURWRFROV\VWHPVZKLFKPHDQVREVHUYLQJWKH LQSXWRXWSXWEHKDYLRURIWKHLPSOHPHQWDWLRQ LHH[HFXWLRQWUDFH ZLWKRXWLQWHUIHULQJ LWV QRUPDO EHKDYLRU >@>@  7KH QDLYH DSSURDFK WR SDVVLYH WHVWLQJ LV WR UHFRUG WKH

H[HFXWLRQ WUDFH DQG WU\ WR ILQG LWV IDXOWV E\ FRPSDULQJ LW ZLWK WKH VSHFLILFDWLRQ >@

>@ >@ 2WKHUDSSURDFKHVWU\WRH[WUDFWWKHFULWLFDOSURSHUWLHVRIWKHVSHFLILFDWLRQLQ

WKHIRUPRIVRPHLQYDULDQWVDQGWKHQWU\WRREVHUYHWKHPRQWKHLPSOHPHQWDWLRQ >@ >@ 

0RVWRISDVVLYHWHVWLQJPHWKRGVDUHIRFXVHGRQWKHFRQWUROSDUWVRIWKHV\VWHPXQGHUWHVW

ZLWKRXWFRQVLGHULQJWKHGDWDSDUWVVRLWLVVXIILFLHQWIRUWKHPWRXVHILQLWHVWDWHPDFKLQHV )60V DVWKHVSHFLILFDWLRQPHWKRG7RWDNHWKHGDWDSDUWRIWKHSURWRFROVLQWRDFFRXQW H[WHQGHGILQLWHVWDWHPDFKLQHV ()60V DUHXVHGWRVSHFLI\WKHV\VWHP()60XVHVSD

UDPHWHUL]HGLQSXWRXWSXWVLJQDOVLQFOXGLQJYDULDEOHSDUDPHWHUVWRHQFRGHGDWDDVZHOODV SUHGLFDWHVDQGDFWLRQV WRFRQWUROWKH ILULQJRI WKHWUDQVLWLRQV E\ PDQLSXODWLQJWKHUHOH

YDQW GDWD 7KHUH DUH VRPH PHWKRGV SURSRVHG WR SHUIRUP SDVVLYH WHVWLQJ XVLQJ ()60 >@ >@ >@  7KHVH PHWKRGV DUH EDVHG RQ H[SORULQJ FRQVWUDLQWV RI WKH YDULDEOHV DQG FRPSDULQJWKHZKROHVSHFLILFDWLRQZLWKWKHLPSOHPHQWDWLRQUHJDUGLQJWKHFRQVWUDLQWVLQ DEDFNZDUGRUIRUZDUGPDQQHU LHWKHQDLYHDSSURDFK 

,QWKLVSDSHUZHUHSUHVHQWDPHWKRGWRSHUIRUPSDVVLYHWHVWLQJEDVHGRQWKHLQYDULDQWV

RQWKH()60,QRXUDSSURDFKILUVWZHKDYHWRH[WUDFWWKHLQYDULDQWVLQWXLWLYHO\IURP WKHSURWRFROVSHFLILFDWLRQUHTXLUHPHQWVUHJDUGLQJRQO\WKHFRQWUROSRUWLRQRIWKHSUR WRFROV$IWHUWKDWWRWDNHWKHGDWDSRUWLRQLQWRDFFRXQWZHFRQVLGHUWKHLQYDULDQWSD

UDPHWHUVDVVRPHYDULDEOHV:HSUHVHQWWZRDOJRULWKPVIRUILQGLQJWKHFRUUHVSRQGLQJ FRQVWUDLQWVRYHUWKHYDULDEOHVRIWKHLQYDULDQWVDXWRPDWLFDOO\7KHDOJRULWKPVXVHWKH XQLILFDWLRQPHWKRG>@IRUFKHFNLQJWKHFRUUHFWQHVVRIWKHJLYHQLQYDULDQWVRYHUWKH

()60 DQG ILQGLQJ WKH FRQVWUDLQWV RYHU LWV YDULDEOHV +DYLQJ WKH LQYDULDQWV DQG WKHLU FRUUHVSRQGLQJFRQVWUDLQWVLQKDQG ZHFDQFKHFNWKHH[HFXWLRQWUDFHZLWKWKHLQYDUL DQWVXVLQJSDWWHUQPDWFKLQJPHWKRGV

,WVKRXOGEHQRWHGWKDW ILQGLQJVXLWDEOHFRQWUROGULYHQ LQYDULDQWVHVSHFLDOO\ ZLWKWKH

KHOSRIDQH[SHUWLVUHODWLYHO\VLPSOHDOVRWKHUHDUHVRPHPHWKRGVWRH[WUDFWDOLPLWHG VHWRILQYDULDQWVIURPDQ()60DXWRPDWLFDOO\>@:HXVHWKHQRWLRQRILQYDULDQWLQ WURGXFHGLQ>@ZLWKOLWWOHFKDQJHVLQWKHGHILQLWLRQVWRH[WUDFWWKHFRQWUROGULYHQLQ YDULDQWVLQWXLWLYHO\

7KHUHVWRIWKHSDSHULVRUJDQL]HGDVIROORZVLQVHFWLRQVRPHSUHOLPLQDU\FRQFHSWV

QHHGHGLQWKHUHVWRIWKHSDSHUDUHGHVFULEHG,QVHFWLRQWKHQRWLRQRIIRUZDUGDQG EDFNZDUG LQYDULDQWV DUH GHVFULEHG ,Q VHFWLRQ  RXU DOJRULWKPV IRU FKHFNLQJ LQYDUL

DQWVRQDJLYHQ()60DQGH[WUDFWLQJFRUUHVSRQGLQJFRQVWUDLQWVDUHSUHVHQWHG6HFWLRQ

 UHSRUWV VRPH H[SHULPHQWV RI WKH DOJRULWKPV RQ WKH 6LPSOH &RQQHFWLRQ 3URWRFRO

6&3 WRVKRZWKHDSSOLFDELOLW\RIWKHPHWKRGLQGHWHFWLQJVXEWOHHUURUVLQVRPHJLYHQ WUDFHVRIWKHSURWRFRO,QVHFWLRQZHFRQFOXGHWKHSDSHU



3UHOLPLQDULHV ([WHQGHG)LQLWH6WDWH0DFKLQH

:HXVH([WHQGHG)LQLWH6WDWH0DFKLQHV ()60V WRVSHFLI\WKHQHWZRUNSURWRFROV

'HILQLWLRQ $Q([WHQGHG)LQLWH6WDWH0DFKLQH ()60 0LVDWXSOH0 6,2[ 7V ZKHUH6LVDILQLWHVHWRIVWDWHV,DQG2DUHWKHILQLWHVHWVRILQSXWDQGRXWSXWSD

UDPHWULFV\PEROVUHVSHFWLYHO\[LVDYHFWRUGHQRWLQJDILQLWHVHWRIYDULDEOHV7LVWKH ILQLWHVHWRIWUDQVLWLRQVDQGV LVWKHLQLWLDOVWDWH

(DFKWUDQVLWLRQW 7LVDWXSOH VVLR3$ ZKHUHVV  6DUHWKHLQLWLDODQGILQDO

VWDWHV RI WKH WUDQVLWLRQ UHVSHFWLYHO\ L  , R  2 DUH WKH LQSXW DQG RXWSXW V\PEROV

SRVVLEO\ZLWKSDUDPHWHUV UHVSHFWLYHO\3LVWKHSUHGLFDWH D%RROHDQH[SUHVVLRQ DQG $LVWKHVHTXHQFHRIDFWLRQV

'HILQLWLRQ  /HW0 6,2[7V EHDQ()60WKHVHTXHQFHLR«LQRQ LVD

SDWK LQ0LIIRUDQ\d Md QLM  ,RM  2DQGWKHUHH[LVWQWUDQVLWLRQVW «WQ  7

DQGQVWDWHVVV«VQV  6VXFKWKDWW  VVLR WQ  VQVLQRQ DQGIRU DQ\MQZHKDYHWM  VMVMLMRM  6XEVWLWXWLRQDQGXQLILFDWLRQ

2XU DOJRULWKPV XVH WKH XQLILFDWLRQ PHWKRG VR ZH ERUURZ VRPH GHILQLWLRQV IURP WKH FRQWH[WRIORJLFSURJUDPPLQJ

'HILQLWLRQ $VXEVWLWXWLRQ T LVDVHWRIELQGLQJVHDFKRIWKHIRUP97VXFKWKDW9 LVDGLVWLQFWYDULDEOHDQG7LVDWHUPT LVFDOOHG DUHQDPLQJ LILWPDSHDFKYDULDEOHWR DQHZIUHVKYDULDEOH

$SSO\LQJDELQGLQJ97WRDQH[SUHVVLRQ(UHSODFHVHDFKIUHHRFFXUUHQFHRI9LQ( E\7$SSO\LQJDVXEVWLWXWLRQT RQDQH[SUHVVLRQ(GHQRWHGE\(T DSSOLHVDOOWKHELQG LQJVLQT WR(VLPXOWDQHRXVO\DQGLQGHSHQGHQWO\

'HILQLWLRQ/HWT ^979P7P `DQG. ^86 8Q6Q `EHVXEVWLWXWLRQV 7KHFRPSRVLWLRQ RIT DQG.GHQRWHGE\TR. LVGHILQHGDV

TR. ^9 7. 9P 7P. `8^8N 6N_8N ^9 «9P``

'HILQLWLRQ $XQLILHURIWZRVLPSOHH[SUHVVLRQV(DQG)LVDVXEVWLWXWLRQT VXFKWKDW

(T )T,IWZRVLPSOHH[SUHVVLRQVKDYHDXQLILHUWKH\DUHVDLGWREHXQLILDEOHZHDOVR VD\WKDW(LVXQLILHGZLWK)E\WKHXQLILHUT$PRVWJHQHUDOXQLILHUDEEUHYLDWHGDVPJX

RIWZRVLPSOHH[SUHVVLRQV(DQG)LVDXQLILHUT WKDWLVPRUHJHQHUDOWKDQDQ\XQLILHURI (DQG )$VDQH[DPSOHWZRH[SUHVVLRQV3 ;I ; DQG3 EI D DUHQRWXQLILDEOHZKLOH WKHPRVWJHQHUDOXQLILHURIWKHH[SUHVVLRQV3 ;I D DQG3 EI <  LV^;E@6XSSRVHWKDWZHZDQWWRSDVVLYHO\WHVWDQ LPSOHPHQWDWLRQ RI WKH 6&3 UHJDUGLQJ WKH IROORZLQJ SURSHUWLHV RI WKH VSHFLILFDWLRQ ZKLFKDUHGHVFULEHGXVLQJWKHLQYDULDQWV

x , % UHIXVHFRQQHFW [ ! UHIXVH ^&21FQI  ` PHDQVWKDW6&3IDLO WRFRQQHFWWKHWZROD\HUV &21FQI  RQO\LIWKHORZHUOD\HUUHIXVHGWKHFRQ QHFWLRQWZLFHEHIRUH UHIXVHFRQQHFW [ UHIXVH  x ,  )   &21UHT [ FRQQHFW \  !  DFFHSW Z   ^ &21FQI ]  `   PHDQV WKDWLI6&3DFFHSWWRFRQQHFWZLWKWKHXSSHUOD\HUDWKLVUHTXHVWHG4R6 &21 UHT [ FRQQHFW \ DQGWKHORZHUOD\HUDFFHSWLWDWDJLYHQ4R6WKHQDFRQQHF WLRQPXVWEHUHDOL]HGEHWZHHQWKHWZROD\HUV x , ) ! DFFHSW [  ^&21FQI \ ` PHDQVWKDWLIWKHORZHUOD\HUDFFHSW WKHFRQQHFWLRQ DFFHSW [ WKLVFRQQHFWLRQPXVWEHUHDOL]HG &21FQI \  x , % ! DFFHSW [  ^&21FQI \ ` PHDQVWKDWDFRQQHFWLRQLVUHDO L]HG &21FQI [ RQO\LIWKHORZHUOD\HUDFFHSWHGLWEHIRUH DFFHSW \ 

1RWHWKDW, DQG, DUHIRUZDUGLQYDULDQWVZKLOH, DQG, DUHEDFNZDUGLQYDULDQWV,QGHILQL

WLRQRIWKHDERYHLQYDULDQWVZHKDYHXVHGDFRQWUROGULYHQDSSURDFKLHLQWKLVVWDJHSD UDPHWHUVRIWKHVLJQDOVDUHQRWLPSRUWDQWVRZHKDYHXVHGVRPHYDULDEOHVLQVWHDGRIWKHP )LQGLQJLQYDULDQWFRQVWUDLQWV

1RZZHDSSO\RXUPHWKRGRQWKHLQYDULDQWVWRILQGWKHLUFRUUHVSRQGLQJFRQVWUDLQWV7DEOH VKRZVWKHWUDFHRIWKHDOJRULWKPV)RUHDFKIRUZDUG EDFNZDUG LQYDULDQWWKHYDOXHRIWKHLQ

WHUPHGLDWHIRUZDUG EDFNZDUG PDWFKHUVHWLH8 LH9 DQGWKHXOWLPDWHFRQVWUDLQWVHWV&6 RYHUWKHYDULDEOHVRIWKHLQYDULDQWVKDYHEHHQVKRZQ 6HHWKHDOJRULWKPVLQILJXUHVDQG  &21UHT TRV

8SSHUOD\HU 121VXSSRUW 5HT4RV

&21FQI )LQ4RV RU&21FQI 

'DWD

5HVHW

6LPSOH&RQQHFWLRQ3URWRFRO FRQQHFW 5HT4RV

DFFHSW TRV RUUHIXVH

GDWD )LQ4RV

/RZHU/D\HU

)LJ ,QWHUDFWLRQVRIWKH6&3ZLWKLWVXSSHU DQGORZHUOD\HUV



DERUW

6 7U\&RXQW  5HT4RV  )LQ4RV 

6 ,2UHIXVH&21FQI  37U\&RXQW  $

,25HVHWDERUW 3 $

,2&21UHT TRV 121VX SSRUW 5HT4RV 3&21UHTTRV! $5HT4RV &21UHTTRV

,2&21UHT TRV FRQQHFW 5HT4RV 3&21UHTTRV  $5HT4RV &21UHTTRV

6

,2UHIXVHFRQQHFW 5HT4RV 37U\&RXQW   $7U\&RXQW 7U\&RXQW

,2DFFHSW TRV &21FQI )LQ4RV 3 $)LQ4RV PLQ DFFHSWTRV5HT4RV

6

,2'DWDGDWD )LQ4RV 3 $

)LJ ()60VSHFLILFDWLRQRI6&3

$SSO\LQJWKHDOJRULWKPVUHYHDOWKDWDOOWKHLQYDULDQWVDUHFRUUHFWUHJDUGLQJWKHFRQWUROSDUWRI

WKHVSHFLILFDWLRQEXWUHJDUGLQJWKHGDWDSDUWRIWKHVSHFLILFDWLRQWKHLQYDULDQWVDUHWUXHFRQGLWLRQ WRVRPHFRQVWUDLQWVZKLFKKDYHEHHQSURGXFHGE\WKHDOJRULWKPV)RULQYDULDQW,WKHUHLVQRW

DQ\FRQVWUDLQWRYHUWKHYDULDEOHRIWKHLQYDULDQWVRLWVKRXOGEHPDWFKHGE\H[HFXWLRQWUDFHV

ZLWK DQ\ YDOXH IRU WKH YDULDEOH [ )RU WKH RWKHU LQYDULDQWV RQO\ VXFK H[HFXWLRQ WUDFHV DUH PDWFKHGZLWKWKHLQYDULDQWVWKDWWKHYDOXHRIWKHLULQSXWRXWSXWSDUDPHWHUVGRHV QRWFDXVHV DQ\ FRQIOLFWZLWKWKHFRUUHVSRQGLQJFRQVWUDLQWVRIWKHLQWHQGHGLQYDULDQW 3DVVLYHWHVWLQJXVLQJWKHFRQVWUDLQHGLQYDULDQWV

1RZVXSSRVHWKDWWKHIROORZLQJH[HFXWLRQWUDFHVDUHJHQHUDWHGE\DIDXOW\LPSOHPHQ WDWLRQRIWKH6&3

x 7UDFH &21UHT  FRQQHFW  UHIXVH&21FQI  x 7UDFH &21UHT  FRQQHFW  DFFHSW  &21FQI  

7DEOH 8VLQJDOJRULWKPVWRH[WUDFWUHTXLUHGFRQVWUDLQWVIRUWKHH[DPSOHLQYDULDQWV ,QYDULDQW ,

%DFN ZDUG

,

)RU ZDUG

,

)RU ZDUG ,

%DFN ZDUG

9 T & G

8 RU9

^ VT&G ` ‡ ^7U\&RXQW ` ‡

8 ^ VT&G ` T ^&21UHTTRV[5HT4RV\` & ^&21UHTTRV  5HT4RV &21UHTTRV` G ‡

8 T & G 9 T & G

^ VT&G ` ^DFFHSWTRV[)LQ4RV\` ^\ PLQ [5HT4RV ` ‡ ^ VT&G ` ^DFFHSWTRV[)LQ4RV\` ^\ PLQ [5HT4RV ` ‡

8 RU9

&RQVWUDLQWVHW &6 &6B, &T ^7U\&RXQW `

9 ^ VT&G ` T ^5HT4RV[` G ^7U\&RXQW\` & &G‰ ^7U\&RXQW \ 7U\&RXQW` ^\ 7U\&RXQW \ 7U\&RXQW` ^7U\&RXQW ` 8 ^ VT&G ` &6B, &T T TR^DFFHSWTRVZ)LQ4RV]` ^[ \ [ ] PLQ Z\ ` ^&21UHTTRV[5HT4RV\ DFFHSWTRVZ)LQ4RV]` & &‰^&21UHTTRV PLQ DFFHSWTRV5HT4RV ` G ^&21UHTTRV  5HT4RV &21UHTTRV )LQ4RV PLQ DFFHSWTRV 5HT4RV ` G ‡ &6B, &T ^\ PLQ [5H T4RV ` &6B, &T ^\ PLQ [ 5H T4RV `

:HNQRZ WKDWDWUDQVLWLRQHUURUKDVEHHQRFFXUUHGLQWKHILUVWWUDFHEHFDXVHWKHVSHFL

ILFDWLRQIRUFHVWZRORRSVRQVWDWHV EHIRUHHYHQWXDOWUDQVLWLRQWRVFRUUHVSRQGLQJWR WKH WKUHH UHTXHVWV 6&3 PXVW GR EHIRUH IDLOLQJ WKH FRQQHFWLRQ ,Q WKLV WUDFH WKH FRQ

QHFWLRQLVVDLGWREHIDLOHGRQILUVWWU\)RUWKH VHFRQGWUDFHWKHUHLVDQRXWSXWHUURU

EHFDXVH WKH ILUVW ,2 FRXSOH VKRXOG EH &21UHT  FRQQHFW   :H FDQ LPDJLQH WKDW WKHWUDFHFRPHVIURPDQLPSOHPHQWDWLRQLQZKLFKWKHDFWLRQRQWKHWUDQVLWLRQIURPV

WRV LV5HT4RV &21UHTTRV±DQGWKHQVXFKDWUDFHLVSURGXFHG7KLVHUURUKDVIRU

FRQVHTXHQFH WR FRQQHFW WKH XSSHU DQG ORZHU OD\HUV ZLWK D 4R6 HTXDOV WR  ZKHQ LW FRXOGEH QRUPDOO\ HTXDOWR

7DEOHV  DQG  VKRZ WKH LQYDULDQWV XVHG LQ WKH FKHFNLQJ RI WKH ILUVW DQG WKH VHFRQG

WUDFHUHVSHFWLYHO\:HWU\ WRLGHQWLI\WKHFRQVWUDLQWV ZLWKWKH YDOXHVRIWKHYDULDEOHV H[WUDFWHGIURPWKHWUDFHV

x 7UDFH 6LQFHWKHDQDO\VLVIRXQG&21FQI  LQWKHWUDFHDQGIDLOHGORRNLQJIRU WKHFRXSOHUHIXVHFRQQHFW [ WKHQWKHWUDFHLVHUURQHRXVUHJDUGLQJWKHLQYDULDQW



,1RWHWKDWWKHIRXQGHUURULVFRQWUROGULYHQVRLWLVQRWQHHGHGWRORRNDWWKH FRQVWUDLQWVDWDOO x 7UDFH 7KHUHDUHWKUHHLQYDULDQWVZKLFKDUHFDQGLGDWHIRUWKLVWUDFH0DWFKLQJ WKHWUDFHZLWKWKHLQYDULDQWV VKRZVWKDWWKHUHLVQRWDQ\FRQWUROGULYHQHUURUVR ZHXVHFRQVWUDLQWVDQGWKHYDOXHRIWKHYDULDEOHVLQWKHWUDFHWRGHFLGHDERXWWKH SRVVLELOLW\RIGDWDGULYHQHUURUV &6, ‰ ^[ \ Z ] ` ^[ \ [] PLQ Z\ `‰ ^[ \ Z ] ` ^   PLQ  `

 LVWUXH LVIDOVHDQG PLQ  LVWUXHVRWKHLQYDULDQW, LVIDOVHRQ7UDFH &6, ‰ ^[ \ ` ^\ PLQ [5HT4RV `‰^[ \ ` ^ PLQ 5HT4RV `

LVWKHPLQLPXPRIDQG5HT4RVRQO\LI5HT4RVLVHTXDOWRVRWKHLQYDULDQW, LV WUXHRQWKHWUDFH7UDFH LI5HT4RV 

&6,‰^[ \ ` ^\ PLQ [5HT4RV `‰ ^[ \ ` ^ PLQ 5HT4RV `

LVWKHPLQLPXPRIDQG5HT4RVRQO\LI5HT4RVLVHTXDOWRVRWKHLQYDULDQW, LV WUXHRQWKHWUDFH7UDFH LI5HT4RV 

$VZHIRXQGDQLQFRQVLVWHQF\LQWKHFKHFNLQJRIWKHLQYDULDQW, ZLWKWKHWUDFH7UDFH

ZHFRQFOXGHWKDWWKHWUDFH7 LVIDOVH&KHFNLQJWKHRWKHULQYDULDQWVRQWKHWUDFHLVQRW QHFHVVDU\EXWILJXUHKHUHDVDQH[DPSOHRIYDULDEOHVLPSOLILFDWLRQ 7UDFH ,

7UDFH , ,

7DEOH 8VLQJLQYDULDQW, WRFKHFNWKHH[HFXWLRQWUDFH7UDFH 

&21UHT  5HIXVH

FRQQHFW  FRQQHFW [

5HIXVH 5HIXVH

&21FQI  &21FQI 

7DEOH 8VLQJLQYDULDQWV,, DQG, WRFKHFNWKHH[HFXWLRQWUDFH7UDFH 

&21UHT  &21UHT [

,

FRQQHFW  FRQQHFW \

$FFHSW 

&21FQI 

$FFHSW [

&21FQI \

$FFHSW Z $FFHSW [

&21FQI ]

&21FQI \

&RQFOXVLRQV 3DVVLYHWHVWLQJPHWKRGVIRUQHWZRUNSURWRFROVFDQEHFODVVLILHGLQWRQDwYHDQGLQYDUL DQW EDVHG DSSURDFKHV ,Q WKH QDwYH DSSURDFK WKH LPSOHPHQWDWLRQ WUDFH ZKLFK LV UH

FRUGHGGXULQJ WKH H[HFXWLRQ RI WKH SURWRFRO XQGHU WHVW LV FRPSDUHG ZLWK WRWDO RI WKH

VSHFLILFDWLRQLQDIRUZDUGRUEDFNZDUGPDQQHU7KLVLVZKHUHLQWKHLQYDULDQWEDVHG DSSURDFK RQO\ FULWLFDO SURSHUWLHV RI WKH VSHFLILFDWLRQ LH LQYDULDQWV  ZKLFK DUH H[

WUDFWHGE\DQH[SHUWDUHFRPSDUHGZLWKWKHLPSOHPHQWDWLRQWUDFH%\XVLQJLQYDULDQW EDVHGDSSURDFKQRWRQO\DORWRIH[WUDSURFHVVLQJLVUHGXFHGEXWDOVRZHFDQIRFXVRQ WKHFULWLFDOSURSHUWLHVRIWKHSURJUDPXQGHUWHVW

3DVVLYHWHVWLQJPHWKRGVFDQEHFRPSDUHGIURPDQRWKHUDVSHFW6RPHPHWKRGVDUHOLP

LWHGWRWHVWLQJRQO\FRQWUROGULYHQDVSHFWVRIWKHLPSOHPHQWDWLRQLHWKHRUGHURIRF

FXUUHQFHVRIWKHLQSXWRXWSXWVLJQDOVZKLOHRWKHU PHWKRGVDUHFDSDEOHRIWHVWLQJERWK FRQWUROGULYHQDQGGDWDGULYHQDVSHFWVRIWKHLPSOHPHQWDWLRQLHWKHYDOXHVRIWKHVLJ

QDO V SDUDPHWHUV )RU WHVWLQJ FRQWURO GULYHQ DVSHFWV LW LV VXIILFLHQW WR XVH )60 IRU VSHFLILFDWLRQZKLOHIRUGDWDGULYHQDVSHFWVLW LVQHHGHGWRXVH()60

,QWKLVSDSHUZHSUHVHQWHGDQHZPHWKRGIRUSDVVLYHWHVWLQJRIERWKFRQWUROGULYHQDQG GDWDGULYHQDVSHFWVRIWKH QHWZRUNSURWRFROVXVLQJDQLQYDULDQWEDVHGDSSURDFK7KH LQWHQGHG SURSHUWLHV RI WKH VSHFLILFDWLRQ DUH H[SUHVVHG XVLQJ VRPH FRQWURO GULYHQ LQ

YDULDQWVJLYHQE\DQH[SHUW$IWHUWKDWXVLQJWKHJLYHQDOJRULWKPVWKHLQYDULDQWVDUH FKHFNHG RQ WKH VSHFLILFDWLRQ RIIOLQH $OVR WR WDNH WKH GDWD GULYHQ DVSHFWV LQWR DF

FRXQWIRUWKHFRUUHFWLQYDULDQWVVRPHFRQVWUDLQWVRYHUWKHYDULDEOHVRIWKHLQYDULDQWV DUHH[WUDFWHG)RUSDVVLYHO\WHVWLQJWKHLPSOHPHQWDWLRQWUDFHVLWLVVXIILFLHQWWRFRP

SDUH RQOLQH WKH WUDFH ZLWK WKH LQYDULDQWV UHJDUGLQJ WKH FRQVWUDLQWV XVLQJ SDWWHUQ PDWFKLQJ$WUDFHLVFRUUHFWZKLOHLWLVPDWFKHGZLWKWKHLQYDULDQWDQGWKHLQYDULDQW V FRQVWUDLQWVDUHQRWFRQIOLFWHGUHJDUGLQJWKHYDOXHVRIWKHVLJQDO VSDUDPHWHUV

7RVKRZWKHDSSOLFDELOLW\RIWKHSUHVHQWHGPHWKRGSDVVLYHWHVWLQJRIWKH6LPSOH&RQ QHFWLRQ3URWRFRO 6&3 XVLQJWKHSUHVHQWHGPHWKRGZDVLOOXVWUDWHG

5HIHUHQFHV

 5/DL³$VXUYH\RIFRPPXQLFDWLRQSURWRFROWHVWLQJ´ -RXUQDORI6\VWHPVDQG6RIWZDUH       '/HHDQG0