treated as the meta data to represent their contents. The problem of content-based music data retrieval is then transformed into the string matching problem.
$Q $SSUR[LPDWH 6WULQJ 0DWFKLQJ $OJRULWKP IRU &RQWHQW%DVHG 0XVLF 'DWD 5HWULHYDO
!"# $ % -897,.9
,Q WKLV SDSHU DQ DSSURDFK IRU FRQWHQWEDVHG PXVLF GDWD UHWULHYDO LV SURSRVHG ,Q WKLV DSSURDFK WKHPDWLF IHDWXUH VWULQJV VXFK DV PHORG\ VWULQJV UK\WKP VWULQJV DQG FKRUG VWULQJV DUH H[WUDFWHG IURP WKH RULJLQDO PXVLF REMHFWV DQG WUHDWHG DV WKH PHWD GDWD WR UHSUHVHQW WKHLU FRQWHQWV 7KH SUREOHP RI FRQWHQWEDVHG PXVLF GDWD UHWULHYDO LV WKHQ WUDQVIRUPHG LQWR WKH VWULQJ PDWFKLQJ SUREOHP $ QHZ DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKP LV DOVR SURSRVHG ZKLFK SURYLGHV IDXOW WROHUDQFH DELOLW\ DFFRUGLQJ WR WKH PXVLF
FKDUDFWHULVWLFV
7R
VKRZ
WKH
HIILFLHQF\
RI
WKH
DOJRULWKP D VHW RI H[SHULPHQWV DUH SHUIRUPHG WR FRPSDUH ZLWK WKH DJUHS DQG WKH IJUHS XWLOLW\ RQ ERWK V\QWKHWLF DQG UHDO PXVLF GDWD
,QWURGXFWLRQ
$V WKH SURJUHVV RI QHWZRUN DQG GDWD FRPSUHVVLRQ WHFKQRORJLHV XVHUV FDQ HDVLO\ DFFHVV D ODUJH DPRXQW RI PXOWLPHGLD GDWD WKURXJK WKH LQWHUQHW ,W LV VWURQJO\ UHTXLUHG WR SURYLGH XVHUV WKH DELOLW\ WR UHWULHYH PXOWLPHGLD GDWD E\ FRQWHQW 5HFHQWO\ PDQ\ UHVHDUFKHUV IRFXV RQ WKH SUREOHP RI FRQWHQWEDVHG UHWULHYDO RI LPDJH GDWD >@>@>@ DQG YLGHR GDWD > @>@>@>@>@ +RZHYHU OHVV DWWHQWLRQ ZDV UHFHLYHG WR WKH DXGLR GDWD 7KH FRQYHQWLRQDO NH\ZRUG EDVHG UHWULHYDO RI DXGLR GDWD DWWDFKHV D VHW RI NH\ZRUGV WR WKH DXGLR REMHFWV XVHUV WKHQ SRVH TXHULHV DJDLQVW WKHVH NH\ZRUGV WR UHWULHYH WKH DXGLR REMHFWV (DFK DXGLR REMHFW LV WUHDWHG DV DQ RSDTXH ODUJH REMHFW DQG WKH LQIRUPDWLRQ ZLWKLQ WKH DXGLR REMHFW LV RPLWWHG 7KH UHVHDUFK LVVXHV KDYH WR EH DGGUHVVHG WR VXSSRUW FRQWHQWEDVHG UHWULHYDO RI DXGLR GDWD VXFK DV GHYHORSLQJ DQ LQGH[LQJ PHFKDQLVP WR H[WUDFW DXGLR IHDWXUHV IURP WKH DXGLR GDWD RUJDQL]LQJ WKH DXGLR IHDWXUHV ZLWK D VXLWDEOH GDWD VWUXFWXUH DQG GHVLJQLQJ DQ HIILFLHQW TXHU\ SURFHVVLQJ DOJRULWKP WR LQFUHDVH WKH SHUIRUPDQFH DQG SURYLGLQJ IULHQGO\ DXGLR TXHU\ LQWHUIDFH 7KH UHODWHG ZRUNV LQ WKH DXGLR GDWDEDVHV DUH VXUYH\HG LQ WKH IROORZLQJ :ROG HW DO > @ FODVVLILHG DXGLR REMHFWV DFFRUGLQJ WR WKHLU DFRXVWLFDO IHDWXUHV VXFK DV ORXGQHVV SLWFK DQG EDQGZLGWK DQG PDS WKHP LQWR IHDWXUH YHFWRUV LQ WKH IHDWXUH VSDFH *LYHQ DQ DXGLR TXHU\ H[DPSOH XVHUV FDQ ILQG WKH DXGLR REMHFWV ZLWK VLPLODU DFRXVWLFDO IHDWXUHV +RZHYHU DFRXVWLFDO IHDWXUHV DUH ORZ OHYHO DXGLR FKDUDFWHULVWLFV ZKRVH VHPDQWLFV LV GLIILFXOW WR XQGHUVWDQG IRU QRQH[SHULHQFHG XVHUV /RKU DQG 5DNRZ >@ GHYHORSHG IXQFWLRQV QHHGHG WR VWRUH DQG PDQLSXODWH DXGLR GDWD 0DQ\
XVHIXO RSHUDWLRQV IRU WKH DXGLR GDWD VXFK DV G\QDPLF FRPSUHVVLRQ DQG ORZSDVV ILOWHULQJ ZHUH SURYLGHG +RZHYHU WKH IXQFWLRQV QHHGHG WR LQGH[ DQG UHWULHYH DXGLR GDWD DUH QRW GLVFXVVHG 3UHYLRXV FRQWULEXWLRQV LQ WKH ILHOG RI PXVLF GDWDEDVHV DUH LQWURGXFHG DV IROORZV )RU WKH H[SODQDWLRQ RI WKH PXVLF WHUPLQRORJLHV XVHG LQ WKLV SDSHU SOHDVH UHIHU WR >@ *KLDV HW DO >@ SURSRVHG DQ DSSURDFK IRU PRGHOLQJ WKH FRQWHQW RI PXVLF REMHFWV ,Q WKLV DSSURDFK WKH PHORG\ RI D PXVLF REMHFW LV UHSUHVHQWHG DV D VHTXHQFH RI WKUHH OHWWHUV 6 VDPH SLWFK 8 SLWFK LQFUHDVHG DQG ' SLWFK GHFUHDVHG 7KH SUREOHP RI PXVLF TXHU\ SURFHVVLQJ LV WKHQ WUDQVIRUPHG LQWR WKH VXEVWULQJ PDWFKLQJ SUREOHP +RZHYHU WKLV UHSUHVHQWDWLRQ PHWKRG LV YHU\ URXJK DQG RWKHU LPSRUWDQW PXVLF IHDWXUHV DUH QRW LQFOXGHG 0RUHRYHU WKH VLPLODULW\ PHDVXUH RI WKHLU VXEVWULQJ PDWFKLQJ DOJRULWKP GRHV QRW WDNH PXVLF FKDUDFWHULVWLFV LQWR FRQVLGHUDWLRQ 3UDWKHU >@ SURSRVHG D OLQNHG OLVW GDWD VWUXFWXUH WR VWRUH WKH VFRUHV RI PXVLF GDWD %\ DSSO\LQJ KDUPRQLF DQDO\VLV PHWKRG WR WUDYHUVH WKH VFRUHV WKH FKRUG LQIRUPDWLRQ FDQ EH GHULYHG +RZHYHU KRZ WR SURYLGH XVHUV ZLWK D IULHQGO\ PXVLF TXHU\ LQWHUIDFH DQG DQ HIILFLHQW PXVLF TXHU\ SURFHVVLQJ WHFKQLTXH LV QRW GLVFXVVHG ,Q >@ ZH XVHG FKRUG WR UHSUHVHQW WKH FRQWHQW RI D PXVLF REMHFW (DFK PXVLF REMHFW LQ WKH GDWDEDVH DQG WKH PXVLF TXHU\ DUH WUDQVIRUPHG LQWR D VHTXHQFH RI FKRUGV 6LQFH XVHUV ZKR DUH QRW H[SHUW LQ PXVLF PD\ QRW EH DEOH WR SUHFLVHO\ VSHFLI\ PXVLF TXHULHV FHUWDLQ HUURUV VKRXOG EH DOORZHG 7KDW LV D IDXOW WROHUDQFH DELOLW\ KDV WR EH SURYLGHG ,Q RXU DSSURDFK WKH PXVLF TXHU\ FRQVLVWLQJ RI WKH KDUPRQLF QRWHV ZLWK FHUWDLQ HUURUV ZLOO EH WUDQVIRUPHG LQWR D FRUUHFW FKRUG WKH IDXOW WROHUDQFH DELOLW\ ZDV HTXLSSHG ,Q WKLV SDSHU ZH JHQHUDOL]H WKH QRWLRQ RI FKRUG WR WKHPDWLF IHDWXUHV DQG SURSRVH DQ HIILFLHQW DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKP ZKLFK WDNHV WKH PXVLF VLPLODULW\ PHDVXUH LQWR FRQVLGHUDWLRQ 7KH UHVW RI WKH SDSHU LV RUJDQL]HG DV IROORZV 6HFWLRQ SURYLGHV DQ RYHUYLHZ RI WKH SURSRVHG DSSURDFK 7KH SHUIRUPDQFH DQDO\VLV RI WKH DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKP LV SURYLGHG LQ 6HFWLRQ )LQDOO\ 6HFWLRQ FRQFOXGHV WKLV SDSHU
'XH WR WKH VSDFH OLPLWDWLRQ WKH IRUPDO DOJRULWKPV DUH VNLSSHG ,Q WKLV SDSHU ZH XVH D PXVLF TXHU\ H[DPSOH WR PRWLYDWH RXU DSSURDFK
$VVXPH WKHUH DUH WZR PXVLF REMHFWV 0 DQG 0 LQ WKH PXVLF GDWDEDVH ZKRVH PHORG\ VWULQJV DUH ¦VROPLPLIDUH UHGRUHPLIDVROVROVRO§ DQG ¦GRPLVROVROUHPLIDID GRUHUHPL§ 7KH PHORG\ VWULQJ RI WKH PXVLF TXHU\ 4 LV ¦GRUHPL§ 7R ILQG ZKHWKHU 0 DQG 0 FRQWDLQ WKH PHORG\ VWULQJ VXEVWULQJ PDWFKLQJ DOJRULWKP VKRXOG EH DSSOLHG )RU H[DFW VWULQJ PDWFKLQJ WKHUH DUH WZR ZHOO NQRZQ DOJRULWKPV WKH .03 DOJRULWKP >@ DQG WKH %R\HU0RRUH DOJRULWKP >@ %RWK WKHVH WZR DOJRULWKPV DUH RSWLPL]HG +RZHYHU WKH\ QHHG WR UHWULHYH DOO HQWLUH PHORG\ VWULQJV LQ WKH GDWDEDVH IRU SURFHVVLQJ 4 7KHUHIRUH WKH ILUVW UHTXLUHPHQW IRU GHYHORSLQJ D VWULQJ PDWFKLQJ DOJRULWKP LV WR UHGXFH WKH DPRXQW RI GDWD QHHGHG WR UHWULHYH IURP WKH GDWDEDVH 6LQFH ZH FDQQRW H[SHFW XVHUV WR SUHFLVHO\ VSHFLI\ PXVLF TXHU\ H[DPSOHV IDXOW WROHUDQFH DELOLW\ LV UHTXLUHG IRU FRQWHQWEDVHG PXVLF GDWD UHWULHYDO 7UDGLWLRQDO DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKPV >@>@>@ DUH GHVLJQHG IRU GRFXPHQW VHDUFKLQJ ZKLFK WDNH DV WKH VLPLODULW\ PHDVXUH +RZHYHU HGLWLQJ GLVWDQFH FDQQRW FRUUHFWO\ UHSUHVHQW WKH VLPLODULW\ FRQVLGHULQJ WKH PXVLF FKDUDFWHULVWLFV )RU H[DPSOH WKH VXEVWULQJV ¦GRPL§ ¦UH PL§ DQG ¦GRUH§ RI WKH PHORG\ VWULQJ ¦GRPLVROVROUH PLIDIDGRUHUHPL§ DOO DUH VLPLODU WR WKH PHORG\ VWULQJ ¦GRUHPL§ ZLWK HGLWLQJ GLVWDQFH RQH +RZHYHU DFFRUGLQJ WR WKH DQG WKH >@>@>@>@ ¦GR§ LV PRUH LPSRUWDQW DQG ¦UH§ LV OHVV LPSRUWDQW 7KHUHIRUH WKH VLPLODULW\ GHJUHH VKRXOG EH ¦GR PL§ ! ¦GRUH§ ! ¦UHPL§ 7KXV WKH VHFRQG UHTXLUHPHQW LV WKDW WKH DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKP VKRXOG DOVR SURYLGH WR EH GLVFXVVHG LQ VXEVHFWLRQV DQG 7R VDWLVI\ WKH ILUVW UHTXLUHPHQW ZH WU\ WR UHWULHYH RQO\ WKH QRWHV LQYROYHG LQ WKH TXHU\ VWULQJ IURP WKH PHORG\ VWULQJV 7R DFKLHYH WKLV ZH ILUVW VWRUH SRVLWLRQV RI WKH VDPH QRWHV LQ D PHORG\ VWULQJ LQWR OLQNHG OLVWV DV VKRZQ LQ )LJXUH D (DFK HOHPHQW LQ WKH OLQNHG OLVWV LV RI WKH IRUP ZKLFK GHQRWHV WKH WK QRWH RI WKH PHORG\ VWULQJ RI WKH WK PXVLF REMHFW LQ WKH GDWDEDVH 7KH QXPEHU RI WKH OLQNHG OLVWV LV WKH VDPH DV WKH VL]H RI WKH DOSKDEHW RI WKH PHORG\ VWULQJV ZKLFK LV DVVXPHG VHYHQ ¦GR§ ¦UH§ ¦PL§ ¦ID§ ¦VRO§ ¦OD§ DQG ¦VL§ 7KH WKUHH OLQNHG OLVWV FRUUHVSRQGLQJ WR §GR§ ¦UH§ DQG ¦PL§ LQYROYHG LQ 4 DUH UHWULHYHG ZLWK WZR GXPP\ QRGHV DQG DV VKRZQ LQ )LJXUH E 7KH RUGHU RI WKH WKUHH OLQNHG OLVWV LV WKH VDPH DV WKH VHTXHQFH RI WKHLU FRUUHVSRQGLQJ V\PEROV DSSHDULQJ LQ 4 ,Q WKLV FDVH RQO\ QRGHV RXW RI WKH WRWDO QRGHV QHHG WR EH UHWULHYHG ,Q JHQHUDO RQO\ QRGHV QHHG WR EH UHWULHYHG ZKHUH LV WKH QXPEHU RI QRGHV LQ WKH PHORG\ VWULQJ RI WKH TXHU\ LV WKH WRWDO QXPEHU RI QRGHV RI DOO PHORG\ VWULQJV LQ WKH GDWDEDVH DQG LV WKH DOSKDEHW VL]H RI WKH PHORG\ VWULQJV 7R VHDUFK IRU WKH VXEVWULQJV RI WKH PHORG\ VWULQJV ZKLFK PDWFK WKH TXHU\ VWULQJ ¦GRUHPL§ ZH ILQG WKH VXEVWULQJV RI WKH PHORG\ VWULQJV ¦GRUH§ DQG ¦UHPL§ DQG PHUJH WKHVH VXEVWULQJV WR ILQG WKH RFFXUUHQFHV RI WKH
VXEVWULQJV ¦GRUHPL§ :KHQ ZH PHUJH WKH VXEVWULQJV ¦GR UH§ DQG ¦UHPL§ LQWR ¦GRUHPL§ ZH KDYH WR PDNH VXUH WKDW WKH ¦UH©V§ LQ ERWK VXEVWULQJV DUH WKH VDPH RQH LQ WKH VDPH PHORG\ VWULQJ LV GRQH E\ DWWDFKLQJ DQ EHWZHHQ WZR HOHPHQWV LQ WKH UHWULHYHG OLQNHG OLVW ZKRVH FRUUHVSRQGLQJ QRWHV LQ WKH IHDWXUH VWULQJ IRUP VXFK D VXEVWULQJ ¦GRUH§ DQG ¦UHPL§ LV GRQH E\ WUDYHUVLQJ WKH GLUHFWHG JUDSK IRUPHG E\ WKHVH H[DFW OLQNV WR JHQHUDWH WKH SDWKV ZKRVH FRUUHVSRQGLQJ VXEVWULQJV PDWFK ¦GRUHPL§ 7KH UXOHV IRU FRQVWUXFWLQJ WKH H[DFW OLQNV DQG WKHLU H[WHQVLRQV IRU WKH DSSUR[LPDWH VWULQJ PDWFKLQJ ZLOO EH LOOXVWUDWHG LQ WKH IROORZLQJ
7KH H[DFW VWULQJ PDWFKLQJ DOJRULWKP JRHV DV IROORZV /HW SRV QL UHSUHVHQW WKH SRVLWLRQ RI QL LQ WKH UHODWHG PHORG\ VWULQJ )RU HDFK SDLU RI QRGHV Q Q WDNHQ IURP WZR DGMDFHQW OLQNHG OLVWV RI WKH VDPH PHORG\ VWULQJ LI SRV Q SRV Q ZKLFK PHDQV ¦QQ§ LV D VXEVWULQJ RI WKH PHORG\ VWULQJ ZH EXLOG DQ IRUP Q WR Q DV VKRZQ LQ )LJXUH F :H DOVR EXLOG DQ H[DFW OLQN IURP WKH QRGH WR Q LI Q KDV DQ RXWJRLQJ H[DFW OLQN DQG IURP Q WR WKH QRGH LI Q KDV DQ LQFRPLQJ H[DFW OLQN DV VKRZQ LQ )LJXUH G 7KH PXVLF VLPLODULW\ PHDVXUH FRUUHVSRQGLQJ WR DQ H[DFW OLQN LV VHW WR RQH VLQFH WKH FRUUHVSRQGLQJ VXEVWULQJ LV WKH VDPH DV WKH FRUUHVSRQGLQJ VXEVWULQJ RI WKH TXHU\ %\ WUDYHUVLQJ WKH H[DFW OLQNV IURP WKH QRGH WR WKH QRGH ZH ILQG WKH TXHU\ PDWFKHV WKH VXEVWULQJ VWDUWHG IURP WKH WK SRVLWLRQ WR WKH WK SRVLWLRQ RI WKH PHORG\ VWULQJ RI WKH PXVLF REMHFW 0 +RZHYHU 0 GRHV QRW FRQWDLQ DQ\ VXEVWULQJ ZKLFK PDWFKHV WKH TXHU\
L
L
L
L
7R SURYLGH IDXOW WROHUDQFH DELOLW\ WKUHH NLQGV RI HUURUV GURSRXW HUURUV LQVHUWLRQ HUURUV DQG WUDQVSRVLWLRQ HUURUV VKRXOG EH DOORZHG )RU H[DPSOH IRU WKH TXHU\ VWULQJ ¦GRUHPL§ WKH PHORG\ VWULQJV ¦GRUH§ ¦UHPL§ DQG ¦GR PL§ DSSUR[LPDWHO\ PDWFK LW ZLWK RQH GURSRXW HUURU WKH PHORG\ VWULQJ ¦GRIDUHPL§ DQG ¦GRUHVROPL§ DSSUR[LPDWHO\ PDWFK LW ZLWK RQH LQVHUWLRQ HUURU DQG WKH PHORG\ VWULQJV ¦GRUH"§ ¦"UHPL§ DQG ¦GR"PL§ DSSUR[LPDWHO\ PDWFK LW ZLWK RQH WUDQVSRVLWLRQ HUURU +RZHYHU DOWKRXJK WKH HGLWLQJ GLVWDQFH RI WKHVH PHORG\ VWULQJV DUH WKH VDPH WKHLU PXVLF VLPLODULW\ PHDVXUHV PD\ EH GLIIHUHQW :H XVH WKH IXQFWLRQ VLP Q WR GHQRWH WKH VLPLODULW\ PHDVXUH ZKHQ ¦Q§ LV GURSSHG RXW LQVHUWHG RU WUDQVSRVHG IURP WKH PHORG\ VWULQJ 7KH FRPSXWDWLRQ RI VLP Q ZLOO EH GLVFXVVHG LQ WKH QH[W VXEVHFWLRQ 7KH ILUVW SURFHGXUH RI WKH DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKP LV WKH VDPH DV WKH H[DFW RQH 7KH QH[W VWHS LV WR ILQG WKH SRVVLEOH GURSRXW HUURUV :H QRWH WKDW WKH YDOXH RI WKH VHFRQG HOHPHQW LQ WKH ¦GR§ OLQNHG OLVW LV DQG WKH YDOXH RI WKH IRXUWK HOHPHQW LQ WKH ¦PL§ OLQNHG OLVW LV ZKLFK PHDQV WKHUH LV D VXEVWULQJ ¦GRPL§ LQ WKH PXVLF REMHFW 0 $ GURSRXW HUURU RFFXUV DQG ZH EXLOG D GURSRXW
GR
UH
PL
ID
VRO
OD
VL
GR VWDUW
UH
PL
GR
UH
PL
GR
UH
PL
VWDUW
HQG
SRLQWHU LQ OLQNHG OLVW
(a)
HQG
VWDUW
(b)
H[DFW OLQN
(c)
HQG
(d)
Figure 1: Exact string matching example.
VWDUW
GR
UH
PL
GR
UH
PL
GR
UH
PL
GR
UH
PL
H[DFW OLQN
HQG
GURSRXW OLQN
VWDUW
HQG
WUDQVSRVLWLRQ LQVHUWLRQ OLQN OLQN
VWDUW
H[DFW OLQN
HQG
GURSRXW OLQN
VWDUW
HQG
WUDQVSRVLWLRQ LQVHUWLRQ OLQN OLQN
Figure 2: Approximate string matching example. OLQN EHWZHHQ WKHVH WZR QRGHV WR GHQRWH LW 7KH UHVXOW LV
! "
#
$%& # " $% ' ( ) *
+ " !
! "
# $, ( ) # $- ' ( ) *
+
' ' ' ! ! # . . # ! . / !
" "
*
& %
. 0 "
" " . 1 ) 2 # )
!
#
3 # # 4% 54%654%754 /54 85
" 9 ! 2 &% 2 &7 2 &, 2 &- 2 &/ 2 &7 2 &8 !
: " ; ; ; 2 ; % " ! . < =N N !N : =N N !N : # : 9
0/9
G
L
W
Σ VLP ' Σ VLP , Σ VLP 7
N
N
N
3
N
N
N
7KH IRUPXOD LV H[SODLQHG DV IROORZV 7KH VLPLODULW\ PHDVXUH RI WKH SDWK 3 LV WKH VXPPDWLRQ RI WKH VLPLODULW\ PHDVXUHV RI WKH IRXU NLQGV RI OLQNV LQ 3 ZKLFK HTXDOV ÷ ü +RZHYHU ø + ∑ ü » × þ ¼ + ∑ ü » Ü þ ¼ + ∑ ü »ç þ ¼
þ =Ä
þ =Ä
þ =Ä
VLQFH D GURSRXW OLQN DQG D WUDQVSRVLWLRQ OLQN VNLS D V\PERO LQ WKH PXVLF TXHU\ VWULQJ IRU D SDWK LQFOXGLQJ G GURSRXW OLQNV DQG W WUDQVSRVLWLRQ OLQNV WKHUH DUH G W V\PEROV VNLSSHG :H QHHG WR DGG G W WR WKH VLPLODULW\ PHDVXUH )XUWKHU VLQFH H G L W Q ZH VKRXOG VXEWUDFW RQH IURP WKH VLPLODULW\ PHDVXUH )LQDOO\ WKH VLPLODULW\ PHDVXUH LV GLYLGHG E\ Q WR QRUPDOL]H LWV YDOXH LQWR WKH UDQJH EHWZHHQ WR )RU H[DPSOH WKH PXVLF VLPLODULW\ PHDVXUH RI WKH ILUVW SDWK LQ )LJXUH LV VLP 3 − WKH PXVLF VLPLODULW\ PHDVXUH RI WKH VHFRQG SDWK LV
VLP 3 − ZKHUH VLP ¦UH§ DQG WKH PXVLF VLPLODULW\ PHDVXUH RI WKH WKLUG SDWK LV VLP 3 − ZKHUH VLP ¦GR§
7R VKRZ WKH HIILFLHQF\ DQG IHDVLELOLW\ RI RXU H[DFWDSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKPV ZH SHUIRUP D VHULHV RI H[SHULPHQWV DQG FRPSDUH RXU DSSURDFK ZLWK WKH DJUHS DQG WKH IJUHS 7KH UHDVRQ IRU FKRRVLQJ IJUHS DV WKH FRPSDULQJ WDUJHW LV WKDW IJUHS LV WKH IDVW DQG VWDQGDUG VWULQJ PDWFKLQJ DOJRULWKP ZKLFK LV ZLGHO\ DYDLODEOH RQ WKH 81,; V\VWHPV $QG DJUHS LV D QHZ WRRO IRU WH[W VHDUFKLQJ DOORZLQJ HUURUV >@ 6LQFH DJUHS DQG IJUHS FDQQRW SHUIRUP DSSUR[LPDWH VWULQJ PDWFKLQJ ZLWK PXVLF VLPLODULW\ PHDVXUHV IRU DSSUR[LPDWH VWULQJ PDWFKLQJ FDVHV ZH RQO\ FRPSDUH ZLWK RXU H[DFW VWULQJ PDWFKLQJ DOJRULWKP 7KHUH DUH WZR UXQQLQJ GDWD VHWV 2QH VHW RI PXVLF REMHFWV LV JHQHUDWHG UDQGRPO\ 7KH RWKHU VHW RI PXVLF REMHFWV FRQWDLQ UHDO PXVLF REMHFWV RI 0,', IRUP >@ $OO PXVLF REMHFWV DUH SDUVHG WR H[WUDFW WKHLU PHORG\ VWULQJV DQG UK\WKP VWULQJV $OO H[SHULPHQWV DUH SHUIRUPHG RQ WKH PHORG\ VWULQJV
)LJXUH DQG )LJXUH LOOXVWUDWH WKH H[HFXWLRQ WLPH YHUVXV WKH OHQJWK RI WKH TXHU\ VWULQJ IRU V\QWKHWLF PXVLF GDWD VHW DQG UHDO PXVLF GDWD VHW UHVSHFWLYHO\ ,Q ERWK FDVHV WKH H[HFXWLRQ WLPH RI IJUHS LV DERXW WKH VDPH DQG WKH H[HFXWLRQ WLPH RI DJUHS JURZV UDSLGO\ DV TXHU\ VWULQJ OHQJWK LV PRUH WKDQ VKRZQ LQ )LJXUH DQG )LJXUH 7KH WLPH FRPSOH[LW\ RI IJUHS LV 2 PQ ZKHUH P LV WKH OHQJWK RI WKH TXHU\ VWULQJ DQG Q LV WKH OHQJWK RI WKH IHDWXUH VWULQJ > @ ,Q WKLV H[SHULPHQW VLQFH WKH OHQJWK RI IHDWXUH VWULQJV JUHDWHU WKDQ LV PXFK ODUJHU WKDQ WKH OHQJWK RI TXHU\ VWULQJV OHVV WKDQ WKH SHUIRUPDQFH RI IJUHS GHSHQGV RQ WKH OHQJWK RI IHDWXUH VWULQJV 2Q WKH RWKHU KDQG WKH WLPH FRPSOH[LW\ RI RXU VWULQJ PDWFKLQJ DOJRULWKP LV 2 PQ$ WKH H[HFXWLRQ WLPH RI RXU DSSURDFK OLQHDUO\ LQFUHDVHV ZLWK WKH OHQJWK RI WKH TXHU\ VWULQJ 0RUHRYHU VLQFH RXU DOJRULWKP RQO\ UHWULHYHV WKH OLQNHG OLVWV LQYROYHG LQ WKH TXHU\ VWULQJ IURP WKH PXVLF LQGH[ LH Q$ LV PXFK OHVV WKDQ RQH EHWWHU SHUIRUPDQFH LV DFKLHYHG WKDQ WKDW RI IJUHS $V ZH KDYH DVVXPHG Q ZLOO QRW EH JUHDWHU WKDQ +RZHYHU ZKHQ Q LV ODUJHU WKDQ $ DOO OLQNHG OLVWV KDYH WR EH UHWULHYHG IRU SURFHVVLQJ DQG WKH SHUIRUPDQFH RI RXU DOJRULWKP ZLOO EH ZRUVH WKDQ WKDW RI WKH XWLOLW\ IJUHS
,Q WKLV JURXS RI H[SHULPHQWV ZH GLVFXVV WKH DSSUR[LPDWH VWULQJ PDWFKLQJ FDVHV 7KHUH DUH WKUHH FDVHV WKH GURSRXW WUDQVSRVLWLRQ DQG LQVHUWLRQ FDVHV )RU WKH H[SHULPHQW IRU VKRZLQJ WKH HIIHFWV RI WKH OHQJWK RI TXHU\ VWULQJV ZH DVVXPH WKDW WKH QXPEHU RI WKH PXVLF REMHFWV LV WKH DYHUDJH OHQJWK RI WKH IHDWXUH VWULQJV RI WKH PXVLF REMHFWV LV IRU UHDO PXVLF GDWD VHW 7KH SHUIRUPDQFHV RI
ERWK H[SHULPHQWV DUH PHDVXUHG E\ WKH DYHUDJH H[HFXWLRQ WLPH RI PXVLF TXHULHV )LJXUH VKRZ WKH H[SHULPHQW UHVXOWV 7KH H[HFXWLRQ WLPHV IRU WKH DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKPV DUH PRUH WKDQ WKDW RI WKH H[DFW VWULQJ PDWFKLQJ VLQFH WKH FRQVWUXFWLRQ RI WKH GURSRXW WUDQVSRVLWLRQ DQG LQVHUWLRQ OLQNV QHHGV H[WUD WLPH +RZHYHU WKH WLPH FRPSOH[LWLHV RI WKH H[DFW VWULQJ PDWFKLQJ DOJRULWKP DQG WKH DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKPV DUH WKH VDPH
,Q WKLV SDSHU ZH SURSRVH DQ DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKP IRU FRQWHQWEDVHG PXVLF GDWD UHWULHYDO ,Q RXU DSSURDFK ZH ILUVW H[WUDFW PXVLF IHDWXUH VWULQJV VXFK DV PHORG\ VWULQJV UK\WKP VWULQJV DQG FKRUG VWULQJV IURP WKH RULJLQDO PXVLF REMHFWV DQG VWRUH WKHP LQ WKH OLQNHG OLVW IRUP 7KHVH IHDWXUH VWULQJV DUH XVHG DV WKH PXVLF LQGH[ IRU FRQWHQWEDVHG PXVLF GDWD UHWULHYDO $ QHZ DSSUR[LPDWH VWULQJ PDWFKLQJ DOJRULWKP LV WKHQ SURSRVHG WR PDWFK WKH IHDWXUH VWULQJV RI PXVLF TXHULHV ZLWK WKH IHDWXUH VWULQJV RI WKH PXVLF REMHFWV LQ WKH PXVLF GDWDEDVH ,Q RXU DOJRULWKP RQO\ WKH OLQNHG OLVWV LQYROYHG LQ WKH PXVLF TXHU\ DUH QHHGHG WR EH UHWULHYHG IRU SURFHVVLQJ )XUWKHUPRUH RXU DOJRULWKP SURYLGHV IDXOW WROHUDQFH DELOLW\ DQG XVHV WKH PXVLF VLPLODULW\ PHDVXUH WR FRPSXWH WKH VLPLODULW\ RI WZR IHDWXUH VWULQJV )XWXUH UHVHDUFK LQFOXGHV WKH IROORZLQJ LVVXHV )LUVW WKH GURSRXW OLQNV WKH LQVHUWLRQ OLQNV DQG WKH WUDQVSRVLWLRQ OLQNV SURSRVHG LQ WKLV SDSHU FDQ RQO\ UHSUHVHQW HUURUV ZLWK HGLWLQJ GLVWDQFH RQH :H VKDOO FRQVLGHU HUURUV ZLWK KLJKHU HGLWLQJ GLVWDQFHV 6HFRQG WR VKRZ WKH SUDFWLFDO HIIHFWLYHQHVV RI RXU DOJRULWKPV DQ H[WHQVLYH SHUIRUPDQFH HYDOXDWLRQ RI WKH 0XVH V\VWHP LV EHLQJ FDUULHG RXW
()* (!* (#* ('* (* (&* ( *
!!!"!#$ %&' +", - . / . 0 1 2 3 4 !' $ 5 %%) &#" ) 6 - 7 8 9 )$ 5 %&& : 2 0 " ;9 ; 34
' )"'%! %%# : 2 4 %%' )! ")!
(%* /C @ - - :9 0 B: 6 >4 " %# %%# ( $* /: 7 6 94 # $% ' # )%" #! %%! ())* = . 5: 0 6 - ; 3 - @ ;9 6 4 '$#"') %%! ()!* = 7 - # H / 8
9 96 %%) ()#* = 0/9 6 9 9: - : 4 !% ) > %% % D )!%")'' / F ()'* 9 / J 0 " ; 3 - @4 ) )"&) %%# () * 0:9 ; 3 " - @ ;9 4 #%"' %%' ()&* 29 # 0 !" *0 # "1 " 79 = % ' () * H - >: > A . 0 9 1 4 )")' %%' ()%* H 0 " - @ :4 ! ! 8 %% )&"! (!$* H: G 08 2 3 1 4 !' $ 5 %%) !"%
¦ G R UH P L§
V WD U W
V WD U W
HQG
¦ G R P L§
V WD U W
HQG
¦UHP L§
V WD U W
HQG
¦VR OUHP L§
V WD U W
HQG
¦ G R UH §
V WD U W
HQG
¦ G R UH UH §
V WD U W
HQG
¦UHP L§
V WD U W
HQG
¦ U H U H P L§
V WD U W
HQG
¦ G R UH UH P L§
V WD U W
HQG
¦ G R UH UH P L§
HQG
Figure 3: Approximate string matching results. fgrep_with_500obj fgrep_with_200obj agrep_with_500obj agrep_with_200obj ours_with_500obj ours_with_200obj
1 time (sec.)
0.8 0.6 0.4 0.2 0 2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
length of the query string
Figure 4: Execution time vs. length of the query string (synthetic music data set). fgrep_with_80obj fgrep_with_40obj agrep_with_80obj agrep_with_40obj ours_with_80obj ours_with_40obj
0.25 time (sec.)
0.2 0.15 0.1 0.05 0 2
3
4
5
6
7
8
9 10 11 12 13 length of the query string
14
15
16
17
18
19
20
Figure 5: Execution time vs. length of the query string (real music data set).
time (sec.)
1 exact dropout transpotion insert
0.8 0.6 0.4 0.2 0 2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
length of the query string
Figure 6: Execution time vs. length of the query string (real music data set).
20
