rhythm deciphering and fine-grained Salsa beat tracking to assist. Salsa dancers with beat counting voice/vibration prompts. The proposed system can be used ...
2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems
6DOVD$VVW %HDW &RXQWLQJ 6\VWHP (PSRZHUHG E\ 0RELOH 'HYLFHV WR $VVLVW 6DOVD 'DQFHUV @ >@ >@ >@ >@ KDYH DOUHDG\ EHHQ FRQGXFWHG RQ WUDFNLQJ EHDWV DXWRPDWLFDOO\ E\ DQDO\]LQJ PXVLF DFRXVWLF VLJQDOV +RZHYHU EHFDXVH RI WKH LUUHJXODU 6DOVD UK\WKPLF SDWWHUQV PDGH E\ GLIIHUHQW /DWLQ LQVWUXPHQWV VLPXOWDQHRXVO\ WKH EHDWV WKHVH DSSURDFKHV GHWHFWHG DUH XVXDOO\ QRW DW WKH FRUUHFW PHWULFDO OHYHO RI 6DOVD EHDWV LH )RU LQVWDQFH WKH PHWULFDO OHYHOV RI WKHVH DSSURDFKHV GHWHFWHG PD\ EH DW KDOI PHWULFDO OHYHO RI 6DOVD EHDWV LH RU RU DW WZLFH PHWULFDO OHYHOV RI 6DOVD EHDWV LH 7KDW LQGLFDWHV WKDW WKHVH DSSURDFKHV ZRXOG OHDG WR D PDVV RI RYHUXQGHU GHWHFWLRQV IRU WUDFNLQJ 6DOVD EHDWV ,Q DGGLWLRQ DQRWKHU GLUHFWLRQ LV XVLQJ WKH PRWLRQ VHQVLQJ GHYLFH HJ 0RWLRQ $QDO\VLV )DOFRQ >@ RU .LQHFW >@ WR REWDLQ WKH ' PRWLRQ RI D GDQFH PDVWHU DQG DQDO\]H WKH
, , 1752'8&7,21 'DQFLQJ LV QRW RQO\ D SHUIRUPDQFH DUW VKRZHG RQ WKH VWDJH E\ SURIHVVLRQDO GDQFHUV EXW D QRUPDO SK\VLFDO H[HUFLVHWKHUDS\ DQG VRFLDO DFWLYLW\ IRU WKH JHQHUDO SXEOLF ZKLFK SOD\V DQ LP SRUWDQW VRFLDO UROH QRZDGD\V ,Q HYHU\ KXPDQ FXOWXUH SHRSOH XVXDOO\ KDYH VRPH NLQGV RI PXVLF ZLWK D UHJXODU EHDW WKDW HOLFLWV V\QFKURQLVHG ERG\ PRYHPHQW LH GDQFLQJ +RZHYHU GDQFLQJ LV DOZD\V GLI¿FXOW HVSHFLDOO\ IRU EHJLQQHUV GXH WR WKHLU VHOIFRQVFLRXVQHVV RU ODFNLQJ VHQVH RI UK\WKP ,PSRUWDQWO\ EHDW SHUFHSWLRQ DQG V\QFKURQLVDWLRQ DUH PHQWDOO\ GHPDQGLQJ IRU SHRSOH ZKR XQOLNH PXVLFLDQV KDYH OLWWOH SULRU H[SHULHQFH LQ EHDW RU UK\WKP GHWHFWLRQ >@ 6DOVD LV D SRSXODU IRUP RI VRFLDO GDQFHV ZLWK RULJLQV LQ &XED D PDMRU FURVVURDG RI 6SDQLVK (XURSHDQ DQG $IULFDQ FXOWXUHV 'LIIHUHQW IURP RWKHU W\SHV RI GDQFHV ¿QGLQJ EHDWV LQ 6DOVD PXVLF LV HYHQ WRXJKHU GXH WR WKH LQWULFDFLHV RI 6DOVD PXVLF 8QOLNH RWKHU W\SHV RI GDQFH PXVLF ZKLFK XVXDOO\ KDYH RQH GRPLQDQW UK\WKP HJ EDVV RU GUXP 6DOVD PXVLF KDV PDQ\ RYHUODSSHG UK\WKPLF SDWWHUQV SURYLGHG E\ GLIIHUHQW /DWLQ LQVWUXPHQWV HJ &ODYH VWLFNV &RQJD GUXPV 7LPEDOH GUXPV SOD\LQJ DW WKH VDPH WLPH +HQFH LW LV FKDOOHQJLQJ WR ¿QG 6DOVD EHDWV IURP WKHVH RYHUODSSHG UK\WKPLF SDWWHUQV MXVW E\ 2155-6814/17 $31.00 © 2017 IEEE DOI 10.1109/MASS.2017.25
81
PRWLRQ GDWD WR JHW WKH WLPLQJV RI PRYLQJ KLVKHU VWHSV DQG WKHQ UHPLQG XVHUV RI WKHVH WLPLQJV XVLQJ YLEUDWLRQ RU FRXQWLQJ YRLFH +RZHYHU WKHVH DSSURDFKHV KDYH WR LQYROYH SURIHVVLRQDO GDQFHUV DQG QHHG GHGLFDWHG PRWLRQ VHQVRUV ZKLFK UHVWULFWV LWV XVDJH VFHQDULRV DQG UHTXLUHV DGGLWLRQDO FRVW 7R DGGUHVV WKHVH LVVXHV ZH LPSOHPHQW D PRELOH GHYLFH EDVHG EHDW FRXQWLQJ V\VWHP SalsaAsst WR DVVLVW 6DOVD GDQFHUV ZLWK DVVLVWHGFRXQWLQJ YRLFHYLEUDWLRQ SURPSWV YLD PRELOH GH YLFHV HJ VPDUWSKRQHV 7KH SURSRVHG SalsaAsst FDQ SHUIRUP UK\WKP GHFLSKHULQJ DQG ¿QHJUDLQHG EHDW WUDFNLQJ QRW RQO\ KHOSLQJ 6DOVD EHJLQQHUV JHW DFFHOHUDWHG OHDUQLQJ SURFHVV GXULQJ SUDFWLFH DW KRPH DQG EH PRUH FRQ¿GHQW RQ GDQFH ÀRRU EXW DOVR HOLPLQDWLQJ SURIHVVLRQDO GDQFHUV¶ HUURUV GXULQJ WKHLU OLYH SHUIRUPDQFH )LJXUH LV DQ LOOXVWUDWLRQ RQ KRZ SalsaAsst DVVLVWV GDQFHUV GXULQJ WKHLU SUDFWLFH VHVVLRQV RU RQ WKH VWDJH 6SHFL¿FDOO\ SalsaAsst LV DEOH WR SURFHVV 6DOVD PXVLF ¿OHV RIÀLQH DQG JHQHUDWH D QHZ PXVLF ¿OH ZKLFK KDV ERWK WKH RULJLQDO 6DOVD PXVLF DQG LWV V\QFKURQRXV EHDW FRXQWLQJ YRLFH :LWK WKH KHOS RI WKH QHZ PXVLF ¿OH ZKLFK FDQ EH SOD\HG WKURXJK WKH %OXHWRRWK VSHDNHU SHRSOH PD\ IHHO PXFK HDVLHU WR OHDUQ 6DOVD GDQFH 0HDQZKLOH SalsaAsst LV DOVR DEOH WR VHQVH WKH 6DOVD PXVLF YLD WKH EXLOWLQ PLFURSKRQH DQG JHQHUDWH UHDO WLPH DVVLVWHGFRXQWLQJ YRLFH WKURXJK KHDGSKRQHV RU YLEUDWLRQ SURPSWV WR UHPLQG WKH GDQFHUV RI 6DOVD EHDW WLPLQJV 0RUH VSHFL¿FDOO\ RXU V\VWHP RQO\ XVHV 6DOVD PXVLF DXGLR VLJQDO IRU GHWHFWLQJ WKH 6DOVD EHDWV 7KH RQVHW GHWHFWLRQ DOJRULWKP ¿VW DQDO\]HV WKH DXGLR VLJQDO LQ IUHTXHQF\ GRPDLQ XVLQJ 6KRUWWLPH )RXULHU 7UDQVIRUP 67)7 WR REWDLQ WKH VSHFWURJUDP DQG WKHQ FRPSXWHV 2QVHW 6WUHQJWK &XUYH 26& %\ ORFDWLQJ WKH ORFDO PD[LPD RI VPRRWKHG 26& ZH FDQ REWDLQ WKH RQVHW SRVLWLRQV 7R GHWHFW 6DOVD EHDWV RXU DOJRULWKP ¿UVW HVWLPDWHV WKH 6DOVD EHDW LQWHUYDO XVLQJ RQVHW SRVLWLRQV DQG WKHQ WUDFN WKH EHDWV IURP RQVHWV EDVHG RQ WKH HVWLPDWHG 6DOVD EHDW LQWHUYDO :H GHYHORS WZR DOWHUQDWLYH PRGHV OfÀine Beat Counting Mode DQG Online Beat Counting Mode LQ RXU V\VWHP IRU ÀH[LEOH XVDJH 7KH RIÀLQH PRGH FDQ SURFHVV DQ\ 6DOVD GDQFH VRQJ ¿OHV WR JHQHUDWH WKH EHDW FRXQWLQJ YRLFHYLEUDWLRQ SURPSWV DORQJ ZLWK WKH RULJLQDO 6DOVD PXVLF 7KH RQOLQH PRGH OHYHUDJHV WKH EXLOWLQ PLFURSKRQH RQ PRELOH GHYLFHV WR UHFHLYH WKH DFRXVWLF VLJQDO RI 6DOVD PXVLF DQG JHQHUDWH WKH UHDOWLPH SURPSWV LH EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ 7KH PDLQ FRQWULEXWLRQV RI RXU ZRUN DUH VXPPDUL]HG DV IROORZV •
•
•
•
([WHQVLYH WHVWV XVLQJ 40 6DOVD GDQFH VRQJV XQGHU 8 HYDOXDWLRQ PHWULFV VKRZ WKDW RXU V\VWHP FDQ DFKLHYH PXFK EHWWHU EHDW WUDFNLQJ SHUIRUPDQFH FRPSDULQJ WR WKH H[LVWLQJ EHDW WUDFNLQJ VROXWLRQV
7KH UHVW RI WKLV SDSHU LV RUJDQL]HG DV IROORZV 6HFWLRQ ,, GLVFXVVHV UHODWHG ZRUN 6HFWLRQ ,,, LQWURGXFHV VRPH EDVLF NQRZOHGJH DERXW 6DOVD DQG WKH RYHUYLHZ RI RXU SURSRVHG V\VWHP 6HFWLRQ ,9 DQG 6HFWLRQ 9 GHVFULEH OfÀine Beat Counting Mode DQG Online Beat Counting Mode WKDW DUH FRUH FRPSRQHQWV RI RXU V\VWHP UHVSHFWLYHO\ ,Q 6HFWLRQ 9, ZH HYDOXDWH WKH SHUIRUPDQFH RI WKH V\VWHP $QG LQ 6HFWLRQ 9,, ZH SHUIRUP GLVFXVVLRQ RQ WKH SURSRVHG SalsaAsst V\VWHP )LQDOO\ ZH FRQFOXGH WKH SDSHU LQ 6HFWLRQ 9,,, ,, 5 (/$7(' : 25. 7KH FUX[ IRU DVVLVWLQJ LQ GDQFLQJ 6DOVD LV WR KHOS SHRSOH WR UHFRJQL]H WKH EHDWV RU WKH WLPLQJV RI PRYLQJ GDQFH VWHSV ,Q JHQHUDO WKH H[LVWLQJ VROXWLRQV XVHG WR DVVLVW GDQFHUV WR UHFRJQL]H WKH EHDW WLPLQJV FDQ EH FDWHJRUL]HG LQWR WZR JURXSV GDQFH PDVWHU UHOLHG DQG GDQFH PXVLF EDVHG 7KH WUDGLWLRQDO ZD\ RI OHDUQLQJ 6DOVD LV WR DWWHQG D GDQFH OHVVRQ LQ ZKLFK D GDQFH PDVWHU FDQ FRXQW DORQJ ZLWK WKH EHDWV RI GDQFLQJ PXVLF IRU WKH EHJLQQHUV ZKR KDYH GLI¿FXOW\ ZLWK UHFRJQL]LQJ WKH EHDW WLPLQJV ,Q DGGLWLRQ VRPH DSSURDFKHV XVH GHGLFDWHG VHQVRUV WR FDSWXUH WKH PRYHPHQWV RI D SURIHVVLRQDO GDQFHU WR JHW WKH FRUUHFW WLPLQJV RI PRYLQJ VWHSV )RU H[DPSOH 1DNDPXUD et al. >@ SURSRVH D GDQFH WUDLQLQJ V\VWHP ZKLFK FDSWXUHV ' PRWLRQ RI D GDQFH PDVWHU DQG DQDO\]HV WKH PRWLRQ WR JHW WKH WLPLQJV RI PRYLQJ KLVKHU VWHSV ,Q RUGHU WR IROORZ WKH SURSHU GDQFH VWHSV ZKLOH GDQFLQJ WKH XVHUV QHHG WR ZHDU DFWLYH YLEURGHYLFHV ZKLFK FDQ JHQHUDWH YLEUDWLRQ WR UHPLQG XVHUV RI WKH VWHS WLPLQJV ,Q DGGLWLRQ 0LVDWR et al. >@ SUHVHQW D V\VWHP WKDW XVHV ERWK DFRXVWLF IHDWXUHV RI GDQFH PXVLF DQG VNHOHWRQ IHDWXUHV IURP PRYHPHQWV RI D GDQFH PDVWHU WR WUDFN WKH EHDWV +RZHYHU WKHVH DSSURDFKHV UHTXLUH PRWLRQ VHQVLQJ GHYLFH HJ 0RWLRQ $QDO\VLV )DOFRQ >@ .LQHFW >@ IRU FDSWXULQJ WKH PRYHPHQWV RI SURIHVVLRQDO GDQFHUV ZKLFK LV FRVWO\ DQG QRW DOZD\V DYDLODEOH )XUWKHUPRUH WKHUH DUH D IHZ H[LVWLQJ VROXWLRQV DXWRPDWL FDOO\ WUDFNLQJ GDQFH EHDWV WKURXJK DFRXVWLF VLJQDO SURFHVVLQJ RI GDQFLQJ PXVLF )RU LQVWDQFH WKH\ LQFOXGH EHDWWHPSODWH PHWKRG >@ %D\HVLDQ IUDPHZRUN >@ G\QDPLF SURJUDP PLQJ >@ >@ DQG WZRVWDWH PRGHO >@ ,Q SDUWLFXODU * 3HHWHUV >@ XVHV EHDWWHPSODWH WUDLQLQJ WR HVWLPDWHG PXVL FDO WHPSR $7 &HPJLO >@ IRUPXODWHV WHPSR WUDFNLQJ LQ D %D\HVLDQ IUDPHZRUN 7KH G\QDPLF SURJUDPPLQJ DSSURDFK LV XWLOL]HG LQ >@ >@ WR KDQGOH EHDW WUDFNLQJ LQ WKH DXGLR VLJQDO 0 'DYLHV DQG 0 3OXPEOH\ >@ SURSRVH D WZRVWDWH PRGHO ZKLFK LQFOXGHV JHQHUDO VWDWH DQG FRQWH[WGHSHQGHQW VWDWH WR ¿QG WKH EHDW SRVLWLRQV IURP PXVLF VLJQDO ,Q DGGLWLRQ WKH ZRUN LQ > @ LV VLPLODU WR RXUV ZKLFK DOVR FDQ UHFRJQL]H DQG FRXQW EHDWV E\ SURFHVVLQJ DXGLR VLJQDO RI GDQFLQJ PXVLF +RZHYHU WKHVH DSSURDFKHV DUH QRW DSSOLFDEOH IRU 6DOVD EHDW WUDFNLQJ 'LIIHUHQW IURP RWKHU W\SHV RI PXVLF 6DOVD¶V UK\WKPLF PDGH E\ /DWLQ LQVWUXPHQWV PD\ DSSHDU DW HYHU\ WZR 6DOVD EHDWV RU
2XU SURSRVHG SalsaAsst FDQ SHUIRUP UK\WKP GHFLSKHULQJ DQG ¿QHJUDLQHG EHDW WUDFNLQJ IRU 6DOVD PXVLF VLJQDO DQG DXWRPDWLFDOO\ FRXQW 6DOVD EHDWV WR UHPLQG WKH GDQFHUV RI WKH WLPLQJV RI EHDWV ZLWKRXW DQ\ DGGLWLRQDO KXPDQ WDVN RU GHGLFDWHG VHQVRUV GHSOR\PHQW :H VKRZ WKDW LW LV IHDVLEOH WR HQDEOH WKH PRELOH GHYLFH WR EHFRPH D YLUWXDO DVVLVWDQW IRU 6DOVD GDQFH RQO\ XVLQJ WKH EXLOWLQ PLFURSKRQH VSHDNHUV DQG YLEUDWRU RI WKH GHYLFH :H GHYHORS D QRYHO EHDW FRXQWLQJ DOJRULWKP IRU 6DOVD PXVLF ZKLFK KDV WKH FDSDELOLW\ WR WUDFN 6DOVD EHDWV DFFXUDWHO\ LQ ERWK UHDOWLPH DQG RIÀLQH PDQQHUV
82
%HDW
&ODYH
7LPEDOH
D 0XVLFDO PHDVXUH
&RQJD
&RZEHOO 0RQWXQR
*XLU|
E 'DQFLQJ PHDVXUH RI 6DOVD ZŝŐŚƚ &ŽŽƚ ĂĐŬ
>ĞĨƚ &ŽŽƚ &ŽƌǁĂƌĚ
^ŚŝĨƚ tĞŝŐŚƚ ƚŽĂĐŬ &ŽŽƚ
>ĞĨƚ &ŽŽƚ ĂĐŬ
^ŚŝĨƚ tĞŝŐŚƚ ƚŽ&ƌŽŶƚ &ŽŽƚ
3OD\ WRJHWKHU ZŝŐŚƚ &ŽŽƚ &ŽƌǁĂƌĚ
6WUHVVHG
8QVWUHVVHG
WĂƵƐĞ
)LJ 5K\WKPLF SDWWHUQV RI GLIIHUHQW /DWLQ LQVWUXPHQWV LQ 6DOVD PXVLF
WĂƵƐĞ
NLQGV RI PXVLF WKH EHDWV RI 6DOVD FRPH IURP YDULRXV /DWLQ LQVWUXPHQWV LQFOXGLQJ &ODYH VWLFNV &RQJD GUXPV 7LPEDOH GUXPV 0RQWXQR &RZ EHOO *XLU HWF (YHU\ LQVWUXPHQW LV SOD\HG ZLWK XQLTXH EHDWV 6RPH RI WKHP DUH HYHQ SOD\HG DW VSHFL¿F KDOI EHDWV )RU LQVWDQFH &ODYHV LV SOD\HG DW DQG HYHU\ WZR EDUV LQ 6DOVD VRQJV )LJXUH VKRZV DQ H[DPSOH WKDW DOO WKHVH LQVWUXPHQWV SOD\LQJ WRJHWKHU PDNHV LW GLI¿FXOW WR GHFLSKHU EHDWV IURP WKH 6DOVD PXVLF 6LQFH 6DOVD PXVLF KDV D EXQFK RI GLIIHUHQW FRPELQDWLRQV RI WKHVH /DWLQ LQVWUXPHQWV WKH UK\WKP RI 6DOVD LV K\EULG DQG FRPSOH[ PDNLQJ LW KDUG WR SHUFHLYH RU SLQSRLQW E\ HDUV (YHQ IRU WKH SURIHVVLRQDO GDQFHUV LW LV DOVR SRVVLEOH WR KDYH HUURU MXGJPHQWV IRU 6DOVD EHDWV GXULQJ OLYH SHUIRUPDQFH ,Q RUGHU WR KHOS 6DOVD EHJLQQHUV WR DFFHOHUDWH WKHLU OHDUQLQJ SURFHVV DQG EH PRUH FRQ¿GHQW RQ WKH VWDJH ZH SURSRVH SalsaAsst ZKLFK FDQ DFFXUDWHO\ LGHQWLI\ WKH EHDWV RI DQ\ 6DOVD PXVLF DQG JHQHUDWH UHDOWLPH EHDW FRXQWLQJ YRLFHYLEUDWLRQ SURPSWV LH EHDWV WR UHPLQG RI ULJKW WLPLQJV RI PRYLQJ KLVKHU IHHW ZKLOH GDQFLQJ
F 6DOVD EHDWV FRUUHVSRQGLQJ WR 6DOVD VWHSV )LJ ,OOXVWUDWLRQ RI WKH UHODWLRQVKLS EHWZHHQ 6DOVD PXVLFDO PHDVXUH GDQFLQJ PHDVXUH DQG LWV FRUUHVSRQGLQJ 6DOVD VWHSV
KDOI 6DOVD EHDW 7KHUHIRUH WKHVH DSSURDFKHV ZRXOG OHDG WR D PDVV RI PLVWDNHQO\ GHWHFWHG EHDWV IRU 6DOVD PXVLF 'LIIHUHQW IURP WKH SUHYLRXV ZRUN SalsaAsst FDQ DFFXUDWHO\ WUDFN WKH EHDWV DW WKH FRUUHFW PHWULFDO OHYHO HPEHGGHG LQ 6DOVD PXVLF E\ H[SORLWLQJ WKH SURSRVHG OfÀine Beat Counting DQG Online Beat Counting DOJRULWKP 0RUHRYHU SalsaAsst OHYHU DJHV PRELOH GHYLFHV HJ VPDUWSKRQH WR UDLVH EHDW FRXQWLQJ YRLFHYLEUDWLRQ SURPSWV WR UHPLQG XVHUV RI WKH WLPLQJV RI 6DOVD EHDWV :LWKRXW DQ\ DGGLWLRQDO KXPDQ WDVN RU VHQVRU GHSOR\PHQW SalsaAsst FDQ EH XVHG FRQYHQLHQWO\ DQG ÀH[LEO\ LQ DQ\ VFHQDULRV HJ OLYH SHUIRUPDQFH SUDFWLFLQJ DW KRPH RU GDQFLQJ VWXGLR WR DVVLVW 6DOVD GDQFHUV ,,, 3 5(/,0,1$5,(6 $1' 6 @ 6DOVD PXVLF LV DUUDQJHG LQ EDUV ,W LV ZULWWHQ LQ 4/4 WLPH VLJQDWXUH WKDW KDV IRXU EHDWV LQ HYHU\ EDU DV LOOXVWUDWHG LQ )LJXUH D $ EDU RU PHDVXUH LV D VHJPHQW RI WLPH FRUUH VSRQGLQJ WR D VSHFL¿F QXPEHU RI EHDWV >@ )RU 6DOVD GDQFLQJ LW LV GDQFHG ZLWK F\FOHV RI WZR EDUV PDNLQJ D WRWDO RI HLJKW EHDWV DV VKRZQ LQ )LJXUH E 0RUH LPSRUWDQWO\ DPRQJ WKHVH HLJKW EHDWV GDQFHUV RQO\ QHHG WR VWHS RQ VL[ EHDWV LH DV VKRZQ LQ )LJXUH F 7KH EHDWV ´´ DQG ´´ DUH XVHG IRU SDXVH 2QH RI WKH PRVW FKDOOHQJLQJ SDUWV IRU 6DOVD GDQFLQJ LV UHFRJQL]LQJ WKHVH 6DOVD EHDWV FRUUHFWO\ 'LIIHUHQW IURP RWKHU
83
Offline Mode Salsa Music File
Online Mode Real-time Salsa Music %
Short Period Listening
%
%
%
% 2QVHW3RVWLRQ
(QHUJ\3URILOH
Onset Detection )LJ 2QVHW SRVLWLRQV RI SOD\LQJ &ODYH LQ WZR EDUV
Salsa Beat Interval (SBI) Estimation Offline Beat Counting
1) Spectrogram: 6SHFWURJUDP GHVFULEHV WKH DXGLR VLJQDO LQ WHUPV RI HQHUJ\ VSUHDG RYHU LWV IUHTXHQF\ FRPSRQHQWV DW GLIIHUHQW WLPHV ,Q SalsaAsst WKH VSHFWURJUDP LV REWDLQHG E\ FRPSXWLQJ WKH ORJPDJQLWXGH RI WKH 6KRUWWLPH )RXULHU WUDQVIRUP 67)7 RI DXGLR VLJQDO &RQVLGHU D OHQJWKN PXVLF VLJQDO x[n] RQ 0 n N − 1 7KH 6KRUWWLPH )RXULHU WUDQVIRUP 67)7 RI x[n] LV GH¿QHG DV
Online Beat Counting
Beat Counting Voice/Vibration Prompts )LJ 6\VWHP RYHUYLHZ RI 6DOVD$VVW
ST F T {x [n]} (m, k) =
N −1
2π
x [n] ω [n − mR] e−j N kN ,
n=0
ZKHUH m LV WKH LQGH[ RI ZLQGRZ IUDPH R GHQRWHV WKH KRS VL]H RI 67)7 ZKLFK LV HTXDO WR 32 VDPSOHV DQG k UHSUHVHQWV WKH kWK IUHTXHQF\ ELQ ω[n] GHQRWHV WKH ZLQGRZ IXQFWLRQ +HUH ZH FRPSXWH WKH 67)7 RI 6DOVD PXVLF VLJQDO ZLWK +DPPLQJ ZLQGRZ IXQFWLRQ ZKLFK LV GH¿QHG DV 0.51 − 0.46cos 2πn ,0 ≤ n ≤ M M ω [n] = , 0 , 2WKHUZLVH
FRUUHVSRQGLQJ WR WKH 6DOVD EHDWV ,Q WKH Online Beat Counting Mode RXU V\VWHP OHYHUDJHV WKH PLFURSKRQH RQ PRELOH GHYLFHV WR UHFHLYH WKH DFRXVWLF VLJQDO RI 6DOVD PXVLF DQG JHQHUDWH WKH UHDOWLPH SURPSWV LH EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ 0RUH VSHFL¿FDOO\ RXU V\VWHP OLVWHQV IRU D VKRUW SHULRG RI WLPH LH 10 VHFRQGV DW WKH EHJLQQLQJ IRU FRPSXWLQJ WKH HVWLPDWHG YDOXH RI 6DOVD %HDW ,QWHUYDO 6%, %DVHG RQ WKH HVWLPDWHG 6%, RXU V\VWHP LV DEOH WR JHQHUDWH WKH UHDOWLPH EHDW FRXQWLQJ YRLFHYLEUDWLRQ SURPSWV DORQJ ZLWK WKH GDQFH PXVLF WR LQGLFDWH WKH WLPLQJV RI EHDWV :H OHDYH WKH GHWDLOV RI OfÀine Beat Counting Mode DQG Online Beat Counting Mode WR 6HFWLRQ ,9 DQG 6HFWLRQ 9 UHVSHFWLYHO\
ZKHUH M LV WKH OHQJWK RI ZLQGRZ IUDPH ZKLFK LV HTXDO WR 256 VDPSOHV DQG n GHQRWHV WKH VDPSOHV LQ ZLQGRZ %\ WKH ZD\ ZH XVH 44.1kHz DV WKH VDPSOLQJ UDWH LQ 67)7 $IWHU FRPSXWLQJ WKH 67)7 WKH VSHFWURJUDP RI WKH PXVLF VLJQDO x[n] FDQ EH REWDLQHG E\ FDOFXODWLQJ WKH ORJPDJQLWXGH RI WKH 67)7 ZKLFK GH¿QHG DV
,9 2 ))/,1( % ($7 & 2817,1* 0 2'( ,Q WKLV VHFWLRQ ZH ¿UVW GHVFULEH Onset Detection DQG WKHQ SUHVHQW Salsa Beat Interval (SBI) Estimation EDVHG RQ WKH RQVHW SRVLWLRQV :H ¿QDOO\ GHVFULEH OfÀine Beat Counting DOJRULWKP WR VKRZ KRZ WR DFFXUDWHO\ GHWHFW WKH 6DOVD EHDWV IURP RQVHWV
Specg {x [n]} (m, k) = log10 (|ST F T {x [n]} (m, k)|). $V VKRZQ LQ )LJXUH D WKH UHG UHJLRQV RI VSHFWURJUDP DUH WKH IUHTXHQF\ UHJLRQV ZLWK D KLJK GHJUHH RI HQHUJ\ 2Q WKH RWKHU FRQWUDU\ WKH WLQW UHJLRQV DUH WKH IUHTXHQF\ UHJLRQV ZLWK D ORZ GHJUHH RI HQHUJ\ 2) Onset Strength Curve: :KLOH VSHFWURJUDP UHSUHVHQWV WKH HQHUJ\ GLVWULEXWLRQ RI GLIIHUHQW IUHTXHQF\ WKH 2QVHW 6WUHQJWK &XUYH 26& LV WKH PHDVXUHPHQW RI WKH HQHUJ\ FKDQJHV LQ VSHFWURJUDP %\ VXPPLQJ WKH SRVLWLYH FKDQJHV DFURVV DOO IUHTXHQF\ ELQV LQ VSHFWURJUDP ZH DUH DEOH WR REWDLQ WKH UDZ 2QVHW 6WUHQJWK &XUYH 26& DV LOOXVWUDWHG LQ )LJXUH E 7KH 26& LV GH¿QHG DV IROORZV>@
A. Onset Detection ,Q WKLV VWHS ZH GHWHFW DOO WKH RQVHWV LQ WKH 6DOVD PXVLF VLJQDO $Q RQVHW UHIHUV WR WKH EHJLQQLQJ RI D VXGGHQ EXUVW RI HQHUJ\ FDXVHG E\ SOD\LQJ LQVWUXPHQWV >@ )RU H[DPSOH )LJXUH VKRZV WKH WLPH GRPDLQ ZDYHIRUP RI SOD\LQJ WKH &ODYH LQ WZR EDUV DQG WKH FRUUHVSRQGLQJ RQVHW SRVLWLRQV ZKLFK LOOXVWUDWHV WKH UHODWLRQVKLSV EHWZHHQ EHDWV DQG RQVHWV :H FDQ ¿QG WKDW WKH RQVHW LV WKH SRVLWLRQ ZKHUH WKH EHDW PD\ DSSHDU 7KHUHIRUH RQVHW GHWHFWLRQ LV WKH EDVLF VWHS IRU WKH IXUWKHU EHDW WUDFNLQJ 7KH SURFHGXUH RI RQVHW GHWHFWLRQ LV WR FRPSXWH WKH VSHF WURJUDP IURP WKH DXGLR VLJQDO WKHQ REWDLQ WKH 2QVHW 6WUHQJWK &XUYH 26& DQG ¿QG WKH ORFDO PD[LPD IURP VPRRWKHG 26& DV RQVHW SRVLWLRQ 7KH IROORZLQJ DUH WKH GHWDLOV
OSC(m) =
N
H(Specg(m, k) − Specg(m − 1, k)),
k=1
ZKHUH Specg(m, k) UHSUHVHQWV WKH VSHFWURJUDP RI WKH kWK IUHTXHQF\ ELQ RI WKH mWK IUDPH N GHQRWHV WKH QXPEHU
84
2QVHW,QWHUYDO VHF
)UHTXHQF\ N+]
6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD 6DOVD
0XVLF,QGH[
7LPH VHF
D 6SHFWURJUDP
)LJ 'LVSHUVLRQV RI 2,V LQ QLQH 6DOVD VRQJV
$PSOLWXGH
7$%/( , ) 5(48(1&< 7$%/( 2) 2, 6 ,1 6 $/6$
2, VHF
7LPH VHF
E 5DZ 2QVHW 6WUHQJWK &XUYH 26&
$PSOLWXGH
6PRRWKHG26& 7UHQG
3URSRUWLRQ
B. Salsa Beat Interval (SBI) Estimation
%DVHG RQ WKH RQVHW SRVLWLRQV ZH FDQ HVWLPDWH 6DOVD %HDW ,QWHUYDO 6%, 6%, GHQRWHV WKH WLPH LQWHUYDO EHWZHHQ WKH DGMDFHQW 6DOVD EHDWV )RU H[DPSOH WKH WLPH EHWZHHQ EHDW 1 DQG 2 LV RQH 6%, ZKLOH WKH WLPH EHWZHHQ 1 DQG 1.5 LV D KDOI RI 6%, 0RUHRYHU WKH 6%, LV D ¿[HG YDOXH IRU D 6DOVD VRQJ >@ 2XU PHWKRG IRU 6DOVD %HDW ,QWHUYDO 6%, HVWLPDWLRQ LV EDVHG RQ WKH REVHUYHG UHODWLRQVKLS EHWZHHQ RQVHW DQG 6DOVD EHDW LQ 6DOVD PXVLF $V VKRZQ LQ )LJXUH GLIIHUHQW 6DOVD LQVWUXPHQWV DUH SOD\HG ZLWK XQLTXH UK\WKPV IRU H[DPSOH WKH &ODYH LV SOD\HG DW ZKHUH WKH RQVHWV FDQ EH REVHUYHG WKHUH +RZHYHU ZKLOH SOD\LQJ DOO WKHVH LQVWUXPHQWV LQ 6DOVD PXVLF ZH FDQ ¿QG WKDW WKH RQVHWV DSSHDU HYHU\ KDOI 6DOVD EHDW 7KDW LV WR VD\ WKH RQVHW LQWHUYDO 2, LV VXSSRVHG WR EH HTXDO WR KDOI 6%, LQ 6DOVD PXVLF :H GHPRQVWUDWH WKDW WKLV RE VHUYHG UHODWLRQVKLS EHWZHHQ RQVHW DQG 6DOVD EHDW LV UHDVRQDEOH WKURXJK DQDO\]LQJ WKH GLVWULEXWLRQ RI RQVHW LQWHUYDOV 2,V LQ 6DOVD PXVLF $V VKRZQ LQ )LJXUH ZH OLVW WKH GLVWULEXWLRQV RI RQVHW LQWHUYDOV 2,V LQ QLQH 6DOVD VRQJV :H FDQ ¿QG WKDW PRVW RI 2,V LQ D 6DOVD VRQJ DUH FRQFHQWUDWHG DURXQG D YDOXH 7KDW LV PRVW RI 2,V DUH DSSUR[LPDWHO\ HTXLYDOHQW 7KHUHIRUH WR HVWLPDWH KDOI 6%, RI D 6DOVD VRQJ HJ 6DOVD ZH GR VWDWLVWLFV RI WKH IUHTXHQF\ RI 2,V DV VKRZQ LQ 7DEOH , DQG FRPSXWH WKH DULWKPHWLF DYHUDJH RI 2,V ZKRVH IUHTXHQF\ SURSRUWLRQ LV KLJK LH > 5% 7KLV PHDQ YDOXH LV DEOH EH UHJDUGHG DV WKH DSSUR[LPDWHG YDOXH RI 2, LH KDOI 6%, 7KH HVWLPDWHG 6%, WKHQ FDQ EH REWDLQHG E\ GRXEOLQJ WKH DSSUR[LPDWHG KDOI 6%,
7LPH VHF
F 6PRRWKHG 26& DQG LWV WUHQG 2QVHW6WUHQJWK&XUYH 2QVHWSRVLWLRQ
$PSOLWXGH
)UHTXHQF\
7LPH VHF
G 'HWUHQGHG 26& DQG 2QVHW SRVLWLRQV )LJ ,OOXVWUDWLRQ RI WKH SURFHGXUH RI 2QVHW 'HWHFWLRQ
RI IUHTXHQF\ ELQV H(x) = IXQFWLRQ
x+|x| 2
LV WKH KDOIZDYH UHFWL¿HU
3) Onset Positions: $IWHU REWDLQLQJ WKH UDZ 2QVHW 6WUHQJWK &XUYH WKH VPRRWKHG 26& DQG LWV WUHQG FDQ EH REWDLQHG E\ OHYHUDJLQJ *DXVVLDQ ORZSDVV ¿OWHU ZLWK GLIIHUHQW SDUDPH WHUV >@ DV VKRZQ LQ )LJXUH F ,Q SDUWLFXODU WKH UDZ 26& LV ¿OWHUHG E\ D *DXVVLDQ ¿OWHU ZLWK D FXWRII IUHTXHQF\ HTXDO WR WKH VDPSOLQJ UDWH GLYLGHG E\ 20 WR REWDLQ WKH VPRRWKHG 26& ,WV WUHQG LV REWDLQHG E\ ¿OWHULQJ WKH VPRRWKHG 26& XVLQJ DQRWKHU *DXVVLDQ ¿OWHU ZLWK D FXWRII IUHTXHQF\ HTXDO WR WKH VDPSOLQJ UDWH GLYLGHG E\ 150 )XUWKHUPRUH ZH FDQ FRPSXWH WKH GHWUHQGHG 26& E\ VXEVWUDFWLQJ WKH WUHQG IURP WKH VPRRWKHG 26& DQG ¿QG LWV FRUUHVSRQGLQJ SHDNV DV WKH RQVHW SRVLWLRQV DV LOOXVWUDWHG LQ )LJXUH G
C. OfÀine Beat Counting $FFRUGLQJ WR WKH GHWHFWHG RQVHW SRVLWLRQV DQG HVWLPDWHG 6%, 6DOVD EHDWV ZRXOG EH VHDUFKHG DFFXUDWHO\ +HUH ZH XVH DQ H[DPSOH DV VKRZQ LQ )LJXUH WR H[SODLQ RXU PHWKRG IRU
85
&DQGLGDWH
$OJRULWKP 2QOLQH %HDW &RXQWLQJ 'DWD 7KH UHFHLYHG UHDOWLPH DXGLR VLJQDO DW VDPSOLQJ UDWH f s = 44.1kHz X = [X(1), X(2), X(3)...] 7KH HVWLPDWHG YDOXH RI 6DOVD %HDW ,QWHUYDO EB 7KH OHQJWK RI ZLQGRZ IUDPH W in = (EB + 0.1) ∗ f s 5HVXOW 7LPH RI %HDWV B = [B(1), B(2), B(3)..., B(i), ...] 5HDOWLPH %HDW &RXQWLQJ 9RLFH DQG 9LEUDWLRQ 3URPSWV EHJLQ B(1) = 0s. ,QVHUW 3URPSWV DW B(1) B(2) = B(1) + EB. ,QVHUW 3URPSWV DW B(2) LI i 64 WKHQ
%HDW %HDW
%HDW V
V
(VWLPDWHG6%,
D 6HDUFKLQJ UHJLRQ KDV RQVHW V &DQGLGDWH %HDW %HDW V
V
i−1
(VWLPDWHG6%,
E 6HDUFKLQJ UHJLRQ KDV QR RQVHW
)LJ ,OOXVWUDWLRQ RI KRZ WR WUDFN 6DOVD EHDW IURP RQVHWV
EHDW VHDUFKLQJ )LUVW RI DOO ZH ¿QG WKH ¿UVW RQVHW SRVLWLRQ DV WKH ¿UVW EHDW SRVLWLRQ DQG ODEHO LW DV %HDW 6WDUWLQJ IURP %HDW ZH DGG WKH HVWLPDWHG 6%, WR JHW WKH FDQGLGDWH RI WKH VHFRQG EHDW SRVLWLRQ LH %HDW :H WKHQ VHDUFK DURXQG WKH FDQGLGDWH WR ORFDWH WKH QHDUHVW RQVHW 6LQFH WKH KXPDQ HDU FDQQRW GLVWLQJXLVK WKH GHOD\DGYDQFH RI WKH VRXQG OHVV WKDQ 0.1s >@ WKH WROHUDWH GHOD\ DQG DGYDQFH IRU WKH GHWHFWHG EHDW DUH 0.1s 7KHUHIRUH WKH VHDUFKLQJ UHJLRQ LV FRQ¿QHG WR ±0.1s DURXQG WKH FDQGLGDWH ,I LW KDV RQVHWV LQ WKH VHDUFKLQJ UHJLRQ DV VKRZQ LQ )LJXUH D ZH UHJDUG WKH QHDUHVW RQVHW SRVLWLRQ DV WKH EHDW SRVLWLRQ DQG ODEHO LW DV %HDW ,I QRW DV LOOXVWUDWHG LQ )LJXUH E WKH FDQGLGDWH ZRXOG EH UHFRJQL]HG DV WKH EHDW SRVLWLRQ 7KHQ ZH UHSHDW WKH VWHSV DERYH WR VHDUFK WKH QH[W EHDW SRVLWLRQ 7KLV PHWKRG LV DEOH WR HOLPLQDWH WKH LPSDFWV RI WKH IDOVH DQG PLVVHG RQVHW GHWHFWLRQV DQG WKHQ ORFDOL]H WKH 6DOVD EHDWV DFFXUDWHO\ 0RUHRYHU WKH EHDW ODEHOV DUH F\FOHG HYHU\ 8 EHDWV 2XU V\VWHP WKHQ JHQHUDWHV EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ SURPSWV FRUUHVSRQGLQJ WR WKH EHDW ODEHOV WKDW GDQFHUV QHHG WR VWHS LH
B(i+1)−B(i)
EB = 1 i−1 // Replace EB with the mean value of previous SBIs HQG ZKLOH i 3 GR 3HUIRUP 2QVHW 'HWHFWLRQ RQ X[B(i − 1) ∗ f s) : B(i − 1) ∗ f s) + W in)] )LQG WKH QHDUHVW RQVHW SRVLWLRQ N DURXQG B(i − 1) + EB LI |N − B(i − 1) − EB| 0.08 WKHQ B(i) = N. ,QVHUW 3URPSWV DW B(i) HOVH B(i) = B(i − 1) + EB. ,QVHUW 3URPSWV DW B(i) HQG HQG // The units of EB, B, N , are all second(s) HQG
Counting LV SURYLGHG LQ $OJRULWKP 6SHFL¿FDOO\ ZH ¿UVW UHJDUG WKH EHJLQQLQJ RI WKH SOD\LQJ 6DOVD PXVLF DV WKH WLPH RI ¿UVW EHDW B(1) DQG WKHQ JHW WKH WLPH RI B(2) E\ DGGLQJ HVWLPDWHG 6%, )RU VHDUFKLQJ WKH QH[W EHDWV ZH ¿UVW JHW WKH FDQGLGDWH EHDW WLPH B(i − 1) + EB WKURXJK DGGLQJ HVWLPDWHG 6%, WR WKH SULRU EHDW WLPH 7KHQ ZH SHUIRUP RQVHW GHWHFWLRQ RQ WKH VLJQDO EHWZHHQ WKH WLPH B(i−1) DQG B(i−1)+EB+0.1 WR REWDLQ WKH RQVHW SRVLWLRQV +HUH 0.1s LV WLPH RI WKH WROHUDWH GHOD\ IRU GHWHFWHG EHDWV )URP WKHVH RQVHWV ZH ¿QG WKH QHDUHVW RQVHW WR WKH FDQGLGDWH EHDW WLPH ,I WKH WLPH GLIIHUHQFH EHWZHHQ WKH QHDUHVW RQVHW DQG FDQGLGDWH LV OHVV WKHQ 0.1s LH WLPH RI WKH WROHUDWH GHOD\DGYDQFH WKH QHDUHVW RQVHW LV UHJDUGHG DV WKH 6DOVD EHDW 2WKHUZLVH WKH FDQGLGDWH VKRXOG EH LGHQWL¿HG DV WKH EHDW $GGLWLRQDOO\ FRQVLGHULQJ WKH EB GHULYHG IURP WKH VKRUW SHULRG KLVWRULFDO GDWD PD\ QRW EH TXLWH DFFXUDWH WKH YDOXH RI EB ZRXOG EH FRQWLQXDOO\ XSGDWHG WR WKH PHDQ YDOXH RI WKH WLPH LQWHUYDOV RI GHWHFWHG EHDWV DIWHU GHWHFWLQJ 64 EHDWV ZKLFK FDQ PDNH EB PRUH DFFXUDWH
9 2 1/,1( % ($7 & 2817,1* 0 2'( ,Q WKLV VHFWLRQ ZH LQWURGXFH KRZ WR WUDFN EHDWV IURP UHDOWLPH 6DOVD PXVLF DQG JHQHUDWH UHDOWLPH EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ SURPSWV RQ PRELOH GHYLFHV 'LIIHUHQW IURP OfÀine Beat Counting ZH GRQ¶W KDYH D 6DOVD VRQJ ¿OH WR HVWLPDWH 6DOVD %HDW ,QWHUYDO 6%, ,QVWHDG LQ Online Beat Counting RXU V\VWHP ¿UVW OLVWHQV IRU D VKRUW SHULRG RI WLPH LH 10 VHFRQGV RI 6DOVD PXVLF WR HVWLPDWH WKH 6%, YLD Onset Detection DQG Salsa Beat Interval (SBI) Estimation ZKLFK KDYH EHHQ LQWURGXFHG VHFWLRQ ,9 %DVHG RQ WKH HVWLPDWHG 6%, IURP WKH VHFRQGV KLVWRULFDO GDWD RXU V\VWHP WKHQ XVHV WKH EXLOWLQ PLFURSKRQH RI PRELOH GHYLFH WR VHQVH WKH UHDOWLPH 6DOVD PXVLF VLJQDO DQG VDYH WKH GDWD LQ WKH EXIIHU :H XVH WZR WKUHDGV IRU UHDOWLPH EHDW FRXQWLQJ 7KH PDLQ WKUHDG LV XVHG WR SOD\ WKH EHDW FRXQWLQJ YRLFH SURPSWV DQG JHQHUDWH YLEUDWLRQ SURPSWV $QRWKHU WKUHDG LV IRU VHDUFKLQJ 6DOVD EHDWV RI WKH UHFHLYHG VLJQDO LQ WKH EXIIHU 7KH DOJRULWKP RI Online Beat
86
9, 3 (5)250$1&( ( 9$/8$7,21
WLPLQJV JLYH WKH JUHDWHVW VFRUH 7KH EHDW DFFXUDF\ WKHQ LV PHDVXUHG E\ VXPPLQJ XS WKHVH VFRUHV • 36FRUH ZKLFK PHDVXUHV EHDW WUDFNLQJ DFFXUDF\ E\ VXP PLQJ WKH FURVVFRUUHODWLRQ EHWZHHQ LPSXOVH WUDLQV RI WKH GHWHFWHG EHDWV DQG WKH JURXQG WUXWK DQQRWDWLRQV FRQVLGHU LQJ GHYLDWLRQV ZLWKLQ 20% RI WKH DQQRWDWHG EHDW LQWHUYDO DV FRUUHFW 2) Continuity-based Metrics: • &0/W ZKLFK ¿QGV WKH FRUUHFW GHWHFWHG EHDWV DW WKH PHWULFDO OHYHO RI 6DOVD EHDWV LH DQG FDOFXODWHV UDWLR RI WKH QXPEHU RI FRUUHFW EHDWV WR WKH OHQJWK RI DQQRWDWLRQ VHTXHQFH • &0/F ZKLFK ¿QGV WKH ORQJHVW VHJPHQW RI FRQWLQXRXVO\ FRUUHFW GHWHFWHG EHDWV DW WKH FRUUHFW PHWULFDO OHYHO DQG FDOFXODWHV WKH UDWLR RI WKH OHQJWK RI WKH ORQJHVW VHJPHQW WR WKH OHQJWK RI DQQRWDWLRQ VHTXHQFH • $0/W ZKLFK ¿QGV WKH FRUUHFW GHWHFWHG EHDWV DW WZLFH RU KDOI WKH FRUUHFW PHWULFDO OHYHO DQG FDOFXODWHV UDWLR RI WKH QXPEHU RI FRUUHFW EHDWV WR WKH OHQJWK RI DQQRWDWLRQ VHTXHQFH • $0/F ZKLFK ¿QGV WKH ORQJHVW VHJPHQW RI FRQWLQXRXVO\ FRUUHFW GHWHFWHG EHDWV DW WZLFH RU KDOI WKH FRUUHFW PHWULFDO OHYHO DQG FDOFXODWHV WKH UDWLR RI WKH OHQJWK RI WKH ORQJHVW VHJPHQW WR WKH OHQJWK RI DQQRWDWLRQ VHTXHQFH 3) Beat Error Histogram: :H FRPSXWH WKH EHDW HUURU DV WKH GLIIHUHQFH EHWZHHQ HDFK DQQRWDWLRQ DQG WKH QHDUHVW GHWHFWHG EHDW 7KH EHDW HUURU RI HYHU\ GHWHFWHG EHDW LV WKHQ GLYLGHG E\ WKH DQQRWDWLRQ LQWHUYDO ZKLFK QRUPDOL]HV WKH EHDW HUURUV EHWZHHQ −0.5 DQG 0.5 ,Q DGGLWLRQ LI DQ\ EHDW HUURUV DUH EH\RQG [−0.5 0.5] WKH\ FDQ EH FRQ¿QHG EDFN LQWR WKH UDQJH [−0.5 0.5] XVLQJ PRGXOR DULWKPHWLF )LQDOO\ ZH XWLOL]H WKHVH EHDW HUURU GDWD WR SRSXODWH D KLVWRJUDP 7KH EHDW HUURU Φ LV GH¿QHG DV > @ ⎧ minj (|a−bi |) ,j = 1 ⎪ ⎨ aj+1 −aj Φ(i) = , ⎪ ⎩ minj (|a−bi |) ,j > 1 aj −aj−1
,Q WKLV VHFWLRQ ZH HYDOXDWH WKH EHDW WUDFNLQJ DFFXUDF\ RI ERWK OfÀine Beat Counting DOJRULWKP DQG Online Beat Counting DOJRULWKP LQ RXU V\VWHP A. Experimental Methodology :H JHQHUDWH D WHVW GDWDVHW ZKLFK FRQVLVWV RI 40 6DOVD PXVLF DXGLR ¿OHV ZLWK WKH JURXQG WUXWK DQQRWDWLRQV RI 6DOVD EHDW WLPLQJV (DFK ¿OH LV DQ HQWLUH 6DOVD VRQJ ZKLFK LV DURXQG 5 PLQXWHV LQ OHQJWK VWHUHR DQG 44.1kHz VDPSOLQJ UDWH $OVR WKH ¿OHV DUH DQQRWDWHG E\ WKH SURIHVVLRQDO 6DOVD GDQFHUV E\ UHFRUGLQJ EHDW FRXQWLQJ YRLFH LH EHDWV WR WKH DXGLR ¿OHV :H PDQXDOO\ QRWH GRZQ WKH JURXQG WUXWK DQQRWDWLRQV DFFRUGLQJ WR WKH EHDW FRXQWLQJ YRLFH LQ HYHU\ PXVLF ¿OH (YHU\ PXVLF ¿OH LQ RXU GDWDVHW LV IHG LQWR OfÀine Beat Counting DOJRULWKP ZKLFK FDQ DXWRPDWLFDOO\ UHFRUG WKH WLP LQJV RI WKH GHWHFWHG EHDWV 7R WHVW WKH Online Beat Counting DOJRULWKP WKH 6DOVD PXVLF ¿OH LV SOD\HG E\ WKH H[WHUQDO VSHDNHU 2XU V\VWHP ¿UVW OLVWHQV IRU VHFRQGV RI WKH SOD\LQJ PXVLF WR FRPSXWH WKH HVWLPDWHG 6%, $IWHU VHFRQGV SOD\LQJ WKH OfÀine Beat Counting DOJRULWKP ZRXOG UHFRUG WKH EHDW WLPLQJV LQ UHDOWLPH ,Q DGGLWLRQ ZH DOVR FRPSDUH WKH EHDW WUDFNLQJ DFFXUDF\ RI RXU DOJRULWKPV ZLWK RWKHU WKUHH EHDW WUDFN LQJ DSSURDFKHV WKH FRQWH[WGHSHQGHQW EHDW WUDFNLQJ V\VWHP LH >@ WKH WZRIROG G\QDPLF SURJUDPPLQJ EHDW WUDFNHU LH >@ DQG WKH EHDW WUDFNHU XVLQJ G\QDPLF SURJUDPPLQJ LH >@ B. Evaluation Metrics )RU HYDOXDWLQJ WKH EHDW WUDFNLQJ DFFXUDF\ DJDLQVW PDQXDO DQQRWDWLRQV ZH XVH HLJKW HYDOXDWLRQ PHWULFV IURP WKH EHDW WUDFNLQJ HYDOXDWLRQ WRROER[ >@ ZKLFK DUH JHQHUDOO\ DGRSWHG LQ EHDW WUDFNLQJ HYDOXDWLRQ )LUVW $FFXUDF\EDVHG 0HWULFV ZKLFK FDQ DVVHVV WKH WRWDO QXPEHU RI FRUUHFW GHWHFWHG EHDWV QRW FRQVLGHULQJ WKH PHWULFDO OHYHO LQFOXGH )PHDVXUH &HPJL O$FF >@ 36FRUH >@ 6HFRQG &RQWLQXLW\EDVHG 0HWULFV >@ LQFOXGHV &0/F &RUUHFW 0HWULFDO /HYHO ZLWK FRQWLQXLW\ UH TXLUHG &0/W &RUUHFW 0HWULFDO /HYHO ZLWK QR FRQWLQXLW\ UHTXLUHG $0/F $OORZHG 0HWULFDO /HYHO ZLWK FRQWLQXLW\ UHTXLUHG $0/W $OORZHG 0HWULFDO /HYHO ZLWK QR FRQWLQXLW\ UHTXLUHG ZKLFK FDQ PDQLIHVW ZKHWKHU WKH GHWHFWHG EHDWV DUH DW WKH PHWULFDO OHYHO RI 6DOVD EHDWV DQG WKH FRQWLQXLW\ RI WKH FRUUHFW GHWHFWHG EHDWV 0RUHRYHU WR YLVXDOL]H WKH EHDW WUDFNLQJ SHUIRUPDQFH ZH XWLOL]H %HDW (UURU +LVWRJUDP > @ WR VKRZ WKH GLVWULEXWLRQ RI WKH EHDW HUURU SUREDELOLW\ 1) Accuracy-based Metrics: • )PHDVXUH ZKLFK FRXQWV WKH QXPEHU RI WUXH SRVLWLYHV T P FRUUHFW GHWHFWLRQ ZLWKLQ D WROHUDQFH ZLQGRZ RI ±70ms IDOVH QHJDWLYHV F N PLVVHG GHWHFWLRQ DQG IDOVH SRVLWLYHV F P H[WUD GHWHFWLRQ DQG GH¿QHV WKH EHDW DFFX P UDF\ DV 2∗T P2∗T +F N +F P ∗ 100% • &HPJLO$FF ZKLFK XWLOL]HV D *DXVVLDQ HUURU IXQFWLRQ ZLWK D VWDQGDUG GHYLDWLRQ RI 40ms WR HYDOXDWH WKH WLPH HUURU EHWZHHQ WKH GHWHFWHG EHDW DQG WKH FORVHVW DQQRWDWLRQ VR WKDW WKH GHWHFWHG EHDWV O\LQJ H[DFWO\ RQ WKH DQQRWDWHG EHDW
ZKHUH b DQG a UHIHU WR WKH VHTXHQFH RI GHWHFWHG EHDWV DQG DQQRWDWLRQV UHVSHFWLYHO\ bi GHQRWHV WKH WLPLQJV RI WKH ith EHDW aj GHQRWHV WKH WLPLQJV RI WKH j th DQQRWDWLRQ C. Results )LJXUH D DQG )LJXUH E SUHVHQW WKH SHUIRUPDQFH RI RXU OfÀine Beat Counting DQG Online Beat Counting DOJRULWKPV 'HYLHV >@ -DQJ >@ DQG (OOLV >@ XQGHU WKH $FFXUDF\EDVHG 0HWULFV DQG &RQWLQXLW\EDVHG 0HWULFV UHVSHFWLYHO\ $V VKRZQ LQ )LJXUH D ERWK OfÀine Beat Counting DOJRULWKP DQG Online Beat Counting DOJRULWKP RXWSHUIRUP DOO WKH RWKHU EHDW WUDFNLQJ V\VWHPV EDVHG RQ WKH SHUIRUPDQFH LQGLFHV RI ) PHDVXUH &HPLO$FF DQG 3VFRUH 7KDW LV RXU V\VWHP KDYH FDSDFLW\ WR ¿QG PRUH FRUUHFW EHDWV WKDQ RWKHU V\VWHPV 0RUHRYHU RWKHU V\VWHPV DOVR VHHP WR SHUIRUP ZHOO ZKHUH WKH )PHDVXUH VFRUHV RI 'HYLHV >@ DQG -DQJ >@ DUH 69.03% DQG 83.24% UHVSHFWLYHO\ +RZHYHU DV VKRZQ LQ )LJXUH E RQFH WKH FRQVWUDLQWV RYHU WKH PHWULFDO OHYHO RI 6DOVD EHDWV DUH
87
100
3URSRUWLRQ
0HDQ6FRUH
)PHDVXUH &HPJLO$FF 36FRUH
80 60 40
%HDW(UURU Φ
SalsaAsst (Online)
Davies [6]
Jang [5]
Ellis [4]
D $FFXUDF\EDVHG 0HWULFV
3URSRUWLRQ
SalsaAsst (Offline)
&0/W &0/F $0/W $0/F
80 60
40
%HDW(UURU Φ
E 6DOVD$VVW 2QOLQH
20
SalsaAsst (Offline)
SalsaAsst (Online)
Davies [6]
Jang [5]
Ellis [4]
3URSRUWLRQ
0
D 6DOVD$VVW 2IÀLQH
100
0HDQ6FRUH
20 0
E &RQWLQXLW\EDVHG 0HWULFV
)LJ &RPSDULVRQ RI EHDW WUDFNLQJ DOJRULWKPV XQGHU $FFXUDF\EDVHG 0HWULFV DQG &RQWLQXLW\EDVHG 0HWULFV
%HDW(UURU Φ
F 'DYLHV >@
UHTXLUHG WKHVH V\VWHPV DUH QR ORQJHU DFFXUDWH WKDW WKH &0/W RI -DQJ >@ LV RQO\ 1.49% 7KH &0/W VFRUHV RI 'HYLHV >@ DQG (OOLV >@ DUH HYHQ HTXDO WR 0% ZKLFK LQGLFDWHV WKDW WKH GHWHFWHG EHDWV RI WKHVH V\VWHPV DUH DOO QRW DW WKH PHWULFDO OHYHO RI 6DOVD EHDWV 2XU SURSRVHG DOJRULWKPV KRZHYHU DUH VWLOO DFFXUDWH ZKHUH &0/W RI OfÀine Beat Counting DOJRULWKP LV 82.28% DQG &0/W RI Online Beat Counting DOJRULWKP LV 65.96% ZKLFK GHPRQVWUDWHV WKDW PRVW EHDWV GHWHFWHG E\ RXU DOJRULWKPV DUH H[DFWO\ DW WKH PHWULFDO OHYHO RI 6DOVD EHDWV QDPHO\ RXU DOJRULWKPV DSSO\ WR 6DOVD PXVLF PXFK EHWWHU WKDQ RWKHUV 2Q WKH RWKHU KDQG &0/F RI RXU DOJRULWKPV UHÀHFWLQJ WKH FRQWLQXLW\ RI WKH FRUUHFW GHWHFWHG EHDWV DW WKH 6DOVD EHDWV PHW ULFDO OHYHO LV RQO\ HTXDO WR 37.04% IRU OfÀine Beat Counting DOJRULWKP DQG 22.41% IRU Online Beat Counting DOJRULWKP %XW WKH ORZ &0/F PHUHO\ LQGLFDWHV WKDW WKH ZURQJ GHWHFWHG EHDWV RI RXU V\VWHP DUH VFDWWHUHG UDWKHU WKDQ FHQWUDOL]HG ZKLFK FDQQRW SURYH WKH 6DOVD EHDW GHWHFWLRQ RI RXU DOJRULWKPV DUH LQDFFXUDWH EHFDXVH RXU DOJRULWKPV REWDLQ D KLJK VFRUH XQGHU &0/W $GGLWLRQDOO\ ZH QRWLFH WKDW WKH $0/W DQG $0/F RI 'HYLHV >@ DUH ERWK JRRG DQG WKH $0/F RI 'HYLHV >@ LV HYHQ EHWWHU WKDQ RXU DOJRULWKPV +LJK $0/W DQG $0/F VFRUHV GHPRQVWUDWH WKDW 'HYLHV >@ FDQ DFFXUDWHO\ ¿QG WKH EHDWV DW WKH WZLFH RU KDOI PHWULFDO OHYHO RI 6DOVD %HDWV 1HYHUWKHOHVV WKH &0/W DQG &0/F VFRUHV RI 'HYLHV >@ DUH HTXDO WR 0% WKDW LV WR VD\ 'HYLHV >@ FDQ PHUHO\ GHWHFW WKH EHDWV DW WKH WZLFH RU KDOI PHWULFDO OHYHO RI 6DOVD %HDWV EXW LV QRW DEOH WR ¿QG EHDWV DW WKH 6DOVD %HDWV PHWULFDO OHYHO )XUWKHUPRUH DV LOOXVWUDWHG LQ )LJXUH ZH XWLOL]H %HDW (UURU +LVWRJUDP WR YLVXDOL]H WKH SHUIRUPDQFH RI RXU DOJR ULWKPV DQG RWKHU WKUHH H[LVWLQJ EHDW WUDFNLQJ DSSURDFKHV 7KH %HDW (UURU +LVWRJUDP LV DEOH WR VKRZ WKH GLVWULEXWLRQ RI DOO EHDW HUURUV LQ RXU GDWDVHW )RU RXU SXUSRVHG OfÀine Beat Counting DOJRULWKP LWV %HDW (UURU +LVWRJUDP DV LOOXVWUDWHG
3URSRUWLRQ
%HDW(UURU Φ
G -DQJ >@
3URSRUWLRQ
%HDW(UURU Φ
H (OOLV >@
)LJ %HDW (UURU +LVWRJUDPV RI RXU DOJRULWKPV DQG RWKHU H[LVWLQJ EHDW WUDFNLQJ DSSURDFKHV
LQ )LJXUH D PHUHO\ H[KLELWV D VWURQJ SHDN FORVH WR WKH HUURU RI 0 LQGLFDWLQJ PRVW EHDWV ZH GHWHFWHG DUH FORVH WR WKH JURXQG WUXWK DQQRWDWLRQV )RU Online Beat Counting DOJRULWKP DV VKRZQ LQ )LJXUH E DOWKRXJK LWV %HDW (UURU +LVWRJUDP VKRZV VRPH HUURUV FRQFHQWUDWH RQ −0.4 PRVW EHDW HUURUV VWLOO IRFXV DURXQG 0 8QOLNH RXU DOJRULWKPV WKH %HDW (UURU +LVWRJUDP RI 'HYLHV >@ VKRZQ LQ )LJXUH F KDV WZR VWURQJ SHDNV DURXQG ±0.2 DQG RWKHU WZR VWURQJ SHDNV DURXQG ±0.5 ZKLFK LQGLFDWHV WKDW D VXEVWDQWLDO SDUW RI GHWHFWHG EHDWV DUH QRW PDWFKHG ZLWK WKH JURXQG WUXWK /LNHZLVH )LJXUH G DQG )LJXUH H GHPRQVWUDWH WKDW -DQJ >@ DQG (OOLV >@ DOVR SHUIRUP EDGO\ GXH WR WKH GLVSHUVLYH GLVWULEXWLRQV RI WKHLU EHDW HUURUV 7KHUHIRUH WKH UHVXOWV XQGHU 8 HYDOXDWLRQ PHWULFV FOHDUO\ GHPRQVWUDWH WKH IHDVLELOLW\ RI RXU SURSRVHG V\VWHP IRU DFFX
88
DORQJ ZLWK WKH GDQFH PXVLF 7KH SURSRVHG V\VWHP FDQ EH XVHG LQ PDQ\ VFHQDULRV HJ KRPH SUDFWLFH GDQFH VWXGLR DQG OLYH SHUIRUPDQFH ZKLFK FDQ QRW RQO\ KHOS 6DOVD EHJLQQHUV JHW DFFHOHUDWHG OHDUQLQJ SURFHVV EXW DOVR HOLPLQDWH SURIHVVLRQDO GDQFHUV¶ HUURUV GXULQJ WKHLU OLYH SHUIRUPDQFH ([WHQVLYH WHVWV XVLQJ 40 6DOVD GDQFH VRQJV XQGHU 8 HYDOXDWLRQ PHWULFV VKRZ WKDW RXU V\VWHP FDQ DFKLHYH PXFK EHWWHU 6DOVD EHDW WUDFNLQJ SHUIRUPDQFH FRPSDULQJ WR WKH H[LVWLQJ EHDW WUDFNLQJ VROXWLRQV
UDWHO\ WUDFNLQJ 6DOVD EHDWV IURP 6DOVD PXVLF VLJQDO FRPSDULQJ WR RWKHU H[LVWLQJ EHDW WUDFNLQJ V\VWHPV 9,, ' ,6&866,21 ,Q WKLV VHFWLRQ ZH PDLQO\ GLVFXVV WKH SRWHQWLDO LPSURYHPHQW RI WKH SURSRVHG SalsaAsst V\VWHP ZKLFK ZLOO EH SUHVHQWHG LQ RXU IXWXUH ZRUN A. Additional Assistance from Inertial Sensors of Mobile Devices
,; $&.12:/('*0(17 7KLV ZRUN ZDV VXSSRUWHG LQ SDUW E\ WKH 1DWLRQDO 6FLHQFH )RXQGDWLRQ XQGHU JUDQW QXPEHUV &16 DQG &16
SalsaAsst KDV WXUQHG RXW WR EH D SURPLVLQJ VWHS WRZDUGV HQDEOLQJ WKH PRELOH GHYLFH WR EH D YLUWXDO DVVLVWDQW IRU GDQFLQJ 6DOVD ZKLFK FDQ KHOS GDQFHUV WR UHFRJQL]H WKH FRUUHFW WLPLQJV RI 6DOVD EHDWV %\ H[SORLWLQJ PRUH VHQVRUV VXFK DV LQHUWLDO VHQVRUV LH DFFHOHURPHWHU DQG J\URVFRSH LQ PRELOH GHYLFHV LW LV SRVVLEOH WR SURYLGH DGGLWLRQDO DVVLVWDQFH WR 6DOVD GDQFHUV )RU LQVWDQFH ZH FDQ XWLOL]H WKH EXLOWLQ LQHUWLDO VHQVRUV RI WKH VPDUWSKRQHV WKDW DUH ZRUQ ZLWK GDQFHUV WR GHULYH WKHLU GDQFLQJ PRYHPHQWV WR VHH ZKHWKHU WKH PRYHPHQWV DUH V\QFKURQL]HG WR WKH GHWHFWHG 6DOVD EHDWV ,W FDQ EH XVHG WR HYDOXDWH GDQFHUV¶ GDQFLQJ V\QFKURQL]DWLRQ GXULQJ SUDFWLFH DQG KHOS WKHP LP SURYH LW E\ SURYLGLQJ DSSURSULDWH IHHGEDFN LI GDQFHUV KDYH DQ\ HUURU PRYHPHQWV ,Q WKLV FLUFXPVWDQFH SalsaAsst ZLOO QRW RQO\ DVVLVW LQ LGHQWLI\LQJ 6DOVD EHDWV EXW DOVR HYDOXDWH WKH DFWXDO GDQFLQJ VWHSV DQG FRUUHFW WKHLU PLVWDNHV LI DQ\
5 ()(5(1&(6 >@ ' 'UREQ\ et al. ³6DOWDWH D VHQVRUEDVHG V\VWHP WR VXSSRUW GDQFH EH JLQQHUV´ LQ CHI’09 Extended Abstracts on Human Factors in Computing Systems $&0 SS ± >@ * 3HHWHUV ³7HPSODWHEDVHG HVWLPDWLRQ RI WLPHYDU\LQJ WHPSR´ EURASIP Journal on Applied Signal Processing YRO QR SS ± >@ $ 7 &HPJLO et al. ³2Q WHPSR WUDFNLQJ 7HPSRJUDP UHSUHVHQWDWLRQ DQG NDOPDQ ¿OWHULQJ´ Journal of New Music Research YRO QR SS ± >@ ' 3 (OOLV ³%HDW WUDFNLQJ E\ G\QDPLF SURJUDPPLQJ´ Journal of New Music Research YRO QR SS ± >@ )+ ) :X et al. ³$ WZRIROG G\QDPLF SURJUDPPLQJ DSSURDFK WR EHDW WUDFNLQJ IRU DXGLR PXVLF ZLWK WLPHYDU\LQJ WHPSR´ LQ ISMIR SS ± >@ 0 ( 'DYLHV et al. ³&RQWH[WGHSHQGHQW EHDW WUDFNLQJ RI PXVLFDO DXGLR´ IEEE Transactions on Audio, Speech, and Language Processing YRO QR SS ± >@ $ 1DNDPXUD et al. ³0XOWLPRGDO SUHVHQWDWLRQ PHWKRG IRU D GDQFH WUDLQLQJ V\VWHP´ LQ CHI’05 extended abstracts on Human factors in computing systems $&0 SS ± >@ 0 2KNLWD et al. ³$XGLRYLVXDO EHDW WUDFNLQJ EDVHG RQ D VWDWHVSDFH PRGHO IRU D PXVLF URERW GDQFLQJ ZLWK KXPDQV´ LQ 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems ,((( SS ± > @ 7 0L]XPRWR et al. ³$ URERW OLVWHQV WR PXVLF DQG FRXQWV LWV EHDWV DORXG E\ VHSDUDWLQJ PXVLF IURP FRXQWLQJ YRLFH´ LQ 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems ,((( SS ± >@ 6 3LHWUREUXQR ³(PERG\LQJ FDQDGLDQ PXOWLFXOWXUDOLVP WKH FDVH RI VDOVD GDQFLQJ LQ PRQWUHDO´ Revista mexicana de estudios canadienses >@ 3 5HQWD ³6DOVD GDQFH /DWLQRD KLVWRU\ LQ PRWLRQ´ Centro Journal YRO QR >@ %DU PXVLF >2QOLQH@ $YDLODEOH KWWSV HQZLNLSHGLDRUJZLNL%DU PXVLF >@ ( ' 6FKHLUHU ³7HPSR DQG EHDW DQDO\VLV RI DFRXVWLF PXVLFDO VLJQDOV´ The Journal of the Acoustical Society of America YRO QR SS ± >@ 6 'L[RQ ³2QVHW GHWHFWLRQ UHYLVLWHG´ LQ Proceedings of the 9th International Conference on Digital Audio Effects YRO &LWHVHHU SS ± >@ ' 'HXWVFK Psychology of music (OVHYLHU >@ & ( 5LFH ³+XPDQ HFKR SHUFHSWLRQ´ Science YRO QR SS ± >@ 0 ( 'DYLHV et al. ³(YDOXDWLRQ PHWKRGV IRU PXVLFDO DXGLR EHDW WUDFNLQJ DOJRULWKPV´ Queen Mary University of London, Centre for Digital Music, Tech. Rep. C4DM-TR-09-06 >@ 0 ) 0F.LQQH\ et al. ³(YDOXDWLRQ RI DXGLR EHDW WUDFNLQJ DQG PXVLF WHPSR H[WUDFWLRQ DOJRULWKPV´ Journal of New Music Research YRO QR SS ± > @ 0 ( 'DYLHV et al. ³2Q WKH XVH RI HQWURS\ IRU EHDW WUDFNLQJ HYDOXDWLRQ´ LQ 2007 IEEE International Conference on Acoustics, Speech and Signal Processing YRO ,((( SS ,9±
B. Remove the Requirement of Pre-listening in Online Beat Counting Mode 'LIIHUHQW IURP RWKHU UHDOWLPH EHDW WUDFNLQJ V\VWHPV WKH Online Beat Counting Mode RI RXU V\VWHP QHHG WR OLVWHQ IRU D VKRUW WLPH SHULRG HJ 10 VHFRQGV RI PXVLF EHIRUH UHDO WLPH 6DOVD EHDWV FRXQWLQJ ZKLFK LV XVHG IRU HVWLPDWLQJ 6DOVD %HDW ,QWHUYDO 6%, ZKLFK GHWHUPLQHV WKH PHWULFDO OHYHO RI 6DOVD PXVLF +RZHYHU VXFK VKRUW SHULRG OLVWHQLQJ LV MXVW D VPDOO VHFWLRQ RI DQ HQWLUH 6DOVD PXVLF ZKLFK ZRXOG QRW EH LQFRQYHQLHQW IRU GDQFHUV WR XVH RXU V\VWHP LQ SUDFWLFH ,Q WKH IXWXUH ZH ZLOO UHPRYH VXFK D UHTXLUHPHQW RI SUHOLVWHQLQJ E\ GHVLJQLQJ D TXLFNHU 6%, HVWLPDWLRQXSGDWH PHWKRG ZKLFK FDQ TXLFNO\ FDSWXUH WKH 6%, ZLWKLQ MXVW 23 PXVLF EHDWV LH OHVV WKDQ 1 VHFRQG DQG XSGDWH LW DFFRUGLQJO\ 9,,, & 21&/86,21 7KLV SDSHU SURSRVHV D EHDW FRXQWLQJ V\VWHP SalsaAsst EDVHG RQ PRELOH GHYLFHV HJ VPDUWSKRQHV WR DVVLVW XVHUV WR GDQFH 6DOVD SalsaAsst FDQ SHUIRUP UK\WKP GHFLSKHULQJ DQG ¿QHJUDLQHG EHDW WUDFNLQJ IRU 6DOVD PXVLF DQG JHQHUDWH EHDW FRXQWLQJ YRLFH RU YLEUDWLRQ SURPSWV WR UHPLQG WKH GDQFHUV RI 6DOVD EHDW WLPLQJV 7ZR DOWHUQDWLYH PRGHV DUH GHYHORSHG WR SURYLGH SURPSWV ZLWK GDQFHUV LQ RIÀLQH DQG UHDOWLPH PDQQHUV UHVSHFWLYHO\ 6SHFL¿FDOO\ LQ WKH RIÀLQH PRGH SalsaAsst LV DEOH WR SURFHVV DQ\ 6DOVD PXVLF ¿OH WR JHQHUDWH D QHZ PXVLF ¿OH ZKLFK SRVVHVVHV ERWK RULJLQDO 6DOVD PXVLF DQG LWV FRUUHVSRQG LQJ EHDW FRXQWLQJ YRLFH SURPSWV ,Q WKH UHDOWLPH PRGH RXU V\VWHP OHYHUDJHV WKH HPEHGGHG PLFURSKRQH RQ WKH PRELOH GHYLFH WR UHFHLYH WKH DFRXVWLF VLJQDO RI 6DOVD PXVLF DQG UDLVH WKH UHDOWLPH SURPSWV LH EHDW FRXQWLQJ YRLFH DQG YLEUDWLRQ
89
