Type of the Paper (Article - MDPI

Download PDF
1 downloads 0 Views 1MB Size Report
Comment
0.29. 0.98. 0.25. 68.97. 1.94. 1.83. 10. 0.08. 0.95. 0.06. 0.97. 0.04. 90.00. 6.60. 6.12. 11. 0.08. 0.95. 0.05. 0.97. 0.04. 89.58. 6.54. 6.04. 12. 0.13. 0.93. 0.07. 0.97.
Supplemental Materials

Figure S1. Daily Diarrhea syndromic surveillance data.

Figure S2. Weekly Diarrhea syndromic surveillance data.

1

Figure S3. The results of the selected algorithms for daily real data.

2

Figure S4. The results of the selected algorithms for weekly real data.

3

Table S1. 42 scenarios used to generate data. Scenario 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

𝜽 0.1 0.1 0.1 0.1 0.1 0.1 -2 -2 -2 -2 -2 -2 1.5 1.5 1.5 1.5 1.5 1.5 0.5 0.5 0.5 0.5 0.5 0.5 2.5 2.5 2.5 2.5 2.5 2.5 3.75 3.75 3.75 3.75 3.75 3.75 5 5 5 5 5 5

𝜷 0 0 0 0.0025 0.0025 0.0025 0 0 0 0.005 0.005 0.005 0 0 0 0.003 0.003 0.003 0 0 0 0.002 0.002 0.002 0 0 0 0.001 0.001 0.001 0 0 0 0.001 0.001 0.001 0 0 0 0.0001 0.0001 0.0001

𝜸𝟏 0 0.6 0.6 0 0.6 0.6 0 0.1 0.1 0 0.1 0.1 0 0.2 0.2 0 0.2 0.2 0 0.5 0.5 0 0.5 0.5 0 1 1 0 1 1 0 0.1 0.1 0 0.1 0.1 0 0.05 0.05 0 0.05 0.05

4

𝜸𝟐 0 0.6 0.6 0 0.6 0.6 0 0.3 0.3 0 0.3 0.3 0 -0.4 -0.4 0 -0.4 -0.4 0 0.5 0.5 0 0.5 0.5 0 0.1 0.1 0 0.1 0.1 0 -0.1 -0.1 0 -0.1 -0.1 0 0.01 0.01 0 0.01 0.01

𝝓 1.5 1.5 1.5 1.5 1.5 1.5 2 2 2 2 2 2 1 1 1 1 1 1 5 5 5 5 5 5 3 3 3 3 3 3 1.1 1.1 1.1 1.1 1.1 1.1 1.2 1.2 1.2 1.2 1.2 1.2

m 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2

Trend 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1

Table S2. The average result of the best CUSUM (glm with trend) algorithm.

Scenario 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

Sensitivity 0.53 0.77 0.85 0.28 0.56 0.79 0.29 0.35 0.42 0.08 0.08 0.13 0.87 0.86 0.85 0.64 0.73 0.74 0.91 0.93 0.98 0.90 0.90 0.91 0.94 0.98 0.97 0.90 0.96 0.97 0.89 0.91 0.92 0.95 0.89 0.89 0.99 0.97 0.96 0.97 0.98 0.97

Specificity 0.95 0.91 0.83 0.97 0.95 0.74 0.95 0.95 0.95 0.95 0.95 0.93 0.87 0.87 0.82 0.91 0.92 0.67 0.81 0.79 0.74 0.80 0.81 0.75 0.77 0.72 0.67 0.78 0.75 0.66 0.81 0.75 0.76 0.83 0.84 0.82 0.73 0.68 0.72 0.65 0.69 0.73

PPV 0.46 0.43 0.35 0.37 0.54 0.22 0.22 0.23 0.29 0.06 0.05 0.07 0.43 0.41 0.33 0.41 0.50 0.15 0.30 0.32 0.34 0.30 0.33 0.31 0.37 0.42 0.40 0.36 0.44 0.36 0.47 0.42 0.43 0.50 0.50 0.47 0.43 0.40 0.46 0.38 0.42 0.43

NPV 0.98 0.99 0.99 0.97 0.98 0.98 0.98 0.98 0.98 0.97 0.97 0.97 0.99 0.99 0.99 0.97 0.98 0.97 0.99 1.00 1.00 0.99 0.99 0.99 0.99 1.00 0.99 0.99 0.99 0.99 0.99 0.99 0.99 1.00 0.99 0.99 1.00 1.00 0.99 0.99 1.00 1.00

5

F measure 0.39 0.48 0.44 0.23 0.42 0.32 0.18 0.21 0.25 0.04 0.04 0.06 0.51 0.50 0.42 0.43 0.52 0.24 0.42 0.45 0.47 0.43 0.45 0.42 0.49 0.54 0.51 0.46 0.55 0.49 0.53 0.50 0.51 0.60 0.57 0.54 0.56 0.51 0.57 0.50 0.54 0.55

sMAPE 98.02 106.82 120.31 79.30 84.47 106.82 64.79 66.32 68.97 90.00 89.58 91.88 103.66 104.63 108.42 54.26 55.25 80.00 131.14 138.22 150.92 108.35 116.31 125.01 108.41 132.05 151.50 93.51 114.79 142.07 77.67 84.93 85.05 60.07 60.96 63.04 79.78 86.32 84.81 89.09 85.40 78.91

RMSE 4.56 5.26 6.97 8.62 9.38 11.67 1.85 1.90 1.94 6.60 6.54 6.63 10.19 10.79 11.53 20.54 21.34 27.15 7.88 10.59 17.41 11.60 14.45 18.96 26.24 60.20 145.46 30.57 62.00 154.67 41.42 44.59 45.91 51.95 54.48 57.28 139.16 140.84 151.83 145.94 145.32 144.36

MAD 4.26 4.66 5.66 8.05 8.46 9.66 1.74 1.79 1.83 6.12 6.04 6.10 9.00 9.27 9.63 18.62 19.13 23.65 6.48 8.00 11.02 9.96 11.82 14.18 19.29 33.28 68.48 24.38 39.29 80.75 34.31 35.81 36.89 44.52 45.88 47.49 101.61 105.76 108.91 110.80 109.63 106.23

Table S3. The average result of the EARS C3 algorithm. Scenario 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

Sensitivity 0.4 0.56 0.64 0.22 0.5 0.52 0.15 0.07 0.11 0.08 0.11 0.15 0.63 0.67 0.67 0.45 0.53 0.48 0.62 0.69 0.74 0.57 0.62 0.71 0.72 0.75 0.74 0.65 0.73 0.72 0.68 0.71 0.73 0.68 0.66 0.73 0.7 0.69 0.69 0.71 0.7 0.71

Specificity 0.94 0.93 0.89 0.93 0.9 0.83 0.99 0.99 1 0.92 0.91 0.9 0.92 0.91 0.89 0.94 0.9 0.81 0.93 0.93 0.92 0.91 0.9 0.88 0.93 0.87 0.81 0.95 0.86 0.81 0.95 0.94 0.92 0.96 0.93 0.89 0.96 0.96 0.95 0.97 0.96 0.95

PPV 0.29 0.35 0.32 0.13 0.24 0.2 0.45 0.32 0.34 0.03 0.04 0.05 0.41 0.4 0.33 0.37 0.3 0.17 0.42 0.47 0.51 0.33 0.38 0.36 0.56 0.43 0.35 0.59 0.41 0.35 0.63 0.62 0.51 0.71 0.52 0.44 0.76 0.71 0.69 0.76 0.75 0.68

NPV 0.97 0.97 0.97 0.96 0.97 0.96 0.98 0.97 0.98 0.97 0.97 0.97 0.97 0.98 0.97 0.96 0.96 0.95 0.97 0.98 0.98 0.97 0.97 0.97 0.97 0.97 0.96 0.97 0.97 0.95 0.97 0.97 0.97 0.97 0.96 0.97 0.96 0.96 0.96 0.96 0.96 0.96

6

F measure 0.33 0.42 0.41 0.16 0.31 0.28 0.14 0.08 0.11 0.04 0.06 0.07 0.48 0.48 0.43 0.37 0.37 0.25 0.48 0.54 0.57 0.41 0.45 0.46 0.61 0.53 0.47 0.59 0.51 0.46 0.63 0.64 0.59 0.67 0.56 0.54 0.7 0.68 0.67 0.72 0.7 0.67

sMAPE 67.78 77.25 88.08 56.16 65.24 83.6 NA NA NA 69.81 69.89 72.52 77.09 77.97 82.75 40.44 44.59 54.87 109.97 110.7 113.87 97.05 101.98 108.34 76.12 89.11 111.8 62.88 79.53 108.27 40.75 42.34 45.22 32.61 33.05 35.01 37.59 37.26 39.36 37.94 37.49 37.4

RMSE 2.79 4.04 6.84 5.14 6.42 10.66 NA NA NA 4.15 4.15 4.23 9.81 10.43 11.9 16.13 17.45 22.06 9.14 12.41 23.44 12.44 16.85 22.53 33.74 88.21 229.95 36.1 86.88 238.19 43.13 49.4 52.54 48.96 50.83 53.79 166.41 169.08 189.27 174.48 173.52 176.42

MAD 2.36 3.08 4.49 4.67 5.5 8.38 NA NA NA 3.68 3.68 3.74 7.35 7.48 8.41 14.16 15.1 18.2 6.41 8.01 12.61 10.2 12.49 15.45 21.04 43.43 106.17 24.6 46.81 120.81 26.54 29.01 31.07 31.94 32.71 34.14 87.25 89.31 98.86 93.15 91.82 93.62

Table S4. The result of the ARIMA algorithm. Scenario

Sensitivity

Specificity

PPV

NPV

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

0.39 0.42 0.34 0.31 0.33 0.27 0.30 0.34 0.37 0.23 0.17 0.19 0.41 0.36 0.35 0.38 0.29 0.22 0.39 0.35 0.31 0.42 0.33 0.27 0.32 0.19 0.15 0.33 0.18 0.16 0.28 0.25 0.21 0.32 0.26 0.20 0.19 0.18 0.16 0.17 0.17 0.17

0.98 0.99 0.99 0.96 0.97 0.97 0.97 0.97 0.97 0.92 0.91 0.91 1.00 1.00 0.99 0.97 0.97 0.96 0.99 0.99 1.00 0.97 0.98 0.99 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00

0.54 0.69 0.82 0.29 0.43 0.47 0.28 0.27 0.32 0.08 0.05 0.08 0.90 0.89 0.84 0.51 0.44 0.34 0.70 0.83 0.98 0.53 0.64 0.73 1.00 1.00 1.00 0.98 1.00 1.00 1.00 1.00 1.00 1.00 0.98 0.99 1.00 1.00 1.00 1.00 1.00 1.00

0.97 0.96 0.96 0.97 0.96 0.95 0.98 0.98 0.98 0.97 0.97 0.97 0.96 0.95 0.95 0.95 0.95 0.94 0.96 0.95 0.94 0.96 0.94 0.94 0.94 0.92 0.90 0.94 0.92 0.90 0.93 0.93 0.92 0.93 0.93 0.93 0.91 0.91 0.90 0.90 0.91 0.91

7

F Measure 0.40 0.48 0.44 0.27 0.35 0.31 0.25 0.26 0.30 0.11 0.08 0.10 0.52 0.47 0.46 0.40 0.33 0.25 0.47 0.46 0.44 0.44 0.41 0.37 0.47 0.31 0.25 0.47 0.30 0.27 0.41 0.38 0.34 0.47 0.39 0.32 0.31 0.30 0.27 0.28 0.28 0.28

sMAPE

RMSE

MAD

86.38 99.28 117.29 60.86 75.01 91.27 42.44 43.23 44.56 66.86 67.90 69.53 99.62 104.77 106.86 43.04 50.04 55.58 133.71 142.84 156.49 105.12 115.86 128.78 111.61 141.95 164.49 85.66 123.30 152.84 64.61 68.10 69.79 42.48 45.53 47.77 64.57 65.41 68.21 64.63 66.33 67.41

3.96 5.23 7.89 5.91 8.05 11.56 1.50 1.52 1.54 4.08 4.15 4.28 11.95 13.38 14.38 16.99 20.31 23.32 11.22 15.48 25.49 14.20 18.57 26.66 37.84 87.49 221.65 38.00 92.52 238.75 46.71 51.05 52.87 46.92 51.17 54.15 164.12 165.35 178.83 176.77 180.95 185.55

3.75 4.98 7.51 5.37 7.37 10.65 1.45 1.47 1.50 3.70 3.76 3.88 11.38 12.71 13.66 15.38 18.33 20.99 10.76 14.83 24.03 13.29 17.51 25.26 36.01 82.37 209.94 35.97 87.14 225.32 44.25 48.06 49.84 44.34 48.05 50.95 153.96 155.85 167.54 164.52 169.81 174.12

Table S5. The result of Holt-Winter algorithm. Scenario

Sensitivity

Specificity

PPV

NPV

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

0.35 0.47 0.46 0.26 0.39 0.44 0.27 0.32 0.35 0.20 0.14 0.21 0.44 0.43 0.41 0.35 0.35 0.33 0.43 0.40 0.34 0.43 0.39 0.34 0.37 0.19 0.16 0.34 0.18 0.17 0.31 0.28 0.26 0.36 0.31 0.31 0.19 0.18 0.17 0.18 0.18 0.17

0.99 0.99 1.00 0.96 0.97 0.98 0.97 0.97 0.97 0.93 0.92 0.92 1.00 1.00 1.00 0.97 0.97 0.96 0.99 1.00 1.00 0.97 0.99 1.00 1.00 0.99 0.99 1.00 0.99 0.99 1.00 1.00 1.00 1.00 1.00 1.00 0.99 0.99 0.99 1.00 0.99 1.00

0.55 0.79 0.91 0.29 0.45 0.62 0.31 0.27 0.32 0.08 0.05 0.09 0.95 0.92 0.94 0.53 0.51 0.41 0.76 0.91 0.96 0.59 0.72 0.89 0.99 0.81 0.83 0.98 0.84 0.86 0.98 0.97 0.95 0.98 1.00 0.99 0.87 0.85 0.85 0.89 0.85 0.86

0.97 0.97 0.96 0.96 0.96 0.96 0.98 0.98 0.98 0.97 0.97 0.97 0.96 0.96 0.96 0.95 0.95 0.95 0.96 0.95 0.94 0.96 0.95 0.94 0.94 0.92 0.90 0.94 0.92 0.90 0.94 0.93 0.93 0.94 0.93 0.94 0.91 0.91 0.90 0.90 0.91 0.91

8

F Measure 0.37 0.54 0.57 0.24 0.39 0.49 0.23 0.25 0.28 0.11 0.07 0.11 0.56 0.55 0.53 0.39 0.38 0.35 0.51 0.53 0.47 0.47 0.48 0.46 0.51 0.30 0.25 0.48 0.29 0.28 0.43 0.41 0.39 0.50 0.45 0.45 0.30 0.29 0.28 0.29 0.29 0.27

sMAPE

RMSE

MAD

75.09 87.72 109.35 53.13 62.57 78.41 25.02 26.20 27.00 57.99 57.78 59.02 92.22 97.15 99.44 37.79 41.20 44.75 127.28 138.25 153.77 97.60 110.19 124.94 105.84 132.57 160.36 81.40 114.75 149.00 60.08 63.73 65.45 40.18 42.50 44.32 60.63 61.86 64.76 58.10 59.73 61.31

3.29 4.28 6.84 4.97 5.95 8.36 1.24 1.26 1.28 3.44 3.40 3.47 10.31 11.41 12.28 14.54 15.42 16.51 9.67 13.88 23.85 12.05 16.18 24.21 34.23 85.94 221.76 35.31 85.67 222.73 43.06 47.21 49.69 44.59 47.73 50.28 154.03 157.11 171.70 153.06 158.83 164.40

3.10 4.05 6.48 4.50 5.45 7.80 1.19 1.21 1.23 3.09 3.06 3.13 9.79 10.83 11.66 13.14 13.96 15.02 9.22 13.18 22.27 11.19 15.23 22.79 32.26 79.14 205.68 33.31 79.39 206.48 40.33 44.18 46.18 41.89 44.77 47.14 142.45 145.97 158.03 141.91 147.32 152.66