Are we fooling ourselves when comparing mean correct RTs, and

Are we fooling ourselves when comparing mean correct RTs, and error percentages? Sven Panis Thomas Schmidt University of Kaiserslautern, Germany

ECVP 2018 Trieste, Italy To exist is to change, to change is to mature - Henri Bergson There are no secrets that time does not reveal - Jean Racine

Event history analysis ●

Developed to describe and model time-to-event data, such as: - RT data - saccade latencies - fixation durations - time-to-force-threshold data - time-to-synchronization data -…

●

Synonyms: hazard analysis, survival analysis, failuretime analysis, duration analysis, transition analysis

Continuous time methods for RT



“Probability density functions can appear nearly identical, both statistically and to the naked eye, and yet are clearly different on the basis of their hazard functions (but not vice versa). Hazard functions are thus more diagnostic than density functions (Townsend, 1990)” when studying the shape of the RT distribution... Holden, J. G., Van Orden, G. C., & Turvey, M. T. (2009). Dispersion of response times reveals cognitive dynamics. Psychological Review, 116 (2), 318-342

Discrete time event history analysis time bin ID (0,40] (40,80] (80,120] (120,160] (160,200] (200,240] (240,280] (280,320] (320,360] (360,400] (400,440] (440,480] (480,520] (520,560] (560,600]

time bin index t 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# Censored 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4

# Events 0 0 0 0 0 0 7 13 26 40 48 37 32 9 4

Risk Set 220 220 220 220 220 220 220 213 200 174 134 86 49 17 8

h(t) 0 0 0 0 0 0 .032 .061 .130 .230 .358 .430 .653 .529 .500

1-h(t) 1 1 1 1 1 1 .968 .939 .870 .770 .642 .570 .347 .471 .500

S(t) 1 1 1 1 1 1 .968 .909 .791 .609 .391 .223 .077 .036 .018

P(t) 0 0 0 0 0 0 .032 .059 .118 .182 .218 .168 .145 .041 .018

h(t) = Prob(T=t | T ≥ t) -> # events / risk set S(t) = Prob(T > t) = [1-h(t)].[1-h(t-1)].[1-h(t-2)]. … .[1-h(1)] P(t) = Prob(T = t) = h(t) . S(t-1) -> # events / risk set for first bin ca(t) = Prob(correct | T = t) -> micro-level SAT function (Pachella, 1974)

# Correct 0 0 0 0 0 0 2 10 24 40 47 37 32 9 4

# Error 0 0 0 0 0 0 5 3 2 0 1 0 0 0 0

ca(t) NA NA NA NA NA NA 0.29 0.77 0.92 1.00 0.98 1.00 1.00 1.00 1.00


time bin index t 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# Censored 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4

# Events 0 0 0 0 0 0 7 13 26 40 48 37 32 9 4

Risk Set 220 220 220 220 220 220 220 213 200 174 134 86 49 17 8

h(t) 0 0 0 0 0 0 .032 .061 .130 .230 .358 .430 .653 .529 .500

1-h(t) 1 1 1 1 1 1 .968 .939 .870 .770 .642 .570 .347 .471 .500

S(t) 1 1 1 1 1 1 .968 .909 .791 .609 .391 .223 .077 .036 .018

P(t) 0 0 0 0 0 0 .032 .059 .118 .182 .218 .168 .145 .041 .018


# Correct 0 0 0 0 0 0 2 10 24 40 47 37 32 9 4

# Error 0 0 0 0 0 0 5 3 2 0 1 0 0 0 0



time bin index t 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# Censored 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4

# Events 0 0 0 0 0 0 7 13 26 40 48 37 32 9 4

Risk Set 220 220 220 220 220 220 220 213 200 174 134 86 49 17 8

h(t) 0 0 0 0 0 0 .032 .061 .130 .230 .358 .430 .653 .529 .500

1-h(t) 1 1 1 1 1 1 .968 .939 .870 .770 .642 .570 .347 .471 .500

S(t) 1 1 1 1 1 1 .968 .909 .791 .609 .391 .223 .077 .036 .018

P(t) 0 0 0 0 0 0 .032 .059 .118 .182 .218 .168 .145 .041 .018


# Correct 0 0 0 0 0 0 2 10 24 40 47 37 32 9 4

# Error 0 0 0 0 0 0 5 3 2 0 1 0 0 0 0



time bin index t 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# Censored 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4

# Events 0 0 0 0 0 0 7 13 26 40 48 37 32 9 4

Risk Set 220 220 220 220 220 220 220 213 200 174 134 86 49 17 8

h(t) 0 0 0 0 0 0 .032 .061 .130 .230 .358 .430 .653 .529 .500

1-h(t) 1 1 1 1 1 1 .968 .939 .870 .770 .642 .570 .347 .471 .500

S(t) 1 1 1 1 1 1 .968 .909 .791 .609 .391 .223 .077 .036 .018

P(t) 0 0 0 0 0 0 .032 .059 .118 .182 .218 .168 .145 .041 .018


# Correct 0 0 0 0 0 0 2 10 24 40 47 37 32 9 4

# Error 0 0 0 0 0 0 5 3 2 0 1 0 0 0 0



time bin index t 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# Censored 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4

# Events 0 0 0 0 0 0 7 13 26 40 48 37 32 9 4

Risk Set 220 220 220 220 220 220 220 213 200 174 134 86 49 17 8

h(t) 0 0 0 0 0 0 .032 .061 .130 .230 .358 .430 .653 .529 .500

1-h(t) 1 1 1 1 1 1 .968 .939 .870 .770 .642 .570 .347 .471 .500

S(t) 1 1 1 1 1 1 .968 .909 .791 .609 .391 .223 .077 .036 .018

P(t) 0 0 0 0 0 0 .032 .059 .118 .182 .218 .168 .145 .041 .018


# Correct 0 0 0 0 0 0 2 10 24 40 47 37 32 9 4

# Error 0 0 0 0 0 0 5 3 2 0 1 0 0 0 0



time bin index t 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# Censored 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4

# Events 0 0 0 0 0 0 7 13 26 40 48 37 32 9 4

Risk Set 220 220 220 220 220 220 220 213 200 174 134 86 49 17 8

h(t) 0 0 0 0 0 0 .032 .061 .130 .230 .358 .430 .653 .529 .500

1-h(t) 1 1 1 1 1 1 .968 .939 .870 .770 .642 .570 .347 .471 .500

S(t) 1 1 1 1 1 1 .968 .909 .791 .609 .391 .223 .077 .036 .018

P(t) 0 0 0 0 0 0 .032 .059 .118 .182 .218 .168 .145 .041 .018


# Correct 0 0 0 0 0 0 2 10 24 40 47 37 32 9 4

# Error 0 0 0 0 0 0 5 3 2 0 1 0 0 0 0



time bin index t 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

# Censored 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4

# Events 0 0 0 0 0 0 7 13 26 40 48 37 32 9 4

Risk Set 220 220 220 220 220 220 220 213 200 174 134 86 49 17 8

h(t) 0 0 0 0 0 0 .032 .061 .130 .230 .358 .430 .653 .529 .500

1-h(t) 1 1 1 1 1 1 .968 .939 .870 .770 .642 .570 .347 .471 .500

S(t) 1 1 1 1 1 1 .968 .909 .791 .609 .391 .223 .077 .036 .018

P(t) 0 0 0 0 0 0 .032 .059 .118 .182 .218 .168 .145 .041 .018


# Correct 0 0 0 0 0 0 2 10 24 40 47 37 32 9 4

# Error 0 0 0 0 0 0 5 3 2 0 1 0 0 0 0


Panis, S., & Schmidt, T. (2016). What is shaping RT and accuracy distributions? Active and selective response inhibition causes the negative compatibility effect. Journal of Cognitive Neuroscience, 28 (11), 1651-1671. 4 Mask Types: No Mask, RELevant, IRRELevant, random LINes 3 Prime Types: No Prime, CONsistent, INCONsistent

Masked priming

Masked priming

Masked priming

Masked priming

Masked priming

Inhibition of return






When comparing mean RT → typically IOR effect of about 30 ms When comparing hazard functions → IOR effect of about 120 ms, timelocked to target onset, just as found in single-cell data

Thanks! ●

●

●

Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modelling change and event occurrence. New York: Oxford University Press Inc.

Allison, P. D. (2010). Survival analysis using SAS: A practical guide, Second Edition. SAS Institute Inc., Cary, NC, USA. R code, presentations, papers, ... → Researchgate

Event history analysis: modeling ●

Inferential statistics:

Continuous time units : Hazard rate λ(t) = f(t) / S(t)

Discrete time units: Hazard probability h(t) = Prob(T = t | T ≥ t) -> first create a data file where each row represents a single bin of a trial of a subject

PARAMETRIC MODELS Exponential model: log λ(t) = β0 + β1X1 + ... + βkXk (log T = β’0 + β’1X1 + ... + β’kXk + σε)

LOGIT HAZARD MODEL logit h(t) = log ( h(t) / [1 – h(t)] ) = [ α1D1 + α2D2 + ... + αJDJ ] + [ β1X1 + ... + βkXk ]

Weibull model: log λ(t) = α log t + β0+ β1X1 + ... + βkXk ...

CLOGLOG HAZARD MODELS cloglog h(t) = log(-log[1-h(t)] ) = [ α1D1 + α2D2 + ... + αJDJ ] + [ β1X1 + ... + βkXk ]

SEMI-PARAMETRIC MODEL Cox regression model: log λ(t) = log λ0(t) + β1X1 + ... + βkXk

cloglog h(t) = log(-log[1-h(t)] ) = [ α0ONE + α1(TIME-c)] + [ β1X1 + ... + βkXk ]