the day god failed

THE DAY GOD FAILED A CASE STUDY OF OVERTRUST IN AUTOMATION José João Martins Sampaio New University of Lisbon

Susan Leavines / Photo Researchers Inc.

António Abreu Guerra NAV - Portugal

The Day God Failed – A Case Study of Overtrust in Automation

Presentation Topics

Human Decision and Overtrust in Automation The Day God Failed Is There a Technological Factor ?

JS/AG -2004-03-24

2


ATC Clearances

FPL DATA

AERONAUTICAL ENVIRONMENT

•ENV. Data (AIS/MET, etc.) •TRACK/FPL Correlation •ATC Coordinations

GROUND AIDS JS/AG -2004-03-24

2 - Human Decision and Overtrust in Automation

Strategic/Tactical Decisions


ATC Clearances

FPL

•Stores DATAVisual and Audio Traffic Environment Characteristics; AERONAUTICAL •Keeps Track of Real Time Traffic Situation; ENVIRONMENT

Strategic/Tactical •Defines Task Priority According to Mental Built in Model; Decisions : •Event Density / Particularities ; •ATC System Built in Functions Characteristics; •ENV. and DataEquipment (AIS/MET, etc.) •Unexpected Events. ATC •TRACK/FPL Correlation •ATC Coordinations OPERATIONAL SYSTEM •SENSITIVE TO

GROUND AIDS JS/AG -2004-03-24

5 Overtrust in Automation 3 - Human Decision and


ATC Clearances

FPL

• Uses Working Memory Information to Built/Update DATA Model of Traffic Environment; Mental AERONAUTICAL • Continuously Updates Cognitive Action Schema ENVIRONMENT

Using Working Memory Feed Back - Experience; • Selects Problem Solution Action Schema - Skill; • Responsible for a Safe Skill Based Performance; •ENV. Data (AIS/MET, etc.) ATC •TRACK/FPL Correlation •ATC Coordinations OPERATIONAL SYSTEM •ATC Clearances GROUND AIDS JS/AG -2004-03-24




GNSS

ADS

GNSS

Data Link

8.33

ATFM

ASM

ATS

ATM SUP. SYS

ATM/HR

Mode S Sensor JS/AG -2004-03-24


AIR TRAFFIC CONTROL : A TECHNOLOGICAL DEPENDENT OCCUPATION ?


GNSS

GNSS

Auto Coordin. Coordin. - OLDI

8.33

Voice Com. Sys (VCS (VCS)) • FPL Data Processing • Radar Data Processing Flight Path Monitoring RAM / CLAM

Trajectory Prediction


MTCD

Problem Detection STCA MSAW APW

Resolution Advisory

JS/AG -2004-03-24


7



GNSS

GNSS

Auto Coordination

8.33

Voice Com. Sys (VCS) • FPL Data Processing: • Radar Data Processing; •Working Memory holds a more diffuse and intermittent Flight Path Monitoring picture framework ; RAM / CLAM

•Keeps Track of “System Behaviour”; Trajectory Prediction •Relies MTCD on System Defined Task Priority ; Problem Detection SENSITIVE TO :


•ATC STCASystem Built in Functions and Equipment “surprises”; •Event Density / Particularities ; MSAW •Unexpected Events. APW

Resolution Advisory

JS/AG -2004-03-24


8



GNSS

GNSS

Auto Coordination

8.33

Voice Com. Sys (V CS) • FPL Data Processing: • Radar Data Processing; •Lack of working memory feed back leads to of Cognitive Action Schema Construction: Flight Pathdisruption Monitoring Degradation of Operational Experience; RAM / CLAM

•Uses Working Memory Stored Information to Trajectory Prediction Strategic/Tactical Built Mental Model of Predicted System Performance: Decisions MTCD New Skills? Problem Detection •Relies on Resolution Advisory Action Schema: STCA Degradation of Operational Skill; MSAW

•Cannot be Responsible for a Safe Operational Skill APW Based Performance - ATCo becomes a spectator.

Resolution Advisory

JS/AG -2004-03-24


9



GNSS GNSS

Auto Coordin. Coordin. - OLDI

8.33

Voice Com. Sys (VCS (VCS)) • FPL Data Processing: • Radar Data Processing; Flight Path Monitoring RAM / CLAM

Trajectory Prediction


MTCD

Problem Detection STCA MSAW APW

Resolution Advisory

JS/AG -2004-03-24



THE DAY GOD FAILED

JS/AG -2004-03-24

10


JS/AG -2004-03-24

11 – The day “God” failed...


The ATCO is no longer the Superman of traffic management, that makes decisions on his own and works isolated… JS/AG -2004-03-24



…Now he must find himself in a more efficient and motivated team environment. JS/AG -2004-03-24



TRM in Portugal • • •

TRM is operational and behavioural oriented : all study cases and team role-playings are taken from our real experience. TRM in Portugal is not mandatory, at this moment, and we don’t think it should be. But there is a clear orientation from top managers to continue with TRM as part of ATC training.

JS/AG -2004-03-24



TOAST: Trm Oriented Simulator Training TOAST: All Trm ATM simulations are team oriented andTraining include unusual Oriented Simulator situations exercises such as recovering from lost of Situation Awareness.

JS/AG -2004-03-24



TRM in Portugal Next steps: 1 - The introduction of simulation exercises on recovery from lost of Situation Awareness, in all ab-initio radar courses. 2 - An initiative on team work preventing RWY incursions (ARM ?). 3 - A prototype course for 5 aviation partners: ATCOs, Pilots, Technical staff, AIS and Cabin Crew (2005): -Outdoor type. -Safety oriented -Big issue: communication. A another ARM experience? 4 - New TRM modules (like automation and conflict generation) to be used as TRM refresher courses.

JS/AG -2004-03-24



TRM

Some results of the final questionnaire in a scale from 1 to 5: (Average of all TRM courses): Questions :

4,4

1. How do you rate this training action : 2. How do you rate the importance of this training action for

4,7

your future work: JS/AG -2004-03-24



TRM

A comment from a portuguese air traffic controller: “This was, in terms of training, the most important thing that happened in my 29 years as an Air Traffic Controller" JS/AG -2004-03-24


A DECISÃO OPERACIONAL NOS SERVIÇOS DE TRÁFEGO The Day God Failed – A Case Study of Overtrust in Automation AÉREO

JS JS/AG - 2001-05-10 -2004-03-24

54

19

A DECISÃO OPERACIONAL NOS SERVIÇOS DE TRÁFEGO The Day God Failed – A Case Study of Overtrust in Automation AÉREO

JS JS/AG - 2001-05-10 -2004-03-24

47

20


“HUMAN FACTORS refer to people: it deals with people, their working environment and relation with the equipment and procedures. It also refers to the relation between people... and it follows a double objective, wich may be defined as safety and eficiency ” - (ICAO Circular 227).

«MAN SHOULD NOT BE REDUCED TO HIS TECHNICAL HOMO FABER, CHARACTERISTIC , NOR TO HIS RATIONAL HOMO SAPIENS CHARACTERISTIC […]. THE TRUE MAN IS FOUND IN THE SAPIENS/DEMENS DUALITY»

( Edgar Morin, The Lost Paradigm 1991) TECHNOLOGICAL FACTORS refer to automation: it deals with complex systems, it´s systemic environment and facilitation of Human decision capacity. It also refers to the respect of Human cognitive and socio-cultural environment... and it follows a double objective, wich may be defined as safety and eficiency ”

JS/AG -2004-03-24 JS - 2001-05-10

55

22

Thank you very much for your attention .