f/-53 - NASA Technical Reports Server (NTRS)

/f/-53

e7 REQUIREMENTS

FOR PSYCHOLOGICAL TOWARDS ECOLOGICAL

MODELS TO SUPPORT TASK ANALYSIS

DESIGN:-

Alex Kirlik Georgia

Institute

of Technology

Abstract Cognitive engineering is largely concerned with creating environmental designs to support skillful and effective human activity. The goal of this chapter is to propose a set of necessary conditions for psychological models capable of supporting this enterprise. An analysis of the psychological nature of the design product is used to identify a set of constraints that models must meet if they can usefully guide design. It is concluded that cognitive engineering requires models with resources for describing the integrated human-environment system, and that these models must be capable of describing the activities underlying fluent and effective interaction. These features are required in order to be able to predict the cognitive activity that will be required given various design concepts, and to design systems that promote the acquistion of fluent, skilled behavior. These necessary conditions suggest that an ecological approach can provide valuable resources for psychological modeling to support design. Relying heavily on concepts from Brunswik's and Gibson's ecological theories, ecological task analysis is proposed as a framework in which to predict the types of cognitive activity required to achieve productive behavior, and to suggest how interfaces can be manipulated to alleviate certain types of cognitive demands. The framework is described in general terms, and illustrated with an example from our previous research on modeling skilled human-environment interaction.

Note: This paper is a draft of a chapter for a forthcoming book, The Ecology of Human-Machine Systems. J. M. Flach, P. Hancock, J. Caird, and K. J. Vicente (Eds.), Hillsdale NJ: Erlbaum. Please do not cite or distribute. Comments are welcome to: [email protected]

This research Technology,

was supported by NASA Ames grant NAG2-656 Robert J. Shively, technical monitor. (NASA-CR-1903a_) PSYCHNLOGICAL

to the Georgia

Institute

of

ReqUIREMENTS F_R HOOrLS TO SUPPORT DESIGN:

T"_ARDS £COL_CICAL TASK Inst. of Tech.) 43 p

ANALYSIS

N92-25732

(Georgia c,3/53

onclas 0091207

INTRODUCTION Modernpsychologyjudgesits progressandproductsby a variety of number

of paradigms

community theory

gives

in current

no less that thirteen

when

you see one?"

computational,

evolutionary)

successful activity

even

thirteen

criteria

individual

research

science

efforts

in certain

formalization model

processes

have emerged

could

this divide

and conquer

paradigms

meets

resulting

array

paradigms Perhaps

its own goals,

but rather,

that comprise

anything,

will have

problems

which

constraints

square

increased

along

overall

number

meeting

of the journeys of constraints.

multiple

Knowing

the least expensive

and the one with the healthiest

menu

and success

able to integrate

efforts

is whether

diverging

between

of

the

in a way that will the many

research

or converging however,

paths.

that although

different

successful hills,

us. A coherent,

however,

do good

by decomposing

restaurant

is not particularly

condition

each of the many

may lead to individually

emerge

concern,

theory.

Rarely,

constraints

paralyzing

The eventual

we may find we have all climbed

the difficulty

path so that

a necessary

on whether

I am concerned,

criterion

most of

mechanisms

or evolution.

concern

are moving

where

is the primary

cognitive

our research

One primary

Research

set of otherwise

we are somehow

decomposed

science

search,

and in still others

and coherent

to a single scientific

to meet a large

themselves.

in town,

helpful

and if

useful

cognitive

solutions

to

the problem

via the

the one with the best

in allowing

one to find a good

meal at a fair price.

The purpose capable

adherence

involve

on whether

are judged

of merit.

realization

not so much

best left for time to decide.

when considered actually

hinges

we have

unification.

in others,

development

into useful

cognitive

this is a question

hillclimbing,

food,

whether

theoretical

strict and dogmatic

theory

products

by a diverse

that the proposed

human

of course,

programs

on any one search

computational

constraint

that the research

(e.g., experimental,

parallel

are suspended

a

do you tell a good cognitive

he mentions

a massively

Reviewing

suggests

"How

so that research

unencumbered

venture,

to wonder

allow for eventual

success

through

(1988)

only one of these standards

may be a demonstration

of research

It is natural

adherents

paradigms

the major

for a theoretical

criteria

thus resembles

can proceed

Claxton

to the question:

meet perhaps

for scientific

For example,

mathematical

answers

has enough

cognitive

Claxton's

psychology,

Each of the thirteen

if their products

in current

constraints.

cognitive

criteria.

of this chapter

of supporting

the design

i.e., cognitive

engineering

1986; Woods

and Roth,

is to identify

of environments

(Fischoff, 1988).

a set of necessary

Slovic,

This effort

to promote

and Lichtenstein, is motivated

conditions skillful

for psychological

and effective

1978; Norman,

by my own limited

human

1986;

success

models activity,

Rasmussen, in attempting

2 to applytheproductsof cognitivescienceto cognitiveengineering.My experienceshaveled beto believethatthecentralproblemthatneedstobeovercometo maketheproductsof cognitivescience morerelevantto designis identifyinga moreproductivesetof dimensionsalongwhich modeling effortscanbedecomposed.We simplymustdecompose thecomplexproblemof cognitive modelingin orderto makeanyheadway.However,for cognitiveengineeringatleast,the decompositionmustbederivedfrom anoverallframeworkcapableof ensuringthattheresulting researchproductscanbereassembled into a coherenttheoryusefulfor design. A descriptionof a solutionto anyproblem,evenif expressedonly asa setof necessary conditions,playsa crucialrolein formulatinga problemdecompositionstrategycapableof ensuringthatthe subproblemsolutionscanbeeffectivelyintegrated.We haveto knowwherewe aregoingif we wantto get there. In termsof thepreviousanalogy,we haveto know thatour goal is a goodsquaremealata fair pricein orderto determinehowto decomposetheproblemof finding anappropriaterestaurant.A necessarysteptowarda moreapplicablecognitivescience,therefore, is a statementof the thesetof constraintsthatmustbemetif a psychologicalmodelis to support design.Theremaybeno goodreasonto expectthatthesetof constraintsthatmustbe satisfiedto supportdesignareidenticalto the setof constraintscognitivescientistsnormallyuseto guidetheir scientificexplorations.In fact,I will suggestbelowthatthenecessaryconditionsfor an acceptable psychologicalmodelin cognitivesciencearequitedifferentthanthenecessary conditionsfor a psychologicalmodelcapableof guidingdesign.Many of thedifficulties involvedwith trying to applycognitivesciencemodelingariseoutof this mismatch.Cognitivesciencehassimply decomposed its centralproblemin a mannerthatis very unfortunatefor thecognitiveengineer. As Carroll(1991)hasnotedin regardto thefailure of psychologyto meaningfullycontributeto understanding theproblemof human-computer interaction(HCI),therealizationthattheproducts of a basicsciencedo notprovideeffectiveresourcesfor applicationcanprovideimportantlessons for the basicscienceitself (alsoseeFlach, 1990b;Gibson,1967/1982;Neisser,1976). The solutionto theproblemof creatinga scientificbasisfor cognitiveengineeringis not merelyoneof improvingthe designer'saccessto researchfindings(e.g.,Meister, 1989),moving researchinto naturalisticor operationalcontexts(e.g.,Klein, 1989),or improvinggeneralizablityfrom experimentalresults(e.g.,Hammond,Hamm,andGrassia,1986),althougheachof thesegoalsis surelyimportant.Rather,I amconvincedthatthesolutionmustlie in a reformulationof the questionsposedby thebasicpsychologicalresearchitself: areformulationdrivenby an understanding of thepsychologicalnatureof thedesignproductandtheknowledgethatis required in orderto createit.

THE

PSYCHOLOGICAL

A standard modeling fixed

approach in cognitive

and to create a description

environment.

The designer,

environment

consistent

with the world. cognitive

environments. cognitive

solving

while the solution

the difference though,

between

of cognitive

function,

and software

just as it would

software

or ink in which

As Carroll

theories

however

psychological nonetheless principles understand

implicit

underlying

(1989)

way of

distinction

While

the scientist

not theories

creates

but merely rather

theories

environments:

than in psychological

view of what the environment

Although

a design

product

is, from

may be implemented

to view the relevant features

design.

is a lack of

restrictive

level at which

have noted,

of how the environment

each design influences

and fragmentary

features

of a psychological

in the design

its design.

Similarly,

you had better know be looked

to look for the relevant

of the VCR,

of that theory

in the

you had better

if you want to understand

at as an assemblage

structure

of the machine,

theory

are real. know

the (good) it is

engineering

If you want to

the operative

or predict

matter,

of

and skillfully

for example,

and electrical

underlying

of physical

think,

Although

VCR interface,

theories

an instantiation

behave,

may be.

of the VCR

the psychological

is actually

how people

of a (good)

of the inner workings

the functionality

product

these theories

the same sense that the physical

in the design

the VCR can of course

and theoretical

and environmental

in technological

to look for the relevant

is only implicit

or predict

problems

it is realized.

real in exactly

with the VCR,

which

be wrong

rudimentary

theory

behavior.

this is the wrong

and Campbell

the designer's

human

feasible

psychological

empirical

psychology

product.

that is best conceived

interact

as a search among possible

sophisticated

creates

that is

is the set of plausible

does not appear to be a faithful

of the design

an

as a search among possible

of the theoretical/applied

on an overly

of a design

and effectively

are symmetrical

of cognitive

that the designer

is based

and software,

product,

perform,

nature

of understanding

in hardware

dichotomy

the creation

in that

of creating

is the set of technologically

problem

These

relatively

behavior

challenge

skillfully

problem

requiring equally

the practices

it is assumed

But this perspective

the standpoint

theories

solution.

for the perpetuation

for the psychological

a mix of hardware

a person's

goal being

for the designer the scientist's

PRODUCT

space for the scientist

to a given task, and the designer's

appreciation

terms.

space

subtlety and difficulty,

One reason,

underlying

how people

terms, the solution

In this sense a theoretical/applied

portraying

activities

underlying

to obtain a given cognitive

of comparable

DESIGN

is to hold a task environment

with the ultimate

We can thus characterize

environments

methods.

behavior,

with the principles

"solutions"

THE

on the other hand, is faced with the opposite

In problem

theories

OF

psychology

of the cognitive

to elicit a desired

maximally

NATURE

physical

human

its design

interaction

as well.

this is the wrong

either for understanding

While level at

electromechanical

4 functionor for understanding userinteraction.Boththe electricalengineerandcognitiveengineer structurethephysicalmatterusingorganizingprinciplesderivedfrom theorieswithin their own disciplines.It is theadequacyof thesetheories,ratherthananyfactssolely aboutthephysical form of themachine,thatdeterminewhethertheVCR will play, andwhethertheusercanplay it. Eachinstanceof humaninteractionwith anyartifactis thusapsychologicalexperimenttesting theassumptions embodiedin theenvironmentaldesign(of.Wise's (1985)construalof an architecturaldesignin termsof scientifichypotheses).Althoughit may befashionablewithin the cognitiveengineeringcommunityto bemoanhowlittle guidancemodernpsychologyprovidesthe designer,thepsychologicalnatureof the designproductis inescapable.The correctresponseto thecurrentandunfortunatelackof applicablepsychologicalresearchis not toattempttodo psychology-freedesign(sincethis is impossible-- the designwill not beapsychologicalbut insteadreflect thedesigner's"folk" psychologicaltheory),butratherto askwhatkindsof psychologicalmodelsareneededto supportcognitiveengineering,andto beginthe long range empiricalandtheoreticalwork necessary to realizethem. This is not to imply, of course,thatcognitiveengineersshouldwait for a moreapplicable psychologyto emergebeforemakingdesigncommitments(this wouldbehopelesslynaive),or shouldturn all attentionto modelingandawayfromthe designof prototypesandexpanding technologicalopportunities.It maybe,asBraitenberg(1984)hassuggested, thathumancapacities for synthesisfar exceedcapacitiesfor analysis,in thesensethatourcreativeproductsreflect a degreeof implicit or tacitknowledgethatis far moreelaborateandrich thantheknowledgewecan explicitly stateandformalize. Thedirectmanipulationinterfacewasnot deducedin anyinteresting sensefrom psychologicaltheory-- in fact, a casecouldbemadethatwe still haveno unified psychologicaltheorythatwouldpredicttheprofoundsuperiorityof directmanipulationover commandinterfacesfor varioustasks.It just sohappenedthatin thiscasethedesignerhadan implicit understanding of howpeoplenaturallyinteractwith theworld thatwascloserto thetruth thananyexplicit andformalpsychologicaltheoryavailableatthattime. While cognitive psychologistsmaybe ableto identifywhy command-lineinterfacesareinefficient in variousways, it wasthe designersandnot psychologicalresearchthatpointedtowardenvironmentsthatsupport moreefficientinteraction,a finding thatshouldprobablyinform psychologicalresearchitself. Thereis a catch,however,to this design-as-research strategy.Assuminga particularprototype of a designconceptis successful,anyusefulgeneralizations thatemergefrom creatingthe prototypewill beat thelevel of thepsychologicalassumptions underlyingthedesign,ratherthanat thelevel of theparticulartechnologiesusedto implementthedesign.To returnto apoint made earlier,thehardwareandsoftwareimplementationis the wronglevel to look for therelevant

featuresof thedesignproduct. Especiallyin HCI, a vastamountof researcheffort hasbeen expendedtrying to answerquestionscomparingvariousinterfacetechnologies,for example, designoptionssuchas scrollingwindows,hypermedia,etc. This researchis of dubiousvalue(see alsoVicente,this volume),sincethe"it depends"answersproducedby sucheffortswill only lead to a neverendingseriesof technology-specific designprinciples,ratherthana stableandgenerative theoreticalaccountof human-environment interaction that can guide design in novel situations. To profit make

from the design-as-research

explicit

the psychological

strategy,

assumptions

system.

A successful

system

demonstrates

implicit

psychological

theory

underlying

to articulate

the theoretical

traditional

experimental

be more efficient surely

than analysis

rides on whether

were validated

strategy

must include

implicit

psychological

of the prototype.

for this purpose.

essential

argued

to that faced

guiding

and behavior

accounts

of cognitive task).

There

designer

and scientist

the other

reasons

heuristic

guidance

A search

is, however, search

over

task) is no more

in opposite

cognitive

models.

will be necessary

guidance

to the cognitive over environments

cognitive

itself

aid to design

an applied

by particular between

by discussing

activities.

will have different

faced

comparable

mode

of cognitive

than is a search

for

environments

(the

these two problems.

The

models

that different

while

types

of

As a result, the theories

properties

Let us examine

might differ

the

problem

a particular

endeavor

in the two cases.

engineer.

the

that the problem

a theoretical

to promote

difference

search

where

and communicated.

therefore,

heuristic

across

tested,

phase

be expected,

best provide

reasoning

the design-as-research

It should

to direct

that

to traditional

over environmental

to the scientist

support

using

hypotheses

one reasons

guidance

reasoning

such demonstrations

directions;

heuristic

best support

from

DESIGN

that are promoted

best provide

would

may

by an analysis

explicit,

but is instead

an important

-- synthesis

and also by showing

for environments

and behavior

progress

no alternative

is a necessary

product,

by

is probably

followed

the researcher

be demanded

impede

program

the possibly

(as would

conclusions

SUPPORT

theory

forcing

the psychological

is made

TO

application

(the designer's activity

phase

synthesis

of the design

is not one of mere

activity

scientist's

nature

by the scientist.

design

to

of the prototype unless

identify

A research

that a good psychological

psychological

by the designer

There

both an initial synthesis theory

Although

may actually

can subsequently

the researcher

than its own success,

is articulated.

methodology)

MODELING I have

other

prior to environmental

the researcher

methodology

nothing

upon

most to the success

-- the hope for generalizable

by the success

experimental

that contributed

the design

assumptions psychology

then, it is incumbent

than the theories how the theories

from the theories

that would

that that

that best

6 Much of ourcurrentunderstanding of cognitive-levelhuman-environment interactionconsists of a setof somewhatindependent triples,eachof which provides a psychologicalmodelof howapersonmight achievea particularbehaviorin a specified environment.Whenthedifficult but importantjob of integratingthisknowledgeinto coherent theoryis attempted,theseeffortstypically focuson integratingacrosstheprocessandbehavior dimensionsratherthanacrosstheenvironmentdimension.The resultis thatunderstanding the environmentalcontributionto behavioris a largelyignoredcomponentin the theoreticalunification. Oneapproach,for example,to achievingtheoreticalunificationof this setof triplesis to integrate acrossthebehaviordimension.Theresultsherearepowerful,typically hybrid "cognitive architectures"(CardandNewell, 1989).Thesegeneralpurposecognitiveframeworkshavethe processingresourcesto producea wide varietyof behaviors,from simplemotorresponsesto complexproblemsolvingandplanning.Yet anotherapproachis to integrateprimarily acrossthe processdimensionin anattemptto showthatthefunctionalityof a wide varietyof existingmodels canbesubsumedundera singleprocessmodelingformalism. Cognitivemodelsdemonstrating howsymbolicprocessingtechniquescanbeimplementedusingneuralnetworkor connectionist formalismsaregoodexamplesof partialtheoreticalunificationalongtheprocessdimension. Theoreticalintegrationsalongtheenvironmentdimension,however,arehardlyeverattempted but arecritically neededto supportthecross-environmental reasoninginherentin design. It should not comeasa surprisethatmostcognitivepsychologistsarenot overlyconcernedwith this typeof theoreticalunification,sincean acceptable scientificproductis a modelof behaviorin a specified environment,andrarelyis reasoningbackwardsfromcognitivetheoryto environmentrequired. Exceptperhapsin experimentaldesignitself,rarelyis the cognitivepsychologistforcedto reason acrossenvironmentsin orderto activatespecifiedcognitivemodes.Significantexceptions(i.e., attemptsattheoreticalintegrationacrossenvironmentalinfluenceson cognitionandbehavior)are Rasmussen's (1986)theoryof multi-levelenvironmentalrepresentation asreflectedin the "abstractionhierarchy,"andHammond,Hamm,Grassia,andPearson's(1987)effortsto obtaina rich enoughsetof environmentalandtaskdescriptorssothatthecognitivemodeunderlying judgmentbehavior(e.g.,analytical,intuitive) canbepredictedandpromotedthrough environmentalmanipulation.Only aunifiedtheoryof theenvironmentalinfluencesoncognition canguidethedesigner'ssearchfor environmentsto activatespecifiedcognitiveprocesses. But whatwould a unifiedtheoryof theenvironmentlook like, andwhattypesof guidance wouldit provide?Whatwouldbeintegratedin sucha theorywould bethediversesetof knowledgeof whatthepsychologicallyrelevantaspectsof theenvironmentmight be,for the purposeof trying to understandor predicthumanbehaviorandperformance.Onemust,for

example,determinewhenit will beappropriatetounderstandtheenvironmentin termsof stimuli andreinforcementsasin behaviorism;cues,criterionandfeedbackasin modelsof judgment; options,chancenodes,choicenodesandprobabilitydistributionsasin decisiontheory;initial states,goal statesandoperators as in Newell and Simon's (1972)problem solving theory; affordances

or consu'aints

and differential Each

equations

of the above

environmental behavior

of environmental

appropriate

will do justice

in complex

selecting

an environmental

and behavior

the cognitive

engineer

can operate

concept,

activity

expressed

maximaIIy

engineering

activities

reasoning

knowledge process

is expressed

scientific design

study. in studies

effectively

experiments

of cognition

of knowledge

necessary

experimental

psychology

the fundamental findings,

capable

difficulty

it may approach

to guide research

models

When

of

the problem

that will assist

the central

problem

in which

will be to create mode

of cognitive

for both of these cognitive that support

the designer's

but perhaps

oddly,

of cognitive activating

activity,

knowledge

guiding

mode

design

is that the knowledge needed

of cognitive

Much

needed

the knowledge activity

this

but rather

those cognitive

the designer

to reason

is quite similar

needs goes beyond

to have actually

designed

for

experimental

underlying

1987, Meister,

in the

activities

has been made of the inability

(e.g., Rouse,

of

in predicting

In other cases

a specified

Guidance

successful,

design.

to guide

the knowledge

faces

of a

behavior

as the selection

the type of understanding

a particular system

productive

engineer

design

of such a theory,

of successfully

that approaches

ability to promote

The reflection

models.

in existing

models.

to which

frameworks

and

and each way is suggestive

that promotes

of a task.

of

of cognition

1986).

will promote.

That is, it is the often tacit and unformalized

over environmental

experimenter's

ways,

cycle,

model,

have the rudiments

not so much

state variables

representation

forms

(Rasmussen,

candidate

in the design

with the demands

environmental

may already

of designing

earlier

different

the cognitive

design

can only come from theoretical

over alternative

Psychology

cases,

the environmental

system

No single

in fact, can be viewed

for an existing

as an environmental

consistent

the many

of the environment

In some

description

activity

has its place.

in various

activity,

theory,

and so on.

interaction

the structure

descriptions.

ecological

description

is manifest

A large part of design

the cognitive

design

theory,

to understanding

in behavior

environmental

(1979)

human-environment

that best describes

must be sensitive.

control

of environmental

structure

model

as in Gibson's

as in manual

forms

structure

observed

different

on action

1984),

the to the type of basic but perhaps

the experimental

these experiments.

a

ISSUES Cognitive

engineering

IN

thus demands

theoretical

basis, and the resulting

the models

used to describe

predicting

environmentally

concerned

as much

Gibson,

psychology,

of attention

techniques

internal

cognitive

situated

behavior

1991).

activity.

A cognitive

cognitive

at the types

model.

There

with a strong formal,

psychology

design

activity

model

and precise

capable

will therefore

of models

does the environmental

cognitive

modeling

must be as explicit,

and of supporting

one looks

rarely

as does the internal

models

as with internal

When

however,

MODELING

for environmental

environmental

with the environment

1979; Anderson,

cognitive

ENVIRONMENTAL

of

have to be

(e.g., Brunswik, produced

receive

1952;

by current

close

are at least three reasons

as

to the amount

for this state of

affairs. First, experimental purposes

of control,

environmental

cognitive

literature).

internal

questions

representation

used by expert

of problem

case that researchers

solving working

environmental can remain

unconfound

the environmental

environmental

manipulations

various

performance.

cognitive

To evaluate such as problem

what problems memory

implicit

the open problem

conditions

within

(but see Larkin

with the result within

a given

from the internal

engineering

activities

will be activated,

solving,

design, decision

In the following

will be, given I will discuss

arise due to the need to represent

various

1987).

are forced

program.

constraints

for cognitive

will have to be solved,

demands

research

these

what decisions design

in the types of

representation

upon

rarely

is it the

to reason

across

widely

There

may be no pressure

on cognition

the

Finally,

environmental

and behavior

to

when

range.

is to determine

under what environmental

and required,

and working will have

for effective

memory,

to be made,

for a particular

two types of constraints

on acceptable

and external

start so far

about

concepts

both internal

be the case that

the issue is not only to understand making,

science

environment

accounts

problem

that assumptions

are made over a very narrow

a candidate

(e.g., differences

and Simon,

paradigm

the cognitive

it may indeed

solvers),

model

in the

of the external

While

of the external

an established

within

the description

problem

any influences

in most papers

in research

such an approach

and novice

activity

conditions,

descriptions

processes

using

-- the environmental

model

of the environment.

that they fail to capture

efficiency

and especially

for the

and impoverished

description

psychological

is even made between

can be answered

representations

downstream

Second,

their environments

simplistic

of the stimulus

of the internal

often no distinction

interesting

Thus,

the length

of the description

and the subject's

varying

(compare

experimental

orientation,

often feel the need to simplify

and are thus able to get by with highly

models

with the length

internal

psychologists

influences

task cognitive

but also to determine and what

working

task. psychological on cognition

models

that

and behavior.

--

Thefirst setof constraintsarestructural.I will arguethatthe a model'sstructuremustbecapable of representingbothcognitiveandenvironmentalorganizationin a single,unifiedformat;i.e., that theappropriateunit of analysisandmodelingmustbe thehuman-environment system,ratherthan thehumanalone. Thesecond set of constraints concern the content of acceptable models. I will suggest

that cognitive

understanding

engineering

fluent,

skilled

that rationalize

detached

create

which

a design

cognitively-intensive features target

human

intellectual promotes

control

activity.

fluent

In most cases

and skilled

of behavior.

and also to identify

the causes

of error-prone

the Integrated

Human-Environment attempts

such as steering

electromechanical

to model

control

feedback

control

system

in order

designed

so that control

demands

language

for environmental

described

in terms

outputs

(heading).

case the input As Flach

The human

notes,

schematic

level of description,

invariance

at the organismic

These

engineers

much

human

transfer

adjust

to changes

noted,

the ability of the human

suggest

"that

control

system

in the dynamics

'the human designer

(McRuer

in order

limits.

that relates

system

controller

familiar

be) to

promotes

to define

the

a design

systems.

Control

theory

the human

(steering

could

in similar

to system

terms.

the human

transfer

control shares

function.

Rather,

of the controlled

as empirical

the human system.

to the environmental

function;

is a scientific

and interminable

and Jex, 1967) were only successful system,

rather

quest." once

the search

than in human

system.

which

to

(1954)

was so great

as to

can lure the

Subsequent

for invariance behavior

that the

appeared

and Taylor

function

(p. 1752)

At this

indicated

function

will-o'-the-wisp

i.e.,

this goal of finding

As Birmingham transfer

command.

behavior.

results

transfer

In this

be a steering

than at the level of the organism-environment however,

be

can be

adjustments)

would

modeling

as a

has a weIl specified

automobile)

manual

of

so that vehicles

(airplane,

inputs

control

with the design

and limits

during

manual

to modeling

was to discover

psychological

function'

models

that capture

concerned

and the output

current

to the level of the human-machine

which

in current

can be described

to response

into a fruitless

activity

"plant"

stimuli

to adjust

transfer

than a design

performance

capabilities

these

were in for a rude awakening

was no single

rather

Engineers

The controlled

level, rather

goal is (or should

interaction

of the automobile

relating

the design

their attention

human

the goal in this endeavor

of the function

than environmental

cognitive

turned

as a feedback

might be the heading

(1990a)

a description

to assess

function

rather

models

skilled

an aircraft.

were within

of a transfer

that assist in

System

systems

modeling.

models

environmental

human-machine

a car or flying

feedback

activity,

We require

effective,

behavior,

shifted

with the world,

that promote

One of the earliest

attempts

interaction

of environments

Modeling

there

is most in need of environmental

alone.

modeling in behavior The

10 crossovermodelof humanmanualcontrolbehaviordevelopedby McRuerandhiscolleaguesis a statementof behavioralinvarianceat thelevelof thehuman-environment system. Why shouldthis finding concerninghumanperceptual-motor behaviorinform our discussionof cognitive-level to expect

human-environment

that cognitive-level

perceptual-motor

flexible,

situation,

In fact, there

and the one pursued as an integrated

for behavioral

expressing transfer

cognitive

function

are a variety

interaction

representations

used to model

less constrained,

model

formalisms

controller

is

to this

both human

and

as a tool in the

capable

in the same language,

both the manual

than is

and thus

response

is to describe

requires

on behavior

structure

that just the opposite

human-environment

such a strategy

constraints

is even

researchers,

unit, and to use this unified Pursuing

to believe

Note also that the correct

control

to be little reason

to environmental

of reasons

interaction.

and external

is that there appears

with the world

by these manual

invariance.

both intemal

The answer

will be any less adaptive

than is perceptual-motor

environment search

behavior

activity.

that case; i.e., that human more

interaction?

of

such as the

and the controlled

system. Although

rarely

level behavior. (Brunswik,

The Lens

1952;

environment. cognitive

used, this approach

on judgment

performance

book

resources

toward

external

environmental

perform

a particular

When

structure

combine

environment descriptions

systems, modeling already

indicate

of how both internal

of understanding

Bennett,

and

for

and Joyce,

framework

and the

consistencies

mechanism

modeling.

1988).

also represents

This framework

"computational"

the processes

approach

being

constraints

that will be engaged

proposed

analyses

However,

for describing understanding of skilled

of environmentally

1988) activity

work remains

will be possible,

to the production

In a pair of penetrating

the kinds

and Holtzblatt,

it is clear that much

a clear shift toward

mutually

some

Analysis system

to determine

as to a number

(e.g., see Brehmer

human-environment

judge

both the environmental

without

the Rational

judgement

and to

task.

1987; Whiteside,

human-machine

to behavior

cognitive-

of human

of both the human

and has been enlightening

describing

the question

we turn to the problem

(Suchman,

contribute

integrated

to address

description

likely not have surfaced

(1990)

to understanding

and analysis

tool in understanding

contribution

by Anderson

step in the direction provides

is a unified

abilities,

out the environmental

The recent

1955)

that would

applied

for the description

As such, it has been a fruitful

in judgment

systems

framework

1956; Hammond,

constraints

partialing

Model

has been successfully

dynamic

typical

"situated" of behavior

to be done before the kinds human

in modem

an integrated

human-

of psychological

interaction

how both environmental

with technological and cognitive

structure

behavior.

of the cognitive-level

ecology

of human-machine

systems,

a

11 Hutchins(1987; 1991)

has suggested

apart from the external

devices

that human

cognition

in the environment

In ship navigation

checklists,

and calculators

can sometimes

be used in lieu of memorial,

operations;

and in modem

aircraft

for memory

of intended

the cockpit.

In such environments,

and environment. describing

both internal

of information

processing

environment

as external

The importance of modem

and limitations, human work

cognition

some

there

possible

mathematical

simply

not be possible,

theoretical

routine

of intimacy hypothesis behavior.

that intensive As Norman

With

thus be difficult

and perhaps

(in preparation) intellect,

amount

activity

than by increased

exploitation

a disembodied

a considerable

rather

Skilled

of environmental

terms.

We have

cognition

of the

be underestimated. cognitive

abilities

be possible

does even

for

the scientist

is out there,

in the diagram

of external

analog

cognitive

measurements,

thoughts

itself.

props

of knowledge,

--

and technical

of this kind would reached

The locus

by the mind

of a process

to any single

like

part of the internal-

(pp. 378-379).

an airplane,

force,

to understanding

driving

is often accompanied

detachment, structure

by a heightened

an observation plays

from the world,

intelligent

lots of deep planning

the

a car, or performing

that leads to the

a key role in productive

has noted: isolated

in terms

the kind of synthesis

in even greater

such as flying

requires

has noted.

to support

to attribute

up such a system.

activity,

(1991)

itself.

systems

of human

or "conclusion"

representation

cannot

But rarely

these things,

the end-state

tasks we find in daily life. with the world

histograms,

both person

and the like.

over how it can even

the synthesis

Without

also apply,

skilled

schema

within

as seen in the description

picture

upon a huge variety plots,

because

that makes

would

underlying

curves,

by the external

network

dismal

as Donald

wiring

heavily

would

tools,

across

of human

and behavior

achievements.

diagram;

at a theory.

synthesis

The same comments

many

cognitive

notations,

-- to arrive

mechanisms

in its modem

depends

directly

a rather

burden

structures

compatible

representations,

aids to thought paints

is no internal

jargon

external

problem

by a scientific

The theoretician

is driven

external

memory

these integrated

the world,

of us in a state of wonder

from external

For example, made

back upon

research

to have resulted

in isolation

turned

and

of the cognitive

is distributed

in mutually

cognitive

with notepads,

procedural,

to external

function

functions

to perform

1991), much

allocated

be understood

interaction

that gave rise to a description

of these environmental

leaving

(Hutchins,

cognitive

cognitive

has been

psychological

human

then, that understanding

metaphor

memories,

for example,

has been

the entire

and external

the computer

1987),

speeds

It is not surprising,

come full circle:

Much

and current

cannot

that have been designed

functions.

computational

(Hutchins,

and behavior

behavior

and decision

requires making,

the level

12 andefficientmemorystorageandretrieval.Whentheintellectis tightly coupledto theworld,decisionmakingandactioncantakeplacewithin thecontextestablished by thephysicalenvironment,wherethestructurescanoftenaid asadistributed intelligence,taking some of the memory and computational burden off the human. (Chapter

10, p. 6)

The human-environment internal

cognitive

environmental through

system

activity

design,

many

that promotes fluent

fluent

as opposed

existing

for effective

aircraft

an "opaque

veil,"

human

example.

and Bohlman

theory

would

cognitively

intensive

modeling

are observed

however, interaction problem

better

and airspace

problems solvers

(Woods

or decision

evolved.

processing work

theories,

it seems

safe flight.

It is natural

What

the situation

commercial as

an active

that it is appropriate

to

could

to view such attempts to create

1988), and training

has yet to be proven

accounts

of training

of activities guide

the

methods

with suspicion,

their own set of human-machine

to make people

effective.

Because

and mental

presumably

tasks, or the design

aids have the potential

problem?

generation,

that better

Such accounts

to think that

kind of psychological

awareness

hypothesis

only natural

in their theoretical

and Roth,

of

adaptation.

the cockpit

I find it most unsettling

inference,

interaction.

thinkers.

the acquisition

in the modern

is so great

is necessary

effective

of maintaining

displays

In

are quite unlike

As a result,

1991)

that

and described.

has gone so far as to describe

for remedying

systems,

makers

of environments

the operators

and Woods,

to ensure

such as inductive

solving

an environmental

of their situations.

necessary

flight crews

to create

cognitive

that the difficulty

a living constructing

in current

since problem

abilities

from cockpit

"theories"

activities

theoretical

in which

(1988)

suggests

the most leverage

of aids to assist

make crews

of

can be engineered

need to be identified

that complex

(e.g., Sarter

such as these are the key to enhancing design

activity

the features

behavior

the reason

Edwards

are sometimes

provide

intensive

(1979)

as constructing

abilities

the

as a function

and the end state is one of only partially

awareness"

As one who tries to make theoretical

behavior,

psychological

is impeded

of the aircraft

of crews

skilled

systems,

in which

is a prime

understanding speak

and effective

of "situation

cockpit

cognitive

is often (but not always)

is that the environments

of behavior

The problem

to be predicted

to allow

with the Worm

engineer

to cognitively

performance

modes

behavior

how necessary

Interaction

human-machine

those environments fluent

of Fluent

the goal of the cognitive

support

for productive

and modeling

manipulation.

The Need for Models

design

necessary

and also to identify

environmental

Since

must serve as the unit of analysis

more "rational"

to

13 The alternativesolution,of course,is to designenvironmentsmoreconsistentlywith the principlesunderlyingskilled,dynamichumaninteractionwith theworld. Pursuingthis strategy, however,requirestechniquesfor environmentalmodelingcapableof representingthe featuresof taskenvironmentsthatbothpromoteandinhibit theacquisitionof fluentmodesof behavior. Only modelsof productive,skilledbehaviorcanprovidetheresourcesfor a task-analyticapproach capableof identifyingfeaturesof anenvironmentaldesignthatareinconsistentwith theprinciples underlyingskilledactivity. Theproblemof identifyingdemandsfor complexcognitiveactivities suchasproblemsolving,planning,decisionmakingposedby a givenenvironmentaldesignmost requiresmodelsof skilled, fluentbehavior,not making

rational

action

models

considerations

Due to their roots

than they are with necessity to describe

and perhaps

all, behavior

process

and often prescribe

such as search

comparative evidence

considerations. behavior

space,

planning,

or artificial

when

The great

or decision

the result

these

performance

inference,

requires

models

Nearly

any,

or the

function.

of these rational

sorts of complex

is

cognitively-intensive

or utility

the sufficiency

of such models

of situations.

of some

to a goal structure

intelligence,

with sufficiency

appeal

hypothetico-deductive

to bear on limiting

identifying

theory

in a huge variety

as being

with respect

for successful

cognitive capable

No empirical methods

activities

of indicating

for

will when

such

are not necessary.

My observations hypothesis

of skilled

human

that cognitively-intensive

i.e., when

effective

thus requires

solutions

will not be available,

skill.

I realize

solutions

to cognitive

However,

intuitions

experiments

the cleverness

in complex for action

solutions approaches

capable

mainly

of behavior of subjects

on laboratory carefully

task demands.

in operational

findings designed

Predicting

when effective

may be skewed

in the graphics

of perception-

counter-intuitive. by the fact that

the availability

my own intuitions

demands

perception-action

opt for perception-action

as being

to preclude

cognitive

by a theory

will typically

even in the laboratory

who are able to short-circuit

off the whir of a disk drive or an aberration

are used only as a last resort;

available.

psychologist

Although

settings,

have led me to the working

can only be provided

that skilled performers

are typically

for meeting

selection

of defining

tasks may strike the cognitive based

systems

are not readily

and this knowledge

that the claim

on cognition

shortcuts

observations

modeling

behavior methods

perception-action

demands

cuing

a problem

of options

However,

be necessary

activities

through

ever be brought

selection.

actually

action

solving,

above are more concerned

can be rationalized

evaluation could

in either economic

such as those mentioned

their ability

action

of problem

activities.

Why is this the case?

action

models

of perception-

are largely

based

I am continually

for complex software.

cognitive I have

on

amazed activity

ceased

to be

at by

14 suprisedandfrustratedby suchcleverness,andhavebegunto view thetendency

toward

perceptual

is no doubt,

selection

however,

of action

as a fundamental

that more empirical

experiments

must provide

and in more

environmental

however, everyday

work

settings.

observations

of the behavior

local area grill.

My interest

in this behavioral

appeared

observational

to possess studies

environment

many

can be readily or by initiating

intervention

and control

In the environment and ovens,

preparation

items within

this type of skilled

to be taking

contemplative

What cook's

First, there

behavior

allows

some

activity.

arose

human-machine

working

because

I have made

busy rush periods

skilled

I have observed

based on the external

selection

based

internal

representation

performance

in my more

at a

in this

limited

24 hour access

And by making

to this

a well timed

limited)

itself automatically and is always

of automated

capacity

action

though,

of intimacy

for

of unpredictable

and in parallel, veridical

maintenance."

(also

serves

capable

Nado,

and

to engage

in

dynamically,

events.

is visible.

Note that the The most

one: the grill area itself.

economies

as compared

The environment of being

as an external

see Reitman,

with the world

to be productive?

of the environment. memory

the following

and arrive

all of this structure

has considerable

as an external

overwhelming.

if ever does the cook appear

for the cook to use is an external environment

simultaneously.

perception-action

contact

are uncertain

such as

the

appear

in the cook's

of interaction

but nearly

orders

and I believe

tight perceptual

Rarely

mode

multiple

and the task demands

interaction, degree

devices

at the grill, to coordinate

while preparing

to a number

representations

serves

activity

Task demands

structured,

selection

care of its own "truth

for rational

intensive

representation"

consistent,

each order,

must be sensitive

"problem

updates

cooks

of a cup of coffee.

manual

sort of action.

cognitive

is highly

problem

Such laboratory

the past two years

and also because

The cook maintains

this perceptually

upon

settings,

over

the cook uses an assortment

is an intensive

environment

efficient

situation

human-environment

with the environment.

and ongoing

time so that

are made available.

day in both complex

of short-order

with substantial

Observe

detached,

the necessary

practice

with the cook, one even has an (albeit

I have studied,

this task in any formalism

seems

However,

over task demands.

Describe

always

every

for the price

conversation

of the many

interaction

There

and enough

As an example,

operational

secured

combined

will be apparent.

conditions

of the same properties

in complex

food order

fryers

this issue.

task solutions

are performed

fairly extensive

setting

behavior.

conducted.

"experiments,"

systems

of skilled

to clarify

and perception-action

are rarely

These

is needed

rich enough

both cognitively-intensive experiments

research

aspect

the

perceptually

memory

and Wilcox,

Action

to action considered

as a

accessed,

store, is internally 1978).

The world

takes

15 When

uncertainties

information through

do occur

underspecifies

using such an external

constraints

on activity),

perception-action

rather

of a steak?

Flip it and see.

underside

the environment, solutions

and thereby

of meats

external

memory

information

in order

of orders

"displays"

variable

to covary

flipping

meats

doneness

And

reduce

cognitive

of doneness,

with a visible

so that the doneness

of the underside.

not only structure

How

well cooked

uses the structure

is the

already

present

in

the cook may organize

or plates 1988),

to serve

environment

the cook may adopt

of the top side can always

generate causing

the strategy

new

a hidden

of continually

be used as a reliable

the cook is both performer

the

as a temporary

and may even

in the controlled

In a very real sense,

can often be resolved

For example,

(also see Beach,

For example,

available

in order to make perception-action

burdens.

constraint

one.

knowledge.

may lay out dishes

to be prepared

by introducing

stored

create

(i.e., perceptually

these uncertainties

the cook

he or she can dynamically

available

placement

than accessing

representation

indicator

and on-line

of the

interface

designer. Skilled

human-environment

structure

as well as a source

environmental similar

interaction

structure

cognitive

created

efficiencies.

organization

enablement displays

used in toasters self-produced

and ovens

processing

structured

demands

the acquisition

information.

in promoting

A model

itself,

contributed economies

Because

through

resist

mechanisms

of skill acquisition

and thus increasingly

and

of the possibility

internal

how the actor's

the

mechanisms

skills will always

efficient

and

at one

memories

role to the timing

of such situated of more

in the static

the external

to the wor/d.

also have to describe

by activity

in cognitive

cognitive

a role

and is thus short-lived,

whether

For example,

plays

in dynamic

external

informative

for

environment to the actor,

of skill development.

of the responsibility

action

mechanisms

skilled,

dynamic

perception-action

to the task.

of the development

would

itself

"on-line"

result

The

strategies

process

of structure,

by the cook,

memory

structure,

interaction

increasingly

over the course Much

to offload

set of environmental

human-environment becomes

But both forms

solutions

solely in terms

a fixed

happens

by the cook play a similar

environmental

description

mereIy

exploited.

and routinized design

to environmental

of the grill area and is thus reflected

or continually

created

own "tricks"

structuring

of the design.

to be subsequently

the environmental

the design

of perception-action

dynamically

faithful

through

during

point in time by the designer

structure

by the cook's

The former

while the latter happened permanent

of environmental

created

to the structure

of this type is thus both a response

for dynamic

at the interface interaction solutions

relies

between

human-environment the performer

upon abilities

to tasks,

and where

to exploit

interaction

lies in the perception-

and the world.

The development

environmental

structure

none naturally

exist,

to create

to obtain

additional

of

of

16 environmentalstructurein sucha wayasto enableperception-action solutions.If suchstructureis not providedby thedesigner,theperformerwiU seekto createit throughactivity thatintroduces newformsof structure.Theproductivityof thismodeof behaviorrequirestheavailabilityof sourcesof informationto specifytheenvironmentalconstraintsto which behaviormustbe sensitivein ordertobeeffective,andtheavailabilityof actionscapableof bothchangingthe environmentandof creatingadditionalsourcesof informationto furtherenabletheperceptual guidanceof activity. Theseare,I believe,featurescommonto nearlyall environmentsin which the acquisitionof fluent,dynamicinteractionis observed.They arealsofeatureslackingin themany technologicalenvironmentsof interestto cognitiveengineering,duelargelyto interfacesthathighly restrictperception-action accessto thecontrolledsystem.Theabsenceof suchfeaturesis one majorcauseof thedifficulty of acquiringskills in suchsystems,andthereasonthattheendstateof learningis oftenoneof only partiallyeffectiveadaptation. TOWARD The previous psychological

discussion models

methodological above.

Many

identified

are suggestive

by Brunswik's

particular,

Brunswik's

and modeling, Hammond, specification cognitive which

1955),

on taking

and Gibson's

engineering

might offer

familiar

with Gibson's

for cognitive views

to cognitive

engineering

environment

interaction

The ecological

(1966,

theories

1979)

as purely

of skilled

(Brunswik,

nicely

interaction

of In

as the unit of analysis 1952;

can be described

blend

was

explanations.

system

identifying

to support

design.

it is important First,

perceptually might

a

1956;

as perceptual

with the claims

with the world,

that

and models

as the unit of analysis.

modeling

especially,

approach

of how knowledge

framework

(1979),

outlining

that have been

on psychological

interaction

initial steps toward

does not require.

by Gibson,

engineering.

on activity

for

to human-environment

the human-environment

system

I will take some

models

approach

in the Lens model

constraints

conditions

of the gaps in our knowledge

for psychological

constraints

is most in need of models

approach

the some

focus on how fluent

take the human-environment

as discussed

important

best represented

of environmental

In the following

action,

can provide

of necessary

It is time now to turn toward

that an ecological

and Gibson's

emphasis

perhaps

features

tools for cognitive

(1952)

a number

design.

to address

of the possibility

pioneered

structure

environmental

of the necessary

may yield fruitful

PERSPECTIVE

on identifying

with the potential

interaction

environmental

ECOLOGICAL

has centered

to support

strategy

discussed

AN

surely

opportunities However,

to first discuss

it does not require guided

activity.

be possible

and for readers

what an ecological

that we conceive

Direct

although

the ecological

perceptual

it is likely

already approach

of all human-

guidance

of

that it is specific

to

17 thoseinformation-richenvironmentsin whichperceptionevolvedor to artifactualenvironments designedto mimic suchenvironments.Thereis noreasonto expectthatevolutionanticipatedthe modernaircraftcockpitor the wordprocessor.In suchenvironmentstheneedfor post-perceptual processes suchasproblemsolvinganddecisionmakingis quite likely. The ecological and information

processing

both contribute Second, studying

fluent

human-machine

arguing

for using

These

in which looks

environments,

much

(1952)

be described

Hammond,

The relations validity. judgment

the other

side.

the organism

of them,

the use of abstract often

found

engineer.

are the places

before

in him, was

that are representative

And for better

fashion,

tO

as a description using

or worse,

terrestrial

where

of

a cockpit

or

environment.

For this reason,

these

the ecological

represents

1975).

The organism

in which forms

concepts

This framework and environment

As shown

has available

criterion

approach

and vary in cue utilization. in that each concept

has a number

1 about

here

mutually

(xi's) which

bear

diagnosis).

and vary in ecological

use of the cues

(rs, i's) to arrive

at a

The framework

is an expression

of

properties

terms.

is a

structure

(e.g., a medical

on one side of the model

of attractive

Insert Figure

a set of cues

forms

(e.g., see

1, the Lens model

and cognitive

to be judged

makes

and environment

concepts

in Figure

may take various

the organism's

in compatible

of parallel

how both environmental

an environmental

the ways

of how the human

the principle

the the cues and the criterion

of parallel

study

to

out.

model

performance.

may take various

the principle

occurs.

a commitment

like Brunswik

than it does the natural

and Steinmann,

which

(re,i'S)

between

Similarly,

be carried

the Lens

Brehmer,

to judgment relations

Gibson,

to the cognitive

abstractions

in an integrated

framework

contribute

against

the types of displays

sense,

of interest

imply

Modeling

offered

Steward,

symmetrical

made

rallied

as the basis of scientific

of interest

should

for Cognitive

Brunswik

specified

environments

or carefully

Gibson

perception;

more like a laboratory

interaction.

does not necessarily

But in a larger

behavior

to be at odds, but may instead

of human-environment

environment.

conditions

a target

engineering

Resources

could

systems.

environmental

are the target

to cognitive

in the natural

be considered

approach

for the study of visual

existing

room

understanding

of an ecological

behavior

displays

the conditions

need not always

to a more complete

the adoption

information

control

approaches

that result

has a counterpart from representing

on

18 Perhapsmostimportantly,theLensmodelframeworkallowsthe modelerto measurethe degree to which theenvironmentalstructurewhichrelatesthecuesto the criterionis reflectedin the mannerin which thecuesarecognitivelystructuredto produceajudgment. High levelsof achievementareanindicationof a highly adaptive cognitive organization; i.e., a cognitive strategy that mirrors previous

comments

activity well.

the environmental

is likely

in this chapter

to be required

In addition,

behavior

of the cue structure

systems.

Did a particular

We often

depending

upon

the answers

the Lens model

representing

skilled

also need models more complex deficiencies.

cognitive

interpretation we go about

abilities

information

and actions

perceptually

guided

about

confronted

can (and should)

my previous system

with result

(facts

information

Note that these environmental as being

properties

Like formalisms

not to represent selection

However,

Second,

would

be

this in how

model

as suggested

(p. 272).

(facts by

Judgmental specifies

the

to gain more diagnostic are exactly

impediments

for rational

for

is a epistemological

only probabilistically

variables

the major

demands

we

has two important

the environmental

of last resort"

for

suggest

and we must be careful

the performer).

environmental

described

to identify

to action

to be judged.

difficulties,

activity

the available

comments

the state of the world,

the Lens model

about

an approach

as the unit of analysis,

in order

in order to keep distinct

is a "cognitive

interaction.

of displayed

The Lens model

as the criterion

of manipulating

I have previously

of human-machine

for developing

behavior

represented.

of conceptual

model

upon when

are not available.

task environments of fluent,

judgment

capable

action

validity

of the reasons

operator

the Lens model

applying

itself to serve

this interpretation

will only be called

criterion,

skilled

judgments

A first step toward

and the cognitive

et al. (1975),

point

However,

this perspective,

can give rise to a number

the world),

Hammond

fluent,

itself is not explicitly

opportunity

formulating

a good starting

interaction.

of modeling

forcasters,

such as these.

offers

From

Weather

usage

adapted

of remedial

as

of both the ecological

incorrect

perfectly

interaction

or both.

and analysis

making

types

view of cognitive

an understanding

that take the human-environment

activity.

are selected.

an action

different

of representing

action

in the design

of the

of less than fully productive

structure,

can yield

an operator

to questions

models

for the purpose

how actions

Quite

the causes

but only an analysis

of a potentially

framework

capable

First,

framework

about

from

As many

human-environment

the cognitive

interests

human-environment

that we not only require

to allow

error result

information?

skilled

for cue utilization

have similar

must be sensitive.

sort of adaptivity-oriented

one to localize

structure,

or was the error the result

not fully diagnostic

Thus,

allows

and their policies

errors.

information,

some

err in their predictions,

for these

behavior

in order to understand

the environmental

frequently

to which

suggest,

the Lens model

to either

for example,

structure

action,

those

features

of

to the development then, it may be quite

19 possibleto interpretthe skilledselectionof actionwithin theLensmodelframework,however, suchamodelis not likely to capturethosespecialfeaturesof taskenvironmentsthatallow for the acquistionof fluent human-environment interaction. The need to explicitly promote

it, suggest

performer

be described.

proposed

perception

results

behavior

the performer.

decision

space descriptions

resulting

from problem

environment, change

solving

particular

position

most concerned affordances,

concerning

with those

capable

perception-action

affordances

in order

of the world

environmental

structure

For our purposes,

in terms

space

the emphasis

of an affordance

for the two deficiencies

the possibility

judgment

that perceptual

perception)

suggests

However,

structure

Regardless

behavior

is still a valuable

Gibson

was

to specify of

in other

situations

or information

must be taken into account

or to combine

actions.

in more elaborate

information

specifying

of the type of either

to an affordance

structure,

tool in understanding

a

how

and behavior. space

is especially

of the Lens model

to the identification information

will

one to any

detection

or impoverished,

can in some cases play the role of the criterion

from passive

(i.e., direct

in cognition

of

of the

for action

is available

may have to engage

structure,

to orient

of affordances

is reflected

the concept

required

description

activity.

as the perceptual

affordance

In such cases, performers

activity

to guide

opportunities.

to select

from theories

does not commit

information

may be restricted

in order

akin to the

descriptions

resources

description

can be described

present

space

it

will result.

perceptual

to action

space,

will be a dynamic

may be detected

to detect the affordance

to compensate

affordance

space

the immediately

or post-perceptual

description

Second,

behavior

with other information

perceptual

resources

interaction

to be productive.

activity

in which

for action

resulting

and the performer's

space environmental

situations

the problem

space

structure

to the environment

other than that specifying for behavior

affordance

of orienting

access

structure

how affordances

and in such cases

information

cognitive

an affordance

as both the environmental

an affordance

an affordance

or the cue space descriptions

In most cases,

Note that creating

theory,

how

An action-oriented

of the opportunities

can be called

from decision

theory,

over time and a dynamic

in terms

of affordances

to understand

for action.

that

of the skilled

and theory

in order

opportunities

description

resulting

physics

description

of the environment

of the world

of how the environment

with his ecological

environmental

The resulting

judgment.

as well as those features

the problem

to environmental

in a description

presents

probabilistic

interaction,

Gibson,

an action-oriented

may orient

approach

f/uent

that we must consider

should

(1979),

represent

that we must relax

identified

because above.

in the Lens model,

and selection

is capable

important

the a priori

assumption

First,

an

thus shifting

of opportunities

of fully specifying

it provides

for action.

the affordance underlying

the

20 Lensmodelthattheavailableinformationis onlyprobabilisticallyrelatedto thecriterion. The questionof determiningtherelationshipbetweentheavailableinformationandtheenvironmental affordancestructureis anempiricalone. The ecologicaltaskanalysisframeworkpresentedbelow is anattemptto integrateBrunswik'sLensmodelandGibson'saffordancetheoryinto a unified frameworkfor modelingskilledhuman-environment interaction. The Framework Integrating

concepts

the ecological framework system.

for Ecological

Task Analysis

from Brunswik's

task analysis

framework

is a symmetrical

symmetries

uses a principle

between

of parallel

cognitive concepts

is evident

perceptual

and the lower

which

represents exploit

to suggest

surface

task analysis

model

structure

between

begins

of environmental

structure

the world,

and the environmental

depth

structure,

so to speak. are models

of environmental

of surface

description

in the Lens model. available

perceptual

structure

controlled

system.

structure surface action

perceptual

structure

structure

the Lens

structure

of surface

model

however,

ecological

of the environment.

as the actions

made

available

of environmental

between

in the middle

shown

and environmental

structure

systems,

which

a similar describes

captures

about

controls.

structure.

The

role to the cue the

the state of the

only the surface

also requires

In a human-machine

of the

one can think of surface

displays

task analysis

and

the surface

action

plays

a distinction

the performer

structure,

perceptual

of the

we rely upon

behind

task analysis

by interface

and action.

descriptions

framework,

from interface

framework

also

here

In human-machine available

model

to capture

framework

exists remotely,

structure

in ecological

The model

as the information

of the environment, action

perceptual

information.

Unlike

of surface

concepts

structure.

at the interface

which

in

the proposed

perception

of two complementary

exists

structure

The two descriptions

description

environmentally

which

results

human-environment

between

action

2 about

model,

the proposed

the upper

As in the Lens model

surface

diagram,

symmetries

theory

of parallel

structure,

between

with the creation

of the environment.

environmental

the integrated

the principle

certain

in the relationship

affordance

2. Like the Lens

and environmental

Insert Figure

Ecological

and Gibson's

in Figure

task analysis

This latter symmetry structure,

model

depicted

arrangement

Not only does ecological

certain

Lens

system,

perceptual

a description

of the

one can think of surface

21 In theLens structure

model

framework,

is the relation

person

is attempting

action

structure

between

the readily

to judge.

and depth

environmental

the relation

change

Similarly,

structure

the person

between

available

is the relation is attempting

the difference

between

given

distinction

reflects

the difference

between

readily

of environmental

potentially

covert

is a description

depth

structure

relationships,

actions

depth

of cognitive

structure

actions

actions.

2 is a description

is reflected

and the

side, the surface/depth

and intended

structure

surface

side, the surface/depth

On the action

on the left side of Figure

and the model

of how environmental

available

state a

between

available

On the perceptual

and inferred.

and depth

the relation

the readily

to effect.

structure

and the environmental

framework

between

reflects

perceptual

information

in the proposed

distinction

model

surface

The

of these

on the right side of the figure

in cognition,

and ultimately,

behavior.

The Principle

of P_II¢I

The description

Concepts:

of environmental

necessary

to capture

perceptual

guidance

specifying

environmental

of action covert

or remote

form,

of activity

depth

process

identify

the congruence

perceptual

action

between

is suggestive

describing

structure

description

environmental structure.

the analysis

a performer structure

model

namely

Various

in detail

of perceptual

perceptually

forms

type of necessary

guided

for cognitive of mismatch cognitive

structure

task to

between

Mismatches

to overcome

are possible: activity,

is to

another

Matches

activity.

activity

some

the ecological

surface

description.

of fully

of action

to specify below,

for

guidance

to infer

guidance

and action

of environmental

of demands

information

perceptual,

As will be described

capable

perceptual

the perceptual

is

The possibility

information

framework,

of surface

structure,

for fluent,

is possible.

of perceptual

Rather,

and action

the

each form of

and a different

type of

solution. process,

must be discussed.

using

envirornmental

of action.

of a different

design

of activity

detection

these two forms

are indicative

of both perception

of the proposed

the models

opporunities

non-specification

interface

structure.

by examining

these two models

Before

In terms

of the environment.

action

indicate

in terms

guidance

as the perceptual

property

proceeds

these two models

remedial

structure

Structure

relies upon the availability

affordances.

the surface

analysis

mismatch

surface

as the use of one form of surface

namely,

between

and Action

how the skilled perceptual

is not to be described

be described

Perceptual

reflects

It is easy to overlook

The model

s perceptual

is described.

the symmetrical of surface

capacities

This description

both the perceptual

of the models

perceptual

structure

as a frame

of reference

is relational

capacities

that such descriptions

nature

are actually

relational

and

is an environmental in which

in the sense

of a performer

of perceptual

the

that the resulting

and the environmental in nature,

as we often

22 speakasif a perceptually-generated differentiationof theenvironmentis purelya functionof the environmentalstructureandnot afunctionof perceptualcapacities.However,the factthatsome formsof environmentalstructureareseenasobjectswhile othersarenot,or thefact thatsome aspectsof theenvironmentareseenasbeingbluewhile othersarered,areasmuchfactsaboutthe perceptualsystemastheyareof theenvironment.A perceptually-oriented environmentalmodelis a relationalconstructcreatedby usingperceptualcapacitiesasaframeof referencein which environmentalstructureis measuredanddescribed. The secondmodelof environmentalsurfacestructurerequiredfor ecologicaltaskanalysisis an action-orienteddescriptionof theenvironment.Themodelof surfaceactionstructureis an environmentaldescriptionusinga performersactioncapacitiesasa frameof referencein whichthe environmentalstructureis described.This descriptionis relationalin the sensethattheresulting environmentalmodelreflectsboththe actioncapacitiesof aperformerandtheenvironmental structure.This action-orientedenvironmentalmodelrepresentstheworld in termsof its opportunitiesfor action. This environmentalmodelthusgenerates a differentiationof theworld in termsof thedegreeto whichvariousspatiotemporal environmentalregionsareconsistentin variousdegreeswith thetakingof variousactions.An action-oriented environmentalmodelis a relationalconstructcreatedby usingactioncapacitiesasa frameof referencein which environmentalstructureis measuredanddescribed. The symmetricalnatureof theperceptually-oriented andaction-oriented environmental descriptionsrequiredfor ecologicaltaskanalysisshouldbe apparent.Thesetwo environmental modelsdiffer only in thatoneis relativizedto thefunctionalityof the performersinputmechanisms while theotheris relativizedto thefunctionalityof theperformers outputmechanisms.Thetwo modelsthusreflecttwo differentprimitivedifferentiationsof the environment,onegeneratedby usingperceptualcapacitiesto understand how theworld is carvedup with respectto perception, andtheothergeneratedby usingactioncapacitiestounderstandhowthe world is carvedup with respectto action. As BarwiseandPerry (1983,p. 11)havesuggested, Theemphasisis onhow theorganismdifferentiatesits environment,on the sortsof uniformitiesit recognizesacrosssituations.Differentorganismscanrip the same realityapartin differentways,waysthatareappropriateto theirown needs,their own perceptual

abilities

and their own capacities

the structure

the environment

with respect

to the environment

independent

of the organism

organism

reflects

properties

displays

for action.

This interdependence

to the organism

is extremely

and the structure

important.

For while

s individuative

activity,

the structure

of the organism

itself.

(My emphasis)

reality

between

of the organism is there,

it displays

to an

23 Neithertheperceptually-oriented environmentaldescriptionnortheaction-oriented environmentaldescriptionresultsin a moreprimitive,privileged,or objectiveontologicalpictureof theworld. However,theclaimis sometimesmadethatanaction-orienteddescriptionof theworld in termsof affordancesis scientificallyillegitimatebecauseit is relativizedto theactor,andis thus in somesensesubjective.Notethatthecorrectresponseto this possiblecriticismis not to adoptthe heroicpositionthataffordancesarein somesenseindependent of theperformer: theyarenot,they arisefrom usingtheperformersactioncapacitiesasaframeof referencefor environmental description.Rather,to counterthis argumentonemustmerelyemphasizethatthe supposedly scientificallylegitimateperceptually-oriented environmentaldescriptionswhich portraya world of objectsandpropertiesarejust asrelativizedto thecapacitiesof theperformerasareaction-oriented environmentaldescriptions.Onemerelytakesperceptualfunctionalityasthe frameof referencefor environmentaldescription,while theothertakesactionfunctionalityastheframeof reference. The Process

of Ecological

I shall call a process are created models

whereby

and mismatches

are identified

What I believe now be stated

quite simply.

human

and environment

(1990)

rational

cognitive ecological

which

Before

task analysis

specify

ecological

attempts

outlining

to define

the analysis

task analysis

is to define

processes

that may be required

Anderson

s rational

analysis,

then,

necessary

cognitive

activities

might be carried

common

perspective.

be its primary cognitive

concern,

activities

to support

the present

However,

ecological

whereas

the current

such as memory,

effective

functionality

a few comments

the

and by for effective

computational-level and Anderson of a variety

comparing

s

of

the goals

of

may be valuable. functionality

leaves

of any cognitive

interaction.

aside the question

In this much

formulation

categorization,

s (1982)

human-environment

task analysis

can

between

interaction,

of vision models,

the necessary

approach out.

or artificial) fluent

in spirit to Marr

in particular

system.

modeling

will be necessary

the necessary

s approach

to cognitive

(natural

processes

process,

in the two

of a human-environment

underlying

functionality

structure

of a human-environment

task analysis

what cognitive

and action

represented

approach

an interface

is thus similar

and Anderson

The goal of ecological

task analysis

to which

perceptual

differentiations

of an ecological

to def'me the necessary

analysis

of surface

with the principles

task analysis

attempts

models.

the degree

is consistent

An ecological

which

an ecological

contribution

helps

models

the environmental

A preliminary

to identify

so such an analysis

theory

between

to be the central

is required

behavior.

environmental

and described,

system

doing

Task Analysis

the two approaches

takes human-environment of rational

and problem

analysis solving.

Like of how any share

a

interaction

is concerned

with

But as Anderson

to

24 himselfhasnoted,all thesecognitive of action

selection,

a problem

solving

words

the nature

and for Anderson, or decision

Anderson

abilities action

making

uses to describe

of the environmental

are useless selection

exercise

is to always

(p. 192).

the processes models

if they do not in some

that result

to be understood

Perhaps

underlying

way serve

action

as the result

it doesnt

much

selection,

but what does

from such a choice

(decision

the goal

matter

of

what matter

is

trees and problem

state spaces). But as our previous intensive

processes

comments

for explaining

focus not on the sufficiency Ecological

in any particular mechanisms

manner,

task analysis

allow

for a description

when

an examination

environmental

The Principle

above,

surface

below.

The result structure,

important structure

task analysis

and action

begin

after creating

problem

state space, decision

structure

to cognitive

structure

process

however, model

tree, linear

processes rules).

of perception

of the environment

guide

possesses

cue-criterion

(e.g., heuristic Rather,

requires

search,

a certain function), comparative

we let the examination

the selection

the bridge

of the model

in

between

and process.

structure.

in detail

It is very depth

That is, we do not depth

structure

(e.g.,

and thereby,

a corresponding

evaluation

of alternatives,

of the surface of depth

with

of the environmental

the environmental

intrinsic

the

the congruence

will be discussed

structure

of

the environment

of a model

and action

selection

of the models

congruence

that modeling

Only

Structure

they differentiate

cognitive

that

activity.

by then examining

is the specification

task analysis the models

in which

models

of activity.

intensive

provide

is governed

in perception-action

with the creation which

of their

interaction.

of action

and Comaitive

must

sorts of governing

mismatches

proceeds

of internal

however,

selection

guidance

in a construal

of examining

that the task environment

linear cue combination

begins structure

The process

and a complementary

result

Depth

in terms of the manner

of this exercise,

by assuming

surface

The analysis

and action.

action

or any other cognitively

ecological

to note that ecological comes

solving,

design,

the use of environmental

indicates

task analysis

into one of

the conditions

what

of the perceptual

models

Environmental

these two models to perception

in terms

Concepts:

structure

respect

depth

problem

and the environment.

between

selection

from cognitively

of human-environment

to meet this goal through

will ecological

of Parallel

on limiting

to define

any instance

of these environmental

making,

As mentioned

human

of action

to support

not by assuming

has as one if its goals

for modeling

attempts

structure

terms of decision

perceptual

but rather

starts

resulting

can be rationalized

Modeling

but rather

therefore,

whether

and action,

behavior.

of these frameworks

task analysis,

selection,

perception

productive

will be necessary

Ecological

all action

or from more efficient

these frameworks

necessity.

suggest,

perceptual

structure,

and action

and thereby

the

25 correspondingstructurefor cognitiveprocesses.Hereinlies themajordifferencebetweenthe proposedapproachandmanycurrentmodelingapproaches in cognitivescience:considerationsof perception-action functionalitydefinethenecessaryfunctionalityfor cognitiveprocesses, rather thandefiningperception-action functionalityby an a priori cognitive model of action selection. I have tried to make the case for sequencing points

in this chapter.

We are obviously

corresponds

to, and helps make

serve

selection.

action

however,

Which

is available

cognitive

the performer,

processes

interaction

best describes

the processes

environmental

models

environmental

structure

is reflected

environmental

and cognitive

limitations

without

Yet I wonder doing

realistically action

selection

environmental

behavioral

purpose

processes

format.

with perceptual models

applied

and action unlimited

the world capacities,

a variety selection

in terms

might

on the other hand,

requires

capable

of the world

of preserving s perceptual

however,

and a between

behavior. necessary

for ecological

the content and action

structure models

of any

that rationalize

low-dimensional are advantageous

be organized

of how the

in a context-free,

general

environmental

structure

of raw

has the potential

to create

environmental

my own view is that any method

many

in a setting

of the fine details

and action

structure,

task

of these descriptions

and are thus suggestive

interaction

the

any ways in which

representations

of the interaction

different

Note,

the congruence

is that relatively

of contexts

skilled human

complexity

The richness

solving

Nevertheless,

underlying

structure.

of the perceptual

dimension.

the mechanisms

about

Such low dimensional

action

identifying

models

of the models

that

ways in which

structure

One great allure of environmental

or problem

across

underlying

Describing

of almost

situation.

depth

of the

model

As a result,

to identify

upon productive

to get more precise

can be used.

in that they can be easily

in order

they make

the cognitive

in the two cases.

are selected,

description

to the richness

making

descriptions

psychological

schematic

injustice

as decision

processes

designs

the best description

the different

of environmental

can be examined

if it is possible

a potential

complex

cognitive

place constraints

a highly

and behavior

to

perceptually

demands

interface.

to describe

that

behavior,

(control-display)

that provides

with another

once a model

models

or biases

This is obviously analysis.

of internal

interface

may be quite unlike

in the two cases in cognition

to which

structure

be engaged

for productive

of the cognitive

model

interaction

framework,

model

Different

of previous

depth

that will actually

of the degree

in terms

with one interface

underlying

corresponding

action.

that the cognitive

will be needed

that like the Lens model

activities

analysis

at a variety

of environmental

will be necessary

can differ radically

with the result

underlying

cognitive

productive

in this fashion

for a model

activities

by a detailed

to specify

for the same task environment upon

looking

sense of, the cognitive

can only be determined

information

the analysis

capable

of

of any reasonable

of the environmental

seemingly

necessary

for the

26 fluentoperationof theperception-action system,severelyovertaxesour highly limited cognitivelinguisticresourcesfor environmentaldescription.I doubtthatatthecurrenttime a taskanalysis techniquecapableof guidinghuman-environment interactionmodelingcanpossiblybemuchmore thana chargeto themodelerto undertakethelong andarduousprocessof identifyingand describingthepotentiallyoverwhelminglyrich interfacebetweentheperformerandtheworld. For thisreason,thefollowing descriptionof the analysisprocesswill consistof an abstract discussionof the possibleresultsof anecologicaltaskanalysis,alongwith a concreteexampleof how suchananalysiscanbeperformedfor a particularbehavioralsituation.Figure3 depictsfour possibleresultsof anecologicaltaskanalysisin termsof thecongruenceof theresulting perceptually-oriented andaction-oriented environmentalmodels.The grid linesin eachof the schematicenvironmentalmodelsindicatethemannerin whichtheenvironmentis spatiotemporally differentiatedwith respectto eitherperceptualcapacitiesor actioncapacities.The four caseswill be describedseparately. InsertFigure3 abouthere

Case I: Perceptual

overspecification

In the first case shown respect

in Figure

to the environmental

different Object

perceptually

situations

displays

of action.

perceptual

so it becomes

perform

this function

Object

used,

salient

to the differentiation

however,

Perceptual

likely that a significant identify

consistencies

relational

in the action

spaces,

naturally amount

to reduce

emerge space.

some

although

to enable

supported

nor supported

of categorical

ability

for coping

space.

a display

to overcome

naturally

through

or instance-based to action.

action.

the dimensionality

perceptual

When

learning Note,

of the

These

displays

information

in

in a manner

based

solution

is not

perceptual

occurring

result

display

for pattern design,

may be required however,

in an

relational

the possibility

configural

Many

with perceptual

that could potentially

this solution.

from perception

for productive

action

When

some

action.

that are differentiated

is one process

available

in the mapping

available

over-differentiated

features

and action

must be perceptually is neither

the originally

recognition

properties

solution

with

for productive

opportunity

with the lower dimensional

pattern

of the perception

is over-differentiated

of opportunities

are an attempt

will have to develop

alignment

recognition

displays

reflected

the performer

overspecification.

in terms

all point to a single

aligned

by organizing

such a way that perceptually identical

environment

are one type of design

overspecification space

3, the perceived

differentiation

distinct

or configural

of action

it is

in order

that there is still

to

27 thepossibilityof fully productiveperformancein all thesecasessinceperceptionmerelyoverspecifiesaction,it doesnot misspecifyaction,asin CaseIV below. Case II: Perceptual

underspecification

Here the perceived differentiation

environment

in terms

information

in order

perceptual

to uniquely

in memory,

information

in order

the displayed such cases. rather

specify

about

specify

effects

of an action,

simulations

are two approaches

performance,

however,

Internalization

Simon

effects

physically

trying

state without

possible.

capacities

of the cognitive

of an action out

solutions

the use of an internal

subtle)

In this situation demands

model,

differentiation Perceptual

to which

specification

of action

fluent

or conceptual

performance learning.

learning

can be expected

The performer

state,

of action

or fast-time Unaided

may be necessary

solving

models

into account

the dimensionality

information

the

or about

may not always

Here,

be

the perceptual

of the perceptual

into alignment

to develop

when

and

initial perceptual 1969).

can eventually

in

of Newell

behavior,

this form of activity

s pre-established

future

to the performer.

of this type, however,

perception

in

displays

Exploratory

to overcome

of the environment

performance

of the downstream

model

(e.g., E.J. Gibson,

in order to increase

into

that may be necessary

for taking

although

this

skill can be acquired.

The problem

that is available

of stimulation

Perceptual

before

keep

memory

about

a number

Predictor

available

activities

available

differentiation

change

differentiation.

dimensions

Case III:

learning

is perceptual

as to bring the perceptual

apparent.

must be taken into consideration. is a method

for aiding

knowledge

in the form of an internal

as descriptions

of the performer

environmental

dynamics

Building

identifies

requires

learning

state information.

A form of perceptual

underspecification

considerable

in the environment;

is that information

this information

to past or future

can be viewed

downstream

action

of

with the perceptual

alternative.

Perception

cause

of the environment,

is one strategy

are not perceptually

for making

will require

of environmental

order to gain access

available.

of the appropriate

and these effects

action

underspecification

than past state, is not perceptually but the selection

history

available

One likely

information

information

displays

perceptually

alternative.

the previous

the appropriate

of perceptual

to the environmental

not enough

action

this memorial

using trend or historical cause

is simply

is that there is hidden-state

and then integrate

interface

There

with respect

the appropriate

something

to uniquely

Another

candidates,

future

action.

of action

must know

information

is underdifferentiated

of productive

underspecification

i.e., the performer

of action

space

so

with the action-oriented requires

become

without perceptual

(perhaps

initially

sensitive.

significant

cognitive

competencies

provide

28 theability for uniquespecificationof productiveaction. No rule-basedinformationintegrationis required. Neitheris informationaboutthehistoryor futureof theenvironmentnecessary. Therefore,no internalizationof environmentaldynamicsis requiredtoprovidetheseformsof information. If it werethecasethatthedesignercould alwaysbecertainthatthe perceptuallyorientedandaction-orientedenvironmentalmodelsof thehuman-environment systemwerecorrect, interfacesthatsupporttheperceptualspecificationof actionwouldbeour undeniabledesigntarget. However,if unanticipatedchangesoccurin eitherthediagnosticityof the displayedinformation, thefunctionalityof theinterfacecontrols,or theenvironmentaldynamics,a fluent,informationally encapsulated perception-action modeof controlmaycarryonwithouttheperformerpayingheedto theseenvironmentaldisturbances. (_Se

IV:

Perceptual

misspecification

In the final case the mapping action

is unruly.

perceived endow

Behavior

situation

misspecification, for finding

might well be productive

invariance

missspecifies

in aggregate

models.

action

do think it is important

misspecification,

that such cases

that the performer

of reasoning

decision

making

(Kahneman

solving

(Kotovsky,

Hayes

attention

to the concrete

models.

There

environment

that a given

productive

knows

(e.g., Wason and Tversky,

presentation

is always

really defined

model

the modeler

1985)

structure

some

1979; Tversky

The long history

no environmental with respect

fails to represent,

situation

that people than do many

stuff that is context to a model;

it is simply

and as a result,

those

I

upon perceptual does not.

I suspect

of findings

abstract

on the 1975),

and problem

pay considerably

more

cognitive

as opposed

aspects

I

to be a case of

1981),

those

with

However,

Johnson-Laird,

and Kahneman,

all demonstrate

of a problem

based

1972;

to be a

these variables.

of a catch-all.

things that the modeler

and Johnson-Laird,

cognitive

can be described

among

selection

opt

that perception

finds what appears

action

and instead

can be considered

under the assumptions

than we may care to admit.

and Simon,

is, of course,

Context

that when

perceptual selection

judgement

it is something

typically

of probabilistic

the covert relationships frankly,

but yet observes

are more frequent

context-sensitivity

structure.

this case because,

of action

depth

the currently

in such situations

the construction

of human

framework

indicating

to emphasize

it is likely

through

and productive

but not because

to overcome

account

and that the environmental

correlations

information

of behavior

of knowledge

performance

task analysis

(judgment),

will have little to say about

information,

Models

amount

available

in this situation,

In fact, the Lens model

and cue-criterion

perceptual

informative.

with a considerable

case of the ecological

cue-cue

the perceptually

or else give up hope for a deterministic

and environmental special

between

is particularly

the performer

of _lction

aspects

to relevant of the

that are rendered

29 incapableof producingbehavioralvariance.Findingswhich demonstrate the intensivecontextsensitivityof cognitionandbehaviorcanbeseento be,in part,a reflectionof thefact thatmany currentenvironmentalmodelsareeitheroverlyabstract,or perhapsevencut acrossthe grainof the perceptually-oriented andaction-oriented environmentaldifferentiationsthatarethebasis of ecological

task analysis.

to complex attention

or probabilistic

to advance

Jagacinski, displayed

modeling

(Sheridan,

provide

and a mixture typical

subjects

control

implementing automation

activities in many

mappings

out as much

cognitive

that give rise

by increased

modehng.

to engage

for interface

systems

apparatus

modern

consisted

operator

and the need to design

Miller

controls,

systems.

world.

was motivated

similar

The selection and

by the introduction

to support

this form of

interaction. The experimental use manual

apparatus

(joystick)

over four additional designed upon

to which and enemy

activity craft)

for this purpose. i.e., to engage base.

Subjects

missiles,

or automatic

cargo

piloted

was confined.

The scout s major

Subjects

the world.

The scout

used the additional

both stationary

and mobile

capacity)

to successfully

mode

levels

of performance

of dynamic

interaction

activity

a text editor

the partially

was to discover equipped

management

objects

shown

mile area (cargo radar

objects,

and unload

it at a home

constraints

(e.g., fuel,

of practice

were required was quite

of much

world

the discovered

each 30 minute

characteristic

of control

specifically

100-square hidden

to act upon

of action

mode

could

with a 1.5 mile radius

complete

the selection

subjects

forested

the entire

craft and to load cargo

of resource

hours

using

showing

was therefore

enemy

to a number

and many

display

over which

used a supervisory

commands

craft primarily

also had to attend in order

Crews

the scout within

map or situation

at skilled

often seamless

control.

color graphical

within

of a scout vehicle,

strings of action

Subjects

The task was quite complex However,

the cockpit

(autopilot)

craft by entering

for the experiment.

a dynamic,

simulated

skilled

experimental to achieve

session. mastery.

rapid and a fluent behavior

to

control

for planning

interfaces

and

At an abstract

and supervisory

is responsible

in

of rich, graphically

in a simulated

systems,

micro-world

(Kirlik,

and discrete

control

craft operating

automated

design

human-machine

in both manual

a system

for (often remote)

in a dynamic

of both continuous

of many

task, in which

existing

interaction

resources

1987) of a set of semi-autonomous

of a supervisory

elaborate

level, the experimental

technology

level, the task required

as by increasingly

that could

of information

the kind of interface

unruly

can be straightened

of human-environment

At a concrete

sources

the apparently

Task Analysis

approaches

1991).

cases

of cognition

modeling

of Ecological

We have performed

that in some

accounts

to environmental

An Example

order

I suspect

and

was observed.

of

30 Theapparent relying

economy

heavily

with some

of control

advantages

of the short-order was possible

identifiable

uncertainties

interface

which

the interface

action

selection.

model

capable

through

design

may have required

example

the simulated

environmental

concerns world.

and cognitive

can be found

in Kirlik,

configuration displayed

of search

in light brown.

modeling

(the unfilled

circle)

The lightly

action:

scout

performed objects,

locomotion

in open rather on the other hand,

were considered influenced refuel

for cargo

search

at home

forested

to resolve

task analysis where

as well as those

of

the control-

situations

in

intensive

the design

mode

of

of a process

description discrete

search

of this model action

paths

as well as a description

selection

in the laboratory

Figure 4 is a depiction

and dark green,

of

task

of a world

here indicated

forested

for the scout

regions,

in white,

here shown

respectively.

Only

were as light

home

base

of the scout are shown in the figure.

Figure

and enemy

regions

with scout

was more efficiently located

4 about

craft requires

than forested

here

consideration radar.

of two capacities

Locomotion

due to the need to navigate performed

in forests.

since fuel expenditure during

of continuous

and heavily

objects

base at some point

were not

behavior.

The open regions,

and sighting

path selection

situations

were used to motivate

as light green

to be more densely

available

to mode

action

to use a more cognitively

(1991).

to subjects.

and the initial location

the world

mode,

for dynamic

Insert

Searching

productive

An ecological

processing

the selection

and Jagacinski

gray and dark gray, were displayed

in relation

behavior

modeling

as it was displayed

representation,

above

were not always

to identify

subject

A more complete

Miller,

upon

information.

subjects

problem

were

that a perception-action

in order

of the task analysis

t_sk analysis

The present

was performed

that subjects

as discussed

unlikely

and actions

perceptual

mimicking

mode

it seemed

information,

a perception-action

The results

as an external

some of the constraints

supported

of successfully

An ecolo#cal

since

with non-specific system

information

However,

from the displayed

associated

led to the hypothesis

of this processing

cook.

in all cases,

this human-environment display

in this environment

upon the rich set of graphical

of the attendant

our example

easily

of behavior

the middle

in lightly

In addition,

for

was most efficiently around

forested

trees.

Sighting

areas because

the fuel range

objects

constraints

rates were designed

so the scout

third of the experimental

session.

had to

31

InsertFigure5 abouthere

Figure5 showssearchpathscreatedby two differentsubjects.Both beginby travelingnorth alongthe boundaryof theinnerforest,turn eastto follow thetop boundaryof this forest,loop backwestalongtheboundaryof theupperforest,turn southalongtheborderof theleft forest, visit homebasefor refueling,thendepartto theeastandthenthe north,at which point thesession terminated.This boundaryhuggingbehaviorresultedfrom theinteractionof the two majorcriteria for searchpathselection:searchingforestswith the 1.5mile radarto discoverobjects,and locomotingthroughopenterrainto coverasmuchareaaspossible. Whatis themostfaithful descriptionof thecognitiveactivitiesunderlyingsearchbehavior? Whatis an appropriatemodelfor thedepthstructureof this environment?Weof coursewould like tofind a modelof depthstructurethatcapturesthoseenvironmentalfeaturesto which cognitionand behaviorwassensitive.Onecouldof courseformulatethisprocessasa constrainedoptimization problemandusea generate-and-test proceduretocreatealternativepathsandthenevaluatethem with respectto anobjectivefunction. However,thereareaninfinite numberof possiblepathsand thecomputationaldemandsappearoverwhelming.Onecouldalsoattemptto describethisprocess in rule-basedterms,usinganenvironmentaldescriptionin termsof perceptuallysalientobjects suchasforestsandtheir borders.With this model,thecognitiveprocesses underlyingsearch behaviorwould bedescribedin termsof themanipulationof symbolsstandingfor discreteaspects of environmentalstructure. Ecologicaltaskanalysis,on theotherhand,suggests thatwe delayassumptions aboutmodeling environmentaldepthstructureandcognitiveprocesses until afterthemodelsof environmental perceptualstructureandactionstructurehavebeenconstructed.An initial cut atconstructingthe perceptually-oriented environmentalmodelrequiredfor ecologicaltaskanalysiswould beto describetheobjectsandpropertiesperceptuallyapparenton this mapdisplay. However,aswill be seenbelow,ecologicaltaskanalysissuggeststhatwe iterateandrefinethis initial perceptuallyorientedmodelafterconstructingtheaction-oriented modelby usingthelatterto helpidentify any initially overlookedperceptualinformationcapableof specifyingtheaction-orientedstructureof thisworld. The informationusedasthebasisfor actionselectionmaybeconsiderablymoresubtle andrich thantheinformationpreservedwhenusingperceptionin a purelydescriptivecapacity (e.g.,Neisser,1988;Bridgeman,1991;Shebilske,1991). Constructingtheaction-oriented environmentalmodelrequiresthe useof actioncapacitiesasa

32 frameof referencefor environmentaldescription.Sincetwo actioncapacitiesunderliesearch behavior(locomotionandsightingobjectswith radar),we mustdescribethe action-oriented structure,or affordances,of this environmentfor bothlocomotionandsightingobjects. We will considersearchaffordancesto bea simplecombinationof thelocomotionandsighting affordances.Figure6 showsthedistributionof locomoting,sighting,andsearchingaffordances asmapsof theworld in which thepathsshowin Figure5 weregenerated.Figure6ashowsthe world asit appearedon themapdisplay. Figure6b showsthe locomotingaffordance,calculated by assigninga valueof zerofor openregions,a valueof -1.5 to lightly forestedregions,anda valueof -2.0to heavilyforestedregions.Thesevalueswereassignedtoreflect thedifficulty of rapidly flying thescoutthroughtheseregionsdiffering in treedensity. Darkerregionson themaps indicatehigheraffordancevalues. InsertFigure6 abouthere

Theaffordancevalueswereselectedby attemptingto constructanobjectivemeasureof the degreeto which relevantactionscouldbeperformedasa functionof environmentalstructure.For example,Figure6c showstheworld sightingaffordancestructure.To constructthis map,a four dimensionalvectorwasassociated with eachworld locationto indicatethepercentageof areathat wouldbecoveredby scoutradarcenteredatthatlocationthatwasopenregion,lightly forested region,heavilyforestedregion,andareabeyondtheworld boundaries.For eachpoint,the inner productof this vectoranda sightingaffordancevectorwastakentodeterminethesighting affordanceof a particularworld location. The sightingaffordancevectorwasthe samefor each world locationandindicatedthedensityof cargoandenemycraft within eachof the four typesof regions.The sightingaffordancevectorhada valueof zerofor openregionsandareabeyond world boundaries,andavalueof 1.0for lightly andheavilyforestedregions. A maximal sighting affordancewouldexist,therefore,in caseswheretheentirescoutradarrangecovereda forested region,andaminimal sightingaffordancewouldexistwhentheentirescoutradarrangecovered eitheranopenregionor areabeyondtheworld boundary.Thegradedstructureof the sighing affordancedistributionresultsfrom thecomplexinteractionbetweenthecircularradarcapabilities of the scoutandtheirregularlyshapedopenandforestedregionsthatdeterminedobjectdensity,or moregenerally,theinteractionbetweenthesubjectsactioncapacitiesandtheenvironmental structure. Figure6d showsthe searchaffordancestructure,createdby simplysummingthevaluesof the

33 locomotingandsightingaffordancesateachworld location(i.e.,a locationaffordssearchif it affordsbothlocomotingandsightingobjects).This maphasbeenrescaledto clearlyindicatelocal optimain thesearchaffordancestructure.Consideredthreedimensionally,this mapindicates peaksandridgesof high searchaffordanceandvalleysandholesof low searchingaffordance.The peak areas

indicate

open regions In order define

the best compromise

and sighing to define

models

objects

congruence

between

our original

perceptually-oriented

oriented

model.

however,

capacities

structure

be necessary

First,

the search

available

provided

to fully specify

perception manner

would in which

locomotion

also have sighting

perceptual

information

perception

would

have to supply

defined

by the 1.5 mile radar radius

needed

to assess

here I will simply forms

however,

affordances

was necessary

structure.

to first identify

did have the perceptual

ability

in a manner

that

information

search

affordance

foveating

(and could

of the world

the search

world

location,

experiments

although

area are surely

conducted),

we may expect would

be an

the information

available

environmental

experiments

of such experiments affordance

the

a circular

of such experiments

of psychophysical

however,

Given

be straightforwardly

the action-oriented

a

of the necessary

area within

that indicates

that the results

affordances,

the nature

are possible,

The result

whether

structure.

psychophysical

manner

is perceptually

at a particular

of forested

Although

what kinds

can identify

we can define

judgments

to specify

we attempt

in an isomorphic

then information

the sighting

Constructing

for the sake of this exercise

the world

To fully specify

this is possible

model

structure,

world

If perception

forest,

when

errors.

of human

What perceptual

differentiates

of the amount

that such perceptual

to specify

structure?

or heavy

of the scout.

is available

environmental

possible.

affordances.

Specifically,

measurement

Using

the displayed

were constructed

perceptually-oriented

the search

Let us assume people

assume

to which

of systematic

empirically-based to specify

the degree

extent

model

a measure

We now ask the question,

knowledge

measure

Note that

than does the action-

in this the world?

to be able to measure

in this case.

differently

the

models.

model

affordances.

affordances

through

we now examine

environmental

the world

affordance

light forest,

to this task, and also to

process,

of the displayed

by locomotion

is open region,

solutions

the perceptually-oriented

possibly

note that the perceptually-oriented

location

certain

within

could

for locomotion

out of alignment.

to the most faithful

to specify

to the differentiation given

differentiates

with knowledge

perception

demands

and action-oriented

or affordances,

together

to find a way in which

would

and cognitive

are apparently

contained

structure,

the depth

perception-actions

structure

model

The two models

the action-oriented

specifies

for fluent depth

the conflicting

regions.

the perceptually-oriented

what information

perceptual

in forested

opportunities

of environmental

between

structure.

model,

to conduct. suggested

(If results

that indicated

34 the subjectscouldnot reliablyestimatethesearchaffordancestructurewe wouldthenbeableto definethekindsof cognitiveactivitiesnecessaryto do sofor behaviorto be productive,i.e., to be in alignmentwith searchaffordances).Whatwould searchbehaviorlook like if subjectswere simplyallowing searchbehaviorto begovernedby theperceptuallydetectedsearchaffordances? We wouldperhapsexpectin thiscasethatthe scoutwouldbeflown up thesteepestgradientin the searchaffordancestructurefrom its presentposition. But while thelocal organizationin search pathsmay bedescribablein thisfashion,searchpathsalsohaveaglobalorganizationthatis not well capturedby this simplesearchmodel. This mismatchbetweenobservedbehaviorandthebehaviorthatwould resultfrom a simple perception-action solutionto this taskis suggestiveof whatkindsof additionalcognitivedemands this taskmakesupontheperformer.Weconstruelong-rangeor globalpathplanningasthe selectionof a sequence of waypointsto bevisited,whereeachwaypointis a peakor ridge in the searchaffordancestructure.We still can,however,construeshort-rangeor local navigationin termsof the simpleperception-action modeldiscussedabovethatresultsin searchaffordance gradientascent.The selectionof anappropriatesequence of waypointsis constrainedtheneedto avoidbacktrackingthroughpreviouslysearchedregions,andalsothe needto returnto homebase atsomepointin themiddlethird of the mission.Thereis no readilyavailableperceptual informationcapableof specifyingtheseconstraintsuponproductiveactivity. Thus,we havea case of perceptualunderspecification of action,wherethe subjectmustapparentlytry out a numberof alternativesolutionsto assessthedownstreameffects(i.e.,consistencywith backtrackingandfuel constraints)of each,priorto selectinga globalsearchpath. InsertFigure7 abouthere

Theresultingmodelis shownin Figure7. At the startof a session,themodelidentifiesthe peakareasin the searchaffordancemapascandidatewaypointsto visit during amission. These peakareasweresubmittedto a generate-and-test mechanismthatattemptedto orderthe waypoints to acceptablymeetbacktrackingandfuelconstraints.Notethatalthoughacognitivelyintensive processis neededfor this purpose,arelativelysmallnumberof waypointsareconsidered,since the searchaffordancestructurecanbeusedtoobtainarelativelylow-dimensionalrepresentation of the world for long-rangepathplanning(i.e.,the setof local optima). Many of the fine detailsof local searchaffordancestructurecanbeignoredduringthisprocess.The outputof this processis anorderedsequence of waypoints.The first waypointis thenselectedasa destinationandwas

35 thusconsideredto possess an affordancefor visiting. The scoutdid not fly in a linearpathto the waypoint,however,sincescoutmotionwasdeterminednotonly by the visiting affordancebut alsoby thelocal searchaffordancestructurein thevicinity of the scout.Detailedmotion commands

for the scout

structure

to operate

force exerted

by second

affordance

basis.

on the visiting

that was very sensitive

visiting

affordance which

one component laboratory Kirlik,

relative largely

Miller,

the perceptua/ly-measurable

weights

to search

An evaluation

by the current

affordance

model

waypoint

values

in a direct structure.

of the process

with the a of motion

relative

to the

in meandering

a large weight

on the

path to the current

This search

model

interaction

in the

human-environment

adequacy

affordance

the direction

resulting

In contrast,

affordance

search

combined

affordance

resulted

of skilled

when

determined

structure.

affordances

of the empirical

and Jagacinski

waypoint,

the local search

process

which

on the local search

provided

disregarded

force field,

of the current

to the local search

in a complete

task.

Large

affordance

motion

waypoint

by considering

upon the scout as an attractive

by the visiting

on a second weighting

were created

model

was

can be found

in

(1991).

CONCLUSION I have tried to make the case that modeling integrated

human-environment

a framework

in which

can be carried details

behavior,

but rather,

In addition, concept, match

it assists

can then specify

task analysis

provides

in perception-action

the design

of modeling

and cognitive

in the identification

and mismatch

to be a theory

the process

the perception-action ecological

framework

environmental

of displays

and controls

human-environment

interaction

global

to flesh

for design. through

structure.

phenomenon of cognitive

instance

is offered

of of humanprocesses

have available.

For an already the process The results

existing

design

of identifying of such an analysis

and action

structure

into

better alignment. Modeling environmental

skilled

structure

promises

to be a messy

that relies upon

business.

the intensive

as

out the

of interaction,

may already

that bring perception

take the

interaction

is in no way a new theory

the scientist

demands

which

task analysis

to be done in order

any particular

resources

theories

human-environment

any new models

models

of cognitive

Ecological

of the more

As such, it does not contribute

it guides

requires

underlying

work remains

The proposed

it may be considered

interaction.

upon

for the mechanisms

task analysis.

although

environment

the search

design

as the unit of analysis.

It is quite clear that much

of ecological

cognition,

relying

out.

system

to support

exploitation

We are led to this conclusion

once

of we

or

36 acceptthepossibilitythattheability to skillfully performwithin sucha widerangeof environments is atestimonyto our adaptiveabilities,ratherthantestimonyto anyabstract,context-free,generalpurposemethodswe mayhavefor the selectionof action. Perhapswe only resortto generalpurposemethodsfor actionselectionwhenwefail to find (or adjusttheenvironmentto allow) moreefficientperception-action solutionswhich areenabledby exploitingmanyof theconcrete detailsof the perceptionandactionstructuresof our world. The searchfor modelsof skilled interactionshouldnot bemisleadby thefact thatwhatpeoplecando mostefficientlymay be hardestto describe,andwhatwe do mostinefficientlymaybeeasiestto describe. ThroughoutthischapterI haveadmittedlytakena cavalierattitudewith respectto a numberof detailedpsychologicalissues.This is not becauseI do not think suchissuesareimportant,but ratherbecauseit is all tooeasy(for me,atleast)to get soenamoredwith psychologicalminutiae thattheforestis lost for the trees.Thus,theemphasishasbeenon theissuehow we will know a cognitivemodelcapableof guidingenvironmentaldesignwhenwe seeone. In orderto takesome positivestepsin this direction,I havehadto pointout thedeficiencesof varietyof existing modelingapproaches in cognitivescience.I havenot donethis out of lackof respectfor such research.It is simplythecase that the constraints that guide modeling in normal scientific practice are quite

different

than the constraints

give rise to problem these solutions support

decompositions,

I quite expect

for decomposing

the cognitive

operative

factors

as many

impediment Those

in human

to design,

environmental

machine

design

against

is in many models

aiding

strategies

the symptoms statement design models

welcome

I have portrayed.

unless

However,

theoretical

than the cause.

must police

that is the explicit

basis.

linked

of the kinds of psychological

community

process:

Constraints

integration,

and criticize

the social

my own proposals and physiological

from

the cognitive

the role of such factors

of our theories

problem

much

design

activity

Obviously

I believe

that treating

models

necessary

to support

for no one is likely

process

of behavior.

solving

of the chaos

is a major

as well.

that I have done for the design over-rationalization

and

as we must to

structures

cleaved

One must be on guard

to current

its own research, goal.

along

to describe

these criticisms

design.

give rise to sets of solutions,

be so easily

a far cry from the explicit

that are too tightly rather

cannot

may claim

in the modeling

be due to the lack of a sound

Perhaps

If the failure

experience

cases

else will come

systems

to support

we aim for theoretical

problem.

assume.

we should

with concrete

practice

modeling

modeling

decompositions

when

that someone

of us normally

have tried to guard design

problem

give rise to new problems

design.

stressors

that must guide

search

Current

over

of the design against

what I

process

proposing

may

design

lest one run the risk of treating the cause cognitive to produce

requires

an explicit

engineering. readily

The

applicable

Acknowledgments

I am grateful was supported Shively,

to Kim Vicente by NASA

technical

monitor.

for helpful

Ames

comments

grant NAG2-656

on a previous to the Georgia

draft of this paper. Institute

This research

of Technology,

Robert

J.

References

Anderson, Barwise, Press.

John R. (1990) Jon, and Perry,

The Adaptive John.

(1983)

Character Situations

of Thought.

Hillsdale,

and Attitudes.

NJ:

Cambridge,

Erlbaum.

MA:

The MIT

Beach, K. D. (1988) The role of external mnemonic symbols in acquiring an occupation. In M. M. Gruneberg, P. E. Morris, and R. N. Sykes, (Eds.), Practical Aspects of Memory.: Current R_search and Issue_, Vol. I. New York: Wiley. Birmingham, H. P. and Taylor, F. V. (1954) A design philosophy for man-machine systems. Proceedings of the I.R.E., Vol. XLI], No. 12, (pp. 1748-1758).

control

Bohlman, L. (1979). Aircraft accidents and the theory of the situation. Resource management the flight deck. Proceedings of a NASA/industry workshop. NASA Conference Proceedings 2120. Braitenberg, Valentino. The MIT Press.

(1984)

Vehicles:

Brehmer, B., and Joyce, North Holland.

C. R. B., Eds.,

Experiments

(1988)

in Synthetic

Human

Psychology.

Judgment:

The

Cambridge,

SJT View.

New

MA:

York:

Bridgeman, Bruce (1991) Separate visual representations for perception and for visually guided behavior. In Pictorial Comml,mication in Virtual and Real Environments. Stephen R. Ellis, Mary K. Kaiser, and Arthur J. Grunwald, (Eds.). New York: Taylor & Francis, (pp. 317-327). Brunswik, Egon. (1952) Encyclopedia of Unified

The conceptual framework of psychology. In, International Science (Vol. 1, No. 10). Chicago, IL: University of Chicago

Brunswik, Egon. (1956) g..xperiments. University

Perception and the Representative of California Press.

Design

Press.

of Psychological

Card, Stuart K., and Newell, Allen. (1989) Cognitive architectures. In Jerome I. Elkind, Stuart K. Card, Julian Hochberg, and Beverly Messick Huey, (Eds.), H0man Performance Models for Computer-Aided Encneering. Washington, D.C.: National Academy Press, (pp. 173-179). Carroll, J. M., and Campbell, human-computer interaction.

R.L. (1989) Artifacts as psychological Behaviour and Information Technology,

Carroll, John M. (1991) Introduction: John M. Carroll, fEd.). New York: Claxton, Claxton, Donald, Press.

Guy. (1988) fEd.), Growth Merlin.

(1991)

The Kittle House manifesto. In Designing Cambridge University Press, (pp. 1-16).

How do you tell a good cognitive Points in Cognition. New York: Origins

theories: The case of 8, (pp. 247-256).

of the Modem

Mind.

theory when Routledge.

Cambridge,

Edwards, E. (1988). Introductory overview. In E.L. Wiener Factors in Aviation. New York: Academic Press.

Interaction,

you see one?

MA:

Harvard

and D.C. Nagel

In G.

University

fEds.)

Human

Fischoff,B., Slovic,P., andLichtenstein,S. (1978) Fault trees:Sensitivityof estimatedfailure probabilitiesto problemrepresentation.Journal of Experimental Psychology: Human Perception and Performance,

4, (pp. 330-344).

Flach, John M. psychophysics. Flach, _,

(1990a) Control with an eye for perception: Precursors Ecological Psychology, Vol. 2, No. 2, (pp. 83-112).

John M. (1990b) 2(3), (pp.

The ecology 191-205).

Gibson, James J. (1979) Houghton-Mifflin.

The Ecolo_cal

Gibson, James J. Houghton-Mifflin.

The Senses

Gibson, Reasons

(1966)

James J. (1967/1982) for Realism. Hillsdale,

of human-machine

Approach

systems

to Visual

Considered

to an active

I: Introduction.

Perception.

as Perceptual

James J. Gibson autobiography. NJ: Erlbaum, (pp. 7-22).

Systems.

Ecological

Boston,

MA:

Boston,

MA:

In E. Reed

and R. Jones,

Hammond, Kenneth R., Hamm, Robert M., and Grassia, Janet. (1986) Generalizing over conditions by combining the multitrait-multimethod matrix and the representitive design of experiments. Psychological Bulletin, Vol. 100, No. 2, (pp. 257-269). Hammond, Kenneth R., Stewart, Thomas R., Brehmer, Berndt, and Steinmann, Derick O. (1975) Social judgment theory. Center for Research on Judgment and Policy Report No. 176, University of Colorado Institute for Behavioral Science. Hammond, Kenneth R., Hamm, Robert M., Grassia, Janet, and Pearson, T. (1987) Direct comparison of analytical and intuitive cognition in expert judgment. IEEE Transactions on Systems. Man. and Cybernetics, Vol. SMC-17, No. 5, (pp. 753-770). Hammond, Kenneth R. (1955) R_vicw, 62, (pp. 255-262).

Probabilistic

Hutchins, Edwin. (1991) How a cockpit NASA Ames Aviation Safety/Automation 1991.

functioning

and the clinical

remembers its speeds. Program Investigators

P. N. (1975) Models of deduction. and Process in Children and Adult_.

Kahneman, D. and Tversky, A. (1979) Econometrica, 47, (pp. 263-291).

Prospect

Kirlik, Alex, Miller, R. A., and Jagacinski, uncertain environment II: A process model submitted for publication. Klein, Gary A. (1989) Recognition-primed Machine Systems Research. Greenwich,

An analysis

at the Second August 14-16,

for Cognitive

In R. J. Falmagne, (Ed.), Hillsdale, NJ: Erlbaum.

theory:

Psychological

Paper presented Meeting eeting.

Hutchins, Edwin. (1988) The technology of team navigation. Institute Report 8804, University of California, San Diego, La Jolla, CA. Johnson-Laird, _Representation

method.

Science

Reasoning:

of decisions

under

risk.

Richard J. (1991) Supervisory control in a dynamic of skilled human-environment interaction. Manuscript

CT:

decisions. In W. B. Rouse, JAI Press, (pp. 47-92).

(Ed.),

Advances

in Man-

Kotovsky,K., Hayes,J. R., andSimon,H. A. (1985) Why aresomeproblemshard? Evidence from thetower of Hanoi. Cognitive Psychology. 17. Larkin, words.

a diagram

is (sometimes)

McRuer, D. T. and Jex, H. R. (1967) A review of quasi-linear 0n Homan Factors in Electronics, HFE-8, (3), (pp. 231-249).

pilot models.

Marr,

Jill H., and Simon, Cognitive Science, David.

(1982)

Herbert A. (1987) Why Vol. 11, (pp. 65-99).

Vision.

New

Meister, Hopkins

David. (1989) Con¢¢ptual University Press.

Meister,

David.

(1984)

Letter.

York:

of Human

Factors

Factors.

Society

Ulric.

Newell, Allen, Prentice-Hall.

(1976)

Cognition

and Simon,

Herbert

and Reality. A. (1972)

New

Bulletin,

York:

Human

27(10),

W.H.

Problem

Donald

Rasmussen, to Cognitive

A.

(in preparation)

Jens. (1986) En_neering.

Things

Information New York:

That Make

MD:

Freeman Solvin¢.

Us Smart.

The Johns

systems.

Address

and Company. Englewood

Cliffs,

and S. W. Draper,

Reading,

Processing and Human-Machine North-Holland.

Transactions

(p. 2).

perceptual

Norman, Donald A. (1986) Cognitive engineering. In D. A. Norman User Centered System Design. Hillsdale, NJ: Erlbaum. Norman, Wesley.

IEEE

Baltimore,

Neisser, Ulric. (1989) Direct perception and recognition as distinct presented to the Cognitive Science Society, August, 1989. Neisser,

a thousand

Freeman.

Aspects

Human

worth

MA:

NJ:

(Eds.),

Addison-

Interaction:

An Approach

Reitman, Walter, Nado, Robert, and Wilcox, Bruce. (1978) Machine perception: What makes it so hard for computers to see? In C. Wade Savage (Ed.), Perception and Co maition Issues in the Foundations of Psychology. Minnesota Studies in the Philosophy of Science. Volume IX. Minneapolis, MN: University of Minnesota Press, (pp. 65-88). Rouse, (pp.

William

B. (1987)

Much

ado about

data.

Human

Factors

Society

Bulletin,

30(9),

1-3).

Sarter, Nadine phenomenon.

B. and Woods, David D. Th_ Intcmat;ional Journal

(1991) Situation awareness: A critical but ill-defined of Aviation Psychology. 1, (1), (pp. 45-57).

Shebilske, Wayne, L. (1991) Visuomotor modularity, ontogeny and training high-performance skills with spatial instruments. In Pictorial Communication in Virtual and Real Environments, Stephen R. Ellis, Mary K. Kaiser, and Arthur J. Grunwald, (Eds.). New York: Taylor & Francis, (pp. 305-315). Sheridan, Factors.

Thomas B. (1987) Supervisory control. In G. Salvendy, New York: John Wiley and Sons, (pp. 1243-1268).

Suchman, Lucy. Communication.

(1987) Plans and Situated Actions: New York: Cambridge University

The Problem Press.

(Ed.),

Handbook

of Human-Machine

of Human

Tversky,A. andKahneman,D. (1981) Theframingof decisionsandthepsychologyof choice. Science.

211,

(pp. 453-458).

Wason, P. C. and Johnson-Laird, P. N. (1972) Content. Cambridge, MA: Harvard University

Psychology Press.

of Reasoning:

Structure

and

Whiteside, J., Bennett, J., and Holtzblatt, K. evolution. In M. Helander, fed.), Handbook Holland, (pp. 791-817).

(1988) Usability engineering: Our experience of Human-Computer Interaction. New York:

Wise, James A. (1985) Decisions in design: Behavioral Decision Making, George Wright

Analyzing and aiding the art of synthesis. fEd). New York: Plenum, (pp. 283-308).

In

Woods, David D. and Roth, E. M. (1988) Cognitive systems engineering. In M. Helander fed.), Handbook of Human-Computer Interaction. New York: North Holland, (pp. 3-43).

and North

f-

J S

/

Figure

Figure

1. The Lens

model

Captions

of Brunswik.

Figure 2. The Ecological task analysis framework. Like the Lens model framework for human judgment, the ecological task analysis framework represents the integrated human-environment system in a symmetrical arrangement. Matches between the perceptually-oriented and actionoriented environmental differentiations indicate opportunities for the fluent perceptual guidance of activity. Structural mismatches between the models of surface perceptual structure and surface action structure are indicative of demands for cognitive activity, and corresponding models of the environmental depth structure. Figure 3. Four possible results of an ecological task analysis. Each case reflects a different type of match or mismatch in environmental perception and action strucuture, and is thus indicative of a different form of cognitive demands, and a different type of remedial interface design solution. Figure

4. Map display

showing

Figure

5. Search

produced

paths

home

base, the scout,

by two different

and forested

subjects

areas.

in the same world

Figure 6. Four representations of the same world. (a) Displayed affordance representation. (c) Sighting affordance representation. representation scaled to emphasize local optima in the affordance

configuration.

representation. (b) Locomotion (d) Search affordance structure.

Figure 7. Process model of search path generation. Local optima in the search affordance structure are organized into a series of waypoints via a generate-and-test scheduler. Detailed scout motion is produced by a combination of attractive forces from both the current waypoint and the local search affordance structure in the vicinity of the scout.