(upbeat gentle music)
- Thank you very much.
We are very excited to present this workshop to you today.
It's a bit of an experiment in and of itself.
We have a vast body of knowledge to cover,
and we've kind of distilled that down to what we think
are the fundamentals for what you,
as software engineers will need to know.
As Penelope said, this is a very interactive workshops.
So we're gonna kind of have you turn
and chat with people in your groups.
And then we have Daniella and Ty will be running some
microphones around at the outset.
So I wanted to kind of start off by getting a sense
of who is in the room.
And we do have to do this little dance of not getting close
to each other.
So,
who in this room is a on-call engineer or
site reliability engineer?
Okay.
- Raise your hand, if you've ever been on-call?
And this will work, this is gonna work.
(audience laughing)
It's fine if you haven't been on call, that's totally cool.
Just wanna get a sense.
- Do we have any marketing product folks?
Customer folks in the room?
One, good, can we are representative voice too?
Okay. Excellent.
All right. Sweet.
Well, let's get into it.
So who are we?
So I am a researcher with jelly.io
and I've been working in high risk,
high consequence industries for most of the last 15 years.
And my interest in software came about during
my doctoral program at the Ohio State University.
And I'd come from an industry where things
are very dramatic, there's a lot of high speed.
There's a lot of high pressure.
There's a lot of adventure in the industries
that I worked in.
And when I was first told that I was working in software,
I was like, "What really?"
And then as I started to get more and more involved
in understanding what and how you do,
I realized this is actually the quintessential occupation
to really understand cognitive work
and to really understand what we're gonna talk about
here today, joint cognitive systems.
So I'm gonna pull in, we're gonna pull in a number
of examples from different industries
and kind of draw that through line for you.
- My name is John Allspaw and like Laura mentioned,
I got addicted.
I mean, enamored about seeing connections
between what we all do.
My background is in software.
I come from,
I would say,
infrastructural bent in my early career,
and worked in a bunch of different places.
None of that background really is interesting.
So I don't wanna spend too much time on it.
The thing that's important that,
that I wanna get across
is that like workshop like this talks that hit on similar
types of themes is all part of what I am pretty confident
is a, perhaps a slower than we would like perspective shift
in the industry.
What we're gonna be talking about today it's so close
to what you all experience
that we sometimes don't really pay much attention to it
because we are so good at it.
All the three describe that later.
- Yeah. And I think one thing that we should also mention
just to follow on that is that we met at Lund University
in the Human Factors and System Safety Design Program.
And I was really struck by the similarities between the work
that I was doing and the things that John was seeing
in the software world.
And so that was almost 10 years ago
and we are both still continually learning.
We're continually surprised by what we find
when we actually look closely at people doing real work
in real world contexts, it's amazingly sophisticated
and it's fascinating, but it is a long journey.
And so our goals with what we want to accomplish here today
is we are in essence coaches here.
And just as if you were to spend two hours on the pitch,
you know, you might know some of the movements,
you might know some of the plays,
but you're not gonna bend it like Beckham necessarily.
So we want to kind of set that expectation right off the bat
that what we're doing here is we're gonna challenge
some of the paradigms, some of the ways
that you've been thinking about how you develop technology,
how do you deploy it into fields of practice?
How do your users interact with it?
How do you interact with the tools that you have?
And so at the end of the day here,
we hope that you walk away with questions.
You walk away with a lot of interest
in kind of pursuing this further.
- Yeah, this is, as Laura mentioned this is an experiment.
I mean, we have two hours.
We're gonna talk about a core concept
that touches every part of an entire field of engineering.
One that also is not very well known.
So the bar is pretty high.
Again, if you're interested, there's something paradoxal.
If we do our job here and you all are into it,
you might actually leave somewhat disappointed
or slightly unsatisfying,
and that you will want to know more
because we simply can't get to it.
That's still a win though for us I think.
- And so one of the things that I would actually correct
about what you just said is like a field of engineering,
but we're gonna talk about multiple sort of interacting
kind of fields.
Some of which you may have heard of,
some of which you haven't.
And when we were talking about like,
well, how many people in the room actually know
what resilience engineering is,
or have heard the term resilience engineering?
Okay. Yeah.
You've probably seen some of it on Twitter.
You might've heard some talks at different conferences.
So what we're gonna break down for you
is kind of these connections between these different fields
and how they relate to software engineering.
And so, as we were talking about this, we were like,
how do we make this connection for people
without getting into this nerdy academic talk?
And John had a really good example.
- Yeah. So see this green bit here,
this smaller circle joint cognitive systems.
This is like a core idea.
Everything within the multidisciplinary field
of cognitive systems engineering relates and connects to it.
Think of it a little bit like this,
if you're at all familiar or even not,
but have heard of the field of statistics, right?
Think of if statistics is it all about normal distributions
and averages and mode and median and all that sort of thing?
No, it's not all of it, but it's certainly
a core part of it, right?
It would be very weird to talk about statistics
without having that as a bit of a framing.
So think about it a little bit like that,
thing is that this workshop
isn't on cognitive systems engineering
'cause frankly we don't have time
and largest spent a dissertation on it.
So we're gonna just focus on this little bit, okay?
Does that make sense?
- Yeah. So we wanted to do a little bit of like bottom line
up front for you here.
And so at a very high level,
these are the core concepts that we're gonna talk about.
The first one is that all work is cognitive work.
So we're gonna kind of break this down a little bit more,
but we're gonna talk about what does it mean
to perceive change in events in your world?
What does it mean to make sense of those?
How do you assess the meaning and implications
of that change as it's happening in your world?
The second core point is that work is always distributed
across different agents.
And so here in the joint cognitive systems world,
we're gonna be talking predominantly about how you share
this cognitive work,
this thinking with both machine agents and human agents.
And so when I talk about machines,
I kind of use that interchangeably,
talking about automation or a number of different kind
of like artificial intelligence.
So we're getting a little hand-wavy with that,
but that's the level of analysis
that we're looking at there.
The third piece is it, these machine co-workers
can actually kind of be jerk sometimes
they don't always help you.
They don't always support you in ways
that can either be slightly frustrating
or they can be catastrophic.
And the last one is that we actually can design
and develop these systems of work for greater safety,
greater productivity, and overall more resilient
and robust systems.
- I think we're at a disadvantage because I didn't manage
to get the computer to make the next slides
on our presenter notes.
So I've no idea what's gonna come next.
- Well, we wanted this to be an interactive workshop.
So we've got a series of exercises throughout the session.
And we wanna start with those.
So with understanding what is a joint cognitive system.
- Right, so I want you to imagine, okay,
that it's the end of your day.
And you know,
the dog's been taken for a walk or you finished dinner,
whatever you do in your, like a routine.
And you're, you know, ready to settle down go to bed
and you get an alert of some sort and you get a page
and go to look at it, it looks something like this.
Okay?
So what we wanna do, I wanna ask you a question,
but before we do that, I wanna see if you'll humorous here
would love, we've got three exercises in this
and it'd be great if we could maybe sort of come up
with, or maybe two tables for a group or yeah.
Two tables for a group.
- [Laura] Just turning around-
- Turn around-
- [Laura] Working with the group behind you.
- Right. And so that would be, we'd have one, two,
three, four, so a lot of groups.
And so I'm gonna ask you to spend a couple minutes
on this question.
And the question is, you get this page alert, okay?
What do you think you would do next?
That's it.
Granted you, there's lots in here.
We want you to talk with your neighbors
about things that might occur to you
in this hypothetical world that you would do next,
given that you've just gotten this, okay?
So spend two minutes talking, anything that comes to mind.
Sounds like you've got some things to talk about.
Of course you might be saying,
I don't know what the hell is happening
or what the story is here.
How are we gonna do this?
Do we want to have.
- So what we'll do is we will,
Daniella and Ty front and the back of the room.
If you've got something to say,
raise your hand and they will run over,
we'll give him a bit of a workout.
So what would you do next?
- What's an idea that's come to ya?
- A brave soul in the back for the first.
- [Man] I checked for urgency.
How quickly has this become a problem?
How quickly is it gonna become a bigger problem?
- Ah, well, can I ask you just real quick?
How do you check for urgency?
- [Man] Well, hopefully I've got something
that's got charts that show me how much free space
or other issues.
- Excellent. Excellent.
Okay. Super.
What else?
Any other ideas?
- [Man] Maybe ping DevOps and go back to sleep.
(all laughing)
- Excellent.
- Legit.
Legit, we've got someone in the back there.
- [Man] I would ask why are we getting this notification
at 99% and not sooner?
I'd probably recalibrate that.
- Yeah. - Yeah.
That's never been where you're like,
why is this not fitting working?
- [Woman] We said that historical context might matter.
Like if we'd seen this alert before and knew what it was,
versus if it was a brand new alert, we'd react differently.
- Mm-mmh. - Yes. Yes.
- Okay. So you have a sense of what nominal
and often nominal is.
Betsy?
- [Betsy] One thing I found kind of scary
was the warning to warning transition
another member of my group pointed that out,
because that means that we don't actually know
if it's been going for two minutes or for much longer.
- Yeah. I see connection between that and what a couple
of other folks have said as well.
- [Woman] I actually thought the same thing
because state is now warning and also was warning,
which either means this is the thing we don't care enough
to do anything about.
And we're just ignoring these warnings
or someone's been ignoring it for too long.
- Okay. So do you go back to bed?
(all laughing)
- All right, this is, thank you.
Thank you. One more.
Any anybody else?
- [Man] I'd probably log into Web 121
and actually make sure it's not a false positive.
- All right.
- Okay, some forms of verification, yeah.
- Yeah, yeah.
So this is excellent.
They did all of the A plus, go ahead.
(audience clapping) - Yeah, thanks again.
So I have a question for you.
So if this was your human coworker
who phones you in the middle of the night
and just told you this and then hung up.
(audience laughing)
How happy would you be with them?
So, yeah, boo.
So this is fundamentally what we're talking about
when we talk about joint cognitive systems
is your coworkers, your colleague, your machine colleague,
you know, kind of underperforming and doing things
that are just kind of mildly irritating like this,
like under specifying what the urgency is,
whether it's important, whether this has happened before.
Sometimes they're just like,
they're hard to collaborate with.
It's frustrating, it gets like confusing.
And it adds a whole bunch of extra work to try and sort out
what's going on.
And sometimes the consequences of having
a really bad team player
or a really poor coworker are deadly.
So this stuff matters and it matters in a way
that is increasing.
As we increase the speed and the scale of critical digital
infrastructure, we move more of societies, you know,
core functions into the cloud
and onto digital infrastructure.
This stuff really matters.
It's not just frustrating.
So here's where we're gonna go.
We've already gone through one of the exercises.
Nice work.
John's gonna talk a little bit about, like,
where did this thinking come from,
and give you a bit of the background.
This is, there's 40 years of literature and of studies
going on in these other high risk,
high consequence type domains.
And that's where we're drawing a lot of this theory.
And a lot of this knowledge from.
We're gonna deep dive into what is cognitive work,
so that we can kinda break that down a little bit more
and understand how it is distributed
across a joint cognitive system.
We're gonna do another little exercise there,
and then we'll kind of walk into,
if we have one machine, one human,
is that a different interaction than many machines,
many humans, spoiler alert, it is.
And then we'll get you to do a little bit of an activity
they're trying to understand, like,
how do we think about distributed cognition
in these kinds of scenarios?
- All right. So here it is.
I'm about to do of extremely abbreviated and cherry picked
history that starts around World War II
to where we are now, ready?
I can tell you a very, very quick story
and just maybe some of you have already heard some of this.
So 1943,
the US Army was like a lot of, not just militaries,
but lots of large multinational organizations.
It was all the rage back then that this idea,
this notion, this stamps that you could hire people
and in the process of hiring them and interviewing them,
you could work out what they're good at.
And then put them in a role where you could do it, right?
It was seen as a problem of selection.
You would take some like competency framework,
people are like this, then you could do this
and like that you can do this.
You would fit the person to the job.
Here's the issue.
What you see here is B-17 at the time,
the most like advanced military apparatus
that had ever really been perceived.
And it was brand new.
It was known as the Flying Fortress.
Here's the issue, in 43, what would happen,
they would keep crashing.
And I don't mean crashing as in go out to battle
and then they got shut down or,
they would go off to battle and these like amazingly
sophisticated machines would be flown back.
They would land on the runway and everybody's almost home
and halfway down the runway,
the landing gears would be retracted
and they would belly flop onto the thing.
Sometimes with bombs still on board.
And so this seems to been an issue
and it was happening with enough frequency
that the army was like, okay, this is a big deal.
What's going on.
It can't be the planes.
Cause they're amazing and they are very sophisticated.
It must be something about the pilots.
And so for example, like, oh,
I'll show you a picture of what these looks like.
So they went to find this, in my opinion, very handsome,
sort of somewhat Hollywood style looking dude,
one of the first psychologists engineering, psychologists,
as they call them back then, Alphones Chapanis
and basically brought him in to ostensibly figure out
what was psychologically wrong with the pilots.
And they said,
because they thought they had a recruiting problem,
Dr. Chapanis please help us work this out.
And so, he shows up the basically says,
"Oh, okay, doctor, we've got all of the pilots here.
You can go and interview and talk to them, we're gonna",
he says, "Okay, okay.
But before we do that,
certainly I wanna talk to them, where do they work?
Show me where they work."
And so he, they bring them to a cockpit, by the way,
this is B-17.
This is how hardcore these things are.
This landed and everybody was fine.
Like, it was a big, big deal,
the Volvo of, I dunno, whatever, right?
And so like, I got, again,
can't be something wrong with the plane.
So he was like, "This doesn't really make sense."
And so they brought him, here's the B-17 cockpit.
And what I pointed out here, there's he noticed this, huh?
So the flap control the thing that you put into place,
you know, activate the flaps to slow the plane down
right next to the landing gear control.
And they said, "Well, where's the other?"
And there was one other plane, the P-47,
it was in service a lot longer.
And he said, "Alright."
And he noticed something about this.
They're nowhere near each other.
They're different colors.
They're different shape.
And as it turned out, as you might imagine,
you see where I'm going.
Is that a lot, if not a majority
of those accidental belly flops happened from pilots
who had previously flown the P-47
and we're adjusting new to the B-17.
It's not that only B-17 pilots did it,
but it was certainly something of interest.
And so what this brought was to this day,
if you take a flight and walk and pass it,
take a look in there, on an Airbus and Boeing aircraft,
you'll notice that the flap controls feel like a flap.
The landing gear fuels like a wheel,
and they're not really next to each other, right?
This is still in place now.
So the idea here is sort of quite succinctly summed up
by what James Reason said, which is that,
"We can't change the human condition,
but we can change the conditions under which humans work."
This was pretty mind blowing at the time.
The idea that you could design the environment of work,
the stuff, the tools that people are using
in such a way where it's less likely
for them to make a mistake,
rather than let's pour all our effort into figuring out
which of the pilots are accident prone, and try to fix them.
Sounds weird and pretty basic now,
but it was really fundamental.
And so this, as soon as that happened,
this is what I just told you is some would say
maybe the origin story of the field of human factors,
a couple of things came from this
and it span throughout the following decades.
The first was safety can be encoded in the design
of technology.
That accidents can be avoided by having more automation.
That procedures can be specified to be objective
and completely specified.
And that operators just have to follow the procedures
to get the work done.
The joke, of course, in aviation,
sometimes in nuclear power was,
what does it take to run a aircraft carrier?
Well, a dog and a human,
the human is there to feed the dog and the dog is there
to make sure the human doesn't touch anything, right.
And the joke that goes with this.
So this notion was that, you could come up with this,
like a bit of a framework that look,
humans are better at these things
and machines are better at these other things.
Sometimes we call that HABA-MABA,
and this persistent, here's this idea, Paul Fitts,
34-7, did literally wrote this out, right?
You might find something familiar in this.
Here are all the things that humans surpass machines
in these abilities.
And likewise, the machine surpassed humans
in these abilities.
So this idea that you take these,
use this to guide how you set up work environments
and design things, sounds pretty intuitive.
It feels like, ah, this was known as functional allocation.
You do this part and I'll do this part.
So, this lasted for a while.
Lasted until roughly around March 28th, 1979
is oddly specific.
Those of you who've ever heard of the accident
at Three Mile Island might not know
that it was so seminal in the world of human factors,
that it completely flipped upside down.
How we understood people made decisions
on how they did their work.
I'm not gonna go into the accident
'cause we don't have enough time,
but this was a seminal watershed moment.
This flipped all of the things that I pointed out earlier,
a good number of them either turned out not to be true
or needed a lot of adjustment.
So these new beliefs, the Three Mile Island
and subsequent accidents later
said was that it revealed that when it comes to safety
and cognitive work, automation is necessary,
but also introduces new forms of challenges and risk,
doesn't come for free.
Second is that rules and procedures
are always under specified.
Raise your hand if where you work, there's some sort of,
you have a runbooks on your Wiki.
Okay.
Raise your hand if you think you could find right now,
one of those runbooks where if you followed it exactly
to the letter, skipping nothing,
you're gonna have a bad time.
Exactly.
So the idea is that these procedures cannot guarantee safety
by themselves.
They have to be interpreted in local context.
Events in these environments will require people
to make decisions and take action that can't be set ahead
a priority put ahead, right?
So therefore they're gonna need support foreshadowing.
Some methods and models for risk rely
on human error categories, accounting,
blah, blah, blah, blah,
a scientific word for this as horseshit.
It doesn't really work.
Sounds really intuitive, but it doesn't work.
And that linear models of accidents, dominoes falling down,
linear chains, blah, blah, blah.
Also don't work.
So this brings us to this notion,
in the mid 80s a number of different folks came together
and talked about this in the wake of Three Mile Island
about all of these topics that we've got thus far.
One of these ideas is the concept
of a joint cognitive system.
In the kernel of this are two perspectives.
The first is that people using technology,
you all have a model, a mental model
of the technical system, whatever that is, right?
Whatever, all of the bucket of descriptors,
applications and infrastructure and architectures and blah,
blah, blah, right?
You have a mental model of how it works, what it can do,
what it can't do, what it looks like when it's broken
and that sort of thing.
You have this story of mental model.
Likewise, the technical system can also be said
to have an image of you.
It has an image of what you might need
of what you might use or know about how it is to be used
and how it can't be used, right?
This is the key part.
This is the perspective shift, right?
So this is sort of the gist,
this brings us up to where we are,
the histories sort of over.
- Kinda, I mean, it brings us to where we are today,
but I have a little bit of a beef with kind of how this gets
presented because I wouldn't be standing here today.
If everything after Three Mile Island was based
on these new views of safety, we have a lot of legacy,
a lot of leftover residue of these old models
of how we think about safety and risk in the workplace.
So, as a researcher, who's kind of studied this domain
for much of the last five years.
Like I have opinions about where the industry is odd
and where it's going and how quickly it's going there.
But I'm actually curious about,
for those of you who in the room
where you think your organization is at relative
to some of the concepts that John just talked about.
So show of hands,
how many of you believe that your company,
or that your company believes that accidents can be avoided
by having more automation?
Okay. Kind of maybe a few on the fence.
They're not too sure.
How many feel like your company thinks that it's necessary,
but it also introduces new forms of challenges and risks?
That's great, that's really inspiring to see.
How about your company believes that you can put everything
in runbooks if your documentation is up to date,
there'll be no problems?
(John laughing)
- [John] I want to talk to you later on that.
- Remember, this is your company's view, not your view.
I know everybody in here probably realizes that's not true.
And how about like that rules and procedures
are always under specified
and so therefore when you're in a local situation,
you have to interpret those?
Yeah. It depends, right?
What about engineers just have to follow the procedures
to get the work done?
How many organizations?
Oh, nice.
Okay. Few in the back there.
And what about the modern software operations
require engineers to make decisions and take actions
that can't be a pre-specified?
Nice. Okay.
So we have a lot of organizations in the room
here who are like already thinking about this idea
that you can put structures in place to help guide actions,
but those are not gonna take you to the end of the road.
At some point, an engineer or a collection of engineers
they're gonna face novel or unanticipated problems
and they're gonna have to adapt in real time.
Unfortunately, during service outages or during
time pressured situations,
it can become a very cognitively demanding situation.
So when we talk about these kinds of, you know,
where we're at today in the software industry,
what we see from working
with a number of different organizations
is that these new ideas are taking hold.
But as we can see in the room,
those old ideas are still very much entrenched
in our design our development process.
And yet what we see is that there's increasing complexity,
increasing speeds, and increasing scales
that these systems are running at.
And so a lot of people had sort of asked me
when I first got started in software engineering like,
what are you doing studying high risk, high consequence work
like software engineers, really?
But technology is so inextricably linked
to any kind of modern work world.
And we're becoming increasingly dependent
on relying on these systems to navigate
much of society's critical infrastructure.
And so we see examples again and again,
of really poor design and very poorly supported,
real time sense making, many of which have pretty profound
implications, both for the organizations that face them
and as well for society as a whole.
So this stuff really matters.
And even if you are only running an app that say generates
random cat memes or something,
which is objectively a public service,
so I'm not calling it out here.
We can still study the ways in which you,
as a software engineer interact with your technology,
how you work in teams to keep that system up and running
and continuously developing.
So we can study and understand how joint cognitive systems
actually work across the spectrum.
So what do we actually mean by a joint cognitive system?
A joint cognitive system takes a systems level view
of people in technology in the real world.
So one of the fundamental differences
that John was sort of describing
in how Chapanis is looked at the world
versus how kind of engineering psychology had taken place
before then was he went out into the real world.
He didn't study this in a lab environment.
He didn't create these kind of toy problems.
And then test, you know, students who are non-experts.
He went out and looked at real world contexts.
And so that is fundamental to join cognitive systems.
Instead of this,
humans are better at, machines are better at, dichotomy.
We look at, what do machines need people for,
and what do people need machines for?
And so when we take that view,
we say they both have cognitive capacities.
Machines can do things at a larger scale.
They can do them faster.
They have capabilities that support and augment human
cognition as well, but they lack context.
They kind of fall down when they're outside
of their prescribed for boundaries.
And lastly, the unit of analysis, therefore, is that,
it's the cooperation between these agents
in joint activity.
And so this seems really self-evident that like,
okay, yeah, you have to understand interactions.
But when you start thinking about cognitive work
being shared across two agents, it becomes less about,
okay, you do this and then you tell me when you're done,
and more about, do you provide the kinds of information
that helped me be ready and prime to take action?
Or do you leave me hanging in the example
that we described earlier?
And so, the primary challenge of thinking about this concept
of joint cognitive systems is how do we make automation?
How do we design automation to be better team players?
So how many of you, while I was talking were just like,
wait a minute, this is inter user interface design
or human computer interaction?
Show of hands.
Okay, only a few.
Perfect. All right.
That's okay.
That's legit because they are related.
Just as we were talking about earlier
of kind of how joint cognitive systems is a concept
that's nested within this broader body
of cognitive systems engineering.
We can also look at the ways in which user interface design,
user experience design and human computer interaction
kind of uses some of the concepts and sort of understand
share some of these same kinds of perspectives,
but they're limited in one really important way.
And that's that they don't necessarily focus
on the cognitive work that's inherent.
They look at sort of task allocation.
They might look at sort of optimizing the human in certain
situations, but they're not necessarily thinking about like,
what is the thinking that's taking place.
So it's about time that we actually talked about
what do we mean by cognitive work?
So cognitive work, the cognitive functions that we have
is basically how we stay alive in the world.
We perceive if there's risks or threats in the world
and we adjust and change our behaviors
to be able to avoid those risks or to minimize
the impacts on us.
We have different sort of priorities and goals
that we want to achieve.
But the world is always changing and doesn't always
kind of provide the opportunities to enact those goals
and priorities.
So we have to continually be adjusting and shifting
reprioritizing relative to the demands on our time,
the demands on our attention,
or the changing demands of the environment around us.
So fundamentally when we think about cognitive work,
it's asking the questions like,
what is actually happening here?
How much information do I have about the situation
that's in front of me,
which might be something as simple as,
not that they are simple, but as, you know,
building a new feature or as, you know,
kind of doing a migration,
it may be something as complex as you know,
our early example with the alert is,
is this something that is gonna eat my lunch
and I need to deal with it right away.
I think that's kind of a Canadian saying,
actually, eat my lunch.
It's about saying what should be happening.
So I see what's happening in the world.
I'm making sense of that,
but is this what I expect to be going on at this point
in time?
Is this off, off my expectations?
What does this actually mean for the goals
that I'm trying to accomplish?
Why is it happening in this way?
Is this something that we have seen before?
Is this deferred maintenance,
is this, you know, increased user volume that's expected?
What's gonna happen next?
So this one's really fundamental when we start thinking
about joint cognitive systems, because when you're designing
that kind of shared cognitive effort,
the ability to anticipate what's gonna be appropriate
action, and how do we move forward in constructive ways
is really important.
And then of course, someone mentioned trajectories,
how quickly is this happening?
How long has this been happening?
Really understanding that what's happening across time
is very fundamental to being able to act appropriately.
And then what options do I actually have here?
And how can I prepare and what resources,
how can I actually cope with this situation as a tendering?
So all of those functions that are coming up
on the right-hand side of the screen there,
observing, anticipating, recognizing change,
re-planning, reprioritizing.
This is the stuff of cognitive work.
This is how you make things happen in your job,
in your day to day lives.
So we've told you a little bit about what cognitive work is.
We're gonna go back to our exercise there
and see if you can actually spot the cognitive work
there as well.
- All right.
So let's go back to this as an aside, by the way,
is anybody recognize the structure mostly curious
of this particular representation of an alert
for those of us who are old in the room
or used to be a alerting system called Nagios,
and this is the default template for what disc space alert
would have.
This is the default.
So we're gonna do exactly what we did before,
where you're gonna talk amongst yourselves.
And it'll give you a different question here.
And the question in this situation,
this is the same alert that we talked about before.
What sort of things do you think you'd need
in order to determine if you can confidently ignore
this alert?
Okay?
What do you think you would need beyond what's here to know
if you can dismiss this as noise or something
I don't have to do anything about, okay?
So take two or three minutes, talk amongst yourselves,
and we're gonna ask you what you came up with, okay?
How about it?
All right, everybody. (indistinct chattering)
So let's see what y'all came up with, right?
So I take that all of you came up with amazing answers.
Every one of you will know, just kidding,
who wants to offer up an idea about what they would need
in order to determine if this is an alert you could ignore?
- [Woman] Very basic,
but what are we even using this disc space for?
- Okay. All right.
Excellent. Excellent.
That's a very good question.
- [Man] Maybe the context has this occurred before,
like say it occurs every day or,
or also how much space has been used over time
to slow growth or fast growth.
- Yeah. Yeah.
So it sounds similar.
That's that's excellent.
It sounds similar to reasonable answers for the first
exercise as well.
Yeah. Yeah, excellent.
Who else?
- [Man] Yeah. I guess what I'm web 121.
What it's like business purposes.
What it's a filling?
- Okay. Yeah.
- So you're thinking in terms of prioritization, then.
Yeah okay.
- [Man] I guess just is the discontinuing to be written to,
or is it stopped?
Has the writing stopped?
- Yeah, yeah, excellent.
Excellent, all right, so-
- We got one more.
- Oh yeah.
One more.
- Oh, two more.
We got time for you both.
- [Man] Need to understand what the last line
is saying there, are we running out of space in a volume
or on the entire server?
The megabyte number seems interesting.
Seems like a lot of space is free if I'm reading that right.
And then the 9%, like where are we talking about?
- Yeah. Excellent.
Super.
- [Woman] Like, who's expected to resolve this alert?
Is it expected to be resolved by a human
or is it expected to be resolved by some sort of automation
that, you know, is there a service monitoring this
that will automatically up the disc space
and this is just letting us know about it?
- Yeah. Which may lead you to wonder,
why are you even telling me.
Anybody else?
Over here?
- [Man] What even is the threshold that we're crossing?
Is it low?
Is it high?
Are we above or are we below?
- Ah, ha great.
- [Man] Is this a production environment?
- Excellent.
(audience laughing) Excellent. Yeah.
Yeah. All right.
So look, I wanna reflect a little bit back
about what you've all, a number of comments,
a number of directions that might be useful to help you
understand is this a big deal or whatever.
I just wanna point out the implicit expectation
that no one questioned,
which is alerts,
you can get alerts that don't require anything
'cause they're annoying.
Right?
The other thing that struck me is a number of you
made comments and with the first exercise as well
about the content of the notification, right?
A lot of you were like, I don't even know what this means,
and I'm not entirely sure if it matters.
You even the confusing, right.
It noticed its disc warning and then it reports free space,
not used space.
That's also a little bit, a little bit strange.
Is 9% what's left or, and is I node equals 99?
Is that important?
I don't know what that looks like.
It doesn't say anything else about that.
Warning, warning for what?
Right?
Even if you were to go out on a limb and say like,
oh, we've got warning and then we've got critical.
Well, how far are we from critical?
The second thing that others pointed out
is that the fact that you got this notification,
let's say you actually did understand,
but you had no idea about what passed,
what of the recent past, right?
Has disc space been alerting all over the place recently?
Has Web 121 been freaking out
in not just disc space recently?
Or is this alert come out of nowhere after no alert
about disc space or from 121 has been seen in the wild
for two years or whatever, right.
And so the there's content that's important.
And you all know the fact that this is a terrible alert.
And again, like Laura inferred later,
if if you were at work, let's say,
imagine we all used to be in an office together
and you're, you know, you're typing away and you know,
and Lisa's next to you.
And Lisa says,
"huh, web 121, disc spaces in warning."
And then like went back to work.
You'd be like, "She, okay?"
You wouldn't do that.
The exchanges you have are fluid, they're really quick,
really quick, right?
There's some expectation that you and Lisa
have some sort of understanding.
Let's say, if you're getting the alert, like, oh yeah.
Web 121 is squawking again.
Oh, is it date 01?
Yeah. All right.
I'll handle it, right?
Those things are very fluid.
So in this way, the alerting system and the content
has a really terrible image of what's important for you,
okay?
What is, I'm gonna push you y'all here.
Given what we talked about in a couple slides ago,
Laura walked us through cognitive.
How would you describe the cognitive work hypothesis
for workshop sake goes into dismissing or not dismissing.
I'll throw out an idea here in that somebody mentioned
is this one of many in the recent past?
Ah, so it turns out that memory has some something
to do with this, right.
Temporality has something to do with this.
Not just that you remember that web 121 is particular
or not particular important or not important
for some reason.
It's in staging, it's in production.
You have stuff understandings when you go to look at this,
right?
What else?
You may have a picture in your mind about thresholds
alerting the most basic form of alerting,
which has all sorts of pros and cons to it,
or static threshold alerts, right?
As somebody pointed out, I'll just put a more,
a finer point on it.
Did it go, let's say it did say that warning is at 85%
and criticals at 90%.
What I heard from all of you
in the last couple of exercises, okay.
One thing I'd wanna know is did it go from 20%
to 85% in the last five minutes?
Or has it been at 84.99% full for the last three years?
And just so happened to over it, right?
That would matter, that would be different.
You see what we're saying?
So the cognitive work that you are doing can be hindered
or helped by the design of the alert in and of itself.
- And I think one of the important things that you mentioned
was this idea that you can actually ask Lisa, like,
what do you mean?
You could ask Lisa some of these questions.
So in CSE parlance or in joint cognitive systems,
it's can you actually interrogate your co-coworker?
Can you actually interact with them and extract new
and different information that's gonna help you
and your sense-making process,
help you in your ability to prioritize it
or to recruit and gather more people
who might have different knowledge bases than you do.
- You'll recognize some of these,
this slide is just pointer Earl Wiener in the early days
of autopilot in aviation ran a number of studies
at NASA Ames in Mountain View, California.
And after the pilots were done,
first time they'd ever encountered a autopilot
recordings of them actually flying with autopilot,
doing some stuff.
They said things like, "What's it doing now?
Why is it doing that?
What will it do next?
How did it get into this state?"
Have these questions ever occurred to you
when you're working in your code?
(audience laughing)
This is what we mean by cognitive work.
And these are the connections between these other domains.
- [Laura] And that was 1989, by the way,
we're still answering those questions.
- Wanna take that?
- Go for it.
- Okay. So what makes a machine a team player?
I'm gonna give a little bit of a description.
Now, I want you to imagine that alert earlier.
Somebody I worked with a long time ago
wrote a like a plugin, I think,
to help in that case, Nagios,
but insert Prometheus or whatever sort of alerting system,
they're all terrible in their own ways.
And what he did was he would shadow people
who were actually on call,
and when they got a disc space alert,
he would try to work out and talk with them,
either wake up with them or the next day.
What exactly the question I asked
we asked in the first exercise,
what do you think you would do next?
And a lot of them gave the same answers.
So he changed the content of the alert
to include some pictorial graphic representation
of the volume.
That's an issue.
Some amount of historical context you'll note
that this looks like it's just a bumped over, right?
Tells you what the threshold was,
what the volume was amongst all of the mounted volumes
and how it has been.
And the frequency of the past alert,
computer knows all this.
You could help it help you by doing this.
And this is an example of when we say context,
everybody needs more context.
Excellent question is, what context specifically?
You'll know there's a whole bunch of stuff
that's not included in here that you could imagine
might be really helpful.
You can't include all context
'cause then you'd have two problems.
You'd be getting the alert and now you have to navigate
what the hell the alert is trying to tell you,
you were already doing that before.
But if but so overload is also an issue.
This is an example, just one of many I'm almost certainly
and this is even a dated screenshot.
This gets you a sense of what makes a machine,
a team player?
- Yeah. So just to put a bit of a finer point on this,
when we talk about the sort of the design characteristics
of what makes a machine a team player is fundamentally
that it is a cognitive aid.
So all of the things that we were talking about
is supporting your cognition, your sense-making,
your ability to put these things into context.
And so the machine needs to be designed and fielded in a way
that supports your ability to make sense of things.
Earlier when I talked about kind of looking at the real
world of work, some important considerations
about what does it mean to be a cognitive aid
includes things like, are you sitting in a quiet room
with nothing else to do just with your fingers
on the keyboard, ready to receive an alert.
Most likely not.
Your attention is distributed across multiple threads
of activity.
You might be in a slack conversation.
You might be in a meeting,
you might be dealing with some other kind of customer
request.
So understanding to John's point about this data scarcity
versus data overload is like,
what information is important at what points in time
to actually serve as an aid and not a hindrance to you.
It supports functions, not just specific tasks.
So it allows you to anticipate what might be useful next.
It allows you to reprioritize to say,
this was important, but given new information,
I need to drop some of these goals,
or I need to like move up certain things over others.
It supports human judgment and expertise.
Again, we are some of the most complex
and sophisticated cognitive agents on the planet
that are able to work in real world conditions.
And so, supporting your judgment and your expertise,
your ability to discern the right course of action
from the wrong in any given situation is critical.
And as we talked about with Lisa,
that ability to be interrogated, can you ask an answer
or does it help lead you to where more information for you
to be able to answer those questions are.
So it has affordances.
So it has some sorts of implicit designs that show you
how to use it, how to interact with it,
and what the limitations of its capabilities may be.
And it aids anomaly detection.
So this is, you know,
many of you talked about when you were sort of identifying
what you would need to know whether or not was something
you could ignore or not.
Is this nominal, is this normal?
Is this expected for this given point in time
or age of this aspect of the component, whatever it is,
it helps you to recognize when there's variability,
that's outside the expected thresholds that you might see.
It aids with this rapid sense-making.
So we've been talking a lot about nuclear power,
about aviation.
A lot of these examples are drawn from healthcare.
These are situations where seconds matter,
microseconds matter.
And so the fact that information is not readily available
to you at different points in time,
actually matters a whole lot.
And it's, you know, related to that,
that it's clear under conditions of time, pressure, stress,
and uncertainty.
Again, if you are responsible
for multiple concurrent activities at once,
you don't want that cognitive agent continually asking you,
like, are you looking at this?
Did you think about this?
Did you check this?
You need to think about the coordination
of those interactions as well,
so that you have smooth transitions
for when the machine takes over,
when the automation is expected to step in,
and when you are expected to sort of step out of that.
So this is kind of this idea of supervisory control.
So far we have been talking about this one-to-one
relationship, one person, one machine,
but how many of you actually work alone
or work in that fashion?
Probably not very many of you, if any,
because real-world work or work in the modern world
is about many people in many machines.
So this is from a paper that, you know,
if you've been kind of interested in resilience engineering
or cognitive systems engineering, you may have seen
joint activity and common ground
in joint activity and coordination.
And, you know, it's not to get all sort of like nerdy
and prescriptive here,
but they talk about this concept of joint activity
of this extended set of behaviors
that's carried out by an ensemble of people
that who are coordinating with one another.
And so, joint activity is this activity that produces
something that is a genuine joint effort.
It's not just you do this and I'll do this in parallel
and we meet together at the end.
There is an element of it that requires you to interact
over a span of time.
And so that span of time could be anything from minutes.
It could be seconds.
You have a brief interaction with a person
or with a machine, or it could be days or months or years.
So it kind of seems almost disingenuous to point out
that any kind of modern work is joint activity,
because this is the stuff of everyday work,
but what's critical here is that we're pointing to this idea
that coordination absolutely matters.
We can't just leave this up to individuals to kind of think
about this as an afterthought.
This has to be a central part of how we design
and deploy these into fields of practice.
So what's important under this context is as well
that all of this coordination actually has a cognitive cost.
So we were talking about how seconds matter,
how microseconds matter.
Well, poor coordination requires you to be thinking more.
It requires you to take extra steps and activities
to be able to ensure that there's sort of that smooth
transition across agents.
And so when we're talking about joint activity,
we are talking about coordination.
So another kind of element of what's really important
when we start thinking about these joint cognitive systems
is that different parties that are engaged,
have different goals and priorities.
So they're gonna have different interests
and want to see things happen in different ways.
And so a fundamental kind of takeaway is this is like,
as humans, I can adapt and adjust my performance relative
to John's goals or interests, I can say,
okay, he wants to step in and do the slide in this way.
I will take less time to do this slide,
or, yeah, it's really important to emphasize this point,
we will adjust on the fly to be able to really like,
you know, make this come across,
but machines really struggle with this.
They can't necessarily adjust and adapt their goals
and priorities as the context change.
So it kind of puts all of this additional effort
to coordinate with machines onto the humans,
to be able to adjust their goals and priorities.
So I'm gonna use a really simple example of coordination
and joint activity.
And that is basically traffic
because people in this situation, when you're driving,
you kind... Not yet no,
you know, how traffic works, right?
There's an expectation of like one person goes,
there's turn-taking, you need to signal a little bit
here and there.
You need to anticipate your participation
in this broader joint activity.
This coordination is happening in real time.
So in this example, people are slowing down,
they're speeding up relative to how they anticipate
the actions of those who are around them.
They're signaling, you know,
if they need to cut across traffic,
they're gonna adjust relative to the flow
and of the situation around them.
So I want you to think about what would it mean
to put a self-driving car into this flow?
(audience laughing)
What do you think would happen in that context?
- Anybody here ever found themselves responding
to an outage?
Some incident, right?
Sometimes Laura and myself spent a number of years
studying in great detail what goes on in matter of fact,
if you would really like to read something mind blowing,
then you'd read Laura's dissertation
on exactly that costs lots happen that we as participants
in that joint activity, do not explicitly acknowledge
that we do, right?
That's the thing about having expertise is that the more
expertise you have, the harder it is for you to describe
how you have that expertise.
Understanding and identifying cognitive work in the wild
is helping pull that apart.
Sometimes when we see a response to an outage,
it can be in, some organizations we've seen really fluid.
And I mean, so fluid, multiple five, six, eight people
all working almost entirely
as if this is what they were born to do.
And when you look at both slack transcripts
and Zoom calls, it sounds like this,
"Hey, yep, I saw it.
What do you think, no I'm gonna check have that,
ah, you know what I think, now I already,
I went, look, it's not, it's not that, ah do, you know what?
Oh, yes it is.
Oh, I think, yeah, it's that, should we?
No, don't no, not let's not this time.
We should probably do this other.
Right, Right.
(audience laughing)
And of course we're like as cognitive systems engineers,
we're like,
awesome, right?
The interviews practically write themselves,
this is what we might just distributed, sorry.
- No, that's okay.
- I improvised just now.
(audience laughing)
I can't tell you how I did it.
- Yeah. What you're talking about is that smooth handoff,
those smooth transitions, that flow.
And so when we juxtapose, you know,
an example of this or what you were just describing
with that earlier, you know,
example of the alert kind of dropping in out of nowhere,
it's a really stark contrast.
So we still have a lot of work to do, awesome.
- So for this last exercise, right,
we're gonna unfortunately, or fortunately,
we're gonna return to the same example,
by the way the alarm problem and all the paradoxes
with what academically described as directed attention
has been a topic in cognitive systems engineering
and human factors for that matter as a whole for decades,
way before internet.
So we're not special with this particular problem.
So is this a joint activity?
Anybody would like to ask or say yes or no?
Yes. Okay.
In what ways?
(indistinct chattering)
Yeah.
Oh yeah.
- [Man] Yeah. We've got a machine who's communicating
to a human and then the human then doing some cognitive work
to figure out next steps and go probably to another machine
to get more information.
- Yes, excellent.
Anybody else?
You nailed it by the way.
It's of course.
So,
what's that?
Yeah. You go ahead.
- Who is not represented in what you just said?
Is there anyone not represented in that description?
- [Man] How many people are on that escalation policy?
- [John] Ah, excellent.
Nice.
- Okay.
- [Man] I was thinking customers or stakeholders
or whoever might care about this system going down.
- Yeah.
- Absolutely.
Anyone else?
- [Woman] The machines that's being alerted on
in the first place?
- Ah, yeah.
Say more about that.
- [Woman] I mean, presumably it's an entire system of things
in the host web 121,
a machine that is being alerted on and presumably
there's probably also all other kinds of monitoring
that's taking place.
- Yeah. Yeah, yeah.
Excellent.
See, I told you they were kind of getting it.
- One more at the back there.
- [Man] Whoever designed the alerting system
and had these logs spit out for this specific reason,
the actual human who set it up.
- Nailing it and nailing it.
- I feel like our work here is done.
(audience laughing)
- All right. Well okay, great, let's go.
So yes, all of you are right.
And I think that that's important before we get,
we're gonna expand, I think a little bit on that,
that idea that what is, and what is a joint cognitive system
can have multiple layers to it.
I wanna give a quick story from for a place that I worked
with this may or may not fit with those of you
who have been on call them in the past.
So this organization that I work with in,
in Brooklyn, New York called Etsy,
there was a period of time when the on-call engineers
were like, struggled with faced a bit of a quandary.
So here's what would happen?
What would happen is,
there are some bits of the infrastructure or the systems
or whatever that for whatever reason might be more,
it's more attractive to do maintenance
on cluster or subsystem, whatever, at night, right?
Or at night in whatever the particular time zone is.
And like I said, it could be for a whole host of reasons.
Sometimes you'd be that,
maintenance would be done by somebody who wasn't on call
at the time somebody else is on call, right?
So, you know, Stephanie's on call
and Stephanie lives in New York,
and Stephanie has gone to bed and,
Raj is gonna work on this cluster, ABC,
and Raj, like, "Okay, I'm gonna do this starting
about nine o'clock.
And I think it's not that big of a deal.
So I think it would totally take an hour.
So what would happen is you, you know, Raj would go like,
"take these nodes out of service,
out of production or whatever.
And then silence the alerts because you know,
is that like, he's gonna turn stuff off and on,
and I'll doing a whole bunch of things to them.
And then like, once that's all done for whoever 50 bucks,
100 whatever nodes or whatever,
then like put everything back and then get on and then maybe
he'd go to bed or something like that.
But what he doesn't wanna do,
he doesn't want to wake up Stephanie,
Stephanie might have had a really crappy week
on call already.
Definitely don't wanna wake up
to find out that like an alert slipped through, woke her up,
but actually it's not that big of a deal, right?
Because the learning system doesn't know
that it was out of production
or something like that, anyway.
So what they did was, because it would happen
actually quite often, because you do, I'll say downtime.
So it was like a temporary silence the notification,
the checks would still run.
'Cause you don't wanna turn that off because you might
forget to turn them back on again.
So you set a timer.
It's like, okay, for the next hour,
don't actually send an alert, still do the checks,
but don't send an alert to somebody
and then like 45 minutes pass and 50 minutes pass
and Raj's like, "All right, cool.
This is gonna be done before the hour's up."
And then something's weird and like,
oh, okay, didn't think this sticky problem.
This is last couple of nodes.
Ooh, I didn't see this.
So he's focused on getting this issue, worked out,
meanwhile,
and sure enough page goes up because the hour flipped
and it's broken, broken, wakes up, wakes up Stephanie,
and Stephanie's like, "Raj, what the fuck?"
And you know, and so they want to do this.
So what they did was they built a little bit of a thing
that said, okay, you know,
we're a cron job or whatever, every minute or so, okay.
What's been given temporary downtime.
Okay. Cluster ABC.
Okay.
Is it less than 10 minutes before that downtime supposed
to expire?
If yes.
Right?.
The person who set the downtime in this case, Raj and,
buzz the shit out of his pagers,
like with an option inside the alert,
Hey, clippy style, it looks like you're about to have
an expiration code downside.
And you're about to be kind of an asshole to your teammates
press here to extend the downtime.
Sounds really simple, but it was an approach to solve
a particular problem.
Now, of course, this is extra code,
which has an opportunity for extra bugs.
That was just an example of a relatively straightforward
thing that gives the alerting system just a bit,
doesn't solve it.
Doesn't turn it into a human,
just a little bit more of support, okay?
So back to all of the ways that you could identify
all of the actors here, agents, right?
There's the alerting, there's system that sends the alert,
it might even be different than the system
that's actually making the checks that fuel the alerts.
There's the thing that being checked.
There's the receiver of it.
Somebody else also said something along the lines
of the on-call schedule.
One of the ways that we can think of this is that layers
of joint cognitive systems can actually broaden
in directions that you might otherwise think
are sort of outside the scope.
When we did a dry run, somebody had mentioned,
well company investors, I don't know.
An example that comes to mind
was let's say a security vulnerability,
a security vulnerability, to some extent of report CVE
is a notification.
I would argue that the investors,
perhaps even board of directors of Equifax,
all of a sudden are somehow interested in alerts.
But these opportunities to see joint cognitive systems
in different areas and what cognitive work means in support
of each other can broaden and vary on different scales.
- Sounds good.
I wanna actually just put a little bookmark here
to riff on something,
'cause I think we're gonna finish a little bit early.
You brought up a lot of like disparate sort of stakeholders
when you were talking about the joint activity.
So I wanna talk after we finish about what that means
when we are learning from incidents and like,
what does that mean when we distribute the sort of learnings
about that?
How can that enhance performance across a system?
So I keep coming back to this point
about like what, you know, speed, scale complexity,
as all of those increase the importance of designing
these things more carefully becomes a more critical.
And so we're coming back to these because as researchers,
as you know, members of the community,
we want to sort of turn this around to you as well
is like, these are practices, these are concepts,
these are theories that can be applied in everyday work
so that we learn and understand more about human
and machine interactions, more about the real world of work
and how these things actually play out
when there's multiple competing goals and priorities.
When there's time pressure,
when you're under production pressure,
because those same kinds of characteristics
of your work world are inherent in a lot of very high hazard
environments as well.
So we need the designers and the developers of these systems
that are being built today to be considering these demands
of running these systems at speed and scale
for purposes that they may not necessarily
have been designed for,
so that we're able to enhance the resilience
and the safety within these systems.
So we're kind of coming to the end of our time
together here.
And we've talked about a number of,
kind of characteristics about
what makes machines good team players,
about how to make your tools
and think about your interactions as part of this larger
joint cognitive systems.
So we're kind of gonna leave you not necessarily
with specific takeaways,
but rather with questions that you can use to inform
and expand some of your existing practices and processes.
So when we talk about making your tools
part of a joint cognitive systems,
it's important to consider the cognitive work
that you are supporting.
What functions are you expecting the automation
to take care of?
Oftentimes it's like, oh, that's toil.
Let's just make a machine do that.
But are there aspects of conducting that work
that help you make sense of and sort of keep the finger
on the pulse of what's actually happening
within that system?
So if you automated a way,
are you actually changing the nature of the cognitive work
needed to make sense of that situation?
So what functions do they support?
Do you consider this across the joint cognitive system?
Considering the joint activity.
So you all were like, kind of blew my mind actually,
how quickly you went into very disparate places
with what is who's involved, who's related to this,
maybe not directly, but also indirectly.
Who's impacted by some of these decisions and actions.
And that is actually really fundamental because the wider
your sort of conception of how your system works,
where pressures and constraints are coming from,
what kinds of influences do different parts
of the system have, the better you're able to anticipate
certain kinds of problems and prematurely adjust
your actions or adjust the way that you're conducting
your work to be able to cope with any kinds
of reverberations that might come out of a different part
of the system.
So who's involved, how do they interact?
What kinds of impacts do we have on each other?
And how will these handoffs between machines
and humans safely take place?
And lastly, what assumptions are you making about the nature
of this work?
So we've kind of implicitly been talking about this
the whole time is, who's doing the work?
What other kinds of work are they doing?
How much time and attention do they have to allocate
to different parts of the task that they're faced with?
So we have some time here for question and answer,
but we wanted to kind of turn this around to the group
and hear a little bit, you know,
at the outset, we called this an experiment.
And as we mentioned,
this is like a big part of other domains
that some of the examples that we've pointed to,
we can't yet point to a lot of really good concrete examples
in software.
And so given that we, as an industry, as software,
hasn't had our Three Mile Island to kind of shift
the thinking within the industry.
I'm curious about what it is that you think
you might take away from some of the things
that you heard today in order to change how you think
about your work.
It's a big question for the end of the workshop.
- And I guess if there are questions that people have still.
- We'll take those.
We got one at the back there,
and there's Ty there's one up here too.
- [Man] Thank you.
Right now, I'm working with a lot of engineer engineers,
like material engineers, mechanical engineers,
and my team is responsible for digitizing and scaling up
a lot of what they're already doing manually cognitively.
And there's a real dichotomy.
This cannot be automated.
What we do is more of an art form than it is a science,
but you need a lot of science to do it.
And a machine will never do this.
And on the other side, there's, you know,
I can write that in a week and eliminate that need
for this person forever.
And what this really shows is to just completely change
that assumption about this discussion
and try to tap into both of them.
And I think it'll make the conversations
that we have at start-up a lot more productive.
So thank you.
- Oh, excellent.
That's great.
So satisfing.
- That's fantastic.
And it is also like, I think even that subtle shift, right?
Because that dichotomy exists right now
in a lot of these conversations is like,
no, we can trust automation.
We don't need humans for this.
And so even that subtle shift of like,
how might we be sharing this load?
How might we be coordinating and handing off work across
these agents is fundamental to so many conversations
right now.
- [Man] Hey, just had a question.
So in the alert example,
what if you're a company that's getting this alert
from a third party company and you can't change the content,
what are your options to kind of reduce the cognitive work
in that case?
- Well,
I'm gonna turn the question back to you in a second,
before I do that,
certainly to aid the cognitive work,
I'm not entirely sure we can reduce it, rather change it.
The receiver of an alert will always be engaged
in cognitive work, we tend to think of less cognitive work
for easy things, but it's actually not less,
it's just different.
Back to the suggestion that, or the scenario that you put,
it's a really good question I maybe would ask,
I would wanna explore what the current situation,
how do people currently handle that?
Getting a work?
Like if you're getting an alert, then somebody getting it,
what do they do when they get it?
They probably have work arounds.
In fact, a significant amount of cognitive systems
engineering work is discovering what work
arounds people have.
Do you wanna say anything?
- Yes, I do.
It's a great question, actually.
So my dissertation work,
which John mentioned was looking at coordination across
sort of large scale distributed systems
and what came out,
like I was studying basically incident response teams.
And what came out right away was that it's not just about
the incident response team.
It's about the vendors.
It's about other teams whose services are dependent on that.
So it became very difficult to draw the boundaries around
who owns what, because they were so tightly interdependent.
And so what you're describing to me is like a failure
of one of the coordinate of agents to really understand
what it is that you need, from your perspective.
And so, I agree with what you're saying
in the sense of like, being able to sort of describe
some of this.
It's hard to go to a vendor and be like,
you should change this.
But I think you can make a compelling case
to be able to say, when we receive your alerts,
you know, here's the information that we don't have.
Here's what we have, like exactly the same process
that we went through with these examples is like,
here's what we lacked.
Here's the questions that we had.
Here's the steps that we took to kind of, you know,
get more context for ourselves
and here's who was important at what points in time.
And so I think like, it's putting some of that work
on to you to be able to describe that to the vendor,
but you could also go back to them and sort of say like,
"Hey, I think you need to look more closely
at how these things are landing in the real context of work
as software vendor or a member of a small startup right now
that would be gold,
like that kind of interaction and collaboration
shifts those boundaries about how do we think about
who's responsible for the coordination?
And it does become much more of a joint activity
when you're sort of thinking about reciprocity,
you're thinking about the work across the system.
Does that make sense?
- [Man] Sure, thank you.
- Yeah.
- [John] Anybody else?
- Betsy, up front here.
- [Woman] So hypothetically,
let's say that you're very bought in,
on all of the ideas presented in this talk,
but that you're embedded in an environment
that is perhaps a bit less so,
what harm reduction techniques are available to you
personally in that kind of situation?
- Say more about that.
- [Woman] Yeah. Sorry.
I genuinely am speaking on behalf of a hypothetical person
who I assume is in this room who, oh,
let's get less hypothetical and talk about situation
that I'm not in anymore,
but that I've definitely faced in my career.
What if you've got alerts blaring all the time
and someone who believes that the problem is the competence
of the engineers, rather than lack of slack on the system,
for example, like what about other situations
of that nature?
How do you bottom up it?
- Yeah.
- I think it's like, it is fundamental to the work
that we are doing.
I think both like in a day-to-day tactical,
like sense in our work,
but also as part of a broader paradigm shift
within software.
So I've seen a couple of like instantiations
of this problem, which is like,
maybe not necessarily that the person is directly
sort of in the line of fire of like,
if the machine can never be wrong,
it's gotta be something with you.
I think that, you know,
the same way I answered that question there,
is like being able to actually let me pause on that.
Well, I was about to say like being able to sort of describe
the complexities or describe some of the...
And Betsy is doing a talk about storytelling
and narrative later,
which I'm very excited about because some of these things,
when we talk about them in very theoretical abstract terms,
they, you know, it becomes an argument
about whether machines are better
or whether humans are better and you lose all of the details
about what makes this work hard, right?
And so I pause there for a moment because I don't think
it's inherently the engineer's responsibility to say,
this is a complex world, there's lots of goal conflicts.
Like, you know, you,
I'm assuming this is someone in a position of power
or influence or authority, you know,
have put certain kinds of pressures and constraints
into the system that make it hard for me
to take any other action than the one I took.
I don't think that's necessarily the role
of the individual engineer,
but I do think that there are sort of like gorilla ways
to surface some of these things in everyday work.
One of the things that the sort of bookmark thing
that I wanted to talk about was like incidents
as an example, as an opportunity to surface normal work,
to surface that this stuff it's not the exception,
it's the rule that you are always facing
these kinds of conflicts in your day-to-day work.
And so I think that's one example of an opportunity to like,
not go down the root cause analysis hole and say like,
oh yeah, it was just this.
But rather to say, whoa,
let's step back and look at all of the, like the,
both the human and the technical,
like factors that were involved in this incident,
because it helps you understand how your system actually
works and how it actually performs.
So while I said,
I wasn't gonna leave it on the individual engineer,
I would say both like, you know,
in that example of you being able to sort of see
the water that you're swimming in and pointed out
and say like this is extra effort,
the coordination, the cognition,
this is extra effort in these ways can be useful
and as well, using structures
within your sort of organization,
to be able to kind of put some more of these things
on the table so that it's not just a heavy lift
for one person.
Did you have anything you wanted to add to that?
- No.
Other than try not to think
of there's you either take these this perspective
or you don't, right.
The fact is whether you all know it,
which is why we wanted to have lots of real world examples
is that the work that you do
is in spite of all the challenges, right?
One of my teachers, one of my professors once said,
two questions that tend to be pretty productive
when you're approaching, understanding how this messy stuff
all works.
And I mean, really, you know, all up and down,
all different scales, complexity, blah, blah, blah,
what makes this job difficult?
What makes them good at it?
There's an implicit assumption that they are,
the fact of the matter is they are,
there's always an opportunity for people to be worse
at what they're doing, (audience laughing)
in which case become curious about what makes it
such that they are not nearly as bad at this
as they could be.
If you can identify that when you're a deluge of alert,
alert, overload, and alert, fatigue, like I said,
in software, we'd love to think of ourselves,
as we're so special and everything good in the world
came from software here, blah, blah, blah, right?
No, that's not true whatsoever.
False alarms happen literally in every important domain,
otherwise they wouldn't have alerts.
It's an unsolvable problem.
Unsolvable, there will always be false positives,
especially for successful organizations.
So to sort of scope in that one way would be to scope.
What are all the situations makes it
such that it's, we can't make progress on getting this huge,
noisy alerting that shit handle.
Another way angle,
only different, not better or worse
is understanding that you can have a system
and you can design alerts and design systems that produce
alerts that are, you know, avalanches
and that come from another source,
other than we're not given time enough to work on it, right.
Sometimes that problem is so hard.
It actually is really hard.
Even if the board of directors say here's a blank check,
go and fix the alerts.
It still could take a lot of work.
It's not just, I want the support and the buy-in on it.
So looking at it from different angles is sometimes
really difficult, but starting slow,
starting tiny little bits, I don't know.
That's my probably not helpful answer.
- Well, I think one of the things we were talking about
is we were putting this slide deck together too,
is like the importance of sometimes just stating the obvious
as well, right?
Like, and we see this a lot when we're working
with organizations, you know,
it's sort of like, particularly around incidents.
It's like, well, what did you think was happening?
And people are kind of reluctant to actually say
what they thought was happening because were like,
it seems really evident to everyone in the room.
And then someone will be like,
well, I actually didn't know that it worked that way.
Or like, you know, and then all of a sudden it opens up,
this whole discussion about like, wait, you thought that,
I thought that.
And then so there's like this broader, you know,
discussion that comes out of just sort of stating something
that seemed quite obvious.
I wanna keep the conversation going,
but I see people are leaving.
So I wanted to put this slide up there
just because these are a couple of resources.
So, you know, take pictures of whatever.
- The first one is a trick.
It's a URL to a ridiculously huge number of references.
- It is.
It's probably like one of the most complete repos of this.
But the common ground paper that I mentioned
is in there as well.
And it's definitely to cold within industry-
- It's a must read.
- Other questions or, comments just,
oh yeah, sorry.
We got one here and went up front Daniella.
- [Woman] Yeah, I was wondering if you could speak more
to the difference between the joint cognitive system model,
where we were first working with like one machine
in one human, and then at the beginning of the talk,
you mentioned multiple humans, multiple machines,
it's a huge difference.
And could you kind of just like reiterate,
like what the difference is because we're all working
in that kind of environment.
- Sorry, can you, the last part.
- That's the environment we're all working in.
- Yeah. - Yeah.
So when we were talking about that kind of joint activity
and thinking about sort of cross scale coordination
and thinking about who and what is impacted
or who and what kind of needs information,
or can you get information from,
it becomes less about kind of, I guess,
handing off information from one party to the next
and more about sort of like we have this collection
of skills and capabilities within an organization,
within a team, whatever it is,
how do we dynamically reconfigure those skills
and capabilities to be able to meet the demands
of the problems that we face.
And I'll make a little distinction here because, you know,
people are like, well, joint activity,
isn't that just kind of like teamwork.
And I think that there's a very fundamental differences
that when we talk about teamwork,
we're talking about groups of people that know each other
that have trained together, that like maybe have history on,
you know, working on certain projects together.
So they have what we call common ground.
They have a basis to understand,
I know what John knows about this world.
I know what he believes about some of these theories.
I have an understanding about like his orientation
to different problems.
And he has that same level of knowledge
about my capabilities and my thoughts and my feelings
and that when it comes to coordinate of activity
is absolutely fundamental for these kinds of smooth kinds
of transitions.
When we, just one second.
When we are talking about joint activity,
it's often these ad hoc groupings,
because you don't really know who you're gonna need
at what points in time and what sorts of like skill sets
you might need to sort of bring to bear on those problems.
And this is not just necessarily an incident response.
This could be responding to, you know,
a competitive threat in the market.
It could be responding to an unanticipated, you know,
acquisition, how do you start working together
with other, you know, new teammates?
And so, when we're thinking about sort of designing
these systems,
we're thinking about how do we provide enough of a basis
to be able to have that common ground,
or have that sort of knowledge,
or to be able to really readily bridge that gap
when you realize I don't actually know enough
about what you know about the situation
for us to work effectively together.
And so part of that, like we could give a whole talk
about that, and you should next time,
part of that is sort of like how we design the system.
Part of that comes back to what we were just talking about
of stating the obvious, right?
One of my favorite things to do is be like,
what kind of assumptions are we actually making here?
You know?
Because what the mental model that I have about
what we're doing, how we're doing it,
and how important it is, is going to be very different
from your mental model.
And if we have good common ground, good basis, you know,
we're kind of in a well-structured joint activity,
that'll be like a smaller gap, but as we move along
and as we approach novel problems
that we weren't sure we were gonna face,
that gap is gonna extend.
And so it's this kind of continuous reconfiguration.
Yeah.
Okay go on about that.
- The only thing I was gonna add to what Laura is saying
on your the shift between, you know,
one person, one machine, to many people, many machine,
it occurred to me, you know,
the Quip about regular expressions.
I will adjust it a little bit.
Some people when looking to solve a problem
with responding to incidents may think,
oh, we'll add more people.
Now they have two problems.
And the coordination, the thing that is absolutely
mind blowing about Dr. McGuire's dissertation
is stuff that is almost entirely invisible to us,
especially teams that do this really well.
But if we're working in trying to understand
what's happening and an incident commander,
or somebody says, oh, can you go check out the database now?
And then Laura goes off to check out the database,
and then I'm doing other things.
All of a sudden, I now have a thing.
And she has a thing that is in addition to figuring out
what the problem is.
I have to remember that Laura went off to go check on
something that is on my stack, right?
If I don't hear back from her,
that might mean a bunch of different things,
especially if I don't hear back from her
in some expected period of time,
she has to remember whatever she finds,
let's say she goes, and actually everything looks normal.
She has to remember to report back on that she found
at normal, she might, depending on how many other people
are responding, might have to explain what she did
to work out what was normal.
So all of these are paradoxical dilemmas, and trade-offs
that increase or are not just increase,
become more complex as more in different layers, right.
And you can expand on all those sorts of things, but.
- Sometimes it almost seems a little bit silly
that example of like, oh, she has to remember to come back
and tell us.
But again, it's like, we're sitting in a nice quiet room
and a very theoretical, you know, like situation.
We're not in the middle of a high pressure.
Like the system is down, you know,
and people are freaking out,
you're getting emails from the VP.
You know, someone is like at an airport trying to log on
a wifi, you know?
So it's like, there's all these like multiple demands
on your attention.
And so again, this stuff matters in real world work.
The types of like friction that can be inherent
in that coordination matters as the time pressure goes up,
as the complexity goes up,
and as the consequences for getting it wrong, go up.
And I think the Boeing example is an excellent,
one of that, you know, you have a machine agent
who is flying the plane in a way that doesn't make sense
for that moment in flight, right.
It's like, why would you push the nose down
when you're on takeoff?
And so then you are starting to like,
what are the signals that are available to the pilots
in that moment?
How much time do they actually have to gather
new information about what's going on and what appropriate
action might look like in that context?
So, yeah, I kind of keep coming back to this idea,
like the more we understand about this in lower consequence
kind of situations,
the more that we can actually start to extrapolate and say,
oh, these are general patterns about how people work
with machines in these kinds of worlds.
Yeah.
There was, I think another one back there.
Yeah.
- [Man] I'm curious.
I feel like all of the examples of like joint actors
were kind of on the same side,
and I'm wondering if that extends to like adversarial actors
and joint activity, if that's something that's considered
or what that relationship means or how you think about?
- Nice, excellent.
How much time do we have?
(audience laughing)
I think you might be right.
So might be a couple of ways.
So they are engaged in a joint activity.
They may or may not in as much as they're both aware
that they're engaged in joint activity,
the common ground coordination and joint activity
and paper does actually does a really good example of that.
People who are opposed, people who are arguing
or engage in joint activity in as much as they both sign up
to what were the others called basic compact,
which is they are both committed
to continuing the argument, right?
Even if their purposes aren't aligned.
So the other thing is that there's a couple of ways
of spinning this, but one way to think about it is,
an adversary doesn't come after people,
if they don't think they're gonna get any value from it.
So to the extent that the target
signals that there is something valuable to go attack
and go get inadvertent or purposefully,
you could say that this is signaling
and that signaling is being hijacked for, you know,
a self-centered purpose.
That's one way of saying it, you know,
both on the attacker side,
you could see that attack if there's very rarely,
is there a single attacker,
but sometimes there's a group of actors that are a part
of they absolutely, as far as they're concerned are hoping
that they have fluid, successful joint activity,
as well as the defenders team at the risk of assigning
yet more reading.
A couple of years before Laura finished her PhD
at the same lab at Ohio State University,
many Matthew Bernard wrote his dissertation
on exactly the same topic on adversarial, you know,
activities.
And in that case was, you know,
in your straightforward sort of open-ended red team,
blue team scenarios.
So yeah, that's the short answer.
- Yeah, I think it's a fascinating problem.
And I would say, you know,
we had an example of they're sort of like military
operations and, you know,
war has fundamentally changed, right?
Like it's not bad guys over here with guns facing this way.
And good guys over here with guns facing this way,
it is more distributed.
It's integrated into safe spaces,
you know, like, so it has changed the way, you know,
those kinds of teams coordinate because it's no longer
appropriate to have sort of a really centralized command
and control, here's the plan go out and enact the plan.
It is much more, here's what we think we're going to see
you when we go into these spaces,
but it becomes about being better at gathering,
like sensing the world for cues that are sort of like,
this is not what we expected to see.
And then being able to, again,
dynamically reconfigure that collection of people
and the broader sort of platoon or network of people
to be able to like face whatever kinds of threats
and risks come up.
And so I think like, in some ways,
like even if it's not an adversarial, like attack,
sometimes when systems go down
and they're like cascading failures,
there's very similar signatures to it as well
in that it's unexpected, it's emergent, you know,
like you're on the balls of your feet all the time ready.
You have to respond and adapt and just look for signals
in the world about what's going on.
Yeah. So it's a good question.
Do we have one up here?
- [Man] First off thank you.
It's been wonderful.
I have more of a comment than a question.
As a junior developer,
I found it really poignant when you were expressing
as somebody gains expertise in a field,
they begin to not understand how they know things,
and coming into the field,
I think it's something that we're looking at,
like humans and computers interacting,
but as senior developers get more senior,
the more computer they become, to assign.
(audience laughing)
So when you're interacting with or onboarding
or writing documentation, yeah.
Interacting with a senior is sort of like interacting
with a very complicated like system
and how to unload that cognitive dissonance
or that difference between that like generational gap too.
- Yeah. - Yep.
- Yeah. Oh, excellent.
A couple of things come to mind and studies of expertise
loom large on this front.
So I think you're absolutely right.
And also what an amazing time it is for you
in that you are given way greater latitude,
at least socially notionally,
not knowing what sort of jerks you work with,
it to ask dumb questions.
And the opportunity is usually just as,
as a sort of almost certainly a gross overgeneralization
more often than not,
engineers, especially,
you know, highly tenured engineers,
very experienced engineers,
very rarely are people asking them to tell stories
or even an opportunity to explain
how they know what they know,
even if they're incomplete in making that.
And the reason for that is pretty simple.
They're a senior engineer,
which means that the organization wants to get
as much senior out of them, so they tend to be very busy.
So finding opportunities and capitalizing
on those opportunities to tell stories
or give descriptions about how this thing works
or how this thing is tricky or how this thing used to be
this other thing.
And here's the rationale for why it is the way
it is right now, all those sorts of things.
Gary Klein, Robert Hoffman wrote an excellent paper
called "Seeing the Invisible."
And they described this as vicarious learning,
incidents are an opportunity to provide
people with expertise,
to tell stories that without the incident
they wouldn't have had an opportunity to do so.
It doesn't really get too much resistance because incidents,
you have to do something,
incidents spark,
nobody says, oh, we just had a big incident.
And rarely would somebody say,
oh, now we're not learning from that and get on.
Let's get on with it.
I'm sure it's fine.
Yeah. There's not spending any time.
We're not gonna do anything in the wake of that incident.
You can harness that and give them,
they may say things that a huge number of people
in the organization didn't know your opportunity
as somebody who is either new to the field
or even new to the company,
nobody says, oh, get a load of this.
Look at this one, asking stupid questions, nobody faults,
because what the hell do you know?
And like, entertaining those,
it's the middle layers that don't know what they don't know
that unfortunately provides a barrier.
I'm happy to talk more about it with you,
but it's a very unique situation
and you won't always have this advantage
because you're gonna be them.
(Laura laughing)
(indistinct chattering)
- Yes, yes.
- Yeah, yeah, yeah.
What was your name?
- [Bronin] Bronin.
- Bronin, yeah, thank you for the question.
'Cause this is the complex sociotechnical systems tract,
right?
And so we've been talking about people in systems,
but I think, you know,
what John pointed to in terms of like creating opportunities
where some of this context actually gets shared
as a way of like filling gaps in our mental model
is really important.
I don't also know that there is psychological safety
in every organization for you to be able
to ask those questions.
So, you know,
we have been a lot of the work we do with companies.
It is like turning it to those senior engineers
and being like state the obvious, you know,
like, so you sort of distribute that ask, right?
The thing I'll say about mental models, that is again,
could be a whole other talk
that I think is really fascinating about your world
is that everybody has a completely like incomplete model
of what's actually going on in your system
and how it actually works.
It's partial and incomplete in some ways.
And within your team, each of those ways
is slightly different.
And that's largely what keeps your system running
is because there's sort of like this overlapping
like knowledge base.
So you kind of draw from each other, what do I know?
What do you know?
But oftentimes when you have these kinds of learning reviews
or these opportunities to talk about how the system works,
you get the senior people getting really like,
what really?
That like, man, I've been working on this
for six or seven years.
And I always thought it worked like this.
And those are really rich opportunities
that if as an organization,
your understanding is engineers
should have all the knowledge and it should be complete.
And they're like faulty in some way,
if they don't know, things is just garbage.
It doesn't work in this world.
You are continuously changing.
It's continuously dynamic.
So the expectation should be that you don't know things.
And you're continually trying to patch those holes.
And as an organization, as a team,
you're continually creating these opportunities
to patch those holes, not, oh, this guy doesn't know
what he's doing.
So we are at time, we're totally happy to hang out,
but we should vacate the room.
I think, otherwise, thank you so much.
You guys have been a great audience.
(audience clapping)
- Thank you.