PAPod 535 - Reimagining Safety: Lessons from the Resilience Master

Show Transcript

WEBVTT

00:00:00.017 --> 00:00:03.137
Fighting zombies and the zombies don't die they

00:00:03.137 --> 00:00:06.377
come back over and over again yeah and

00:00:06.377 --> 00:00:12.237
the zombies are out in force now and none of these things for example all the

00:00:12.237 --> 00:00:20.257
insane statements relative to the washington crash none of them are in a sense

00:00:20.257 --> 00:00:25.437
they're all old they're all things we've seen and fought and dealt, had to deal with before,

00:00:25.697 --> 00:00:29.997
but there's kind of a new intensity, a new,

00:00:30.657 --> 00:00:34.277
blatantness, a new, a new kind of misdirect.

00:00:34.677 --> 00:00:38.077
But again, it's as always, safety is controversial.

00:00:38.337 --> 00:00:46.537
Safety is never not controversial, even though we all want to pretend it is. Oh, my God.

00:00:44.240 --> 00:00:55.920
Music.

00:00:55.057 --> 00:00:57.657
Hey, everybody, and welcome to the Pre-Accident Investigation Podcast.

00:00:58.017 --> 00:01:03.837
I am your host, Todd Conklin, and it is so good to be with you.

00:01:04.197 --> 00:01:07.537
What a week. Oh, my goodness. What a couple of weeks.

00:01:08.137 --> 00:01:14.177
You know the James Reason case because this podcast is the origin of the James

00:01:14.177 --> 00:01:17.717
Reason podcast, so you know how that happened.

00:01:18.177 --> 00:01:22.597
The conversation didn't stop when we talked about Reason. In fact, it went on.

00:01:22.837 --> 00:01:27.937
And the great thing about this conversation is if you don't take a note in this

00:01:27.937 --> 00:01:32.017
conversation, I'm not sure I guess what would make you take a note.

00:01:32.477 --> 00:01:37.117
Because the podcast you're about to jump into is a sweet podcast.

00:01:39.057 --> 00:01:42.177
Oh, it has got lots and lots of content.

00:01:42.837 --> 00:01:47.337
And it's an interesting conversation, and I'm so excited you're here to be a

00:01:47.337 --> 00:01:50.397
part of it. I think you're going to enjoy it immensely.

00:01:50.817 --> 00:01:57.597
We're going to talk to David Woods. Now, David Woods is really the origin of

00:01:57.597 --> 00:02:02.937
something that's in The Ohio State University called the Cognitive Systems Engineering Laboratory.

00:02:03.377 --> 00:02:08.017
And we've talked before about this, and it's something to think about.

00:02:08.137 --> 00:02:12.857
Now, David put a series of videos together, and it's the easiest way for me

00:02:12.857 --> 00:02:19.677
to tell you how to find the videos, is just go to YouTube and look up David Wood's Resilience.

00:02:19.997 --> 00:02:24.257
And there's a series of 12 videos, and I think even more are coming.

00:02:24.697 --> 00:02:29.237
They're all pretty short, six, eight minutes, maybe nine minutes tops.

00:02:30.117 --> 00:02:33.557
Completely worth watching. You will love them.

00:02:33.717 --> 00:02:37.977
And I would start in order, but I would definitely watch it because you're going

00:02:37.977 --> 00:02:44.457
to hear kind of where the contemporary academic thinking is now from the master

00:02:44.457 --> 00:02:46.137
of contemporary academic thinking.

00:02:46.297 --> 00:02:52.997
There's just no question in my mind that they get no better or smarter or wiser

00:02:52.997 --> 00:02:59.677
or more just profoundly futuristic than David Woods.

00:03:00.390 --> 00:03:06.290
He's, Professor Woods is remarkable. And he's remarkable in the way he sees the world.

00:03:06.450 --> 00:03:10.310
And he's just consistently seen these things for a long time.

00:03:10.430 --> 00:03:15.870
And now he's at a point in his academic career where, and this is the best time

00:03:15.870 --> 00:03:20.110
ever, where now things are starting to kind of come to a sharp point.

00:03:20.570 --> 00:03:24.510
And he's got some pretty strong ideas. And he's going to talk a bunch,

00:03:24.650 --> 00:03:28.130
well, in the entire podcast today is about the fundamentals.

00:03:28.630 --> 00:03:31.890
Now, you know I'm a big believer in first principles. We've talked about it

00:03:31.890 --> 00:03:35.450
a ton of times on the podcast. If you don't have fundamentals,

00:03:35.450 --> 00:03:40.190
you can't have the next step, and the next step is applied use of these ideas.

00:03:40.390 --> 00:03:45.210
They're based upon these fundamentals. They support the theory and support the

00:03:45.210 --> 00:03:47.850
ideas, and much of the work we do, you and I do,

00:03:47.990 --> 00:03:55.430
is based upon these very fundamentals, and that's what those videos are brilliant in describing.

00:03:56.210 --> 00:04:00.990
Please, if you get a chance, and I urge you to make a chance,

00:04:01.190 --> 00:04:03.390
watch them because they're really worthwhile.

00:04:03.690 --> 00:04:08.530
Let's jump into this podcast because the great thing about when you talk to

00:04:08.530 --> 00:04:14.230
David Woods is you just need to get him started and then kind of sit back and

00:04:14.230 --> 00:04:16.370
take notes, which is exactly what I do.

00:04:16.510 --> 00:04:19.050
You'll hear that as we progress through, But I think you're going to find this

00:04:19.050 --> 00:04:20.650
podcast super interesting.

00:04:20.890 --> 00:04:26.010
So without any more ado, remember David Woods, Cognitive Systems Engineering Laboratory.

00:04:26.510 --> 00:04:32.170
And just go to YouTube and look up David Woods Resilience and you'll see these

00:04:32.170 --> 00:04:33.350
videos. Here's the conversation.

00:04:33.530 --> 00:04:36.730
It's just the three of us, you, me, and Dave Woods.

00:04:40.730 --> 00:04:47.210
So, Todd, why don't we talk about compromise? All right. Let's compromise and talk about compromise.

00:04:48.070 --> 00:04:54.130
Well, what does compromise mean in safety? What does compromise mean in systems engineering?

00:04:55.130 --> 00:04:58.870
So I want to tell a couple stories.

00:04:59.510 --> 00:05:05.710
One story is really simple. What do I tell people as they finish up their PhD

00:05:05.710 --> 00:05:11.850
or graduate work and go off into the world to fight battles over safety and

00:05:11.850 --> 00:05:15.790
design and the future and automation?

00:05:15.790 --> 00:05:19.790
And I go to them Hey look if you,

00:05:20.645 --> 00:05:24.665
If you never compromise, you're not doing your job.

00:05:25.125 --> 00:05:29.305
If you always compromise, you are not doing your job.

00:05:31.685 --> 00:05:39.545
The need to compromise is continuous navigation in order to push people further

00:05:39.545 --> 00:05:40.905
than their comfort zone.

00:05:41.305 --> 00:05:46.925
Because they want to retreat and retrench in what's familiar and comfortable when it comes to safety.

00:05:46.925 --> 00:05:51.565
And we see this in the reaction to failures I wrote about with my colleagues

00:05:51.565 --> 00:05:54.305
in the original Behind Human Error in 1994,

00:05:54.305 --> 00:05:58.425
which all emerged as

00:05:58.425 --> 00:06:03.285
soon as I hit the ground in the real world from grad school and walked into

00:06:03.285 --> 00:06:08.965
nuclear control rooms in the aftermath of accidents and started looking at accidents

00:06:08.965 --> 00:06:15.005
and redesigning control rooms to help operators handle accidents in the making.

00:06:15.005 --> 00:06:18.465
The pressures to compromise

00:06:18.465 --> 00:06:21.705
are we're there i mean i tell

00:06:21.705 --> 00:06:25.385
the story of walking in as you

00:06:25.385 --> 00:06:28.125
know i was still 27 years old i hadn't

00:06:28.125 --> 00:06:31.445
even made it to 28 and i am

00:06:31.445 --> 00:06:39.965
supposed to run a study looking at new post three mile island accident upgrades

00:06:39.965 --> 00:06:45.005
to the control room and we had been designing an innovative one based on the

00:06:45.005 --> 00:06:50.985
octagon pattern display to show you a dynamic picture of where are you moving?

00:06:51.085 --> 00:06:56.385
Are you moving away from the normal configuration for the operating context?

00:06:56.645 --> 00:06:59.525
How are you moving away? You got diagnostic cues.

00:06:59.665 --> 00:07:03.725
You had a big picture overview, an overview of health safety,

00:07:03.825 --> 00:07:06.925
what was deteriorating, what was recovering, what was stabilizing.

00:07:07.245 --> 00:07:10.965
And there were other approaches which were compromising.

00:07:11.105 --> 00:07:15.305
Instead of taking this as an opportunity for innovation. They were taking it

00:07:15.305 --> 00:07:20.305
as an opportunity, how to change it as little as possible. It was really those operators.

00:07:20.685 --> 00:07:23.425
It was really that utility, right?

00:07:23.605 --> 00:07:30.085
Second-rate organization, second-rate people, right? All of these things we've heard all along.

00:07:30.765 --> 00:07:37.545
And so the sponsor says, and I'm sitting in the back of the room and the sponsor

00:07:37.545 --> 00:07:39.685
goes, I know I'm the one who's going to execute, right?

00:07:39.885 --> 00:07:46.125
And the sponsor goes, oh, it's simple. We're going to compare different difficult situations.

00:07:46.525 --> 00:07:48.805
We'll look in this high-fidelity simulator.

00:07:49.385 --> 00:07:55.705
We can track errors and response time, and then we can do a statistical analysis,

00:07:55.725 --> 00:07:58.645
and we'll know if any of these new backfits really help or not.

00:07:59.301 --> 00:08:02.221
And I'm going, wait a minute. In the back of the room, I'm thinking,

00:08:02.361 --> 00:08:05.661
there's no way this will work. They're solving dynamic problems.

00:08:06.961 --> 00:08:10.441
There's thousands of variables. There's variations. By the way,

00:08:10.561 --> 00:08:14.801
when you run one of these things, depending on an action that could be correct,

00:08:14.961 --> 00:08:17.961
but exactly when it's taken, it's continuous change.

00:08:18.141 --> 00:08:21.061
Something can get worse. Something else may not be as hard.

00:08:21.281 --> 00:08:26.201
They may innovate and recognize something and act early, diffusing the situation

00:08:26.201 --> 00:08:31.641
before things get too far. In other cases, they may just be going slow and things

00:08:31.641 --> 00:08:34.901
continue to get bigger and bigger and more difficult to handle.

00:08:35.681 --> 00:08:40.041
They can misstep. They can missee. The information can be misleading.

00:08:40.301 --> 00:08:44.101
We designed misleading information cues into the scenarios.

00:08:44.641 --> 00:08:47.601
And I'm sitting there going, there's no way this is going to work.

00:08:47.981 --> 00:08:53.901
And I spoke up. I started to say, you know, it violates every assumption of

00:08:53.901 --> 00:08:58.201
these kinds of statistical analyses. And what was the comment back?

00:08:58.321 --> 00:09:02.861
Oh, no, no, no, no, no, no, no, no, no, no, no. Just go collect these numbers

00:09:02.861 --> 00:09:06.741
and run the analysis of variance, and that's it.

00:09:07.141 --> 00:09:12.081
And I'm going back on, this is a fiasco. I have to do underground research.

00:09:12.521 --> 00:09:17.081
I had to compromise and not compromise, because I knew down the road they'd

00:09:17.081 --> 00:09:21.041
be unhappy when I came back with null results showing absolutely nothing.

00:09:21.661 --> 00:09:25.581
It would have been boring and expensive with no information value.

00:09:25.701 --> 00:09:27.921
I had to create a different kind of study.

00:09:28.101 --> 00:09:33.581
I had to go back and invent process tracing for the modern systems and world

00:09:33.581 --> 00:09:38.981
from the history of protocol analysis in the early 20th century and problem solving.

00:09:39.221 --> 00:09:45.861
That was like chess, not how do you recognize unexpected behavior in a dynamic,

00:09:46.321 --> 00:09:49.001
complex, highly interdependent system.

00:09:50.401 --> 00:09:55.961
It's a little example. Now, think about human factors. Think about safety.

00:09:56.581 --> 00:09:59.341
What are we asked to do all the time? Compromise.

00:10:00.130 --> 00:10:03.650
What does everybody say? Well, you got to get your foot in the door.

00:10:04.110 --> 00:10:09.150
What do they say? You've got to reach out. You've got to touch them where they

00:10:09.150 --> 00:10:13.430
live in terms of their expectations and mental models about safety.

00:10:13.690 --> 00:10:17.630
And then you can start to move them. And then you can start to connect.

00:10:18.070 --> 00:10:21.410
You build it up. You get your foot in the door.

00:10:21.850 --> 00:10:31.090
So what did I start with as a joke in my 1999 presidential address for human factors.

00:10:31.710 --> 00:10:35.790
I said, you know, I've seen this all the time. I hear this all the time.

00:10:36.030 --> 00:10:40.530
You know, when they crack the door open, get your foot in there and then you

00:10:40.530 --> 00:10:43.370
can expand it and get inside and really help.

00:10:43.670 --> 00:10:45.710
Right. I mean, this is common.

00:10:46.130 --> 00:10:48.130
And I said, you know what really happens?

00:10:48.830 --> 00:10:53.910
It says they slammed the door shut. And then later when problems arise or accidents

00:10:53.910 --> 00:10:58.990
happen that have human factors, human systems, human automation problems, what do they do?

00:10:59.170 --> 00:11:02.890
They pick up your foot and wave it around and say, hey, I have human factors.

00:11:03.070 --> 00:11:04.350
It's right here. They lift their foot.

00:11:06.170 --> 00:11:11.690
So what does that mean for today and in today's context?

00:11:12.170 --> 00:11:14.390
Well, I have good news.

00:11:15.010 --> 00:11:22.510
The good news is what we started really, it's now 25 years since I first called

00:11:22.510 --> 00:11:29.090
for resilience engineering based on NASA accidents and investigations that occurred in 99,

00:11:29.610 --> 00:11:33.410
20 years since we had the first meeting on resilience engineering,

00:11:33.630 --> 00:11:38.110
is we've actually made progress on fundamentals.

00:11:38.910 --> 00:11:43.770
And, well, fundamentals, what's the reaction I get to fundamentals?

00:11:44.690 --> 00:11:50.690
Fundamentals, you know, they go, but those fundamentals, they don't show anybody

00:11:50.690 --> 00:11:53.570
what to do practically tomorrow in their organization.

00:11:53.850 --> 00:11:55.370
I'm looking at them like, what do you mean?

00:11:55.710 --> 00:12:02.870
I go, you know, nobody says, hey, you know, we've been building airframes,

00:12:03.050 --> 00:12:06.510
airplanes for decades. We know how to build them.

00:12:06.990 --> 00:12:10.650
Hey, this AI system, look at this wing.

00:12:11.270 --> 00:12:15.490
It's beautiful. And, you know, it's going to be great.

00:12:15.610 --> 00:12:21.290
This AI program generated a new image of an aircraft wing. Let's go build it.

00:12:22.610 --> 00:12:25.790
Wait a minute. Don't you have to take it to a wind? You have to put it together.

00:12:25.990 --> 00:12:29.910
You have to run some tests and simulation and the wind tunnel.

00:12:30.110 --> 00:12:33.390
It's a radically different wing. Oh, no.

00:12:34.390 --> 00:12:37.770
Aerodynamics is a solved problem. We don't need to do those tests.

00:12:37.890 --> 00:12:40.650
We don't need to do those analyses. You should be laughed out the door.

00:12:41.110 --> 00:12:44.350
Actually, today, I wonder whether you would still be laughed out the door.

00:12:44.610 --> 00:12:48.730
You should be laughed out the door, right? That's not engineering.

00:12:48.730 --> 00:12:50.250
That's not systems engineering.

00:12:51.821 --> 00:12:57.681
Yet that's what people who work on complex systems and safety are asked to do every day.

00:12:58.081 --> 00:13:02.561
You know, it feels good this way. You know, this is consistent with what we

00:13:02.561 --> 00:13:04.001
learned in an MBA program.

00:13:04.381 --> 00:13:09.481
You know, it would cost. Hey, that's not going to happen to us.

00:13:11.021 --> 00:13:15.881
Compromise. Compromise. Push it, you know, just make a small change.

00:13:16.581 --> 00:13:23.601
And you see this in what happened in the twin Boeing 737 MAX accidents with

00:13:23.601 --> 00:13:25.401
the MCAS automated system.

00:13:25.981 --> 00:13:30.641
And, you know, and then all these rationalizations and discounting mechanisms

00:13:30.641 --> 00:13:33.801
are out there in a case where, what did we do?

00:13:33.921 --> 00:13:37.121
We didn't listen to the fundamentals. We took safety for granted.

00:13:37.121 --> 00:13:41.261
It. We don't have to go back and test. We don't have to check.

00:13:41.441 --> 00:13:46.161
We don't have to make sure certain things in an integrated human automation

00:13:46.161 --> 00:13:49.021
system work. Do we understand what they are?

00:13:49.261 --> 00:13:55.161
Yeah, but compromise because otherwise it might raise some regulatory risk.

00:13:55.381 --> 00:13:57.181
It might delay the schedule.

00:13:57.581 --> 00:13:59.741
They killed 246 people.

00:14:00.652 --> 00:14:04.992
And then the other irony in this is it's cost them $30 billion and counting.

00:14:05.212 --> 00:14:07.312
I don't know where they are, $50 billion now?

00:14:07.832 --> 00:14:12.572
You know, the idea there's an economic case for safety has been blown up by Boeing.

00:14:13.172 --> 00:14:16.792
They said, I can save money now, so let's take shortcuts.

00:14:17.812 --> 00:14:22.472
Engineering shortcuts, safety shortcuts. And when it blows up,

00:14:22.572 --> 00:14:23.972
but they didn't think it would happen to them.

00:14:24.072 --> 00:14:28.092
It hadn't happened to them because they'd been taking these shortcuts for a

00:14:28.092 --> 00:14:32.072
long time. and their cumulative toll was building.

00:14:32.632 --> 00:14:38.472
So where do we stand? The good news is we have foundations. What's that like?

00:14:38.692 --> 00:14:44.072
It's like physics, aerodynamics, only it's for adaptive systems in this universe.

00:14:44.632 --> 00:14:50.792
Adaptive systems at a human system scale, that means technology is part of human systems.

00:14:51.032 --> 00:14:56.032
And we have fundamentals, we have laws, we have theorems, right?

00:14:56.132 --> 00:14:59.032
People always would say, where's your numbers don't you

00:14:59.032 --> 00:15:02.412
have numbers you know every time that

00:15:02.412 --> 00:15:05.572
happened to me when i started about alarms 1982

00:15:05.572 --> 00:15:08.912
30 years old and go

00:15:08.912 --> 00:15:14.832
okay let me show you the math for how how why alarms are misdesigned and how

00:15:14.832 --> 00:15:19.332
to do it better and you know what they said i didn't mean that math i don't

00:15:19.332 --> 00:15:24.152
understand that math and and and it gives a different answer than i expected

00:15:24.152 --> 00:15:26.592
and and if i to follow that answer,

00:15:26.832 --> 00:15:29.772
it'll cost me money. I'll have to do something different.

00:15:30.372 --> 00:15:32.132
Different. Oh my God, different.

00:15:33.032 --> 00:15:38.392
And so I said, wait a minute. You asked for the math. Here it is.

00:15:38.752 --> 00:15:44.032
Well, now guess what? We've got some math. We've got some advanced theorems. They're pretty weird.

00:15:44.312 --> 00:15:49.872
What do they show? Well, the world is inherently messy. The world is inherently nonlinear.

00:15:50.232 --> 00:15:53.672
All these compromises, is let's pretend it's linear.

00:15:54.532 --> 00:15:59.352
Let's pretend these are mostly independent parts. We can add in a few interactions.

00:16:00.806 --> 00:16:06.606
And again, this is an old story. This went back to the Lewis report in 1978,

00:16:06.606 --> 00:16:12.846
a year before the Three Mile Island accident happened, when they were asked to check on the accuracy,

00:16:13.786 --> 00:16:20.926
of probabilistic risk assessments on the safety of nuclear power plants. And what did they say?

00:16:21.466 --> 00:16:26.486
Basically, there's too many interactions and interdependencies, right?

00:16:26.626 --> 00:16:30.646
And it's hard to take into account the adaptive behavior of people.

00:16:31.186 --> 00:16:37.466
And so the only thing we can be sure of is that the analyses are an order of

00:16:37.466 --> 00:16:40.026
magnitude more uncertain than they pretend to be.

00:16:40.626 --> 00:16:44.866
An order of magnitude more uncertain, right? What were they highlighting?

00:16:45.106 --> 00:16:48.126
The complexity that's evident in our world everywhere.

00:16:48.646 --> 00:16:54.186
So the fact that we have laws, and yes, some of these you may say are provisional.

00:16:54.186 --> 00:16:56.986
Theorems can be overturned by new work.

00:16:57.506 --> 00:17:03.406
Laws can be qualified. Laws may be modified as we gain more evidence.

00:17:03.646 --> 00:17:08.826
But that's what the foundations are. That's our physics. That's our aerodynamics.

00:17:10.126 --> 00:17:14.726
And so I run through this and I go, and some of these are actually very simple.

00:17:15.746 --> 00:17:19.206
Let's take one. And we'll take one from social science first,

00:17:19.206 --> 00:17:24.126
because one of the differences about resilience as a systems engineering approach

00:17:24.126 --> 00:17:29.806
is that we take key ideas from all kinds of things that are non-traditional

00:17:29.806 --> 00:17:31.526
and don't normally talk to each other.

00:17:31.826 --> 00:17:38.386
Because that's what about adaptive systems in the human slash biological world.

00:17:38.706 --> 00:17:44.006
So let's take one, reciprocity. So Eleanor Ostrom gets the Nobel Prize in 2009

00:17:44.006 --> 00:17:51.006
for studies and work synthesizing the role of reciprocity in systems and human

00:17:51.006 --> 00:17:53.566
systems, human system scales,

00:17:53.766 --> 00:17:57.426
and how these things work to make systems that are adaptive.

00:17:58.510 --> 00:18:03.370
And that they can persevere over challenges, over longer timescales.

00:18:03.830 --> 00:18:07.690
And there's different ways to think about reciprocity. Well,

00:18:07.770 --> 00:18:13.870
it turns out in the fundamentals that a particular way to think about reciprocity is fundamental.

00:18:14.310 --> 00:18:19.830
So if you want to build a safe system, you need to build reciprocity between

00:18:19.830 --> 00:18:23.050
the different units, roles, and layers in your organization.

00:18:23.050 --> 00:18:28.590
Now, we see a lot of lip service to that, and we have always, right?

00:18:29.050 --> 00:18:33.870
But do you really build reciprocity? What does it really take?

00:18:34.130 --> 00:18:37.390
Now, we can describe that very simply. You don't have to know the theorems.

00:18:37.490 --> 00:18:41.190
You don't have to get into complex nonlinearities.

00:18:41.410 --> 00:18:47.910
You have to simply say and look at your system and say, where is reciprocity demonstrated?

00:18:47.990 --> 00:18:54.330
Where does reciprocity break down? how does management actually tell everyone

00:18:54.330 --> 00:18:57.090
that there is reciprocity going on?

00:18:57.210 --> 00:19:03.490
Or is it only ad hoc between horizontally between a few local roles because they're what?

00:19:03.670 --> 00:19:08.010
Committed to make the system work, fill the gaps, close the holes.

00:19:08.550 --> 00:19:11.330
Be the ad hoc source of resilient performance.

00:19:11.490 --> 00:19:18.290
So they find ways to reciprocate under stress in order to make the system work,

00:19:18.290 --> 00:19:27.750
generally in an environment that downplays or restricts or works against forms of reciprocity.

00:19:28.670 --> 00:19:33.550
And we can see this in successful stories of organizations that are highly adaptive

00:19:33.550 --> 00:19:36.290
to a dynamic world with changing risks.

00:19:36.630 --> 00:19:43.050
And so we can see these things play out. So it's very practical, But it's different.

00:19:43.510 --> 00:19:48.390
And that's when the compromise, the bad sense of compromise creeps in.

00:19:48.750 --> 00:19:54.430
No, we can't go that far. We can't do that. That doesn't fit with standard management practices.

00:19:54.850 --> 00:20:00.810
And what do they end up with? A slow organization that doesn't recognize things

00:20:00.810 --> 00:20:04.410
fast, that can't adapt quickly to changing circumstances.

00:20:04.410 --> 00:20:10.430
That's stuck in stale approaches in a world that is increasingly telling us

00:20:10.430 --> 00:20:13.850
about the turbulence and change that's ongoing.

00:20:14.190 --> 00:20:20.170
The thing we know for sure is that new kinds of challenges and shocks will occur.

00:20:20.530 --> 00:20:23.810
We don't know which ones, but we know they'll occur. Now, we know something

00:20:23.810 --> 00:20:26.310
about what makes such events shocking.

00:20:26.790 --> 00:20:28.450
That's back in the fundamentals.

00:20:29.050 --> 00:20:31.910
We know a bunch of stuff about how

00:20:31.910 --> 00:20:35.150
to be adaptive that's what's in the fundamentals do you

00:20:35.150 --> 00:20:38.910
want to put it into practice or do you want to say no no no no we're just we're

00:20:38.910 --> 00:20:42.770
just repairing a little bit over here that that's out of whack and a little

00:20:42.770 --> 00:20:48.350
bit over there and this discounting and narrowing that we wrote about in behind

00:20:48.350 --> 00:20:53.910
human error 40 years ago it's a 40 30 years ago 30 years ago 30 years ago.

00:20:55.378 --> 00:21:02.698
Plays out. So today's version of this isn't different than older versions. It's just that it's now.

00:21:03.198 --> 00:21:08.038
But the difference is we do have tools and techniques to deal with it that come

00:21:08.038 --> 00:21:09.238
out of those foundations.

00:21:09.518 --> 00:21:13.618
We have techniques to build reciprocity. We have tools that could help you tell

00:21:13.618 --> 00:21:16.278
whether you have reciprocity. Let's take another one.

00:21:17.138 --> 00:21:20.018
Saturation. This comes from engineering. It's in the textbooks.

00:21:20.238 --> 00:21:21.878
It's in the textbooks, for God's sake.

00:21:22.178 --> 00:21:32.298
And it gets ignored. But the textbooks try to hide and compromise away the implications that any agent,

00:21:32.598 --> 00:21:38.098
any system that's trying to control a dynamic, influence a dynamic world can

00:21:38.098 --> 00:21:39.698
saturate its responses.

00:21:40.038 --> 00:21:44.558
What does that mean? Hey, you got finite resources. The world keeps changing.

00:21:44.938 --> 00:21:48.998
Others are adapting around you. So whatever scale you're at,

00:21:49.058 --> 00:21:54.538
not just a control system, at every scale, this idea of saturation matters.

00:21:54.818 --> 00:21:59.658
What was competent can still be surprised.

00:21:59.958 --> 00:22:04.978
What was increasingly competent over before will get surprised because those

00:22:04.978 --> 00:22:07.118
limits will matter eventually.

00:22:07.118 --> 00:22:11.758
And the world will continue to throw events that surprise, right,

00:22:11.978 --> 00:22:19.118
the model you use to build competence or the changing model you use to enhance a form of competence.

00:22:19.798 --> 00:22:23.518
And there's a bunch of assumptions about the way we think about the world that

00:22:23.518 --> 00:22:25.398
turn out to be wrong and the fundamentals.

00:22:25.698 --> 00:22:31.918
So what's going on in the real world is the need to revise and reframe.

00:22:33.223 --> 00:22:37.063
And so we can talk about that. It's in the fundamentals. Can people do it?

00:22:37.203 --> 00:22:40.803
Yes. Is it hard to do? Yes. Do machines do it? No.

00:22:41.043 --> 00:22:43.723
Does anyone try to make machines reframe? No.

00:22:44.343 --> 00:22:48.663
So let's go back to saturation. It turns out it's universal.

00:22:49.143 --> 00:22:53.603
It turns out in adaptive systems, they adapt before they reach saturation.

00:22:53.823 --> 00:22:57.363
Why? Because if you hit saturation, your effectiveness at responding to whatever

00:22:57.363 --> 00:22:59.983
is going on just went to zero or near zero.

00:23:00.303 --> 00:23:02.763
What does that mean? That's a brittle collapse.

00:23:03.223 --> 00:23:07.863
Brittle collapse has big penalties. It's a step change in penalties.

00:23:08.203 --> 00:23:10.143
Could be an economic penalty.

00:23:10.503 --> 00:23:15.963
Could be a weakening of how your system works that will mean the next time it's

00:23:15.963 --> 00:23:17.363
challenged, it won't work as well.

00:23:18.303 --> 00:23:24.303
And it has costs right now. So biology wants to avoid brittle collapse.

00:23:24.463 --> 00:23:29.123
This is why I can summarize a lot of the resilience engineering as we build

00:23:29.123 --> 00:23:32.463
competent but brittle systems because we ignore these fundamentals.

00:23:32.683 --> 00:23:36.763
We want to use new tools and techniques that we're developing in resilience

00:23:36.763 --> 00:23:40.823
engineering to build competent and extensible systems.

00:23:41.103 --> 00:23:45.963
Yes, should you do work to build up new forms of competency,

00:23:46.263 --> 00:23:49.463
take advantage of new technical capability? Sure.

00:23:50.043 --> 00:23:55.663
It doesn't change the larger equation about how adaptive systems work,

00:23:55.783 --> 00:24:00.703
about how adaptive systems are challenged, about how adaptive systems can malfunction.

00:24:01.063 --> 00:24:06.043
Those fundamentals hold, just like aerodynamics holds, just like physics holds.

00:24:07.083 --> 00:24:12.383
So everyone needs to understand saturation, approaching saturation,

00:24:12.523 --> 00:24:13.983
what are the forms of adaptation.

00:24:15.264 --> 00:24:19.664
That can go on. There's some basic classes. There's opportunities for research

00:24:19.664 --> 00:24:24.844
to develop and understand other forms of adaptation that go on approaching saturation

00:24:24.844 --> 00:24:29.204
so that systems can be extensible as well as competent.

00:24:29.544 --> 00:24:31.804
Again, it's pragmatic.

00:24:32.384 --> 00:24:35.384
There's things to do, but it's different.

00:24:35.784 --> 00:24:40.504
Now, there's one other challenge for us, and that is it's not as mature.

00:24:40.984 --> 00:24:48.464
Why do people think about aerodynamics as or control engineering as easy and

00:24:48.464 --> 00:24:53.504
resilience engineering is, oh, we can't do that, is because the others are mature.

00:24:53.724 --> 00:24:58.924
So they've created very efficient tools, tooling and tools.

00:24:59.204 --> 00:25:01.824
You don't have to go back to first principles.

00:25:02.324 --> 00:25:10.904
You don't have to do custom craft work to translate laws into engineering practice, right?

00:25:11.064 --> 00:25:16.284
We have engineering practices. We have software that supports those engineering practices.

00:25:16.764 --> 00:25:21.924
But what do they tell us? They go, you have to fit the old engineering practices. You have to compromise.

00:25:22.364 --> 00:25:29.764
And you go, but those practices don't address the complexity penalties that arise as growth goes on.

00:25:30.064 --> 00:25:39.464
New roles, new scales, new interdependencies, a more tangled network of roles

00:25:39.464 --> 00:25:41.864
and interdependencies across layers.

00:25:42.504 --> 00:25:48.384
Tangled, hard to see, hard to notice where the connections are going to matter

00:25:48.384 --> 00:25:50.684
as the world continues to change.

00:25:51.444 --> 00:25:56.584
So we're not as mature? Yes. Are we different?

00:25:57.064 --> 00:26:04.104
Yes. Does that mean we're not actionable? Not in the least. Are you willing to change?

00:26:04.644 --> 00:26:11.484
Or are you going to stay stuck in old ways when the world is shouting at you?

00:26:12.064 --> 00:26:16.984
It's different. Your old assumptions don't hold anymore.

00:26:17.344 --> 00:26:22.844
You need to revise, reframe how you do things if you want to be adaptive to

00:26:22.844 --> 00:26:26.884
be viable in the world we live in now.

00:26:27.164 --> 00:26:31.964
You need to be able to synchronize across roles and layers, for example,

00:26:32.124 --> 00:26:33.464
by building reciprocity.

00:26:33.804 --> 00:26:38.984
You need to be able to be highly responsive when disturbances arise.

00:26:40.244 --> 00:26:45.244
And that means you have to understand how to adapt ahead of reaching saturation,

00:26:45.524 --> 00:26:51.104
whether that's about a mass casualty event in a hospital system or whether that

00:26:51.104 --> 00:26:58.084
is operating on a societal scale when a pandemic arises or when an industry

00:26:58.084 --> 00:27:03.304
is threatened with collapse due to economic and political factors.

00:27:04.327 --> 00:27:11.507
So the challenges out there really demand the foundations we have behind resilience engineering.

00:27:11.787 --> 00:27:18.087
They really ask to say, well, let's invest and build the maturity of these tools as we move forward.

00:27:18.347 --> 00:27:20.247
And let's not compromise.

00:27:21.287 --> 00:27:25.987
And I want to close by saying, and you can have the link, I wrote a preface

00:27:25.987 --> 00:27:30.927
for the report from our 20th anniversary meeting on resilience engineering.

00:27:30.927 --> 00:27:37.867
And it highlights a few of these simple points about what's a systems engineering

00:27:37.867 --> 00:27:41.967
area, even though this one is very different, that includes the human realm,

00:27:42.227 --> 00:27:43.567
organizational factors,

00:27:43.967 --> 00:27:47.527
other kinds of things we normally think of as separate from engineering.

00:27:47.807 --> 00:27:51.827
And that all gets integrated because it's about adaptation and complexity.

00:27:52.687 --> 00:27:58.127
And so it comes back and runs through some of these different points about how

00:27:58.127 --> 00:28:00.287
do you and when can you compromise?

00:28:01.007 --> 00:28:06.167
And what we have to be able to do in our field is recognize that there are certain

00:28:06.167 --> 00:28:08.527
fundamentals that say you can't compromise on.

00:28:08.647 --> 00:28:11.487
And we have to stand up and say you can't compromise on this.

00:28:11.647 --> 00:28:16.187
And if they want to go to some fly-by-night compromiser, I will compromise on

00:28:16.187 --> 00:28:18.387
anything if you give me the work.

00:28:18.627 --> 00:28:20.627
I will justify anything you want to do.

00:28:21.087 --> 00:28:25.467
I'll pretend it matters. No, no, no. We have real science.

00:28:25.667 --> 00:28:30.947
We have real theorems. We have real formal theories like how physics works only

00:28:30.947 --> 00:28:35.207
for the biological, human, technological, adaptive world.

00:28:35.447 --> 00:28:37.907
So you want to challenge them, challenge them.

00:28:38.327 --> 00:28:42.527
But you don't get to just compromise on them. You just don't get to pretend

00:28:42.527 --> 00:28:46.287
they don't exist. Sorry, we've made advances.

00:28:47.727 --> 00:28:50.427
So you've got to say how did you

00:28:50.427 --> 00:28:53.767
take this account into account in your engineering trade-offs

00:28:53.767 --> 00:28:59.667
not say i don't need to take it into account you need to do testing that says

00:28:59.667 --> 00:29:05.047
you're addressing these kinds of issues not say that's too expensive and this

00:29:05.047 --> 00:29:09.507
is too new or too different and you have to be willing to architect your systems

00:29:09.507 --> 00:29:13.587
in different ways because the way you architect systems now,

00:29:13.847 --> 00:29:16.147
in the technological,

00:29:16.647 --> 00:29:25.707
human technological, human organizational scopes, those layers are guaranteed to be slow and stale.

00:29:25.847 --> 00:29:28.947
They're guaranteed to become fragmented under stress.

00:29:29.167 --> 00:29:35.267
They're guaranteed to come to be poised to be for brittle collapse.

00:29:35.267 --> 00:29:38.807
They will appear competent over time.

00:29:39.047 --> 00:29:42.927
The statistics will look good while brittleness grows.

00:29:43.567 --> 00:29:46.887
And then there will be a collapse at shocks.

00:29:47.147 --> 00:29:53.827
But these are actually predictable characteristics of the way we build competent

00:29:53.827 --> 00:29:58.227
but brittle systems when we could have built competent and extensible systems.

00:29:59.115 --> 00:30:05.895
So the piece on this and the entire Foundations for Resilience Engineering video series,

00:30:06.215 --> 00:30:12.775
which I think you'll put the links up for as well, all of this is trying to

00:30:12.775 --> 00:30:18.675
highlight the fundamentals that guide practical action, the fundamentals that

00:30:18.675 --> 00:30:20.315
generalize across settings.

00:30:20.575 --> 00:30:26.575
They point us towards the tools and techniques that are starting to emerge.

00:30:26.575 --> 00:30:32.215
Now, granted, I don't have as much discussion in the video series so far on

00:30:32.215 --> 00:30:37.255
the techniques and tools and more examples of putting those techniques into action.

00:30:37.455 --> 00:30:42.435
But we have more and more papers that are available that illustrate them.

00:30:42.555 --> 00:30:44.475
And we're working on more videos.

00:30:45.095 --> 00:30:49.475
As you well know, post-production on these things takes effort and time.

00:30:50.195 --> 00:30:56.595
And we're all, what, nearly saturated. In fact, that is one of the hallmarks of modern systems.

00:30:56.895 --> 00:31:02.495
They operate under faster, better, cheaper pressure. They operate nearly saturated regularly.

00:31:02.835 --> 00:31:08.355
That doesn't mean you don't have periods that ease off, but the experience of

00:31:08.355 --> 00:31:13.775
approaching overload, approaching saturation, is regular and tangible to everybody.

00:31:13.775 --> 00:31:16.755
Right because what does everybody do if you're dealing with

00:31:16.755 --> 00:31:19.835
a friend and you're a little late and you're in your exchanges and

00:31:19.835 --> 00:31:22.975
whatever what do you always say i'm sorry but i had a period of overload it's

00:31:22.975 --> 00:31:26.555
like everybody's has periods of overload everybody's

00:31:26.555 --> 00:31:33.275
behind getting back on things why because new events happen which change your

00:31:33.275 --> 00:31:40.095
strategy for managing overload shifting things around and so we're all experiencing

00:31:40.095 --> 00:31:43.075
the dynamics of resilience and brittleness.

00:31:43.395 --> 00:31:49.495
We're all experiencing local adaptations in order to survive and the kinds of

00:31:49.495 --> 00:31:53.135
sacrifices, reconfigurations, reprioritization that go on.

00:31:53.315 --> 00:31:58.715
The issue isn't that we don't adapt, that there isn't some resilience. It's that it's ad hoc.

00:31:59.781 --> 00:32:04.201
And therefore, it will be insufficient and less effective than it could be.

00:32:04.481 --> 00:32:09.921
And that's why we need these larger layers of the organization to join with

00:32:09.921 --> 00:32:15.361
the rest of us in building adaptive capacity, enhancing our ability to demonstrate

00:32:15.361 --> 00:32:16.441
resilient performance.

00:32:16.841 --> 00:32:20.981
And we have plenty of examples from biology at different scales,

00:32:21.261 --> 00:32:25.281
from human systems, from cognitive systems, human automation,

00:32:25.501 --> 00:32:26.701
well-designed systems.

00:32:27.321 --> 00:32:30.901
They're just, the human technology ones just aren't the norm.

00:32:31.041 --> 00:32:35.121
The human organizational ones just aren't the norm because we're stuck in these

00:32:35.121 --> 00:32:36.761
old, stale beliefs. Yeah.

00:32:38.021 --> 00:32:42.601
Phenomenal. No, that was really good. I mean, that was phenomenal.

00:32:43.261 --> 00:32:48.981
Thank you. Well, I had to tell the old stories because I'm old. No, they need to be.

00:32:49.601 --> 00:32:53.581
That's exactly the conversation that the world needs right now.

00:32:53.741 --> 00:32:58.781
It's brilliant. Well, and we need this conversation in the context of safety

00:32:58.781 --> 00:33:03.161
and Jim Reason and past battles and compromises and, you know,

00:33:03.281 --> 00:33:08.281
how this cheese got, you know, is in some ways a compromise to help people understand.

00:33:08.281 --> 00:33:13.421
And then what happens is people deliberately take advantage of the opportunity

00:33:13.421 --> 00:33:17.461
to misunderstand, to diffuse it, to make it safe.

00:33:17.680 --> 00:33:23.760
Music.

00:33:24.621 --> 00:33:30.261
I told you you'd love this podcast. It is just one little gem of information

00:33:30.261 --> 00:33:35.461
after another little gem of information, and they all sort of add up to a big gem of information.

00:33:35.941 --> 00:33:40.481
More to come, because the one promise I got David to make with us is that we do this again.

00:33:40.641 --> 00:33:44.141
It is always so much fun to listen to him. Thanks for listening.

00:33:44.441 --> 00:33:47.101
This means so much to me. Thanks for being a part of it.

00:33:47.661 --> 00:33:50.781
Learn something new every single day. I know you did today. Have as much fun

00:33:50.781 --> 00:33:52.901
as you possibly can. Be good to each other. Be kind to each other.

00:33:52.901 --> 00:33:54.221
And for goodness sakes, you guys.

00:33:55.120 --> 00:34:06.744
Music.