Going "Code Dark": An Emergency Response to Cyberattacks

00;00;00;02 - 00;00;50;08

Tom Haederle

In most hospitals, highly trained staff members know just what to do in the case of an

emergency, be it a fire or hurricane or even an active shooter. But what happens when

that emergency comes in the form of a dangerous cyber-attack? 

Welcome to Advancing Health, a podcast brought to you by the American Hospital

Association. I'm Tom Haederle with AHA Communications. Children's National Hospital

in Washington, D.C. has developed a sort of system wide off switch that staff members

can use if it is clear that the hospital has come under a broad based ransomware or

malware attack.

00;00;50;15 - 00;01;14;27

Tom Haederle

It's called Code Dark and it empowers staff members to disconnect critical patient care

devices from the system as quickly as possible in the event of an outside cyber-attack,

which has become increasingly common over the past several years. Here to describe

Code Dark and how it works is Nate Lesser, vice president and chief information

security officer with Children's National Hospital, in conversation with John Riggi, AHAÕs

national adviser for cybersecurity and risk.

00;01;15;09 - 00;01;20;06

Tom Haederle

Gentlemen, over to you.

00;01;21;05 - 00;01;48;06

John Riggi

Thank you, Tom, and thanks to everyone for joining us again for another Advancing

Health podcast. In the series, I host, we focus on national cyber and risk issues in which

we feature highly accomplished leaders from the healthcare field, cyber security

industry, and government. They provide us with their frontline perspective in the never

ending battle against the serious cyber threats we all face in the health care field and as

a nation.

00;01;48;23 - 00;02;21;13

John Riggi

Today, I'm truly pleased to have with me Nate Lesser, my friend and colleague from

Children's National Hospital. Nate Lesser is a vice president and the chief information

security officer at Children's National. Nate is an engineer and information security

expert who has spent the last two decades working in various positions inside and

outside of government, including serving as the deputy director of the Mist National

Cyber Security Center of Excellence, the Cybersecurity National Lab.

00;02;21;25 - 00;02;54;27

John Riggi

So, Nate, thanks for joining us today to spend a little time to talk about what I believe is

a great initiative that you all developed at Children's National in that certainly has

national application and relevance, unfortunately, in this time of dramatically increased

ransomware attacks against hospitals. You developed a special system known as Code

Dark. In general terms, Nate, please describe what Code Dark is and why did you

develop it?

00;02;55;08 - 00;03;22;00

Nate Lesser

Sure. And thanks for having me. It's great to be here, and thanks for your leadership

with the American Hospital Association. We're very grateful to have you in that role.

Code Dark is really very straight forward. It aligns well with the emergency operations

plan of the hospital where we have the rest of the codes that we call and is built to

empower our staff across the entire hospital to become cyber first responders.

00;03;22;16 - 00;03;37;09

Nate Lesser

It's a code that we would call in the instance where our network becomes the attack

vector and is being used and exploited by attackers to infect devices across the

hospital, as we've often seen with broad based ransomware attacks.

00;03;38;00 - 00;04;01;16

John Riggi

Thanks, Nate. So interesting that you were internally there with your emergency

management team. And what I think is one of the great features of this code that you

develop is that, again, focused internally and working with your emergency

management team. What are the most significant ways establishing Code Dark will

reduce cyber risk?

00;04;02;08 - 00;04;26;09

Nate Lesser

Well, foundationally, we've seen that not only do we need to get ahead of the curve,

right? We talk about this all the time in cybersecurity. We have to be more proactive.

We have to be more predictive. But we also have seen that when a hospital has

unfortunately been subject to a successful ransomware attack, they're often in downtime

procedures for weeks or even more than a month.

00;04;26;26 - 00;04;57;08

Nate Lesser

And Code Dark really focuses not on how do we prevent the attack from happening?

We have other work and in flight, of course, to do that. But it helps us to focus on how

do we get back up and running as fast as possible, how do we how do we recover from

a successful attack if, unfortunately, we are in the same boat as many of our colleagues

across health care in the United States and are hit by one of these sophisticated

attacks, how do we get the system back up and running as fast as possible?

00;04;57;21 - 00;05;24;23

Nate Lesser

And to do that, we really felt we needed to coordinate not just within information

security, not just within information technology, but across the entire staff of the hospital.

And the great news is that, like all other hospitals we have in place, communication and

coordination channels to address system wide emergencies, whether it's a hurricane or

an active shooter, a fire in one of our facilities.

00;05;25;11 - 00;05;41;07

Nate Lesser

So we just leveraged the work that they did to establish those communication channels

and built an additional code that empowers our staff to disconnect devices as quickly as

possible if we're in one of those ransomware or broad based malware attack scenarios.

00;05;41;25 - 00;06;12;03

John Riggi

Yeah, thanks for that, Nate. And I think you brought out a key point, which I often also

stress, is that to leverage the existing capabilities and expertise of emergency

preparedness, folks that have planned for other high impact events such as hurricanes,

natural disasters, active shooters, those systems and capabilities, response capabilities

in place and often cyber incidents are not included there.

00;06;12;12 - 00;06;26;20

John Riggi

And truly educating the staff that is already prepared, mentally prepared to respond to

such high impact events. When would you call a code dark and what are you asking

specifically the hospital staff to do?

00;06;27;14 - 00;06;54;18

Nate Lesser

Yeah, it's a great question. So we would call it in any instance where we are confident

that there is fast spreading malware moving across our network. So where we are

actively combating an attack, we simultaneously want devices off the network as quickly

as possible. If the network becomes the attack vector, then like most other hospitals

weÕll be in a position where we can try and centrally bring down the network.

00;06;55;03 - 00;07;21;02

Nate Lesser

But that is a challenge of course, and it doesn't happen instantaneously. So in

partnership with the information security teams efforts to combat the attack, we're

asking our staff to get devices off the network. So what does that mean? Individual

computers? We ask them to put them into airplane mode. Phones and tablets, the same

thing, airplane mode, get it off the wireless network. 

00;07;21;02 - 00;07;33;17

Nate Lesser

Medical equipment. We're asking everyone to pull those devices, literally disconnect

network cables so that our devices are off the network and more of our devices will be

protected from this fast moving attack.

00;07;34;15 - 00;08;02;24

John Riggi

I think this is really very, very astute of you and the staff to understand that it will take

time to shut down the network in the midst of a high impact ransomware attack. And

ultimately, delay is the enemy. That time delays the enemy of the hospital when you're

trying to prevent the network and the face of rapidly propagating ransomware.

00;08;02;24 - 00;08;21;23

John Riggi

And I think that's really ingenious to ask the staff to go around, put devices in airplane

mode and unplug cables. Let me just one follow up question here. And obviously, how

do they know which cables to unplug without causing some other type of damage or

malfunction in the machine?

00;08;21;23 - 00;08;48;21

Nate Lesser

Yeah, it's a really great question. So a lot of training, that's the answer. So it's this is you

know, we think of this as a relatively simple response scenario, but to do it well and to

ensure staff are not in a position where, like one of our leaders said to me when I first

got here, hey, if you told me I have a ransomware on my device, I ransomware on my

device, or if I saw a ransomware message, I would probably throw my laptop out the

window because I don't know what else to do and it's too scary.

00;08;48;21 - 00;09;14;20

Nate Lesser

You know, that was very triggering for me. It was good to know that that's the mindset of

and this is a relatively senior leader here, one of our providers. It helped us to level set

and be able to say: okay, we need to provide visuals, actual pictures from across the

hospital. This is not only what a network table looks like, but this is what the network

cable looks like, plugged into the back of a infusion pop.

00;09;14;26 - 00;09;48;27

Nate Lesser

This is what a network cable looks like when it's plugged into a large piece of medical

equipment, like an MRI, and ensure that our staff not only have the opportunity to get

that training from us, but also have the opportunity to ask us questions. So there's a lot

of routing involved, there's a lot of training, there's a lot of direct interaction, and we

produce material too, like badge cards, lanyards that people wear around their neck and

stickers that we put on the side of medical devices across the hospital that say, What

should I do in response to it in response to Code Dark?

00;09;49;29 - 00;10;14;22

John Riggi

Fantastic. Nate clearly you didn't just issue a two-line guidance or order to the staff that

if this happens, just start unplugging cables. So clearly a lot of thought put into this. And

I think the cards on the lanyards that folks literally have around their neck at all time is a

way to have folks ensure they have access to the information during an emergency.

00;10;15;03 - 00;10;36;24

John Riggi

We all know, all of us that have been involved in emergency response in one man or

another know that during the midst of a crisis, folks will not remember key steps unless

they've been trained and the information is readily available. Nate, what steps would

you suggest for other hospitals looking to implement a similar Code Dark?

00;10;37;17 - 00;11;06;12

Nate Lesser

We've been making this information as widely available as we can. Certainly we are

happy to share, I think, you know, through American Hospital Association, through the

HISAC, CHIME and other collaborators we work with. As a cybersecurity industry, we've

gotten very good, I think at sharing information, kind of what I think of as laterally right

across hospitals, across organizations, and that's essential.

00;11;06;12 - 00;11;27;09

Nate Lesser

So we've learned from that activity that we also need to think about sharing vertically,

right? So that's how we got into this position of saying, okay, we need to bring the entire

staff of the hospital along with us. We need to think not just in terms of, you know,

information security, information technology and then the, you know, providers, but also,

you know, who's working in the kitchens and who's guarding the doors?

00;11;27;10 - 00;11;47;13

Nate Lesser

And what does it look like when we are in one of these downtime scenarios and how do

we reduce the blast radius? But as far as sharing this information broadly across the

community, we're happy to do it. We have been doing some of it with your health and

with others, and we're more than happy to do that in any form that we can find.

00;11;48;05 - 00;12;20;21

John Riggi

Nate, thanks so much. We certainly appreciate sharing your knowledge and best

practices out to the larger community. Unfortunately, as we've seen there, and when

one hospital or health system is attacked, there is in fact an effect, a blast radius

regionally. And sometimes even statewide. So truly a collaborative amongst information

security personnel and really all the leadership of hospitals to share best practices and

lessons learned from some of these high impact ransomware attacks.

00;12;20;28 - 00;12;43;12

John Riggi

I say this all the time, truly mean it because we've seen the effect that to defend one is

to defend all. Nate, truly appreciate that. And if an organization would like to contact you

directly, could you give us your email or an email that perhaps folks could write to you

and get some more information?

00;12;44;04 - 00;13;09;13

Nate Lesser

Yeah, they should absolutely feel free to reach out to me directly. You can probably put

it in the show notes. My email address is just my first initial last name at Children's

National.org. And you know, I would say to the question that you asked before, I guess

I'd add one more point, which is I think that the internal collaboration, especially with

your emergency management team, something that you preach a lot, John, is really

essential, right?

00;13;09;13 - 00;13;34;04

Nate Lesser

If you're trying to do this in a vacuum, again, whether it's vertically or horizontally, you're

not going to be successful. So for us, the key was how do we build it? Really, our entire

cyber incident response program into the broader emergency operations plan of the

hospital. How do we leverage things like our administrator on call structure and our

emergency communication structures?

00;13;34;19 - 00;13;48;11

Nate Lesser

I think that's an essential piece. And if you're starting out, if you haven't, you know, put

one of these programs in place yet, if you don't have a Code Dark, but you're interested

in doing it, I would start there. I'd start with a collaboration with your emergency

management team.

00;13;49;06 - 00;14;08;00

John Riggi

Nate, I certainly appreciate you sharing your knowledge and the details of this best

practice that you all developed here at Children's National. But there are those who are

concerned with the possibility of increased risk due to the public sharing of best

practices and lessons learned. How do you respond to those concerns?

00;14;08;25 - 00;14;33;21

Nate Lesser

Well, I think it's a really good question, right? So out of the gate, we tend to think about

cybersecurity information security as something that we try to keep it sensitive. So we

try to keep it secret. And certainly that is an essential function of information security.

The reality is our attackers are very good at sharing information and we have to be very,

very good at sharing best practices by sharing a best practice.

00;14;33;22 - 00;15;02;08

Nate Lesser

Are we certainly do, I think, elevate the practice of all of our partners and our entire

industry in health care. And I would say that those who are concerned that perhaps we

at Children's National or anyone who gets this kind of exposure or is out front offering

suggestions makes us a bigger target. I would only argue that the attackers know who

we are.

00;15;02;09 - 00;15;33;21

Nate Lesser

We're not hiding under the covers, right? We are not successfully obscuring our identity

by keeping our experience and cases where we have something we have developed by

keeping it secret. I think we're only serving to harm the rest of the industry by not

sharing, by not collaborating. So at the end of the day, I would say, you know, if

anything, being out front in sharing best practices makes us individually and as a

community less of a target.

00;15;33;21 - 00;15;50;08

Nate Lesser

Attackers that are running a business and they will look for the softest targets they can

find. So I would encourage everybody who listens to this podcast to participate in the

dialog, to share feedback. If you see something that you think we can improve, if you

take a look at our Code Dark and you have feedback, we'd love to hear it.

00;15;50;19 - 00;15;57;05

Nate Lesser

If you want to implement it and have different ways of doing it that you think might be

better, that's fantastic. Please share it with the rest of the community.

00;15;58;07 - 00;16;27;27

John Riggi

Great advice, Nate. And it really points out that Code Dark is not a system or program

that stands alone. It's really a byproduct of that productive interaction with your

emergency management team. I could not agree with you more, Nate, that often that

cyber incident response plans are developed in a silo without sufficient coordination with

the emergency management team.

00;16;27;27 - 00;16;58;12

John Riggi

As I did in the government, I often find there are pockets of expertise and knowledge

within an organization that's often siloed, and we don't have to spend a lot of money,

often for outside consultants or high tech solutions. It's simply a matter of, as you said,

coordinating internally, leveraging the existing expertise and programs. Really it

becomes a force multiplier. Yeah, to limit that ransomware blast radius that you and I

often speak about internally and externally.

00;16;59;01 - 00;17;23;29

John Riggi

Thanks again, Nate, for sharing your knowledge and your innovative program on Code

Dark, no doubt will help defend your networks and help to defend the networks of health

systems across the United States. And especially thank you to all the health care

providers at Children's National and across the United States for everything you've done

to care for our patients in our communities during the pandemic.

00;17;23;29 - 00;17;44;07

John Riggi

And now this has been John Riggi. You're national advisor for Cybersecurity and Risk.

Stay safe, everyone.