In most hospitals, highly-trained staff members know just what to do in the case of an emergency, whether it is a fire, hurricane, or even an active shooter. But what happens when that emergency comes in the form of a dangerous cyberattack? Children’s National Hospital in Washington, D.C., has developed a system-wide “off” switch that staff can activate if it is clear that the hospital has come under a broad-based ransomware or malware attack.
00;00;00;02 - 00;00;50;08
In most hospitals, highly trained staff members know just what to do in the case of an
emergency, be it a fire or hurricane or even an active shooter. But what happens when
that emergency comes in the form of a dangerous cyber-attack?
Welcome to Advancing Health, a podcast brought to you by the American Hospital
Association. I'm Tom Haederle with AHA Communications. Children's National Hospital
in Washington, D.C. has developed a sort of system wide off switch that staff members
can use if it is clear that the hospital has come under a broad based ransomware or
00;00;50;15 - 00;01;14;27
It's called Code Dark and it empowers staff members to disconnect critical patient care
devices from the system as quickly as possible in the event of an outside cyber-attack,
which has become increasingly common over the past several years. Here to describe
Code Dark and how it works is Nate Lesser, vice president and chief information
security officer with Children's National Hospital, in conversation with John Riggi, AHAÕs
national adviser for cybersecurity and risk.
00;01;15;09 - 00;01;20;06
Gentlemen, over to you.
00;01;21;05 - 00;01;48;06
Thank you, Tom, and thanks to everyone for joining us again for another Advancing
Health podcast. In the series, I host, we focus on national cyber and risk issues in which
we feature highly accomplished leaders from the healthcare field, cyber security
industry, and government. They provide us with their frontline perspective in the never
ending battle against the serious cyber threats we all face in the health care field and as
00;01;48;23 - 00;02;21;13
Today, I'm truly pleased to have with me Nate Lesser, my friend and colleague from
Children's National Hospital. Nate Lesser is a vice president and the chief information
security officer at Children's National. Nate is an engineer and information security
expert who has spent the last two decades working in various positions inside and
outside of government, including serving as the deputy director of the Mist National
Cyber Security Center of Excellence, the Cybersecurity National Lab.
00;02;21;25 - 00;02;54;27
So, Nate, thanks for joining us today to spend a little time to talk about what I believe is
a great initiative that you all developed at Children's National in that certainly has
national application and relevance, unfortunately, in this time of dramatically increased
ransomware attacks against hospitals. You developed a special system known as Code
Dark. In general terms, Nate, please describe what Code Dark is and why did you
00;02;55;08 - 00;03;22;00
Sure. And thanks for having me. It's great to be here, and thanks for your leadership
with the American Hospital Association. We're very grateful to have you in that role.
Code Dark is really very straight forward. It aligns well with the emergency operations
plan of the hospital where we have the rest of the codes that we call and is built to
empower our staff across the entire hospital to become cyber first responders.
00;03;22;16 - 00;03;37;09
It's a code that we would call in the instance where our network becomes the attack
vector and is being used and exploited by attackers to infect devices across the
hospital, as we've often seen with broad based ransomware attacks.
00;03;38;00 - 00;04;01;16
Thanks, Nate. So interesting that you were internally there with your emergency
management team. And what I think is one of the great features of this code that you
develop is that, again, focused internally and working with your emergency
management team. What are the most significant ways establishing Code Dark will
reduce cyber risk?
00;04;02;08 - 00;04;26;09
Well, foundationally, we've seen that not only do we need to get ahead of the curve,
right? We talk about this all the time in cybersecurity. We have to be more proactive.
We have to be more predictive. But we also have seen that when a hospital has
unfortunately been subject to a successful ransomware attack, they're often in downtime
procedures for weeks or even more than a month.
00;04;26;26 - 00;04;57;08
And Code Dark really focuses not on how do we prevent the attack from happening?
We have other work and in flight, of course, to do that. But it helps us to focus on how
do we get back up and running as fast as possible, how do we how do we recover from
a successful attack if, unfortunately, we are in the same boat as many of our colleagues
across health care in the United States and are hit by one of these sophisticated
attacks, how do we get the system back up and running as fast as possible?
00;04;57;21 - 00;05;24;23
And to do that, we really felt we needed to coordinate not just within information
security, not just within information technology, but across the entire staff of the hospital.
And the great news is that, like all other hospitals we have in place, communication and
coordination channels to address system wide emergencies, whether it's a hurricane or
an active shooter, a fire in one of our facilities.
00;05;25;11 - 00;05;41;07
So we just leveraged the work that they did to establish those communication channels
and built an additional code that empowers our staff to disconnect devices as quickly as
possible if we're in one of those ransomware or broad based malware attack scenarios.
00;05;41;25 - 00;06;12;03
Yeah, thanks for that, Nate. And I think you brought out a key point, which I often also
stress, is that to leverage the existing capabilities and expertise of emergency
preparedness, folks that have planned for other high impact events such as hurricanes,
natural disasters, active shooters, those systems and capabilities, response capabilities
in place and often cyber incidents are not included there.
00;06;12;12 - 00;06;26;20
And truly educating the staff that is already prepared, mentally prepared to respond to
such high impact events. When would you call a code dark and what are you asking
specifically the hospital staff to do?
00;06;27;14 - 00;06;54;18
Yeah, it's a great question. So we would call it in any instance where we are confident
that there is fast spreading malware moving across our network. So where we are
actively combating an attack, we simultaneously want devices off the network as quickly
as possible. If the network becomes the attack vector, then like most other hospitals
weÕll be in a position where we can try and centrally bring down the network.
00;06;55;03 - 00;07;21;02
But that is a challenge of course, and it doesn't happen instantaneously. So in
partnership with the information security teams efforts to combat the attack, we're
asking our staff to get devices off the network. So what does that mean? Individual
computers? We ask them to put them into airplane mode. Phones and tablets, the same
thing, airplane mode, get it off the wireless network.
00;07;21;02 - 00;07;33;17
Medical equipment. We're asking everyone to pull those devices, literally disconnect
network cables so that our devices are off the network and more of our devices will be
protected from this fast moving attack.
00;07;34;15 - 00;08;02;24
I think this is really very, very astute of you and the staff to understand that it will take
time to shut down the network in the midst of a high impact ransomware attack. And
ultimately, delay is the enemy. That time delays the enemy of the hospital when you're
trying to prevent the network and the face of rapidly propagating ransomware.
00;08;02;24 - 00;08;21;23
And I think that's really ingenious to ask the staff to go around, put devices in airplane
mode and unplug cables. Let me just one follow up question here. And obviously, how
do they know which cables to unplug without causing some other type of damage or
malfunction in the machine?
00;08;21;23 - 00;08;48;21
Yeah, it's a really great question. So a lot of training, that's the answer. So it's this is you
know, we think of this as a relatively simple response scenario, but to do it well and to
ensure staff are not in a position where, like one of our leaders said to me when I first
got here, hey, if you told me I have a ransomware on my device, I ransomware on my
device, or if I saw a ransomware message, I would probably throw my laptop out the
window because I don't know what else to do and it's too scary.
00;08;48;21 - 00;09;14;20
You know, that was very triggering for me. It was good to know that that's the mindset of
and this is a relatively senior leader here, one of our providers. It helped us to level set
and be able to say: okay, we need to provide visuals, actual pictures from across the
hospital. This is not only what a network table looks like, but this is what the network
cable looks like, plugged into the back of a infusion pop.
00;09;14;26 - 00;09;48;27
This is what a network cable looks like when it's plugged into a large piece of medical
equipment, like an MRI, and ensure that our staff not only have the opportunity to get
that training from us, but also have the opportunity to ask us questions. So there's a lot
of routing involved, there's a lot of training, there's a lot of direct interaction, and we
produce material too, like badge cards, lanyards that people wear around their neck and
stickers that we put on the side of medical devices across the hospital that say, What
should I do in response to it in response to Code Dark?
00;09;49;29 - 00;10;14;22
Fantastic. Nate clearly you didn't just issue a two-line guidance or order to the staff that
if this happens, just start unplugging cables. So clearly a lot of thought put into this. And
I think the cards on the lanyards that folks literally have around their neck at all time is a
way to have folks ensure they have access to the information during an emergency.
00;10;15;03 - 00;10;36;24
We all know, all of us that have been involved in emergency response in one man or
another know that during the midst of a crisis, folks will not remember key steps unless
they've been trained and the information is readily available. Nate, what steps would
you suggest for other hospitals looking to implement a similar Code Dark?
00;10;37;17 - 00;11;06;12
We've been making this information as widely available as we can. Certainly we are
happy to share, I think, you know, through American Hospital Association, through the
HISAC, CHIME and other collaborators we work with. As a cybersecurity industry, we've
gotten very good, I think at sharing information, kind of what I think of as laterally right
across hospitals, across organizations, and that's essential.
00;11;06;12 - 00;11;27;09
So we've learned from that activity that we also need to think about sharing vertically,
right? So that's how we got into this position of saying, okay, we need to bring the entire
staff of the hospital along with us. We need to think not just in terms of, you know,
information security, information technology and then the, you know, providers, but also,
you know, who's working in the kitchens and who's guarding the doors?
00;11;27;10 - 00;11;47;13
And what does it look like when we are in one of these downtime scenarios and how do
we reduce the blast radius? But as far as sharing this information broadly across the
community, we're happy to do it. We have been doing some of it with your health and
with others, and we're more than happy to do that in any form that we can find.
00;11;48;05 - 00;12;20;21
Nate, thanks so much. We certainly appreciate sharing your knowledge and best
practices out to the larger community. Unfortunately, as we've seen there, and when
one hospital or health system is attacked, there is in fact an effect, a blast radius
regionally. And sometimes even statewide. So truly a collaborative amongst information
security personnel and really all the leadership of hospitals to share best practices and
lessons learned from some of these high impact ransomware attacks.
00;12;20;28 - 00;12;43;12
I say this all the time, truly mean it because we've seen the effect that to defend one is
to defend all. Nate, truly appreciate that. And if an organization would like to contact you
directly, could you give us your email or an email that perhaps folks could write to you
and get some more information?
00;12;44;04 - 00;13;09;13
Yeah, they should absolutely feel free to reach out to me directly. You can probably put
it in the show notes. My email address is just my first initial last name at Children's
National.org. And you know, I would say to the question that you asked before, I guess
I'd add one more point, which is I think that the internal collaboration, especially with
your emergency management team, something that you preach a lot, John, is really
00;13;09;13 - 00;13;34;04
If you're trying to do this in a vacuum, again, whether it's vertically or horizontally, you're
not going to be successful. So for us, the key was how do we build it? Really, our entire
cyber incident response program into the broader emergency operations plan of the
hospital. How do we leverage things like our administrator on call structure and our
emergency communication structures?
00;13;34;19 - 00;13;48;11
I think that's an essential piece. And if you're starting out, if you haven't, you know, put
one of these programs in place yet, if you don't have a Code Dark, but you're interested
in doing it, I would start there. I'd start with a collaboration with your emergency
00;13;49;06 - 00;14;08;00
Nate, I certainly appreciate you sharing your knowledge and the details of this best
practice that you all developed here at Children's National. But there are those who are
concerned with the possibility of increased risk due to the public sharing of best
practices and lessons learned. How do you respond to those concerns?
00;14;08;25 - 00;14;33;21
Well, I think it's a really good question, right? So out of the gate, we tend to think about
cybersecurity information security as something that we try to keep it sensitive. So we
try to keep it secret. And certainly that is an essential function of information security.
The reality is our attackers are very good at sharing information and we have to be very,
very good at sharing best practices by sharing a best practice.
00;14;33;22 - 00;15;02;08
Are we certainly do, I think, elevate the practice of all of our partners and our entire
industry in health care. And I would say that those who are concerned that perhaps we
at Children's National or anyone who gets this kind of exposure or is out front offering
suggestions makes us a bigger target. I would only argue that the attackers know who
00;15;02;09 - 00;15;33;21
We're not hiding under the covers, right? We are not successfully obscuring our identity
by keeping our experience and cases where we have something we have developed by
keeping it secret. I think we're only serving to harm the rest of the industry by not
sharing, by not collaborating. So at the end of the day, I would say, you know, if
anything, being out front in sharing best practices makes us individually and as a
community less of a target.
00;15;33;21 - 00;15;50;08
Attackers that are running a business and they will look for the softest targets they can
find. So I would encourage everybody who listens to this podcast to participate in the
dialog, to share feedback. If you see something that you think we can improve, if you
take a look at our Code Dark and you have feedback, we'd love to hear it.
00;15;50;19 - 00;15;57;05
If you want to implement it and have different ways of doing it that you think might be
better, that's fantastic. Please share it with the rest of the community.
00;15;58;07 - 00;16;27;27
Great advice, Nate. And it really points out that Code Dark is not a system or program
that stands alone. It's really a byproduct of that productive interaction with your
emergency management team. I could not agree with you more, Nate, that often that
cyber incident response plans are developed in a silo without sufficient coordination with
the emergency management team.
00;16;27;27 - 00;16;58;12
As I did in the government, I often find there are pockets of expertise and knowledge
within an organization that's often siloed, and we don't have to spend a lot of money,
often for outside consultants or high tech solutions. It's simply a matter of, as you said,
coordinating internally, leveraging the existing expertise and programs. Really it
becomes a force multiplier. Yeah, to limit that ransomware blast radius that you and I
often speak about internally and externally.
00;16;59;01 - 00;17;23;29
Thanks again, Nate, for sharing your knowledge and your innovative program on Code
Dark, no doubt will help defend your networks and help to defend the networks of health
systems across the United States. And especially thank you to all the health care
providers at Children's National and across the United States for everything you've done
to care for our patients in our communities during the pandemic.
00;17;23;29 - 00;17;44;07
And now this has been John Riggi. You're national advisor for Cybersecurity and Risk.
Stay safe, everyone.