[01:19] What’s Going on in OT Security?

Rachael Lyon: We have Rob Lee, the founder and CEO at Dragos. It’s an industrial cybersecurity company on a mission to safeguard civilization joining us today. Welcome to the podcast, Rob. 

First, congratulations on the 200 million funding, that was fantastic, and with folks like BlackRock. You had some really interesting observations that came out of that. 

Robert: There's a lot of folks in the industry today, whether it's policymakers, CEOs, or practitioners, they’re asking why now? What's going on in OT security? It seems like all of a sudden, it just blew up in the discussion. Singapore has their OT national plan, Australia has their mandatory reporting laws. ICS has a specific national security memorandum at the beginning of this year. There's just a lot happening. 

Eric: Do you think that Colonial Pipeline accelerated that? 

Robert: It didn't hurt the discussion for sure, but SolarWinds was more impactful. A lot of the reporting publicly on SolarWinds was about compromises on the IT side of the house. But there's nothing actually all that different about SolarWinds compared to things we've seen before. If you think about it, there's a lot of hoopla made about it in the media. But it was like, "Nah, okay, espionage." 

But what wasn't covered, and this is the piece that made everybody uncomfortable, is that there were a lot of compromises of the OEMs. The original equipment manufacturers themselves and service providers led to SolarWinds in the OT networks getting directly accessed and compromised at a control layer. 

 

Is it IT or OT Security?

Robert: Sometimes when we talk about compromising critical infrastructure, you have to ask, "Is it IT, is it OT?" But even in OT, it is like, "Okay, was it like the data in-store and did you actually have control?" What happened in SolarWinds is the adversary had remote direct control over critical portions of critical infrastructure. That terrified everybody because, for years, we've taken this sort of segment firewall, prevention-only approach to OT. 

Even when we knew the software version, the forensics, and we knew everything to look for, we still couldn't in most OT networks. That lack of detection response capability from a policy level is what really scares folks. But writ large, the difference of what's happening lately is not any one event. The main thing is, back in the late '90s, early 2000s when the United States started talking about critical infrastructure and cyber protection. No CEO or board was dismissive of that. 

They said, "National security, military, president, everything good, let's go reduce cyber risk." From a board level, it was, "Go reduce cyber risk." It wasn't IT or OT, it was just reduced cyber risk. But at the time related to culture, connectivity, state of the industry, everything went to the enterprise and the OT strategy was segmented off. 

While these companies go through digital transformation, industry 4.0, as they go through connectivity and transformation, we're seeing that that strategy was not effective or not good for where we're going. Boards, presidents, and congressmen, and everybody else were waking up to the fact of, "Wait, what do you mean the side of our business that isn't the critical infrastructure is a side that we spend all the money on?" 

 

A Pendulum Swing

Robert: What we're seeing is a pendulum swing back of a CEO and a board not knowing they needed to specify the difference between IT and OT. Now there's a, "Oh, I guess we have to specify. Please go reduce the risk on the side of the house that generates revenue. The environmental impact and safety impacts, please go put focus on that." That's what we're seeing. 

Eric: Why do you think that was missed for so long? The board, the CEO expected, "We're a critical manufacturer or whatever, we have equipment that has to work, we have something, Colonial Pipeline, we got a pump gas." 

Robert: There's a lot of things that went into it. Not to overgeneralize, but there are two big things that hurt. That time, early 2000s, late '90s when people really started focusing on this probably for the first time in mass, the industry was in a different place. I don't think it was anything malicious. The IT staff of those companies went to their operation staff and said, "Hey, we're supposed to protect you, what do you want to do?" 

Eric: What do you need? 

Robert: The operation staff wasn't being malicious either. It was just like, "We're not connected, I don't know what you're talking about. We don't use the internet, we don't access other systems." 

Eric: These are my systems, we got them. 

Robert: "Yes, just go do your stuff. We get it, you're resource constraint, we're resource constraint, go do your thing." That turned into a narrative that carried on for 20 years. But there was a story that probably only lasted 10. For the last 10 to 15 years for most industries, that hasn’t been true. 

 

The Second Point in OT Security

Robert: It hasn't caught up with people yet. That leads to the second point, which is at a board level. I'm not trying to put anybody down here by any means. The CISO reporting to the board and talking to the board is a new phenomenon in security. That's only the past couple of years in mass. A lot of those CSOs, in my opinion, don't know how to talk to the board. 

A lot of the conversations they have is, "Look at our net cyber security framework. Look at our scoring, look at our FICO-like score. Here's how many scans we saw, it's like numbers, and metrics, and trying to chase financials." All of the finance people show metrics, we got to show metrics. At a CEO and board level, they look at that and go, "Oh, well, we're good." 

Eric: We got a lot of data. They seem like they had a handle on it. 

Robert: But they don't stop and ask, "Wait, is that all of our business or is that just the enterprise?" I've been in a lot of these board meetings lately where that question gets asked. The CSO is like, "Well, no, that's just the enterprise." Like, "Wait, what? You mean all of this stuff we've been seeing was about the noncritical portion of our company?" It's this awakening moment happening at these boards. 

Eric: Who owns the OT security side of a given business normally? 

Robert: Most people haven't defined the problem. There is an expectation at the board level for most of these companies that the security owner owns the risk. That's a bad expectation. 

Eric: Can you give us an example? 

 

[07:51] OT Security Ransomware Case

Robert: Whether it's the CSO, CIO, or CISO, sometimes the CRO. 

Eric: Somebody on the traditional IT side, the same person who owns the risk for email. 

Robert: Whoever that is, you got it. But that's a bad expectation even when it comes to enterprise risk. The CSO or CIO does not own the risk, they're the advisor of what risk they see. Then the CEO and the board own the risk and go resource across the business to implement. You're always going to have the VP of operations, or some plant manager, or so forth. They’re supposed to own the risk of their assets. 

But you've got to be able to create the understanding at a board level of what risks do we accept? Which one do we want to be prepared for? A lot of my conversations there are around scenarios like screwed the metrics. What does an OT-specific ransomware case look like? What does a safety system compromise look like, the scenario they want to go through and the controls that the security advisors in collaboration with operations would say are important? 

Then it's a binary discussion, do you want to invest in that or not? It's not, "Well, I can nickel and dime this piece," or, "I want to have a 35 versus a 36." It's, "Do you want to be covered across the prevention detection response, against that scenario, yes or no? If so, go invest in it." The answers are going to be, there's collaboration. But the risk owners are the operations side of the house and you'll never get out of that. That's appropriate. 

 

Typical OT Security Operator

Robert: But they have to be partnered with a CSO or similar and that CSO or similar has to understand the difference. If you're just copying and pasting your IT governance strategy or IT security controls in the plant, you're going to cause more damage than Russia. 

Eric: You mentioned prevention, detection, and response. Prevention, stop it from happening. Detection, if it does happen for whatever reason, know it. Then response, when it happens, how quickly can you do something about that? Do you find that the typical OT security operator thinks in that mindset? To me, that's more of an IT security mindset. 

We've been talking about that in some form. Pick your company, protect, detect, correct, whatever you want to call it, for a while. Do OT security operators think about it that way or do they just think, "I got to keep the machinery moving?" 

Robert: In a security construct, they do not. But I would actually argue that most IT security people don't articulate that either. They may be able to say it, but if you look at where their investments go, it is a massive prevention bias across the community. They're not idiots, it's what they've been told to do. If you look at IEC 62443 as a standard, if you look at NIST cybersecurity framework, if you look at NERC CIP, or whatever framework you want, CMMC, we did the analysis. 

We found that anywhere between 75% and 95% of all of the controls are prevention based. We've told the community forever, do prevention. Then they divide up anywhere from 25% to 5% across detection, response, and recovery. We wonder, why aren't we resilient? It's literally, you gave them advice to only worry about prevention. 

 

Prevention Bias

Robert: My point is, IT security largely has a prevention bias. I don't think the operation staff has been talked to like adults on that. They've been told, "Well, you need to patch this system, install antivirus, and update your passwords." That's three preventive controls. They're going to focus on those things. 

Eric: And multifactor authentication, then just throw in that zero trust, it's all prevention. You have to have a role for prevention. But when you look at CISA these days, they're spending all their time there. Now, they are rolling out logging and EDR, which is more on the hunting side, the detection. 

Robert: But even then, it's like you need to implement MFA in this way, then you need to do monitoring. What would you like us to monitor, for what, and what use cases? It’s very high level. Go back really quick to the OT security thing, and I'll pivot on the CISA one. The beautiful thing is that the idea of detection and response is very core to operations if you break out at the security discussion. 

The ability to have a temperature alarm to be able to detect when something is going wrong and the ability to respond to it, it's there. I love the safety culture. When you look at HAZOPs' process where they go in and try to assess a plant to determine, "Is it unsafe? If it's unsafe, what would we do about it? Do we redesign portions of it? Or do you put some detections and some safety systems that are correct? What is the process?" 

 

From a Scenario Approach

Robert: They think about it from a scenario approach, they don't say, "What's our fishing protection?" The same way they don't say, "What's our temperature protection?" They say, "How would I think through on safe events in the end and how would I deal with that? If you actually tie into safety from that level of the communication approach, it goes very well. 

But to your point on CISA, it's a very preventative focus, but I will tell you there's something I like about it. It is the simplicity of here's the thing we want you to do. 

Eric: You have to do it. 

Robert: I don't agree with all the things that come out like TSAs, security regulations on pipelines. We're atrocious. If you follow those regulations, verbatim, you are shutting down pipelines, I promise you, they're not good. 

Eric: It's like a zero-risk approach?

Robert: Well, they had everything from adopting solar to here is exactly how to implement a patch on this type of system. It was all over the place and it’s very prescriptive. You can't do a lot of things, there's a lot of IT security controls issues. If there's a flow meter off of a pipeline that literally you can't do anything to impact operations with it, why are we obsessing about it? 

Or, "Here's a vulnerability and a compressor station out in the middle of nowhere that has never been exploited." The vulnerability doesn't even add any new risk to the system, now you've got to patch it in 35 days. Why are we pushing that? It's the prescriptive nature that I don't like about it. But if you look at what CISA is doing, I like that they're saying, "You know what, please go to the MFA". 

 

[13:27] Ten Different Government Agencies

Robert: Sure, they'll recommend other things. But if you watch Jen Easterly on Twitter, you watch Eric Gold telling these folks, "It's MFA." I love that they're saying, "You got to do this thing." A lot of times, infrastructure owners get FBI, DoD, DoE, the base commander, TSA. They have so many people come and say, "Do these two or three things." 

At the end of it, you're getting asked by 10 different government agencies to do two or three different competing things. You're just analysis paralysis at the end. For the whole government to say, "Have an instant response plan and tabletop exercise on ransomware. Please implement MFA." Great, let's go knock that one out and then next year talk about another one. 

Eric: They just came out with the top 200 or 300 vulnerabilities, and got a patch within a couple of weeks. 

Robert: I'm not saying everything's great. 

Eric: But to your point, that's very prescriptive, at least. "Hey, go patch this." God only knows how many vulnerabilities are out there, but they're saying, "Go do this." With MFA, I heard Jen also speaking, quoting her, I don't know where the data came from. If you're using MFA, you avoid 99% of the likelihood of being compromised essentially, your credentials being stolen. 

Robert: There's some data getting pulled from ASD that they've done over the years in Australia, the four critical controls. Verifiably, where you can implement MFA, it should be one of the universal controls. But patching isn't the same to the point. When our team looks at the vulnerabilities, we're a technology company but we have an Intel team as well. 

 

Ability to Impact Operations

Robert: When the Intel team digs in and looks at it, we found that only about 7% of ICS vulnerabilities a year are worth a damn at all. Why are we running over to operations who are already constrained, already dealing with a lot of stuff, complaining about these 93%? "Well, it's a CVSS 9.3." "Yes, but does it actually have any ability to impact operations at all?" "Well, no." 

"Then move on to something else." Let's spend the resources on a risk-based approach and get some value there. To your point on the vulnerability set, it was, "Go patch these." There's nine different things you could do about a vulnerability, patching is just one option. Why are we being prescriptive about patching? 

Eric: They don't even know where all the assets are. 10 plus years into CDM on .gov, we still don't have a baseline on where all the systems and capabilities are. 

Robert: That's where I see people talking a lot about ESPON and for whatever's good for enterprise. I'm not an enterprise security person, so I'm not going to critique it one or the other. But on the operations side, a lot of the ransomware cases we get called into, why? Not anything crazy. It's just they literally didn't know what was in their network. They had no visibility and asset identification. 

There was prevention of atrophy over the years. Something they didn't realize they had got compromised, and boom, they got ransomware. I'm not concerned about ESPON, I'm concerned that nobody knows what's on their network. How can you not have that and then jump to, "Well, what's in the device?" You gotta take it down a couple of steps first. 

 

The Difference Between Enterprise Security and OT Security

Robert: At times, we talk to the community. Whether it's the government, or practitioners, or vendors, here's what you should be doing. But what level of maturity they're at in their journey, it's very based on each individual company. 

Eric: Even within companies, individual groups and organizations. 

Robert: That's the other issue. Look at the difference between enterprise security and OT security. With enterprise security, here's my enterprise security project, and a Gantt chart over four years with 30 controls. It's like, "For every one IT network the oil company has, they have 500 IT OT networks." You can't take this one enterprise project to roll out to the OT side of the house. It's like, that's a lot of different plants, refineries, and things. You can't treat it that way. 

Eric: What should they do? How should they handle it then? What's the multifactor authentication of OT, if there is such a thing? 

Robert: If it's a remote connection, wherever possible, I would still roll out MFA. If you look at the problem from two lenses, one being Intel-driven, what have the threats actually done? I don't care about your new research and your cool things. You're going to show up black hat, just ground reality of what's actually happened, the consequence of whether or not it's actually happened. We know it could and it would have a big consequence of life safety. 

You take it from an Intel and a consequence perspective. If you look at the scenarios we can come up with in any given company, you'd probably be talking about five or six scenarios. Look across those five or six scenarios and think about what controls were the most impactful. 

 

One Giant OT Security

Robert: You will generally come to about five. You'll come to the fact that you need a defensible architecture. If you've got one giant IT, OT security, merge network, your instance is going to suck. If you don't have span ports on a switch, you're not even going to monitor anything anyway. So you first come to defensible architectures. 

The second thing you'll come to is visibility and monitoring. Can I actually get network traffic analysis, east-west traffic, understand what's in my network, detect vulnerabilities, detect threats? You can get a lot of value out of just the monitoring category. 

Then you move into having MFA on remote connections where you can, and if you can't, then you have compensating controls back and defensible architectures for it, jump host or whatever. Then you get into a key vulnerability management program. Don't care about the 93%. Those 7% that have an internet-facing data historian that's got remote code execution on it, I better fix that one. 

Then the last one is an ICS-specific instant response plan that you have tabletop exercises. Those five controls would put you into a world-class OT security program. Then you go talk about all the other things like, "What about application whitelisting?" You go talk about all those things later but do not pass go until you get those five. 

However, this is the interesting piece on industry that goes to your question, I don't want those five everywhere. It sounds bad to say that, but we've got some assets like a wellhead that might be generating $1,000 a month. Why am I rolling out five security controls on that? 

 

CEO Level Boarding Conversation

Robert: For the next thing that needs to happen, it really is an executive level. You could have a board-level conversation, but it's at least a CEO-level boarding conversation. What are our high assets, our medium-level assets, and our low-level assets in terms of criticality? 

Factor in revenue, health and safety data, and environmental data. Make the master list one to 1,000 or whatever it is in terms of physical assets like plants, figure out what that is. You might be a power company that's got some distribution substations that are more critical than certain transition substations. Figure out what that list is, and then say, "Cool, we know security is these five things," as an example. 

Then I want to know the top 25, 30. Let's have the cut line of the top critical assets that are high are taken care of now. Don't tell me at the end of the five-year journey that my lowest asset is protected at the same level as my highest assets when it took five years to get there. Go roll out good security at the high assets. Then we might find, based on revenue, or cost, or whatever else, we don't want to do all five at the mediums. Maybe you want to do three of those five.  

You know what, in the lows, maybe it's one or two. Maybe we have a defensible architecture and instant response plan. But we're not going to put in a bunch of monitoring and everything else, sites that aren't that critical. The security team should be making sure that they're aware of the new scenarios that come over the years. 

 

[21:48]The Way You’re Doing OT Security

Robert: Then determine, "Do we need to add anything? Do we need to adjust based on it? Are we getting value out of the things we've already spent on?" The executives need to determine, "We were okay with 25% before. But based on where the company is and transformation, now we want to extend it down to 35%. How do we move some of those medium-class assets into high-class assets?" It's not easy, but you can follow that simple framework. We have found it has been an immeasurable change in these companies. 

Eric: You're talking about standard risk management, it's just not being applied. 

Robert: It is standard risk management based on scenarios and understanding the most common thing that happens in instant response engagements for us. "You've got 200 plants, where would you like us to start triaging?" "Well, I don't know." "Which ones are you high?" "The finance group, it's this, the risk group, it's this." They haven't even thought through what is important to the business to be able to have those conversations. 

And they haven't rehearsed on what questions is your CRO, legal, everybody else is going to need answers to make sure the collection and the data that you have in those plants can facilitate those questions. It isn’t like this is unheard of or completely novel in an approach. It's just not getting done. The way you're going to do an OT security is different from what you do on your enterprise approach. There are different stakeholders, different people involved, different threats, and different risks. 

Eric: What percent of operators do you think do this well? 

Rachael: 20%. 

Eric Trexler: I was going to go with 3%. 

 

Live In a Secure World

Rachael: I was speaking aspirationally. My first thought was five. 

Robert: You want to live in a secure world, that's okay. 5% is being an optimist. 

Eric: What is the number based on your knowledge and experience? 

Robert: In places that we have visibility, I don't really know what's going on in Chinese infrastructure these days. But I would say 5% is a very generous answer. That's the answer I normally give to folks, but it matters what industry we're talking about. The electric power industry in North America is moving at a more mature, click rate, speed than the food and beverage industry in Saudi Arabia. There's a geo and industry vertical aspect to these things.  

The defining feature to me, of when an industry is doing well, is two things. One, there is alignment at an executive level across pure companies. You can try to solve for risks in your company all day long. But if you look at the electric sector as an example, they have the electric sector sub coordinating council, CEO-led group. 80% of the CEOs in the country are all part of it. They all get together and talk about this stuff, and there's an alignment. 

The second thing is, right, wrong, and different. I don't care what anyone wants to say about the government. Your government partners for critical infrastructure are really important. When you have an alignment that the government is engaging industry in an open and transparent discussion about the risks that it sees, why do we see these risks? What kind of outcomes do we want to see? 

 

Really Good Movements

Robert: But they stay out of the “how”. They leave the how-to an industry that is collaborating together on what it means to actually accomplish those goals. We see a lot of really good movements and you would get above the 5%. If we look at the electric power space in the United States, by numbers only, it's not 5%. But by regional footprint, it's well above 5%. 

What I mean by that is, a Southern company is doing a lot more than 100 different small co-ops in terms of the meters it's serving. It doesn't mean the co-op's not important, but just footprint-wise, it's well above 5% by numbers, well below 5%. But you start talking again about what's going on in the pharmaceutical industry. If it's not FDA related, it's not happening. So, I'd say 5% is extraordinarily generous. 

Eric: If you think about electrical generation, we need power to do things. If somebody attacked, that country would respond in some way, I would hope. 

Robert: I don't think so. This dream that the military's going to come in swinging is a fantasy. 

Eric: I'm looking more like Colonial Pipeline. Nobody expected the president of the United States to get involved when they launched a ransomware attack against Colonial Pipeline. The government stepped in there. 

Robert: There's a lot of your critical infrastructure community members that have been told that, not just like Rob Lee says no, military commanders and staff have come over. I worked with one, one-spaced, space company. The CEO was convinced if there was ever a cyber attack on their space-spaced assets and the control systems, that the US government was going to respond. 

 

Who Is Going to Respond?

Eric: Who did you think was going to respond? 

Robert: I was like, "Where are you getting that? What do you mean?" They were like, "No, the military, they'll go to war over it." I'm like, "Said who?" "Oh, yes, the general of whatever this combatant commands." It wasn't SpaceCom at the time. "But one of the combatant commands NORTHCOM or something, he told me over dinner." You do know generals don't declare war. There's no power there. 

Eric: They don't even really operate in the states. 

Robert: It feels like Lord of the Rings, you have no power here. The military does not get to decide where or who it engages. I was like, "You're wrong." He's like, "No, general so and so has sworn to me." I'm like, "That's not happening." But there are a lot of infrastructure owners who believe that there's this backstop that doesn't exist. 

Eric: Right, but if we lose power, that's a problem. We're talking about lives in many cases. 

Robert: It's a problem and I worry about the psychological impacts sometimes more than the real impact. Maybe we lose the distribution grid in a small portion of DC. That is enough to swing elections, in terms of how people are scared about it, or forget the loss of life. 

Eric: Colonial Pipeline, guys are filling Ziploc or plastic bags with fuel. Some rationality exists. 

Robert: The images of that absolutely. 

Eric: Pick a pharmaceutical company, we know they're targeted by China. China has a massive problem. Just pick on something like cancer, they've got the biggest cancer problem in the world. 

 

Red Dawn

Eric: They have a motive to steal intellectual property. What I don't understand is, when a pharmaceutical company doesn't protect their intellectual property or their systems well. You know the government's not doing much of anything there when your IP is stolen. It's just gone. That is your future. 

That's more tangible for a pharma organization to understand, than a regional electric co-op. They saw Red Dawn. They're just going to the 2021 modern version. The Russians aren't para trooping in anymore. They're coming in via cyber, but the government will be there to protect me. When you're talking about IP, most businesses have a better understanding of the risk of IP loss. But what you're saying is, not really. 

Robert: Somewhere in their organization, they do. I bet you, there's a CFO, or a VP of operations, or somebody that could articulate that. 

Eric: It doesn't translate to the actual risk. We need to do something to protect that IP because that's not getting translated out. It's in the 10-K as a risk to the business. 

Robert: I was in a manufacturing company where we just went through this. The CFO is chiefly aware that the bottom line is a billion dollars, minimum estimate is a billion. First of all, I don't think there was alignment at the executive level on that. But on the security piece of it, "Here's what we're doing on our enterprise security programs. Look at all these, we need to talk about these, our insider threat program." 

I again stopped them like, "You know the intellectual property is in the manufacturing side. Everything you're talking about is enterprise." They're like, "Oh." None of those things are going to help. 

 

[29:22] The OT Security Folks

Robert: I'm not saying it's not going to help, but it's not the problem. But then there was a complete misalignment on what was intellectual property. The security folks started thinking, "Oh, there's a recipe that could be stolen." I'm like, "I guarantee you not." They're like, "No, it's the ingredient list." I'm like, "I don't think it is, you should talk to your plant folks." We brought in the plant folks. They're like, "What, the ingredient list? No, it's the way we produce it. It's the manufacturing line itself.” 

Eric: It's the methods. 

Robert: It's the efficiency, it's the methods. They're like, "Oh, we're not looking at that at all." I'm like, "Yes." So, I would say, you're right. That intellectual property theft is very tangible, but it doesn't mean it translates into IT versus OT. It doesn't mean it translates into what is intellectual property. How would we protect it? Again, I'm not trying to put the security industry down by any means. There are wonderful folks here. But what does every security person do going into a company? 

"Oh, well, here's our top critical controls on patching and this, they just apply it wholemeal anyway. It's like, that's not necessarily tailored at all to what we're trying to solve. If I'm a security person, I go into any company in the world. Day one, I started asking about our vulnerability and management program. Are we encrypting our data at transit and rest? What are we doing on EDR? I would sound like a pro. Those three things I just mentioned have zero to do in its entirety with OT security discussion. 

 

A Hard Nuance to Get Off

Robert: This association though, we must be careful here. Just because that's what it is doesn’t mean anybody has done wrong. That is a really hard nuance to get off. Go back to where we were, when we made the decisions, how they propagate the standards, the frameworks, the guide. You look at our infrastructure owner-operator community, we're rocking. 

We need to change because things have changed. We’ve got to do better because we now understand more.  

But it's always like these 5% numbers, I get scared to throw them out in front of Congress. Every now and then, I'll go testify, and they'll ask that question. I'm like, "Oh, I don't want to say this because some poor CEO of a power company is going to get it. I can't believe you're not protecting nationals." But they're following every piece of guidance they've been given. I always try to throw in there that our community is awesome. We've got to change because our world has changed. 

Eric: We're focused on the wrong things. 

Rachael: It's interesting you brought up the testimony. This is always a fascinating topic when it comes to Congress because there's an education component. It’s not easy to understand. When you're testifying in July, you testified before the house of representatives. How do you balance that to make them truly understand the landscape and the level of threat and the work that needs to be done? 

Robert: You're right, Congressmen, and women, and others have farm aid, and everything else they have to deal with. Cyber is one component of it, then there's IT versus OT something like, "Well, oh gosh." 

 

Talking Points

Robert: First, there's a reason they want you to testify. This one was ransomware. The first half of your testimony has got to be grounded in. You came to this meeting expecting this, here's what I'm delivering on that topic. Then you can introduce, if you're lucky, two, maybe three more talking points. 

My talking points were really around trying to help them understand that we didn't have a good understanding of the problem. I don't want to get into the specifics, or get in front like, "Here's a model that works in this standard. You need to do these five controls." 

Number one for them is understanding. One of the things I want to position is that we don't have full alignment in understanding the risk anyway, period. I love our federal partners, I came from the government. It cuts me deep enough, red, white, and blue. That being said, vendors get in front of Congress and try to be objective. 

They know if I'm a vendor, I can't get there and front a pitch. Nobody wants to pitch Congress, you need to be objective. When government agencies get in front of Congress, that is their funding source. I'm not trying to put them down, but all they do is pitch, that's it. Like, "We're experts, here's what we know. I need extra budget in the CISA. 

Eric: It's like private equity. 

Robert: EPA, they're going for a venture round and it's not malicious. They've been asked to do that. They're not bad people for doing it, but it's very much. I'm here from the government, the government can help. If you give me resources, I can help more. 

 

OT Security Expertise

Robert: One of the things I try to dispel is this idea that anybody has got a monopoly on the problem. One of the talking points I've taken to Congress a number of times is, you do know, through investments with the government, the expertise on OT cyber security is not in the government. They're like, "No, look at all these experts. Like, "No, those are your IT security experts." Or these or that. OT security expertise is at the infrastructure and owner and operator level. They're the ones doing the mission. 

Just that one change, changes a whole lot about how you're going to ask questions, and who you're going to ask questions to. It changes it from, "These i**, let's go regulate them," to, "Oh, maybe we need to ask them what they're doing." That one thing is like a strategic shift that's important. 

The second talking point that I took to them was the government and its roles and responsibilities. It has some, which are poorly defined but they exist. Their role and responsibility, in my opinion, is to define the why and the what. Why do we care about this thing in the first place? What's the risk that we see? And what would we like the outcome to be? 

I want to increase the ability to detect and respond in our current overture, whatever it is. But define the why, so there's alignment on the risk. Then from a national perspective, help me understand the what, in the context of national security. I shouldn't be telling a CEO how to manage business risk. But, power company and pharma company, whatever you have business risk, do whatever you want. You have national security risk. 

 

National Security Risk

Robert: On the national security risk, here's the why, here's the what, but leave the how to the infrastructure owners and operators. Don't tell the pipeline industry how to implement security. There's not going to be one answer that benefits all the pipeline industry, let alone across all of the infrastructure. My two talking points there largely were, go understand that you've got partners in the private sector. 

They have more expertise on this and they have a seat at the table, you got to consider them. The government's role and responsibility is not a flyaway instant response kit. We've got plenty of firms that do that but, really alignment around the why and the what, and leaving the how to the private sector. When you get in front of Congress, it's that level of messaging, at least in my experience, that works really well for them. 

Eric: I'm thinking about it from an IT perspective, which is where a lot of my background is. When you start to get to the state and local level, there's less capability. There are fewer resources, less control, less understanding of the problem. You don't have the budgets to hire, you don't have the budgets for the system, you don't understand. What you're saying is, with OT security, you almost flip it. 

The strength is at the edge, it's with the companies that are sitting. If you're the congressperson from Oklahoma, national cyber security, you probably feel a million miles away from being able to control it. But if you have power generation or something in your district, the strength to protect is right there is what you're saying. It's an entirely different argument. 

 

[36:52] The Salt River Project

Robert: One of the greatest companies I've ever worked with is the Salt River Project. They're not a huge company, they're public power and public water. But that team knows everything about what matters to the operations side of the house. They're service-oriented in what they do. They don't think about security like, "This is a security risk, let's do it." So, they get it. If I force that update, there's nobody at the field to go and reset it if something goes wrong and it's going to call somebody in on another 12-hour shift. 

"We better not do that one, here's what we can do differently." Which actually takes against it and they think through that stuff that you're not going to think through it from DC. The operator sitting at that SCADA system, even non-security, knows what's physically possible on that system or not, in a way that policy isn't going to get there. 

The expertise is at the edge, but it doesn't mean that we don't want to influence it whenever else. You talk about state and local level, it's always interesting to me. I love CISA. But one of the places that I feel so bad for CISA is anything that goes wrong at all, ever, full stop, in terms of cybersecurity, Jen Easterly, or Chris Krebs, "What are you doing about this?" 

Eric: Even when it goes well with the election, you still can get tweeted and fired. 

Robert: They're not resourced to do the cybersecurity mission, no one is. What lane of it are they going to take? It comes off like I'm critiquing CISA, but I actually just feel bad for them. I'll see them in front of Congress. 

 

A Flyaway Team

Robert: It's like, "Why don't you have a flyaway team to go help Chevron or whoever if they have an incident? It's like, "What?" Because they can depend on Dragos, Crowdstrike, Mandiant, anybody. Throw a dart, pick one, they got everybody. Why are we asking them to go do that? But are there state and local infrastructure problems? Are there public water companies that have nothing? And are there election issues that CISA could help those out with?" 

I want to see CISA going and helping out the Texas Public Utility Commission and the state and local level. Long before they're going and talking to Southern Company, a Fortune 1000, about what we can do for you. That's a better investment of resources. On that topic too, this is the piece that gets slightly frustrating. Cyber security is not the number one problem. I love our cyber security discussions. 

The electric power industry, at the beginning of this year, there was a 100-day action plan that kicked off from the White House. It increases the ability to do monitoring in our infrastructure. Beautiful, that's where the electric power industry was. It was a smart move, an amazing move by the Biden administration. Nunberg and her folks, well done, worked out really well. 

Now, they're talking about going to the water industry and doing that. They're like, "Oh, you can't do that." I'm like, "Well, what do you mean? I don't understand electricity, this was really successful." "Because the electric industry had the infrastructure built over the years to be there. But there are over 50,000 water and wastewater systems in the United States. You hear about Oldsmar, most of those facilities don't have IT staff, forget security. 

 

Shared IT Resource

Robert: How am I going to take somebody who's got a shared IT resource between four companies and tell them that they need a rollout of advanced cybersecurity monitoring? "That's not where they are." "I wish they were. We got to talk about how do you go from here to there? You can't just go out and say, "Roll out MFA" 

They're going to be like, "What's MFA now? "Well, MFA, roll it out." "Hold on, I'm the one guy that's got to get access to four different regional plants. By EPA regulations, if a pump fails and I'm not there within 60 minutes, it's a reportable incident. I can't physically drive to all of the sites, I have to have TeamViewer to get access to it." "Oh no, you got to use this." 

S** that. The budget for that program is three times the amount of revenue that we generate at this water facility in a year. There's a whole economics thing long before this security discussion at some of these infrastructure players. 

Eric: In some of the most remote parts of the country, good people, don't get me wrong, but, try explaining something like ransomware to them, in many cases. 

Robert: We've got an electric utility, and I hope they don't mind me calling him out. Cordova, Alaska provides power to 2000 people or something. Their CEO is their security guy because he's taken it up and he's serious about it. They've implemented phenomenal security especially for that size of infrastructure and they've done extremely well. 

Eric: He has an interest. 

 

Hope for the Clays of the World

Robert: He has an interest and he cares about it, but why aren't we helping there? That's where I go like, "Okay, that's our strategy. It’s to hope for the Clays of the world to get this done." When it's smaller or infrastructure sites, how much your Public Utility Commission will allow a water gas or electric player to resource, says more about their security than any policy framework, regulation, whatever ever will. 

We can't depend on the Clays of the world, we have to figure out what's the resourcing challenge. It's not yelling at people for not taking it seriously, it's understanding what the problem is and what their mission is. Again, your invest-your-own utilities, rock on protecting the world, they're amazing. It's well above 5%. You're 1000 smaller public power sites that don't have the same level of resourcing. 

Let's not yell at them about the cybersecurity discussion, that's where the resourcing is. I'm from Cullman, Alabama. Try convincing the Cullman board, the Public Utility Commission side of the house, that Cullman, Alabama rural cooperative should raise the electric bill by 10 cents per person. It comes on like nothing. But tell them that they should raise it by 10 cents per person because one day Russia might hack into it.  

Eric: Give me that dime back. You say, "Why aren't we helping them?" 

Robert: I'm talking in the federal discussion now.  First, I would be very thoughtful with the resourcing. We like to throw a lot of money at programs. Here's another $100 million to the Department of Energy to come up with some answer. It's always on what's the next-gen thing. We haven't rolled out last gen. It's like, "Let's just do something." 

 

Only Two Hands to Play

Robert: If you only had two hands to play, what are those two hands? One of them goes down to the Public Utilities Commission. The Public Utilities Commission is a good organization that is there because your utilities are a monopoly. If you didn't have a Public Utilities Commission, no matter how much I love our infrastructure players, abuse would happen, it would. When you're in a monopoly, abuse happens. 

To have a Public Utilities Commission that is standing in the way of abuse that happens for your local utilities is a really smart thing. But what is the cyber savviness of that Public Utility Commission? Nonexistent. When a power company gets in front of another amazing power company, Southern California Edison is great too but Sempra is another really good one. 

Sempra got in front of the Public Utility Commission Board, and without getting into anything confidential, just said, "Some of the things you're asking us to do are adverse to security that we've been told at a federal level not to do." The Public Utility Commission ghosted them on it. From the Public Utilities Commission's perspective, it was, "Here's the big power company just trying to do something to screw over people." 

No matter how much you build trust, that comes up every now and then because their role is the regulator. And so, go back to resourcing. I don't need $200 million to come over to DOE, I need CISA to talk to Public Utility Commissions and say, "Here's what right looks like. If your companies come to you asking you for platinum level gold coding on every distribution substation, it's probably not a good investment. 

 

[44:39] The Most Impactful Thing to OT Security

Robert: But if it means this classification quantification, it's these types of initiatives, here's the why and the what, you really ought to pay attention to them. We need the influence there." That one motion would cost the federal government zero. It would probably be the most impactful thing to cyber security if you do it at the utilities' level. What are those pressure points that the why, the what, and the amplification that we see at a national security level, can be influenced across our communities. That's impactful. 

Eric: But they're not sexy like a $2 billion investment program on the latest anti-ransomware toolset. 

Robert: If I can give $200 million to the department of energy to come up with a new AI-based tool, I'm going to get another vote. "Ah, it looks good, and it's sexy." 

Eric: It'll get reported at least. 

Robert: It'll get reported and program managers get excited about it. It's, "Look at what we did and whatever else," and it's not going to help. Not to be political, one way or the other, but I'm not a big fan of using taxpayer money to bankroll new companies. That's not the best use of taxpayer money half the time. There is no lack of venture capital for good ideas on cybersecurity. 

You want to go bankroll a new nuclear reactor because we're changing our green energy portfolio in this country. Nuclear energy is really good for baseload, and there's not a market there, go do that. But with cybersecurity tools, there's more than enough market. You don't need taxpayers to front-load that market. These are some of the things that balance out. 

 

Helping with OT Security Capabilities

Eric: The election in 2020, CISA seemed to take a pretty good approach to it. Obviously, it was a whole government issue. But like critical infrastructure, I feel there was an edge component, they had to work with state and local components. They really didn't have power. Even though everybody thinks the Marines would come in if anybody messed with it, they had to influence, to educate, and to be available in case of an issue. 

A lot of it was really getting the message out, the guidance. It seems to have worked pretty well, at least from everything I've seen. Is that a fundamentally appropriate model? Probably needs some adoption for helping with these OT security capabilities that are distributed to the edge. 

Robert: When Chris Krebs was there and ran that playbook, behind the scenes, talking to some of our members in the Senate, I told them, "Take that. Bottle it, resell the crap out of it. What you just did, that lightning in a bottle, capture, run that play another 100 times." 

What they did was exactly, partner, amplify, "Let's figure out where the state and locals need help." They did a great job and didn't get close to the size and scope of that problem. To me, a lot of the election infrastructure security discussions are restoring confidence. For CISA to be apolitical and to be the cyber security focus of the government. To partner with these players and help amplify that story and go, "You can have confidence in elections." 

 

A Really Important Mission

Robert: That's a really important mission. No vendor is doing that, and they can't. If Dragos shows up at a power company, and I say, "I've got the ball on an instant response. I'm not trying to pick a fight here, but I guarantee you, more people would have confidence in Dragos doing that IR case than CISA. But for Dragos to show up and go, "Hey, election security committee, we can protect you," versus CISA, CISA's going to be a thousand times better to do that. 

The confidence that comes with that is important. To your question on OT, can you do that with EPA? Partner with EPA, and then go reach out to the local water companies and not private-owned companies. American Water, Aqua America, these fantastic privately owned water companies need to do their own security. There’s revenue and generation capacity for it. There are other things they can get from federal partners but it's not, let me show up and do an assessment. 

But to reach down to the Public Utility Commission, work with them closely, and say, "Here's the problem your water companies have. This is what we need to change in the ecosystem to allow them to foster." That is not a let me show up and run a tool in your network. That's let me help change the conversation and that the government can do that. 

Eric: Or even provide you with tools. I could see CISA going out, "Here's a $500 million investment in tools for OT." It’s going nowhere, of course. 

Robert: There's definitely things to explore there, but can they pick it up and use the tools? Do they have the people to process the staffing? 

 

A Perfectly Fine Approach

Robert: There are lots of conversations there. I still like that, if you're going to go down the level of giving resources after we've established the pit pressure points in summer. $500 million is going to run out real fast. But after we've shown that investments can be made, is it okay to get to that level? Sure it is, but I don't want to pick a tool. I don't want to favor a vendor. 

And I don't want, "Local water company, here's our point of view." This is where the government always shies away from picking winners and losers at all. They go, "You can do anything you want, but half of those choices are bad. We're not going to tell you which one." That's s**. But they say, "You gotta be this tall to ride the ride. This is the framework. Look, there's 50 vendors that meet these qualifications. Pick from one of those and we'll resource it." That's a perfectly fine approach. 

Eric: Guidance. 

Robert: We talked about supply chains in this country. We don't have a position at a federal government of being able to pick winners and losers. It's like, "You have no supply chain security, let's stop pretending." When you can identify that there's this Chinese-based company that is actively doing things to subvert the supply chain. We're afraid to come out and say it publicly as a government, it's that we don't have supply chain security. Let's move on. 

Rachael: I lived in New York for 15 years, on the subway. There were all these signs, see something, say something. I love that that's your origin story for Dragos. 

 

Words of Wisdom

Rachael: You're going in front of Congress, having the conversations, and doing the education. That's the really critical element here. People just don't know, they just don't understand. It's so encouraging to see folks like you who are out there, trying to address the issue. Do you have any words of wisdom, for folks who see an issue but they don't know how to get started to try and address it? 

Robert: The best thing we can all do is have some empathy. These infrastructure owners and operators in places, they're just wonderful people. It's easy to get out in front and go, "TSA did something s** in its regulation." But the members of TSA, there were four of them that were tasked with this national-level problem. They did the best that they could with what they had. They're not bad people. The outcome may have been undesirable, but you know what, they're good folks. Let's figure out what the problem was and address that. 

Too often, with InfoSec, there's a lot of vilification that happens. We love the idea that there's a bad guy. There are good people trying to handle really hard problems. A little bit of empathy and an understanding of what the mission is, not what I want to come and pitch. It’s not what's the security control I'm patching about, not where's my bias, but what's the mission we're trying to solve. Let's learn that first and then everything else will fall into place. That's what gets you to a good place. 

Rachael: Thanks everybody for joining this week's podcast. Thank you Rob for joining us today. Amazing insights. I love that we can share them out to the world. Until next time, stay safe and talk to you next week. 

 

About Our Guest

 

Robert M. Lee is a recognized pioneer in the industrial security incident response and threat intelligence community. He gained his start in security as a U.S. Air Force Cyber Warfare Operations Officer tasked to the National Security Agency where he built a first-of-its-kind mission identifying and analyzing national threats to industrial infrastructure. He went on to build the industrial community’s first dedicated monitoring and incident response class at the SANS Institute (ICS515) and the industry recognized cyber threat intelligence course (FOR578).

Forbes named Robert to its 30 under 30 (2016) list as one of the “brightest entrepreneurs, breakout talents, and change agents” in Enterprise Technology. He is a business leader but also technical practitioner. Robert helped lead the investigation into the 2015 cyberattack on Ukraine’s power grid, he and his team at Dragos helped identify and analyze the CRASHOVERRIDE malware that attacked Ukraine’s grid in 2016 and the TRISIS malware deployed against an industrial safety system in the Middle East in 2017.

Robert is routinely sought after for his advice and input into industrial threat detection and response. He has presented at major security conferences such as SANS, BlackHat, DefCon, and RSA and has testified to the Senate’s Energy and National Resources Committee. As a non-resident national security fellow at New America, Robert works to inform policy related to critical infrastructure cyber security and is regularly asked by various governments to brief to national level leaders.