Agencies scrambling to get a grip after SolarWinds hack

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

It’s hard to tell but it’s likely a mad scramble is going on among federal information security officers as the extent of the SolarWinds hacking becomes more visible. For more likely scenarios, Federal Drive with Tom Temin turned to the former National Security Agency chief information security officer, now with Fidelis Cybersecurity, Chris Kubic.

Interview transcript:

Tom Temin: Mr. Kubic, good to have you on.

Chris Kubic: Good morning. And it’s a pleasure to be here with you today.

Tom Temin: And I’m going to ask a question, which you may or may not be able to answer. But you know, a couple of years ago, the NSA lost some of its own hacking tools that it used in investigations and whatever intelligence gathering it does. Is there any sense that this is evidence of those tools being used by people that had them out in the wild?

Chris Kubic: As you already mentioned, that’s not really something that I can comment on today.

Tom Temin: All right, well, you are former NSA, that’s the way it goes. So put yourself back in your old job as a federal agency, cybersecurity operative and what is the reaction do you think is going on? And what do people do first, when they hear about this?

Chris Kubic: Well, you know, the first order of business is to figure out whether, you know, particular department or agencies’ networks have been compromised. So there’ll be a huge effort to really do that discovery and damage assessment phase, to really understand whether or not they are within the scope of this particular attack. And from there, it really determines what their next steps would be.

Tom Temin: I mean, some agencies know they have been hit. But is it possible that even this couple of weeks after we know about this, that they don’t all know? And does it take time to do all the discovery required, given the complexity of the systems today?

Chris Kubic: Yeah, I think what you mentioned is spot on. It’s going to take some time to sort out exactly who’s been compromised. I mean, we know certainly folks that use the SolarWinds software that was part of the supply chain attack would be within scope, and certainly need to be concerned. But you know, we don’t know all the details yet. So there’s potentially other attacks out there, you know, other ways that the attacker gained entry. And as we discover those, those could certainly expand the scope of this. But certainly those people that use SolarWinds need to dive in. But you mentioned another key point is the complexity of networks, networks are interconnected. So adversaries, attackers can certainly hop between networks, they have capabilities, techniques to do that. So just because you didn’t use SolarWinds doesn’t necessarily mean you’re not potentially compromised. So that’s that detailed work that needs to be done to really dive in, do the analysis, understand what systems were compromised by the attacker, and really get your head around the full scope of the attack.

Tom Temin: And how do people know precisely what to look for? Because often, these malware pieces are very small chunks of code. And so they might be hard to find among your stacks and stacks of software that are running all over the place?

Chris Kubic: There’s really kind of two answers to that: First of all, we’ve already learned a lot from what FireEye has publicly posted as far as the different attack techniques used by the attackers. And we certainly know SolarWinds is an attack vector. So certainly looking for evidence that those same attack techniques that we already have signatures for were used, that’s a tip off that the adversaries were in your network. But you know, there’s also advanced forensic techniques that can be used that really dive in and look for just evidence that systems have been corrupted. But you know, it’ll be a challenge for folks, because the attackers use very good tactics, they’re very careful in how they attack the system. So they likely didn’t leave a whole lot of breadcrumbs behind. So you really have to have a skilled set of analysts that understand the attack techniques used by adversaries and look for evidence that those attack techniques were used within systems.

Tom Temin: Because once you have been infected, and it came from SolarWinds, you’re a couple of layers removed from the original IP addresses that might have sent these out. Is there any possibility that a given agency can detect the original source at this point?

Chris Kubic: Well, you know, I think at this point, there were specific versions of SolarWinds that were compromised. So I think if your systems currently have or ever had any of those versions, specifically, during the timeframe that you know, from March on, you just need to assume that you are potential victim, and you need to take actions assuming that your systems have been compromised.

Tom Temin: And people look for different vectors expecting attacks. And there’s lots of monitoring systems that have been in place for a long time. And those keep getting more sophisticated. Did this come from a vector that nobody ever dreamed would come from, and therefore nobody was watching? Because solar winds is or was a trusted supplier. And patching is something that every agency attempts to keep up with?

Chris Kubic: Well, so I wouldn’t say that it’s not a an attack vector that has been used in the past, certainly, supply chain attacks have been used across history. So it’s not a new attack technique, so to speak, but you kind of hit on a key point. A lot of defenses are tuned to be able to detect known threats. So they have signatures for vulnerabilities that – in systems, this is a new attack vector. So there were new signatures that were going to readily pick this up. So that’s where more sophisticated technologies need to come into play once they can really look for anomalous behaviors within networks and kind of flag those to the security operations team to investigate because the traditional signature-based defenses just aren’t going to cut it when you have a new threat, you know, a new attack vector like we saw in this particular attack.

Tom Temin: We’re speaking with Chris Kubic. He’s chief information security officer at Fidelis Cybersecurity, formerly with the NSA, and do you imagine the NSA is probably lending advice to agencies at this point, too?

Chris Kubic: Yeah, I’ve certainly wouldn’t have any insights into that. But you know, I fully expect that all of the expertise across the government from NSA and from others, I mean, there’s plenty of folks with very good analytic skills, very good forensic analysis skills across the government. And I got to believe that they’re all leaning forward and helping out these agencies that have been hit, because, you know, this does take a skilled analyst to really dive in and find the minimum breadcrumbs that the attacker would have left behind. And that expertise is certainly in short supply, both in government as well as commercial space. But I gotta believe that the federal government’s doing everything to help out any of those impacted at the federal level, as well as at the state or local level.

Tom Temin: And given your earlier point that this is not a brand new vector, that people have been using supply chain attacks since there have been online computers, what should the government have been doing differently? Or what should it do differently? Now to make sure that it’s in tune with that vector?

Chris Kubic: That’s a really good question. You know, supply chains are inherently very complex. I mean, you have lots of third party suppliers, you have components that are built all over the world. And we now have global economy. So not everything is developed here in the U.S., it’s developed in a global scale. So it can be very challenging to try and secure that supply chain. There’s certainly been lots of good work done in that space. And the vendors in particular, the folks that are building products, they’re interested in making sure their supply chains are secure, because not only can it cause an attack like this, but it also in a lot of cases, supply chain attacks, create fraudulent versions of their products, people, you know, essentially stealing their intellectual property and selling knockoff versions of the product. So they’re very interested from a bottom line in securing their supply chains, and they’ve done a lot of good work in that arena. But I think when it comes to supply chain, given the complexity of it, you’re not really going to completely fix the supply chain problem, you really need to kind of manage the risk to the supply chain. And managing risk has to happen on the vendor side, as well as the people that are actually buying the products, you know, on the vendor side, they need to do the due diligence to make sure they’re fully vetting their supply chains, you know, doing audits and those types of things, understanding the secure processes that their vendors or suppliers have in place to protect the supply chain. And then the folks on the you know, the buying end need to make sure they’re buying from trusted sources and making sure that they’re properly, you know, installing and configuring their products, making sure they’re getting the software from authorized sources, and making sure that people that are maintaining their systems, you know, are also trusted as well, because really, on the buying side, you know, the supply chain is also very broad. So you have to look at it from a cradle-to-grave stand [point].

Tom Temin: And as a federal CISO what would you say to SolarWinds at this point, or any vendor to which this happens and causes so much apparent widespread damage?

Chris Kubic: Well, yeah, I mean, I think any vendor could potentially be a target. So you know, it’s hard to kind of throw stones at a vendor, the attacker demonstrated really good opsec techniques based on what’s been published in the press. So I think it would be hard for anyone to really detect this type of attack. But you know, in this particular case, the attacker was able to kind of inject software into the tools within SolarWinds that they use to build their gold copy of the software. And so it’s very critical to make sure that the gold copy that you develop, that authoritative source of your software is trusted. And that’s where that vendor due diligence comes in, they need to make sure that you know, if they have third parties that are providing software to them, that they vetted those third parties, it helps to do independent verification of the software, have a independent team look at the software to look for, you know, signs that something’s been inserted. And they need to secure their internal systems where they do those software builds to make sure that somebody can’t inject bad software into your build process and have it be distributed out to customers, as we saw with the solar winds attack.

Tom Temin: And you spent a long time in the federal government and now you’re on the outside looking in at it as a would be contractor and so on. What’s the difference in point of view? What are you noticing differently from the new point of view? And it must be frustrating to have so much you can never talk about having been at NSA for all those years.

Chris Kubic: Yeah. You know, but certainly that knowledge couples over so, and you know where I can I like to help out on the commercial side and apply the knowledge I have on the federal side. So I think that’s a healthy thing. But, you know, I think really, whether you’re talking a federal CISO, or whether you’re talking private industry, there are plenty of private companies that, we believe have been impacted by this as well. And time will tell what the extent of that is, but it’s not just a federal thing. It’s really, you know, a cross-commercial and federal government, and as I mentioned before, local and state governments could be impacted as well. So I think probably the federal CISOs are feeling a lot of the same pain that’s being felt really across the entire landscape.

Tom Temin: Chris Kubic is chief information security officer at Fidelis Cybersecurity. He’s formerly with the NSA. Thanks so much for joining me.

Chris Kubic: It was my pleasure.

Tom Temin: We’ll post this interview at FederalNewsNetwork.com/FederalDrive. Subscribe to the Federal Drive at Podcastone or wherever you get your shows.

Related Stories

Comments

Sign up for breaking news alerts