How do agencies tell if cybersecurity money was well spent?
How do you measure the value of spending on prevention? That's a recurring challenge for federal agencies sensitive to the dangers of cybersecurity breaches.
Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.
It’s an age old problem, but a practical one. How do you measure the value of spending on prevention? That’s a recurring challenge for federal agencies sensitive to the dangers of cybersecurity breaches, and to appropriators who vote on the money. So how do you justify cybersecurity spending? For some of the latest thinking, the Federal Drive with Tom Temin turned to Paul Rosenzweig, former Homeland Security cyber official, now a senior fellow at the R Street Institute.
Interview transcript:
Tom Temin: Paul, good to have you on.
Paul Rosenzweig: Great to be here. Thanks for having me.
Tom Temin: Just recently, in the past days, the Defense Information Systems Agency had to send out those letters telling people that their data was breached, a couple of 100,000 people, and therefore they’ll be getting free credit monitoring, etc. etc. So sometimes these breaches do prove the importance of spending. But when nothing happens, it’s a tough one.
Paul Rosenzweig: It is a tough one. Right now, if I’m the chief information security officer at any enterprise, whether it’s DISA or private sector commercial enterprise, I go to my management and I say, “give me $5 million for this new upgrade in security.” They’ll say, “great, what do we get for it?” And I’ll answer, “well, it makes us safer.” And they’ll say, “great, how much safer?” At which point I go, “hmmm huh..” because there’s no way off quantifying the benefit of the cybersecurity improvements. Or to put it more accurately, there’s no transparent, auditable generally accepted way of doing that. We know that adding a new firewall probably helps some or changing to two factor authentication probably helps some But we have no way of rigorously quantifying that in any way that that would address budget makers in Congress so that they can make comparative tradeoffs of resource allocation and say, well we’d rather invest five million there than over here. That is a problem. It means, essentially, that cyber security is an art. It’s not a science. It’s not a measurable science yet, and until it is something like that, it will never be able to get the resource allocation that’s appropriate.
Tom Temin: And there’s no private sector model for this, say from the energy sector or some of the banking sectors where they’ve got equal cyber security risks?
Paul Rosenzweig: Essentially, no. One of the things that I did early on in studying this problem was try and survey all of the different ways in which people have used metrics to try and assert improvements in cybersecurity. My colleague at R Street, Kathryn Waldron, put together a bibliography of all of the different methodologies of which there are four or five dozen, none of which are agreed upon, none of which are commensurate with each other, all of which are kind of proprietary to different corporations. Most of what we do right now is cybersecurity by checklist, Right? I give you a list of things to do and you check the boxes on all them. A lot like FISMA, the Federal Information Security Management Act stuff. And that gives you a nice report that you put up on the desk and you never look at again. There is no dynamic way of doing it. In fact, when we did this survey, there was a substantial minority of people who said that it was impossible to do. That the cyber realm was so dynamic, so constantly changing that any effort to actually measure a security of an enterprise at a particular point in time was doomed to be outdated before the measurement was even completed, which was pretty grim. I mean, when you think about it, the fact that we would rely for 25% of our economy on an area, a sector whose safety cannot be decisively measured, that would be really pretty scary from a policy point of view.
Tom Temin: And that check list idea, in some ways, that resembles flight, where there are extensive checklists before you can fly, especially a plane with passengers or, in reality, any plane. And given the safety record of commercial aviation in the United States, you could say that the checklist approach and whatever technology is behind all of the safety measures works pretty well.
Paul Rosenzweig: It does in areas like that where the problem is a static one. The problem in cybersecurity is that its dynamic, and in particular, we have human adversaries who are adapt to our solutions. In the aviation field or, for example, automotive safety. The problems of metal-wear and fatigue, construction, manufacturing problems — those are relatively well bounded and readily, easily quantifiable. The challenge here in cyber is that as soon as I build a checklist for problem A and think I’ve solved it, the adversaries move onto problem B or, attack vector C, is one of the reasons why some people think it’s an insoluble problem. For myself. I think that there are ways of measuring risk reductions and overall vulnerabilities that could probably be developed, but we aren’t there yet.
Tom Temin: When you mention the automotive industry, that brings to mind another issue with cybersecurity. I just heard this on a panel the other day. The car engineers always say the one thing we can’t fix is the nut behind the wheel. Many cybersecurity incidents, the root cause is not so much lack of installed technology, but simply that somebody clicked on something they should not have. The phishing attack is one of the most potent things that can’t be controlled because you can’t stop all email. So maybe the training and education of your own staff is something that maybe doesn’t get enough emphasis in all these budget discussions.
Paul Rosenzweig: I think that’s exactly right, and that’s actually one of the things that a good comparative metric might help with. Kevin Mitnick, the famous hacker once said, “there’s no patch for human stupidity.” And that’s essentially saying, you can’t fix the nut behind the wheel. But what we can do is ask questions like if I have $5 million or $50 million, is it better invested in a new firewall or a new enterprise intrusion detection system? Or is it better invested in training and education of my workforce and testing of them? And the answer right now to that question is, I don’t know. I don’t know how to make the comparisons between them, and because of that, we tend to default to technological solutions, which are much easier to kind of think of as a box. You just buy a thing, you plug in the widget and you’re better off, rather than an ongoing human relations problem. That is never soluble, only subject to mitigation.
Tom Temin: And you watch a lot of the activities of federal agencies. How would you assess the cybersecurity and Critical Infrastructure Security Agency, CISA, at Homeland Security? Are they on the right path do you think in terms of their role in federal cyber security?
Paul Rosenzweig: I think that that creating CISA as an operational agency was a positive step for cyber security. It’s good to have some place in the federal government, at least for the commercial world in the private sector, to have an input. I would say that they are still an agency in formation. It’s only been a couple of years. They’re still putting together their strategic approach. They’re still building up their capabilities. It’s probably too early to give them a real grade. Incomplete is the right grade. But if I had to give him a grade right now, it’d be a B, it could be higher. The main problem that CISA has right now is that much of the focus of leadership above the system level at DHS and in the White House is on other aspects of homeland security, most notably immigration. And so there’s a real lack of executive focus on cyber security problems and issues, which is, as everybody who listens to the Federal News Network knows, executive focus is what drives the federal government’s attention in ways that we all understand and it’s lacking right now, So CISA is doing well. But it’s not getting the love and attention it needs from senior management at DHS or the White House.
Tom Temin: I guess, to underscore what you say, there is no cybersecurity coordinator at the White House level that there had been for about 10 years up until now.
Paul Rosenzweig: That’s exactly right. And, you know, in the early part of President Trump’s administration, the Homeland Security adviser, Tom Bossert had a wealth of experience in cyber matters as well as other homeland security matters. The Secretary of Homeland Security fora while , Kirstjen Nielsen, had done much of her early work in cyber related matters. And at that time there was a focus on some form on cybersecurity. Today, the current acting head of DHS is not focused on cybersecurity, he’s focused on immigration. There is, as you say, no cybersecurity coordinator at the White House and the Homeland Security adviser, the newly named Homeland Security adviser, is not a cyber expert either. I would characterize it as that the bureaucracy in the federal government is doing good work at the level that it can, but without executive focus, it can’t make great great changes.
Tom Temin: Thanks so much.
Paul Rosenzweig: Ok, thank you very much. It was great to be with you.
Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.
Related Stories
Exclusive 