Defense News

DoD cyber readiness tests get a major overhaul

The department looks to replace an adversarial, check-the-box mentality around cyber readiness with one that actually helps commanders manage their risk.

Jared Serbu@jserbuWFED

February 28, 2024 7:36 am

5 min read

Starting this week, the Defense Department is implementing a major revamp of the regime it uses to test the cyber readiness posture of its commands, bases and agencies, aiming to move from a compliance-focused checklist mentality to one that tries to help commanders understand how their cyber risk actually affects their missions.

The new process seeks to put a friendlier — and hopefully more helpful — face on the Command Cyber Readiness Inspections (CCRIs) DoD has used for more than a decade. In fact, they’re no longer called “inspections” at all. Cyber Operational Readiness Assessments (CORAs) will take the place of CCRIs starting on Friday.

And although a lot is changing under the hood, officials argue the name change is important in and of itself.

“The main thing we want to see is that people aren’t just preparing for an inspection: When we’re not there, we want them to always be in an assessment mode,” John Porter, the director of network readiness and security inspections at Joint Force Headquarters-DoD Information Networks (JFHQ-DoDIN) told reporters Tuesday. “We want the team to feel like they’re not just having another inspection done and folks are looking to ding them. I want them to say, ‘I have an assessment coming, and we’re ready, but this assessment will help us to identify how we can get stronger.’”

Risk assessments replace pass-fail

As a practical matter, one of the biggest things that will change under the CORA process is the site visits will no longer be pass-fail tests. Under CCRI, a score of 70 or above was considered “passing.”

Instead, the assessment teams from JFHQ-DoDIN and the military services will use a mix of intelligence data and cyber threat information from the MITRE Corporation’s “ATT&CK” database to figure out how vulnerable a particular organization is to current threats. And importantly, even if a command hasn’t completely eliminated a vulnerability from its IT infrastructure, it will now get credit if it’s taken steps to mitigate the risk that threat might have on its missions.

“When you allow for those remediations, now, all of a sudden, you see the assessor working together [with the command] on understanding it so they can make those changes,” said Charles Wille, the deputy director for readiness and security inspections at JFHQ-DoDIN. “I think there’s a tonal shift there that helps the entire organization digest that this is not just another inspection. The wording has been shifted to risk wording, versus terms like compliant and non-compliant. The mission is what matters to those leaders. If it’s the Defense Commissary Agency, we want to connect it to their ability to sell groceries. If you’re the Defense Logistics Agency, and we can show how this will impact logistics, you’ve got a lot more buy-in in the cyber domain.”

Meanwhile, CORA will also use a risk calculus to decide which organizations get assessments, and how often. Rather than inspecting commands on fixed timelines, DoD will conduct the visits based on a “multifactor” analysis that takes needs and assessment team resources into account. Some bases and commands might CORAs multiple times a year; others might go for several years without one.

“We’re actually using threat intelligence and mission priorities to choose when and where to go,” Wille said. “A lot of times, under the old regimen, it was like, ‘Hey, you failed inspection, we’ll be back in six months.’ Well, if they’ve done a good job of remediating, why go there instead of somewhere else that matters? So the focus is to be at the right place at the right time with the limited resources we have and capture that data, so that we have it for multiple uses — both to feed into these great automated tools that private industry has put out there, but also so that we can make sure they’re telling us what we need to know.”

New roles for automation

And officials are hoping their own automated tools will help stretch their limited assessment resources a bit further. The centerpiece is one called the DoD Inspection Analysis Tool (DIAT), which won a DoD CIO award for innovation last year. It started as a simple database to store the results of CCRI inspections, but it’s since evolved into a more robust platform the department uses to automate its assessment workflows and decide how to conduct them. Eventually, the goal is to let individual commands use DIAT to help grade their own cyber readiness between their formal site visits.

“The end state is to have a continuous monitoring feature,” Porter said. “If we’re coming in every six months or once every year, how do we fill those gaps in between? We want to be able to have a real-world look at everyone, and also empower them to constantly assess their own environment. And then we could come in and make sure that folks are where they need to be. That combination is where we want to get to: continuous monitoring and continuing to empower individuals to monitor themselves, because we know we cannot do this alone.”

Even though DoD is trying to take a less adversarial and more partner-focused approach to the assessments, there could still be serious consequences if a CCRA uncovers major cyber vulnerabilities. In the most extreme cases, JFHQ-DoDIN still has the authority to order commands to disconnect from Defense networks.

“But we’re trying to set clear lines between high risk and low risk,” said Nicholas DePatto, JFHQ-DoDIN’s inspections branch chief. “We’re trying to get away from the days of ‘Oh, my shredder doesn’t have oil, so I failed an inspection.’ We want to have a clear line between high risk — what we care about — and the low-risk, low-threat things we don’t care about. And I think CORA enables that mindset.”