Defense and Homeland Security acquisition officials say they are making a serious effort to confront a criticism that’s been a subject of decades of Government Accountability Office reports and blue ribbon commissions: Quite often, the government pushes off the testing of the critical technologies that underpin its multi-billion dollar systems until it’s already signed contracts to buy them.

The new initiative, dubbed “Shift Left,” aims to push program managers to do much more of their testing of key systems and their individual components during their developmental phase rather than waiting until after the “Milestone C” decision, when the government already has committed to the production phase of an acquisition.

By that point, programs are so far along in the process that the inevitable problems that come up during the operational test and evaluation phase are much more costly and time consuming to fix. Or they’re not fixed at all.

“We’re the Consumer Reports for the Department of Defense,” said Dr. David Brown, the deputy assistant secretary of Defense for developmental test and evaluation. “And I don’t read Consumer Reports after I bought something just to figure out whether I’m going to be sorry about what I bought. I read it early.”

In an effort to keep surprises from popping up late in a program’s development, DoD is moving more of its overall testing activity from the “operational” test phase to the developmental phase, Brown told the International Test and Evaluation Association’s annual symposium in Arlington, Virginia, Tuesday. The department also is broadening the scope of developmental test and evaluation to include measures that have not traditionally gotten a close look under the DT&E system.

Brown said “Shift Left” will insist that DoD’s tests evaluate systems under the real-world conditions in which warfighters will use them, and also include rigorous checks of a system’s interoperability and cybersecurity long before an end-product is built.

“Cybersecurity used to be handled as a certification that we did very late in the process, just before we got an interim authority to operate. That needs to be done very early. It needs to be started early in the systems engineering process,” he said. “Interoperability testing is the same way. You can’t test interoperability features in at the end. You have to build them in with well-informed systems engineering, and the informed part comes from systems engineering. The same can be said about mission context. This stuff needs to be used in an operational context, and if it can’t, that’s not something you can wait to discover after you’ve already committed to a design and a manufacturing process. That has to be engineered into the system, and that engineering needs to be well informed.”

Borrowing from DoD’s playbook

Dr. Stephen Hutchison, Brown’s immediate predecessor as DoD’s top developmental test and evaluation official, originated the “Shift Left” initiative while he was in the Pentagon. He became director of test and evaluation at the Department of Homeland Security in December 2013, and said he has brought the concept with him into the new job at DHS.

The change in culture DHS will need to make is very similar, since Homeland Security built its current test and evaluation infrastructure with manpower and concepts that it borrowed from DoD’s practices a decade or so ago.

“We brought great DoD testers over to stand up our organization, and the organization was based on operational testing,” Hutchison said. “You’re going to get great information out of operational testing, but our problem is that it’s not at the right time. It doesn’t help you at the point where it’s most important, before the decision to start production. If you don’t have things right at that point in time, those deficiencies in the system are either going to be too costly to fix or they’re going to be addressed in some future increment. In the meantime, you’ve deployed a system that has shortcomings to the warfighters who are counting on it in order to do their jobs. That, to me, is the definition of acquisition malpractice.”

The Pentagon’s interest in developmental testing has ebbed and flowed over the last several decades. The department’s DT&E capabilities began to wither the mid-1990s, but Congress took an interest in the issue in recent years, and as part of the 2009 Weapon Systems Acquisition Reform Act, it told DoD to create a director for DT&E and ordered to military services to devote more resources to developmental testing.

But so far, Congress has not officially mandated that DoD test all of its systems to ensure they meet their key performance parameters before they move into a production phase.

Developmental is different than operational testing

Frank Kendall, the undersecretary of Defense for acquisition, technology and logistics, said he views developmental testing as having a very different mission than operational testing.

Operational tests are supposed to be used as an independent assessment of whether a program that’s rolling off of the assembly line delivered the capability it was supposed to.

Developmental testing, on the other hand, is less of a pass/fail test.

Kendall said program managers should view the early testing efforts DoD mandates as part of their own acquisition team so that the technology they’re planning for their programs is likely to come to fruition by the time a product is built.

“And it should all be integrated very carefully into a program’s plan,” Kendall said. “Developmental testing is one of our key ways to eliminate risk in a program and to learn about what we can actually do instead of what we think we can do. Then we inform decisions about a program’s design. Shift Left says that we want to do all of those things as early as we can.”

RELATED STORIES:

Senators seek cost cuts for F-35 fighter jet

DoD, DHS see more, earlier testing as a possible fix to troubled programs

DoD plots third chapter in Better Buying Power initiative

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.