Two of the largest agencies are looking at increasing the amount of testing and evaluating of their often-troubled acquisition programs as the panacea to systemic problems.

The Defense and Homeland Security departments are pushing project managers to test technology or weapons systems earlier in the acquisition lifecycle to understand and solve potential roadblocks sooner.

The idea of testing sooner may seem like common sense — the sooner you find the problems, the better it is for the program.

But as Michael Gilmore, DoD’s director of operational test and evaluation, has found in his 24 years in government, common sense doesn’t always rise to the top.

“It’s clear that testing, in and of itself, is not the reason programs get delayed. It’s not the reason costs grow in programs. The results of the testing, you bet. The results of the testing often lead program managers in the services and the acquisition authorities in OSD to try to fix the problems that the testing uncovered, which is what testing should do,” said Gilmore during a speech Tuesday at the National Defense Industrial Association Test and Evaluation Conference in Washington. “So when people claim that testers are driving billions of dollars in costs to the problems, the facts simply do not support the claim. That claim is not factually based, period. That doesn’t mean there aren’t people who strongly believe it’s true. But it is not factually based. The facts indicate the converse — the earlier that you test, the more information you get about the problems with the program, the sooner you will be able to fix them at a lower costs. That’s what the facts support.”

Gilmore so strongly believes that DoD is short-changing the value of testing and evaluation that he submitted a series of recommendations to Congress as part of its acquisition reform efforts.

Sens. Carl Levin (D-Mich.) and John McCain (R-Ariz.), chairman and ranking member of the Homeland Security and Government Affairs Subcommittee on Investigations, are collecting suggestions from Defense and industry experts on ways to improve the military’s procurement processes.

Workers need better skills

Gilmore’s recommendations included integrating test and evaluation planning into the requirements phase, including scientists and engineers earlier on in the acquisition lifecycle planning phases and making it easier for DoD to hire “technical excellent workers.”

“I think engineers and scientists need to be just as involved in generating requirements as the operators. Certainly the operators must be involved. The operators are the people engaged in the fight, trying to kill the enemy so they aren’t killed, trying to prevail on the battlefield and protect our security. So the operators must play a key role in generating the requirements. That’s not a profound observation,” he said. “But engineers and scientists must as well, and for that matter the test community should, which is composed of scientists and engineers. We know what the problems are. We know what the physical limitations are.”

Gilmore said two prime of examples of where scientists and engineers could have made a huge difference and saved the DoD tens of billions of dollars was with the Army’s Future Combat Systems and Joint Tactical Radio System programs.

When the Army developed requirements for the Joint Tactical Radio System (JTRS), it asked for performance measures that weren’t possible and engineers knew they weren’t possible for decades, he said. But since engineers weren’t consulted, the requirements were included in the RFP and ended up costing the Army $8 to $10 billion for a system that, in the end, didn’t deliver on its promises.

Gilmore said the Future Combat Systems (FCS) also suffered from outsized requirements, in part because the senior officials didn’t heed the concerns or suggestions from engineers. The Army ended up cancelling FCS and wasting between $15 billion and $20 billion over the last decade.

Culture change slowly happening

But while these two examples demonstrate a lack of understanding of the value of scientists and engineers, Gilmore said he already sees change across some parts of DoD.

He said the Army now is testing and evaluating technology much earlier in the process through the Network Integration Evaluation program.

Additionally, Adm. James Winnefeld, vice chairman of the Joint Chiefs of Staff, issued a memo in January 2013 letting military services and agencies submit a request to the Joint Requirements Oversight Council to change program requirements if needed.

In addition to including scientists and engineers in the requirements development process, Gilmore said other ways to ensure test and evaluation are part of the acquisition planning earlier on is for programs to develop a draft concept of operations and a draft test and evaluation plan even before they come up with the requirements and release the requests for proposals.

Gilmore said these draft documents likely will change, but it gives both the agency and the vendors a starting point to understand what’s important to the end-user community and what the goals or measures will be for the program.

This concept of including engineers and scientists earlier also means DoD needs better qualified and more of them.

Gilmore said DoD needs a technically excellent workforce to have successful projects.

“Congress has granted me special hiring authority for term appointments, and I’m using it. But that is not the solution to the problem, as far as I’m concerned,” he said. “I know Sean Stackley [the Navy’s assistant secretary for Research, Development & Acquisition] is urging Congress to grant the department special hiring authorities for scientists and engineers, and I’m a 100 percent for that. I see a need for that,” he said. “I’ve never understood how the DoD which attempts to develop and field the most complex systems that anybody on the planet tries to get to work and do that competently without not just a technically competent workforce, but a technically excellent workforce. I’ll say today, we do not have a technically excellent workforce.”

Reorganized T&E at DHS

The Homeland Security Department faces many of the same challenges as DoD.

Steve Hutchinson, the acting director of acquisition support and operational analysis in the Science and Technology Directorate, said he’s trying to bring his experience from DoD to DHS.

Hutchinson, who previously served as principal deputy for developmental test and evaluation in the Office of the Secretary of Defense before coming to DHS, said one of the first things he did was to improve the way testing and evaluating is organized.

“The main idea was to align the test organization to the DHS mission space. There are five stated DHS missions plus then the Homeland Security enterprise where there are a lot of systems in that portfolio,” Hutchinson said. “Our main concern is to answer the ‘so what?’ question in terms of whether the capability satisfies the users’ need in the field, so we need to understand the mission context. So by aligning to mission, it gives our personnel a better understanding of mission space and how the capability is going to be used and that will provide us a better evaluation and test design to answer that so what question.”

He said the new organizational structure is approved and DHS is filling some vacant positions to complete the transition.

Additionally, DHS last month stood up its Joint Requirements Council with the Coast Guard Rear Adm. Mark Butt as the chairman for the first year.

The council is one of Secretary Jeh Johnson’s initiatives under his Unity of Effort program. It brings together all the users, engineers, key decision makers around strategic programs and investment decisions.

Hutchinson said the new set up will help DHS make more efficient test and evaluation based on risk and mission needs.

Beyond the organizational structure, Hutchinson said his goal is get testing and evaluations done sooner in the acquisition lifecycle process — or as he puts it, move T&E to the left.

“The most important question in acquisition is, ‘are we ready to begin producing the item?’ So that’s all based on better testing to the left. That’s why the emphasis is to shift left,” he said. “If you answer the mission question while you are in production, that’s a two-for-one deal. You answer the mission question plus you get the technical out of your test activity. It’s all part of answering the so-what question, better informing the acquisition decision makers and better use of the resources we have. If we shift left, we will improve acquisition outcomes. As you know, everyone on the Hill had got a lot of attention on acquisition outcomes and they know we are bad at it. So what we can do in testing is help that way.”

RELATED STORIES:

House panel targets DoD acquisition reform, but will that be enough

Internal watchdog blasts DoD’s health IT efforts

Johnson proclaims management philosophy for DHS as a ‘unity of effort’

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.