Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.
The Air Force’s senior leadership has set a goal of shaving a collective 100 years from the schedules in the service’s acquisition programs. And officials say there are already some significant signs of success, especially in the world of software development. Projects that used to take years are getting at least partially fielded in weeks or months.
Via a project it calls Kessel Run, the service had already gotten a fair degree of attention in defense IT circles. One of the Boston-based agile development office’s first goals was to roll out new mission planning applications for Air Operations Centers within 120 days, a feat that would have normally taken years via the military’s traditional approach to software acquisition.
But officials say they’re finding ways to institutionalize that sort of speed throughout the Air Force. Against the 100-year target, they have already mapped out strategies that they believe will cut 62 years from existing acquisition schedules.
The main techniques the Air Force plans to employ are agile development, rapid prototyping and more use of software-as-a-service, said Susan Thornton, the director for information dominance programs in the office of the Air Force’s assistant secretary for acquisition.
“Air Force leaders at a lot of different levels are encouraging risk-taking to achieve results, but it’s not just risk for risk’s sake — it’s not just running with scissors,” she said Wednesday at a conference hosted by AFCEA’s Northern Virginia chapter. “It’s understanding the risks sufficiently to make decisions to pursue that higher risk path when it has the potential for high payoff. It’s speed with rigor, and structuring our efforts to fail small, fail fast. Cultural change is difficult, and it’s probably the single hardest thing we’ll do, but the times and the conditions are right. And we’re seeing results from that cultural shift today.”
One example is the Air Force’s effort to move all of its contracting offices to a single contract management tool, called CON IT.
Instead of awarding a traditional development contract, the service partnered with agile development experts at the U.S. Department of Agriculture. The team wound up mostly reusing a software package the Defense Information Systems Agency was already employing. With some tweaks, they deployed it to users in U.S. Southern Command in less than nine months.
Growth of practical application
It went into day-to-day use in SOUTHCOM in February, and the deployment pace has picked up since then: 42 contracting offices across the broader Air Force are using it as of now, and they’ve used CON IT to award more than 14,000 contracts.
“But that’s just part of the story,” Thornton said. “Because of what we were doing with CON IT, we were were able to respond quickly when Hurricane Michael devastated Tyndall Air Force Base back in October. Contracting is really important in a disaster, and getting their contract professionals back online quickly was really important. 24 hours after the call, the new accounts were active, and a few days later, [their legacy] servers were recovered from the base and transported to Maxwell Air Force Base to migrate their data to the CON IT cloud based environment. That gave their users access to existing contract data and contracts from any location pretty much worldwide.”
Much of the shift to rapid development is being led by a program executive office the Air Force stood up in September specifically to build on the success of Kessel Run, aiming to extend the use of leading edge agile dev-ops methodologies across the service’s acquisition enterprise.
Steven Wert, the program executive officer, said his office is in charge of about 100 separate programs.
“We do a lot of software, and across the board, what we’re not doing anymore is a year of development followed by a year of developmental testing and operational testing. That’s very inefficient,” he said. “I’m not a purist about agile dev-ops. If we can get to a cadence of releases to avoid accumulated risk, if we can get our teams to work directly with end users, and if we can leverage automated test, we are much better place than how we’ve traditionally done business.”
Wert said that’s partly because the Defense Department’s traditional approach to weapons system testing — involving long, dedicated periods in which independent testers examine a system and then submit detailed, written reports — makes very little sense when the service is trying to embrace a model of continuous, iterative software deliveries.
“Most of you have been there: That point in the program where you go through initial operational testing and the [Air Force Operational Test and Evaluation Center] declares your program not effective, not suitable and not mission capable. That’s what I saw immediately,” he said. “But If we can deliver a minimum viable product and work very closely with end users and then spin on that product, we have dramatically reduced delivery risk. Another key is the cost of delay. To set up a software program to develop for three to five years and then go through a year or two of test means you’re delivering absolutely nothing for six or seven years. That’s an insane way to do business, but it’s the cost of delay. Those 400 requirements that you wrote in your requirements document — there might be only two or three the warfighter really wanted. If we make them them wait six or seven years, that’s an insane way to operate.”
Early stage planning, development
In the early going, the work done by PEO Digital is producing new software releases at a much faster clip than DoD’s testing procedures are designed for.
In the case of one program, the Air Force’s Integrated Strategic Planning and Analysis Network, developers had pushed out 15 new software versions in the time it took the test community to examine just one earlier release of ISPAN.
The system, which helps the Air Force and U.S. Strategic Command game out nuclear planning scenarios, had been in the works for several years, but it’s another case in which the Air Force has dramatically sped up the development process, Thornton said.
“They’re accelerating capability delivery by years, through the use of prototyping and agile dev-ops. They had gone through a more traditional operational test event, but because of how they were doing their development, they quickly fixed the errors that had been identified during that event, and they actually had corrected the bulk of those errors prior to the report even being released,” she said. “That gives you also a sense of the power of this: when you align the developers and the operators, you can really quickly identify where things are going wrong, fix them and address them.”
Software is Air Force acquisition’s biggest problem, Roper says
In other cases, the Air Force has managed to fix hundreds of bugs and make other software improvements that wouldn’t have otherwise been fielded to users if it had decided to conduct its software development process in the traditional sequence that waited for initial operational test and evaluation results.
Wert said that’s exactly what happened with the system that Kessel Run built for Air Force tanker flight planning.
The tool had been in development since 2013, bogged down, he said, by excessive layers of Pentagon oversight. Even after the Air Force delegated some acquisition decisions to lower levels and decided to pivot to an agile methodology, an early test report declared the system to be not operationally effective.
But Wert said he was able to convince officials at Air Mobility Command to greenlight its deployment to the operational force before they otherwise would have.
“We showed them how fast we could work with their end users and spin improvements to this capability,” he said.
Four major software releases were sent out to the mobility fleet while testers were still examining an older version of the system. The result: AMC successfully used the system to plan more than 39,000 flights in a more fuel-efficient way.
“How do you measure success in agile dev ops? Results. You actually deliver software,” Wert said. “When you compare and contrast and look backwards to how we have done traditional acquisition programs, you realize that on many of our bigger programs, the only measure of success is whether you’ve had a victory over the bureaucracy. Even with our most successful programs, when they had an engagement in the Pentagon, you’d feel like you needed a shower afterwards. It’s just nonsense that goes on.”