If there was ever a case for agencies to change the way they plan and manage information technology projects, here it is.

High-level data from the Federal IT Dashboard compiled exclusively for Federal News Radio shows the average time it takes agencies to complete an IT program is 1,018 days and the average cost is $23.2 million per program.

The Defense Department, on average, takes the longest time to deliver value — 1,915 days per IT Dashboard program at an average cost of $115.5 million per program.

On the other side of the spectrum is the Energy Department, which takes on average 272 days per IT Dashboard program at an average cost of $2.6 million.

But before you gasp for air and call for hearings and inquiries, this raw data doesn’t tell the entire story.

Current and former federal chief information officers say the data, in and of itself, is interesting, but also shows the problem with the IT Dashboard and the way agencies track projects.

“I see this as an artifact of different agencies defining projects differently or breaking them down to different levels of granularity. Almost an apples to oranges kind of thing,” said Simon Szykman, who spent seven years as a federal CIO in the Commerce Department, and now is chief technology officer for federal services for Attain. “Inherently there’s a problem. Many times major IT programs include multiple projects so maybe there needs to be more transparency in how agencies break down programs for public reporting?”

The problems with the data on the IT Dashboard are not new. The Government Accountability Office has been pressing OMB since the portal’s inception about keeping the information current to ensure maximum transparency.

But this is more than just a dashboard problem. Agencies have a difficult time getting out of the cycle of waterfall projects and into agile development for many reasons.

Some current and former CIOs point to the budgeting, planning and implementation challenges agencies face annually through the capital planning and investment control (CPIC) processes, requiring business cases, years of advanced planning and minute details.

“What you see in CPIC and what that aggregate number gives you is a mix of waterfall projects with long durations, agile projects with short durations and things in operations and maintenance that may be listing a six-month or year-long basis. So rolling up that data, which is done on a regular basis, doesn’t give you a true view of how the agencies are actually doing its projects,” said Ann Dunkin, the CIO of the Environmental Protection Agency. “And there have also been some real incentives in CPIC in the past to put in fewer longer projects and not put in development and focus on O&M because there is less work associated with that. So all those things together skew everyone’s data to the longer durations and the larger dollar values.”

Duncan said EPA does use the IT Dashboard data, but must dig into it more deeply to understand what is going on with each and every project.

But stepping back from the process challenges agencies, the data also highlights in pretty stark terms the need to change how CIOs and mission owners look at projects and measure success.

The Office of Management and Budget gave agencies modular contracting guidance in 2012, encouraging them to deliver capabilities in six months or less, and 90 days or less for specific tasks within the projects.

In 2014, OMB reported that agencies using agile are delivering capabilities on average 20 days faster than those using waterfall.

Steve Cooper, the Commerce CIO and former CIO at the Homeland Security Department and the Federal Aviation Administration’s Air Traffic Organization, said he was a bit surprised by the information, but more importantly, it reinforces the direction his agency is heading — toward agile development.

“What we are pushing very hard across all of our programs, projects and initiatives at Commerce is shorter duration between both measurement and the delivery of tangible value as defined by the customer. In this case, the customer is the recipient of the delivered functionality of the program, project or initiative,” he said. “By shortening that duration, and what we are pushing very hard is literally incremental development or agile of inside smaller duration of six months or less, we actually want to see and review with the customer on about an every four-to-six week basis and ask how are we doing?”

Cooper said he wants to make sure the mission customer is receiving new functionality or capability on a regular basis.

EPA on path to use dev/ops

Over at EPA, Duncan said she is moving projects toward agile development processes.

“We have three camps here. We have folks how are doing agile where the average project duration is extremely short, but it varies even there. Are you calling your project one sprint? Or are you calling your project 20 sprints? How are you calculating that project length?” she said. “Then you have waterfall projects. We are working on them. We are still trying to improve them. We are having those conversations with people about how do you do things differently. But some still have durations of six months to a year in between when they deliver functionality.”

EPA, like other agencies, is spending most of its IT budget on legacy systems. Duncan said upgrading older systems tend to fall into the waterfall category.

“We are trying to drive things out of waterfall and into agile, and then by default we will drive our average duration down,” she said. “

Elena Larsen, the chief of staff for EPA CIO, said through the use of PortoflioStat reviews, the agency has pockets of excellence around iterative or dev/ops use. Larsen said among the lessons learned is for project managers to get away from hard dates because that drives buggy code. She said successful projects have goals like completing two high-quality releases a year, both of which meet user needs.

By the way, EPA had some of the better aggregate numbers — 374 days and $1.7 million on average per project.

Joe Klimavicz, the Justice Department CIO, said while his agency isn’t using agile across all IT projects, the ability to deliver capabilities quickly is at the forefront of his approach.

“I remain mindful of the benefits derived from an agile approach and future projects’ business cases must include such an approach,” he said. “DOJ is within the targets for average activity length specified by OMB’s guidelines of no more than six months.”

But moving to agile isn’t so easy and that’s the rub.

Richard Spires, a former DHS CIO and now the CEO of Learning Tree, said the reality that most agencies face is the budgeting, planning and procurement processes are little different than a decade ago.

It may take a program a year or even two to get to the point where it can begin development. So that makes the overall program schedule longer. Think about it, if an agency started a project in 2015 by writing a business case, but didn’t get funding until 2017, then that’s already two years before the program even gets off the ground. If the agency started the market research phase or the draft RFP development, agencies may have to put that information in as a business case to get the budget process moving.

“We clearly need to work on how to accelerate the means by which agencies can get budget allocated for new initiatives, and streamline the procurement process,” he said. “It’s easy to state, but it has been very difficult to make significant progress in a system that typically requires many stakeholders to approve, includes a lot of oversight, and fosters a risk-adverse culture.”

Changes to CPIC process in the works

EPA’s Duncan takes it one step further, emphasizing the challenges with the CPIC process. She said it’s not a great tool for agile projects.

Larsen added CPIC is more of an accountability tool than a project management tool for OMB.

“What we are trying to do under Federal IT Acquisition Reform Act (FITARA) and even some of the new CPIC development coming out of OMB is to really move to more agile delivery and processing. We need to make sure we are trying to drive that decision making and those practices under CPIC,” she said. “There are some new tables that coming out under CPIC that instead of focusing on projects, are trying to focus on releases with numbers of sprints and iterations in-between. We haven’t had a chance to test drive that. It just came out this summer. But once we do, then we can see if that’s really driving the behavior we want to see in the projects.”

OMB led a CPIC summit year where a group of federal experts came up with ideas and suggestions for how to improve the process.

Duncan said she hopes the changes will make it easier to report CPIC data, break out O&M spending versus modernization spending, and give agencies a better overall understanding really how long projects are taking until mission areas get value from IT systems.

Commerce’s Cooper said to shorten the duration and costs of IT projects, CIOs need to change the mindset of the agency.

“One of the things that we have been working both within Commerce and with OMB and [federal CIO] Tony Scott is everybody has begun to recognize that the metrics that actually are the metrics we all are supposed to use in the CIO and IT scorecard are geared toward waterfall methodology. Therefore, they don’t lend themselves well to PTO or the Census Bureau or BEA who already are using agile methodologies,” he said. “What we are trying to do in Commerce, but also in concern with other CIOs and with Tony’s office, is to revise those metrics. When you revise the metrics and actually build the metrics around agile development, you will see two things right away. First of all, the average duration will immediately decrease because you are not trying to measure from inception of investment to first production. That is what the duration measures today. If it’s going to take me three years, four years or five years to get to full functionality using the metrics in the scorecard today, I don’t know unless we change the metrics, I’m not certain we will see the duration come down dramatically because the metrics themselves are not reflecting iterative development.”

Cooper said he’s pushing to measure the duration between delivery of value and have a different set of metrics that measure the program or project itself.

“The real value is when we discover something that might go wrong or has gone wrong early, we absolutely save money in the time it takes to fix it,” he said. “We are reducing the full cost of the program every time we catch something early.”

Return to the Reporter’s Notebook

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.