Sometimes it takes an academic eye to spot the flaws in a practical system, like federal IT acquisition

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

Why exactly are so many federal information technology acquisition programs late, over budget, and not always delivering hoped-for results? It might be one of the most studied topics in public management. Now a supply chain and operations researcher at the University of Minnesota may have an answer. Dwai Roy joined the Federal Drive with Tom Temin with analysis.

Interview transcript:

Tom Temin: Mr. Roy, good to have you on.

Dwai Roy: Thank you so much Tom for having me, it’s my pleasure.

Tom Temin: And you’ve published a paper that’s kind of academic sounding, so we’re gonna try to do this in plain English. But what precisely were you trying to discover in your research here?

Dwai Roy: Thank you for that question, Tom. So in this research, my co-authors and I take a closer look at the US federal government spending on technology programs. Like you mentioned earlier, these programs are enterprisewide technology initiatives that consist of interrelated application development and maintenance projects. And they help the federal government to improve operations and deliver essential services to the public. So an example of a federal technology program is the myUSCIS platform, which enables users including international students like me to access information about immigration services. It’s worthwhile to note that in the past 10 years, the US federal government has been spending close to $70 billion annually in executing these technology programs. And the execution of these programs is monitored against a baseline, which is an aggregate plan representing the program’s plan, budget, schedule and scope. And recent reports have found that federal technology programs are rebaselined multiple times resulting in significant additional expenditure of taxpayer dollars. For example, a Government Accountability Office report finds that baseline changes are quite common across federal technology programs, with more than 50% of such programs being rebaselined at least once and more than 25% being rebaselined twice or more.

Tom Temin: Sure. So you are trying to find out what causes this rebaselining?

Dwai Roy: Exactly. In this study, what we are trying to do is to identify the drivers of baseline changes in federal technology programs.

Tom Temin: Alright. And how did you go about the research? What did you look at to try to ascertain this?

Dwai Roy: So in terms of the study methodology, to conduct the study what we did was we collected detailed execution level data on 240 federal technology programs across 24 agencies. 240 technology programs, which in turn consisted of around 800 projects. And these projects, in turn consisted of around 4,000 work units. And because these programs were executed by 24 federal agencies, they were quite representative of the kind of work which the federal government is doing. And just for the information of the audience, this data is actually publicly available on the federal IT dashboard, a publicly accessible website that tells federal agencies and commoners like me to view details about these technology programs and track their progress over time.

        Read more: Contracting

Tom Temin: And by the way, did you have any particular trouble in dealing with the USCIS platforms as a foreign student?

Dwai Roy: No, I think my experience so far has been wonderful. So I think the federal government is doing a great job when it comes to helping international students like us to get into the universities, and a very seamless process.

Tom Temin: So you took all this data on 4,000 work units, so that could be a software development subset of work or the acquisition of machinery, that type of thing.

Dwai Roy: And even maintenance of a legacy system. So it could be application development or maintenance projects.

Tom Temin: And how did you analyze all of this data?

Dwai Roy: So like I said earlier, using this data sample of 240 programs, we tested for hypothesis on the interrelationships between programs, scope, granularity, competency, execution, methodology – and how do they affect the baseline changes in the program? And what we did was we estimated a regression model that accounts for agency level characteristics and also a number of programs specific characteristics, such as the duration of the program, the budget of the program, when was the program started, and so on.

Tom Temin: Alright, by the way, we’re speaking with Dwai Roy, he’s a supply chain and operations researcher at the University of Minnesota. And what did you find? What causes agencies to have to rebaseline? Which is just another way of saying we had no idea how long it would take or what it would cost, here’s the latest estimate.

        Want to stay up to date with the latest federal news and information from all your devices? Download the revamped Federal News Network app

Dwai Roy: So if I were to summarize the three key findings from the study, so first, baseline changes in programs have greater scope that, is programs that have more projects, they can be reduced if federal agencies and contractor firms, they invest greater efforts in componentizing a program into smaller work units, and in identifying managers with high levels of technical and practical knowledge in program management, a competency which is critical for managing multiple interrelated projects. Second, our study highlights the role of number of baseline changes as a value in process metric for program managers and federal agencies to monitor program execution. And finally, what we find is federal technology programs using the agile methodology experienced more baseline changes. In programs with agile methodologies, scope can be higher as these programs can often lack sufficient upfront effort in developing the initial baseline, and depends too much on making adaptations during execution. However, this upfront effort is very critical for better managing adaptations, and for avoiding the time consuming approval process needed for revising a baseline.

Tom Temin: Because everybody’s talking about their adoption of agile methodology. But it sounds like you’re saying that’s fine, go ahead and use agile but keep the scope limited and just do it in an agile way, rather than letting agile branch off into more and more functionality that you might discover along the way.

Dwai Roy: Exactly. I think you summarized it very well there. The only point that we are trying to make is yes, when you’re using agile make adaptations when the program is executed, but keep these adaptations within a certain limit and spend some upfront effort and planning what exactly are you going to do so as to avoid making large adaptations during program execution.

Tom Temin: Got it. So you’ve kind of found a statistical basis for some eternal truths that people sensed but maybe never really could prove.

Dwai Roy: Vigorous econometric evidence. That’s what we provide in the study.

Tom Temin: Alright, so at the conclusion of your studies here in the United States, do you want to work as a government IT person, or maybe work for the Government Accountability Office and get after some of these things before they get out of hand?

Dwai Roy: So like I mentioned earlier, I’m an international student, but I’ll be joining the Darden School of Business as an assistant professor this fall.

Tom Temin: Alright, so you’ll be nearby in Virginia. And that’s a step closer to the power center, hu?

Dwai Roy: Exactly. That’s one of the reasons why I chose to be at Darden because I had multiple offers from across the world.

Tom Temin: Alright, who knows what might come in after people hear this. Dwai Roy is a supply chain and operations researcher at the moment at the University of Minnesota’s Carlson School of Management. Thanks so much for joining me.

Dwai Roy: Thank you so much Tom, this was a pleasure.

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.