Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne. 

Two key laws require federal agencies to make their decisions based on evidence and performance information. And according to the Government Accountability Office, they’re making progress. According to GAO’s latest survey of more than 4,000 federal managers, they’re using performance information more often than at any time since the office started the survey in 2007. Federal News Network’s Jared Serbu talked about the results with the acting director for strategic issues at the GAO, Alissa Czyz, on Federal Drive with Tom Temin.

Interview transcript:

Alissa Czyz: What we mean by performance information here is it’s really part of a broader suite of evidence, as you alluded to, it’s a key building block for sound decision making. So it’s any information used to track progress on how a program is meeting its intended goals or objectives. So it can take a variety of forms, it can be quantitative, or can be qualitative. It measures different aspects of performance, quality or efficiency, customer satisfaction, could be a customer satisfaction survey, for example. In our report, we provide a couple of examples of what agencies are doing, and NASA in particular, we talk about how they’ve created kind of the central repository of information on their acquisitions projects. So their agency leaders kind of have a one stop shop for looking at their projects across the board and making informed decisions. So that’s just one of kind of several examples of what performance information could include.

        Learn how federal agencies are preparing to help agencies gear up for AI in our latest Executive Briefing, sponsored by ThunderCat Technology.

Jared Serbu: And since performance information can be so many different kinds of things, it seems like on its face a difficult thing for GAO to study. So can you talk a little bit about how you actually designed your survey here and how you gathered data on whether or not agencies are actually increasing the use of evidence?

Alissa Czyz: Sure. And let me take a step back and talk about the survey. So GAO has been periodically surveying federal managers across 24 major federal agencies way back since 1997. So we do this periodically, every few years. And we ask a lot of questions about different types of management activities. So this one being how much are managers using performance information in their program management, and what are their views of how much their agencies are doing that. We also kind of break our questions down into different types of evidence. So we look at evaluations and research and statistics, but this is performance information here. And kind of how we develop the survey is that we have, it’s a long survey, but we have about 11 questions that specifically asked federal managers about their views, the extent to which they’re using performance information in various activities, like looking at allocating resources to programs or identifying problems, those sorts of things. And like many surveys that you’ve probably filled out, it’s on a five point scale. So one being, they’re not using performance information at all, a five being they’re using it to very great extent. And what we do is we average the scores of those 11 questions really focused on performance information, and we come up with what we call an index score. And we’ve been doing this index since 2007. In our most recent survey that we’re reporting on today, for the 2020 survey, we actually saw the highest result of that index since we’ve been measuring it since 2007.

Jared Serbu: And is there any way to break down those results to show what types of information is being increasingly used? Or is there any other explanation that GAO came away with to describe what agencies have been have been doing well, what the government at large has been doing well to cause these increases?

Alissa Czyz: Right. And so I’ll give you a word of caution. So we see this as very positive that our survey is showing that managers are saying that they’re using performance information more in their activities, and they see their agencies doing that. And that’s generally across the government, so that was 16 of 24 agencies that we surveyed. The word of caution is this is according to federal managers. So for this report, we’re reporting on a survey, so we’re reporting on perceptions of managers. We didn’t go into the agencies for this work and get behind what was going on in the agencies or kind of where in the agencies this might be happening more than others. Some of these are quite large agencies, as you can imagine. We have ongoing work now as part of looking at the implementation of the Evidence Act that you referenced, that we’ll be going into select agencies and looking at some of their activities here. But it’s an important thing, perception is very important. So this looks like it’s on the right track. I think a lot of the kind of increased emphasis from Congress on the GPRA Modernization Act and the Evidence Act that you alluded to, of really trying to get agencies to have a more systematic approach to how they run their programs is perhaps starting to pay off. But we’ll be doing some more digging and kind of seeing what specifically is happening at these agencies.

Jared Serbu: I guess one other way to talk about generalizing these results across the government, is can you maybe say a few words about what GAO thinks the specific practices are that encourage the use of performance information and evidence, and how closely those kinds of practices are correlated with the the increases in the survey data that you saw?

Alissa Czyz: Sure. That’s a great question. And in fact, in our survey we did ask about specific practices that we’ve found in our work for many years can be associated with increased use of performance information in decision making. So we asked questions to federal managers about kind of management commitment to being driven by information for decision making. How much are our goals and objectives being aligned with the program, how much emphasis is there on improving the usefulness or the quality of information, or developing the capacity, tools and training for staff to use information? And then another practice is communicating effectively, communicating performance information. So we asked about these kind of key practices in our survey. And what we found was that they were positively correlated with increased results. So out of six of the 10 questions that specifically asked about these practices that we’ve previously identified, we saw results increase in this survey compared to our prior survey in 2017.

Jared Serbu: Yeah, and I guess trying to correlate those two things is probably pretty important because if all you did was ask managers for their perceptions about whether they’re using more performance information and evidence, it might not necessarily tell you a lot if they don’t have something behind that to actually inform them about what quality data is, or what meaningful performance information is, if it were solely perception based. Again, that may not tell you a lot.

Alissa Czyz: Exactly. And one of the things we really drill down on in this survey was this data driven reviews. So those are kind of regular reviews by agency leaders using information to evaluate programs. And we really saw that those managers whose programs were subject to those kind of high level, data driven reviews were much more likely to say that performance information was being used in decision making. So it really does come from the top there. And but that’s exactly what we did, Jared, we tried to tie what’s going on here to some specific practices. And that would be helpful for other agencies that might not have done so well to look to try to implement some of those practices.

        Read more: Agency Oversight

Jared Serbu: And this isn’t the sort of GAO report where you come away with specific recommendations to specific agencies, but any takeaways that you want to share with us on what OMB or other agency leaders can do in general to increase agencies’ use of this kind of performance information?

Alissa Czyz: Right, we didn’t make specific recommendations in this report. But in fact, we had a report come out in July, just a few months before on other parts of the survey, and that was more broadly on agencyies’ ability to develop and use evidence in their decision making. And what we found in that report was that it was really uneven across the federal government. Some agencies were doing a really good job of being able to do this and provide their staff and managers with the tools and training to develop quality evidence and informed decision making, and others were not so far along there. So just a couple months ago, we made a recommendation to OMB to leverage our survey results to kind of better understand what’s going on at agencies and kind of direct some actions were needed. And as I mentioned before, we’ve got some current work kind of trying to get behind what’s going on at specific agencies. So I imagine we’ll be able to help OMB target their efforts a little bit more with our current work.

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.