People

EPA’s Evidence Act officer honored for using data in program evaluation

Everybody talks about using data to inform decision making about meeting the mandates of the Evidence Act, but fewer actually do it.

Tom Temin@tteminWFED

October 6, 2021 9:41 am

9 min read

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

Everybody talks about using data to inform decision making about meeting the mandates of the Foundations for Evidence-Based Policymaking Act, a.k.a. the Evidence Act, but fewer actually do it. Katherine Dawes is one of those outstanding exceptions. She’s the acting evaluation officer at the EPA and as of last week, a recipient of the Data Coalition’s 2021 Annual Datum Awards. She joined Federal Drive with Tom Temin to talk more about her role.

Interview transcript:

Tom Temin: Ms. Dawes, good to have you on.

        Join us May 2 at 2 p.m. ET for a discussion with agency and industry leaders on network modernization strategy, 5G, AI and plans for the future at the Naval Network Warfare Command and NASA, sponsored by T-Mobile. | Register now!

Katherine Dawes: Thank you for having me, Tom.

Tom Temin: Let’s begin with your title as evaluation officer or acting evaluation officer; we should point out you are a career federal employee with the EPA. What does the evaluation officer do there?

Katherine Dawes: So it’s a required position for the Evidence Act. The Evidence Act has three required positions. The evaluation officer is one we also have a chief data officer and a statistical official. And the role of the evaluation officer is to help coordinate the overall agency response to implement in the Evidence Act. But the position also has some specific standalone responsibilities for program evaluation. And using program evaluation as one advancing program evaluation is one of the many different tools and strategies that we have for responding to the need for building and using evidence. And a big part of that is how do we use data, as a part of delivering on the Evidence Act?

Tom Temin: Program evaluation has been a fairly standard function, as long as there’s been programs, program analysts and program evaluators. So did you do this type of work before the Evidence Act and how does this change the job of program evaluation?

Katherine Dawes: So program evaluation, it is long standing in the federal landscape. However, it’s been one of those items that has uneven investment across the federal family. So part of the Evidence Act’s vision was let’s bring all agencies to this, to using this tool and using it consistently. EPA has had program evaluation as part of its work before, but because we didn’t have a law or statutory authority for consistently supporting it, it’s had its ups and downs at the agency. And we’re not unique in that space. Other federal agencies have experienced that as well. We do have agencies that are leaders in evaluation and had – and they have statutory and ongoing authority and even set asides for funding program evaluation. But that’s not the case across the full federal family. So the Evidence Act was to help bring that home, and EPA has embraced that enthusiastically. And in terms of saying, yeah, we’re going to start using program evaluation to help us with our decision making, to help us improve our program design, and how we’re delivering our services to the American people.

Tom Temin: Because EPA has a number of functions, depending on the medium that they’re controlling – air, water, and so forth – and each one of those has many, many programs. So the programs continue whether or not they’re evaluated. So this sounds like a way of doing better evaluation, not so much for budget justification but for program improvement. Would that be fair to say?

Katherine Dawes: Certainly decisions get made based on any evidence, information, and program evaluation is part of it. But the real animating vision of the Evidence Act is let’s improve programs. Let’s make sure we have the best, highest and best information to make decisions in government so that we have the best working government that we can have. And so that’s the animating vision of the Evidence Act. And certainly, that is what federal evaluators are embracing and trying to help agencies deliver on.

Tom Temin: And give us an example, name a program that might be evaluated, and what are the elements, the data elements that define the program, such that you can have some objective evaluation of it?

        Read more: People

Katherine Dawes: If you look at our strategic plan, we have a draft strategic plan. People can look at it now. And you’ll see in there that we have a learning agenda. And one of the areas of the learning agenda is about safe drinking water. And how can we ensure that the strategies that we’re using to help safe drinking water systems comply with the regulations that we have out and lead to better environmental outcomes, right? So the evaluations that we’re doing there, so we’re having these really important discussions with our co-regulators about how can we test different strategies? How can we set up evaluations to allow us to know which strategies are most effective and where they’re most effective, and with which systems they’re most effective. Different types of compliance and enforcement strategies have different levels of effectiveness depending on how you deploy them. And so we want to learn more about that. And so it’s working with our current regulators. This is exciting. It’s working with states and tri-local regulators to identify, where do we want to set up the valuation so that we learn, and then do better on delivering compliance?

Tom Temin: And what might be some of the data elements, for example, how clean the water is? There’s lots of data that relates to that, what their, say, inputs are to clean water, how much – I don’t, I’m making this up – how much chlorine they have to use, or how much whatever else they can water?

Katherine Dawes: So in this case, it would be more about what is the data that’s letting us know that these systems are actually in good working order? So that’s the kind of data that we’re talking about. The environmental data that you’re talking about is something that the agency has been using very effectively over time to understand the difference, whether our water is getting cleaner or is our air easier to breathe and the like. But what the Evidence Act is asking us to do is saying, well, the program elements that you’re putting in place, the inputs that you’re putting in, and the outputs that we’re getting they’re actually leading to cleaner air, cleaner water. That’s something that we have not been able to do as much as we would like. But the Evidence Act is giving us a big opportunity to take advantage of that.

Tom Temin: We’re speaking with Katherine Dawes, she’s acting evaluation officer at the EPA, and a recipient of the Data Coalition’s Datum Awards. And tell us more about that the Datum Awards – and what do you think you did to deserve this one?

Katherine Dawes: It was very interesting to get the – it was a complete surprise, I did not expect to be getting this award for the evaluation officer function. The Data Coalition, as you know, has been an organization that is advocating for actually better quality data. And then actually the use of better quality data, and with the evidence that they wanted to recognize that this kind of work and these different roles that we have implemented the Evidence Act are all important. So they could have just recognized the chief data officers, right? But they decided no, we actually want to recognize the other roles because of the importance of using data. One of the reasons why they said I got the award is I have been part of the federal evaluation community for a number of years advocating for better evaluation policy at the federal level that will allow more federal agencies to invest in program evaluation and support evaluation to help us continuously improve government. And because of the work that I’ve done there as part of this network, and I’m just representative of a broad network of people who have been consistently investing in this.

Tom Temin: And you mentioned that this job is required by the Evidence Act. Is that full time for you? And if so, what did you do before this?

Katherine Dawes: It is is full time for me. I’ve done a number of different jobs at EPA. I was the director of the Evaluation Support Division, which was charged with supporting evaluation studies and evaluation capacity building across the agency. I did that for a number of years. And then the agency transitioned for a time to focusing very strategically and very deep investment in Lean and Six Sigma tools and building a management system for continuous improvement. And I was a part of that as well. I transitioned for a time out of program evaluation. But now I’m back to it, because the Evidence Act said, “Hey, we need evaluation, we need people who are experienced with evaluation and to promote it within the agencies.” And so I came back to it after being away from it for a while.

        Want to stay up to date with the latest federal news and information from all your devices? Download the revamped Federal News Network app

Tom Temin: And you must be impressed over the years with how much the technology has improved the ability to gather data, and to put data together and to apply tools to data to understand what it’s telling you. That’s really come a long way, hasn’t it?

Katherine Dawes: Oh, my goodness. So I happen to teach program evaluation at the George Washington University for the Trachtenberg School [of Public Policy and Public Administration]. And I like to tell my students, “Hey, I used to actually open up boxes with pieces of paper in them. That’s how we did the data.” We did not have Excel, we did not have, that’s even basic and we did not have some of these tools for data visualization that we have now that are really, really exciting. I mean, it’s an extraordinary change in the role of supporting evidence building and it’s really an exciting – actually, it’s an exciting component of being an evaluation officer in the federal government right now is how do we bring all of these new tools to bear to help advance decisionmaking at the agencies?

Tom Temin: Katherine Dawes is the acting evaluation officer at the EPA and she’s a recipient of a Data Coalition Datum Award. Thanks so much for joining me.

Katherine Dawes: Thank you, Tom.