The Biden administration is looking to better understand whether trillions in COVID-19 spending have effectively helped underserved communities recover from the pandemic.
To do so, the administration is tapping into the expertise of agency chief evaluation officers and evaluation offices across the federal government.
The Evidence Act required agencies to stand up much of this evaluation infrastructure in less than three years, but analysis of pandemic spending gives this community one of its first major opportunities to assess whether the federal government is getting the most bang for its buck.
While evaluation officers are a relatively new addition to most agencies, the General Services Administration’s Office of Evaluation Sciences has helped agencies assess the effectiveness of their programs for the past six years.
GSA Administrator Robin Carnahan, speaking at an OES forum Tuesday, said the expansion of these evaluation capabilities across government will help agencies determine whether programs are having their intended impact.
To that end, Carnahan said GSA is launching a series of evaluations into American Rescue Plan spending, in order to determine what spending programs worked effectively and whether they helped ensure an equitable recovery from the pandemic.
“We know that putting this kind of evidence together, gathering that is going to help us make better decisions and improve how government works,” Carnahan said.
OES has completed more than 100 collaborations across 21 agencies over the past six years. Projects include increasing vaccination rates, cutting opioid abuse and reducing federal improper payments.
Pompa Debroy, the strategy and operations lead at OES, said agencies will soon expand the federal government’s capacity to conduct this type of evaluation work.
The Office of Management and Budget expects agencies to post their final learning agendas on Evaluation.gov by February 2022.
Clarence Wardell III, senior advisor for policy implementation and delivery for the White House Domestic Policy Council, said agencies should consider reviews of pandemic spending programs as a “learning opportunity” to continuously improve program design.
“Where we get into trouble is when we presume to know how these programs will actually materialize on the ground, where we presume to know who will access them and what barriers are most are most critical for folks to overcome,” Wardell said.
“There are a lot of questions about, are we reaching all parts of America equitably? And how we can expand our reach and education to seek those services to be able to participate?” Bossie said.
Brittany Borg, acting director of SBA’s Office of Program Performance Analysis and Evaluation, said the agency partnered with OES and its network of city partners and academics to better understand how small businesses were recovering from the pandemic.
“We knew at the SBA that this was an unprecedented opportunity to understand how this aid would reach small businesses and affect them,” Borg said. “Rarely is aid so universally needed across all geographies, all stages of business, all sizes of business, all industries of business. So we knew that this would be an event also that could have disproportionate effects and perhaps exacerbate inequalities that we already see in the small business community.”
Borg said SBA leadership has shown interest in the results of this evaluation work and is “actively consulting on how to interpret this evidence into future program design and implementation.”
Exodie Roe III, GSA’s associate administrator for the Office of Small and Disadvantaged Business Utilization, said the agency is looking at its acquisition data and to better understand barriers that small businesses face in winning federal contracts.
Despite these positive scores, Roe said GSA can still use data to better connect with business owners from underrepresented backgrounds.
“There are businesses out there who have not been as successful in landing federal awards or meeting the demands to even be part of the federal procurement process. We know there are specific business development challenges and information gaps with maximizing opportunities,” Roe said.
Pam Coleman, OMB’s associate director of performance and personnel management, said agencies should feel compelled to ask these types of questions under the Evidence Act, rather than view it as a set of deadlines to meet.
“The culture of curiosity, I think, is what we need to instill, and is the one thing that we may be lacking,” Coleman said.
Andrea O’Neal, senior advisor to the GSA administrator on equity, said agencies are improving the maturity of their data on equity and stakeholder engagement, and in the process, developing an evaluation culture across the workforce.
“It’s a relatively new muscle that government is exercising at this depth and scale. What that has required is a fair amount of capacity building along the way, recognizing where we do have data gaps, or where our infrastructure for this type of work is not sufficient for the kinds of analyses we’d like to do,” O’Neal said.
That evaluation work, however, still has some hiccups. Debroy said OES, in its first year, invested in 130 projects that didn’t come to fruition.
“For every project that launches, there may be three or four that were just glimmers in our agency collaborators’ and team members’ eyes that unfortunately did not pan out,” Debroy said.