Contracting

Here’s what you did not know about contract award protests

The government releases too little information about bid protests.

Tom Temin@tteminWFED

February 25, 2022 1:13 pm

9 min read

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

The government releases too little information about bid protests. More information would help companies and the government craft better procurement policies and measure them over them. Those are the contentions of the Federal Drive with Tom Temin‘s next guest, George Washington University law student Will Dawson. He wrote a program that analyzed 7,000 protests spanning 20 years. And he’s not even yet a lawyer.

Interview transcript:

Tom Temin: Mr. Dawson, good to have you on.

        We want to hear from federal technology leaders about how you are addressing current cyber demands and how your agency is making the most of its resources to improve its cyber posture. | Take our quick survey

Will Dawson: Thanks so much for having me, I really appreciate it.

Tom Temin: And your paper has really made its way around legal circles concerned with bid protests. Tell us what it is you’re setting out to do here with this piece of research.

Will Dawson: So the original point of it was simply to analyze some very fundamental information from Joe’s bid protest reports. After my initial findings, I presented it to some of my professors at GW, and they expressed interest in trying to find more data. And I was able to generate that. And it sort of went from there and discussing what practitioners I found, there’s a lot of data missing, and eventually just came to the realization of exactly how little information about the macroscopic bid protest process there was currently available.

Tom Temin: So if there was so little information available, what were you able to analyze, then?

Will Dawson: So I was able to analyze the protest decisions issued by the GAO. Unfortunately, the Federal Circuit decisions were not available because of some programmatic nuances. These types of programs do best against text that is very formulaically laid out. There are aspects of the GAO decisions that favor that. Unfortunately, the [Federal Circuit] decisions are a little bit more dynamically written. And so they’re not as able to be processed.

Tom Temin: But if the GAO is not giving out enough information about the cases, then what I’m driving at is then how were you able to get that information and analyze it?

Will Dawson: That’s a good question. So basically, what it was, is that they only issue nine data points every year in their congressional reports macroscopically on what’s going on. So my program was able to do was to dive into each individual case, extract data from that, and then collate that data and analyze it. Once again, on a macroscopic level, I suppose they do technically release the data. But it’s distributed across such a wide number of documents that it’s not really feasible for someone to ingest and analyze it in their head as it’s presented. So I was basically able to extract information from that and condense it into a report.

Tom Temin: So almost like a bot approach to go across those documents and grab that data and put it into an algorithm. Would that be a fair way to put it?

        Read more: Contracting

Will Dawson: Yes, yeah. Thank you. That’s a good way to put it.

Tom Temin: And what were some of your top line findings, then, when all of the data can be analyzed?

Will Dawson: My primary findings — and the paper is available now on SSRN; I won’t go into too many of the details — but it was largely there are 26 different protest grounds that I found over time that the GAO tends to sustain. The top four grounds for sustained protests were unreasonable valuation solicitation. The solicitation was awarded or rejected for reasons outside the scope of solicitation, agencies’ conclusions were unsupported by the record, and then there’s a lack of meaningful discussions being held. So I mean, I think that’s really helpful for practitioners, right, to look at, okay fine, there’s an error in this protest, but it’s something that we’re going to be upheld on with the GAO. I think the most interesting finding personally was not actually one that I necessarily generated data on, but it’s the question of when a protest is sustained, there’s a specific type of protest end up — is that likely to result in the protester being awarded the contract downstream? Of course, that a protest is sustained does not mean a contract is and subsequently get awarded to that protester. So that, to me, I think is the biggest piece of missing data, right. As a business owner, are you being effectively rewarded for raising this concern with the government? Which I appreciate may not be an attractive framing to some, but I think ultimately, for business owners, that’s the question that they’re pursuing when they file a bid protest.

Tom Temin: In other words, even when they win, they lose.

Will Dawson: Exactly.

Tom Temin: We’re speaking Will Dawson. He’s a third year law student at George Washington University. And how do you think this information, if it was available widely, could change the process or help people that are bidding and protesting and maybe help the GAO itself?

Will Dawson: So I think the information that I’ve processed is helpful in analyzing long term trends. What I would really like to see is that the program that I wrote is also able to be applied forward, and I’ve set it up to be plugged in to the existing system and extract data from it. And I think what would be most helpful is identifying, year over year, what trends are evolving for policymakers, as well as contract attorneys to advise their clients, like I said, about what’s being upheld. I think the most important thing really is for assessing the health of the bid protest system. You hear a lot of people that are proponents of it, and detractors of it, but ultimately it’s impossible to make an informed decision on this when we don’t have enough information. I like to equate it to a fire department whose only annual reports are how many fire trucks are sent out into the field. That’s an interesting figure. Without knowing how many actual fires there are, or how many alarms were set off, right, that figure is useless. Equally, the information issued in the GAO’s annual reports are interesting, but they lacked the broader context to really inform people about what’s going on in the system in substantive ways. Because ultimately, bid protests are meant to be a regulatory mechanism. So without knowing how many of them are validated, and how many of them are wasting government’s time, you can’t really say what I don’t think the system is broken. And yes, they do tell you how many protests are filed every year, but you can have multiple protests on a given contract. So you can have a highly defective contract that’s creating a lot of protests; you can have a lot of defective contracts and only getting one protest. And currently we can’t determine that number, and as such, the system can’t be improved.

        Sign up for our daily newsletter so you never miss a beat on all things federal

Tom Temin: And just a detail point, you looked at protests that were lodged after contracts were awarded. That is to say, not at the solicitation stage.

Will Dawson: It was both at the solicitation stage and after award. To my understanding, I have been able to analyze every published merits decision from 2000 to 2019, both pre- and post-award.

Tom Temin: Because pre-award and solicitation-level protests have really been on the rise, especially for some of the governmentwide acquisition contracts that agencies are trying to establish. So that’s a good piece of insight. And then you’ve also proposed a type of form, or data gathering framework, in future protests that you propose that could maybe give this information from that one.

Will Dawson: Yes. So to my original point about the program that I wrote, we have information at a very, very high level with those nine data points. And then we have the discrete merits decision. Merits decision has only covered about 25% of protest that are filed. And so what my program proposes is to extract data on every single protest that’s filed. Moving forward, you would end up with 23 different data points — 15 would be entered, 8 would be extrapolated from there. What that would empower you to do is then understand what happened with denied protests and sustained protests pre-award and post-award, and just give you a lot more granular insight into the system. The way that it’s set up, it programmatically extracts the data, so at most, depending on the data that’s available — of course, I don’t know what the GAO actually has on hand — at most, it would be a minute or two of additional work per protest.

Tom Temin: And if a reward is changed to a different contractor, or that kind of thing happens that you said doesn’t happen necessarily. But if it does switch, that’s beyond the GAO involvement, though, isn’t it? That’s between the agency and the original bidders. So it seems like you would need the contracting officers, perhaps, to have a reporting requirement, if something changes in terms of the award after the protest is finished.

Will Dawson: Yeah. So I guess this is what I was getting to about the system integration. Depending on how much information is shared, you should be able to extract that data against some of the information that’s published by USA spending. One of the most compelling things as a way of framing this issue is that the GAO releases nine data points every year. USAspending releases 280 data points on, you know, millions of contracts a year. So they’re releasing, I think, like 1.8 billion data points, right, on everything that the government is doing. GAO is giving us nine. You know, there’s sort of a Goldilocks zone there in between. The government’s clearly not adverse to this information being out there. And some of the information, as I said, would have to be drawn off USAspending or other sources. But I do think it would be best for the information to come out of GAO, just because they are sort of the de facto protest clearing.

Tom Temin: And has GAO seen your report?

Will Dawson: I’m not sure. I heard from people that they have. They have not contacted me. I’m not sure at this point.

Tom Temin: All right, but give them a call. They tend to pick up the phone and talk. So that’s my advice. Having dealt with them quite a bit. They’re they’re very open to that kind of discussion. Will Dawson is a third year law student at George Washington University. Thanks so much for joining me.

Will Dawson: Of course. Thanks so much for having me.