How contractors can make sure they say what they mean when using artificial intelligence to write proposals
It takes a lot of thought to buy artificial intelligence. Contractors trying to sell the stuff, need to put some careful thinking into their proposals and not rush...
It takes a lot of thought to buy artificial intelligence. Contractors trying to sell the stuff, need to put some careful thinking into their proposals and not rush through them. Then there’s an even bigger question for contractors: using AI to draft the proposal itself. Anything could happen. For some tips, Federal Drive with Tom Temin spoke with attorney Craig Smith, a partner at Wiley Rein.
Interview Transcript:
Tom Temin Is that something that clients are asking you about these days? Do contractors want to apply artificial intelligence to, I guess, generative AI to the writing of proposals?
Craig Smith I think it’s on everyone’s mind whether I talk to clients or I talk to peers in industry. It’s something that feels maybe a little inevitable that at some point this is going to be an unremarkable activity, no different from sending a proposal by email. But maybe the analogy to think about is it’s like self-driving cars. It can feel like something that’s coming, and it’s felt like something that’s coming for a while. But right now, with where we are with the metaphorical rules of the road, you don’t want the car that’s driving itself into wet cement.
Tom Temin Sure. And just on the legalistic side of this, before we get into the bigger question, federal contracts can have hundreds of clauses. Sometimes not every clause is applicable in their contract writing systems will, maybe if they’re smart enough, leave those clauses out. But then there are clauses that, what do they say, operate in law, even if it’s not specifically included in a particular transaction. Could that be a tripping point for AI? Oh, it wasn’t in there, but it operates in law, therefore you’re in trouble.
Craig Smith That’s probably an area where you could see an AI tool being developed to check for those types of clauses that are missing. And I think there are some technical aspects that really sophisticated AI companies are going to figure out, because federal procurement, while it’s not runic text, there are unique aspects.
Tom Temin Comes close.
Craig Smith It feels like it at times. But it makes writing a proposal for a federal agency different from writing a commercial proposal or even a comparable agency in another country or a state. Those are technical aspects that with time and enough work, I think can get solved. But there are other risks out there, Tom, that I think we’re still just seeing a lot of uncertainty around and are going to take some time to sort out.
Tom Temin What do you see as the top risks? In my mind, something that could get you in False Claims Act territory would be one of them.
Craig Smith That’s exactly right. So imagine the term that’s been around probably for decades, you’d probably know it well is vaporware. The idea of writing a proposal about something that doesn’t exist yet. And you could see a generative AI tool writing a proposal for a new widget or a new service that you have no intention of ever offering. The way to guard against that, of course, is the touchstone, I think, of using any AI tool for writing any content for a company which is having humans involved in the process and reviewing to make sure, are we offering these widget? Are these our indirect rates? Is this responsive to what we actually want to offer the agency for this particular opportunity?
Tom Temin Because there’s also the problem or the challenge for these generative tools anyway, is that they decay over time. That is to say they multiply the error that they introduced, and it’s like multiplying fractions. Each time you run it through again, it gets a little further off course to the point where some of them can’t add two plus two anymore after they’ve been run through a thousand cycles.
Craig Smith And that’s an area where I think it’s interesting. What goes in to the black box, sometimes we don’t know all of it for an AI generative tool. And then how the black box works, we don’t always know. And as a lawyer, the uncertainty is something that is what keeps me up at night. And so I think the more proprietary a tool is, the less I know about it, the less interested I am in necessarily shouldering that kind of risk. So I think you’re exactly right. If I don’t know how the tool is working, then I feel less comfortable. But if I have to spend more and more time understanding how the tool works and maintaining it, it’s also, as a practical matter, unclear how much additional value I’m getting out of it.
Tom Temin Right. So the last thing that a bidder should do then is take the solicitation, throw it into the chat, and then mail off to the agency what came out.
Craig Smith That is absolutely right. For a host of reasons. We’ve all heard about the lawyer who submitted the brief with made up cases. You can imagine that any tool that’s out there that’s open source or generally available or even at a modest fee. Any of those tools might propose things that violate one of the basic edicts you learn as a capture or a lawyer. You got to respond to what the solicitation asked for, not for what you think the agency necessarily needs in the best case. There are any number of reasons for just saying don’t send it in, pull it back out and ship it off.
Tom Temin We’re speaking with Craig Smith. He’s a partner at the law firm Wiley Ryan. And you have compiled another list of recommendations. What are some of the other things companies thinking of using AI should keep in mind when writing solicitations or bids.
Craig Smith There are a number of questions that when I huddle with our colleagues, we have something of an informal AI working group, we think about how does the tool actually work? So one question is, what goes into it? Probably the open source items, items available on the web. That’s understandable. But are we getting proposals that are just from our company, which may not be enough to help tune a model to writing proposals for pick your agency? Or is it getting proposals that maybe are collected through Freedom of Information Act requests, which are publicly available in a sense, but maybe a couple of years behind where you need them to be. Or they’re getting proposals maybe from other companies that are using the tool, depending on the terms of service, maybe that’s a permissible thing for the company to do, in which case I’d start to wonder, do I, in a very indirect way, have access to a competitor sensitive information in a way that might create some risk under something like the Procurement Integrity Act. So that’s one question we talk about. We’re thinking another is the flip side of that. What’s the company doing with the information I submit. So is the tool using servers located only in the United States? That would be important to me if I had to submit any export controlled information that might be part of the proposal process. Another question is, will my proposal be folded into the model for tuning, that you mentioned, Tom, or for other purposes? Will my contents, maybe my secret sauce start showing up in proposals generated for my direct competitors? So those are the sorts of questions we talk about internally that we would encourage any company considering a generative AI tool to really think closely about.
Tom Temin Yeah, you answered a question I was going to ask. Is there a possibility of some kind of a community generative AI that is just focused on federal contracting for the contractor community to use. But if something from Lockheed is going to infect something that Northrop is trying to say, or whatever the case might be, then probably there wouldn’t be too much uptake of it.
Craig Smith I think that’s the challenge that any company that is interested in offering an AI tool they face is how do we how do we get enough proposals to where the tool is useful, but not in a way that any particular contractor feels uncomfortable about what we’re doing with it. And then there’s another component to this, is just having proposals only tells you how proposals get written. It doesn’t tell you which ones are good. And so the other question is what are the evaluations of these proposals? Sometimes you could get that from public information like Government Accountability Office bid protest decisions will often have the ratings included, but that may be only a portion of it, and that’s only a fraction of a fraction of federal procurements. So how we’re going to have access to that, and then also making sure there’s access to the solicitation. What’s the proposal responding to? Many are available publicly on the system for award management, but many are not available on publicly available tools that I can just find through an Internet search.
Tom Temin So maybe the real question is, if you’re trying to save time here, and time is money for a law firm or for a contractor, is can the ChatGPT, for lack of a better word, the generative AI make this proposal for me? And then I have to spend a lot of time reading it and vetting it with the human brain and the people that really know what they’re doing. Will that process be shorter than simply writing it from scratch? Maybe that’s the essential question.
Craig Smith I think that’s part of the question. And then the other part of the question is, what risks do I take on top of the kind of costs going one way or the other? And I want to make sure listeners understand the answer for me is not, no, we’re never going to do this. I can see a world where using AI to generate parts of proposals could be as common as an agency, using AI to help with parts of their evaluation. What I think of as more a tool, no different from using a macro in a spreadsheet software or a formatting function in word processing software. You wouldn’t just hit the button and then just never look at it afterwards. There would still be human review, and you have carefully vetted the software before you use it.
Tom Temin Craig Smith is a partner at the law firm. Wiley Ryun, thanks so much for joining me.
Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.
Tom Temin
Tom Temin is host of the Federal Drive and has been providing insight on federal technology and management issues for more than 30 years.
Follow @tteminWFED
