DHS’ list of AI use cases found inaccurate, GAO says

Among the dozens of mandates in recent executive orders, there is bunch about artificial intelligence. Among them, a requirement for agencies to inventory their AI use cases for purposes of cybersecurity. Perhaps strangely, the Homeland Security Department put together an inaccurate list, according to the Government Accountability Office, which found a few problems. For more, the Federal Drive with Tom Temin spoke with GAO’s Director of Science, Technology Assessment and Analytics, Candice Wright.

Interview Transcript: 

Tom Temin And so their list of AI use cases for cyber security was somehow inaccurate. Is this a big deal or a little deal?

Candice Wright Well, we think it was important to really take a look at how DHS was implementing AI for cybersecurity use cases. They had an overall inventory of 21 AI use cases in their 2022 inventory, and they had identified two specifically related to cyber. This was a first attempt for GAO to apply its AI accountability framework, which was developed in 2021. And so, with agencies beginning to publish their inventories, we thought it was a good time to start taking a look at their efforts to implement AI.

        Join us May 7 at 2 p.m. ET where government and industry leaders will discuss the benefits of digital-centric engineering vs. document-engineering approaches, best strategies and lessons learned, sponsored by Arcfield. | CPE eligible

Tom Temin Right. So, this framework then was developed way ahead of the executive order on artificial intelligence.

Candice Wright Indeed, we recognized that with the growing use of AI and lots of interest and curiosity about it, we thought it was important that there would be a framework to help entities, particularly those who have an oversight role, but also those who are managing and overseeing AI projects, to be able to identify key practices and considerations that should be thought of as they’re developing, designing and deploying such systems.

Tom Temin And before we get to the list of cybersecurity related use cases, you have a long list of best practices. I think there were something like 16 of them, and DHS was only following four of them. Tell us about the elements of the framework and what they help an agency accomplish.

Candice Wright So the framework is actually organized around four key principles: governance, data, performance and monitoring. And really with these practices, the framework is really to help ensure that there is accountable and responsible use of AI. The particular systems that we identified that DHS had on its inventory, again, there were 21 overall, but there were two that they identified as being cyber related. As we said about to do our work, we began having conversations and discussions with the agency officials, and it became quite apparent that one of the two systems that they had identified related to cyber actually did not have characteristics of AI at all. And so, through our discussions with the agency, you know, we thought it was important for them to really take a look at the processes that they have in place for determining what systems actually end up on their AI use case inventory.

Tom Temin Well, do you think that they were characterizing something as AI, even if it was not just as a way of saying, oh yeah, we’re doing AI here in cyber security?

Candice Wright I think there’s just been so much attention and focus on it of late. People may not necessarily have the necessary background to fully take the necessary steps to assess whether or not the system truly is, or the capability truly is AI. And so, given a lot of the attention and focus that’s been happening, you know, certainly there’s a rush to submit information for these use case inventories. But it’s really important that if you have individual components within an agency submitting such information, that there be a body that is verifying and validating the submissions and ensuring that they really are characteristic of AI use cases, right.

Tom Temin Agencies may be running into the tendency to just characterize simple automation, or even orchestrated multiple automations as AI when it’s not really strictly AI.

Candice Wright Exactly. And so, one of the models that we identified and that they acknowledged was not AI was one that had a predictive modeling component, but certainly was not AI. As we explore that in greater detail with the agency.

        Read more: Artificial Intelligence

Tom Temin  And if they have mischaracterized one of their inventory items, then what’s the practical effect of that?

Candice Wright Well, we think it’s important that if agencies are going to be publishing this information and making it available to the public to indicate what their use cases are, that that information be accurate, it be complete, and that it be reliable, because that’s a really important element in establishing transparency, but more importantly instilling public trust and confidence.

Tom Temin We’re speaking with Candice Wright. She’s director of science, technology assessment and analytics at the GAO. And the fact that just a couple out of one agency sample was inaccurate. Could that be an indicator that there’s maybe something systemic in the government that needs to be tightened up?

Candice Wright What it definitely raises the question that perhaps, you know, more should be done to take a look at what use cases were submitted and ensure that it is accurate. And that and necessary updates are occurring.

Tom Temin And again, you have DHS fully implementing four of 11 key practices that are related to those governance data performance and monitoring areas. So even if those applications actually were AI, they still weren’t executing up to snuff.

Candice Wright Exactly. And so, we specifically analyzed and reviewed 11 practices in the agency’s implementation of their system, which ended up being the automated PII detection system. And so, we focused on looking at that to identify the ways in which they were implementing these practices. As you mentioned, they were fully implemented before but had mixed results in implementing the remaining seven. I one of the things that we identified is that data sources and reliability in particular, were the areas that required the most attention. And much of this was because we really couldn’t find any evidence that the agency had addressed any of the key considerations for documenting the sources of the data and the origins of the data for the system. And in addition to that, we also found that they didn’t have any evidence that any data reliability assessments had been conducted.

Tom Temin So those are pretty basic practices for careful AI.

        Want to stay up to date with the latest federal news and information from all your devices? Download the revamped Federal News Network app

Candice Wright Well, the underlying data, such a key element, especially when you’re going to be using these systems to make recommendations, made decisions. And so, it’s important to ensure that the underlying data are sound and that they are representative of the solution that you’re trying to attain.

Tom Temin And what were your main recommendations for homeland in this case?

Candice Wright So for DHS, with respect, again, to the one system that we were actually able to review because we could confirm that it was AI, we made eight recommendations for them to really focus on, first of all, updating their inventory to make sure that it is accurate, make sure that they’re expanding their process to not just receive information from their components, that something is AI and should be on the inventory, but really to validate the accuracy of that submission. In addition, with respect to the other issues that we identified around governance, around data, as well as performance and monitoring, many of the recommendations there were about ensuring that they have the appropriate documentation to provide evidence, especially for those in the oversight community, but also for the agency as well, who’s managing and overseeing the implementation of the system, because you’ll often have people coming and going on these systems. And so, you want to make sure that you have that documentation to be able to refer back to as the system is being developed, but more importantly, being launched and operated so that you can ensure that it’s operating as intended and achieving the expected outcomes.

Tom Temin And they pretty much went along with you.

Candice Wright Yeah. So, the agency actually did agree, and concur with the eight recommendations. And so, we’ll be monitoring that over time to see how they implement the recommendations.

Tom Temin And will you be looking at other agencies inventories to make sure that they characterize AI that really is AI and then are following the best practices?

Candice Wright Well, I certainly can say that AI oversight is going to be a huge part of our work going forward for the foreseeable future. Again, as you look at the ways in which agencies are starting to adopt use of the technology. Last year, we actually had another team that issued our government wide review on agencies implementation across the 23 largest civilian agencies. And there they found that over 1200 use cases had been identified by these agencies. So, we’ll be continuing to monitor what’s happening there. In that report, we actually made 35 recommendations to 19 different agencies. So, a lot to look at. And certainly, there’s a lot of congressional action and attention on this topic. So more to come in the future.

Tom Temin And by the way, even though you are a congressional agency, does GAO have an inventory? And would you live up to what you ask of the agencies on there?

Candice Wright Well, certainly GAO is also walking the talk on this topic. We actually published, recently published our inventory of AI use cases where we have about eight use cases that we have identified. They’re in different stages of concept exploration to prototyping. One project in particular organizes large volumes of text that might be found in various public documents, such as public comments from regulations.gov. And we think that these are really important steps to help us in gaining insight about the benefits and the limitations of using AI, but more importantly, that it can help us in evaluating how other agencies are using AI and help us in providing our oversight support to Congress.

Tom Temin Fundamentally, agencies have to understand what AI is and what is not AI. That’s the basic step in getting better at AI.

Candice Wright There certainly is going to be a need to make sure that we’re building knowledge within the federal government to understand the technology. And as you said, what is or isn’t, but more importantly, ensuring its responsible practices to develop the system, design the system, deploy it, monitor it, and ensure that it’s performing as desired and intended.

Tom Temin You might say AI is a hammer, but not everything out there is a nail.

Candice Wright That’s certainly one way to look at it.

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.