Data is the fuel for the advancement of AI

The federal government is continuously tracking and focusing on the advancement of artificial intelligence, even more so with another round of Senate hearings in mid-September. The White House is making it clear that defining standards and best practices for this technology is a priority to ensure safe and ethical use. AI is becoming more prominent across government agencies and is already affecting mission outcomes in key areas such as cybersecurity, healthcare and supply chain rationalization.  

 This rapidly evolving technology offers federal agencies the opportunity to harness its advancements for innovation and security – and at the center of AI’s primary capabilities and offerings is data. AI can be challenging to control, but modern data architectures and data management solutions can assist in securing and implementing AI. Effective data management solutions also support a future where AI can be used ethically and effectively. 

 The current direction of AI

The federal government has multiple projects in process to help agencies enable, implement and utilize AI to support mission objectives. These directions and memos from the government are set to provide guidance for how agencies should properly manage AI. 

In August, the Department of Defense (DoD) announced AI Task Force Lima, a generative AI task force that is committed to helping the department harness the power of AI in a responsible and strategic manner. The government is utilizing AI to solve specific operational problems that federal agencies are running into – however, the complexity of AI is that it is easy to use and difficult to control.  

The U.S. is not the only country making strides with AI. Foreign adversaries are also using this technology to exploit our resources in the cyber realm. Luckily, the Biden-Harris Administration announced a National AI Strategy in development. This is essential as it will provide federal agencies with guidance when working to take full advantage of the benefits of AI while mitigating the risks.  

The transformative power of generative AI affords the opportunity for the nation to make modernized technological leaps and bounds towards advancements; however, it does come with the risk of serious defensive implications. It’s also key to understand the role of data modernization within the AI process – it helps negate the issues that arise from the advancement of AI by enabling teams access to critical data.   

Data is at the core

AI is only as good as the data it is trained on, and sound data management is key to having clean data that can be used for mission success. With the use of data lakes, a centralized repository that allows for the storing of structured and unstructured data, AI can perfect the process and produce the most effective, dependable results. Data lakes also assist in tailoring solutions for complex problems.  

Functional AI relies on data lakehouses – data lakehouses facilitate data literacy and data-driven operations by enhancing trust in data through governance. They increase flexibility to expand AI and analytics while making data more accessible, providing self-service analytics, ensuring data quality and simplifying data security. Sorting and organizing clean, aligned data strengthens AI processes by providing the technology with trusted information.  

Data lakehouses also provide end-to-end capabilities throughout the entire data lifecycle. The advanced data management solution helps organizations run quick analytics on all data; the complex architecture allows running AI and various analytics on the same data without ever moving or locking the data. The advanced analytics of open data lakehouses can provide faster, more accurate and more educated responses to mission needs, all while enabling modern data architectures to problem-solve efficiently.  

With the movement of massive amounts of data, it’s impossible to not think about security’s role in this whole transaction. Government must be cognizant of how AI impacts data privacy while federal agencies consider the challenges of safeguarding personal, customer and company data among the rapidly evolving AI technology realm and innovations that are fueled by data.  

Privacy and security  

A concern of AI technology is privacy and security of the data that is being poured into the technology. It can store users’ past searches and activity to inform future research and development. As security is of utmost importance for government agencies, this must be avoided.   

One option to ensure data privacy is to use enterprise-developed and -hosted language learning models (LLM); open source models can enable agencies to host their AI solutions in-house within their teams without spending a fortune on research, infrastructure and development.  

For example, LLM development enables automation, freeing government employees to do vital higher-level decision-making and essential tasks by allowing the technology of AI to handle lower-level decision-making. Interactions with this model would also be kept in house, eliminating the privacy concerns associated with SaaS LLM solutions, like ChatGPT and Bard.   

In the ever-evolving landscape of AI, it is vital to acknowledge and act upon the significant role that modern data architectures and proper data management approaches provide. The presence of AI is only as successful as its data capabilities. Data lakes and the use of data lakehouses enable the use of AI and LLMs to offset the workload for agencies across the government and manage data more efficiently, all while remaining secure from threat actors.   

Modern data approaches will accelerate the implementation of AI and reduce the chances of creating misinformed solutions. Moving forward, these AI solutions backed by clean, true data will support accurate, timely decisions to assist government agencies in driving their mission forward.  

Rob Carey is president of Cloudera Government Solutions. 

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.