Data Benefits AI; AI Benefits Data

Unlocking the Full Potential of AI: The Crucial Role of Data

The relationship between Artificial Intelligence (AI) and data is symbiotic. AI relies on high-quality, well-structured, and accessible data to function effectively, while data, in turn, becomes more valuable and actionable with the help of AI. This interdependence is highlighted in Amazon Web Services’ (AWS) recently published Blueprint Executive Blueprint for Enterprise AI Transformation, which emphasizes the need for organizations to rethink their data strategy to fully leverage AI.

The blueprint’s authors note that as large language models and foundation models become more widely available, the differentiator lies not in the model itself but in the quality, structure, and accessibility of the data powering it. This means that organizations must open up information from a variety of new or previously underused resources, including unstructured and multimodal formats such as video, audio, text, and code, with real-time accessibility.

Overcoming Data Challenges

However, managing and transforming data to support AI is a significant challenge. According to a survey by the IBM Institute for Business Value, only 26% of chief data officers are confident that their data capabilities can support new AI-enabled revenue streams. The top data barriers they face include accessibility, completeness, integrity, accuracy, and consistency. To overcome these challenges, organizations must modernize their data architecture, break down silos, and establish common governance structures.

Ashwin Patil, principal and data engineering and analytics practice leader at Deloitte Consulting, notes that “agentic AI is the technology most impacting data management today.” Agentic AI can significantly augment or automate the manual process of profiling data, performing quality checks, building business rules, and integrating data across applications. This can help organizations to unlock the full potential of their data and support AI-enabled revenue streams.

The Role of Large Language Models (LLMs) in Data Management

Large Language Models (LLMs) can be deployed directly into a data infrastructure, introducing a semantic understanding layer that fundamentally changes how unstructured data is stored, discovered, classified, and acted upon. Jim Liddle, AI entrepreneur and former chief innovation officer of data intelligence and AI at Nasuni, notes that this goes beyond traditional AI assistants layered on top of legacy file systems. Instead, it enables organizations to interpret content, context, and usage patterns, unlocking dormant data that has historically been trapped in archives or file shares.

LLMs can also help experts speed up the time to discover, deliver, move, and troubleshoot data across data management platforms, while lowering the bar of entry for less experienced users. Aron Semle, chief technology officer at HighByte, notes that LLMs are making data more accessible to a broader audience, which could grant decision-makers more control over their data and help them use it more effectively.

Implementing an AI-First Data Strategy

To properly integrate data into an AI-focused system, organizations should follow a structured approach. The AWS authors recommend five steps: conducting a data audit, modernizing data architecture, building internal capabilities, ensuring a human is in the loop, and measuring progress. By following these steps, organizations can unlock the full potential of their data and support AI-enabled revenue streams.

Conducting a data audit involves establishing use cases that take advantage of low-hanging fruit, unifying relevant data in a secure and scalable storage solution, and implementing appropriate guardrails. Modernizing data architecture requires breaking down silos, defining data product owners, and establishing common governance structures. Building internal capabilities involves upskilling data teams in prompt engineering, vector databases, and responsible AI, while training teams across the organization on AI fundamentals and responsible use.

Ensuring a human is in the loop, along with the use of LLMs to provide feedback, can help continuously monitor and improve data quality and model performance. Finally, measuring progress involves tracking business outcomes, operational performance metrics, and data and trust metrics such as retrieval precision rate, factual consistency score, and daily active users.