AI will not solve the problems of Big Data

You cannot turn your head to politely sneeze without encountering someone who isn’t a specialist in AI or a technology company that claims to have a product that uses AI to solve a significant business problem. 

AI—whether it is people who know it or companies that are integrating it or enterprises that are using it—seems to be about as abundant as the oceanic water on the planet, or as prevalent as the stars in the night sky. But as much as AI seems to be everywhere and solving everything, remember that there are different types of AI that do different things, it is a technology tool, and AI cannot do it all, and all by itself.  

One big problem that AI cannot solve is the problem of Big Data: the lack of data, the overwhelming amount of data, the accessibility of the data, the gaps within the data, the different structures of the data, the timeliness of the data, or the accuracy of the data. 

Let’s look at a framework for big data that I’ve used previously, such as in my first book, and apply it to the challenges for AI:

• Volume: Enterprises have a difficult time storing the vast amounts of data they are collecting. Data storage is not a cost-free endeavor. As more devices are connected to the internet and more (and more) data is collected, the sheer volume of data will continue to be overwhelming for enterprises to maintain and keep long enough for AI to analyze.  Whether it is the longevity of the data or the range of the data, the volume of the data is becoming a problem for enterprises.
• Variety:  Even text data is often found in various format states, such as X12-EDI, CSV files, Excel files, Word documents, PDF documents, emails, and in different supply chain systems (e.g., ERP, EDI, WMS, TMS, etc.).  And some of these format states undergo natural changes that must be maintained on a regular basis and kept up-to-date when customizations are performed. Is the AI using raw data or aggregated data, because it will likely matter for the final output. 
• Velocity:  As businesses need transactions to speed through their systems faster and faster to keep up with operational—and customer—demands, we may find that this speed is simply too fast for AI models to absorb and process, given the computing time required to analyze the data. Will this necessitate limiting the volume of data to accommodate the timely processing of the data?
• Veracity:  What data represents the “source of truth” for the AI model, and how available is this data? Disconnected systems could pose a problem in ascertaining which system’s data is the “truth source.” Replicated data can pose a challenge if the timeliness of the data in the “source of truth” system is materially different from where a copy of it exists in a (more accessible) data warehouse.  And certainly, all data—whether internal or external—should come from a trusted source, whether that source is a person, system, or entity (customer, supplier, vendor).
• Relevancy:  Care should be taken that any AI model is focused on the data relevant to the problem at hand. Introducing unrelated data could distort the output, just as forgetting, foregoing, or limiting (filtering) the data could likewise cause misleading results. 

According to a report by Impinj, which is an implementor of RAIN RFID, found in a survey of 1,000 supply chain managers performed in August 2024 in order to assess the state of supply chain integrity for 2025 that just 33% of the respondents consistently obtain accurate, real-time inventory data. Furthermore, the top challenges and concerns in implementing AI are reported as:

• Data accuracy (43%)
• Data availability (39%)
• Real-time data access (36%)

AI models can handle lots of data and certainly that’s what they have been built and trained on.  AI can help identify problems and revelations with the data set, but AI alone will not solve the problems of Big Data. For companies to be successful with AI, the foundational data needs to be accurate, complete, timely, and trustworthy.  Before you engage on an AI project, make certain that your software systems and the data within are up to the task.   

To read the full Impinj report, go to: https://www.impinj.com/about-us/news-room/2024/new-impinj-research-reveals-top-headaches-for-supply-chain-managers-for-2025     

SC
MR