Whether we realize it or not, AI or artificial intelligence is all around us. E-commerce sites like Netflix or Amazon are a few of the more prominent users of AI — enjoy those additional product recommendations that seem to know exactly what you need? Those are based on data inputs to enhance the user experience and increase sales. Similarly, Siri, Alexa, and even self-driving cars rely on artificial intelligence, shifting manual human tasks to a machine.
But how accurate is artificial intelligence, and what should your organization consider before making business decisions around it?
Understanding Artificial Intelligence (AI)
Before we dive into the accuracy of artificial intelligence (AI), let’s first talk about what AI is, and what it isn’t.
In its simplest form, artificial intelligence (AI) is the process of using computers and data to understand human intelligence. Using computer science, robust datasets, and machine learning algorithms, AI enables complex problem-solving by analyzing large amounts of data, identifying patterns, and making predictions. But this begs the question, “can machines think?”
The answer is yes — and no. A machine can imitate human thinking; however, artificial intelligence (AI) and machine learning is about statistical, not functional relationships. Statistical relationships work with random and randomly determined variables. Functional relationships also work with variables, but they are neither random nor stochastic. Let us illustrate further.
When Newton observed an apple falling from a tree, he developed a functional relationship between force, mass, and acceleration due to gravity. With this relationship (F=ma), many things can be “determined” from that equation.
Unfortunately, AI does not work that way. In this scenario, AI would observe thousands or millions of different things falling from different types of things and different heights. Based on “seeing” all that happens (being trained), the AI system could then make predictions (inference) about how other things will fall. Thus, the AI system (or “thinking”) is statistical. Specifically, it will get some things right and some things wrong. The question or risk that follows, then, is how “accurate” is artificial intelligence in making predictions?
The Accuracy of Business Predictions Using Artificial Intelligence
The demise of many organizations using artificial intelligence (AI) stems from fallacies regarding the accuracy of the model. When brands get narrowly focused on accuracy, they spend valuable resources on over-optimizing the system, and as a result, many organizations lose out on massive opportunities.
A perfect illustration is Netflix. In their quest to improve AI algorithms, Netflix offered a million-dollar prize to anyone who could improve their "watch next" recommendation algorithm by 10%. The competition ran for three years, and ultimately, two teams built algorithms that exceeded the accuracy threshold. But after awarding the million-dollar prize, Netflix ended up dumping the winning algorithm. Why? It was too costly to implement. “The increase in accuracy on the winning improvements did not seem to justify the engineering effort needed to bring them into a production environment," per the Netflix public announcement. So instead, Netflix implemented a lower-ranked, simpler, and less-costly algorithm that improved accuracy by 8.43% — 1.5% less than the winning submission.
In business systems, rather than asking how accurate is our artificial intelligence model, the more important question is how accurate does the AI system need to be?
In the current state of AI, the logical assumption is that with more and more data along with more computing power, training an AI system can become more accurate. But, the output must be balanced with the input.
Defining Business Objectives for AI Systems
AI can be a great intelligence tool, but an organization can also waste a lot of money if objectives aren’t clearly defined. What problem are you trying to solve?
More data and more computing power also translate to more specialized skills to “wrangle” that data — and at a greater cost. So, company executives needs to understand that not only does the objective need to be clearly identified, but the data needs to be trained to that identified objective, AND the required accuracy needs to be defined.
For instance, people creating a fun system based on AI (i.e., recognizing dogs and cats) may not need to be that accurate. However, a system used by a veterinarian to determine a symptom in a dog should be more accurate. Likewise, a self-driving car that can determine the difference between a dog in the road versus a pile of leaves needs to be exceedingly accurate. The accuracy required for a specific intelligence system directly relates to data, training time, effort, and computing power needed and is a critical part of business cases.
It’s easy for people to get caught up in the excitement of artificial intelligence (AI) when they start learning about what’s possible. But remember, you’re designing an AI solution for a specific objective or getting help to reach a particular goal, from efficiency gains to increasing customer experience and online sales to reducing operating expenses, to name a few.
When deciding your business case for using AI, consider how mission-critical accuracy is to the project and then take an educated, savvy approach to define the outcomes you need.
Implementing Data + AI Accuracy Checkpoints
To judge AI merely by the accuracy percentage is a misleading metric. After all, AI is not always right. Your success with artificial intelligence will depend on your expectations, how well you’ve identified your objectives, and the extent you’ve trained your data. Do you have a good data foundation? Are you including all relevant inputs to produce the best output? Are your performance metrics aligned with your objectives?
AI success is the continual optimization of human and machine actions. Use these accuracy checkpoints to evaluate what’s needed in your model.
1. Define AI and Human Roles
AI can help with processing large sets of data, but in many applications (such as a medical diagnosis), the human still needs to make the decision. Determine the hand-offs, what you expect from artificial intelligence, and where human intervention is still needed.
2. Understand Data + Intelligence Limitations
Artificial intelligence systems are based on technology to solve problems and produce results. Focus on the defined business goal and ask how AI can help to achieve the most accurate results.
3. Evaluate Existing Processes
Run a series of trials on your AI model— it should produce consistent, repeatable results. If it’s not, you may have gaps in your processes, and your AI system may not be ready to launch.
4. Check Data Quality
The adage, “garbage in, garbage out,” is especially true in artificial intelligence (AI). Poor data will deliver poor results; be mindful of the data quality you are analyzing. If you have large data sets, tools such as data lakehouse services may be useful.
5. Understand the Model’s Process
You should be able to trace the “thought process” of your model and understand how it arrived at the results. If you find your AI recommendations are off, use this trail to track down where errors may be occurring.
6. Continually Add Data and Adjust your AI Model
Algorithms thrive on learning. Keep adding new data to your artificial intelligence system to ensure it continues to improve and produce relevant and accurate results.
The Power of Artificial Intelligence
Artificial intelligence coupled with machine learning has the ability to transform a business. However, executives need to understand obtaining the highest AI accuracy percentage is not the goal. Rather, balancing the accuracy needed according to the defined objective will produce more efficient outcomes in the long run.
Interested in artificial intelligence, machine learning, deep learning or big data? Contact us! To learn more about BOSS AI, check out our overview pricing documentation.