Congratulations, you’ve identified processes to innovate with data science and AI. After this step, you’re almost ready to leave the AI exploration phase. To get ready for the experimenting phase, you need to get a high-resolution, bird’s eye view of the project. In this post, we share how to use the AI canvas for this. Follow the steps in this blog to be well prepared and start the fun: letting AI developers put their money where their mouth is by building a proof of concept.
Getting the right people around the canvas
Ideally, all relevant stakeholders are represented when filling in the canvas: end-users, application domain experts, operators, process owners, product managers, data managers, marketing, and many more. Even better is when a data scientist or AI expert is also present to help guide the session. Can’t get everyone in the same room right now? No problem! This guide aims to get you started on your own. You won’t be able to fill in all of the fields, but this will show you your blind spots. At Verhaert, we can help you fill these in at any stage.
The 4 main components
The canvas has 4 main blocks. Your AI story starts with defining your goal. Next, we define what we need to predict before tackling the specifics of the learn block. Finally, you’ll evaluate the impact of your system after production. These four main parts will be explained more in depth in the following sections. To make this guide practical, we will have a running example to help you understand each canvas section better.
Goal
The goal captures the added value of the AI solution. This step is all about defining the value propositions of your system. To help you with this exercise, ask the following questions:
- What are you trying to do?
- Why is it important?
- Who is going to use it?
- Which of the objectives are we serving?
- How will this algorithm benefit you and other end users
- Who is going to be impacted by this? (mention workflow and interfaces)
Need some inspiration? Check our example here!
Predict
The Predict part will make you think more in-depth about the desired outputs of your system. During this step, you need to imagine that the algorithm already exists. This will help you think about how these predictions will be used, how they generate value and how you can build trust in the predictions. Take your time to go through this process and ask AI experts for help if necessary. Now, let’s dive deeper into the subfields of Predict!
Prediction task
What would you like to get out of the AI algorithm? This is what we mean by output definition. By defining your desired output, you also define the type of task you are facing. You can use the AI use case chart to help you do this, don’t worry about its feasibility yet.
Summary ML/AI capabilities © Verhaert
Imagine you have your AI algorithm that you now want to start getting predictions from. This process is called inference or scoring, the predictions the output. And in order to get an output you will, of course, need to give your AI algorithm an input.
Defining the input to your AI algorithm can be scary. A good way to start is by listing all the information humans are using currently to do the same task. Domain experts are most knowledgeable about the domain because they can think more easily about what inputs would be useful to predict your desired output. Will this be an image? Maybe it is an excel sheet with numbers?
Decisions
In this step you need to think about how you will use the predictions. How can you fulfill your value proposition with the predictions? Make sure to mention the relevant decision-making parameters or parameters you know will influence your process.
Making predictions
How often would you like to get a prediction? Every second, every hour, every minute? Is it the end-user who chooses to get a prediction? Answering these questions will let you determine the prediction schedule. The prediction policy is a bit trickier and depends a bit on the input you defined. The prediction policy can state that you need to have captured a certain amount of quality input before you can make a prediction.
Offline evaluation
Your system is almost ready to be used in production! Before you do so, you still need to evaluate how good the predictions actually are. This part helps you define how to test your system. You should create representative test cases, similar to the real-world environment Think about relevant metrics to assess whether or not your system works well. Define acceptance criteria that help you decide when to bring the AI solution into production.
Not sure how to answer these questions? Check our example here!
Learn
Until now you assumed you already have an AI algorithm that makes predictions. You have determined what your desired output is, and you already thought about the input to your algorithm. It is now time to pay attention to how this AI algorithm can become reality, how it can Learn. Don’t worry you don’t need to be an AI expert to fill in this part. However, the help of an A.I expert in this step can still be very useful.
Data collection
Your AI algorithm needs to learn. Finding and collecting the correct data and boundary conditions is an important step in your AI project. You have already thought about the different data sources that could be useful for your problem in the Predict step. Now think about how you could collect data for all those different sources to allow development.
Data sources
The quality and performance of your AI algorithm depends on the quality of the learning data. Data sources make you think about which data will help you solve your problem. Make a list of all the data you, as a domain expert, think is useful to solve the problem. The input you defined in the Predict section can be helpful here. And remember, it is always better to list too many data sources than too few.
Building models
You’re almost done! Now it’s time to think about how your AI algorithm will be built. You will probably need some help from an AI expert for this step. The building strategy defines how your AI algorithm will learn. Can you provide it with examples of input and a corresponding desired output? You and/or an AI expert can think of different types of algorithms that can fit your solution. Maybe your solution needs multiple algorithms? Maybe it can be solved with an optimization algorithm instead of a machine learning approach. The building schedule makes you think about a rough timeline. How much time can you spend in exploring different options? How much time will you spend on refining one option? Will the model be retrained regularly after it is deployed?
Features
Think of features as characteristics that represent data, the same way that you can describe a person by identifying their characteristics. As a domain expert, you probably know which characteristics can be extracted from the data or which characteristics are needed. Make a list of all the characteristics you think can be useful.
Need some inspiration? Check our example here!
Evaluate
Your system is in production, congrats! How will you monitor the performance of your system over time? How can you tell if your value propositions really have the impact you foresaw? The final table in the AI Canvas will make you think about these questions before your system is in production.
- Which metrics could be gathered to quantify value creation?
- Which metrics could be gathered to verify AI model performance?
- How will we set up a IoT/data pipeline to gather this data?
Not sure how to answer these questions? Check our example here!
Congratulations, you’re officially ready to move to the next phase of development: experimenting. Good luck!
