AI is on the roadmap, where do I start?
Marc Lecoultre, Co-founder of wazzabi.ch speaking at Innovation Leaders 2018 event on Machine Learning.
Have you ever heard from your board or management “We need to do something with AI, we need to show the market that we are ahead of the competition”? I hear this very often when I visit my clients for the first time. The next sentence is most often “but we don’t know where to start”.
Through this article, I would like to share my experience of implementing AI-based services for businesses. I practiced AI, or rather Machine Learning (ML) I should say to be precise, for over 15 years. I have worked on dozens of projects in various companies and industries and I can not say that either one is more prepared to implement ML.
You may think that you are reading yet another article on AI and you are not going to learn anything; before turning the page, I would like to reassure you immediately, this is not an article on AI like those you’ve probably already read, you will not find generic facts about AI, not even a description of case studies already seen. I am going to talk about AI for ALL, AI for YOU, without math or theory. I will present a real working example that will demystify the whole subject.
I would like to convince you that AI is concrete, and not only for global or high-tech data driven companies. Too often, my clients get lost in the overabundance of information, in my opinion, they are not getting the right information.
Have you ever tried to find business information on AI or research for companies that could help you in Switzerland? It is not an easy task. It’s hard to find talent that understands what it takes to implement AI-based services. It’s not like a traditional IT project where suppliers are well identified.
You hear about AI everywhere and every day, in newspapers, on the radio, on television, there are dozens of conferences every month on AI. You have probably read many articles about different use cases made by others, most of them come from the United States and are far from your daily concerns. You may be even frustrated, everyone tells you that the AI is like the Holy Grail, and you’re like a little kid looking at this amazing toy that thinks it can never have it.
When I visit customers, they are often a little nervous at the beginning of the meeting. They feel uncomfortable, they don’t really know what we are going to tell them. They do not master the subject as might be the case when you meet a vendor who will set up a new IT infrastructure. In the end, we saw people smile, as if they were relieved. It motivates me every day to do what I do at MLlab.ai, to democratize AI and make it accessible to all.
But then, what can AI really do for you? For enterprise applications, there are primarily two types of domains where AI can impact: improve decision-making and drive operational improvement. I could make an endless list of use cases you’ve heard about. To illustrate the fact that you do not get the right information about the potential of AI, there is this interesting case: reduce the churn rate by using social media data. At first it seems promising to use social media data to learn more about your customers and infer their behavior. If you are a well-known B2C company and sell your product around the world you may receive additional information from this type of data. If you are the manager of a political campaign you could also get useful information…
Remain serious, have you ever tried to search on Twitter or Facebook the number of posts referencing your brand or product? You are probably marketing an excellent product or service, but your customers do not talk about it on social media. We did the test for one of our client, a health insurance company. We have collected all posts on Swiss health insurance and found less than a thousand documents and most of them reflect on news in the media and almost none speak of specific insurance companies. It was useless…
This is not “AI for All” but rather “AI for them”! Remember, you have been asked to do something with AI for your business. Now that you have read articles, watched videos, searched the Internet you are probably sitting at your desk and I hear you say, it’s not for us.
But yes, it’s for you. You probably will not use social media data, but your data, in most cases, is sufficient. You probably have several years of historical, structured and unstructured data. Based on it, we can build useful AI services like the one I’m going to show you now.
The case I want to share with you is the optimization of a back office we did for a client. Suppose you have customer requests with different complexity that you receive each day, such as insurance claims, loan applications, tax returns, customs declarations, subscription to a service offered by your company. We have encoded the complexity from 0 to 4, 0 being the least complex and 4 the most complex.
In your team you have people with different skills and capacities, some can read perfectly Italian and German but not English, others do not work on Wednesdays, and so on. How do you divide the workload among your team members so that the processing time is small but of high quality?
How would you do that? With a traditional approach, you would probably design a rule-based system that infers the complexity of each new request from them. You can clearly see the disadvantages of this approach. You will end up with a static system that will not be able to take into account changes, either of the behavior of your customers or changes in the skills of your team. The system may be impossible to maintain because of the large number of rules needed to describe the complexity of the actual process.
We have developed a system that learns the complexity of a task and then assigns it to the right people or groups of people. Using ML algorithms, we can create a self-adaptable system that you can test from Excel. Yes, I just said Excel, I told you, this article aims to demystify AI, what is more common than an Excel sheet?
How does it work? For this case, we used Microsoft Azure Machine Learning Studio to develop the application. It is one of the well-integrated machine learning tools available today in the cloud, it is visual and intuitive. The result of this development process is a web service that you can call using the Azure Machine Learning add-in found in Excel. All you need to do is add the web service by providing the URL and API key. When you save the Excel workbook, your web services are also saved. You can share the workbook with other users and allow them to use the web service.
In this Excel sheet, there are two specific regions: one containing the data of one or more client requests, and the other showing the prediction of the complexity. For the request, you find fields like the customer identifier, the age, the annual income, the number of children, … In the prediction zone, you have five columns, one for each complexity, they show the probability that the request is of this complexity. The last column corresponds to the final prediction, the one with the highest probability.
When you change the values of a request, Excel sends the data to the web service and receives its complexity prediction in return. You can play and test different scenarios to validate the predictive model.
I will now present how we have done and give some details on the different steps needed to implement such services. In the following, there are no technical details, it is a high-level overview of the process needed to get this example to work.
We start the process by creating what we call an experiment. As part of this experiment, we will train a model to learn to attribute complexity to unknown demands, i.e. new customer requests. As with all supervised ML problems, we need historical data from which the system will learn. In this case, we use a set of 20’000 queries with their associated complexity, the label.
After loading the data, the second step is called feature engineering. It is the process of using domain knowledge of the data to create new features that make machine learning algorithms work better. For example, you can extract date and time information from a timestamp and create features such as weekend, day or night, day of week, year, …
Then, you need to divide your dataset into at least two subsets called training set and test set. We will not use the third set, called validation set, in this example for simplicity, although for your implementation, I recommend using the validation set. The training set is used to train the model and the test set to test its accuracy on data that has not been seen. This will tell us if the model is able to generalize.
In our example, we decided to train two algorithms, a neural network and a decision tree. Both models are scored with the test set and then compared to each other. In our example, we used the overall accuracy to decide which model to use for production. In this case, the decision tree did better with an overall accuracy of 0.94 versus 0.88 for the neural network.
Here we finished, to deploy our web service we just need to click a button and it is deployed and available from Excel or any other application using a simple http POST request. As you can see, this is not rocket science but be very careful here, it requires the right skills, methodology and management organization to be successful.
What are these ingredients necessary for an AI project to succeed? First, you need affordable computing power to run the training phase of your model; some algorithms can be greedy in terms of computing. You can find many specialized cloud infrastructures and start right away. You do not need to invest in an on-site infrastructure that costs you a lot of money to buy, install and maintain.
Then you need sophisticated algorithms to build your model. The good news is that the open source community is big in AI. Many very good algorithms are available in libraries for languages like R and Python, two programming languages commonly used for machine learning.
The previous two components are required but not sufficient to implement an AI project successfully. The third ingredient is data. Data is at the heart of your project and is a very important success factor. If you plan to implement a specific AI service such as the one presented, you will need clean and reliable data from your organization.
I would like here to clarify something that is not always understood. To require data does not mean to collect as much data as possible. An AI project will not start with your data but with your strategy, what does your company want to achieve with AI? It does not matter what data already exists.
This does not mean either deleting data that could be used in the near future. I have seen companies delete their data: to save a few megabytes of disk space, a company decided to keep a summary of orders after one year and delete the lines of its orders. Data storage costs nothing today, please keep the data that you already collect, even the logs of your website, application logs that are usually deleted every n days are full of information.
You have to compromise, when you are dealing with data you need a good data strategy and data governance that will determine what data to keep or delete. Data governance refers to the overall management and caretaking of data, from its creation to its deletion, covering usability integrity and security.
Apart from the technical and organizational aspect, an important part of good data governance relies on building a data culture within your organization. At every layer, there should be a culture of data being a valuable asset and to care of it.
You do not have to wait for the entire data strategy to be in place before starting your AI projects. It can be built in parallel and benefit from your first experiences.
Last but not least, you need talent that know how to use this technology, manipulate data and lead an AI implementation project. The problem: the skills required are scarce and most of them are absorbed by the largest companies in the world. You will need to find these skills outside your company, and to partner with specialized suppliers in AI. For these kinds of projects, a great experience helps a lot. It’s not just about technical knowledge, we use a lot of our guts and we know what algorithm will work or not without always being able to explain it.
Now is the right time to get onboard and use AI intelligently for the benefit of your processes, products and services. I encourage you to discover the potential of AI for your own business and concretely asses the relevance of this approach.
We created an AI Starter Kit. It is a flat rate offering, which allows your company to discover the potential of AI to solve one of your problems. At the end of it, you will have an AI-based service implemented like the one I presented in this article.
In conclusion, I would like you to remember that, yes, AI can be complex and requires specific skills, experience and an adapted management to be implemented and deployed successfully. But I also would like you to feel that “AI for all” can become “AI for you”.
Do you want to learn more about the subject of Machine Learning? Check out also these articles:
- Machine Learning – Introduction for decision makers
- Initiating a machine learning project: the skills your company needs
- The steps to a successful machine learning project
- Chris Tucci’s talk: Introduction to Machine Learning and Artificial Intelligence
- Jérôme Kehrli’s talk: Machine learning – from skepticism to application in fraud prevention