10 Questions before starting a Machine Learning POC

37% of organizations have implemented artificial intelligence in some form according to Gartner. Since AI is a a broad category of algorithms covering cognitive, natural language, pattern recognition and other areas, I suspect the number of organizations experimenting with machine learning or deep learning is a lot smaller.

Yet there are daily stories of different organizations benefiting from machine learning algorithms. This week I read about Frito Lay using ML to determine chip texture, MIT reporting on ML used to create extra-delicious basil, and an ML that helps to detect gunfire.

Last week I posted a more futuristic prediction  when software architecture will be driven by machine learning. This week I want to follow up with these more practical steps on getting started with machine learning.

Ten steps to evaluate before starting a machine learning POC

A few months ago, I shared a few of the critical questions that help define winning AI experiments. In this post, I share a more detailed checklist taking you from value, feasibility, solution types, and program management.

Here are the questions:

  1. What is the business value if delivered - Describe and ideally quantify the business outcome first. How does identifying chip texture drive value for Pepsi? How much more basil can be sold if the public recognizes the improvements?  If the business value is small, then I would suggest looking at non-ML options because the cost to experiment with machine learning is relatively high.
  2. Do you have data (or can you get it easily) - This of course must be followed up with many additional questions on the quantity and quality of data. If you don't have data or its quality needs to be improved, then you have some work to do before getting to machine learning.
  3. Do you know what's good? - Machine learning algorithms need to be trained either with training data (superviser learning) or value functions (reinforcement and other unsupervised learning). Some ML problems are hard because identifying the correct answers isn't trivial and identifying training data is prone to human biases. Identifying cats in pictures is easy, identifying sought-after criminals is a lot harder.   
  4. How good is good enough? When machine learning is being used to automate or augment workflows, human behaviors, or decision making, then one critical question is knowing the level of accuracy required compared to the underlying costs. If your organization has three people in a workflow that requires very high accuracy, then this may not be an ideal candidate to apply machine learning.
  5. How feasible is it to implement? - If you get through questions 1-4, then question 5 asks to look at the feasibility and cost of the complete solution. First, you should consult experts to validate the overall complexity of the ML required. For example, ML applied to images is a lot easier and requires substantially less infrastructure compared to ML applied to video. Second, the investment in ML is often only a part of the overall solution and organizations should calculate the total cost before diving into what is possibly the hardest part.


  6. Can you buy the AI?- Before asking the ML enthusiasts in your organization to prototype, perform some research and vendor analysis to see whether and how vendors may have already tackled the challenges and provide solutions. Looking for AI in sales - there are solutions. Chances are your marketing tools have some AI inside. Better not reinvent the wheel when the wheel can be expensive to invent! 
  7. Do you really have the skills? - All the public cloud providers have ML offerings and you may have some data scientists who have done some modeling in Python, but that doesn't mean you have the necessary and infrastructure ready to handle the targeted POC.    
  8. Who is on the team? - Want success? Start by formally defining who is on the team and their roles. Machine learning as in any innovation or R&D process requires a team with different people collaborating, learning, and deciding on how to process feedback and adjust approaches. Consider the ML agile team where you will likely need business experts, data scientists, platform experts, business analysts developers, and devops engineers depending on the type of ML and solution platform.
  9. How will you track experiments? - Machine learning requires experimentation on selecting algorithms, configuring and tuning networks, improving data quality, optimizing infrastructure, and validating results. I strongly advocate using agile practices to manage data-driven processes on everything from discovering data sources to agile data science (covered in my book, Driving Digital). If you are already using an agile tool for application development, consider how to configure it to support ML POCs.   
  10. Is your organization patient? - Lastly, succeeding at artificial intelligence requires a commitment to agile experimentation. If you are asking to have an ML POC completed with results in two, two-week sprints then I would suggest pausing and taking a dose of reality pills. Your culture needs to support experimentation, questioning, pivots, learnings, and failures before embarking on a machine learning proof of concept.

So, given all these steps, it doesn't surprise me that only a 1/3 of organizations have started their AI journeys. If you need help getting started, ask StarCIO to run a data-driven workshop to innovate, prioritize, and plan!

No comments:

Post a Comment

Comments on this blog are moderated and we do not accept comments that have links to other websites.

Share