Fundamentals of MLOps, ModelOps, and Successful Machine Learning Lifecycles

In the state of AI in 2020 report, 50% of respondents assert that their companies have adopted artificial intelligence in at least one business function. Excluding what this report labels as "high performing AI companies," only 17% of all other respondents have a clearly defined an AI vision and strategy.

MLOps vs ModelOps in Machine Learning - Isaac Sacolick

Who are the high performers? The high performers claim that 20% or more of their organizations’ enterprise-wide earnings in 2019 were attributable to using AI. Not surprisingly, only 8% of respondents self-assessed their companies as high performers.

Sorry folks - this is a glass-half-full story. If only high performers can tap the talent, processes, and technology required to leverage machine learning, it can create another disruptive moat for SMB's implementing digital transformation. SMBs, must also experiment with machine learning and target  to succeed with AI.

Understanding the Machine Learning Lifecycle

While there's a lot to understand, strategize, prioritize, plan, implement, monitor, and manage in the machine learning lifecycle, it's not insurmountable. In my 22nd episode of 5 Minutes with @NYIke, I share What MLOps and ModelOps are all about.

  • MLOps is similar to DevOps focussing on collaboration, automation, deployments, and infrastructure.
  • ModelOps is similar to the Software Development Lifecycle (SDLC), centering on model development, testing, releasing, and monitoring. 

Watch the 5-minute video and then read below on some very important differences in machine learning and software development.

If you liked the video, please subscribe! 

How Machine Learning Differs from the SDLC and DevOps

Now here's where there are some stark differences between machine learning and software development

  1. MLOps must address multiple scalability models - Automating the infrastructure as code (IaC) has more variability for machine learning use cases than software development. Data science team's infrastructure needs are very episodic depending on how often they train models, data set sizes, model topology, number of parallel experiments, costs, among other factors. Testing models and running them in production have different performance considerations depending on data volumes, model response times, model usage factors, cost, and performance requirements. 
  2. ModelOps Must Bring Successful POCs -> Production - While machine learning is a science that requires experimentation, the goal must be to bring successful models to production. We can rely on end-users and agile product owners to steer business needs and value propositions back to implementation goals and priorities in software development. With machine learning, experimenting is the journey, and production-enabling the model an important step. But business value is only achieved when that model gets utilized in another application, service, data visualization, or business process.   
  3. MLOps and ModelOps Must Address Model Drift - We're concerned about incidents, performance, reliability, security, and defects with production software applications and services. With machine learning models, we also have to be concerned about model drift and when models need to be retrained.

Strategic Consideration for Machine Learning Implementations 

Oh yes, there are many more considerations and choices in machine learning. Some like frameworks and platforms have commonalities in software development, others are specific to machine learning.  

  • Bias in data and whether training data has sufficient sample across targeted segments
  • Feature reuse has all the complexities of building data catalogs and reusable microservices
  • Model explainability so that subject matter experts, end-users, and auditors understand how models arrive at their predictions. 
  • Evolving frameworks - Keras, Pytorch, Tensorflow, Scikit-learn,  
  • Evolving algorithms - SVM, Bayes, KNN, K-Means, Random Forests, Reinforcement Learning
  • Evolving platforms - Alteryx, Azure ML Studio, Databricks, Dataiku, Google Cloud ML, RapidMiner, SAS, SageMaker, Watson,

What does this mean for the 92% of companies that are not high AI performers? AI and machine learning aren't easy, but staying on the sidelines is too risky. That's why it's critical to learn more about developing an AI and machine learning strategy

No comments:

Post a Comment

Comments on this blog are moderated and we do not accept comments that have links to other websites.


About Isaac Sacolick

Isaac Sacolick is President of StarCIO, a technology leadership company that guides organizations on building digital transformation core competencies. He is the author of Digital Trailblazer and the Amazon bestseller Driving Digital and speaks about agile planning, devops, data science, product management, and other digital transformation best practices. Sacolick is a recognized top social CIO, a digital transformation influencer, and has over 900 articles published at InfoWorld,, his blog Social, Agile, and Transformation, and other sites. You can find him sharing new insights @NYIke on Twitter, his Driving Digital Standup YouTube channel, or during the Coffee with Digital Trailblazers.