Scaling Predictive Insights through Machine Learning Operations

With the proliferation of the cloud and its innate ability to scale, businesses are asking how they can use this power to make their data ­­machine learning ready, enabling automation behind the predictive modeling process, increasing the quantity of predictions, and improving the quality of outputs from existing data science functions/teams. The push for more automation behind a business' predictive applications and data services is what the industry has begun to call Machine Learning Operations, or MLOps for short.­­

What is MLOps?

MLOps is a set of systems and processes that enable businesses to automate the management and lifecycle of machine learning models, easing the burden of data scientists to focus on the more labor and time-consuming tasks that come with the job: individual model training, evaluation, deployment, and monitoring.   

In a business context, a mature MLOps implementation can provide real-time customer insights pertaining to churn, pricing, and lifetime value – thus elevating an organization’s ability to make data-driven decisions. For example, if the average time for a data scientist to build a predictive model for a single, given product is 50 days, how can a business manage 1,000+ models? MLOps:

• Facilitates collaboration and communication between data scientists, ML engineers, and IT professionals and allows enterprises to bring data scientists' models to production faster, with fewer chances of manual errors.
• Enables multiple predictive models to train and deploy automatically and in real-time, allowing more trial and error when spinning up and down modeling concepts, as well as models to improve as new data becomes available.
• Uses a unified data foundation to ensure quality and validity across all predictive models and significantly reduce the amount of one-off data analysis required at the individual model level

Why is MLOps a new focus point for businesses?

While many businesses have a great analytics and data science teams, a lack of MLOps can inhibit a team’s ability deliver intelligence to leadership in terms of speed, scale, and quality of insights. Sources indicate that only 50% of all U.S. businesses have deployed a single model to production,[i] and 80% of models get thrown out mid-build.[ii] By automating the data and machine learning pipelines, MLOps increases the success rate of AI/ML projects and ensures a quick way to gain insights for leadership. Further, by empowering data science teams with readily available access to machine learning ready data and associated tools, ROI can be realized much faster.

Microsoft currently uses a scale of 0-4 to indicate an organization's MLOps maturity.[iii] The defining characteristic of a highly mature MLOps organization is that it wraps all ML-specific processes around key business functions and within other data integration services, enabling a cleaner method for delivering predictive intelligence to decision-makers.

How can businesses benefit?

As businesses become more dependent on data to drive decision making, automation becomes fundamental to all business functions and their supporting data platforms. For example, rather than tasking a data scientist with building predictive models on a per-use-case basis, a business can benefit from pairing the data scientist with a data engineer to architect a reusable data construct, enabling the parallel design of related models.

Breaking the mold and becoming an MLOps driven organization, however, is not a linear path and many IT-driven companies – like Uber – cite a six-year timeline to move from a level 1 to a level 3 organization.[i] Moving up the curve requires buy-in from all areas of the business, a commitment to trial and error, a heavy focus on governance and, finally, an end-to-end data strategy to tie all business needs to prospective components of a future-state MLOps development lifecycle.   

Given that every business is uniquely defined by how it applies its data to decision-making, MLOps may mean different things to different organizations. While the task of achieving a high-level MLOps framework may seem daunting, there are many ways businesses can accelerate their maturity via existing open-source tools and/or process improvements. These include: 

• Applying existing assets, such as step functions, to connect pieces of the data pipeline.
• Using containers to simulate small machine learning tasks.
• Testing new open-source functions like AutoML to automate smaller pieces of the machine learning workflow.
• Supplementing legacy statistical software with open source and justifying the necessity for paid services like Databricks, i.e., build vs. buy.

For organizations beginning to explore MLOps and the ML engineering role, organizations can move up the curve through establishing a baseline and implementing against an initiative roadmap to improve the MLOps maturity level. Organizations just getting started in the ML engineering space can leverage quick solutions to enhance existing machine learning capabilities and/or work directly with business leaders to directly implement a highly mature MLOps development lifecycle.

[i] Gualtieri, Mike and Carlsson, Kjell, PhD (2020, August). Implement ModelOps to Operationalize AI. Forrester. https://www.forrester.com/report/Introducing-ModelOps-To-Operationalize-AI/RES160698

[ii] Janakiram, MSV (2021, January). MLOps – the Missing Piece in the Enterprise AI puzzle. Forbes. https://www.forbes.com/sites/janakirammsv/2021/01/05/mlopsthe-missing-piece-in-the-enterprise-ai-puzzle/?sh=5ba9853624ad

[iii] Azure Maturity Model. Microsoft. (2022). https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model

[i]v Feature Store Summit, Uber (2022, October). Presenters: Amit Nene, Nicholas Marcott. https://content.logicalclocks.com/hubfs/Feature%20Store%20Summit%20Presentations%20PDFs/Uber%20-%20Feature%20Store%20Summit.pdf