The last decade has seen the rise of AI & Machine Learning, which now powers product experiences beyond those of the big technology companies. More than ever before, models must perform and function correctly to deliver business value. The cost of deploying a slow or bad model, or not detecting undesirable behavior quickly, could significantly impact customer experience and the business' bottom line.
‍
Though it’s easy to get a machine learning model into production, deploying reliably and catching performance issues remain key challenges. The trade-offs of different design decisions can result in production outages or unhealthy models. In response, a proliferation of tooling (open source and proprietary) and the Machine Learning Operations (MLOps) practice emerged.
‍
In this course we’ll explore where deployment & inference fit into MLOps. Then we’ll review common strategies and tactics that help stop bad models from reaching production and reduce the mean time to resolution of production model issues. We’ll cover common patterns for scaling inference and discuss strategies & tactics within the context of popular open source frameworks (where applicable) while applying them to common machine learning contexts. The course will not assume a particular model framework or architecture to ensure broad content applicability.
‍
Learners will be able to improve production model reliability and performance using tooling-agnostic MLOps patterns. By the end of the course, learners will be able to answer these questions:
‍

What components should my model deployment system have?

Where will my current approach to deployment & inference break down?

Is my current production architecture limiting my model performance?

How can I increase my model throughput?

How can I reduce outages/MTTR by applying model observability approaches?

Session 1 - What makes an ideal ML production system?

November 29th

8:30 am (PST)

Identify key characteristics of common ML outages in order to avoid them by trying to break a few common scenarios.

Enumerate components required for robust deployment and observability by dissecting and analyzing accepted best practices.

Session 2 - Model training, representations, and inference

December 1st

8:30 am (PST)

Demonstrate why model representations are the cornerstone of reliability.

Illustrate common approaches to scaling model inference & throughput via mini case studies.

Connect the impact of inference SLAs on model architectures and representations.

Session 3 - Patterns for successful production model deployment

December 6th

8:30 am (PST)

Demonstrate common patterns to increase production robustness.

Make informed trade-offs between production architectures and internal operations by exploring how Stitch Fix made these decisions.

Session 4 - Approaches to model observability

December 9th

8:30 am (PST)

Apply common approaches to increase model observability via review of industry tooling.

Choose appropriate patterns for avoiding outages/reducing MTTR via mini case studies.

Manage outage instances successfully (without losing your cool) .