🚨 Black Friday Sale 🚨 Purchase a course through an individual or team subscription and get $400 | Redeem by registering for a course | Expires Monday 5pm PT

Designed to help make organizations

Data Driven

Product Leaders

Program managers and product managers that are focused on metrics like growth and revenue, and prioritization decisions

Data Scientists

Data Scientists and Data Science managers who help map strategic decisions to actionable experimental designs and then interpret the results in a trustworthy manner

The motivation and basics of A/B testing (e.g., causality, surprising examples, metrics, interpreting results, trust and pitfalls, Twyman’s law, A/A tests)
Cultural challenges, humbling results (e.g., failing often, pivoting, iterating), experimentation platform, institutional memory and meta-analysis, ethics
Hierarchy of evidence, Expected Value of Information (EVI), complementary techniques, risks in observational causal studies

Engineering Leaders

Engineering managers, directors, VPs, and CTOs who want to make their organizations data-driven with metrics and A/B tests

Safe deployments
Triggering, especially in evaluating machine learning models
The benefits of agile product development

Senior ML Engineers, ML Platform Engineer and ML Product Managers

who want to reduce engineering time spent on model maintenance, improve user experience and reduce churn of your ML-powered products and manage risk associated with live model deployments

You will:
  • Set up monitoring to automatically diagnose issues like data drift, model pipeline issues, and dips in model performance in production
  • Build automatic retraining schedules that minimize the risk of model drift and resultant user churn
  • Implement active learning workflows that increase model accuracy using targeted human feedback
You should:
  • Have the ability to train machine learning models using a framework like scikit-learn, tensorflow, or pytorch
  • Have 1+ years of full-time experience in an ML or ML-adjacent engineering role

Learn live from a world-class

Instructor

Learn live from a world-class

Instructor

Josh Tobin is the cofounder and CEO of Gantry. Previously, Josh worked as a deep learning & robotics researcher at OpenAI and as a management consultant at McKinsey. He is also the creator of Full Stack Deep Learning (fullstackdeeplearning.com), the first course focused on the emerging engineering discipline of production machine learning. Josh did his PhD in Computer Science at UC Berkeley advised by Pieter Abbeel.

Learn live from world-class

Instructors

Josh Tobin is the cofounder and CEO of Gantry. Previously, Josh worked as a deep learning & robotics researcher at OpenAI and as a management consultant at McKinsey. He is also the creator of Full Stack Deep Learning (fullstackdeeplearning.com), the first course focused on the emerging engineering discipline of production machine learning. Josh did his PhD in Computer Science at UC Berkeley advised by Pieter Abbeel.

Recommended by

Industry Experts

No items found.

Expense
the cost

90% of our learners expense the full cost of our courses to their employer. This includes leading startups and enterprises alike.

Check Expense Approval At Your Company

Exclusive Content

to advance your business

Get access to exclusive content through live sessions, meetups and our Student Portal (even after you finish the cohort). Ask questions and get personal feedback directly from your instructors and others taking the course.

Join a diverse and experienced

Community

This cohort gives you access to a rich community of like-minded professionals from some of the best businesses in the world. Even after the course ends, you will continue to learn and build with each other.

About Josh's

Live Cohort

No matter how good your model was when you trained it, it won’t remain that way for long in production. That’s because unlike in academia, in the real world, data distributions are almost never static. So if you want to use machine learning in production, your goal should be to build a continual learning system, not a static model.

In this course, we’ll teach you how to build a continual learning system progressively. We’ll show you how to monitor your models and debug production issues. We’ll build a basic automated retraining strategy with best practices like CI/CD. Finally, we’ll cover advanced extensions of continual learning like active learning.


Session 1: Set up Reliable Logging and Auditing for Your ML Models

4-6 pm PST
  • Learn where continual learning is applicable and why it is helpful, recognize the difference between continual learning, online learning, and continuous retraining
  • Deploy the machine learning model we’ll work on throughout the course, and learn how to log and audit the predictions of their machine learning models using Gantry

Session 2: Configure Monitoring That Automatically Detects Production Issues

4-6 pm PST
  • Choose the metrics to monitor most applicable to your use case, and understand the pros and cons of different kinds of metrics
  • Configure metrics and set alerts for the model you deployed in Session 1 
  • Practice the two kinds of debugging: top-down debugging (aka error analysis) and bottom-up debugging

Session 3: Implement Periodic Retraining

4-6 pm PST
  • Choose a retraining cadence that optimizes cost versus benefit for your models
  • Implement periodic retraining for your course project
  • Recognize when to move away from simple periodic retraining

Session 4: Implement Continuous Integration and Deployment

4-6 pm PST
  • Understand why testing models beyond accuracy is a critical part of the continual learning process, and how to use testing to create consensus on model performance among stakeholders
  • Pick subsets of your data to test on that provide an accurate representation of model performance in a fraction of the time
  • Implement model testing and continuous integration for your course project

Session 5: Implement Active Learning

4-6 pm PST
  • Understand the pros, cons, and use cases for the different curation strategies: user feedback, manual curation, and active learning
  • Implement active learning for your course project

Session 6: Evaluate the Tradeoffs Behind Online Learning

4-6 pm PST
  • Consider automatic retraining based on triggers instead of a fixed schedule
  • Learn the basics of online learning, and when to apply it

Still have questions?

We’re here to help!

Do I have to attend all of the sessions live in real-time?

You don’t! We record every live session in the cohort and make each recording and the session slides available on our portal for you to access anytime.

Will I receive a certificate upon completion?

Each learner receives a certificate of completion, which is sent to you upon completion of the cohort (along with access to our Alumni portal!). Additionally, Sphere is listed as a school on LinkedIn so you can display your certificate in the Education section of your profile.

Is there homework?

Throughout the cohort, there may be take-home questions that pertain to subsequent sessions. These are optional, but allow you to engage more with the instructor and other cohort members!

Can I get the course fee reimbursed by my company?

While we cannot guarantee that your company will cover the cost of the cohort, we are accredited by the Continuing Professional Development (CPD) Standards Office, meaning many of our learners are able to expense the course via their company or team’s L&D budget. We even provide an email template you can use to request approval.

I have more questions, how can I get in touch?

Please reach out to us via our Contact Form with any questions. We’re here to help!

Book a time to talk with the Sphere team

Join us for a next generation

Learning Experience