Three Levels of MLOps
This Google Cloud source lists three levels of MLOps. The higher models offer significant benefits the more complex a project is, but are also more difficult to implement. MLOps level 2 requires collaboration between data scientists, software/platform engineers, data engineers, devops engineers. MLOps processes also require tighter integration with DevOps tooling.
Level 0: Manual process
- Target of development is a model, not a pipeline.
- Little concern for production, so if a model needs to go to production a lot of work is still to be done.
- No eye for monitoring and retraining.
Suitable for situations where there is little concern for production time for good reasons:
- Side projects
- Studying new techniques
- Kaggle competitions
- Proof of concepts
Level 1: ML pipeline automation
- Development target is ML pipeline
- Reusable pipeline components
- Code, artifact and experiment tracking
- Easy retraining
- Symmetry between development and production environments
- Monitoring in production
Advantages:
- Process standardization
- Rapid prototyping
- Fast go-to-market
- Avoid model drift through periodic retraining
- Possible to learn in production if a feedback loop is included
- Easy handover of project to others
Level 2: CI/CD pipeline automation
This level builds forth on level 1 and assumes there are existing pipelines in production.
- Development target are pipeline components
- Continuous integration: code changes in the components are merged if tests are passed
- Continuous deployment: if a component is updated, changes are deployed to production if tests are passed
- Continuous training: release a new model automatically after a trigger, e.g. uploading new data
Advantages:
- Rapid iteration on production pipelines and models
- Facilitates A/B testing of changes by deploying to a certain part of the target audience
- Easier collaboration across larger (>10) teams
- Continuous improvement in production rather than only shipping periodically