MLflow
A MLflow project has three main components:
- Project code
- Reusable and reproducible by sticking to conventions
- MLFlow is language-agnostic. Use the right language for the right task. You can even mix and match.
- Environment definition of runtime code dependencies
- Uses Conda or Docker
- Note that Conda is not limited to Python and is also language-agnostic
- Note that Conda != Anaconda. Conda is the package manager, Anaconda offers an integrated set of packages.
- Project definition: what is contained in the project and how does the user interact with it?
- API and CLI tools for running projects and chaining projects into workflows
Project dependencies: defining the environment
Basic example of conda.yml:
name: download_data
channels:
- conda-forge
- defaults
dependencies:
- requests=2.24.0
- pip=20.3.3
- mlflow=1.14.1
- hydra-core=1.0.6
- pip:
- wandb==0.10.21
Things to note:
nameis the environment name- The order in the list of channels matters because it defines the lookup order of dependencies
- Best practice is to list to full version of a working environment to ensure compatibility of dependencies
- If some dependencies are not available in
condayou can nest a definition e.g. withpipspecific installs (seewandbabove). - Versions are indicated with a single
=forcondabut==forpip
You can also use Docker.
See https://www.mlflow.org/docs/latest/projects.html#specifying-an-environment.
Conda vs. Docker
Considerations for Conda:
- Easy to use, little DevOps skills needed
- Self-contained, all info in conda.yml
- Not completely isolated, still dependent on host
- Cannot use Kubernetes (which requires Docker)
Considerations for Docker:
- Complete isolation
- No annoying issues when switching between Windows and Linux
- Can run anywhere, also on Kubernetes
- Requires more skills to setup
- You need a separate Dockerfile in addition to the MLproject file, so your config is not self contained.
- Running locally requires root privileges
- Which means that adding a user to the docker group effectively gives that user root permissions
Project definition in MLproject
NOTE: the filename is MLproject without extension.
Example:
name: download_data
conda_env: conda.yml
entry_points:
main:
parameters:
file_url:
description: URL of the file to download
type: uri
artifact_name:
description: Name for the W&B artifact that will be created
type: str
default: output.csv
command: >-
python download_data.py --file_url {file_url} \
--artifact_name {artifact_name}
other_script:
parameters:
parameter_one:
description: First parameter
type: str
command: julia other_script.jl {parameter_one}
Things to note:
- You must always define a
mainentry point, which will serve as the default - Every entry point section must define the command to run
- The parameters within brackets will be interpolated with the user input.
Running an MLflow project from CLI
From a local folder:
mlflow run /path/to/the/local/folder -P file_url=https://...
NOTE: for each parameter you provide a -P flag.
If you want to specify a different entry point, use -e:
mlflow run /path/to/the/local/folder -e other_script
From a remote Git repo:
mlflow run git@github.com/my_username/my_repo.git
Run a specific version tag or branch, e.g. to only run released code versions:
mlflow run git@github.com/my_username/my_repo.git -v 1.2.3
Running an MLflow project from a Python script
import mlflow
mlflow.run(
# URI can be a local path or the URL to a git repository
# here we are going to use a local path
uri="my_project",
# Entry point to call
entry_point="main",
# Parameters for that entry point
parameters={
"file_url": "https://...",
"artifact_name": "my_data.csv"
}
)
Running in the cloud
There are several options:
- Databricks: native support by using the
-bflag:mlflow run -b - Execute pipeline on a compute instance you spin up yourself
- Hydra launchers
- Supported backends: Joblib (parallel local jobs on one CPU), Ray, Redis Queue, SLURM
- Kubernetes without Kubeflow
- Quick and dirty but not optimal because entire pipeline is run within a single container
- In other words, you lose the benefits of the composition of the pipeline into multiple steps
- E.g. include miniconda in Docker build so that components can still use conda.
- Kubernetes with Kubeflow
- Install miniconda and mlflow in Docker image
- You can relatively easily convert your mlflow pipeline with main.py to a Kubeflow pipeline
Setting the storage backend
By default MLflow will store results locally.
One particular issue I ran into is that when running on an AzureML compute the default backend is the AzureML workspace.
In this particular instance I wanted to store results locally.
The mlflow Python package has the functions mlflow.get_tracking_uri() and mlflow.set_tracking_uri().
However, when running mlflow run . the backend stored in the environment variable MLFLOW_TRACKING_URI is used.
So, in order to set a local file or sqlite backend, change this environment variable, e.g. with:
export MLFLOW_TRACKING_URI=file://$(pwd)/mlruns
Experiment tracking
Example:
experiment_id = mlflow.create_experiment("experiment 1")
with mlflow.run() as run:
# Log an hyper parameter
run.log_param("hyperparam1", 100)
# Log a metric
for i in range(10):
run.log_metric("foo_metric", i)
# Log an artifact
path_to_my_artifact = "./my_artifact.csv"
run.log_artifacts(path_to_my_artifact)