Deploy Charmed MLflow and Kubeflow to EKS

This guide shows how to deploy Charmed MLflow alongside Kubeflow on AWS Elastic Kubernetes Service (EKS). In this guide, you will create an AWS EKS cluster, connect Juju to it, deploy the MLflow and Kubeflow bundles, and relate them to each other.

Requirements

Create an EKS cluster

See the EKS creation guide to learn how to create an EKS cluster where Charmed MLflow will be deployed.

Note

Make sure to change the value of instanceType under managedNodeGroups[0].instanceType from t2.2xlarge to t3.2xlarge, as worker nodes of type t3.2xlarge or larger are required to deploy both MLflow and Kubeflow.

Set up Juju

First, install Juju with:

sudo snap install juju --channel=3.6/stable

Connect it to Kubernetes (K8s):

juju add-k8s kubeflow

Create the controller:

juju bootstrap --no-gui kubeflow kubeflow-controller

Note

You can use any name for the controller.

Add the kubeflow model to your Juju controller:

juju add-model kubeflow

Note

You must choose kubeflow as the model name to connect MLflow to Kubeflow.

Deploy MLflow

Deploy the MLflow bundle as follows:

juju deploy mlflow --channel=2.22/stable --trust

Deploy Kubeflow

To deploy Kubeflow along with MLflow, run the following command:

juju deploy kubeflow --trust  --channel=1.10/stable

Once the deployment is completed, you will see the following message:

Deploy of bundle completed.

Note

The bundle components need some time to initialise and establish communication with each other. This process may take up to 20 minutes.

Check the status of the components with:

juju status

The deployment is ready when all the applications and units in the bundle are in active status. You can also use the watch option to continuously monitor the statuses:

juju status --watch 5s

During the deployment process, some of the components statuses may momentarily change to blocked or error state. This is an expected behaviour, and these statuses should resolve by themselves as the bundle configures.

Set credentials for your Kubeflow deployment:

juju config dex-auth static-username=admin
juju config dex-auth static-password=admin

Deploy Resource dispatcher

The Resource dispatcher operator is an optional component which distributes K8s objects related to MLflow credentials to all user namespaces in Kubeflow. This enables all Kubeflow users to access the MLflow model registry from their namespaces. Deploy it as follows:

juju deploy resource-dispatcher --channel 2.0/stable --trust

See Resource Dispatcher for more details.

Then, relate the Resource dispatcher to Charmed MLflow as follows:

juju integrate mlflow-server:secrets resource-dispatcher:secrets
juju integrate mlflow-server:pod-defaults resource-dispatcher:pod-defaults

To deploy MLflow models using KServe, create the required relations as follows:

juju integrate mlflow-minio:object-storage kserve-controller:object-storage
juju integrate kserve-controller:service-accounts resource-dispatcher:service-accounts
juju integrate kserve-controller:secrets resource-dispatcher:secrets

Integrate MLflow with Kubeflow dashboard

You can integrate the MLflow server with the Kubeflow dashboard as follows:

juju integrate mlflow-server:ingress istio-pilot:ingress
juju integrate mlflow-server:dashboard-links kubeflow-dashboard:links

Now you should see the MLflow tab in the left-hand sidebar of your Kubeflow dashboard at:

http://10.64.140.43.nip.io/

Note

The address of your Kubeflow dashboard may differ depending on your setup. Check its URL by running:

microk8s kubectl -n kubeflow get svc istio-ingressgateway-workload -o jsonpath='{.status.loadBalancer.ingress[0].ip}'