Deploy Charmed MLflow 3 with Terraform (Experimental)¶
Warning
MLflow 3 support is experimental and is not intended for production workloads. latest/edge charm channels are employed, since Charmed MLflow 3 is currently only available there, and such channels are under active development and may change at any time.
This guide describes how to deploy Charmed MLflow 3 (experimental) using Terraform, as its own standalone product, via the Charmed Kubeflow Terraform solutions.
Note
Despite the name of the Terraform solutions, this guide does not deploy any component of Charmed Kubeflow, but only Charmed MLflow together with:
The components the MLflow application needs:
a PostgreSQL database as its backend store
an S3-compatible storage integrator as its artifact store
The necessary charm ecosystem around the MLflow charm:
a service mesh based on Istio in ambient mode
an identity and access management layer based on the Canonical Identity Platform and OAuth2 Proxy
Requirements¶
Terraform version 1.6 or later
Git to download the Terraform source
- A Kubernetes cluster:
of version 1.32 to 1.34
preconfigured to work with Istio (e.g., on Canonical K8s, configure Cilium for Istio)
with at least three external IP addresses available for ingress load balancers
- The following environment variables set on your CLI as desired for DNS of external IP addresses:
AUTH_HOSTNAMEfor the hostname to be employed to authenticate to MLflow (e.g.,auth.mlflow.com)API_HOSTNAMEfor the hostname to be employed to reach MLflow’s API for external, machine-to-machine access (e.g.,api.mlflow.com)UI_HOSTNAMEfor the hostname to be employed to reach MLflow’s user interface (e.g.,ui.mlflow.com)
- Some S3-compatible storage bucket preconfigured, and the following environment variables set accordingly on your CLI:
S3_ENDPOINTfor the endpointS3_ACCESS_KEYfor the access keyS3_SECRET_KEYfor the secret keyS3_BUCKET_NAMEfor the bucket name
The Juju client installed, with a controller bootstrapped on your Kubernetes cloud - the Terraform Juju provider operates on the controller your Juju client is currently connected to
kubectl, already configured to access your cluster
Deploy Charmed MLflow 3 and Its Charm Ecosystem¶
1. Download the source of the Terraform solution by cloning the repository¶
git clone https://github.com/canonical/charmed-kubeflow-solutions.git
2. Move into the directory of the Terraform solution for Charmed MLflow 3 with ambient-mode Istio and Identity and Access Management (IAM)¶
cd charmed-kubeflow-solutions/terraform/tests/mlflow-ambient-iam
3. Initialize Terraform to download provider and modules¶
terraform init
4. Deploy the whole solution¶
terraform apply -auto-approve -parallelism=1 \
-var create_model=true \
-var external_auth_hostname=$AUTH_HOSTNAME \
-var external_m2m_hostname=$API_HOSTNAME \
-var external_ui_hostname=$UI_HOSTNAME \
-var s3_endpoint_global=$S3_ENDPOINT \
-var s3_access_key_global=$S3_ACCESS_KEY \
-var s3_secret_key_global=$S3_SECRET_KEY \
-var s3_bucket_global=$S3_BUCKET_NAME
Note
With this command, latest/edge channels are selected by default for all of the deployed charms but the Istio and IAM ones, which is necessary because Charmed MLflow 3 is currently only available there.
Terraform will create four different Juju models across which all the charm applications will be deployed and integrated: istio-system, iam-core, iam and mlflow.
5. Wait for all of the charm applications in the iam-core model to become active¶
juju status --model iam-core --watch 5s
6. Add your hostnames to the internal DNS of Kubernetes and to the DNS of your host¶
Note
The hostnames set above do not have to be registered with a public DNS provider if (and only if) the following steps are applied.
Find the IP addresses that correspond to your hostnames from the deployed load balancers:
AUTH_IP=$(kubectl -n iam-core get svc traefik-lb -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
API_IP=$(kubectl -n mlflow get svc istio-ingress-k8s-m2m-istio -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
UI_IP=$(kubectl -n mlflow get svc istio-ingress-k8s-ui-istio -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
For Kubernetes:
copy the code block rendered by running this command:
cat <<EOF hosts { $AUTH_IP $AUTH_HOSTNAME $API_IP $API_HOSTNAME $UI_IP $UI_HOSTNAME fallthrough } EOF
add it (with proper indentation) to the CoreDNS ConfigMap of your cluster, inside the
.:53block:kubectl -n kube-system edit configmap $(kubectl -n kube-system get configmap -o name | grep coredns | head -n1 | cut -d/ -f2)
make CoreDNS restart to let the ConfigMap change take effect:
kubectl -n kube-system rollout restart deployments/coredns
For your host:
echo "$AUTH_IP $AUTH_HOSTNAME" | sudo tee -a /etc/hosts
echo "$API_IP $API_HOSTNAME" | sudo tee -a /etc/hosts
echo "$UI_IP $UI_HOSTNAME" | sudo tee -a /etc/hosts
7. Wait for all of the charm applications in the istio-system, iam and mlflow models to become active¶
juju status --model istio-system --watch 5s
juju status --model iam --watch 5s
juju status --model mlflow --watch 5s
How to Access Charmed MLflow 3¶
Refer to Configure RBAC Across MLflow Workspaces in Charmed MLflow 3 (Experimental) to configure user grants across MLflow workspaces and control programmatic and UI access to Charmed MLflow 3.