Troubleshooting Gateway address unavailable

Whenever Traefik is used to ingress your Kubernetes workloads, you might in some specific cases encounter a “Gateway Address Unavailable” message. In this article, we’ll go through what you can do to remediate it.

Caution

In this article, we will assume that you are running MicroK8s on either a bare-metal or virtual machine. If your setup differs from this, parts of the how-to may still apply, although you will need to tailor the exact steps and commands to your setup.

Checklist

  • You have run juju trust traefik --scope=cluster

  • The MetalLB MicroK8s add-on is enabled.

  • Traefik’s service type is LoadBalancer.

  • An external IP address is assigned to Traefik.

Possible causes

The MetalLB add-on isn’t enabled

Check with:

$ microk8s status -a metallb

If it is disabled, you can enable it with:

$ IPADDR=$(ip -4 -j route get 2.2.2.2 | jq -r '.[] | .prefsrc')
$ microk8s enable metallb:$IPADDR-$IPADDR

This command will fetch the IPv4 address assigned to your host, and hand it to MetalLB as an assignable IP. If the address range you want to hand to MetalLB differs from your host IP, alter the $IPADDR variable to instead specify the range you want to assign, for instance IPADDR=10.0.0.1-10.0.0.100.

No external IP address is assigned to the Traefik service

Does the Traefik service have an external IP assigned to it? Check with:

$ JUJU_APP_NAME="traefik"
$ kubectl get svc -A -o wide | grep -E "^NAMESPACE|$JUJU_APP_NAME"

No available IP in address pool

This can happen when:

  • MetalLB has only one IP in its range but you deployed two instances of Traefik, or when Traefik is forcefully removed (--force --no-wait) and a new Traefik app is deployed immediately after.

  • The ingress add-on is enabled. It’s possible that Nginx from the ingress add-on has claimed the ExternalIP. Disable Nginx and re-enable MetalLB.

Check with:

$ kubectl get ipaddresspool -n metallb-system -o yaml && kubectl get all -n metallb-system

You could add more IPs to the range:

$ FROM_IP="..." TO_IP="..."
$ microk8s enable metallb:$FROM_IP-$TO_IP

The Load Balancer service type reverted to ClusterIP

Juju controller cycling may cause the type to revert from LoadBalancer back to ClusterIP.

Check with:

$ kubectl get svc -A -o wide | grep -E "^NAMESPACE|LoadBalancer"

If Traefik isn’t listed (it’s not LoadBalancer), then recreate the pod to have it re-trigger the assignment of the external IP with kubectl delete . It should be LoadBalancer when Kubernetes brings it back.

Integration tests pass locally but fail on GitHub runners

This used to happen when the github runners were at peak usage, making the already small 2cpu7gb runners run even slower. As much of a bad answer as this is, the best response may be to increase timeouts or try to move CI jobs to internal runners.

Verification

Verify that the Traefik Kubernetes service now has been assigned an external IP:


$ microk8s.kubectl get services -A

NAMESPACE         NAME                     TYPE           CLUSTER-IP       EXTERNAL-IP                 PORT(S)
cos               traefik                  LoadBalancer   10.152.183.130   10.70.43.245                80:32343/TCP,443:30698/TCP   4d3h
                                                                            👆 - This one!

Verify that Traefik is functioning correctly by trying to trigger one of your ingress paths. If you have COS Lite deployed, you may check that if works as expected using the Catalogue charm:

# curl http://<TRAEFIKS_EXTERNAL_IP>/<YOUR_MODEL_NAME>-catalogue/
# for example...
$ curl http://10.70.43.245/cos-catalogue/

This command should return a long HTML code block if everything works as expected.