How to manage controllers¶

Bootstrap a controller¶

Given a cloud you’ve already added to Juju, to bootstrap a Juju controller in that, use the bootstrap command followed by the name of the cloud and the name you want to assign to your new controller. For example:

juju bootstrap aws aws-controller

You can also add many different options, to specify the cloud credentials to be used, to select a specific cloud region, to specify a storage pool, to constrain the controller or workload machines, to configure the deployment in various ways, to pass a cloud-specific setting, to choose a specific juju agent version, etc.

See more: juju bootstrap, cloud-specific reference docs, List of constraints, List of controller configuration keys

View all the known controllers¶

To see a list of all the controllers known to the juju client, run the controllers command:

juju controllers

Sample output for a case where there is just a single controller boostrapped into the localhost cloud:

Use --refresh option with this command to see the latest information.

Controller             Model       User   Access     Cloud/Region         Models  Nodes    HA  Version
localhost-controller*  controller  admin  superuser  localhost/localhost       1      1  none  3.0.0

By specifying various options you can also choose a specific output format, an output file, etc.

See more: juju controllers

View details about a controller¶

To view detailed information about a controller, use the show-controller command, optionally followed by one or more controller names. For example, below we examine a controller called localhost-controller:

juju show-controller localhost-controller

By specifying various options you can also choose an output format, an output file, or get an output that includes the password for the logged in user.

See more: juju show-controller

Switch to a different controller¶

To switch from one controller to another, use the switch command followed by the name of the controller. For example, below we switch to a controller called localhost-controller-prod:

juju switch localhost-controller-prod

See more: juju switch

Configure a controller¶

See related: Configure a model

Set values. A controller configuration key can be assigned a value during controller-creation time or post-creation time. The vast majority of keys are set in the former way.

To set a controller’s configuration at controller-creation time, use the bootstrap command with the --config followed by the relevant <key>=<value pair(s). For example, the code below creates a controller localhost on a cloud lxd and at the same time configures the controller such that the bootstrap-timeout key is 700 seconds:

juju bootstrap --config bootstrap-timeout=700 localhost lxd

To set a controller’s configuration once it’s already been created, use the controller-config command followed by the relevant <key>=<value pair(s). For example, the code below configures an existing controller named aws so as to record auditing information, with the number of old audit log files to keep being set at 5.

juju controller-config -c aws auditing-enabled=true audit-log-max-backups=5

See more: juju bootstrap, juju controller-config

Get values. To get a controller’s current configuration, run:

juju controller-config

This will output a list of configuration keys and their values. This will include those that were set during controller creation, inherited as a default value, or dynamically set by Juju.

See more: juju controller-config

Manage constraints for a controller¶

Manage a controller’s connection to the client¶

To add / remove details of a controller to / from your Juju client, you need to register / unregister the controller.

Register a controller¶

The procedure for how to make an external controller known to your local client varies slightly depending on whether the controller is private or public.

Register a private controller. To register a private controller, use the register command followed by your unique registration key – that is, copy-paste and run the line of code provided to you by the person who has added you to the controller via the juju add-user command. For example:

juju register MFATA3JvZDAnExMxMDQuMTU0LjQyLjQ0OjE3MDcwExAxMC4xMjguMC4yOjE3MDcwBCBEFCaXerhNImkKKabuX5ULWf2Bp4AzPNJEbXVWgraLrAA=

This will start an interactive session prompting you to supply a local name for the controller as well as a username and a password for you as a new juju user on the controller.

The command also has a flag that allows you to overwrite existing information, for cases where you need to reregister a controller.

See more: juju register, Add a user

Register a public controller.

First, check that your public controller meets the prerequisites: Your client must be able to connect to the controller API over port 17070. Note: Juju takes care of everything else, and in most cases it takes care of this requirement too: for all clouds except for OpenStack Juju defaults to provisioning the controller with a public IP, and even for OpenStack you can choose to bootstrap with a floating IP as well.

Then, to register the public controller, use the register command followed by the DNS host name of the public controller. For example:

juju register public-controller.example.com

This will open a login window in your browser.

By specifying various flags you can also use this to reregister a controller or to type in your login information in your terminal rather than the browser.

See more: juju register

Unregister a controller¶

To remove knowledge of the controller from the juju client, run the unregister command followed by the name of the controller. For example:

juju unregister localhost-controller-prod

Note that this does not destroy the controller (though, to regain access to it, you will have to re-register it).

See more: juju unregister

Make a controller highly available¶

Collect metrics about a controller¶

Each controller provides an HTTPS endpoint to expose Prometheus metrics.

See more: Charmhub | juju-controller > Endpoint metrics-endpoint > List of metrics

To feed these metrics into Prometheus, you must first configure Prometheus to scrape them.

You can do that automatically via Juju relations or manually.

Configure Prometheus automatically¶

Added in version 3.3.

Important

As the required Prometheus charm is only available for Kubernetes, this option requires a Kubernetes cloud.

If you’re already on a Kubernetes cloud: While it is possible to deploy Prometheus directly on the controller model, it’s always best to keep your observability setup on a different model (and ideally also a different controller and a different cloud region or cloud).

To configure Prometheus to scrape the controller for metrics automatically, on a Kubernetes cloud add a model; on it, deploy prometheus-k8s, either directly or through the Canonical Observability Stack ; offer prometheus-k8s’ metrics-endpoint for cross-model relations; switch to the controller model and integrate the controller application with the offer; wait until juju status --relations shows that everything is up and running; query Prometheus for your metric of interest / set up a Grafana dashboard and view the metrics collected by Prometheus there.

Example workflow using Prometheus and Grafana from COS

Assumes your controller application and Prometheus are on different models on the same Kubernetes cloud and that you are deploying Prometheus (prometheus-k8s) through the Canonical Observability Stack bundle (cos-lite). However, the logic would be entirely the same if they were on the same controller but different clouds (multi-cloud controller setup) or on different controllers on different clouds (except in some cases you may also have to explicitly grant access to the offer).

First, deploy COS, offer Prometheus, and integrate Prometheus with your controller.

$ juju add-model observability

$ juju deploy cos-lite

$ juju status -m cos-lite --watch 1s

$ juju offer prometheus:metrics-endpoint

$ juju switch controller

$  juju integrate controller admin/observability.prometheus

$  juju status --relations

Now, either query Prometheus directly or set up a Grafana dashboard and view the metrics there.

See more:

Configure Prometheus manually¶

Tip

Useful if your Prometheus is outside of Juju.

Important

The Prometheus server must be able to contact the controller’s API address/port 17070. (Juju controllers are usually set up to allow this automatically.)

To configure Prometheus to scrape the controller for metrics manually:

On the Juju side create a user for Prometheus and grant the user read access to the controller model (e.g., juju add-user prometheus, juju change-user-password prometheus, juju grant prometheus read controller – where prometheus is just the name we’ve assigned to our Juju user for Prometheus).
Either: On the Prometheus side, configure Prometheus to skip validation. Or: On the Juju side, configure the controller to store its CA certificate in a file that Prometheus can then use to verify the server’s certificate against (juju controller-config ca-cert > /path/to/juju-ca.crt).
Add a scrape target to Prometheus by configure your prometheus.yaml with the following:

scrape_configs:
  job_name: juju
    metrics_path: /introspection/metrics
    scheme: https
    static_configs:
      targets: {ref}`'<controller-address>:17070']
    basic_auth:
      username: user-<name of Juju user for Prometheus, e.g., 'prometheus'>
      password: <password of Juju user for Prometheus>
    tls_config:
      ca_file: /path/to/juju-ca.crt

Back up a controller¶

Create a controller backup¶

Important

Only supported machine (non-Kubernetes) controllers.

To create a backup of a controller configuration / metadata, use the create-backup followed by the -m flag and the name of the target controller model. For example, assuming a controller called localhost-controller, and the standard controller model name (controller), we will do:

juju create-backup -m localhost-controller:controller

(Alternatively, you can switch to the controller model and use this command without any arguments or use the -m flag followed by just controller. However, due to the delicate nature of data backups, the verbose but explicit method demonstrated above is highly recommended.)

Sample output:

backup format version: 1
juju version:          3.0.0
base:                  [email protected]

controller UUID:       ca60f7e9-647b-4460-8232-fe75749e17c7
model UUID:            a04d7604-3073-45b7-871f-030ac0360fb4
machine ID:            0
created on host:       juju-360fb4-0

checksum:              BrOGsXIK375529xlXJHX7m23Amk=
checksum format:       SHA-1, base64 encoded
size (B):              114919198
stored:                0001-01-01 00:00:00 +0000 UTC
started:               2022-11-09 09:06:46.800165238 +0000 UTC
finished:              2022-11-09 09:07:05.133077079 +0000 UTC

notes:

Downloaded to juju-backup-20221109-090646.tar.gz

The backup is downloaded to a default location on your computer (e.g., /home/user). A backup of a fresh (empty) environment, regardless of cloud type, is approximately 75 MiB in size.

The create-backup command also allows you to specify a custom filename for the backup file (--filename <custom-filename). Note: You can technically also choose to save the backup on the controller (--no-download), but starting with juju v.3.0 this flag is deprecated.

See more: juju create-backup

Download a controller backup¶

Important

Only supported machine (non-Kubernetes) controllers.

Starting with Juju 3.0, this flag is deprecated.

Suppose you’ve created a backup with the --no-download option, as shown below (where controller is the name of the controller model).

$ juju create-backup -m controller --no-download
WARNING --no-download flag is DEPRECATED.

backup format version: 1
juju version:          3.0.0
base:                  [email protected]

controller UUID:       ca60f7e9-647b-4460-8232-fe75749e17c7
model UUID:            a04d7604-3073-45b7-871f-030ac0360fb4
machine ID:            0
created on host:       juju-360fb4-0

checksum:              tjqEvlspc88mYQmjV9u/m4i+prg=
checksum format:       SHA-1, base64 encoded
size (B):              114919131
stored:                0001-01-01 00:00:00 +0000 UTC
started:               2022-11-09 09:08:51.314128218 +0000 UTC
finished:              2022-11-09 09:09:10.296320799 +0000 UTC

notes:

Remote backup stored on the controller as /tmp/juju-backup-20221109-090851.tar.gz

As you can see from the output, this has resulted in the backup being saved remotely on the controller as /tmp/juju-backup-20221109-090851.tar.gz.

To download the backup, use the download-backup command followed by the remote location of the backup. In our case:

juju download-backup /tmp/juju-backup-20221109-090851.tar.gz

This will output the name of the downloaded backup file. In our case:

juju-backup-20221109-090851.tar.gz

This file will have been downloaded to a temporary location (in our case, /home/user).

See more: juju download-backup

Restore a controller from a backup¶

Important

Only supported machine (non-Kubernetes) controllers.

To restore a controller from a backup, you can use the stand-alone juju-restore tool .

First, download the juju-restore tool and copy it to the target controller’s ha-primary machine (typically, machine 0). To identify the primary controller machine, you can use the juju show-controller – its output will list all the machines and the primary will contain ha-primary: true:

juju show-controller
...
  controller-machines:
    "0":
      instance-id: i-073443a840f1a3626
      ha-status: ha-enabled
      ha-primary: true
    "1":
      instance-id: i-0be2c1b818e54a2ba
      ha-status: ha-enabled
    "2":
      instance-id: i-0b4705ede7d3c0faa
      ha-status: ha-enabled
...

Then you can copy the restore tool:

# Download the latest release binary (Linux, AMD64):
wget https://github.com/juju/juju-restore/releases/latest/download/juju-restore
chmod +x juju-restore

# Switch to the controller model:
juju switch controller

# Copy juju-restore to the primary controller machine:
juju scp juju-restore 0:

Second, assuming that during the create-backup step you chose to save a local copy (the default option), use scp to copy the file to the same controller machine, as shown below. (Note: If you’ve used create-download with the --no-download option, you can skip this step – the backup is already on the primary controller machine.)

juju scp <path-to-backup> 0:

Now, SSH into this machine and run ./juju-restore followed by the path to the backup file, as shown below. All replica set nodes need to be healthy and in PRIMARY or SECONDARY state.

# SSH into the controller machine
juju ssh 0

# Start the restore! (it will ask for confirmation)
./juju-restore <path-to-backup>

The juju-restore tool also provides several options, among which:

--yes: answer “yes” to confirmation prompts (for non-interactive mode)
--include-status-history: restore the status history collection for machines and units (which can be large, and usually isn’t needed)
--username, --password, and related options: override the defaults for connecting to MongoDB
--allow-downgrade: restore from a backup created with an earlier juju version
--manual-agent-control: (in the case of restoring backups to high availability controllers) stop and restart juju agents and Mongo daemons on the secondary controller machines manually
--copy-controller: clone the configuration of an old controller into a new controller (download the latest juju-restore to see this option).

For the full list of options, type: ./juju-restore --help

See more: juju-restore

Upgrade a controller¶

The procedure depends on whether you’re upgrading your controller’s patch version (e.g. 2.9.25 → 2.9.48) or rather its minor or major version (e.g., 3.1 → 3.4 or 2.9 → 3.0).

Upgrade a controller’s patch version¶

To upgrade your controller’s patch version, on the target controller, use the juju upgrade-controller command with the --agent-version flag followed by the desired patch version (of the same major and minor):

juju upgrade-controller --agent-version <current major. current minor. target patch>

For example, assuming a controller version 3.0.0, to upgrade to 3.0.2:

juju upgrade-controller --agent-version 3.0.2

Upgrade a controller’s minor or major version¶

It is not possible to upgrade a controller’s minor or major version. Instead, you should:

Use a client upgraded to the desired version to bootstrap a new controller of that version. For example:

snap refresh juju --channel=<target controller version>
juju bootstrap <cloud> newcontroller

See more: Upgrade juju, Bootstrap a controller

Recreate your old controller’s configuration (settings, users, clouds, and models) in the new controller (on machine clouds, through our dedicated tools for backup and restore). For example:

# Create a backup of the old controller's controller model
# and make note of the path to the backup file:
juju create-backup -m oldcontroller:controller
# Sample output:
# >>> ...
# >>>  Downloaded to juju-backup-20221109-090646.tar.gz

# Download the stand-alone juju-restore tool:
wget https://github.com/juju/juju-restore/releases/latest/download/juju-restore
chmod +x juju-restore

# Switch to the new controller's controller model:
juju switch newcontroller:controller

# Copy the juju-restore tool to the primary controller machine:
juju scp juju-restore 0:

# Copy the backup file to the primary controller machine:
juju scp <path to backup> 0:

# SSH into the primary controller machine:
juju ssh 0

# Start the restore with the '--copy-controller' flag:
./juju-restore --copy-controller <path to backup>
# Congratulations, your <old version> controller config has been cloned into your <new version> controller.

See more: Back up a controller (see esp. Create a controller backup and Restore a controller from a backup)

Migrate your models from the old controller to the new, then upgrade them to match the new controller’s version.

# Switch to the old controller:
juju switch oldcontroller

# Migrate your models to the new controller:
juju migrate <model> newcontroller

# Switch to the new controller:
juju switch newcontroller

# Upgrade the migrated models to match the new controller's agent version:
juju upgrade-model --agent-version=<new controller's agent version>

See more: Migrate a model, Upgrade a model

juju change-user-password <user> --reset
# >>> Password for "<user>" has been reset.
# >>> Ask the user to run:
# >>>     juju register
# >>> MEcTA2JvYjAWExQxMC4xMzYuMTM2LjIxNToxNzA3MAQgJCOhZjyTflOmFjl-mTx__qkvr3bAN4HAm7nxWssNDwETBnRlc3QyOQAA
# When they use this registration string, they will be prompted to create a login for the new controller.

See more: Manage a user’s login details

Remove a controller¶

Destroy a controller¶

Important

Any model in the controller that has disabled commands will block a controller from being destroyed. A controller administrator is able to enable all the commands across all the models in a Juju controller so that the controller can be destroyed if desired. This can be done via the juju enable-destroy-controller command: juju enable-destroy-controller.

A controller can be destroyed with:

juju destroy-controller <controller-name

You will always be prompted to confirm this action. Use the -y option to override this.

As a safety measure, if there are any models (besides the ‘controller’ model) associated with the controller you will need to pass the --destroy-all-models option.

Additionally, if there is persistent storage in any of the controller’s models, you will be prompted to either destroy or release the storage, using the --destroy-storage or --release-storage options respectively.

For example:

juju destroy-controller -y --destroy-all-models --destroy-storage aws

See more: juju destroy-controller

Use the kill-controller command as a last resort if the controller is not accessible for some reason.

Kill a controller¶

The kill-controller command deserves some attention as it is very destructive and also has exceptional behaviour modes. This command will first attempt to remove a controller and its models in an orderly fashion. That is, it will behave like destroy-controller. If this fails, usually due the controller itself being unreachable, then the controller machine and the workload machines will be destroyed by having the client contact the backing cloud’s API directly.

See more: juju kill-controller

How to manage controllers¶

Bootstrap a controller¶

View all the known controllers¶

View details about a controller¶

Switch to a different controller¶

Configure a controller¶

Manage constraints for a controller¶

Share a controller with other users¶

Manage a controller’s connection to the client¶

Register a controller¶

Unregister a controller¶

Make a controller highly available¶

Collect metrics about a controller¶

Configure Prometheus automatically¶

Configure Prometheus manually¶

Back up a controller¶

Create a controller backup¶

Download a controller backup¶

Restore a controller from a backup¶

Upgrade a controller¶

Upgrade a controller’s patch version¶

Upgrade a controller’s minor or major version¶

Remove a controller¶

Destroy a controller¶

Kill a controller¶