How to move existing LXD instances between servers

If you use the LXD client, you can move or copy instances from one LXD server (remote or local) to another.

Note

Remote servers are a concept of the LXD client. Therefore, there is no direct equivalent for moving instances in the API or the UI.

However, you can export an instance from one server and import it to another server.

To move an instance from one LXD server to another, use the lxc move command:

lxc move [<source_remote>:]<source_instance_name> <target_remote>:[<target_instance_name>]

Note

When moving a container, you must stop it first. See Live migration for containers for more information.

When moving a virtual machine, you must either enable Live migration for virtual machines or stop it first.

Alternatively, you can use the lxc copy command if you want to duplicate the instance:

lxc copy [<source_remote>:]<source_instance_name> <target_remote>:[<target_instance_name>]

Tip

If the volume already exists in the target location, use the --refresh flag to update the copy (see Optimized volume transfer for the benefits).

In both cases, you don’t need to specify the source remote if it is your default remote, and you can leave out the target instance name if you want to use the same instance name. If you want to move the instance to a specific cluster member, specify it with the --target flag. In this case, do not specify the source and target remote.

You can add the --mode flag to choose a transfer mode, depending on your network setup:

pull (default)

Instruct the target server to connect to the source server and pull the respective instance.

push

Instruct the source server to connect to the target server and push the instance.

relay

Instruct the client to connect to both the source and the target server and transfer the data through the client.

If you need to adapt the configuration for the instance to run on the target server, you can either specify the new configuration directly (using --config, --device, --storage or --target-project) or through profiles (using --no-profiles or --profile). See lxc move --help for all available flags.

Live migration

Live migration means migrating an instance while it is running. This method is supported for virtual machines. For containers, there is limited support.

Live migration for virtual machines

Virtual machines can be moved to another server while they are running, thus without any downtime.

To allow for live migration, you must enable support for stateful migration. To do so, ensure the following configuration:

Note

If you are using a shared storage pool like Ceph RBD to back your instance, you don’t need to set size.state to perform live migration.

Note

When migration.stateful is enabled in LXD, virtiofs shares are disabled, and files are only shared via the 9P protocol. Consequently, guest OSes lacking 9P support, such as CentOS 8, cannot share files with the host unless stateful migration is disabled. Additionally, the lxd-agent will not function for these guests under these conditions.

Live migration for containers

For containers, there is limited support for live migration using CRIU. However, because of extensive kernel dependencies, only very basic containers (non-systemd containers without a network device) can be migrated reliably. In most real-world scenarios, you should stop the container, move it over and then start it again.

If you want to use live migration for containers, you must enable CRIU on both the source and the target server. If you are using the snap, use the following commands to enable CRIU:

snap set lxd criu.enable=true
sudo systemctl reload snap.lxd.daemon

Otherwise, make sure you have CRIU installed on both systems.

To optimize the memory transfer for a container, set the migration.incremental.memory property to true to make use of the pre-copy features in CRIU. With this configuration, LXD instructs CRIU to perform a series of memory dumps for the container. After each dump, LXD sends the memory dump to the specified remote. In an ideal scenario, each memory dump will decrease the delta to the previous memory dump, thereby increasing the percentage of memory that is already synced. When the percentage of synced memory is equal to or greater than the threshold specified via migration.incremental.memory.goal, or the maximum number of allowed iterations specified via migration.incremental.memory.iterations is reached, LXD instructs CRIU to perform a final memory dump and transfers it.