Create QEMU VMs with up to 1024 vCPUs

Note

Please bear in mind that invoking QEMU manually may sometimes require your user to be part of the kvm group.

For a long time, QEMU only supported launching virtual machines with 288 vCPUs or fewer. While this was acceptable a decade ago, nowadays it is more common to see processors with 300+ physical cores available. For this reason, QEMU has been modified to support virtual machines with up to 1024 vCPUs. The caveat is that the user has to provide a few specific (and not trivial to guess) command line options to enable such a feature, and that is the gap that this document aims to fill.

Supported QEMU versions

Currently, support for VMs with more than 288 vCPUs is present in the following QEMU versions:

  • QEMU 6.2 (Ubuntu 22.04 Jammy)*

  • QEMU 8.0.4 (Ubuntu 23.10 Mantic)*

  • QEMU 8.2.1 (Ubuntu 24.04 Noble)

* A special QEMU machine type needs to be used in this case. See below.

From Ubuntu 24.04 Noble onwards, there is native support for more than 288 vCPUs and using the regular ubuntu machine type should work out of the box.

Ubuntu 22.04 Jammy

If you are using QEMU on Jammy and want to create VMs with more than 288 vCPUs, you will need to use either of the special pc-q35-jammy-maxcpus or pc-i440fx-jammy-maxcpus machine types.

The command line needs to start with:

qemu-system-x86_64 -M pc-q35-jammy-maxcpus,accel=kvm,kernel-irqchip=split -device intel-iommu,intremap=on -smp cpus=300,maxcpus=300 ...

In the example above, the virtual machine will be launched using 300 vCPUs and a pc-q35-jammy-maxcpus machine type. You can adjust the option according to your use case.

The kernel-irqchip=split -device intel-iommu,intremap=on command line options are required, to make sure that the VM is created with a virtual IOMMU with interrupt mapping. This is needed due to some idiosyncrasies present in this scenario.

Note that both machine types for Jammy are supported in subsequent versions of Ubuntu, so you should be able to migrate your virtual machines to newer versions of QEMU in Ubuntu without problems.

Ubuntu 23.10 Mantic

If you are using QEMU on Mantic, the special machine types are named in a similar fashion to Jammy’s: pc-q35-mantic-maxcpus or pc-i440fx-mantic-maxcpus. Therefore, you command line to create a virtual machine with support for more than 288 vCPUs on Mantic should start with:

qemu-system-x86_64 -M pc-q35-mantic-maxcpus,accel=kvm,kernel-irqchip=split -device intel-iommu,intremap=on -smp cpus=300,maxcpus=300 ...

In the example above, the virtual machine will be launched using 300 vCPUs and a pc-q35-mantic-maxcpus machine type. You can adjust the option according to your use case.

The kernel-irqchip=split -device intel-iommu,intremap=on command line options are required, to make sure that the VM is created with a virtual IOMMU with interrupt mapping. This is needed due to some idiosyncrasies present in this scenario.

Note that both machine types for Mantic are supported in subsequent versions of Ubuntu, so you should be able to migrate your virtual machines to newer versions of QEMU in Ubuntu without problems. As noted in the previous section, it is also possible to create virtual machines using the special Jammy machine types on Mantic.

Ubuntu 24.04 Noble

From Noble onwards, the regular ubuntu machine type supports up to 1024 vCPUs out of the box, which simplifies the command used to create such virtual machines:

qemu-system-x86_64 -M ubuntu,accel=kvm,kernel-irqchip=split -device intel-iommu,intremap=on -smp cpus=300,maxcpus=300 ...

Although the regular machine type can now be used to launch the virtual machine, it is still necessary to provide some special command line options to make sure that the VM is created with a virtual IOMMU with interrupt mapping.