Engine manifest

Each engine is accompanied by a manifest file outlining the engine’s components and configurations. The manifest also describes the engine’s system requirements. It is required during the selection process in order to pick the most optimized engine for the host machine.

The engine manifest should include the following attributes:

name

A unique name for the engine.

description

The description of the engine

vendor

The name of the responsible organization

grade

Indicates the stability and compliance of the engine. Only stable engines get auto selected. (supported values: stable and devel)

devices

Lists of required computing devices. It’s possible to indicate if more than one device is required, or if there are multiple options, using the anyof and allof keywords. For example, an engine requiring a generic CPU, combined with either an Nvidia GPU with compute capability 7.0 or an AMD GPU with 8GB vRAM should list the CPU under allof and GPUs under anyof.

  • type - type of the device (cpu, gpu, npu, nil)

  • Remaining field types depend on the type. See other device-specific properties below.

memory

Required system memory to load the model. Memory capacities should be given either in gigabytes or megabytes, with G or M as suffixes.

disk-space

The total size of engine components plus runtime additions. Disk capacities should be given either in gigabytes or megabytes, with G or M as suffixes.

components

List of snap components required by the engine

configurations

Default engine configurations

Device-specific fields

Device entries may also have the following device-specific fields:

CPUs

architectures: CPU architecture in Debian nomenclature (amd64, arm64). This field is mandatory. The remaining fields are architecture specific.

  • amd64:

    • manufacturer-id - reported by CPUID instruction

    • flags - list of required CPU flags

    • family-id - not used

  • arm64:

    • implementer-id - from Main ID register

    • part-number - from Main ID register

    • features - not used

PCI peripherals

  • bus: pci

  • vendor-id - PCI vendor ID hex number.

  • device-id - PCI device ID hex number

  • device-class - not used

  • subvendor-id - not used

  • subdevice-id - not used

  • programming-interface - not used

GPUs

  • bus - the bus or protocol used by the device (supported values: pci, default: pci)

  • vram - minimum required Video RAM

  • compute-capability - applicable to NVIDIA (vendor-id 0x10de) only. A list of MAJOR.MINOR version strings prefixed with a comparison operator (==, <=, >=, >, <[1]). MAJOR and MINOR must both be integers.

When bus is set to PCI, this object inherits all PCI peripheral fields.

NPUs

  • bus - the bus or protocol used by the device (supported values: pci, default: pci)

When bus is set to PCI, this object inherits all PCI peripheral fields.

Example YAML serialization of an engine manifest

  # metadata
  name: manifest-template # <vendor>-<id>
  description: This is an engine manifest template
  vendor: Engine Ltd
  grade: stable

  # hardware requirements
  devices:
    # at least one is required
    anyof:
      - type: cpu
        manufacturer-id: AuthenticAMD
        architecture: amd64
      - type: cpu
        manufacturer-id: GenuineIntel
        architecture: amd64
        flags: [avx2, avx512]
    # all are required
    allof:
      - type: gpu
        vendor-id: "0x10de" # Nvidia Corporation
        # device-id:
        vram: 4G
        compute-capability: ["==5.3", ">=6.2"]
  memory: 4G
  disk-space: 15G

  # required snap components
  components:
    - llamacpp
    - model-q4-k-m-gguf

  # default config
  configurations:
    key: value