Devices

Backends, supported hardware and how a device is chosen.

Device	Backend	Hardware
`npu`	OpenVINO GenAI	Intel NPU (Core Ultra)
`intel-gpu`	OpenVINO GenAI	Intel GPU (integrated or Arc)
`vulkan`	llama.cpp (llama-server)	any GPU with a Vulkan driver: AMD (RADV), NVIDIA (proprietary driver or NVK), Intel (ANV)
`cpu`	llama.cpp (llama-server)	any x86_64 CPU, Intel or AMD
`openvino-cpu`	OpenVINO GenAI	any x86_64 CPU

OpenVINO is only needed for Intel's NPU and GPUs. AMD and NVIDIA go through llama.cpp.

Choosing a device

Setup measures every device your machine has. On a Ryzen 5 5500 with an RTX 3050:

  device   per message
  vulkan         2.4 s  <- selected
  cpu            4.0 s

Run smart-commit bench again after changing drivers. Force a device with smart-commit device use npu.

More numbers are in the measurements and the devices guide.