Linux Plumbers Conf 2010

One Line Summary

Abstract

All recent servers with more than one CPU socket rely on NUMA (non-uniform memory access) to solve scalability problems. If guests fit entirely into one NUMA node (both in terms of memory and required vCPUs), the current implementation both in KVM and Xen work quite well. But if larger guests exceed one of these resources, the performance may degrade.
The solution is to allow the hypervisor to assign resources from multiple nodes to one guest and to tell the guests about it by injecting a virtual NUMA layout.
While it is comparatively easy to generate the appropriate ACPI SRAT table and inject it into the guest (which is already implemented in QEMU), the host binding part is more complicated. The main problem is
the static nature of a one-time allocation and binding, as is limits the guest to be scheduled on a certain subset of nodes. This works against the nice default load-balancing feature of virtual machines.
The current first implementations in Xen and KVM rely on pinning the guests, which is OK for a rather static virtualization setup and resembles the old partitioning approach of early virtualization solutions.
In the talk I will sketch both the implementations in Xen and KVM and show the effects caused by the different architectures. Possible extensions are shown. Also demonstrated are the performance effects of NUMA un-awareness in the hypervisor and the guests. Particular attention is paid to the potential performance enhancements caused by spreading guests across different nodes to let them use separate resources.
The talk targets both developers and administrators looking for improved virtualization performance on larger systems.

Tags

KVM, Virtualization, QEMU, Xen, NUMA

Speaker

• Andre Przywara

  AMD
  • Website: http://www.amd64.org/

  Biography

  After graduation from the University of Technology in Berlin, Germany in 2002 with a diploma thesis on realtime embedded Linux and working there for a four years (including lecturing) I joined AMD in Dresden, Germany in 2006. At the OSRC I worked on several fields in the virtualization area both in Xen and KVM, contributing to topics like NUMA (QEMUs guest NUMA emulation), CPUID (including guest multi-core emulation) as well as cross-vendor migration. I presented some topics at the Berlin Linux-Tag and at the Chemnitzer Linux-Tage events.