One Line Summary

Abstract

The Machine from Hewlett Packard Enterprise (HPE) showcases a breakthrough
architecture with many exciting new technologies. Unsurprisingly, The boot
phase of The Machine runtime lifecycle looks much like a legacy cluster of
networked systems or nodes. The highly-touted Fabric-Attached Memory (FAM)
needs a running OS to be accessed; as such it can be ignored at boot time.

The only devices available in the ARM SoC at boot are behind PCIe in a
classic list: serial, SDHC, USB, and network. The SDHC and USB will be used
heavily during initial node bringup and verification. Dependence upon dozens
of these devices in a mechanically-challenged full-rack demonstration should
induce cringes in any sysadmin.

The only sane choice is a network boot followed by diskless operation. The
legacy solution to diskless boot of Linux typically involves an NFS-mounted
root file system but that was deemed unsuitable in this environment.

There is an external server in The Machine implementation called the Top of
Rack Management Server (ToRMS). Among other things it provides The Machine
Manifesting Service (TMMS). TMMS manages the end-to-end lifecycle for all
the nodes in The Machine:

- Preparation of kernel and file system images
- PXE service
- Primary DNS server for nodes

This presentation will start with a quick review of The Machine hardware and
development emulator and simulator. We will dissect TMMS use of standard
Debian and Linux tools to generate the boot images (kernel and initrd) and
present them to the network for PXE boot. Of particular focus is the custom
glue performed all along the way (such as converting the transient initrd
into a persistent root filesystem).

Benefits of the current approach are fairly straightforward and simple.
Standard tools (vmdebootstrap, chroot, and qemu-static-aarch64) are used
in image creation. dnsmasq has triple duty for PXE (DHCP and TFTP) and DNS
guarded by iptables. The root FS is in RAM, minimizing software interaction
and hardware impact. The implicit “fresh root FS on boot” satisfies some of
The Machine’s multi-tenant security challenges.

Discussion can start and be guided along lines of extending connectivity
between constituent parts. A partial list could include

- Prototype performance/capacity/security constraints
- More secure or capable boot protocol (encrypted, routable, broadcast)
- Console logging and crash dump without local storage
- More intelligent integration of systemd than current hacks
- Adaptation to RPM-based distros on nodes
- Introducing hybrid NVM file system approach

Tags

Debian, vmdebootstrap, PXE, diskless, rootfs

Speaker

• Rocky Craig

  Hewlett Packard Enterprise
  Biography

  Linux Master Technologist and Principal Lead, Hewlett Packard Enterprise

  Rocky is employed by Hewlett Packard Enterprise, a spinoff of HP where he
  started in 1978. He has served many roles in the technical space of technical
  workstations and HP-UX and Linux servers. Rocky is currently co-architect
  of a Debian-based distribution for The Machine, a new architecture featuring
  non-volatile memory. He is driving the NVM management for The Machine,
  extending prototype work started several years ago. Rocky is also leading the effort to configure Linux for booting on the nodes of The Machine.