Linux Plumbers Conference 2008

Physical PCI slots: A little bit goes a long way - Alex Chiang

Biography

Alex Chiang is the shortest kernel developer in the HP Open Source and Linux Organization in Fort Collins, Colorado. He has been known to submit various hacky patches related to ACPI, PCI, ia64, and crfs. He toiled in the HP-UX salt mines until a fairy penguin led him through the tunnels to his freedom in 2007. In his spare time, he enjoys scaring himself pantsless in the Colorado high country.

Abstract

Recently, PCI slots gained first-class citizenship, and on ACPI enabled systems, the pci_slot driver can expose them in sysfs. The information provided by this simple driver has surprisingly powerful implications for fundamental user tasks.

Some basic problems can occur during OS installation. The network interface used to download the install kernel and ramdisk from the boot server may not be the same interface used by the installer to download the rest of the OS image [1]. This problem is especially prevalent on large, scale-up systems with many network ports where only a few have link.

Another installation issue occurs on systems with multiple OS images installed on various forms of storage, connected to a plethora of HBAs. Although this situation is unlikely to occur in a production environment, it is quite common for a kernel developer. Installing onto the wrong disk and blowing away an environment that finally reproduces a bug is possibly one of the worst feelings caused by technology, ever.

Troublesome hardware presents its own issues. Currently, there is no way to know into which physical slot a PCI card is inserted. During normal use, this lack of knowledge is mysterious but benign. However, if a card starts failing, the user wants to know quickly which device to replace, and the lack of a physical slot mapping forces the user to consult installation records in the best case, and perform a binary yank-and-see search in the worst case.

All of these problems can be solved by hooking up various userspace tools to take advantage of the pci_slot driver.

Installers such as anaconda could present physical slot information for NICs or HBAs and reduce much sysadmin heartburn
lspci, lsscsi, and friends could be extended to display the physical slot number to aid hardware troubleshooting, or simply mitigate the existential crisis of not knowing exactly what card /dev/sda might be hooked up to
udev scripts could be written, reducing the number of systems that require pci=bfsort to obtain a sane mapping between kernel device names and silk-screened chassis labels

This talk aims to:

quickly discuss the above use cases
briefly describe the effort required by system vendors to provide this information
spend the majority of the time discussing other ways that userspace can leverage this new knowledge.

Finally, the author notes that although the underlying implementation behind this proposal is ACPI based, a new driver could be written to take advantage of the new PCI slot infrastructure and recent Type 9 extensions in SMBIOS that provides the same functionality.[2]