Platinum Sponsors

  • Intel
  • IBM

Gold Sponsors

  • NetApp

Silver Sponsors

  • HP
  • Google
  • MontaVista
  • Sandisk


  • Portland State University
  • Linux Foundation

Press Partners

  • Linux Journal
  • Linux Weekly News
  • Linux Pro Magazine

Sponsorship opportunities

For more information on sponsorship opportunities, please contact Angela Brown. Linux Plumbers Conf sponsorship packages.

Kernel support for user debugging: ptrace, utrace, and what's next - Roland McGrath (Red Hat)


Roland McGrath first contributed code to the free software community when he posted the "catname" patch for Hack 1.0.3 to Usenet. He went on to start hacking for the GNU Project in 1987 and worked there full-time between 1989 and 1996. Later he was involved in research on operating systems for the University of Utah and MIT. He has always worked on development tools and operating systems, and has contributed to a variety of free software projects. In 2002 he joined Red Hat, where his work began to include development of the Linux kernel. He has since participated in kernel work on NPTL, vDSO, signals, core dumps, and ptrace.


Every developer wants better tools to debug their applications. These tools rely on the kernel for the means to control and examine other user processes. The main Linux facility for this is ptrace.

The ptrace interface has proven difficult for tool developers. Its problems have stifled the ease of creating novel tools and components to expand the debugging and tracing landscape, and have often made maintenance and extension of existing tools difficult.

Meanwhile, the existing ptrace kernel internals have long been a source of difficult bugs and a thorn in the side of kernel maintainers. This code has been difficult for many maintainers, leaving architecture support code and all ptrace code often neglected and abandoned as a quagmire. Those interested in better debugging facilities have found it difficult to make progress.

This talk will first briefly describe the main problems with the ptrace interface from the user perspective, and conclude that ptrace has nothing left to offer. It will then move to in-kernel issues of concern to kernel developers and architecture maintainers, quickly listing problems with the old code.

The primary focus of the talk will be recent work to improve the picture inside the kernel, going from the bottom up. In each area, it can delve into as much technical detail as the audience wants.

First steps establish uniform interfaces for architecture code. The user_regset interface and other cleanups went into 2.6.25 and 2.6.26 kernels. These and other new internal API additions will complete the new definition of what each architecture's code can do to get the most out of any new tracing and debugging features. The talk will cover everything an architecture maintainer needs to know to make new features available to users and keep those features' details out of the maintainer's hair.

Next, work (not yet integrated) that completes the cleanup of all interactions between the core kernel and the implementation of tracing and debugging. This fully documents the requirements and expectations of each hook into the core and architecture code, making debugging features tractable without knowledge of many corners of the kernel. For kernel and architecture maintainers not interested in these features, it draws the line past which they need not consider the details.

The presentation of current work concludes with utrace. This baseline of low-level tracing and control features on user threads is exported as a new API for kernel modules. Analogous to the block layer below filesystems, it provides a platform on which to build debugging and tracing features efficiently and safely, and lets separate features coexist well. This work hopes to open the field for casual kernel developers to work on new features in this area. The talk will explain the utrace API in full.

Finally, we'll discuss some work in progress and ideas for new things built on utrace, including user-level interfaces for debuggers to use in place of ptrace.