Don’t you love it when your compiler thinks hard so you don’t have to? Rust’s built-in static analysis is praised for providing all kinds of safety guarantees for your code. Today, it’s not about your code, or even my code; it’s about how calling Linux ioctls through a type-safe abstraction layer exposed a bug in an ioctl definition and Rust’s type-checker was the first one to bark about it!

iocuddle is a library for improving the safety of ioctl calls from Rust. But what’s so unsafe about ioctls that we need a crate for it in the first place? The Linux kernel’s ioctl mechanism is a minimal interface that allows Linux module developers to provide APIs to userspace that don’t necessarily fit the mold of the primary module classes: char, block, and net. To this end, the ioctl function definition must be broad enough to avoid constraining the interfaces that module authors can expose.

Let’s take a look:

#include <sys/ioctl.h>

int ioctl(int fd, unsigned long request, ...);

Most readers will probably agree that this minimal interface allows for a large number of valid inputs. In fact, there’s not much out there that this function won’t accept at compile-time or at run-time. As a result, the kernel only needs one syscall instead of allowing driver developers to add zillions for their specific module.

Unfortunately for userspace programmers, this means a great deal of care must be taken for each ioctl call site to ensure the correct data are passed in. If a mistake is made, it won’t be discovered until run-time.

Readers who are already familiar with the Rust programming language may already know that, among other things, Rust ensures type-safety with the ferocity of one thousand suns. Furthermore, most programmers in general would probably agree that they’d rather be notified of an error at compile-time rather than run-time.

iocuddle exists to help provide these guarantees to Rust applications and libraries. It shifts the care that must be taken from each call site and funnels it into one place: the initial iocuddle::Ioctl definition. It is here that the crate author must ensure they’ve found the correct ioctl type from the Linux headers and that they understand the ioctl’s “direction” (though this is also found in the Linux headers). After that, the Rust definition of the ioctl is complete, and client code can enjoy compile-time type-checking of what they’re sending in to an ioctl call.

And yet… it wasn’t working for me! The internet lied to me! Rust is no panacea!1!1 (joking, joking…)

Despite quadruple checking that I had the right ioctl type:

const KVM: Group = Group::new(0xAE);

against the Linux kernel’s:

uapi/linux/kvm.h:#define KVMIO_0xAE

as well as checking that I had the right ioctl number:

pub const PIN: Ioctl<Write, &Command<Pin>> = unsafe { ENC_OP.write(0xbb) };

against the Linux kernel’s:

uapi/linux/kvm.h:KVM_MEMORY_ENCRYPT_REG_REGION   _IOR(KVMIO, 0xbb, struct kvm_enc_region)

My ioctl call was failing with ENOTTY, no appropriate ioctl.

Have you spotted the bug? Oops. I expressed the iocuddle::Ioctl as a Write direction (_IOW) ioctl, not _IOR as it’s shown here in the Linux kernel source. Silly me! Under the covers, iocuddle performs the same ioctl construction as the ioctl constructor macros. So indeed, mislabeling the ioctl direction will result in an ioctl with ENOTTY.

I had been debugging for a while, and was getting tired, so I just quickly changed the direction in my iocuddle::Ioctl definition and found my program fails to compile. More specifically, I’m unable to define this Ioctl such that I can send in my own data that the kernel needs to fulfill my request! That can’t be right, according to the documentation for this ioctl, I’m supposed to supply information:

4.111 KVM_MEMORY_ENCRYPT_REG_REGION

Capability: basic
Architectures: x86
Type: system
Parameters: struct kvm_enc_region (in)
Returns: 0 on success; -1 on error

Readers who are already familiar with Linux ioctls may already agree that iocuddle’s types weren’t wrong, they just weren’t letting me express an illegal state. Why is this an illegal state? Let’s look into ioctl directions a little more closely:

The Linux kernel ioctl definitions are built using macros that provide hints as to the direction that data flows during the course of the ioctl call:

• _IOR indicates that userspace will read data from the kernel. Generally, this means that userspace provides some buffer or struct for the kernel to write to, and if the ioctl succeeded, then userspace can inspect the contents of that buffer or struct for whatever information the ioctl promised to give them.
• _IOW indicates that userspace will supply data to the kernel. If the ioctl succeeds, then userspace can assume that the kernel took whatever actions were described in the ioctl and its relevant buffers.
• _IOWR indicates that data can flow in both directions for this ioctl. This is quite common, as one ioctl may be used to multiplex different commands. Often times, the data passed in to the kernel is a struct that describes what subcommand to execute during the course of the ioctl.

So really, iocuddle’s types were forcing me to define types that make sense for the semantics of the ioctl. A Read direction iocuddle::Ioctl has no business allowing the caller to send in its own buffer. The iocuddle::Ioctl will zero out its own buffer and let the kernel scribble on it before handing that to the caller.

Indeed, everything about the documentation for this ioctl suggests that data flows from userspace into the kernel, and while Rust’s type-checker didn’t point that out in so many words, it’s the only thing at compile-time that pointed out something is wrong with this ioctl.

While it is a great emotional victory to know that this ioctl should have been constructed with _IOW, it is a futile one, for the ioctl has already been etched into the stone tablet that is the Linux syscall ABI and it so it shall be… forever.