Research Associate and Software Engineer
Flux Research Group
School of Computing
University of Utah
linkedin source code
office: 3490C Merrill Engineering Building
voice: +1 801-581-3045 fax: +1 801-585-3743
mail: 50 S. Central Campus Dr., Rm. 3190
Salt Lake City, UT 84112
I'm a software engineer in the Flux Research Group at the University of Utah, and I design and build cool, new software that many people use. My interests span many aspects of computer systems. Usually I'm happiest when hacking lower in the software stack. For instance, during a 3-year project we called A3, I built Stackdb, a C debugger library for the entire system stack (but especially for VMs) — and debugging tools, and security forensics and defense software built atop it. But I've worked in many areas of systems software: VMs and debugging, distributed systems, languages, networks, network management, security, and mobile sensor networks and embedded systems.
- Testbeds: CloudLab, Emulab, Apt, NCR (National Cyber Range), ProtoGENI
We needed software to let us look inside a VM and monitor its execution and memory state, at a source code level, to trace and diagnose malicious software actions. First, I wrote a VMI (virtual machine introspection) software package that utilized source code debugging symbols to unearth semantically-meaningful information about the execution state of the kernel inside the VM. However, we wanted to perform whole-system security analyses — imagine getting multiple execution traces for all processes involved in an exploit. Then imagine the specific scenario of being able to get a stack trace for a remotely-exploited kernel bug, at all source code levels from the start of the attack in a PHP script, through the C library in userspace, through the system call layer in the kernel, and finally to whatever kernel subsystem contained the nasty bug. Pretty cool — you'd have a PHP stack, a C stack (since the PHP interpreter and runtime are implemented in C) in userspace, and a C stack from the kernel's execution.To get us to this point, I designed a stackable debugger library. The idea was that you'd have a base debugger driver that would "attach" to the lowest layer of the system you wanted to debug (i.e., a VM and the OS kernel inside it). Then, you'd stack another debugger driver on top, which would "attach" to the base driver in order to read and write memory, implement breakpoints, etc (i.e., attach to a userspace process in an OS in a VM). Finally, you could attach another driver to debug a language runtime (i.e., PHP atop the userspace process).
The insight is that even though only the base driver attached to
a real debugging API (i.e., the Xen VMM, a GDB server, or
ptrace) to pause the target, read and write memory,
or set breakpoints — the overlay drivers could emulate all
those operations in terms of the underlying driver — so
you have the same interface going all the way up the stack of
Our most active testbed project is CloudLab. CloudLab is a testbed that is designed to help researchers develop and test new or modified cloud software. CloudLab is based on the original Emulab software and its more recent ProtoGENI extensions. I developed and maintain our OpenStack profile (source repo). A CloudLab profile is a description of an experiment---required hardware, network configuration, OS configuration---and a set of installation scripts that turn raw Ubuntu machines into an instant, on-demand OpenStack cluster. This profile is heavily optimized for quick setup time---so you can have a new shiny OpenStack on bare metal in less than 15 minutes. I'm also developing server- and client-side support for sharing raw CloudLab resources (PCs and L2 network resources) with other federated clusters (i.e., HPC clusters, Condor pools, etc), at times of reduced usage demand from CloudLab users (or at times of higher demand from an HPC pool). This is quite interesting, due to the interaction of multiple different clusters, which all have different notions of node value, resource guarantees, reservations, scheduling, resource assignment, etc.
For the NCR, I did a bunch of work helping to rethink and redesign testbed software for better extensibility, scalability, and security (MLS contexts). Utah developers implemented some of those ideas in the Utah Emulab software, and within the prototype system Utah and the John's Hopkins Applied Physics Lab team built (which was based on Emulab). The primary challenges the NCR imposed on testbed design involved secure isolation and scale of experimentation; the first is tricky to do while maintaining full testbed automation; the latter involves challenges that you'd face in any large distributed system.
SeaCat is motivated by the desire to provide secure end-to-end application data containment between an application and a remote service. This kind of thing is really useful in healthcare applications, where you want to let a doctor access patient medical records securely from an insecure device like his/her mobile phone — but you want to make sure that only the medical records app on the phone can talk to the medical records server, and vice versa. You also have to restrict the doctor from copying data between apps. My role is in reconfiguring and modifying Linux OSes to securely isolate applications in containers and bind them to least-privilege network and IPC access. I advised one of our grad students on doing this in Ubuntu, and did it myself on an Ubuntu Touch for mobile phones and tablets. The work here is to (1) minimizing the TCB (trusted computing base) of the device, so that the only OS components and userspace programs that absolutely require root privileges run in a single privileged container, and (2) all other services and apps run in other, less-privileged containers. We haven't achieved either of these things in full, but we've gone far enough to isolate applications from each other and bind them to privileged, isolated network access using SDN (software-defined networking).
Here's the automatically-maintained list.
I like hacking: my dev hobbies tend to be whatever interesting software I'm working on at work. I do a lot of live audio engineering at my church, which is fun because I get to mix on top gear and a great PA, and learn about operating digital audio networks. I relax by reading, especially anthropology novels. I enjoy exploring remote areas, including ghost towns in the past. A few years ago, I took up road biking in the Wasatch mountains. At first I focused on time trialing my way up single canyons, but as I got into better shape I took on longer rides and more elevation. I have old GPS data and elevation plots here, but they're not current.
I've taken lots of pictures around Utah and on conference trips; view the gallery.