Overview

As explained in the overview, the LCD microkernel consists of two parts - an architecture-dependent part, and an architecture-independent part:

                                LCD Microkernel
             +-------------------------+---------------------------+
             |  Architecture-dependent |  Architecture-independent |
             |       part (VT-x)       |         part              |
             +-------------------------+---------------------------+

The architecture-dependent part provides an interface for creating VT-x containers, modifying guest physical address spaces (EPTs), entering VT-x containers, and so on. This interface is in the lcd-domains/arch/x86/include/asm/lcd_domains headers. The architecture-independent part is the user of this interface.

In order to understand the architecture-dependent part in a lot of depth, you'll have to read a lot of the Intel manual. We adapted code from the Dune project at Stanford, which was adapted from KVM.

The architecture-independent part provides an internal interface (in lcd-domains/include/lcd_domains/microkernel.h). This is used by kliblcd to translate function calls in the LIBLCD interface, like lcd_sync_send, into LCD microkernel-internal function calls.

The architecture-independent part uses libcap to track resources, as explained here.

Creating LCDs

The non-isolated LCDs or kLCDs are created when the lcd_create_module_klcd() is invoked (see for example . lcd_create_klcd

Running LCDs

This section should give you good sense of the relationship between all of the parts. It explains what happens when you invoke lcd_run in the LIBLCD interface on an LCD you just created.

When you create an LCD, a kernel thread is created and associated with the LCD. When you invoke lcd_run, this wakes up the kernel thread. This shows the sequence for a non-isolated thread invoking lcd_run:

         |
         |                                               
         |                                            
  non isolated thread                            
         |                                                                    .----> wake_up_process:
      lcd_run                kliblcd                    LCD microkernel      |
         |                 +---------+   __lcd_run    +----------------+     |             /
         '---------------> |         |--------------> |                |-----              |  LCD kthread
                           +---------+                +----------------+                   /
                                                                                           |
                                                                                           V

The LCD kthread will start executing __lcd_kthread_main in lcd-domains/microkernel/run.c.

Shortly thereafter, the LCD kthread will invoke lcd_arch_run (defined in lcd-domains/arch/x86/microkernel/run.c) to enter into the LCD (VT-x container):

                                                               LCD
                                                        +-----------------+
                                                        |                 |
                                                        |                 |
                                                        |                 |
                                                        |        A        |
                                                        |        |        |
                                                        +--------|--------+
                                                                 |
    Host Linux                                                   |
  +--------------------------------------------------------------|------------------------------+
  |                                                              |                              |
  |                                                              | VMLAUNCH/VMRESUME            |
  |                                                              |                              |
  |                                                              '-------<------<---------.     |
  |      |                                                                                |     |
  |      |                                                                                |     |
  |  LCD kthread                    _______________ LCD microkernel __________________    |     |
  |      |                         /                                                  \   |     |
  |      |                          +--------------+                  +--------------+    |     |
  | __lcd_kthread_main [start]      |              |                  |              |    |     |
  |      |                          |  Arch-indep  |  lcd_arch_run    |   Arch-dep   |    |     |
  |      '------------------------> |     Part     |----------------> |     Part     |----      |
  |                                 |              |                  |    (VT-x)    |          |
  |                                 |              |                  |              |          |
  |                                 +--------------+                  +--------------+          |
  |                                                                                             |
  +---------------------------------------------------------------------------------------------+

Note that lcd_arch_run is a "blocking" call. It doesn't return until the kthread exits out of the LCD. This happens when the code inside the LCD or an external interrupt triggers a VM EXIT. When this happens, the kthread returns back into the architecture-independent part, which will handle the exit. For example, if the LCD does a hypercall, this triggers an exit, and the architecture-independent code handles the hypercall (in lcd-domains/microkernel/run.c).