User level threading for a Library OS

Library operating systems have long been researched to study the impact of pushing many functionalities out of the kernel and into userspace, for the purposes of security, compatibility and performance. The Drawbridge project showed that a commercial, large operating system (OS) like Windows can be refactored into a Library OS supporting standalone apps, providing access to low-level resources using only a small set of ABIs. The Graphene project generalizes this idea to the Linux OS, and further adds secure multiprocessing abstractions and new classes of applications to the mix.

An interesting artifact of this design decision is the clear demarcation between the unprivileged, userspace component (the library OS) and the privileged, kernel component. This well-defined boundary lends itself well to execution on systems with heterogeneous cores, where different cores have different capabilities for varying energy and performance characteristics. In our case, we specifically look at cores which support different protection domains. For example, on the Intel architecture, simple cores will support only ring 3, while complex cores support ring 0 as well. The fact that a library OS limits most of the unprivileged OS code to userspace, with the privileged code being accessed via a narrow API, makes it very well suited for experimentation with heterogeneous multi-core systems. Thus, most of the OS, that is the userspace component, can be executed on a simple core and the kernel component using the complex core.

In traditional OSes, user level threading is one model of multi-threading which is employed for its performance benefits. In contrast to kernel level threads, they are light-weight since most of the thread management, including context switching and thread scheduling is done in user space without involving the kernel. Moreover, the application can be in control of the threading, instead of being limited by the kernel. However, this approach is not without its disadvantages; the most prominent ones being: user level threads cannot simultaneously run on multiple processors and no user level thread can proceed if one of them is blocked performing a system call.

Despite the suitability of the Graphene Library OS for our purpose, threading is something that is still kernel based. In this project, we aim to support multi-threaded apps on heterogeneous systems by moving the responsibility of creating and managing threads from the kernel to userspace as provided by a user level threading implementation.

Green threads AKA User level threads

This work is an attempt to introduce User Level Threading (ULT) into a library OS with a view towards enabling its execution on systems with heterogeneous cores, specifically those that support different protection domains. As a first step towards this, we aim to provide a light-weight implementation of ULT in Graphene to be tested on Linux. This way we ensure that a multi-threaded application does not have to make system calls into the kernel for thread creation and management purposes. In order to accomplish this, we adhere to a constraint that user threads will not execute any system calls. This is because system calls execute in privileged mode and our aim is to restrict threading to a simple core, with system calls being executed on the complex core.

A key component of this effort is the implementation of non-blocking system calls in order to ensure that other schedulable user threads are not blocked by a system call being executed on behalf of a single user thread.

Current status

For more details regarding the current green threads implementation, along with information on future work, please refer to the project report.


For information on the latest regarding Green threads, please contact Prof. Mike Ferdman at COMPAS lab. For information regarding the Graphene project, please contact Prof. Donald Porter and Chia-che Tsai at the OSCAR lab.