In this lab, we will use the virtual memory system to add to xv6 three features common in many modern operating systems: catching NULL pointer dereferences, automatic stack growth, and kernel-provided code and data mapped to user-mode address spaces — which we shall refer to as Virtual Dynamic Shared Object (VDSO) following Linux's convention.
This lab will heavily interact with all virtual-memory-related functionality in xv6, including your implementation of COW fork. So, any lurking bugs from Lab 2 can cause a lot of unnecessary headaches here. Make sure you debug and vigorously test your COW fork before staring this one.
As in Lab 2, this assignment does not involve writing many new lines of code. The difficulty is figuring out what to change in xv6 and writing careful unit tests for each change. We strongly recommend starting early, carefully thinking about — and vetting — the chagnes before developing any code. Also, write as many test cases as you can.
Do the following to pull and merge the Lab 3 code with your existing Lab 2 code (after making sure you have committed your Lab 2 code on branch lab2):
$ git commit -am "final lab2 commit" $ git pull $ git checkout -b lab3 origin/lab3 Branch lab3 set up to track remote branch refs/remotes/origin/lab3. Switched to a new branch "lab3"
The git checkout -b command shown above actually does two things: it first creates a local branch lab3 that is based on the origin/lab3 branch provided by us, and second, it changes the contents of your xv6 directory to reflect the files stored on the lab3 branch. Git allows switching between existing branches using git checkout <branch-name>, though you should commit any outstanding changes on one branch before switching to a different one.
Note: In the above commands, we are assuming that the "origin" remote refers to read-only repo. If you have changed the name of the read-only remote, make sure to use the correct name.
Next, you will need to merge the changes you made in your last lab (lab2) branch into the new (lab3) branch, as follows:
$ git merge lab2 Merge made by recursive. ... x files changed, y insertions(+), z deletions(-)
In some cases, Git may not be able to figure out how to merge your changes with the new lab code (e.g. if you modified some of the code that is changed in the new lab handout). In that case, the git merge command will tell you which files are conflicted, and you should first resolve the conflicts (by editing the relevant files) and then commit the resulting files with git commit -a.
A null pointer is a pointer with address 0 in it. So derefrencing a null pointer is nothing but trying to read or write virtual address 0. In most cases, this is an indicator of a programming bug, and you would like the OS to let you know if your program ever attempts it.
In its current address space layout, xv6 loads user code into the very first part of the address space. Thus, if you dereference a null pointer, you will not see an exception (as you might expect); rather, you will see whatever code is the first bit of code in the program that is running. Try it and see!
Thus, the first thing you might do is create a program that dereferences a null pointer. It is simple! See if you can do it. Then run it on Linux as well as xv6, to see the difference.
Exercise 1. Write two simple user-mode test programs, one that reads from a null pointer and another one that writes to a null pointer. Call them np_read.c and np_write.c. Run them on your existing xv6 and see that they don't fail.
The next step will be looking through the code to figure out where there are checks or assumptions made about the address space. This should be fairly easy now that you are familiar with how xv6 creates and initializes a process's address space, in particular in fork() and exec() — using auxiliary functions such as allocuvm(), mappages(), deallocuvm(), etc. — and where kernel checks pointers passed as system call arguments — e.g., using argptr()). You may need to change parts of some or all of these functions when you change the address space layout.
Exercise 2. Change xv6's process address space layout so that the 0th page of every process (other than the first process, init) maps to an invalid PTE — i.e., a PTE without PTE_P set. This way, you will get a page fault if your program ever dereferences a null pointer.
To make this work, we need to change how xv6 compiles user programs. In stock xv6, programs are compiled so that the .text section is placed at virtual address 0. But now that we are going to mark that address as invalid, this needs to change. The necessary changes are already included in the Makefile. Take the diff between the newly pulled Makefile and that of the lab2 branch, and make sure you understand what the required modifications are — they are quite simple really!
Once you are done, run your np_read and np_write programs to confirm that they indeed get page faults and are killed. Make sure you haven't broken anything in the kernel by passing all the tests in usertests. This completes part 1.
In its current form, when initializing an address space in exec(), xv6 sets aside two pages for the user-mode stack: one contains the stack itself, and the other one is used as a guard page to catch stack overflows. There is no existing support for automatic stack growth if a program needs more than one page of stack. We are going to change that once and for all!
Exercise 3. Write a program that causes stack overflow on existing xv6 code (perhaps using recursive function calls), and make sure that you get a page fault when the overflow happens.
We would like to allow the stack to grow to upto 1MB (that is 256 pages, not just one), as indicated by MAX_STACK in memlayout.h. We don't want to hard-code the 1MB limit in the kernel code; that's why we use the MAX_STACK macro.
Exercise 4.
Change xv6 to enable incremental stack growth upto MAX_STACK size. Your code should do the following:
MAX_STACK+PGSIZE bytes of the address space for stack (MAX_STACK for the stack and one page for the final guard page). Let's call this space stack VMA. This initialization happens in the exec() function.allocuvm() from 2*PGSIZE to MAX_STACK+PGSIZE. Doing so would allocate 1MB of physical memory space upfront.
To help you track the stack location and size, you may want to add new fields to struct proc to keep track of the first and last allocated pages of the stack.
Exercise 5. Use your-previously-overflowing program of Exercise 3 to confirm that your program does not get killed if it needs more than one page of stack space. Modify the program to test successful execution upto 1MB of stack space, and getting killed beyond that.
Exercise 6.
Now that you have changed the process address space layout, you need to review parts of xv6 code that check user-provided system call arguments to make sure they point to valid addresses. Hint: argptr() is one such piece of code, but there are more. Make the necessary changes and write test programs to confirm user programs cannot crash the kernel by passing invalid addresses that fall in the not-yet-allocated parts of the stack VMA.
Exercise 7. Finally, make sure your new code works well with COW fork. Devise a few test programs to test that interaction. Document them in your README file.
As always, make sure you haven't broken anything in the kernel by passing all the tests in usertests. This completes part 2.
Now, after the ordeal of Part 2, it's time to reward yourself by doing something cool but not so complicated: VDSO. We have done most of the heavy lifting for you already; you just need to add a tiny bit of code.
As discussed in the class lectures, VDSO is a technique to provide kernel-level functionality to user-mode programs without making a system call. In other words, the goal is to turn system calls into normal function calls.
To this end, kernel maps the code and data necessary to implement some simple but frequently used calls, e.g., getpid(), to the user-mode address space. In our case we would like to make two such functions available to xv6 programs:
vdso_getpid() which returns the PID of the current process, and vdso_getticks() which returns the number of the timer ticks passed since the machine was booted.
Our approch is quite simple. First, we need to write the code that will be mapped to the user space. You will find this code in vdso_impl_entry.S (just one instruction) and vdso_impl.c (main code). This code is compiled and included in the kernel as a binary blob (similar to how the code for the init process is included in the kernel). Read the Makefile to learn how it is done. In this process, the linker will create two symbols, _binary_vdso_impl_start[] and _binary_vdso_impl_size[], which encode the location of this blob in the kernel binary.
This code is compiled with the assumption that it will be loaded in the user part of the address space at address VDSOTEXT which is 3 pages above KERNBASE. So, when a process is created, the physical page containing this code should be mapped at that address in its address space. To make your life easier, we have already done this mapping in a new function in vm.c, called allocvdso(). allocvdso() is called when a process address is initialized in fork() and exec().
Mapping of the VDSO code page is done in STEP 1 of allocvdso(). Read the code and make sure you understand what it is doing. You will be implementing steps 2 and 3 in which you will map the data pages.
There are two main functions in vdso_impl.c: _vdso_getticks() and _vdso_getpid(). These are the implementations of our two VDSO functions (there is also an auxiliary _vdso_getentry() that we'll discuss later). Read the implementations of these functions. To find the data they need, they assume there is some page, with the particular piece of data they need, mapped at a specific virtual address in the address space. You should be able to identify these addresses by reading the (very simple) code of these functions.
Exercise 8.
Implement STEP 2 in allocvdso(), by following the instructions included in the comments. This maps the page that contains the PID of the process, and will be used by the code in _vdso_getpid(). Remember that this page should be mapped as "read-only" in the address space.
Exercise 9.
Implement STEP 3 in allocvdso(), by following the instructions included in the comments. This maps the page that contains the number of clock ticks, and will be used by the code in _vdso_getticks(). Again, remember that this page should be mapped as "read-only" in the address space.
_vdso_getticks() returns whatever ticks value it finds in this page. Obviously, this value needs to be updated on each timer tick. We have provided a function to increment this value (inc_vdso_ticks() in sysvdso.c). You need to figure out where in the kernel you should call this function to update the ticks value.
Now, we have most of the needed machinery to make our VDSO calls work. The only missing part is the answer to the following question: how can a program know the address of _vdso_getpid() and _vdso_getticks() in order to call them? The answer: it has to ask the kernel. So, there is a new system call, vdso_entry, to just do that. Its implementation, sys_vdso_entry() in sysvdso.c, uses the mysterious _vdso_getentry() function mentioned above to find the addresses of these functions. Read its code, and how it is used by vdso_getpid() and vdso_getticks() in xv6's user-mode library (ulib.c) to unravel the mystery.
That's it! We have found a way to turn two kernel-level system calls to user-level function calls.
Exercise 10.
Write a few test programs to use vdso_getpid() and vdso_getticks(). In particular, check that the return value of vdso_getpid() is the same as the getpid system call. Also, run vdso_getticks() in a loop to see it returns monotonically-increasing timer tick values (well, it should, if your implementation is correct!).
This completes the lab.
You must include a file named README-lab3 with this assignment. The file should describe what you did, what approach you took, results of any measurements you made, which new files are included in your submission and what they are for, etc. Feel free to include any other information you think is helpful to us in this file; it can only help your grade.
If you are handing in late, add an entry to the file slack.txt noting how many late hours you have used both for this assignment and in total. (This is to help us agree on the number that you have used.) This file should contain a single line formatted as follows (where n is the number of late hours):
late hours taken: n
If you submit multiple times, we will take the latest submission and count late hours accordingly.
To submit your lab, type make handin in the xv6 directory. If submission fails, please double check that you have committed all of your changes, and read any error messages carefully before emailing the course staff for help.