From OSR

Prototype Nway LWK

This is a prototype OS that allows us to explore different feature sets with a very stripped down, simple OS. It does not have support for many devices or any interactive features, but is only a few thousand lines of code.

CVS Access

The Nway CVS repository is hosted on www.osresearch.net. The following environment variable needs to be set if checking out from remote:

   CVS_RSH=/usr/bin/ssh

The repository can then be checked out with (the repository path is only needed for the checkout):

   cvs -d www.osresearch.net:/home/cvs co nway

Then cross your fingers, wiggle your nose, and:

   cd ./nway
   make

Roadmap

Kernel

In roughly order of importance:

64-bit support

~~PAE support Bug 109~~

We have support for 36-bit addressing and 2 M pages now (Hudson 17:23, 17 January 2007 (EST))

Memory models with the x86-64 ABI (Bug 92

We're not exactly compliant with the x86-64 ABI for memory mapping models. One summary of the models.

~~SMP and threaded cores Bug smp~~

Our target is large N n-way machines, which we can simulate with bochs. This is mostly in place now for booting SMP nodes, although the AP cores just spin once they reach 32-bit protected mode. Much locking will be required to fix the region, pagetable, console and other structures. (Hudson 16:31, 16 January 2007 (EST))

User level SMP is now in place as well, allowing the user code to divy up the processors as they see fit. (Hudson 17:19, 17 January 2007 (EST))

~~Locking Bug 108~~

Locks are implemented using CMPXCHG (Hudson 12:33, 22 January 2007 (EST))

~~Page table free list~~

I implemented this since it was fairly easy. We're were using the os_bits field in the first entry of a page table to track which page tables from the slab are inuse. This meant a linear search to find a free one; it now builds a list using the ->frame field of the PTE to track the next PTE in the list. Since the ->present bit is not set, the MMU hardware will ignore the frame field. (Hudson 17:42, 8 January 2007 (EST), updated 18:38, 8 January 2007 (EST))

Better exception handling

User pointer verification Bug 94

We trust all user pointers passed in now. That's not right!

~~System calls do not match Linux registers Bug 97~~

This is fixed for i386, but will need to be repeated for x86_64 (Hudson 11:42, 10 January 2007 (EST))

Region overwrite and failure semantics

The lwk panics if overwrite is not specified and a page is already mapped.

SYSENTER support

Currently all entry to the kernel is via INT. SYSENTER / SYSCALL is an order of magnitude faster and will help reduce latency.

Nway Discovery API

How do we make the cores, NUMA spaces, etc visible to the user?

~~Serial console~~ Bug 111

PCI support
Ethernet card support

Depends on PCI.

NUMA mapping of page tables Bug 106

The user should be able to select where page tables are allocated. This way the NUMA nodes don't have to cross the bus for translations. This would introduce holes into the address space, but may be worth it for users that want it.

Doxygen comments Bug doxygen

To aid new developers in understanding the code, we should add doxygen comments on all the major functions.

Userspace

~~C userspace~~

Done. A linker script locates the user binary at 0x4000_0000 and _lwk_main() handles the mappings. (Hudson 17:44, 8 January 2007 (EST))

ELF header decoding Bug 99

I'll do this one, too. It's pretty easy and I've done it a few times already... (Hudson 17:42, 8 January 2007 (EST))

~~Region mapping~~

Implemented, but there is still no bounds checking (Hudson 21:43, 7 January 2007 (EST))

~~uClibc linking (shared with kernel?)~~

Not shared with the kernel, but linked on its own. Sharing the text segment from it would be very difficult and not worth the 20 kb it would save us. (Hudson 21:43, 7 January 2007 (EST))

Add support for Thread-Local Storage.

This will require understanding the ELF extensions described here and updating crt0 to set things up properly. The kernel will probably have to have a separate GDT for each core.

Releases

None of these releases are intended for the general public to use. They are internal development versions.

Release 0.01

The 10^-2 release adds serial console and SMP user mode with user control of threads. The SMP locking is broken, especially with the console access.

Release 0.001

The 10^-3 release is a big step forward. We're getting to the point of almost having a real operating system! The user code is now 100% C and starts at main(), while our crt0.o handles the region setup and remapping prior to the user main.

User apps are loaded at 0x0100_0000 and the _lwk_main() routine creates regions for them

at the start virtual address of 0xC000_0000 (1 GB). Once the text and data regions are created, it does a long jump to main() in the new address space.

The system calls are working, although they are 100% trusted at this point and don't check for many failure modes.

printf() from the user goes to the VGA console.

Page faults from the user correctly reflect the faulting address. The discussed user level page fault handlers have not been implemented yet (we abort the OS on fault).

Page fault (#14) errors correctly reporting faulting address in register cr2

Have I mentioned how much I hate the i386 architecture?

18 KB tar file is attached. We've grown to about 2500 lines of code, which seems like alot, but we have lots of comments and most of the files are very, very short. It's time to re-organize the lwk directory a bit to move the system call handlers out of the high-level interrupts.c file.

  42   112   583 lwk-0.001/liblwk/lwk-user.lds
  16    31   232 lwk-0.001/liblwk/Makefile
 110   237  1703 lwk-0.001/liblwk/syscalls.h
  36    72   442 lwk-0.001/liblwk/libc.c
  73   236  1504 lwk-0.001/liblwk/crt0.c
  96   201  1223 lwk-0.001/liblwk/syscall.h
  20    57   392 lwk-0.001/liblwk/liblwk.h
  32    66   496 lwk-0.001/tests/Makefile
  18    54   342 lwk-0.001/tests/user-test.c
  64   200  1284 lwk-0.001/lwk/lwk.h
  34   103   603 lwk-0.001/lwk/user-test.S
  44    97   621 lwk-0.001/lwk/Makefile
 185   776  4278 lwk-0.001/lwk/trap.S
  79   179  1321 lwk-0.001/lwk/region.h
  20    49   356 lwk-0.001/lwk/console.h
  56   154   808 lwk-0.001/lwk/lwk.lds
 183   473  3216 lwk-0.001/lwk/interrupts.c
 279   805  5530 lwk-0.001/lwk/main.c
  52   207  1387 lwk-0.001/lwk/multiboot.h
  45   111   844 lwk-0.001/lwk/console-vga.h
  52   126   774 lwk-0.001/lwk/console.c
 247   803  4950 lwk-0.001/lwk/region.c
 111   396  2218 lwk-0.001/lwk/head.S
  36    73   517 lwk-0.001/lwk/semaphore.h
 127   369  2413 lwk-0.001/lwk/interrupts.h
 321   960  5947 lwk-0.001/lwk/pagetable.h
  16    33   220 lwk-0.001/lwk/context.h
  57   148   874 lwk-0.001/lwk/console-vga.c
  25    50   318 lwk-0.001/Makefile
  77   259  1489 lwk-0.001/Makefile.common
2553  7437 46885 total

Release 0.0001

The release team has declared that the real 10^-4 release (0.0001) is now available. The tree has been re-designed to split the lwk, the user library and the user code into separate directories now and added a top level makefile.

The other big news in this release is that you can now have a C user runtime and the system calls have real wrappers around them. The ELF image is still not in the right place in memory, but I'll fix that shortly in user land using the region code that will soon be implemented.

The user code links against uClibc now as well, but since the memory addresses aren't right it doesn't actually work yet.

Release 0.00001

The 10^-5 release has real syscalls that are directed with an interesting way to do the dispatch inside of the assembly handler. There are no error checks in the do_con_printf() syscall (INT $0x20), so you can crash the kernel with it.

The other major change is that the pte_slab[] is not allocated but instead extends from the end of the image to the KERNEL_END value. This is specified in the lwk.lds script to make it page aligned and at the right location.

Release 0.000001

The 10^-6 release is the first one that can boot via grub, setup everything and run a minimal user mode program. Current features:

It boots!
It creates the regions
It jumps into user mode
User mode does a syscal via INT 0x80 to print the arg in %eax
User mode sleeps
User mode does another syscall
User mode segfaults itself

Unimplemented features on the roadmap:

Semaphores (waiting until SMP)
Functional user mode syscalls
User level memory management
Job load (user-test.o is bundled into the kernel)
Interrupts
Segfault does not reflect faulting address in cr2 (!?)
Recovery from any interrupt
Serial console
PCI
Ethernet
SMP

Transcript of a boot and user mode run:

   LWK: Built on Jan  4 2007 at 15:42:41
   Memory: 639 KB + 255 MB (flags=000007e7)
   Kernel: '/lwk console=vga'
   User memory: 239 MB
   syscall: decafbad!
   syscall: deadbeef!
   Exception 13 at eip = 001b:0100101b sp = 0023:0feffff0
   err = 00000000 efl = 00010056 cr2 = 00000000 cr0 = 80000011
   eax = deadbeef ebx = 00000000 ecx = 00103024 edx = 01001000
   esp = 0007ffe4 ebp = 0009ffe0 esi = deadbeef edi = 00102969
   
   ***** PANIC! UNHANDLED INTERRUPT: HALT *****

Navigation

Nway LWK