Warning: Parameter 1 to Language::getMagic() expected to be a reference, value given in /opt/local/apache2/htdocs/wiki/includes/StubObject.php on line 58
Xen - OSR

Xen

From OSR

Jump to: navigation, search

Contents

Xen Documentation

Hypercall Notes

  • update_va_mapping() needs a machine frame number, not a local frame number.

Building Xen 3.0.4-1 

hg clone http://xenbits.xensource.com/xen-3.0.4-testing.hg
  • Apply the xen-3.0-xt patch
  • Run 'make world'
    • This will download linux-2.6.16 (39 MB) and configure it as dom0
    • Expect about 30 min for the build
    • Make another pot of coffee...


Booting on Bochs

  • Install the files
    • Copy xen/xen.gz /tmp/loop/
    • Copy linux-2.6.16.33/vmlinuz /tmp/loop/vmlinuz-xen0
    • sudo make -C linux-2.6.16.33-xen modules_install
  • Generate the ramdisk 
   sudo mkinitrd -v /tmp/loop/initrd-xen0.img 2.6.16.33-xen
  • Configure /tmp/loop/grub/grub.conf for bochs with the serial console: 
   title Linux-Xen
       kernel /xen.gz com1=9600,8n1
       module /vmlinuz-xen0 console=ttyS0,9600
       module /initrd-xen0.img
  • sync ; sync ; sync
  • Boot bochs
    • Expect constant error messages like: 
   [CPU0 ] allow_io(): IO addr 3d5 (len 1) outside TSS IO permission map (base=8000, limit=67) #GP(0)
   01170730750e[CPU0 ] write_virtual_checks(): write beyond limit, r/w
   01170731425e[CPU0 ] read_virtual_checks(): read beyond limit


Porting to Redstorm

Setup

  • Copy xen kernel into full-nightly tree: 
objcopy -O binary xen ~/src/full-nightly/install/linux/bootable/vmlinux-ss-lustre26.bin
  • Modify ~/src/full-nightly/install/linux/bootable/parameters file to specify: 
console=ttyS0
earlyprintk=ttyS0
ip=dhcp
ro
nfsroot=192.168.1.1:/rr/current_2.6,nolock,hard,intr,nfsvers=3,tcp,rsize=16384,wsize=16384
pci=lastbus=5

Xen defaults to mapping /dev/ttyS0 to its own console; we reuse it to get our console messages out. The lastbus is most important -- otherwise the ethernet card will not be detected. The NFS root parameters are spelled out since we are using TCP; the default is UDP and the server is not listening.

However, this still fails to open the initial console on the root filesystem. rscsmw2 is a 2.4.21 server with a broken nfsd that will not properly allow access to /dev files on a read-only export. This means that we can't access the system interactively. If you wait about a minute you will be able to ssh into rsclogin108 to run non-interactive commands.

Starting udev by hand ('mount -t tmpfs none /dev ; udevstart') creates the files and fixes the access to them, but still doesn't allow interactive logins.

initramfs / Linux / Xen

  • Build initramfs cpio file FIRST
  • Build Linux kernel (links in usr/initramfs_data.cpio)
  • Build xen kernel (links in ../linux-2.6.16.38-xen/vmlinux)
  • bjcopy -O binary xen to install directory
  • rs_init
  • rs_boot --l

Booting

Debugging

  • xen-syms has the binary before stripping, so you can use 'objdump -D xen-syms' to trace code execution.
  • Console
  • SeaStar

Good Linux Boot 

rscsmw2:~/nightly/modules: rs_boot --l n0.cx24y2
booting n0.board0.cx24y2 rs_role i
Booting LINUX on n0.board0.cx24y2
Executing /net/home/tbhudso/src/full-nightly/install/bin/snos64/bootlinux -n 0 -d -v -c /cluster_new//vms/cx24y2c0 -i /net/home/tbhudso/src/full-nightly/install
starting cpuid
 sync_api at addr 0xffffffe0: 0xfeedfeed 0x0 0x0 0xf5a
syncapi is 0xfeedfeed 0x0 0x0 0xf5a
dual core flag is 0
starting htlink
 sync_api at addr 0xffffffe0: 0xfeedfeed 0x0 0x0 0xf5a
syncapi is 0xfeedfeed 0x0 0x0 0xf5a
starting coldstart+ dispatch loop
 sync_api at addr 0xffffffe0: 0xf00df00d 0x6c646 0x0 0x0
starting sys64/ucode on core 0
entry point at 0xf0000
 sync_api at addr 0xffffffe0: 0xf00df00d 0x91a8c 0xf0000 0x3a
starting sys64/linuxpci on core 0
entry point at 0xf2639
 sync_api at addr 0xffffffe0: 0xf00df00d 0x294dd9 0xf2639 0x0
starting sys64/linuxmptable on core 0
entry point at 0xf0d0a
 sync_api at addr 0xffffffe0: 0xf00df00d 0x2aa7d4 0xf0d0a 0x1
starting sys64/cache on core 0
entry point at 0xf0000
 sync_api at addr 0xffffffe0: 0xf00df00d 0x48aa1c 0xf0000 0x0
starting sys64/linuxcmdline on core 0
entry point at 0xf0735
 sync_api at addr 0xffffffe0: 0xf00df00d 0x648883 0xf0735 0x90000
Bootdata ok (command line is earlyprintk=rcal0 load_ramdisk=1 ramdisk_size=100000 acpi=off console=ttyL0 rootfs=nfs-shared bootnodeip=192.168.1.1 bootpath=/rr/current_2.6 bootproto=dhcp cray_bin= pci=lastbus=5 quiet harness=1)
booting XT3 x86_64 kernel... Linux version 2.6.5-7.252-ss (geeko@buildhost) (gcc version 3.2.3) #2 Fri Jul 14 08:50:36 MDT 2006
Current console loglevel 7 Setting to 15
These are the boot parameters provided to Linux:
EXT_MEM_K 64512 0xfc00
ALT_MEM_K 2094528 0x1ff5c0
MOUNT_ROOT_RDONLY 0x1
RAMDISK_FLAGS 0x0
ORIG_ROOT_DEV 133 0x85
AUX_DEVICE_INFO 0x0
LOADER_TYPE 0x32
KERNEL_START 1048576 0x100000
INITRD_START 262144000 0xfa00000
INITRD_SIZE 2542938 0x26cd5a
CMD LINE [earlyprintk=rcal0 load_ramdisk=1 ramdisk_size=100000 acpi=off console=ttyL0 rootfs=nfs-shared bootnodeip=192.168.1.1 bootpath=/rr/current_2.6 bootproto=dhcp cray_bin= pci=lastbus=5 quiet harness=1]
Number of E820 maps 3
Address 0 for 0
Size 655360 for 0
Type 1 for 0
Address 1048576 for 1
Size 4226809856 for 1
Type 1 for 1
Address 4294967296 for 2
Size 2147483648 for 2
Type 1 for 2
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
 BIOS-e820: 0000000000100000 - 00000000fc000000 (usable)
 BIOS-e820: 0000000100000000 - 0000000180000000 (usable)
init_memory_mapping
kernel direct mapping tables upto 10180000000 @ 8000-f000
Scan SMP from 0000010000000000 for 1024 bytes.
INITRD_START 0xfa00000 INITRD_SIZE 0x26cd5a pfn 0x6442450944
On node 0 totalpages: 1572864
  DMA zone: 4096 pages, LIFO batch:1
  Normal zone: 1568768 pages, LIFO batch:31
  HighMem zone: 0 pages, LIFO batch:1
Intel MultiProcessor Specification Dump
mpf_sig bytes _MP_
mpf_phy addrs 0x20 0000010000000020
mpf_len 1
mpf_spec 4
mpf_cksum 128
mpf_feature1 0
mpf_feature2 0
mpf_feature3 0
mpf_feature4 0
mpf_feature5 0
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
Intel MultiProcessor Config table Dump
mpc_sig bytes PCMP
mpc_len 136
mpc_spec 4
mpc_checksum 0x76
OEM bytes are:
CRAY INC
Product Id bytes are:
XT3         
OEM ptr is 0x0 oemsize 0 oem_count 10
APIC address is 0xfee00000 -18874368
OEM ID: CRAY INC <6>Product ID: XT3          <6>APIC at: 0xFEE00000
Dumping mpc Proc table
Proc TYpe 0
Proc local apicid 0
Proc local apicver 16
Proc cpuflag 0x3
Proc cpufeature/signature 0xf5a
Proc featureflag 0x381
Processor #0 15:5 APIC version 16
    Bootup CPU
Dumping mpc BUS table
BUS TYpe 1
BUS busid 1
BusType  bytes are:
PCI   
Bus #1 is PCI   
Dumping mpc BUS table
BUS TYpe 1
BUS busid 2
BusType  bytes are:
PCI   
Bus #2 is PCI   
Dumping mpc BUS table
BUS TYpe 1
BUS busid 3
BusType  bytes are:
PCI   
Bus #3 is PCI   
Dumping mpc IO APIC table
IO APIC TYpe 2
IO APIC ID 1
IO APIC apicver 17
IO APIC flags 1
IO APIC apicaddr 0xfecff000 -19927040
I/O APIC #1 Version 17 at 0xFECFF000.
Dumping mpc IO APIC table
IO APIC TYpe 2
IO APIC ID 2
IO APIC apicver 17
IO APIC flags 1
IO APIC apicaddr 0xfecfe000 -19931136
I/O APIC #2 Version 17 at 0xFECFE000.
Dumping mpc_config_intsrc table....
MPC intsrc type 3
MPC intsrc irqtype 0
MPC intsrc irqflag 15
MPC intsrc srcbus 2
MPC intsrc srcbusirq 4
MPC intsrc dstapic 1
MPC intsrc dstirq 0
Int: type 0, pol 3, trig 3, bus 2, IRQ 04, APIC ID 1, APIC INT 00
Dumping mpc_config_intsrc table....
MPC intsrc type 3
MPC intsrc irqtype 0
MPC intsrc irqflag 15
MPC intsrc srcbus 2
MPC intsrc srcbusirq 5
MPC intsrc dstapic 1
MPC intsrc dstirq 1
Int: type 0, pol 3, trig 3, bus 2, IRQ 05, APIC ID 1, APIC INT 01
Dumping mpc_config_intsrc table....
MPC intsrc type 3
MPC intsrc irqtype 0
MPC intsrc irqflag 15
MPC intsrc srcbus 3
MPC intsrc srcbusirq 4
MPC intsrc dstapic 2
MPC intsrc dstirq 0
Int: type 0, pol 3, trig 3, bus 3, IRQ 04, APIC ID 2, APIC INT 00
Dumping mpc_config_intsrc table....
MPC intsrc type 3
MPC intsrc irqtype 0
MPC intsrc irqflag 15
MPC intsrc srcbus 3
MPC intsrc srcbusirq 5
MPC intsrc dstapic 2
MPC intsrc dstirq 1
Int: type 0, pol 3, trig 3, bus 3, IRQ 05, APIC ID 2, APIC INT 01
Setting APIC routing to flat
Processors: 1
set_pte_phys ffffffffff5fd000 to fee00000
spp_getpage 0000010006441000
mapped APIC to ffffffffff5fd000 (        fee00000)
set_pte_phys ffffffffff5fc000 to fecff000
mapped IOAPIC to ffffffffff5fc000 (00000000fecff000)
set_pte_phys ffffffffff5fb000 to fecfe000
mapped IOAPIC to ffffffffff5fb000 (00000000fecfe000)
Built 1 zonelists
Kernel command line: earlyprintk=rcal0 load_ramdisk=1 ramdisk_size=100000 acpi=off console=ttyL0 rootfs=nfs-shared bootnodeip=192.168.1.1 bootpath=/rr/current_2.6 bootproto=dhcp cray_bin= pci=lastbus=5 quiet harness=1
WARNING more than 4GB of memory but no IOMMU.
WARNING 32bit PCI may malfunction.
Initrd: Mounting /proc /sys /dev ...
Initrd: done.
Initrd: inserting Cray portals driver.
Initrd: Inserting RCA driver
modprobe: FATAL: Module rca not found.

insmod /lib/modules/2.6.5-7.252-ss/kernel/drivers/net/e1000/e1000.ko 
Inserting the e1000 driver, or trying anyway
Inserting the tg3 driver, or trying anyway
Inserting the r8169 driver, or trying anyway
modprobe: FATAL: Module r8169 not found.

Inserting the s2io driver, or trying anyway
Initrd: Invoking dhcpcd on eth0...
Initrd: dhcpcd complete
Initrd: Using rootfs config: nfs-shared...
192.168.1.1:/rr/current_2.6 on /new_root type nfs (ro,nolock,hard,intr,vers=3,tcp,rsize=16384,wsize=16384,addr=192.168.1.1)
Initrd: rootfs mount succeeded.
Initrd: Going to pivot_root...
Initrd: Completed root pivot
Initrd: Re-mounting /proc...
Initrd: done.
Initrd: Mounting NID specific /etc...
WARNING: did not find NID specific /etc. Using default.
Initrd: done.
Initrd: Setup /dev
Initrd: Setup /tmp
Initrd: done.
**********************************************************
    HOSTNAME: rsclogin108
**********************************************************
Retrieved from "http://www.osresearch.net/wiki/index.php/Xen"