Setting up Stable Xen Dom0 with Fedora: Xen 4.0.1 with Xenified Linux Kernel 2.6.32.13 in Fedora 12

By: Zhiqiang Ma In: Linux, Virtualization

Please refer to
Xen solution
for the more latest stable Xen Dom0 solution.

How to set up Xen Dom0 with Xenified Linux kernel in Fedora 12 will be introduced in this post. We use Xen 4.0.1 from xen.org and Xenified Linux kernel 2.6.32.13. This solution have been tested quite stable in our cluster.

Hardware:

Dom0′s hardware platform:

Motherboard: INTEL S5500BC S5500 Quad Core Xeon Server Board
CPU: 2 x Intel Quad Core Xeon E5520 2.26G (5.86GT/sec,8M,Socket 1366)
Memory: 8 x Kingston DDR-3 1333MHz 4GB ECC REG. CL9 DIMM w/Parity & Thermal Sensor
HD: 4 x WD WD10EARS 1 TB, SATA II 3Gb/s, 64 MB Cache

Linux system:

Fedora 12 x86_64

SELinux is disabled. Please refer here for detail: Disabled SELinux on Fedora.

ext3 is recommended for the file system of disk partition for /boot.

Update the system:

# yum update

The Xen and libvirt packages in Fedora should not be installed to avoid conflict.

# yum erase xen* libvirt

Build and install Xen hypervisor and tools

Download Xen 4.0.1

$ wget http://bits.xensource.com/oss-xen/release/4.0.1/xen-4.0.1.tar.gz
$ tar xf xen-4.0.1.tar.gz

Build Xen and tools

$ make xen
$ make tools

You may need to install packages depended by this. You can try this for solving the dependencies:

# yum groupinstall "Development Libraries" -y;
yum groupinstall "Development Tools" -y;
yum install transfig texi2html
libaio-devel dev86 glibc-devel
e2fsprogs-devel gitk mkinitrd
iasl xz-devel bzip2-devel
pciutils-libs pciutils-devel
SDL-devel libX11-devel gtk2-devel
bridge-utils PyXML qemu-common
qemu-img mercurial -y

Install Xen and tools

$ make install-xen
$ make install-tools

Build and install xenified Linux kernel

Download Linux kernel 2.6.32.13

$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.13.tar.bz2
$ tar xf linux-2.6.32.13.tar.bz2

Download 2.6.32 Xen patches v2

$ wget http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.32-2.tar.bz2
$ mkdir xen-patches-2.6.32-2
$ tar xf xen-patches-2.6.32-2.tar.bz2 -C xen-patches-2.6.32-2

Apply Xen patches

Apply all the patches downloaded above following the patch number. This patch.sh script can be used (we assume the patch and the kernel are in the same directory):

patch.sh:

#!/bin/bash
for P in `ls ../xen-patches-2.6.32-2/6*.patch1 | sort`
do
    patch -p1 -s -i $P
    if [ $? = 0 ]; then
        echo $P applied
    else
        echo "Error processing "$P
        exit 1
    fi
done

Put this script into Linux source directory and execute:

$ sh ./patch.sh

Configure Xenified Linux kernel

A working configuration file that I used can be downloaded directly from here:

config-for-xenified-linux-2.6.32.13

Just download this file, put it into the kernel source code file directory and rename it to .config .

Other than use my configuration file, you can also configure it by yourself by using “make menuconfig”.

Make sure you build the kernel with these components enabled:

Processor type and features  --->
 [*] Symmetric multi-processing support
 [*] Support sparse irq numbering
 [*] Enable Xen compatible kernel
 Preemption Model (No Forced Preemption (Server))  --->

Device Drivers  --->
 XEN  --->
 [*] Privileged Guest (domain 0)
 <*> Backend driver support (NEW)
 <*>   Block-device backend driver (NEW)
 <*>   Block-device tap backend driver (NEW)
 <*>   Block-device tap backend driver 2 (NEW)
 <*>   Network-device backend driver (NEW)
 (8)     Maximum simultaneous transmit requests (as a power of 2) (NEW)
 [ ]     Pipelined transmitter (DANGEROUS) (NEW)
 < >     Network-device loopback driver (NEW)
 <*>   PCI-device backend driver (NEW)
 PCI Backend Mode (Virtual PCI)  --->
 [ ]     PCI Backend Debugging (NEW)
 < >   TPM-device backend driver (NEW)
 <M>   SCSI backend driver (NEW)
 <M>   USB backend driver (NEW)
 <M> Block-device frontend driver
 <M> Network-device frontend driver
 <M>   Network-device frontend driver acceleration for Solarflare NICs (NEW)
 <M> SCSI frontend driver (NEW)
 <M> USB frontend driver (NEW)
 [*]   Taking the HCD statistics (for debug) (NEW)
 [ ]   HCD suspend/resume support (DO NOT USE) (NEW)
 <*> User-space granted page access driver (NEW)
 <*> Framebuffer-device frontend driver (NEW)
 <*>   Keyboard-device frontend driver (NEW)
 [*] Disable serial port drivers (NEW)
 <*> Export Xen attributes in sysfs (NEW)
 (256) Number of guest devices (NEW)
 Xen version compatibility (no compatibility code)  --->
 [*] Place shared vCPU info in per-CPU storage (NEW)

Build kernel

$ make -j16

Install modules and kernel

# make modules_install install

Configure grub

Add one entry for Xen in /boot/grub/grub.conf. This is an example entry:

title Xen 4.0.1 - Xenified Linux 2.6.32.13
  root (hd0,0)
  kernel /xen-4.0.1.gz console=vga vga=ask noreboot
  module /vmlinuz-2.6.32.13 ro root=/dev/mapper/VolGroup-LogVol_root noiswmd LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us
  module /initramfs-2.6.32.13.img

The root and other parameters may be different depending on the configuration.

Make Xend and Xendomains services automatically start when system boots

# cd /etc/init.d/
# chkconfig --add xend
# chkconfig --add xendomains

Check whether Xend and Xendomains services are automatically started in level 3-5:

# chkconfig --list | grep xend

It should be like this:

xend               0:off    1:off    2:off    3:on    4:on    5:on    6:off
xendomains         0:off    1:off    2:off    3:on    4:on    5:on    6:off

Enjoy the fun now!

After booting the system, you can try to use xm to check xen info

# xm info

Then xm command can be used to start up DomUs.

This is one working configuration file for one DomU that I use:

name="10.0.1.201"
vcpus=2
memory=16384
disk = ['tap:aio:/lhome/xen/vm-10.0.1.201/vmdisk0,xvda,w' ]
# disk = ['file:/lhome/xen/vm-10.0.1.201/vmdisk0,xvda,w' ]
vif = ['bridge=eth0']
bootloader = "/usr/bin/pygrub"
on_reboot = 'restart'
on_crash = 'restart'

Here we use the blktap backed VBD device which has much better performance than Linux blkback backed VBD device.

Making the performance more stable

Allocating dedicated CPU core and memory for Dom0 may provide more stable performance for the Xen platform. Please refer to Managing Xen Dom0′s CPU and Memory/ for detailed instruction.

Problems

Problems and solutions can be found here: Problems during Installing Xen Dom0 in Fedora