This document describes how to compile, package and install the Xenomai real time Linux kernel extension (version 2.0) on the Debian GNU/Linux sid distribution.
Please note that I am currently working on packaging Adeos and Xenomai cleanly for Debian. I have recent Adeos packages readily available from my Debian APT repository. I will hopefully have packages ready for Xenomai version 2.1, although I am still working on packaging the kernel-space part only. I will post more information in my weblog.
Xenomai is the continuation of RTAI's previous development branch called Fusion, that has been split from RTAI on 2005-10-08. Fusion / Xenomai is now independent from RTAI, hence this article does not address the installation of any RTAI branch (e.g. Magma), although the installation processes should be very similar. For more information about the two projects, see:
Xenomai relies on the Adeos real-time microkernel, to handle real time interruption dispatching. Adeos is implemented as a patch to the Linux kernel.
In Debian GNU/Linux, the Adeos patch is normally provided in package kernel-patch-adeos, but this package has not been updated for Linux kernels versions more recent than 2.6.11 (as of 2005-11-01) while the current version of Linux in Debian (and the one that I is considered in this article) is 2.6.14.
This can be checked by looking for the line that defines the KVERSIONS variable in the /usr/src/kernel-patches/i386/apply/adeos script file that is installed by the kernel-patch-adeos package.
It is therefore necessary to download the patches directly from the Adeos website, at http://download.gna.org/adeos/, and to build a new Debian GNU/Linux package for it.
Download the Adeos patch for Linux kernel version 2.6.14, that is now named I-pipe, using the following commands:
> mkdir -p /tmp/rtai/adeos > cd /tmp/rtai/adeos > wget http://download.gna.org/adeos/patches/v2.6/adeos/i386/adeos-ipipe-2.6.14-i386-1.0-09.patch
Get the sources of the current Debian GNU/Linux kernel-patch-adeos package (version 20050809-1 as of 2005-11-01):
> mkdir -p /tmp/rtai/adeos/srcpackage > cd /tmp/rtai/adeos/srcpackage > apt-get source adeos > cd adeos-20050809
Add the downloaded patch into the source package, which is very easy because this package uses Debian's kpatches helper tools for building the package:
> cp ../../adeos-ipipe-2.6.14-i386-1.0-09.patch . > cat >> debian/kernel-patch-adeos.kpatches <<EOF Patch-file: adeos-ipipe-2.6.14-i386-1.0-09.patch Kernel-version: 2.6.14 Architecture: i386 Path-strip-level: 1 EOF
Change the package version number (you can put the extension text you like after the revision number -1, but there are a few restrictions as specified in the Debian Policy):
> dch --newversion=20050809-1.rlenglet
When the dch command makes you edit the changelog file, add a comment line like:
* added patch for kernel version 2.6.14 from upstream)
Install packages required for building this package:
> sudo apt-get build-dep kernel-patch-adeos
Build the package:
> debuild
Install the newly generated package:
> cd .. > sudo dpkg -i kernel-patch-adeos_20050809-1.rlenglet_i386.deb
Now, check that the installed package is OK. The command:
> grep 2.6.14 /usr/src/kernel-patches/i386/apply/adeos
should output the two lines:
KVERSIONS=(blahblahblah 2.6.14) PATCHFILES=(blahblahblah/adeos-ipipe-2.6.14-i386-1.0-09.patch.gz")
Check that the patch file has been correctly installed by the package:
> ls /usr/src/kernel-patches/diffs/adeos/adeos-ipipe-2.6.14-i386-1.0-09.patch.gz
It is possible to use either Debian's patched Linux kernel sources, or the vanilla Linux kernel sources. Since Debian's patched kernel works fine with Xenomai and it is more natural to use it on Debian, we will use it in the following instead of the vanilla sources. Install Debian's patched Linux kernel sources:
> sudo apt-get install linux-source-2.6.14 > cd /usr/src > sudo tar xjf linux-source-2.6.14.tar.bz2 > sudo ln -s linux-source-2.6.14 linux > cd linux-source-2.6.14
If you are already running a 2.6.14 version of the Linux kernel from an official Debian binary package, e.g. package linux-image-2.6.14-1-686, you should use the kernel configuration from that kernel (that activates everything as modules, etc. as always in Debian) as the initial configuration file for the Xenomai-enhanced kernel:
> sudo cp /boot/config-2.6.14-1-686 ./.config
Otherwise, edit a new initial configuration (although it may be difficult to create such a hand-configured Linux kernel work with Debian the first time: it must have initrds enabled, etc.):
> sudo make menuconfig
To build the kernel, any GCC compiler available in Debian's sid distribution is OK: either 3.3, 3.4 or 4.x versions have been tested to work fine (and all Debian kernel images are built using GCC 4.0 without trouble).
Build the kernel image package with the Adeos patch applied, using Debian's kernel-package commands:
> sudo apt-get install kernel-package
> sudo make-kpkg --added-patches adeos \
--config menuconfig --initrd \
--append-to-version -1-686-ipipe --stem linux \
kernel_image
When this command makes you edit the Linux kernel configuration, it is necessary to activate I-pipe and to deactivate some kernel features (essentially some power management features) that would ruin the latencies in the real-time kernel:
Processor type and features:
Interrupt pipeline is enabled (this is to enable the Adeos implementation - I-pipe).Local APIC support on uniprocessors and IO-APIC support on uniprocessors are enabled.Power management options (ACPI, APM):
ACPI (Advanced Configuration and Power Interface) Support:
Processor.APM (Advanced Power Management) BIOS Support:
APM (Advanced Power Management) BIOS support.CPU Frequency scaling:
CPU Frequency scaling.Once the package has been created, install it:
> cd .. > sudo dpkg -i linux-image-2.6.14-1-686-ipipe_10.00.Custom_i386.deb
If you use either LILO or Grub for bootloading, you need not do anything to make this kernel selectable at boot time: this is done automatically thanks to the Debian GNU/Linux kernel packaging mechanisms. Otherwise, you need to configure your bootloader properly. Reboot your computer, and boot with the new kernel.
Check that Adeos is working. The command:
> dmesg | grep -i I-pipe
should output the line:
I-pipe 1.0-09: pipeline enabled.
And the command:
> cat /proc/ipipe/Linux
should show that Linux has been registered into the Adeos/I-pipe interrupt pipeline.
If the /proc/ipipe directory does not exist, Adeos/I-pipe has not been integrated and enabled into the currently running kernel: you must check that you have performed all the steps above.
Packages for RTAI already exist in Debian GNU/Linux sid (rtai, rtai-doc and rtai-source), but there is no package yet for Xenomai (or for RTAI's previous Fusion branch it is the continuation of).
Until a package is created for Xenomai, the following will guide you to install it "by hand", not in a package-managed way.
Download the latest version of the Xenomai sources (as of 2005-11-01, the latest and first official release is 2.0):
> mkdir -p /tmp/xenomai > cd /tmp/xenomai > wget http://download.gna.org/xenomai/stable/xenomai-2.0.tar.bz2 > tar xjf xenomai-2.0.tar.bz2 > cd xenomai-2.0
Configure Xenomai:
> make menuconfig
and in the configuration application:
General:
Installation directory to /usr/lib/xenomai.Linux build tree to /usr/src/linux.Enable expert configuration mode.Nucleus:
Interrupt shield support.Watchdog support (this is sometimes useful).Scalability:
O(1) scheduler.Machine (x86):
Enable SEP instructions for syscalls to improve latency.SMI workaround:
Enable SMI workaround to improve latency.APIs, enable all options.Drivers, enable all options.
To increase performance, you may enable the Enable SEP instructions for syscalls option in the configuration application, ONLY IF two conditions are met: your CPU supports the SEP instruction, AND your system has NPTL enabled and your RTAI applications use it.
To check if your CPU supports the SEP instruction, the command:
> cat /proc/cpuinfo | grep '^flags.*sep'
must output one line that contains the sep keyword if it supports it.
To make applications use the NPTL threads library instead of the older LinuxThreads library, you must run a Linux kernel of the 2.6 branch (which is the case if you run the Adeos-patched 2.6.14 kernel installed as described above), AND you must install NPTL-enabled C libraries:
> sudo apt-get install libc6-i686
I have tested that this works in practice.
Compile Xenomai:
> make all
Install Xenomai:
> sudo make install-nodev
You should run Udev to automatically manage Xenomai's device files in /dev.
However, if Xenomai is installed with the install target instead of install-nodev, Xenomai's Udev rules are appended to to the /etc/udev/udev.rules file, which is not the "Debian way of doing".
In Debian, Udev rules should instead be added by creating new files in the /etc/udev/rules.d/ directory.
Therefore, to correctly install Xenomai's Udev configuration files you must instead run:
> sudo install --mode 644 nucleus/udev/*.rules /etc/udev/rules.d/
Alternatively, only if you don't run Udev, you can create static device files by hand in /dev:
> for n in `seq 0 31`; do f="/dev/rtp$n"; sudo mknod -m 666 $f c 150 $n; done > sudo mknod -m 666 /dev/rtheap c 10 254
Xenomai requires to have local APIC enabled for the CPUs by the kernel, which is always the case for SMP systems, but is not always the case by default for single-CPU systems.
For instance, local APIC is not automatically enabled on my (single-CPU) IBM ThinkPad X31 notebook.
To force the enabling of the local APIC at kernel startup, the lapic parameter must be passed to the kernel at boot time.
For instance, using Grub as a bootloader, only the lines starting with # kopt in the /boot/grub/menu.lst file must be modified, to append the lapic keyword (preceded by a space).
For instance, on my system, I have modified the line:
# kopt=root=/dev/hda4 ro
to become:
# kopt=root=/dev/hda4 ro lapic
Then, update-grub must be called to update the boot loader:
> sudo /sbin/update-grub
Then, you must reboot to activate this option.
To solve the paths to Xenomai's libraries, modify LD's configuration:
> sudo echo "/usr/lib/xenomai/lib" >> /etc/ld.so.conf > sudo ldconfig
Load the Xenomai modules into the kernel:
> sudo insmod /usr/lib/xenomai/modules/xeno_hal.ko > sudo insmod /usr/lib/xenomai/modules/xeno_nucleus.ko > sudo insmod /usr/lib/xenomai/modules/xeno_native.ko
Those modules print kernel logs when starting. The command:
> dmesg
should at least output the following three lines if nothing went wrong:
I-pipe: Domain Xenomai registered. Xenomai: hal/x86 loaded. Xenomai: real-time nucleus v2.0 (Surfing With The Alien) loaded. Xenomai: starting native API services.
To measure the real-time interrupt latencies of your setup of Linux / Adeos and Xenomai, Xenomai provides the xeno-test command:
> /usr/lib/xenomai/bin/xeno-test
See the man page for options:
> man /usr/lib/xenomai/man/man1/xeno-test.1
This application uses sudo(8) to execute latency test real-time applications.
You must therefore properly configure sudo for your user, to let it execute Xenomai's test applications as root.
If xeno-test execute without problem, any Xenomai application should execute fine also.
If when loading the xeno_hal.ko module, you get the following line in dmesg's output:
Xenomai: Intel chipset found and SMI workaround not enabled,
you may encounter high interrupt latencies.
your computer has SMI enabled, which could significantly increase latency: SMIs are non-maskable interrupts that can happen even if a real-time interrupt handler is being executed, hence increasing its execution time / latency.
Xenomai provides a workaround to disable SMI interrupts, which must be enabled at configuration time: enable Enable SMI workaround in the Machine (x86) / SMI workaround configuration section.
If some devices do not work anymore when loading Xenomai modules with the SMI workaround enabled, it probably means that those devices rely on SMI. You should try to re-enable some SMI interrupts in Xenomai's SMI workaround configuration, one by one. In the configuration application:
Machine (x86):
SMI workaround:
Globally disable SMI.Enable Intel-Specific USB2 SMI (disable if it has no effect).Enable legacy USB2 SMI (disable if it has no effect).Enable periodic SMI (disable if it has no effect).Enable TCO SMI (disable if it has no effect).Enable microcontroller SMI (disable if it has no effect).Enable APM SMI (disable if it has no effect).Enable legacy USB SMI (disable if it has no effect).Enable ACPI BIOS SMI (disable if it has no effect).
Copyright © 2005 Romain Lenglet.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
A copy of the license is available from the GNU project web site at: http://www.gnu.org/copyleft/fdl.html.