…and still be able to mount a root file-system stored on a USB mass-storage.

The idea is to build a very small kernel with the bare minimum compiled-in and all the rest as modules which are stored on the “rootfs” device. Once the “rootfs” device has been mounted by the kernel, the kernel can load any additional modules from there. Therefore, our kernel has the following compiled-in features:

• device drivers for the “rootfs”: USB mass-storage.
• File-systems: ext3.
• Misc: BSD process accounting, /proc support, inotify support, NO initrd (we do not need one as we can mount the “rootfs” device directly), NO compiled-in wireless support (only by modules, thus you cannot download a “rootfs” over-the-air by PXE, for example), NO swap support (Bifferboard I/O is too slow for swapping).
• Size: very small, only 918224 bytes.

Why would someone need such a kernel?
The size of the bootable kernel image (+the initrd ramdisk, if any) on a Bifferboard single-chip-computer is limited to:

• 974848 bytes with Biffboot v2.0
• 983040 bytes with BiffBoot v1.X

Furthermore, some patches and special configuration is required for the RDC chip which is the heart of the system. The creator of Bifferboard has done this for us already – he developed the patch and created a minimal config for the 2.6.30.5 Linux kernel.

In order to merge the Bifferboard minimal kernel config with a config where all modules are enabled, I do the following:

• Make a kernel config with all possible modules enabled by executing “make allmodconfig“. The problem with this config is that it has every possible option selected as “Yes”, not only the modules. Therefore, I substitute every “Yes” (which is not a module) to “No” by executing “perl -pi -e ‘s/=y/=n/g’ .config“. This way I have only config entries which say “CONFIG_SOME_OPTION=m”.
• Download the other minimal kernel config which I want to merge with priority over the “all modules config”. I make a “grep =y .config-biff > .config-biff-yes“. This way I leave only the “Yes” selected kernel config options, nothing more.
• Finally, I can merge the config files into one by concatenating them. The file which is concatenated last has the most priority. This is how Kconfig merges the config lines and resolves conflicts or redefinitions of the same kernel option.
• There is however a problem with this automatic way of generating and merging an all-modules kernel config – there are sections in the kernel config which add no additional code to the kernel (thus add no space either) but they “hide” their child sub-sections. One has to go through the kernel menu manually and select with “Yes” every menu option which has a sub-menu associated with it. You can easily recognize such menu options by the “—>” ending after their menu title. I’ve created a third config which is also being merged as last which selects all such options as “Yes” (multiple CONFIG_SUBMENU_EXAMPLE=y).
• If you want to overwrite anything at the very end, you can create a fourth config file and merge it as very last.

Here is a Bash script which does what I’ve currently described: http://www.famzah.net/download/bifferboard/obsolete/build-biff-kernel-2.6.30.5-deb.sh.

Note that when you have no initrd and boot from a USB mass-storage device, you have to add “rootdelay=30” (or less) to your kernel command line. It takes some time for the USB mass-storage devices to get initialized. If there is no “rootdelay” option specified, the kernel tries to mount the “rootfs” device immediately which ends up in Kernel panic – not syncing: VFS: Unable to mount root fs. This very useful article describing the initial RAM disk (initd) in detail helped me to find out why the original Ubuntu kernel+initrd gave no kernel panic and was able to mount the root file-system from my USB stick, but at the very same time my custom kernel couldn’t do it. I did some initrd debugging and found out that it simulates the kernel command line option “rootdelay” – it polls if the “rootfs” device has been detected, every 0.1 seconds.

UPDATE: The option “rootwait” is what I was actually looking for. It is similar to “rootdelay=NN”, only that it waits forever for a root device and continues with the boot immediately after the root device is found, thus the kernel wastes no time in just waiting for “NN” seconds to elapse.

You can read my next article which gives detailed instructions on how to build a kernel suitable for Bifferboard and package it as .deb files.