usb 1-1.4: rejected 1 configuration due to insufficient available bus power

The “1-1.4″ ID may be different for your configuration.

If, and only if, you are absolutely sure that your USB hub and/or hardware configuration have a safe way to actually supply enough power, you can override this barrier and force the device to be activated despite of the error message. A possible situation is where you manually applied 5V external power on your USB device and/or USB hub, like I did on my Bifferboard.

Here is how you can override the power safety mechanism:

echo 1 > /sys/bus/usb/devices/1-1.4/bConfigurationValue

Replace “1-1.4″ with your USB device ID. Be careful and have fun!

Resources:

• Error insufficient available bus power RT2573.

I’m using an external hard disk via USB which is partitioned in two parts – /dev/sda1 (1GB) and the rest in /dev/sda2. Once you have installed Debian on Bifferboard, here are the commands which further transform your Bifferboard into a secure NAS:

apt-get update
apt-get -y install nfs-kernel-server

vi /etc/default/nfs-common
  # update: STATDOPTS='--port 2231'
vi /etc/default/nfs-kernel-server
  # update: RPCMOUNTDOPTS='-p 2233'

mkdir -m 700 /root/.ssh
  # add your public key for "root" in /root/.ssh/authorized_keys

echo '/mnt/storage 127.0.0.1(rw,no_root_squash,no_subtree_check,insecure)' >> /etc/exports
mkdir /mnt/storage
chattr +i /mnt/storage # so that we don't accidentally write there without a mounted volume

cat > /etc/rc.local <<EOF
#!/bin/bash

# allow only SSH access via the network
/sbin/iptables -P FORWARD DROP
/sbin/iptables -P INPUT DROP
/sbin/iptables -A INPUT -i lo -j ACCEPT
/sbin/iptables -A INPUT -p tcp --dport 22 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT # TCP initiated by server
/sbin/iptables -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT # DNS traffic

# mount the storage volume here, so that any errors with it don't interfere with the system startup
/bin/mount /dev/sda2 /mnt/storage
/etc/init.d/nfs-kernel-server restart
EOF

# allow only public key authentication
fgrep -i -v PasswordAuthentication /etc/ssh/sshd_config > /tmp/sshd_config && \
  mv -f /tmp/sshd_config /etc/ssh/sshd_config && \
  echo 'PasswordAuthentication no' >> /etc/ssh/sshd_config

reboot

There are two things you should consider with this setup:

1. You must trust the “root” user who mounts the directory! They have full shell access to your NAS.
2. A not-so-strong SSH encryption cipher is used, in order to improve the performance of the SSH transfer.

On the machine which is being backed up, I use the following script which mounts the NAS volume, starts the rsnapshot backup process and finally unmounts the NAS volume:

#!/bin/bash
set -u

HOST='192.168.100.102'
REMOTEPORT='22'
REMOTEDIR='/mnt/storage'
LOCALDIR='/mnt/storage'
SSHKEY='/home/famzah/.ssh/id_rsa-home-backups'

echo "Mounting NFS volume on $HOST:$REMOTEPORT (SSH-key='$SSHKEY')."
N=0
for port in 2049 2233 ; do
	N=$(($N + 1))
	LPORT=$((61000 + $N))
	ssh -f -i "$SSHKEY" -c arcfour128 -L 127.0.0.1:"$LPORT":127.0.0.1:"$port" -p "$REMOTEPORT" "root@$HOST" sleep 600d
	echo "Forwarding: $HOST: Local port: $LPORT -> Remote port: $port"
done
sudo mount -t nfs -o nfsvers=2,proto=tcp,rsize=8192,wsize=8192,intr,rw,bg,port=61001,mountport=61002 "127.0.0.1:$REMOTEDIR" "$LOCALDIR"

echo "Doing backup."
time sudo /usr/bin/rsnapshot weekly

echo "Unmounting NFS volume and closing SSH tunnels."
sudo umount "$LOCALDIR"
for pid in $(ps axuww|grep ssh|grep 6100|grep arcfour|grep -v grep|awk '{print $2}') ; do
	kill "$pid" # possibly dangerous...
done

Resources:

• Tunneling NFS over SSH « HowtoForge.

Here is how we performed the benchmarks, in order to answer the above question:

• 41 MB test file with random data, which cannot be compressed – GZip makes it only 1% smaller.
• A slow enough system – Bifferboard. Bifferboard CPU power is similar to a Pentium @ 100Mhz.
• The other system is using a dual-core Core2 Duo @ 2.26GHz, so we consider it fast enough, in order not to influence the results.
• SCP file transfer over SSH using OpenSSH as server and client.

As stated at the Ubuntu man page of ssh_config, the OpenSSH client is using the following Ciphers (most preferred go first):

aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,
aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,
aes256-cbc,arcfour

In order to examine their performance, we will transfer the test file twice using each of the ciphers and note the transfer speed and delta. Here are the shell commands that we used:

for cipher in aes128-ctr aes192-ctr aes256-ctr arcfour256 arcfour128 aes128-cbc 3des-cbc blowfish-cbc cast128-cbc aes192-cbc aes256-cbc arcfour ; do
        echo "$cipher"
        for try in 1 2 ; do
                scp -c "$cipher" test-file root@192.168.100.102:
        done
done

You can review the raw results in the “ssh-cipher-speed-results.txt” file. The delta difference between the one and same benchmark test is within 16%-20%. Not perfect, but still enough for our tests.

Here is a chart which visualizes the results:

The clear winner is Arcfour, while the slowest are 3DES and AES. Still the question if all OpenSSH ciphers are strong enough to protect your data remains.

It’s worth mentioning that the results may be architecture dependent, so test for your platform accordingly.

Resources:

• Arch Linux forums / Fastest ssh cipher

• biffavrprog (slow and you probably need to manually add your AVR chip’s definitions to its database).
• avrdude 5.10 via sysfs patched by Radoslav Kolev (very fast and supports everything by avrdude, not tested by me yet) – look at the Bifferboard AVR discussion article for more info.
• avrdude 5.10 via sysfs patched by Biff (even faster but there is a caveat with “rdc321x_gpio”, not tested by me yet) – look at the Bifferboard AVR discussion article for more info.

…read on if you are interested in the story of the above solutions and how they came to the world.

The open-source community is very strange by its nature. Motivating people to work for free is not a straightforward task, as you’ll see from the following story.

I needed an AVR programmer as I’m switching from Microchip to Atmel AVR microcontrollers, because their toolchain is free and open-source. And they also match the price and features of Microchip in general, at least for my needs. I didn’t want to buy any hardware programmers, as I already had the Bifferboard which supports GPIO.

So I decided to code an AVR programmer in Perl myself, for fun and education, and because the C code of avrdude seems too unclean and lacks documentation. That’s how biffavrprog was “born” and I offered it to the Bifferboard mailing list. You can review the thread about it.

You’ll notice that shortly after that, like about an hour later, I was criticized about why I re-implemented it all in Perl from scratch. Many valid arguments were mentioned, I explained my reasons too, but the most important lesson here is that… After a day, Radoslav Kolev developed a patch for avrdude, later on Biff made it faster, and in the end I managed to get the community involved in something of which they were only thinking doing “some day”.

An interesting way to inspire the open-source community…

Required Software
You can install the required software by the following command, no need to compile anything:

apt-get install i2c-tools

Kernel modules
The instructions are fairly the same here, except that there is no module “i2c-core” and you don’t need to load it:

modprobe rdc321x_gpio
modprobe i2c-algo-bit
modprobe i2c-gpio


You have two options on where to connect the I2C pins (SDA and SCL):

• The difficult one – connect them to the JTAG pins by disabling the JTAG first. You will need to solder on the Bifferboard which may void your warranty.
• The easy one – connect them to the Serial console pins. There is no soldering involved here but the trade-off is that you cannot use these pins for your Serial RS-232 console which you may need for debugging or for other purposes. But you could always attach another serial console via USB by using an FT232R chip, for example. This option is my personal favorite here.

Both options work fine, I’ve tried them myself. Here are the corresponding commands:

# using the JTAG pins #11 and #13, soldering required to enable them
modprobe i2c-gpio-custom bus0=0,11,13

# using the Serial console pins #7 and #8, no soldering involved here
modprobe i2c-gpio-custom bus0=0,8,7


Finally, you need to load one more additional module and you are done:

modprobe i2c-dev

Application software
The original Slackware article gives an example on how to query your I2C temperature sensor.

Boot time
Total boot time: 1 minute 11 seconds (standard Debian Lenny base installation)

The boot process goes like this:

• Initial boot. Mounted root device (5 seconds wasted on waiting for the USB mass storage to be initialized). Executing INIT. [+21 seconds elapsed]
• Waiting for udev to be initialized (most of the time spent here), configuring some misc settings, no dhcp network, entering Runlevel 2. [+41 seconds elapsed]
• Started “rsyslogd”, “sshd”, “crond”. Got prompt on the serial console. [+9 seconds elapsed]

Therefore, if a very limited and custom Linux installation is used, the total boot time could be reduced almost twice.

CPU speed
Calculated BogoMips: 56.32
According to a quite complete BogoMips list table, this is an equivalent of Pentium@133MHz or 486DX4@100MHz.

According to another CPU benchmarks comparison table for Linux, Bifferboard falls into the category of Pentium@100Mhz.

Memory speed
A “dd” write to “/dev/shm” performs with a speed of 6.3 MB/s.
The MBW memory bandwidth benchmark shows the following results:

bifferboard:/tmp# wget

http://de.archive.ubuntu.com/ubuntu/pool/universe/m/mbw/mbw_1.1.1-1_i386.deb

bifferboard:/tmp# dpkg -i mbw_1.1.1-1_i386.deb
bifferboard:/tmp# mbw 4 -n 20|egrep ^AVG
AVG Method: MEMCPY Elapsed: 0.15602 MiB: 4.00000 Copy:
25.637 MiB/s
AVG Method: DUMB Elapsed: 0.06611 MiB: 4.00000 Copy:
60.502 MiB/s
AVG Method: MCBLOCK Elapsed: 0.06619 MiB: 4.00000 Copy:
60.431 MiB/s

For comparison, my Pentium Dual-Core @ 2.50GHz with DDR2 @ 800 MHz (1.2 ns) shows a “dd” copy speed to “/dev/shm” of 271 MB/s, while the MBW test shows a maximum average speed of 7670.954 MiB/s. Bifferboard is an embedded device after all…

Available memory for applications
My base Debian installation with “udevd”, “dhclient3″, “rsyslogd”, “sshd”, “getty” and one tty session running shows 24908 kbytes free memory. You surely cannot put CNN.com on this little machine, but compared to the PIC16F877A which has 368 bytes (yes, bytes) total RAM memory, Bifferboard is a monster.

Disk system
All tests are done on an Ext3 file-system and a very fast USB Flash 8GB A-Data Xupreme 200X.
A “dd” copy to a file completes with a write speed of 6.1 MB/s.
The Bonnie++ benchmark test shows the following results:

Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
bifferboard.lo 300M   822  96  5524  61  4305  42   855  99 16576  67 143.2  12
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP /sec %CP
                 16  1274  94 14830 100  1965  85  1235  90 22519 100 2015  87

bifferboard.localdomain,300M,822,96,5524,61,4305,42,855,99,16576,67,143.2,12,16,1274,94,14830,100,1965,85,1235,90,22519,100,2015,87

Therefore, the sequential write speed is about 5.5 MB/s, while the sequential read speed is about 16.5 MB/s.

It’s worth mentioning that while the write tests were running, there was a very high CPU System load (not CPU I/O waiting) which indicates that the write throughput of Bifferboard may be a bit better if the file-system is not a journaling one. However, the tests for “Memory speed” above show that writing to “/dev/shm” (a memory-based file-system) completes with a rate of 6.3 MB/s. Therefore, this is probably the limit with this configuration.

Network
Both Netperf and Wget show a throughput of 6.5 MB/s.
The packets-per-second tests complete at a rate of 8000 packets/second.
Modern systems can handle several hundred thousand packets-per-second without an issue. However, the measured network performance of Bifferboard is more than enough for trivial network communication with the device. During the network benchmark tests, there was very high CPU System usage, but that was expected.

Encryption and SSH transfers
The maximum encryption rate for an eCryptfs mount with AES cipher and 16 bytes key length is 536 KB/s. The standard SSH Protocol 2 transfer rate using the OpenSSH server is about the same – 587.1 KB/s. If you try to transfer a file over SSH and store it on an eCryptfs mounted volume, the transfer rate is 272.2 KB/s, which is logical, as the processing power is split between the SSH transfer and the eCryptfs encryption.
You can try to tweak your OpenSSH ciphers, in order to get much better performance. The OpenSSH ciphers performance benchmark page will give you a starting point.

Conclusion
Bifferboard performs pretty well for its price. It’s my personal choice over the 8-bit 16F877A and the other 16-bit Microchip / ARM microcontrollers, when a project does not require very fast I/O.

1. Kernel installation on the Bifferboard.
2. Rootfs installation on a USB device or an SD/MMC card.

Kernel installation on the Bifferboard

Download a pre-built kernel binary image:

• vmlinuz-2.6.30.5-bifferboard-ipipe (909K)
• vmlinuz-2.6.32-bifferboard (906K) – EXPERIMENTAL

The kernel is compiled with (almost) all possible modules, so your Bifferboard should be able to easily use any device supported on Debian. Once you have downloaded the kernel image, you can then upload it to the Bifferboard, as advised at the Biffboot Wiki page. You have two options to upload the kernel – via the serial port or over the ethernet. Both work well.

Example: Assuming that you have the Bifferboard SVN repository checked out in “~/biffer/svn“, you have downloaded the “vmlinuz-2.6.30.5-bifferboard-ipipe” kernel image in “/tmp“, your Bifferboard has a MAC address of “00:B3:F6:00:37:A9“, and you have connected it on the Ethernet port “eth0” of your computer, here are the commands that you would need to use:

cd ~/biffer/svn/utils
sudo ./bb_eth_upload.py eth0 00:B3:F6:00:37:A9 /tmp/vmlinuz-2.6.30.5-bifferboard-ipipe

Rootfs installation on a USB device or an SD/MMC card

Once you have the kernel “installed” on the Bifferboard and ready to boot, you need to prepare a rootfs media. This is where your Debian installation is stored and booted from. Download one of the following pre-built rootfs images (default root password is “biffroot”):

• Debian “lenny” for Bifferboard – minimal (download size 95M, uncompressed size 346MB).
• Debian “lenny” for Bifferboard – developer (download size 129M, uncompressed size 468MB).

The “developer” version adds the following packages: build-essential, perl, links, manpages, manpages-dev, man-db, mc, vim. Note that for each image you will need at least 100MB more free on the rootfs media.

In order to populate the rootfs media, you have to do the following:

1. Create one primary partition, format it as “ext3″ and then mount the USB device or SD/MMC card.
2. Extract the archive in the mounted directory.
3. Unmount the directory.

Example: Assuming that you have the Bifferboard SVN repository checked out in “~/biffer/svn“, you have downloaded the “minimal” rootfs image in “/tmp“, and you are using an SD/MMC card under the device name “/dev/mmcblk0“, here are the commands that you would need to use:

sudo bash
mkdir /mnt/rootfs
cd ~/biffer/svn/debian/rootfs
./format-and-mount.sh /dev/mmcblk0 /mnt/rootfs
tar -jxf /tmp/debian-lenny-bifferboard-rootfs-minimal.tar.bz2 -C /mnt/rootfs
umount /mnt/rootfs
# CHANGE THE DEFAULT ROOT PASSWORD!

When you have the USB device or SD/MMC card ready and populated with the customized Debian rootfs, plug it in Bifferboard, attach a serial cable to Bifferboard, if you have one, and boot it up.

That’s it. Enjoy your Bifferboard running Debian.

Update: As already mentioned in the comments below, you would probably need to set up swap too. Here is my recipe:

# change "128" (MBytes) below to a number which suits your needs
dd if=/dev/zero of=/swapfile bs=1M count=128
mkswap /swapfile
swapon /swapfile # enables swap right away; disable with "swapoff -a"
echo '/swapfile none swap sw 0 0' >> /etc/fstab # enables swap at system boot

Using a file for swap on a 2.6 Linux kernel has the same performances as using a separate swap partition as discussed at LKML.

P.S. If you want to build your own customized Debian rootfs image for Bifferboard – checkout the Bifferboard SVN repository and review the instructions in “debian/rootfs/images.txt“.

References:

• dpkg: extract specific file.
• Other similar Debian projects for Bifferboard – Bifferboard Wiki “Home‎ > ‎’Desktop’ Linux Distributions‎ > ‎Debian” page.
• Pinouts for the standard single USB Bifferboard.
• Bifferboard with 2 USB ports – hardware specification and pinouts.

1. You need to checkout the whole Bifferboard SVN repository.
2. The scripts are located in the directory “/debian/rootfs“. Execute them from the checked out repository on your local computer.

First you have to mount a medium on which we are going to install the Debian system. Generally, you have two options:

• Using a USB Flash drive:
  

  
## MAKE SURE THAT YOU UPDATE THIS
$ export ROOTDEV=/dev/sdc1
$ sudo mkfs.ext3 $ROOTDEV
$ sudo tune2fs -c 0 -i 0 $ROOTDEV
$ export MNTPOINT=/mnt/diskimage
$ sudo mount $ROOTDEV $MNTPOINT


• Using a Qemu image:
  

  
$ export MNTPOINT=/mnt/diskimage
$ export IMGFILE=hd0.img
$ sudo mount -o loop,offset=32256 "$IMGFILE" $MNTPOINT


Once we have the medium mounted at $MNTPOINT, we can proceed with installing Debian there and configuring it for Bifferboard:

$ export DBS_OS_VERSION=lenny
## replace "bg." with your local archive, or just omit it
$ export DBS_LOCAL_ARCHIVE=bg.
$ sudo debootstrap --arch i386 ${DBS_OS_VERSION} $MNTPOINT/ http://ftp.${DBS_LOCAL_ARCHIVE}debian.org/debian
## ... go grab a pizza or something ... this will take a while
$ sudo cp /etc/resolv.conf $MNTPOINT/etc/
$ sudo mount proc $MNTPOINT/proc -t proc
$ sudo chroot $MNTPOINT
##
## We are now in the "chroot" environment as root
##
/# apt-get -qq update && apt-get install wget
/# cd /root && wget http://bifferboard.svn.sourceforge.net/viewvc/bifferboard/debian/rootfs/include/debootstrap-postconfig.sh
/root# chmod +x debootstrap-postconfig.sh && ./debootstrap-postconfig.sh
/root# passwd root
/root# exit
##
## Back to our machine
##
$ sudo umount $MNTPOINT/proc
$ sudo umount $MNTPOINT


Now you have a minimum Debian installation customized for Bifferboard in the following way:

• Custom kernel for Bifferboard installed by a .deb package.
• Ethernet interface configured as DHCP client.
• Temporary directories /tmp and /var/tmp mounted on a RAM-disk.
• All APT sources “main contrib non-free” enabled.
• Serial console on ttyS0 (115200 8N1).
• RTC (real-time clock) kernel modules blacklisted – the Bifferboard has no RTC.
• IPv6 disabled – takes a lot of resources and we won’t use it anyway, for now.

I may add any further customizations if needed. You can always review the debootstrap-postconfig.sh script for details on what is being configured.

You can use this image/disk as a rootfs which you can boot directly on Bifferboard or try in Qemu. Note that you have to install our Debian kernel on Bifferboard prior to booting this rootfs.

Used resources:

• Debian Administration: Installing new Debian systems with debootstrap
• Installing Ubuntu from a Unix/Linux System

You can download the already built .deb packages for Debian “lenny” at the following addresses:

• 2.6.30.5:
  • Linux kernel 2.6.30.5 binary image (18M)
  • Header files for the 2.6.30.5 Linux kernel (5.6M)
• 2.6.32 – EXPERIMENTAL:
  • Linux kernel 2.6.32 binary image (19M)
  • Header files for the 2.6.32 Linux kernel (5.9M)

On my Bifferboard, I use the following Kernel command line to boot this kernel:

rootwait root=/dev/sda1 console=uart,io,0x3f8

For Qemu, because of some USB mass-storage emulation issues, the line looks like:

rootwait root=/dev/sda1 console=uart,io,0x3f8 irqpoll

Update: There are (more up-to-date) automated scripts which you can use for the below actions:

• You need to checkout the whole Bifferboard SVN repository.
• The scripts are located in the directory “/debian/kernel“. Execute the “build.sh” script from the checked out repository on your local computer, on a Debian “lenny” system.

If you want to build the packages yourself, you need to execute the following commands on a Debian “lenny” machine (a virtual machine or a chroot()’ed installation work too):

famzah@FURNA:~$ sudo apt-get install kernel-package fakeroot build-essential ncurses-dev tar patch
famzah@FURNA:~$ export KVERSION=2.6.30.5
famzah@FURNA:~$ rm -rf /tmp/tmpkern-$KVERSION
famzah@FURNA:~$ mkdir /tmp/tmpkern-$KVERSION
famzah@FURNA:~$ cd /tmp/tmpkern-$KVERSION && wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-$KVERSION.tar.bz2
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ tar -xjf linux-$KVERSION.tar.bz2
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ sudo mkdir -p /usr/src/bifferboard && sudo chown $USER /usr/src/bifferboard
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ mv linux-$KVERSION /usr/src/bifferboard/
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ cd /usr/src/bifferboard/linux-$KVERSION
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ wget 'http://www.famzah.net/download/bifferboard/obsolete/bifferboard-2.6.30.5-12.patch' -O bifferboard-2.6.30.5-12.patch
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ patch --quiet -p1 < bifferboard-2.6.30.5-12.patch
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ wget http://www.famzah.net/download/bifferboard/obsolete/build-biff-kernel-2.6.30.5-deb.sh
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ chmod +x build-biff-kernel-2.6.30.5-deb.sh
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ ./build-biff-kernel-2.6.30.5-deb.sh
# When "make menuconfig" is displayed, just EXIT and SAVE the configuration.
#
# After the build, you can find the two .deb packages in "/usr/src/bifferboard".

Used resources:

• Bifferboard Kernel Patch for 2.6.30.5 (old, mirror copy)
• Ubuntu Wiki guide for building a custom kernel from the upstream mainline Linux kernel

The idea is to build a very small kernel with the bare minimum compiled-in and all the rest as modules which are stored on the “rootfs” device. Once the “rootfs” device has been mounted by the kernel, the kernel can load any additional modules from there. Therefore, our kernel has the following compiled-in features:

• device drivers for the “rootfs”: USB mass-storage.
• File-systems: ext3.
• Misc: BSD process accounting, /proc support, inotify support, NO initrd (we do not need one as we can mount the “rootfs” device directly), NO compiled-in wireless support (only by modules, thus you cannot download a “rootfs” over-the-air by PXE, for example), NO swap support (Bifferboard I/O is too slow for swapping).
• Size: very small, only 918224 bytes.

Why would someone need such a kernel?
The size of the bootable kernel image (+the initrd ramdisk, if any) on a Bifferboard single-chip-computer is limited to:

• 974848 bytes with Biffboot v2.0
• 983040 bytes with BiffBoot v1.X

Furthermore, some patches and special configuration is required for the RDC chip which is the heart of the system. The creator of Bifferboard has done this for us already – he developed the patch and created a minimal config for the 2.6.30.5 Linux kernel.

In order to merge the Bifferboard minimal kernel config with a config where all modules are enabled, I do the following:

• Make a kernel config with all possible modules enabled by executing “make allmodconfig“. The problem with this config is that it has every possible option selected as “Yes”, not only the modules. Therefore, I substitute every “Yes” (which is not a module) to “No” by executing “perl -pi -e ‘s/=y/=n/g’ .config“. This way I have only config entries which say “CONFIG_SOME_OPTION=m”.
• Download the other minimal kernel config which I want to merge with priority over the “all modules config”. I make a “grep =y .config-biff > .config-biff-yes“. This way I leave only the “Yes” selected kernel config options, nothing more.
• Finally, I can merge the config files into one by concatenating them. The file which is concatenated last has the most priority. This is how Kconfig merges the config lines and resolves conflicts or redefinitions of the same kernel option.
• There is however a problem with this automatic way of generating and merging an all-modules kernel config – there are sections in the kernel config which add no additional code to the kernel (thus add no space either) but they “hide” their child sub-sections. One has to go through the kernel menu manually and select with “Yes” every menu option which has a sub-menu associated with it. You can easily recognize such menu options by the “—>” ending after their menu title. I’ve created a third config which is also being merged as last which selects all such options as “Yes” (multiple CONFIG_SUBMENU_EXAMPLE=y).
• If you want to overwrite anything at the very end, you can create a fourth config file and merge it as very last.

Here is a Bash script which does what I’ve currently described: http://www.famzah.net/download/bifferboard/obsolete/build-biff-kernel-2.6.30.5-deb.sh.

Note that when you have no initrd and boot from a USB mass-storage device, you have to add “rootdelay=30″ (or less) to your kernel command line. It takes some time for the USB mass-storage devices to get initialized. If there is no “rootdelay” option specified, the kernel tries to mount the “rootfs” device immediately which ends up in Kernel panic – not syncing: VFS: Unable to mount root fs. This very useful article describing the initial RAM disk (initd) in detail helped me to find out why the original Ubuntu kernel+initrd gave no kernel panic and was able to mount the root file-system from my USB stick, but at the very same time my custom kernel couldn’t do it. I did some initrd debugging and found out that it simulates the kernel command line option “rootdelay” – it polls if the “rootfs” device has been detected, every 0.1 seconds.

UPDATE: The option “rootwait” is what I was actually looking for. It is similar to “rootdelay=NN”, only that it waits forever for a root device and continues with the boot immediately after the root device is found, thus the kernel wastes no time in just waiting for “NN” seconds to elapse.

You can read my next article which gives detailed instructions on how to build a kernel suitable for Bifferboard and package it as .deb files.