/contrib/famzah

Enthusiasm never stops

Tag Archives: debian


by 2 Comments

Find the repository of all installed packages on Debian or Ubuntu

It turns out that there is no standard “apt” command which lists where a package was installed from. You may need this information if you have added additional APT repositories to your Debian/Ubuntu installation. I see a lot of questions at the forums (1, 2, 3, 4) and the proper solution tends to be “parse apt-cache output yourself”. Here is my solution which is very similar to this one:

#!/bin/bash
set -u

errors=0

for PKGNAME in $(dpkg -l|grep ^i|awk '{print $2}'); do
        INFO="$(apt-cache policy "$PKGNAME")"
        IVER="$(echo "$INFO" | grep Installed: | awk '{print $2}')"
        IPRIO="$(echo "$INFO" | fgrep "*** $IVER" | awk '{print $3}')"
        REPO="$(echo "$INFO" | fgrep -A1 "*** $IVER" | tail -n+2 | head -n1 | awk '{print $2 " " $3}')"

        echo "$PKGNAME repo=$REPO"

        if [ "$REPO" == '' ]; then
                errors=$(( $errors + 1 ))
                echo "ERROR: Unable to find the repo for package \"$PKGNAME\"" >&2
        fi
done

if [ "$errors" -ne 0 ]; then
        echo "ERROR: $errors errors encountered" >&2
        exit 1
else
        exit 0
fi


by Leave a comment

The “cp” command may corrupt your files on Debian Wheezy

We recently had two files corrupted on Debian Wheezy (the current “stable” release). The first one had some garbage, instead of the real data, the other had only zero characters. Only a small part of the files of about 3K was corrupted. This affects both “ext3” and “ext4” file-systems.

It turns out to be a free-memory read bug in cp from coreutils-[8.11..8.19] reported to GNU in Oct/2012. Almost a year ago it was also reported to Debian in Apr/2014 with severity “grave“.

Today we test if the bug is fixed using the PoC given in the original GNU bug report:

$ perl -e 'for (1..3333) { sysseek (*STDOUT, 4096, 1)' -e '&& syswrite (*STDOUT, "a" x 1024) or die "$!"}' > j

$ valgrind cp j j2

==13175== Memcheck, a memory error detector
==13175== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==13175== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==13175== Command: cp j j2
==13175== 
==13175== Invalid read of size 4
==13175==    at 0x8051229: ??? (in /bin/cp)
==13175==    by 0x153FFF: ???
==13175==  Address 0x424ed0c is 1,356 bytes inside a block of size 1,440 free'd
==13175==    at 0x40283EE: realloc (vg_replace_malloc.c:632)
==13175==    by 0x805820B: ??? (in /bin/cp)
==13175==    by 0x153FFF: ???

...

==15843== ERROR SUMMARY: 15 errors from 9 contexts (suppressed: 25 from 6)

It turns out that the bug is not fixed in Debian. Unfortunately, upgrade of the “coreutils” package from Jessie is not an option, where this bug is not present. The “coreutils” package from Jessie depends on a newer “libc6” and futhermore would introduce too many (untested) changes to the core utils.

Here is how to rebuild the “coreutils” package by applying the “cp” data corruption patch:

root@machine1:~# cowbuilder --login

COW-machine1:~# apt-get update
COW-machine1:~# apt-get upgrade

COW-machine1:~# mkdir /root/coreutils
COW-machine1:~# cd /root/coreutils

COW-machine1:~/coreutils# apt-get source coreutils
COW-machine1:~/coreutils# apt-get build-dep coreutils

COW-machine1:~/coreutils# cd coreutils-8.13
COW-machine1:~/coreutils/coreutils-8.13# wget 'http://git.savannah.gnu.org/cgit/coreutils.git/patch/?id=64aef5fb9afecc023a6e719da161dbbf450908b8' -O cp-avoid_data_corrupting_free_memory_read.patch

COW-machine1:~/coreutils/coreutils-8.13# patch -p1 < cp-avoid_data_corrupting_free_memory_read.patch
COW-machine1:~/coreutils/coreutils-8.13# DEBFULLNAME='Admin Team' DEBEMAIL='box@example.com' dch --local '~patched' 'Local build with cp data corruption patch'
COW-machine1:~/coreutils/coreutils-8.13# dpkg-buildpackage -b -rfakeroot

root@machine1:~# cp /var/cache/pbuilder/build/cow.1385/root/coreutils/coreutils_8.13-3.5~patched1_i386.deb /root/tmp/

Finally, you need to install the “.deb” file on your system and prevent APT from auto-upgrading it. You’d need to recompile it every time Debian “stable” releases a mainstream update for “cureutils”. This doesn’t happen that often. Furthermore, we hope that Debian will react to the bug report and will fix the bug in their source tree for Wheezy “stable”.


by Leave a comment

Cron job custom timezone

The default cron job daemon on Debian and Ubuntu does not support per-user timezones (see crontab(5) man page).

Here is a solution which runs hourly cron tasks in the timezone which you specified. For example, if you want to run “test.sh” at 5 AM and 7 PM in timezone Europe/Sofia, you need to create the following “cron.hourly” script:

#!/bin/bash
/path/to/run-at.sh 5,19 Europe/Sofia $(( 15*60 )) /var/run/run-at-test.state /path/to/test.sh

Here is the source code of “run-at.sh”:

#!/bin/bash
set -u

# XXX
# We work exclusively with global variables.
# Functions are used just to separate logic and for self-documenting.

function display_usage() {
	if [ "$1" -ge 5 ]; then
		return; # enough parameters
	fi

	cat >&2 <<EOF
Usage: $0 HOURS TZ WARN_TIME STATE_FILE COMMAND [ARGS...]
Runs COMMAND every day at the specified HOURS.

The execution is accounted and considered successful
only if COMMAND exits with 0. If there was no (successful)
execution within the WARN_TIME hours specified period,
a warning is being issued.

The STATE_FILE must pre-exist, so make sure that you
create it before the first run, or you will get a
warning.

This script is intended to be run in "cron.hourly".

Arguments:
 - HOURS -- comma-separated list; example: 5,12,19
 - TZ -- time zone; example: Europe/Sofia
 - WARN_TIME -- minutes; example: 360
 - STATE_FILE -- fill path to a writable file
 - COMMAND and ARGS to be executed on HOURS

Example:
 $0 5,19 Europe/Sofia \\
   \$(( 15*60 )) /root/.run-at-test.state \\
   date -d now +%c
EOF
	exit 1
}

function check_state_file_oldness() {
	# find file with mtime less than $WARN_TIME minutes
	if [ "$(find "$STATE_FILE" -type f -mmin -"$WARN_TIME")" != "$STATE_FILE" ]; then
		# file not found -> this will be indicated by 'find' on STDERR too
		# file is too old ("-mmin" condition not met)
		echo "WARNING: No successful run in the last $WARN_TIME minutes" >&2
	fi
}

function split_hours() {
	HOURS="${HOURS//,/ }" # replace "," with " "
	# https://blog.famzah.net/2013/02/17/
	#   bash-split-a-string-into-columns-by-white-space-without-invoking-sub-shells/
	HOURS=( $HOURS ) # now an ARRAY
}

function validate_tz() {
	# naive check; tested on Debian
	if [ ! -e "/usr/share/zoneinfo/$WANT_TZ" ]; then
		echo "ERROR: TZ seems to be invalid." >&2
		exit 1
	fi
}

function get_now_hour_in_tz() {
	NOW_HOUR="$(TZ="$WANT_TZ" date +%H)" # get current hour in 24h-format using the $WANT_TZ
}

function check_if_we_should_run_or_exit() {
	RUN=0
	for h in "${HOURS[@]}" ; do
		if [ "$h" -eq "$NOW_HOUR" ]; then
			RUN=1
			break
		fi
	done

	if [ "$RUN" -eq 0 ]; then
		exit 0
	fi
}

function execute_command_and_get_exit_code() {
	"$@" # execute the command
	EC="$?"
}

function update_state_file_mtime() {
	if [ "$EC" -eq 0 ]; then
		touch "$STATE_FILE"
	fi
}

#### ### ### ###

display_usage "$#"

# parse_argv

HOURS="$1" ; shift
WANT_TZ="$1" ; shift
WARN_TIME="$1" ; shift
STATE_FILE="$1" ; shift
# the rest in "$@" is the command to be executed

check_state_file_oldness

split_hours
validate_tz
get_now_hour_in_tz
check_if_we_should_run_or_exit

execute_command_and_get_exit_code "$@"
update_state_file_mtime


by Leave a comment

Re-compile a Debian kernel as a .deb package

Here is my success story on how to re-compile a Debian/Ubuntu kernel, in order to enable or tune kernel features which are not available as kernel modules:

# Install required software for the kernel compilation
apt-get install fakeroot build-essential devscripts
apt-get build-dep linux-image-$(uname -r) # make sure you have the appropriate "deb-src" in "sources.list"
apt-get install libncurses5-dev # required for "make menuconfig"
apt-get install ccache # to re-compile the kernel faster (http://wiki.debian.org/OverridingDSDT)

# Prepare some environent variables for our architecture, for later use
ARCH=$(uname -r|cut -d- -f3)
CPUCNT=$(( $(cat /proc/cpuinfo |egrep ^processor |wc -l) * 2))

# Get the kernel sources
rm -rf /root/krebuild && mkdir /root/krebuild
cd /root/krebuild
apt-get source linux-image-$(uname -r)
cd linux-$(uname -r|cut -d- -f1|cut -d. -f1-2)* # cd linux-3.2.20

# http://kernel-handbook.alioth.debian.org/ch-common-tasks.html # 4.2.5 Building packages for one flavour
# The target in this command has the general form of target_arch_featureset_flavour. Replace the featureset with none if you do not want any of the extra featuresets.

# Prepare a Debian kernel to compile
fakeroot make -f debian/rules.gen setup_${ARCH}_none_${ARCH} >/dev/null
cd debian/build/build_${ARCH}_none_${ARCH}
make menuconfig # make any kernel config changes now
cd ../../..

# No debug info => faster kernel build
perl -pi -e 's/debug-info:\s+true/debug-info: false/' debian/config/$ARCH/defines
echo binary-arch_${ARCH}_none_${ARCH}
vi debian/rules.gen # find the Make target and change DEBUG and DEBUG_INFO to False/n respectively

# Bugfix: http://lists.debian.org/debian-user/2008/02/msg01455.html
vi debian/bin/buildcheck.py +51 # add "return 0" right after "def __call__(self, out):"

# Compile the kernel
time DEBIAN_KERNEL_USE_CCACHE=true DEBIAN_KERNEL_JOBS=$CPUCNT \
	fakeroot make -j$CPUCNT -f debian/rules.gen binary-arch_${ARCH}_none_${ARCH} > compile-progress.log

# If needed, the linux-headers-version-common binary package (http://kernel-handbook.alioth.debian.org/ch-common-tasks.html -> 4.2.5)
#fakeroot make -j$CPUCNT -f debian/rules.gen binary-arch_${ARCH}_none_real

# Install the newly compiled kernel
cd ..
dpkg -i linux-image-*.deb
#dpkg -i linux-headers-*.deb # only if you need them and/or have them installed already


by Leave a comment

Secure NAS on Bifferboard running Debian

This NAS solution uses OpenSSH for secure transport over a TCP connection, and NFS to mount the volume on your local computer. The hardware of the NAS server is the low-cost Bifferboard.

I’m using an external hard disk via USB which is partitioned in two parts – /dev/sda1 (1GB) and the rest in /dev/sda2. Once you have installed Debian on Bifferboard, here are the commands which further transform your Bifferboard into a secure NAS:

apt-get update
apt-get -y install nfs-kernel-server

vi /etc/default/nfs-common 
  # update: STATDOPTS='--port 2231'
vi /etc/default/nfs-kernel-server 
  # update: RPCMOUNTDOPTS='-p 2233'

mkdir -m 700 /root/.ssh
  # add your public key for "root" in /root/.ssh/authorized_keys

echo '/mnt/storage 127.0.0.1(rw,no_root_squash,no_subtree_check,insecure,async)' >> /etc/exports
mkdir /mnt/storage
chattr +i /mnt/storage # so that we don't accidentally write there without a mounted volume

cat > /etc/rc.local <<EOF
#!/bin/bash

# allow only SSH access via the network
/sbin/iptables -P FORWARD DROP
/sbin/iptables -P INPUT DROP
/sbin/iptables -A INPUT -i lo -j ACCEPT
/sbin/iptables -A INPUT -p tcp --dport 22 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m state --state ESTABLISHED -j ACCEPT # TCP initiated by server
/sbin/iptables -A INPUT -p udp -m state --state ESTABLISHED -j ACCEPT # DNS traffic

# mount the storage volume here, so that any errors with it don't interfere with the system startup
/bin/mount /dev/sda2 /mnt/storage
/etc/init.d/nfs-kernel-server restart
EOF

# allow only public key authentication
fgrep -i -v PasswordAuthentication /etc/ssh/sshd_config > /tmp/sshd_config && \
  mv -f /tmp/sshd_config /etc/ssh/sshd_config && \
  echo 'PasswordAuthentication no' >> /etc/ssh/sshd_config

reboot

There are two things you should consider with this setup:

  1. You must trust the “root” user who mounts the directory! They have full shell access to your NAS.
  2. A not-so-strong SSH encryption cipher is used, in order to improve the performance of the SSH transfer.

On the machine which is being backed up, I use the following script which mounts the NAS volume, starts the rsnapshot backup process and finally unmounts the NAS volume:

#!/bin/bash
set -u

HOST='192.168.100.102'
SSHUSER='root'
REMOTEPORT='22'
REMOTEDIR='/mnt/storage'
LOCALDIR='/mnt/storage'
SSHKEY='/home/famzah/.ssh/id_rsa-home-backups'

echo "Mounting NFS volume on $HOST:$REMOTEPORT (SSH-key='$SSHKEY')."
N=0
for port in 2049 2233 ; do
	N=$(($N + 1))
	LPORT=$((61000 + $N))
	ssh -f -i "$SSHKEY" -c arcfour128 -L 127.0.0.1:"$LPORT":127.0.0.1:"$port" -p "$REMOTEPORT" "$SSHUSER@$HOST" sleep 600d
	echo "Forwarding: $HOST: Local port: $LPORT -> Remote port: $port"
done
sudo mount -t nfs -o noatime,nfsvers=2,proto=tcp,intr,rw,bg,port=61001,mountport=61002 "127.0.0.1:$REMOTEDIR" "$LOCALDIR"

echo "Doing backup."
time sudo /usr/bin/rsnapshot weekly

echo "Unmounting NFS volume and closing SSH tunnels."
sudo umount "$LOCALDIR"
for pid in $(ps axuww|grep ssh|grep 6100|grep arcfour|grep -v grep|awk '{print $2}') ; do
	kill "$pid" # possibly dangerous...
done

Update, 29/Sep/2010 – performance tunes:

  • Added “async” in “/etc/exports”.
  • Removed the “rsize=8192,wsize=8192” mount options – they are auto-negotiated by default.
  • Added the “noatime” mount option.
  • Put the SSH username in a variable.

Resources:


by 4 Comments

Debug Debian or Ubuntu /etc/network/interfaces

Here is a debug idea for your Debian or Ubuntu server or network station, if you do fancy stuff with your network configuration, or if you are in trouble even with a standard configuration.

Let’s first review some documentation and namely the one of ifup(8) and ifdown(8). Here is an excerpt from it:

KNOWN BUGS/LIMITATIONS
The program keeps records of whether network interfaces are up or down. Under exceptional circumstances these records can become inconsistent with the real states of the interfaces.

Moreover, if the ifup(8) command fails in the middle/end of configuring an interface, then the interface is marked as “down” in the state database but is actually configured, i.e. its actual state is not reverted to a non-configured actually “down” interface. As a result, ifdown(8) doesn’t want to bring down the interface later, even though it’s configured to some point. Furthermore, if ifdown(8) fails in the middle of the de-configuration, you are not notified properly by an error message.

Why would you care so much? If all ifup(8) and ifdown(8) procedures don’t complete well, most probably `/etc/init.d/networking restart` will not work as expected, and you also won’t be able to bring up or down certain interfaces by calling “ifup $IFACE” or “ifdown $IFACE”.

Let’s see how we can have better control and debug info. Here is a somehow complicated “/etc/network/interfaces” example which could cause you some trouble and is not that easy to debug:

# The primary network interface
auto bond0
iface bond0 inet static
        address 192.168.7.13
        netmask 255.255.255.0
        network 192.168.7.0
        broadcast 192.168.7.255
        gateway 192.168.7.8
        pre-up /sbin/ifconfig eth0 up
        pre-up /sbin/ifconfig eth1 up
        pre-up echo bond0 > /sys/module/aoe/parameters/aoe_iflist
        pre-up echo 100 > /sys/class/net/bond0/bonding/miimon
        pre-up echo 1 > /sys/class/net/bond0/bonding/mode
        post-up /sbin/ifenslave bond0 eth0 eth1
        post-up /sbin/ip link set bond0 txqueuelen 1000
        down /sbin/ifenslave -d bond0 eth0 eth1
        post-down /sbin/ifconfig eth0 down
        post-down /sbin/ifconfig eth1 down

The problem in my case was that I used “post-down” instead of “down” for the “/sbin/ifenslave -d bond0 eth0 eth1” but that wasn’t obvious for me – I spent almost an hour trying to figure out why my “ifup” and “ifdown” (and the whole `/etc/init.d/networking` script on boot and restart) weren’t working as expected.

How can you debug it?
You can add a test for successfulness after each statement and also add one very final debug message in each “post-up” and “post-down” interfaces(5) section:

# The primary network interface
auto bond0
iface bond0 inet static
        address 192.168.7.13
        netmask 255.255.255.0
        network 192.168.7.0
        broadcast 192.168.7.255
        gateway 192.168.7.8
        pre-up /sbin/ifconfig eth0 up || echo FAILED break point 1
        pre-up /sbin/ifconfig eth1 up || echo FAILED break point 2
        pre-up echo bond0 > /sys/module/aoe/parameters/aoe_iflist || echo FAILED break point 3
        pre-up echo 100 > /sys/class/net/bond0/bonding/miimon || echo FAILED break point 4
        pre-up echo 1 > /sys/class/net/bond0/bonding/mode || echo FAILED break point 5
        post-up /sbin/ifenslave bond0 eth0 eth1 || echo FAILED break point 6
        post-up /sbin/ip link set bond0 txqueuelen 1000 || echo FAILED break point 7
        post-up echo Successful UP for interface $IFACE
        down /sbin/ifenslave -d bond0 eth0 eth1 || echo FAILED break point 8
        post-down /sbin/ifconfig eth0 down || echo FAILED break point 9
        post-down /sbin/ifconfig eth1 down || echo FAILED break point 10
        post-down echo Successful DOWN for interface $IFACE

Note the very last “post-up” and “post-down” debug statements which we added, they must always be the last “post-up” and “post-down” statements:

        ...
        post-up echo Successful UP for interface $IFACE
        ...
        post-down echo Successful DOWN for interface $IFACE

If you don’t see the “Successful UP for interface $IFACE” or “Successful DOWN for interface $IFACE” for each of the configured interfaces, then something with your network start-up script went wrong (`/etc/init.d/networking`).

The step-by-step debug statements (“… || echo FAILED break point XX”) should help you determine where exactly the problem was.

Note that the “echo” debug statements here will always exit successfully which will not interrupt your network script as it would have done it if the debug “echo” was missing.


by 35 Comments

Running Debian on Bifferboard

There are three major steps in installing Debian on your Bifferboard:

  1. Kernel boot command line.
  2. Kernel installation on the Bifferboard.
  3. Rootfs installation on a USB device or an SD/MMC card.

Kernel boot command line

Since Biffboot v3.3, dated 19.July.2010, the kernel boot command line no longer specifies an external block device for the root file system. As a result of this, you need to update the boot configuration before you can boot from a USB device or an SD/MMC card. You have two options to configure the boot command line:

You need to set the kernel boot command line (“Kernel cmndline”) to:

console=uart,io,0x3f8 root=/dev/sda1 rootwait

Kernel installation on the Bifferboard

Download a pre-built kernel binary image:

The kernel is compiled with (almost) all possible modules, so your Bifferboard should be able to easily use any device supported on Debian. Once you have downloaded the kernel image, you can then upload it to the Bifferboard, as advised at the Biffboot Wiki page. You have two options to upload the kernel – via the serial port or over the ethernet. Both work well.

Example: Assuming that you have the Bifferboard SVN repository checked out in “~/biffer/svn“, you have downloaded the “vmlinuz-2.6.30.5-bifferboard-ipipe” kernel image in “/tmp“, your Bifferboard has a MAC address of “00:B3:F6:00:37:A9“, and you have connected it on the Ethernet port “eth0” of your computer, here are the commands that you would need to use:

cd ~/biffer/svn/utils
sudo ./bb_eth_upload.py eth0 00:B3:F6:00:37:A9 /tmp/vmlinuz-2.6.30.5-bifferboard-ipipe

Rootfs installation on a USB device or an SD/MMC card

Once you have the kernel “installed” on the Bifferboard and ready to boot, you need to prepare a rootfs media. This is where your Debian installation is stored and booted from. Download one of the following pre-built rootfs images (default root password is “biffroot”):

The “developer” version adds the following packages: build-essential, perl, links, manpages, manpages-dev, man-db, mc, vim. Note that for each image you will need at least 100MB more free on the rootfs media.

In order to populate the rootfs media, you have to do the following:

  1. Create one primary partition, format it as “ext3” and then mount the USB device or SD/MMC card.
  2. Extract the archive in the mounted directory.
  3. Unmount the directory.

Example: Assuming that you have the Bifferboard SVN repository checked out in “~/biffer/svn“, you have downloaded the “minimal” rootfs image in “/tmp“, and you are using an SD/MMC card under the device name “/dev/mmcblk0“, here are the commands that you would need to use:

sudo bash
mkdir /mnt/rootfs
cd ~/biffer/svn/debian/rootfs
./format-and-mount.sh /dev/mmcblk0 /mnt/rootfs
tar -jxf /tmp/debian-lenny-bifferboard-rootfs-minimal.tar.bz2 -C /mnt/rootfs
umount /mnt/rootfs
# CHANGE THE DEFAULT ROOT PASSWORD!

When you have the USB device or SD/MMC card ready and populated with the customized Debian rootfs, plug it in Bifferboard, attach a serial cable to Bifferboard, if you have one, and boot it up.

That’s it. Enjoy your Bifferboard running Debian.

Update: As already mentioned in the comments below, you would probably need to set up swap too. Here is my recipe:

# change "128" (MBytes) below to a number which suits your needs
dd if=/dev/zero of=/swapfile bs=1M count=128
mkswap /swapfile
swapon /swapfile # enables swap right away; disable with "swapoff -a"
echo '/swapfile none swap sw 0 0' >> /etc/fstab # enables swap at system boot

Using a file for swap on a 2.6 Linux kernel has the same performances as using a separate swap partition as discussed at LKML.

Update 2: As announced by Debian, Debian 5.0 (lenny) has been superseded by Debian 6.0 (squeeze). Security updates have been discontinued as of February 6th, 2012. Thus by downloading and installing the images provided here, you’re using an obsolete Debian release. If that’s not a problem for you, read on. You need to change the file “/etc/apt/sources.list” to the following using your favorite text editor:

deb http://archive.debian.org/debian lenny main contrib non-free
deb-src http://archive.debian.org/debian lenny main contrib non-free
deb http://archive.debian.org/debian-security/ lenny/updates main contrib non-free
deb-src http://archive.debian.org/debian-security/ lenny/updates main contrib non-free

P.S. If you want to build your own customized Debian rootfs image for Bifferboard – checkout the Bifferboard SVN repository and review the instructions in “debian/rootfs/images.txt“.

References:


by Leave a comment

Debian rootfs installation customized for Bifferboard

Update: There are (more up-to-date) automated scripts which you can use for the below actions:

  1. You need to checkout the whole Bifferboard SVN repository.
  2. The scripts are located in the directory “/debian/rootfs“. Execute them from the checked out repository on your local computer.

First you have to mount a medium on which we are going to install the Debian system. Generally, you have two options:

  • Using a USB Flash drive:


    ## MAKE SURE THAT YOU UPDATE THIS
    $ export ROOTDEV=/dev/sdc1
    $ sudo mkfs.ext3 $ROOTDEV
    $ sudo tune2fs -c 0 -i 0 $ROOTDEV
    $ export MNTPOINT=/mnt/diskimage
    $ sudo mount $ROOTDEV $MNTPOINT

  • Using a Qemu image:


    $ export MNTPOINT=/mnt/diskimage
    $ export IMGFILE=hd0.img
    $ sudo mount -o loop,offset=32256 "$IMGFILE" $MNTPOINT

Once we have the medium mounted at $MNTPOINT, we can proceed with installing Debian there and configuring it for Bifferboard:

$ export DBS_OS_VERSION=lenny
## replace "bg." with your local archive, or just omit it
$ export DBS_LOCAL_ARCHIVE=bg.
$ sudo debootstrap --arch i386 ${DBS_OS_VERSION} $MNTPOINT/ http://ftp.${DBS_LOCAL_ARCHIVE}debian.org/debian
## ... go grab a pizza or something ... this will take a while
$ sudo cp /etc/resolv.conf $MNTPOINT/etc/
$ sudo mount proc $MNTPOINT/proc -t proc
$ sudo chroot $MNTPOINT
##
## We are now in the "chroot" environment as root
##
/# apt-get -qq update && apt-get install wget
/# cd /root && wget http://bifferboard.svn.sourceforge.net/viewvc/bifferboard/debian/rootfs/include/debootstrap-postconfig.sh
/root# chmod +x debootstrap-postconfig.sh && ./debootstrap-postconfig.sh
/root# passwd root
/root# exit
##
## Back to our machine
##
$ sudo umount $MNTPOINT/proc
$ sudo umount $MNTPOINT

Now you have a minimum Debian installation customized for Bifferboard in the following way:

  • Custom kernel for Bifferboard installed by a .deb package.
  • Ethernet interface configured as DHCP client.
  • Temporary directories /tmp and /var/tmp mounted on a RAM-disk.
  • All APT sources “main contrib non-free” enabled.
  • Serial console on ttyS0 (115200 8N1).
  • RTC (real-time clock) kernel modules blacklisted – the Bifferboard has no RTC.
  • IPv6 disabled – takes a lot of resources and we won’t use it anyway, for now.

I may add any further customizations if needed. You can always review the debootstrap-postconfig.sh script for details on what is being configured.

You can use this image/disk as a rootfs which you can boot directly on Bifferboard or try in Qemu. Note that you have to install our Debian kernel on Bifferboard prior to booting this rootfs.

Used resources:


by Leave a comment

Create a Qemu image file which you can mount in both Linux and Qemu

The idea is to be able to easily manage a Qemu image outside of Qemu, natively on Linux. This can help you to alter the files on the Qemu image easily on Linux and then test the modified Qemu image on a Qemu virtual machine.

You can download an empty, formatted with Ext3 Qemu raw image at the following URL address:

There is nothing special about how you can achieve this yourself:

  • Create an empty Qemu image file.
  • Run an installation CD of Debian (or any other Linux) under Qemu and use the empty image as an available hard disk.
  • Partition and format the Qemu hard disk (resp. the Qemu image file) using the Linux installer.
  • Interrupt the Linux installer, stop Qemu, mount the Qemu image on Linux and clean it up.

You can achieve the above using the following commands:

# make sure to update the .iso URL if needed
$ wget http://cdimage.debian.org/debian-cd/5.0.3/i386/iso-cd/debian-503-i386-netinst.iso
$ qemu-img create hd0.img 2G
$ qemu -hda hd0.img -cdrom debian-503-i386-netinst.iso -boot d
# continue with the installation to the point where you can set up the partitions
# set up a primary partition using the entire disk space, do not set up a swap partition; save changes to disk and continue
# interrupt the installation (for example from the second console by executing "halt"), stop the virtual machine, we will not need it any further
$ sudo mkdir -p /mnt/diskimage
$ sudo mount -o loop,offset=32256 hd0.img /mnt/diskimage
$ sudo rm -r /mnt/diskimage/*
$ sudo mkdir -m 0700 '/mnt/diskimage/lost+found'
$ sudo umount /mnt/diskimage

Now we have an empty Qemu image which we can mount in both Linux and Qemu.

Here is an example on how to mount this image in Qemu:

qemu -usb -usbdevice disk:hd0.img

Do not use the image simultaneously as Linux mount and Qemu hard disk.

Used resources:


by Leave a comment

Build a Debian Linux kernel for Bifferboard as .deb packages

In my previous article I explained why and how to build a very small Linux kernel with all possible modules enabled which would help us to run a standard Debian installation on Bifferboard.

You can download the already built .deb packages for Debian “lenny” at the following addresses:

On my Bifferboard, I use the following Kernel command line to boot this kernel:

rootwait root=/dev/sda1 console=uart,io,0x3f8

For Qemu, because of some USB mass-storage emulation issues, the line looks like:

rootwait root=/dev/sda1 console=uart,io,0x3f8 irqpoll

Update: There are (more up-to-date) automated scripts which you can use for the below actions:

  • You need to checkout the whole Bifferboard SVN repository.
  • The scripts are located in the directory “/debian/kernel“. Execute the “build.sh” script from the checked out repository on your local computer, on a Debian “lenny” system.

If you want to build the packages yourself, you need to execute the following commands on a Debian “lenny” machine (a virtual machine or a chroot()’ed installation work too):

famzah@FURNA:~$ sudo apt-get install kernel-package fakeroot build-essential ncurses-dev tar patch
famzah@FURNA:~$ export KVERSION=2.6.30.5
famzah@FURNA:~$ rm -rf /tmp/tmpkern-$KVERSION
famzah@FURNA:~$ mkdir /tmp/tmpkern-$KVERSION
famzah@FURNA:~$ cd /tmp/tmpkern-$KVERSION && wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-$KVERSION.tar.bz2
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ tar -xjf linux-$KVERSION.tar.bz2
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ sudo mkdir -p /usr/src/bifferboard && sudo chown $USER /usr/src/bifferboard
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ mv linux-$KVERSION /usr/src/bifferboard/
famzah@FURNA:/tmp/tmpkern-2.6.30.5$ cd /usr/src/bifferboard/linux-$KVERSION
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ wget 'http://www.famzah.net/download/bifferboard/obsolete/bifferboard-2.6.30.5-12.patch' -O bifferboard-2.6.30.5-12.patch
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ patch --quiet -p1 < bifferboard-2.6.30.5-12.patch
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ wget http://www.famzah.net/download/bifferboard/obsolete/build-biff-kernel-2.6.30.5-deb.sh
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ chmod +x build-biff-kernel-2.6.30.5-deb.sh
famzah@FURNA:/usr/src/bifferboard/linux-2.6.30.5$ ./build-biff-kernel-2.6.30.5-deb.sh
# When "make menuconfig" is displayed, just EXIT and SAVE the configuration.
#
# After the build, you can find the two .deb packages in "/usr/src/bifferboard".

Used resources: