Fedora

Today it's voting day for me. As Hendrik already mentioned, it's time to vote for the European Parliament.

Besides that, I had to fill in three local ballots. One for my town council, and two times for the regional council.

But there's an even more important vote going on: Fedora has three elections running, one for the next Release name, one deciding about 5 seats on the next FESCO and three seats for the Board.

Unfortunately my preferred candidate, Cthulhu, wasn't running for any of the elections which meant I indeed had to settle for the lesser evil.

In case you haven't voted yet, do so at https://admin.fedoraproject.org/voting/.
You can find out about the candidates and their platforms at the nominataion page for FESCO or the nomination page for the Board.
Another good way of understanding the candidates and their intentions is the Questionaire every candidate was asked to fill in. This was the first time we've ever tried gathering questions from the Fedora contributors and submitting them to the candidates. The answers are very interesting and can be found in either plain text or as an OpenOffice Spreadsheet. My suggestion would be to look at the spreadsheet, it's easier to compare the different stances of each candidate.
And then, there are of course the Town Hall Meeting transcripts. The #fedora-townhall is a moderated IRC Chanel where the candidates are taking questions from the audience and try to answer them. The logs can be found on the normal Fedora Election pages.
There are a lot of worthwhile candidates, not the least your's truly. So don't waste your vote.

With current virtualization technology it is often desirable to install an absolute minimal system and then only add the one service running on the system.

Unfortunately, this is not as easy as the "Minimal Installation" of RHEL (I'm not even going to think about fedora) is rather huge and contains lots of unnecessary gems people do not want on a server.

The easiest way of achieving a minimal installation is to use a kickstart file and select only the necessary packages.

The following kickstart file only installs a usable base system but leaves out the unnecessary stuff often being installed on a "server" installation.
The %post script is used to do some final clean up.
In this case i386 packages are removed if running on a x86_64 system in order to prevent having a mixed userland.
For people not running with IPv6, v6 support is disabled/removed as well.

This template kickstart file should be customized and then be published on a ftp or http server.
The anaconda installer can be instructed to use this file by passing ks=http://your.server/minimal.ks on the command line.

#
# Minimal RHEL5 Installation
# http://blog.vodkamelone.de/archives/151-Red-Hat-Enterprise-LinuxCentOS-5-minimal-installation.html
#

install
# Mirror URL
url --url Your Mirror URL, e.g. http://mirror.centos.org/centos/5/os/x86_64
lang en_US.UTF-8
keyboard de-latin1-nodeadkeys
network --device eth0 --bootproto dhcp
# Your root password
rootpw --iscrypted your root password as a crypted string
firewall --enabled --ssh
firstboot --disable
authconfig --enableshadow --enablemd5
selinux --enforcing
# Timezone, change as needed
timezone --utc Europe/Berlin
bootloader --location=mbr
# Append the following line if you need serial console support
#--append="console=tty0 console=ttyS0,115200n8r"
# or for Xen:
#--append="console=tty0 console=xvc0"
key --skip
logging --host=You syslog server
skipx
# uncomment if you only need a text mode installation
#text
reboot
services --disabled ip6tables
clearpart --initlabel --all
autopart

# Packages selection.
%packages --nobase
kernel
yum
openssh-server
openssh-clients
dhclient
audit
man
logrotate
tmpwatch
vixie-cron
crontabs
# Remove some stuff we do not need.
-iptables-ipv6
-system-config-securitylevel-tui
-gnu-efi
-Deployment_Guide-en-US
-redhat-release-notes
-cryptsetup-luks
# Remove some further packages
-hal
-pm-utils
-dbus
-dbus-glib
# If you're using xen, the following packages can be removed as well
#-setserial
#-hal
#-pm-utils
#-kudzu
#-dbus
#-dbus-glib

# Run a post script to clean up a bit
%post
chvt 3
(
echo "Disabling IPv6"
sed -i -e 's/\(NETWORKING_IPV6=\).*/\1no/' /etc/sysconfig/network

cat << EOF >> /etc/modprobe.conf
# disable IPv6
alias net-pf-10 off
EOF

echo "Disabling Zeroconf"
grep -q '^NOZEROCONF=yes' /etc/sysconfig/network || sed -i -e '/^NETWORKING=yes/a NOZEROCONF=yes' /etc/sysconfig/network

# Running on x86_64? Remove i386 rpms
if [ "$(uname -m)" == "x86_64" ]; then
	echo "We're on x86_64, removing unwanted i386 libraries"
	rpm -qa --queryformat='%{n}-%{v}-%{r}.%{arch}\n' | grep '\.i[3456]86$' | xargs rpm -ev
	echo "done"
fi

# Adding ssh key
# You could add your ssh key here
#echo "Adding ssh key"
#mkdir -p /root/.ssh
#chmod 700 /root/.ssh
#echo 'your ssh key' > /root/.ssh/authorized_keys
#chmod 600 /root/.ssh/authorized_keys


# Running on XEN? Add serial console if not already configured
if [ -f /proc/xen/capabilities ] && [ $(cat /proc/xen/capabilities | wc -l) -eq 0 ]; then
	echo "Adding XEN serial console support"
	# Check it's not already configured and add it and allow root-logins
	grep -q xvc0 /etc/inittab || sed -i -e '/^# Run gettys/a co:2345:respawn:/sbin/agetty xvc0 9600 vt100-nav ' /etc/inittab 
	grep -q xvc0 /etc/securetty || echo xvc0 >> /etc/securetty
fi

) 2>&1 | tee /root/ks-post.log
chvt 1

Den üblichen Verächtigen dürfte bekannt sein, dass unsere nicht vertrauenswürdige Familienministerin gerade dabei ist die Weichen für das neue, kindersichere und reingewaschene Internet zu stellen.

Böse Zeitgenossen nehmen gerne das ebenso böse Wort "Zensur" in den Mund und vergessen dabei an die Kinder zu denken. Frau von der Leyens Internetsperren weisen jedoch den richtigen Weg.

Da dies ja ein hauptsächlich technisches Blog ist, hier also ein kleines Rezept für mod_rewrite um Besuchern der eigenen Webseite schonmal einen kleinen Vorgeschmack zu geben auf die Zukunft des deutschen Internets:

RewriteEngine on
RewriteCond %{HTTP_REFERER} ![den_eigenen_domainnamen_bitte_hier_einfügen] [NC]
RewriteCond %{HTTP_REFERER} !bmi\.pifo\.biz [NC]
RewriteRule ^(.*)$      http://bmi.pifo.biz/?http://%{HTTP_HOST}/$1 [R,L]

Sobald dieser Konfigurationsschnipsel in einer .htaccess Datei im Webroot einer Webseite abgelegt wurde, wird die Funktionsweise der bundesdeutschen Filterliste realitätsnah simuliert. Der Besucher wird beim ersten Aufruf einer Webseite auf eine täuschen echte Simulation der STOP Seite für gefährliche Internetangebote umgeleitet.

Sobald er sich von dieser humoristischen Seite auf die eigentlich besuchte Seite durchgeklickt hat, kann er diese jedoch normal durchklicken... Eigetnlich auch nicht anders als die blauen Free-Speech Schleifchen, die man vor vielleicht 12 Jahren überall im Web vorfand.

Ein kleines Beispiel für die in Zukunft sicher regelmäßig auftretenden, jedoch sehr bedauerliche Einzelfällen von Fehlkategorisierungen: blog.vodkamelone.de auf der Sperrliste. :-)

A friend of mine wanted a small embedded linux router. After evaluating several options such as Routerboard, Soekris or ALIX systems, he decided to get himself a Microtik Routerboard 433.

The RB433 is a small MIPS board based on the Atheros AR7100 chipset with a 300MHz CPU, 64MB RAM, 3 100Base-TX ethernet ports and three slots for MiniPCI Cards.

The Routerboard manufacturer Microtik delivers these systems with a software called "RouterOS". I haven't looked any closer at it but it seems to be Linux based system with some proprietary userspace management applications. RouterOS seems mostly to be just a Nortel-ish command line interface and a fugly webinterface. Some people claim that RouterOS is kinda nifty, but it's definitely not hackable enough considering the plans my friend had with his device.

To solve his dilemma, we did what everyone else does in a similar situation, we put a real Linux on it:
OpenWrt to the rescue!

Getting to know the Routerboard

When connecting the power to the Routerboard, the system beeps after a short time and outputs some status messages to the serial port. In order to read these, one has to connect to the serial port via a serial crossover cable and use a terminal program.

Minicom is one such terminal program. Personally though, I prefer cu from the uucp package as it is rather lightweight. All one has to type is cu -l ttyS0 -s 115200 and the bootup messages from the routerboard connected to COM1 will be visible. If you're using any other terminal program, the console settings are the usual 115200bps, 8 data bits, No parity bits and 1 stop bit.

RouterBOOT booter 2.15

RouterBoard 433

Authorization: Passed
CPU frequency: 300 MHz
  Memory size:  64 MB

Press any key within 2 seconds to enter setup

Now is a good time to press any key to enter the setup mode in order to see what the device can do.
You'll be presented with the following screen:

RouterBOOT-2.15
What do you want to configure?
   d - boot delay
   k - boot key
   s - serial console
   o - boot device
   u - cpu mode
   f - cpu frequency
   r - reset booter configuration
   e - format nand
   g - upgrade firmware
   i - board info
   p - boot protocol
   x - exit setup
your choice: 

Change the bootmode to tell the device _not_ to boot from the local flash chip (called NAND) but from the network. To do that, press "o" and "e".
Afterwards press "x" to leave the setup.
The device will try to boot and get its kernel from the network.

RouterBOOT booter 2.15

RouterBoard 433

Authorization: Passed
CPU frequency: 300 MHz
  Memory size:  64 MB

Press any key within 2 seconds to enter setup
trying dhcp protocol...........................................................
kernel loading failed

So it seems the device is looking for a kernel.
Let's build one for it to boot sucessfully from...

Building OpenWrt Kamikaze

In order to correctly install the OpenWrt system a linux host is needed to build the kernel image on. I've been using Fedora 9 from the Fedora Project which did the job perfectly. Any other recent distribution should work equally well.

First, check out the current development code via Subversion to have the greatest and latest code:

[athienem@localhost ~]$ mkdir ~/openwrt
[athienem@localhost ~]$ cd ~/openwrt
[athienem@localhost openwrt]$ svn co https://svn.openwrt.org/openwrt/trunk/
[...]
Updated to revision 13193.
[athienem@localhost openwrt]$ 

In order to install the system correctly we'll be needing two different OpenWrt images:
One image is a JFFS2 or SquashFS image to install onto the target device.
The other image is the so called ramdisk image which can be booted over the network and contains a minimal shell. This image will be used to install the JFFS2 or SquashFS image onto the device and will never again be used. Think of it as a "rescue image".

Both images are basically built the same way.
First, change to the svn checkout directory called trunk and execute make menuconfig to configure the OpenWrt image.

[athienem@localhost openwrt]$ cd trunk/
[athienem@localhost trunk]$ make menuconfig

This command will start the ncurses interface to generate a .config file. It should look familiar to people having built kernels before.
Make sure that "Atheros AR71xx [2.6]" is selected as the target system:

This will make sure that the resulting kernel is bootable on the Routerboard 433.

The next step is to select the target image format, chose ramdisk for now:

Now select "< Exit >" in the main menu and confirm that you want to save your new OpenWrt configuration.

The next step is to actually build the image by calling "make

*** End of OpenWrt configuration.
*** Execute 'make' to build the OpenWrt or try 'make help'.
[athienem@localhost trunk]$ make
Checking 'working-make'... ok.
Checking 'case-sensitive-fs'... ok.
Checking 'working-gcc'... ok.
Checking 'working-g++'... ok.
Checking 'ncurses'... ok.
Checking 'zlib'... ok.
Checking 'gawk'... ok.
Checking 'bison'... ok.
Checking 'flex'... ok.
Checking 'unzip'... ok.
Checking 'bzip2'... ok.
Checking 'patch'... ok.
Checking 'perl'... ok.
Checking 'wget'... ok.
Checking 'gnutar'... ok.
Checking 'autoconf'... ok.
Checking 'non-root'... ok.
Collecting target info: done
Collecting package info: done
Checking 'bison'... ok.
Checking 'automake'... ok.
 make[2] tools/install
[...]
 make[2] target/install
 make[3] -C target/linux install
 make[2] package/index
[athienem@localhost trunk]$ 

Everything went fine and there should be a ramdisk image in elf format:

[athienem@localhost trunk]$ ls -all bin/openwrt-ar71xx-vmlinux-initramfs.elf 
-rwxrwxr-x 1 athienem athienem 3735060 2008-11-13 22:27 bin/openwrt-ar71xx-vmlinux-initramfs.elf
[athienem@localhost trunk]$ 

The next step is to build the system image to be installed on the device. Execute make menuconfig again but this time select either squashfs or jffs2 as the target image format instead of ramdisk:
After saving the config, execute make again. This time, it should be much faster as nearly everything is already compiled.

#
# using defaults found in .config
#


*** End of OpenWrt configuration.
*** Execute 'make' to build the OpenWrt or try 'make help'.

[athienem@localhost trunk]$ make
++ mkdir -p /home/athienem/openwrt/trunk/staging_dir/toolchain-mips_gcc4.1.2
++ cd /home/athienem/openwrt/trunk/staging_dir/toolchain-mips_gcc4.1.2
++ mkdir -p bin lib include stamp
 make[1] world
[...]
 make[2] target/install
 make[3] -C target/linux install
 make[2] package/index
[athienem@localhost trunk]$ 

Now the bin/ directory should be filled with some files:

[athienem@localhost trunk]$ ls -all bin/
total 23656
drwxrwxr-x  3 athienem athienem    4096 2008-11-08 18:25 .
drwxrwxr-x 15 athienem athienem    4096 2008-11-13 22:44 ..
-rw-rw-r--  1 athienem athienem     710 2008-11-13 22:46 md5sums
-rw-rw-r--  1 athienem athienem 1499367 2008-11-08 18:25 openwrt-ar71xx-rootfs.tgz
-rw-rw-r--  1 athienem athienem 1441792 2008-11-08 18:25 openwrt-ar71xx-root.squashfs
-rw-rw-r--  1 athienem athienem 2492740 2008-11-13 22:46 openwrt-ar71xx-uImage.gz
-rwxrwxr-x  1 athienem athienem 2248838 2008-11-08 18:25 openwrt-ar71xx-vmlinux.bin
-rwxrwxr-x  1 athienem athienem 2258096 2008-11-08 18:25 openwrt-ar71xx-vmlinux.elf
-rw-rw-r--  1 athienem athienem 1048576 2008-11-08 18:25 openwrt-ar71xx-vmlinux.gz
-rwxrwxr-x  1 athienem athienem 3725815 2008-11-13 22:46 openwrt-ar71xx-vmlinux-initramfs.bin
-rwxrwxr-x  1 athienem athienem 3735072 2008-11-13 22:46 openwrt-ar71xx-vmlinux-initramfs.elf
-rw-rw-r--  1 athienem athienem 2555904 2008-11-13 22:46 openwrt-ar71xx-vmlinux-initramfs.gz
-rw-rw-r--  1 athienem athienem 2293760 2008-11-13 22:46 openwrt-ar71xx-vmlinux-initramfs.lzma
-rw-rw-r--  1 athienem athienem  786432 2008-11-08 18:25 openwrt-ar71xx-vmlinux.lzma
drwxrwxr-x  3 athienem athienem    4096 2008-11-08 17:50 packages
[athienem@localhost trunk]$ 

Booting OpenWrt on the RouterBoard

To boot the routerboard, a dhcp server is needed to tell the bootloader on the Routerboard which IP address it should use and where to get it's bootable kernel image.
The tftp server is needed to actually serve said image to the RouterBoard.

Under Fedora linux, installing both just needs the command yum install -y dhcp tftp-server. To activate both services, chkconfig can be used as root:

[root@localhost ~]# chkconfig dhcpd on
[root@localhost ~]# chkconfig tftp on

The configuration for the dhcpd needs to be adapted to the local circumstances. The setup I've been using was a crosslinked cable between the notebook and the Routerboard with a manually configured IP address of 192.168.23.254/24. All that is configured in that file is to assign the RouterBoard an IP address and tell it to boot the file vmlinux. Adapt the following file as needed for your own circumstances:

[root@localhost ~]# cat /etc/dhcpd.conf 
# Global Parameters

authoritative;

max-lease-time 604800;
default-lease-time 3100;

ddns-update-style none;
ddns-ttl 7200;

allow booting;
allow bootp;
one-lease-per-client true;

subnet 192.168.23.0 netmask 255.255.255.0 {
	option routers 192.168.23.254;
	option subnet-mask 255.255.255.0;
	option broadcast-address 192.168.23.255;
	ignore client-updates;
}

group {
	host routerboard {
		hardware ethernet 00:0c:42:32:43:8a;
		next-server 192.168.23.254;
		fixed-address 192.168.23.2;
		filename "vmlinux";
	}

}
[root@localhost ~]# 

Start the dhcp server by calling service dhcpd start, if there are any problems, look into /var/log/messages and fix the issues noted there.

The tftp-server has already been activated earlier but might need a service xinetd restart to be really started. Do that.
Then copy the ramdisk image named openwrt-ar71xx-vmlinux-initramfs.elf to the tftproot. This is /tftpboot on older systems or /var/lib/tftpboot/ on newer systems. Name the copied file vmlinux.

If everything is working fine, the system should boot:

RouterBOOT booter 2.15

RouterBoard 433

Authorization: Passed
CPU frequency: 300 MHz
  Memory size:  64 MB

Press any key within 2 seconds to enter setup...

trying dhcp protocol... OK
resolved mac address 00:1C:23:03:AA:F8
Gateway: 192.168.23.254
transfer started ............................ transfer ok, time=1.68s
setting up elf image... OK
jumping to kernel code
Linux version 2.6.26.7 (athienem@localhost.localdomain) (gcc version 4.1.2) #1 Sat Nov 8 18:11:40 CET 2008
console [early0] enabled
CPU revision is: 00019374 (MIPS 24K)
Determined physical RAM map:
 memory: 04000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Zone PFN ranges:
  Normal          0 ->    16384
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0:        0 ->    16384
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
Kernel command line: root=/dev/mtdblock2 rootfstype=squashfs,yaffs,jffs2 noinitrd console=ttyS0,115200 init=/etc/preinit
Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes
Writing ErrCtl register=000227c0
Readback ErrCtl register=000227c0
PID hash table entries: 256 (order: 8, 1024 bytes)
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Memory: 60768k/65536k available (1762k kernel code, 4700k reserved, 312k data, 1572k init, 0k highmem)
SLUB: Genslabs=6, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Mount-cache hash table entries: 512
net_namespace: 484 bytes
NET: Registered protocol family 16
MIPS: machine is MikroTik RouterBOARD 433/AH
registering PCI controller with io_map_base unset
PCI: mapping irq 33 to pin1@0000:00:13.0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP reno registered
NET: Registered protocol family 1
squashfs: version 3.0 (2006/03/15) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
yaffs Nov  8 2008 18:08:56 Installing. 
msgmni has been set to 118
io scheduler noop registered
io scheduler deadline registered (default)
Serial: 8250/16550 driver $Revision: 1.90 $ 1 ports, IRQ sharing disabled
serial8250.0: ttyS0 at MMIO 0x18020000 (irq = 11) is a 16550A
console handover: boot [early0] -> real [ttyS0]
ag71xx_mdio: probed
eth0: Atheros AG71xx at 0xba000000, irq 5
eth1: Atheros AG71xx at 0xb9000000, irq 4
NAND flash driver for RouterBoard 4xx series version 0.1.10
NAND SPI clock 25000 kHz (AHB 150000 kHz / 6)
FLASH SPI clock 25000 kHz (AHB 150000 kHz / 6)
NAND device: Manufacturer ID: 0xad, Chip ID: 0x76 (Hynix NAND 64MiB 3,3V 8-bit)
Scanning device for bad blocks
Bad eraseblock 828 at 0x00cf0000
Creating 3 MTD partitions on "NAND 64MiB 3,3V 8-bit":
0x00000000-0x00040000 : "booter"
0x00040000-0x00400000 : "kernel"
0x00400000-0x04000000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
split_squashfs: no squashfs found in "NAND 64MiB 3,3V 8-bit"
Atheros AR71xx SPI Controller driver version 0.2.2
Atheros AR71xx hardware watchdog driver version 0.1.0
Registered led device: rb4xx:yellow:user
TCP vegas registered
NET: Registered protocol family 17
802.1Q VLAN Support v1.8 Ben Greear 
All bugs added by David S. Miller 
Freeing unused kernel memory: 1572k freed
Algorithmics/MIPS FPU Emulator v1.5
[sighandler]: No more events to be processed, quitting.
[cleanup]: Waiting for children.
[cleanup]: All children terminated.
- preinit -
Press CTRL-C for failsafe

Please press Enter to activate this console. br-lan: Dropping NETIF_F_UFO since no NETIF_F_HW_CSUM feature.
device eth0 entered promiscuous mode
PPP generic driver version 2.4.2
ip_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (1024 buckets, 4096 max)
wlan: trunk
ath_hal: module license 'Proprietary' taints kernel.
ath_hal: 2008-10-02 (AR5210, AR5211, AR5212, AR5416, RF5111, RF5112, RF2413, RF5413, RF2133, RF2425, REGOPS_FUNC, DFS, XR)
ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk)
ath_rate_minstrel: look around rate set to 10%
ath_rate_minstrel: EWMA rolloff level set to 75%
ath_rate_minstrel: max segment size in the mrr set to 6000 us
wlan: mac acl policy registered
ath_pci: trunk
PCI: Enabling device 0000:00:13.0 (0000 -> 0002)
Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies
wifi0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboA rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: H/W encryption support: WEP AES AES_CCM TKIP
ath_pci: wifi0: Atheros 5212: mem=0x10000000, irq=33
eth0: link up (100Mbps/Full duplex)
br-lan: port 1(eth0) entering learning state
br-lan: topology change detected, propagating
br-lan: port 1(eth0) entering forwarding state



BusyBox v1.11.2 (2008-11-08 17:55:16 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 KAMIKAZE (bleeding edge, r13141) -------------------
  * 10 oz Vodka       Shake well with ice and strain
  * 10 oz Triple sec  mixture into 10 shot glasses.
  * 10 oz lime juice  Salute!
 ---------------------------------------------------
root@OpenWrt:/# 

If something didn't work out, check your system log to see what happens. Adding the "-s" parameter to the tftpd binary might be useful as it will log single requests.

Permanently installing OpenWrt on the RouterBoard

As we have an accessible Linux system running now on the RouterBoard the available tools such as scp and mtd can be used to copy the needed files onto the NAND device and thus permanently install OpenWrt on the device.

Under Linux the NAND device is partitioned and can be accessed through the mtd framework which exports some information to userspace through the /proc filesystem:

root@OpenWrt:/# cat /proc/mtd 
dev:    size   erasesize  name
mtd0: 00040000 00004000 "booter"
mtd1: 003c0000 00004000 "kernel"
mtd2: 03c00000 00004000 "rootfs"

As can easily be seen, there are three "partitions" available. Leave the one called "booter" alone, it might be important and contain the bootloader. I haven't checked. All we're interested in is "kernel" and "rootfs". The former contains the kernel, the latter the root filesystem.

To install the elf kernel binary named openwrt-ar71xx-vmlinux.elf, it has to be transferred onto the RouterBoard and written onto the second mtd partition. Make sure that the file is called kernel.
Note: OpenWrt configures its IP address to be 192.168.1.1 on bootup. You might have to change this with ifconfig.

root@OpenWrt:/# scp athienem@192.168.23.254:openwrt/trunk/bin/openwrt-ar71xx-vmlinux.elf /tmp/
root@OpenWrt:/# mount /dev/mtdblock1 /mnt/
yaffs: dev is 32505857 name is "mtdblock1"
yaffs: passed flags ""
yaffs: Attempting MTD mount on 31.1, "mtdblock1"
root@OpenWrt:/# mv /tmp/openwrt-ar71xx-vmlinux.elf /mnt/kernel
root@OpenWrt:/# ls /mnt
kernel      lost+found
root@OpenWrt:/# umount  /mnt/
save exit: isCheckpointed 0
root@OpenWrt:/# 

The kernel image is installed.
The rootfs is even easier, as the mtd-device does not need to be mounted at all to write the squashfs image called openwrt-ar71xx-root.squashfs in my case.
This just needs to be written as is onto the mtd block device named rootfs:

root@OpenWrt:/# scp athienem@192.168.23.254:openwrt/trunk/bin/openwrt-ar71xx-root.squashfs /tmp/
root@OpenWrt:/# cat /tmp/openwrt-ar71xx-root.squashfs > /dev/mtdblock2
root@OpenWrt:/# 

After a few seconds the squashfs image has been written and the device can be rebooted. Don't forget to disable the network boot in the Bios:

root@OpenWrt:/# reboot
root@OpenWrt:/# br-lan: port 1(eth0) entering disabled state
device eth0 left promiscuous mode
br-lan: port 1(eth0) entering disabled state
eth0: link down
Restarting system.


RouterBOOT booter 2.15

RouterBoard 433

Authorization: Passed
CPU frequency: 300 MHz
  Memory size:  64 MB

Press any key within 2 seconds to enter setup..

RouterBOOT-2.15
What do you want to configure?
   d - boot delay
   k - boot key
   s - serial console
   o - boot device
   u - cpu mode
   f - cpu frequency
   r - reset booter configuration
   e - format nand
   g - upgrade firmware
   i - board info
   p - boot protocol
   x - exit setup
your choice: 

Press "o" twice and "x" once to continue booting normally from the NAND.

RouterBOOT booter 2.15

RouterBoard 433

Authorization: Passed
CPU frequency: 300 MHz
  Memory size:  64 MB

Press any key within 2 seconds to enter setup..
loading kernel from nand... OK
setting up elf image... OK
jumping to kernel code
Linux version 2.6.26.7 (athienem@localhost.localdomain) (gcc version 4.1.2) #2 Sat Nov 8 18:25:41 CET 2008
console [early0] enabled
CPU revision is: 00019374 (MIPS 24K)
Determined physical RAM map:
 memory: 04000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Zone PFN ranges:
  Normal          0 ->    16384
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0:        0 ->    16384
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
Kernel command line: root=/dev/mtdblock2 rootfstype=squashfs,yaffs,jffs2 noinitrd console=ttyS0,115200 init=/etc/preinit
Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes
Writing ErrCtl register=000227c0
Readback ErrCtl register=000227c0
PID hash table entries: 256 (order: 8, 1024 bytes)
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Memory: 62208k/65536k available (1762k kernel code, 3252k reserved, 312k data, 124k init, 0k highmem)
SLUB: Genslabs=6, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Mount-cache hash table entries: 512
net_namespace: 484 bytes
NET: Registered protocol family 16
MIPS: machine is MikroTik RouterBOARD 433/AH
registering PCI controller with io_map_base unset
PCI: mapping irq 33 to pin1@0000:00:13.0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP reno registered
NET: Registered protocol family 1
squashfs: version 3.0 (2006/03/15) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
yaffs Nov  8 2008 18:08:56 Installing. 
msgmni has been set to 121
io scheduler noop registered
io scheduler deadline registered (default)
Serial: 8250/16550 driver $Revision: 1.90 $ 1 ports, IRQ sharing disabled
serial8250.0: ttyS0 at MMIO 0x18020000 (irq = 11) is a 16550A
console handover: boot [early0] -> real [ttyS0]
ag71xx_mdio: probed
eth0: Atheros AG71xx at 0xba000000, irq 5
eth1: Atheros AG71xx at 0xb9000000, irq 4
NAND flash driver for RouterBoard 4xx series version 0.1.10
NAND SPI clock 25000 kHz (AHB 150000 kHz / 6)
FLASH SPI clock 25000 kHz (AHB 150000 kHz / 6)
NAND device: Manufacturer ID: 0xad, Chip ID: 0x76 (Hynix NAND 64MiB 3,3V 8-bit)
Scanning device for bad blocks
Bad eraseblock 828 at 0x00cf0000
Creating 3 MTD partitions on "NAND 64MiB 3,3V 8-bit":
0x00000000-0x00040000 : "booter"
0x00040000-0x00400000 : "kernel"
0x00400000-0x04000000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
split_squashfs: no squashfs found in "NAND 64MiB 3,3V 8-bit"
Atheros AR71xx SPI Controller driver version 0.2.2
Atheros AR71xx hardware watchdog driver version 0.1.0
Registered led device: rb4xx:yellow:user
TCP vegas registered
NET: Registered protocol family 17
802.1Q VLAN Support v1.8 Ben Greear 
All bugs added by David S. Miller 
VFS: Mounted root (squashfs filesystem) readonly.
Freeing unused kernel memory: 124k freed
Please be patient, while OpenWrt loads ...
Algorithmics/MIPS FPU Emulator v1.5
- preinit -
Press CTRL-C for failsafe
- init -

Please press Enter to activate this console. br-lan: Dropping NETIF_F_UFO since no NETIF_F_HW_CSUM feature.
device eth0 entered promiscuous mode
PPP generic driver version 2.4.2
ip_tables: (C) 2000-2006 Netfilter Core Team

nf_conntrack version 0.5.0 (1024 buckets, 4096 max)
wlan: trunk
ath_hal: module license 'Proprietary' taints kernel.
ath_hal: 2008-10-02 (AR5210, AR5211, AR5212, AR5416, RF5111, RF5112, RF2413, RF5413, RF2133, RF2425, REGOPS_FUNC, DFS, XR)
ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk)
ath_rate_minstrel: look around rate set to 10%
ath_rate_minstrel: EWMA rolloff level set to 75%
ath_rate_minstrel: max segment size in the mrr set to 6000 us
wlan: mac acl policy registered
eth0: link up (100Mbps/Full duplex)
br-lan: port 1(eth0) entering learning state
br-lan: topology change detected, propagating
br-lan: port 1(eth0) entering forwarding state
ath_pci: trunk
PCI: Enabling device 0000:00:13.0 (0000 -> 0002)
Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies
wifi0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboA rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: H/W encryption support: WEP AES AES_CCM TKIP
ath_pci: wifi0: Atheros 5212: mem=0x10000000, irq=33



BusyBox v1.11.2 (2008-11-08 17:55:16 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 KAMIKAZE (bleeding edge, r13141) -------------------
  * 10 oz Vodka       Shake well with ice and strain
  * 10 oz Triple sec  mixture into 10 shot glasses.
  * 10 oz lime juice  Salute!
 ---------------------------------------------------
root@OpenWrt:/# 

Done.

OpenWrt has been installed on the device and can be used and configured as usual.

For more information about configuring, using and customizing OpenWrt see the Kamikaze Manual, the OpenWrt Wiki or use the source. For network related configuration issues, /lib/network/config.sh and the files in /lib/wifi/ are a good start.

As most readers of this blog are probably aware, pre-releases of the Linux Kernel 2.6.27 are able to trash the NVRAM/EEPROM of certain Intel Network cards. As usual, lwn.net has a nice writeup of the issue including some background information.
I had a similar problem in the past and as a favour for some friends of mine, I've written down a small description to restoring the ethernet firmware in these cards. This guide should serve as a good primer on reflashing your broken nic but probably needs to be adapted for your own use case.

NB: Instead of just giving a command by command description of what I did, I'll try explaining a bit more about the background and the process of fixing the problem at hand. Maybe this gives other people some insight into valuable problem solving skills.

Some years back we bought quite some Tyan S5112 machines for bawue.net.
The idea was to fit these machines with the Tyan m3289 server management card, an IPMI card allowing remotely powercycling of the machines and offering a serial console via the network.

In order to have the whole setup work, the IPMI management module needs support from the network interface in order to receive IP packets while the machine is powered off. After contacting the Tyan support, we were offered a firmware file to flash into the network adapter activating the needed "management mode". This firmware file came in the form of a .bin file and an accompanying eeupdate.exe file for flashing the firmware image.

The mainboard has two ethernet controllers, with the 82547EI one being the controller utilized by the management card. The lspci output on this board looks as follows:

[root@selene ~]# lspci|grep Ethernet
01:01.0 Ethernet controller: Intel Corporation 82547EI Gigabit Ethernet Controller
03:02.0 Ethernet controller: Intel Corporation 82541GI Gigabit Ethernet Controller
[root@selene ~]#

The instructions for flashing the firmware were relatively simple: Boot with a DOS bootdisk and execute eeupdate -nic=1 -d 82547EI.eep
After I pressed enter, nothing much happened. The system wasn't reading anything from the disk, there was no progress message on the screen, just the general start-message of the firmware update tool.

When nothing happened after 5 minutes of waiting, I foolishly reset the system. Big mistake!.

On the next boot of the system, there were no PXE messages from the network card and during bootup the e1000 linux driver only threw out the ominous message The EEPROM Checksum Is Not Valid without loading the network interface.
It turns out, I just trashed the firmware of my on-board network interface.
Calling the eeupdate program again did not work as it bailed out because the eeprom was corrupted
At such times, I firmly believe in the use of swearwords accompanied by heavy googling for some advice. Alas, googling told me only that I broke my hardware and needed new one.

As returning hardware because of a problem is akin to giving up, which is generally unacceptable, I decided to look into the issue a bit more and find a workable solution to unbrick the network interface.
To recap, I had the following:

• One on-board network interface with a corrupted nvram
• One eeprom image for said network interface (For reference: 82547EI.EEP)
• No working flash program
• No working driver
• One working Linux system without network
• One working Linux notebook with network

Luckily, the working Linux system and network access is all I needed:

The first step was getting the sources of the e1000 driver from the project page. As this was a few years ago, I chose the version 7.3.15 which was current at this time.

After untaring the sources, a quick grep -R 'The EEPROM Checksum Is Not Valid' e1000-7.3.15 turned up one hit in e1000-7.3.15/src/e1000_main.c:

So there is a function called e1000_validate_eeprom_checksum responsible for checking the validity of the eeprom. During the main initialization of the card this function is called and in case the checksum is not valid, the error handler err_eeprom is executed which aborts the module load.
On a hunch, I removed the whole check logic containted in this function located in e1000-7.3.15/src/e1000_hw.c. After I was done, the whole function body consisted only of a "return 0" statement meaning that the checksum check will always succeed.

Building the modified module by calling make in the src dir resulted in a e1000.ko file which could be loaded into the running kernel by executing "insmod ./e1000". (Note, this will probably not work with current kernels as the buildscripts have changed. Use a current version of the e1000 driver instead.)
To my great relief the driver returned the following message:

Intel(R) PRO/1000 Network Driver - version 7.3.15
Copyright (c) 1999-2006 Intel Corporation.
e1000: 0000:01:01.0: e1000_probe: (PCI:33MHz:32-bit) ff:ff:ff:ff:ff:ff
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
e1000: 0000:03:02.0: e1000_probe: (PCI:33MHz:32-bit) 00:e0:81:55:f2:01
e1000: eth1: e1000_probe: Intel(R) PRO/1000 Network Connection

So even though the mac address of the card is broken, at least the card is somewhat detected and I can work on restoring the eeprom.

For modifying low-level settings of network interfaces under Linux one can usually use the fabulous ethtool utility.
Using the -e parameter dumps the eeprom values of the specified network interface onto the screen. This is great for getting a backup of the eeprom:

[root@selene ~]# ethtool -e eth1 | head -n 5
Offset		Values
------		------
0x0000		00 e0 81 55 f2 01 10 02 ff ff 06 20 ff ff ff ff 
0x0010		ff ff ff ff 0b 64 76 10 86 80 76 10 86 80 84 b2 
0x0020		dd 20 22 22 00 00 90 2f 80 23 12 00 20 1e 12 00 
[root@selene ~]# 

Even better is the -E parameter as it allows changing a single byte at a specified address in the eeprom:

[root@selene ~]# ethtool -E eth0 magic 0x10198086 offset 0x0 value 0x00
[root@selene ~]# 

This command would change the byte at the address 0x0 (the first byte) into the value 0x00. The 0x10198086 value is the "magic" value needed to "unlock" this write operation. Depending on the driver and the card this value is different for each system. In the case of the intel e1000 driver, the magic value is the Device ID and Vendor ID of the selected network card. This value can be gathered by examining the lspci -n output.
As I was in a hurry back then to get the machine working again, I didn't try to find out what exactly the magic value was but just commented out this check in the e1000_ethtool.c file.
For reference, the patch of my modifications to the e1000 driver are e1000-repair.patchdownloadable as a unified diff.

Now, that I could change single values in the eeprom, it was time to take a look at the Tyan provided eeprom file:

[root@localhost root]# head -n 5 82547.eep 
E000 2A81 0855 0A10 FFFF FFFF FFFF FFFF 
FFFF FFFF 640B 1019 8086 1019 8086 B200 
1F35 002A 0E00 0012 0E00 20DD 7777 1F95 
0001 1F73 0098 1F72 3FB0 0009 1200 3649 
00CF 8FA7 290E 0305 0CCA FFFF FFFF FFFF 

Comparing this eeprom file with the dump taken earlier from the second network interface in the machine showed that the .eep file from intel was in "mixed-endian" format, meaning I had to shuffle the values around a bit before being able to rewrite the image. The file contains the eeprom values as groups of two bytes each in reversed order. The first four byte-values in the file are 0xe0 0x00 0x2a 0x81 while in the eeprom they would be 0x00 0xe0 0x81 0x2a.

After I found the correct byte ordering, I could simply call ethtool -E manually with the correct addresses and just write each byte into the eeprom or automate this and reduce the possibility of mistakes. Naturally, automation it is. Back then I chose to do this script in PHP as a small exercise in command-line-interface programming.
The PHP script can be downloaded as eepromer.php and executed by calling php eepromer.php on the shell.
In order to explain it's working, the code is printed below:

At the start, the variable file contains the filename to read in. This file is then opened and read into memory as it is only 6K large. The PHP String Tokenizer function is used to extract the values from the script and the bytes in each extracted group are then swapped around to put them into big endian byte-order. When the eeprom file has been completely parsed the ethtool commands to write the gathered data into the eeprom are printed to STDOUT:

[root@localhost root]# php eepromer.php | head -n 5
ethtool -E eth0 magic 0x00 offset 0x0 value 0x00
ethtool -E eth0 magic 0x00 offset 0x1 value 0xE0
ethtool -E eth0 magic 0x00 offset 0x2 value 0x81
ethtool -E eth0 magic 0x00 offset 0x3 value 0x2A
ethtool -E eth0 magic 0x00 offset 0x4 value 0x55
[root@localhost root]# 

By piping the output of this quick-and-dirty script into a shell (php eepromer.php | sh), the content of the .eep file is written for real into the eeprom. The last step is changing the first 6 bytes of the eeprom (offset 0x0 to 0x5) to the original mac address of the network interface.
After this has been done, the network card is considered repaird or unbricked.

Now, this explanation should give anyone some hints on fixing his network card's eeprom should this be needed because of problems with the kernel releases mentioned in the beginning. It is unlikely that following the procedure above to the letter is going to have any usable results as every system and situation is different.
The major problem would be, that not everyone has a working .eep file from Intel available after his own card is trashed. My suggestion would be to look for a friend who has exactly the same card. This can be achieved by using lspci:

[root@selene ~]# lspci | grep Ethernet
01:01.0 Ethernet controller: Intel Corporation 82547EI Gigabit Ethernet Controller
03:02.0 Ethernet controller: Intel Corporation 82541GI Gigabit Ethernet Controller
[root@selene ~]# lspci -n | grep '01:01\.0'
01:01.0 0200: 8086:1019
[root@selene ~]# 

If a friend has the same network card as indicated by the Vendor and Device ID (8086 == Intel, 1019 == 82547EI Gigabit Ethernet Controller in my example) he should be able to take eeprom dump by calling ethtool -e [device] > /tmp/eeprom-[device].dump.:

[root@selene ~]# ethtool -e eth0 | head -n 8
Offset		Values
------		------
0x0000		00 e0 81 55 f2 00 10 0a ff ff ff ff ff ff ff ff 
0x0010		ff ff ff ff 0b 64 19 10 86 80 19 10 86 80 00 b2 
0x0020		35 1f 2a 00 00 0e 12 00 00 0e dd 20 77 77 95 1f 
0x0030		01 00 73 1f 98 00 72 1f b0 3f 09 00 00 12 49 36 
0x0040		cf 00 a7 8f 0e 29 05 03 c8 0c ff ff ff ff ff ff 
0x0050		ff ff ff ff ff ff ff ff ff ff ff ff ff ff 02 06 
[root@selene ~]# 

This dump file can then be written into your own nvram by an easier procedure then described above. After all, no endianness swapping is necessary as ethtool already returned the data correctly. A bit of reformatting of the import is necessary however but can be accomplished in a simple bash script:

This script, which can be written as one single line, will remove the header and other superfluous data from the dumpfile, leaving only the values itself which are then echoed to STDOUT in the form of an ethtool command.

[root@selene ~]# magic=0x0; j=0; for i in `sed -e '1,2d' /tmp/eeprom-[device].dump | cut -c 9- |
tr -d "\n"`; do echo ethtool -E magic $magic offset 0x$(printf %x ${j}) value 0x${i}; j=$(($j + 1)); done | head -n 5
ethtool -E magic 0x0 offset 0x0 value 0x00
ethtool -E magic 0x0 offset 0x1 value 0xe0
ethtool -E magic 0x0 offset 0x2 value 0x81
ethtool -E magic 0x0 offset 0x3 value 0x55
ethtool -E magic 0x0 offset 0x4 value 0xf2
[root@selene ~]#

Piping this into a shell will restore your eeprom meaning only the mac address has to be reverted to the old one. The correct working of the above bash line should be tested however, as the output of ethtool differs depending on the card and the driver.

Should the network interface not even be visible on the PCI bus anymore (possible due to the usage of the ibautil.exe tool mentioned on some webpages) reflashing the main bios might work for some systems. The flashrom utility from the coreboot project might come in handy for this.
Otherwise it might be necessary to rewrite the necessary part of the eeprom first through the SPI, a serial programming interface, in order to have it enumerate again on the pci bus.
Should this be the case, it might be easier to just return your mainboard for repair though.
Should you be willing however, to try your luck and certainly void any little bit of warranty you had left, Intel offers a nice manual explaining the firmware format of it's ICH8 system.

Hans wrote about a different way of preparing a bootable USB stick for a Fedora installation then I did in Installing Fedora 9 from a (small) USB stick.
Unfortunately, his blogpost is set to only accept comments from "friends" and does not even accept the Fedora OpenID url as a "friend". :-( Thus I'll reply as an article:

While my recipe wrote the data directly onto the USB stick without creating any partitions ont it, Hans suggests to format the USB stick similar to a normal hard drive with a Master Boot Record and a single partition holding the data. This is based on the hope that it makes it more likely that the stick is in fact bootable.

Fedora, and in extension Red Hat Linux before it, has always created partition-less diskboot.img files. The commands I listed in my article are taken directly from the sourcecode of the anaconda-runtime scripts, which have in the past generated the shipped diskboot.img file.

I might not remember the exact date but I'm certain I've been booting installation images from USB sticks for more than 5 years now and never had a problem with the partition-less disk-image as found on the installation CDs in the /images-folder.
In fact, I remember some problems with an older EPIA-M board which absolutely refused to boot from an USB stick formatted as either a hard-drive or a ZIP-drive with the data contained on the 4th partition.
But that was in 2003 and I'd really expect current BIOS implementations to correctly boot from whatever disk is plugged in. Yeah, right...

While working with the "new" ASUS Pundit-R350 I found the IO performance of the system severely lacking.

When doing anything IO heavy on the box system load goes up and the system becomes basically unresponsive. I noticed that while preparing an image for an USB stick:

[root@workstation ~]# dd if=/dev/zero of=usbstick.img count=1024 bs=1M
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 300.266 s, 3.6 MB/s
[root@workstation ~]# uptime
 14:51:02 up 7 min,  2 users,  load average: 4.53, 3.63, 1.66
[root@workstation ~]#

3.6 MB/s transfer rate is completely unacceptable. I'm feeling as if 1994 called and asked for the Red Hat Linux 3.0.3 (Picasso) install CDs.
To solve this mystery, why a SATA disk is running that slow a bit of digging is needed.

Usually, such slow IO access means the drive in question is not using DMA transfer but one of the slower PIO modes.
This can be checked with the hdparm utility:

[root@workstation ~]# hdparm -I /dev/sda | grep '[DP][MI][AO]:'
	DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 ∗udma5 udma6 
	PIO: pio0 pio1 pio2 pio3 pio4 
[root@workstation ~]# 

As can be seen, the udma5 entry has an asterisk in front of it, identifying the currently used transfer mode is in fact UDMA/100.
Considering the claimed transfer mode and the actual speed I'd suspect a kernel problem. The tool for viewing the kernel log is dmesg. Using this command in combination with grep allows one to quickly find anything related to ata:

[root@workstation ~]# dmesg | grep ata
 BIOS-e820: 000000003bf40000 - 000000003bf50000 (ACPI data)
Memory: 965836k/982272k available (2233k kernel code, 15736k reserved, 1120k data, 284k init, 64768k highmem)
      .data : 0xc062e7b5 - 0xc0746800   (1120 kB)
Write protecting the kernel read-only data: 908k
libata version 3.00 loaded.
scsi0 : pata_atiixp
scsi1 : pata_atiixp
ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xff00 irq 14
ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xff08 irq 15
ata1.00: ATAPI: _NEC DVD_RW ND-4550A, 1.06, max UDMA/33
ata1.00: configured for UDMA/33
ata2.00: ATA-7: ST3250823AS, 3.03, max UDMA/133
ata2.00: 488397168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata2.00: simplex DMA is claimed by other device, disabling DMA
ata2.00: configured for PIO4
EXT3-fs: mounted filesystem with ordered data mode.
[root@workstation ~]#

The line ata2.00: simplex DMA is claimed by other device, disabling DMA and ata2.00: configured for PIO4 explain the problem. The disk was in fact only being accessed by one of the PIO modes. Googling for the first line, gives several results. Of interest are the different hits from the Arch-Linux, the Ubuntu and the Fedora forums. Seems other people have the same problems. The first hit however is from the Ubuntu Wiki suggesting that the problem is caused by the ata_generic module which prevents the pata_atiixp module from correctly driving the interface.

It quickly turns out however, that the wiki entry is incorrect and the forums are completely useless as nobody really looked into the issue at all. Koen, explains this nicely.

A bit of googling for Simplex DMA shows that the kernel driver is simply buggy/the way fedora loads sata drivers is buggy.
Happily, cebbert (one of our kernel wranglers) already applied a patch fixing this problem.

This nicely shows that even though fedora's way of trailing the bleeding edge quite closely may lead to strange hardware problems from time to time, the fix is usually pretty quickly applied.
Considering that the relatively large number of users allows for a larger testing base and therefore improving the software for every user in the long run I find this is an acceptable compromise.

I've recently started to unpack some of the boxes from my last move. In there I found an older Booksize PC, an ASUS Pundid-R350. It's not the nicest system I've ever had, the fan is incredibly noisy and it's constantly switching to full speed when the system is loaded. I guess I should swap the fan for something much quieter, but haven't had the time yet.

It took me some time to figure out the correct settings to have lm_sensors running on the device as there's been no template configuration to be found on google. To rectify this omission, I'm dumping mine on my blog:

# Asus Pundit-R350
chip "it8712-∗"

# Voltage monitors as advised in the It8705 data sheet

    label in0 "VCore 1"
    label in2 "+3.3V"
    label in3 "+5V"
    label in4 "+12V"

    # Seem not to be connected on the Pundit-R
    ignore in1
    ignore in5
    ignore in6
    ignore in7
    ignore in8
    ignore cpu0_vid

    compute in3 ((6.8/10)+1)∗@ ,  @/((6.8/10)+1)
    compute in4 ((30/10) +1)∗@  , @/((30/10) +1)

# Set some upper and lower limits

    set in0_min 1.4 ∗ 0.95
    set in0_max 1.4 ∗ 1.05
    set in2_min 3.3 ∗ 0.95
    set in2_max 3.3 ∗ 1.05
    set in3_min 5.0 ∗ 0.95
    set in3_max 5.0 ∗ 1.05
    set in4_min 12 ∗ 0.95
    set in4_max 12 ∗ 1.05

# Temperature
    label temp1       "CPU Temp"
    set temp1_type 3
    set   temp1_max   75
    set   temp1_min   15
    label temp2       "M/B Temp"
    set temp2_type 2
    set   temp2_max   55
    set   temp2_min   15
    ignore temp3

# Fans
   set fan1_div 2
   set fan1_min 2500
   ignore fan2
   ignore fan3

With this configuration written to /etc/sensors3.conf the freshly installed fedora 9 system correctly shows some sensible sensor readings:

[root@workstation ~]# sensors
it8712-isa-0260
Adapter: ISA adapter
VCore 1:     +1.39 V  (min =  +1.33 V, max =  +1.47 V)
+3.3V:       +3.26 V  (min =  +3.14 V, max =  +3.47 V)
+5V:         +5.00 V  (min =  +4.76 V, max =  +5.24 V)
+12V:       +12.03 V  (min = +11.39 V, max = +12.61 V)
fan1:       3994 RPM  (min = 2657 RPM, div = 2)
CPU Temp:    +65.0°C  (low  = +15.0°C, high = +75.0°C)  sensor = thermal diode
M/B Temp:    +50.0°C  (low  = +15.0°C, high = +55.0°C)  sensor = transistor

[root@workstation ~]# 

My notebook is running on Red Hat Enterprise Linux. 5.2 This is very nice on the one hand, as I do have stable software and it's very unlikely that I'll kill my system with a yum update call right before having a presentation. That happened in the past when running rawhide, but that's perfectly okay. I just have different requirements now.
On the other hand, being on a stable operating system means I'm not having access to the latest and greatest releases of all software. A problem users on Debian stable are well acquainted with.

One of the problems I'm having is with NetworkManager 0.6.4-8.el5. One of the wireless networks I'm using is changing it's keys each quarter. This is actually a good idea. Unfortunately, NetworkManager on some systems seems not to be able to associate to the wireless network after the first keychange.
It's trying again and again to associate, asks for a new password but even though the new password/hex-key is entered, no association is possible.

A quick workaround is possible though requiring removal of the password and the network data from the Gnome keyring as well as the gconf registry. After this has been done, NetworkManager is able to associate with the wireless network:

Make sure that the gnome-keyring-manager as well as the gconftool-2 binaries are installed:

[root@localhost ~]# rpm -q gnome-keyring-manager GConf2
gnome-keyring-manager-2.16.0-3.el5
GConf2-2.14.0-9.el5
[root@localhost ~]#

The next step is to remove the wireless network from the Gnome registry with the help of gconftools-2. Substitute <network-name> accordingly:

[athienem@localhost ~]$ gconftool-2 --recursive-unset /system/networking/wireless/networks/<network-name>
[athienem@localhost ~]$

The next step is to fire up gnome-keyring-manager from the shell or the Applications-menu under "System-Tools". Navigate to the entry named "Passphrase for wireless network <network-name>", select it and click on the "Keyring" entry in the menu and choose "Delete Key".
Close gnome-keyring-manager.

You're done. Now NetworkManager has lost all knowledge about the existing wireless network and the entry can be added again as usual. NM should be able to associate then.

With the release of Fedora 9, anaconda has introduced a new consolidated boot image:

"Consolidated network booting ISO image, replacing old boot.iso, diskboot.img, and rescuecd.iso."

The problem with this approach is, that the 12MB diskboot.img file has disappeared, which could be used to boot a system with a small USB stick. Awesome! Thank you very much. ∗facepalm.jpg∗

But luckily, it is not very hard to recreate this file on the shell and write it onto an USB stick afterwards. This serves as another installment of my small article series announced earlier.

I started with the boot.iso image which I downloaded from http://download.fedoraproject.org/pub/fedora/linux/releases/9/Fedora/i386/os/images/boot.iso and loopback mounted on /mnt:

[root@localhost ~]# wget -q -c
http://download.fedoraproject.org/pub/fedora/linux/releases/9/Fedora/i386/os/images/boot.iso
[root@localhost ~]# mount boot.iso /mnt -o loop,ro
[root@localhost NotBackedUp]# ls /mnt/
images  isolinux
[root@localhost NotBackedUp]# 

Now the steps to create the USB pendrive image are quite simple. A file is created, formatted and made bootable with syslinux and the appropriate files from the boot.iso are copied over:

[root@localhost ~]# cd /tmp/
[root@localhost tmp]# mkdir pendrive
[root@localhost tmp]# cd pendrive/
[root@localhost pendrive]# dd if=/dev/zero of=diskboot.img bs=1M count=12
12+0 records in
12+0 records out
12582912 bytes (13 MB) copied, 0.0433931 seconds, 290 MB/s
[root@localhost pendrive]# mkdosfs diskboot.img 
mkdosfs 2.11 (12 Mar 2005)
[root@localhost pendrive]# syslinux diskboot.img 
[root@localhost pendrive]# mkdir mount
[root@localhost pendrive]# mount diskboot.img mount/ -o loop
[root@localhost pendrive]# cp -r /mnt/isolinux/∗ mount/
[root@localhost pendrive]# grep -v local mount/isolinux.cfg > mount/syslinux.cfg
[root@localhost pendrive]# rm -f mount/isolinux.∗
[root@localhost pendrive]# umount -f mount
[root@localhost pendrive]# rm -rf mount/
[root@localhost pendrive]# cd
[root@localhost ~]# umount -f /mnt
[root@localhost ~]#

After these steps, a diskboot.img file is located in /tmp/pendrive. This file can be written to an USB stick with a command such as dd if=diskboot.img of=/dev/sdb. Special care should be taken to write to the correct device (/dev/sdb in this case)and _not_ overwrite one of your own harddrives.

When working with disk images the admin is often faced with the problem of mounting the contained partitions inside of these images.

Such full-disk images can be the containers of virtual machines for xen as fount in /var/lib/xen/images or disk-images ripped from dying drives with the help of recovery tools such as dd_rescue.

Regardless of the origin, these images do contain everything needed to run a machine, a partition table, a boot sector, at least one, but often several partitions inside of one container.
Mounting these images through a normal loop mount does only result in the first image to be mounted, often not what is wanted.
In the past, suggestions ranged from imaging single partitions to using dd to "cut" the single partitions out of the full-disk image. Crude workarounds as we've had a perfect tool called device mapper for some time now.

Executing fdisk on the my test disk-image shows several partitions, among some ingorable warnings:

[root@workstation ~]# fdisk -l disk.img 
You must set cylinders.
You can do this from the extra functions menu.

Disk disk.img: 0 MB, 0 bytes
4 heads, 32 sectors/track, 0 cylinders
Units = cylinders of 128 ∗ 512 = 65536 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
disk.img1   ∗           1          32        2032   83  Linux
disk.img2              33         216       11776   83  Linux
disk.img3             217         705       31296   83  Linux
disk.img4             706         978       17472    5  Extended
disk.img5             706         859        9840   83  Linux
disk.img6             860         978        7600   83  Linux
[root@workstation ~]# 

Thus, we know we do have an image file containing several partitions.

Now, how to handle this?

Mounting this test-image gives an error in my case. Often though the first partition gets sucessfully mounted.

[root@workstation ~]# mount disk.img /mnt/ -t ext2 -o loop,ro
mount: wrong fs type, bad option, bad superblock on /dev/loop1,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

[root@workstation ~]# 

The solution to this is called kpartx, a tool to map the partitions contained inside such an image into device-mapper block devices. This tools is contained in the kpartx RPM and can be easily installed by executing yum install kpartx if it is not already installed on your system.

Execution is quite simple. To list the contained partitions and see onto which loop-devices they would be mapped, the following command is needed:

[root@workstation ~]# kpartx -l disk.img 
loop0p1 : 0 4064 /dev/loop0 32
loop0p2 : 0 23552 /dev/loop0 4096
loop0p3 : 0 62592 /dev/loop0 27648
loop0p5 : 0 19680 /dev/loop0 90272
loop0p6 : 0 15200 /dev/loop0 109984
[root@workstation ~]# 

To actually map these partitions onto the loop-devices, call kpartx with the -a instead of the -l parameter:

[root@workstation ~]# kpartx -a -v disk.img 
add map loop0p1 : 0 4064 linear /dev/loop0 32
add map loop0p2 : 0 23552 linear /dev/loop0 4096
add map loop0p3 : 0 62592 linear /dev/loop0 27648
add map loop0p5 : 0 19680 linear /dev/loop0 90272
add map loop0p6 : 0 15200 linear /dev/loop0 109984
[root@workstation ~]# 

The partitions are then available in /dev/mapper/ to be mounted accordingly and the containing files to be accessed:

[root@workstation ~]# ls -all /dev/mapper/
total 0
drwxr-xr-x  2 root root     160 2008-06-11 15:06 .
drwxr-xr-x 13 root root    4180 2008-06-11 15:06 ..
crw-rw----  1 root root  10, 60 2008-06-11 13:28 control
brw-rw----  1 root disk 253,  0 2008-06-11 15:06 loop0p1
brw-rw----  1 root disk 253,  1 2008-06-11 15:06 loop0p2
brw-rw----  1 root disk 253,  2 2008-06-11 15:06 loop0p3
brw-rw----  1 root disk 253,  3 2008-06-11 15:06 loop0p5
brw-rw----  1 root disk 253,  4 2008-06-11 15:06 loop0p6
[root@workstation ~]# 
[root@workstation ~]# mount /dev/mapper/loop0p5 /mnt/ -o loop,ro
[root@workstation ~]# mount | grep '/mnt'
/dev/mapper/loop0p5 on /mnt type ext2 (ro,loop=/dev/loop1)
[root@workstation ~]# ls /mnt/
backup  conf.tar.gz     etc                  manifest.txt  tool
bin     default.list    factory_default.md5  rc.reboot     upgrade.list
cfg     default.tar.gz  lost+found           test
[root@workstation ~]# 

After the partitions have been unmounted, the device-mapper mappings can be removed by calling kpartx with the -d parameter. The image file can then be used again as it is not in use anymore by the device mapper.

[root@workstation ~]# umount /mnt/
[root@workstation ~]# kpartx -d -v disk.img 
del devmap : loop0p1
del devmap : loop0p2
del devmap : loop0p3
del devmap : loop0p5
del devmap : loop0p6
loop deleted : /dev/loop0
[root@workstation ~]# ls -al /dev/mapper/
total 0
drwxr-xr-x  2 root root     60 2008-06-11 15:16 .
drwxr-xr-x 13 root root   4080 2008-06-11 15:16 ..
crw-rw----  1 root root 10, 60 2008-06-11 13:28 control
[root@workstation ~]# 

I finally decided to try to breath some life into my weblog again and write some hopefully useful technical articles.

I'm currently planning on concentrating on stuff I find out fiddling with new toys (usually embedded stuff running Linux) or little hints or tricks I'm using when administrating systems.

The content of these articles might not all be new and many people already know about them, but I found it's mostly "advanced" administration stuff many people do not know about and can profit from these.

If there's anything which should be described in more detail or anything else, I'm available on the usual IRC networks (IRCnet, oftc, freenode etc.) as ixs.

The first article explains kpartx and how to use it to access partitions inside full-disk images.

We've gotten a new document scanner for bawue.net in order to better handle our documents and our snail-mail communication.
The Canon ScanFront 220P is a network scanner which does (among other things) optical character recognition and is able to send the resulting document as a PDF via e-Mail or upload it to a FTP Server or a SMB Network share.

The device itself is running Windows CE in order to accomplish all that which is fine as it therefore needs no computer connected to it as some other scanners do.

A real problem for the Linux user howerver is that Canon seems to have handed the development of the scanner's web interface to some clueless and moronic developers.
There's simply no excuse for designing a web interface which only works with Internet Explorer and doesn't even allow a Firefox user to log into the interface.
I'd expect MUCH better from a global company such as Canon, especially when the scanner in question costs about US$ 2,150.00

If the device would have been running Linux and the sources would have been delivered as requested by the GPL I'd have a working scanner here and Canon's development team would have had a nice unified diff in the mail fixing their problems. That way, all they are getting is an acrid mail.
Another reason to favour open source operating systems for any development work. Some companies seem to clearly get this.

In the meantime tough I need a working scanner. Greasemonkey to the rescue: I've written a small GPL3 licensed Greasemonkey User script fixing the problems in the ScanFront webinterface. Naturally, one has to have greasemonkey installed to use this script.

It currently fixes the login prompt, makes the "New User" button work, fixes an onload-recursion in the address book and makes the job-control window work when selecting the destination address for a scanjob.
Most of these problems stem from the fact that the Internet Explorer XML parser seems to be somewhat sloppy and does not make a difference by getElementById and getElementsByName.
My userscript works around these issues but a real fix can only be delivered by Canon.

Today was my first day as a Red Hat employee.

Even though I'm just working there as an intern for the next six months before starting work on my Diploma thesis, it has been rather uneventful.

Ohh right, today is saturday. Weekends for teh win!
So I'll start on monday then with the "New Hire Orientation" in Munich. According to the guys on irc, it's gonna be fun.

Yeah, right!

Michael linked to an IAT Test which claims to be able to find out your unconscious preference for either Microsoft Software or Open-Source software, ruling out self-presentation or controlled responding.
Wikipedia has somewhat more about the Implicit Association Test.

In this case, the participant is asked to please sort stimulus words (Joy, Evil etc.) and pictures (Tux, Firefox Logo, Microsoft Word Logo and similar). By combining "good" stimuli with "bad" attitudes and measuring response time and comparing the results from the test round where "good" stimuli is paired with "good" attitudes the IAT claims to be able to offer insight into a person, regardless of his self-reports.

I am quite wary of tests such as these as I believe them to be useless against a participant who intends to game the test. Especially if the test claims to be objective.

The first trial I did resulted in being attested to have "a strong automatic preference for Open Source products over Microsoft products".
This sounds about right.

The second trial, I tried gaming the system. As the participant is asked to sort the words and images into two combined groups which change, it was clear that the IAT test is based on reaction measurements. If sorting the words into the "Microsoft + Good" group is being done faster by the participant then sorting the words into the "Microsoft + Bad" group (compared to the "OSS + Good" group vs "OSS + Bad" group), it is assumed that the association between good and Microsoft is closer then the association between bad and Microsoft. Thus the user has a unconscious preference for Microsoft.

This much I gathered by guessing and the result was that I consciously delayed my responses at the appropriate time, thus resulting in a slight preference for Microsoft over OSS.

A short Google-Search later I was reading a paper by Klaus Fiedler and Matthias Blümke from the University of Heidelberg confirming my speculation and resulting in the third test run certifying "a moderate automatic preference of Microsoft over Open Source Software"...

So much for the test. :-)