Intro to GDB remote debugging

A tiny introduction to remote debugging using GDB:

http://blip.tv/file/602952

Alexander Issaris

This August I became a daddy for the first time. Here are some pics of our little boy...

Debugging the Linux kernel using Eclipse/CDT and Qemu

A screencast demonstrating roughly the same thing is available at: http://blip.tv/file/586651
For iTunes users there's a videopodcast at: http://takis.blip.tv/rss/itunes/

Download the Linux kernel sourcecode from http://www.kernel.org/. For example, the current kernel version is 2.6.23, a direct link would be http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.9.tar.bz2

Extract the Linux kernel sourcecode:

cd /usr/local/src
tar xvjf linux-2.6.23.9.tar.bz2

We will build the Linux kernel in a different directory:

mkdir -p /mnt/build/linux-2.6

Then copy the provided kernel configuration into this directory giving it a new name ".config". The following commands will then use this as a base-configuration to start from.

Next, we'll configure the kernel. Just keep pressing enter to use the default answers to all the questions that the kernel configuration program will ask you.

cd /usr/local/src/linux-2.6.23
make oldconfig O=/mnt/build/linux-2.6

Next, make the kernel a bit easier to debug:

make menuconfig O=/mnt/build/linux-2.6

And enable the following options: In the "Kernel hacking" menu enable both "Compile the kernel with debug info" and "Compile the kernel with frame pointers".

Now, we'll fire up Eclipse with the CDT plugin. You can download Eclipse with the CDT plugin from http://www.eclipse.org/downloads/. You'll need to download "Eclipse IDE for C/C++ Developers".

Get rid of the intro screen.

You'll get an empty workspace as shown in the screenshot. First disable automatic building, by using the "Window->Preferences" menu, selecting "General->Workspace" and deselecting "Build automatically". Eclipse will perform a time consuming indexing operation which you can disable by using the "Window->Preferences" menu,  selecting "C/C++->Indexer" and switching from "Fast C/C++ Indexer" to "No Indexer".

Start a new project, by using File->New->Project...

Then select "C Project", "Makefile project", "Empty Project".

Now enter a project name and specify a specific directory for the project sourcecode. To do this, first uncheck the "Use default location" checkbox.

Finally click "Finish".

If you hadn't disabled indexing, Eclipse will now start indexing the Linux kernel sourcecode. This will take a long time.

You'll see a progressbar which might give you an indication on how long it might take to complete.

Eclipse finished indexing the kernel sourcecode. Now, we're ready to configure our debugger. Right-click on the project-name in the left pane (Project explorer) and select "Properties".

We want to modify the default build command and the location where the build files should go.

Uncheck "Use default build command" and enter make CC=gcc-3.4 O=/mnt/build/linux-2.6

Modify the build location by clicking the "File system..." button and browsing to /mnt/build/linux-2.6

Through the menu-bar select "Project->Build all" or press "Ctrl-b".

After some time the Linux kernel build will be completed and you see "bzImage is ready" appear in the Eclipse Console output.

Next, we'll run our kernel binary using the Qemu system emulator. The nice thing about Qemu is that besides the normal virtual HD, floppy and ISO image booting, it can also boot Linux kernels directly. And, Qemu provides a GDB-stub to which we can connect from our Eclipse debugger. The "-s" switch activates this GDB-stub. The "-S" switch makes sure Qemu doesn't start running before we're ready (it freezes the CPU at startup).

Because the CPU is "frozen" at startup, the Qemu window won't show anything useful yet.

Through the menubar, select "Run->Debug Configurations...". Double-click "C/C++ Local Application". Modify the "C/C++ Application" textentry to point to the actual Linux kernel, being /mnt/build/linux-2.6/vmlinux

Click on the "Debugger" tab, and in the "Debugger" listbox select the "gdbserver Debugger". Next, modify the "Stop on startup at:" to "start_kernel". Below this, you'll notice a frame named "Debugger Options"; click the "Connection" tab in this frame and modify the "Type" to "TCP" and the "Port number" to 1234. Continue by clicking the "Debug" button.

Eclipse might compile and link a bit, but will finally launch the debugger and ask if you want to switch to the debugging perspective. Say yes.

The next screenshot shows the debugging perspective. Just like with normal applications, you'll see that the line it is about the execute is highlighted.

In the Qemu window, you'll notice some output already. This is the output which happened in functions preceding the start_kernel() function.

...

By using "Run->Step over" or pressing the "F6" key, you can execute the kernel code line by line and examine what's happening.

If you want to see the assembly instructions which are being executed, you can add a view which displays this by selecting "Windows->Show View->Disassembly".

There's a register view too, as can be seen in the next screenshot. Registers who's contents has been altered by the previous execution step are highlighted in yellow.

You can add breakpoints, inspect variables, inspect memory and much more, but as you keep running the kernel you'll run in trouble as we did not specify a true harddisk image for Qemu. So, you'll get the following output in the Qemu window, because the Linux kernel could not find a root filesystem on our fake harddisk image "/dev/zero".

That's it. Hopefully the above is useful (and fun) for anyone :)

Last updated on 20080108.

Why GIT?

For those of you who do not know what GIT is: GIT is a version control system similar to systems like CVS, Subversion or Bitkeeper.

In short, my four reasons for using GIT are:

1. Because it is distributed, it allows people to work on projects without having an account
2. It is really fast
3. All revisions of every file _are_ always locally available for you to inspect
4. Easy branching
   

Distributed

The one thing I adore most about GIT, is that it is a distributed system. Not by itself ofcourse, but this gives you incredible freedom (from a software developers perspective). To explain what is so great about this, I'll illustrate how development used to go with CVS and Subversion and similar systems.

Lets assume you are not a member of the project you want to work on. Then you'd have to check out their code from their software repository, make some changes and then -as you want to get it back and integrated- make a patch and send it to the projects mailinglist.
This doesn't sound that bad. Until you actually start working this way. The problem is that you do not have any versioning system for your own changes. You are working on a read-only repository and have no way to check in small changes.

So, what do some developers do (as I did)? They set up a local repository and import the external projects code. They have write access to that local repository, so they can start make changes while having their changes versioned.

The problem with this approach is that you will most likely want to keep synchronized with the official project sourcecode. So now and then you'd have to try and create a patch and solve lots of conflicts.

So, you'd say, okay, lets synchronise more often, so that the patches keep nice and small, and the conflicts would be easy to fix. Ofcourse, but the problem is that there is no automated way to do this, so it would involve a lot of work.

GIT makes this trivial. With GIT you'd make a clone of the projects repository. Within this clone you can do as you please, as it is your local repository. Now and then (very often) you just pull in the changes from the main project repository and you solve the conflicts that might occur.

How difficult is it to set such an environment up?

git-clone http://your.favorite/project.git

That's it.

Now, you enter the newly created project directory and start hacking and committing your new changes with:

git-commit -a

After a while you want to resynchronize with the main project repository, so you do a:

git-pull

GIT will get all the new code from the main repository, and will try to automatically merge it. If it fails, it will mark the code with the familiar ">>>" markers, as in both CVS and Subversion. After solving the conflicts, you commit again.


Or, you can keep one branch in your own repository identical to the main repository, and create a separate branch for your own development:

git branch mywork

git checkout mywork

Do whatever stuff you want to do in this branch and commits as much as you want to. Afterward you can pull from the main (called master) branch to your branch:

git pull . master

So, this command means that you want GIT to pull in the changes from the master branch into your current branch (being the one you were hacking in).

Speed
Another great benefit of GIT is that it's fast, really fast. I once did a comparison of GIT versus CVS, Subversion, TLA and Bazaar (and possibly others, can't recall). I used the Linux kernel source tree as contents for my revision control systems, and imported the 2.6.0 kernel. Then proceeded committing the 2.6.1 diff, 2.6.2, etc. After going up until 2.6.something, I then made a normalsized patch, and did a commit, GIT was blazingly fast while the other were unworkably slow. Recently Jo Vermeulen did a similar test and published the results on his blog. His tests focus on Bzr being comparable to GIT performancewise.


Full history at your fingertips
The entire sourcecode of the FFmpeg project, with every revision of every file included, fits into 9MiB using GIT. This means that you can have the entire development history of the project on your local harddrive. As it can be stored easily on your harddrive, it is very fast to access, as no calls need to use network connections at all.

Easy branching
Another nice thing about GIT is the ease with which one can create branches. And, in contrary with f.e. SVN and CVS, you feel comfortable to create branches _all the time_. Why? Because
you can delete them whenever you want, and no traces will remain. So, after the following commands, the repository will be the same as before the commands:

git-branch profile
git-branch h264
git-branch -D h264
git-branch -D profile

So, I typically create branches for whatever patch I am about to create. In fact, I actually just start working on something, and if it starts out being something worth keeping, I create a branch and commit the just created codechanges in that newly created branch.

Diffs between branches are easy too:
git diff profile..h264

Pulling in changes from a different branch into the current one:
git pull . somebugfix

Using GIT
If you want to use GIT, you'll better enjoy using the commandline, as the most powerful features are available through the commandline. There are some GUI's available too, mostly for inspecting codechanges.

There's a GUI included, called gitk:


Such as QGit:


And, for the Curses lovers, tig:


Update: As Uoti Urpala commented on the FFmpeg mailinglist, and as I should have mentioned above, the distributed nature of GIT isn't unique. There are a lot of other distributed revision control systems: Mercurial, Bazaar, Bitkeeper, SVK, TLA/Arch, darcs, ... In fact, I tested a few of those a long time ago, and noticed that performance was suboptimal for some (TLA/Arch and at the time Bazaar - but as I said that was a long time ago) and some seemed a bit immature, others were closed-source and commercial software. So, for me the choice was rather obvious. Recently, I have been told Bazaar has made excellent progress performancewise, so that might be an interesting candidate too. I never really tried Mercurial, although I did have the impression that it might have all the advantages GIT has...

Update2: Jo Vermeulen pointed me to this mail on the Cairo mailinglist where similar advantages concerning GIT are being illustrated.

DosBox on Nokia 770

As I am experimenting with videoplayback on the Nokia 770, I had a look at both GTK+ and SDL for this purpose. After a quick hello-world style SDL application, I decided to get something cute running on the device.

I had a quick look at the Maemo Application Wishlist, and saw DosBox. This looked interesting to me, so I had a quick go at getting it up and running on the device. It compiled just fine in Scratchbox.

Several issues showed up. The first being related to SDL screen initialisation: you should always init SDL in 800x480 16 bits per pixel mode, fullscreen, with hardware accelerated SDL surfaces. Second issue was exactly the same as with Google LibJingle: it quickly ran out of memory and got killed by the kernel. Skimming through the sourcecode, followed by a quick hack fixed the first issue, while a swapfile fixed the second (16M swap appears to be enough).

Then as it seemed to work perfectly, I noticed that the virtual keyboard on the N770 was not accessible from within SDL, so I couldn't really do anything in my cute little DosBox :o) Anyways, likely it is possible to let DosBox start an app right from the commandline of an xterminal, and if the DOS application only requires a mouse, it could still be useful :)

Furthermore, DosBox seems to use the Alsa sequencer for MIDI support, that doesn't seem to work either, as there appears to be no sequencer. I haven't looked into this.

The binary is available here. The patch , horrible as it is -remember, it was a quick 2 min hack with grep and sed- is available here: Dosbox ugly N770 patch. It is truly a stupid patch, but as I'm distributing the binary, the GPL requires the availability of the code.

Books

Fiction
Angels and Demons - Dan Brown ***
Digital Fortress - Dan Brown **
The Da Vinci Code - Dan Brown ***
Deception Point - Dan Brown *
Big Fish - Daniel Wallace **
The Last Juror - John Grisham


History
Little Big Man - Thomas Berger **
Baudolino - Umberto Eco ***

Horror
At the Mountains of Madness - H.P. Lovecraft *
The Call of Cthulhu - H.P. Lovecraft *
The Lurking Fear - H.P. Lovecraft ***
The Outsider - H.P. Lovecraft ***
The Statement of Randolph Carter - H.P. Lovecraft
The Thing on the Doorstep - H.P. Lovecraft ***

Science fiction
Day of the Triffids - John Wyndham ***
Crime and Punishment - Fyodor Dostoyevsky ****


Non-fiction

History
The life of Greece - Will and Ariel Durant ****
Caesar and Christ - Will and Ariel Durant ****
The Age of Reason Begins - Will and Ariel Durant ***
The Age of Voltaire - Will and Ariel Durant ***
Rousseau and Revolution - Will and Ariel Durant ***

Other
Stupid White Men - Micheal Moore **
Dude, Where's My Country? - Michael Moore **
Adventures In A TV Nation - Michael Moore *

FOSDEM 2006

I went to the Free and Open Source Developers European Meeting (FOSDEM) in Brussels this weekend. I attended some quite interesting presentations about Plan9, DTrace and Xen.

Some of these presentation were recorded, and the videos are available for free.

That's me on the picture :) It was taken by a former classmate of mine, Dag, with his Nokia mobile phone.

Printing huge posters

I recently needed a huge poster of one of my own pictures. Luckily, on a GNU/Linux system, all the software you need for doing so, is already available and just works.

Here's the commands needed to convert a picture called "mypicture.jpg" to a big poster in a Postscript file called myposter.ps:

convert mypicture.jpg mypicture.ps
poster -mA4 -p4x4A4 mypicture.ps >myposter.ps

The parameter -mA4 selects the paper size you want to print the poster with. The parameter -p4x4A4 sets the poster size as being 16 A4 pages.

You can print using whatever printing application you like, be it lpr on the commandline or a graphical application such as Evince on GNOME systems (such as the Ubuntu Breezy system I am using).

Migrated my repositories

I've recently migrated my repositories to a new hosting provider. The Breezy repository is available at http://issaris.be/breezy and my new Maemo repository
with packages for the Nokia 770 is available at http://issaris.be/maemo.

Nokia 770 OSGi

Nokia770 running OSGi

These screenshots show a Nokia 770 with the Oscar OSGi implementation running on top of the JamVM Java virtual machine and GNU Classpath classlibraries.

Transcoding video for the K750i

This works fine for me:
ffmpeg -i sourcevideo.avi -s qcif -b 256 -acodec aac -ab 128 -ac 2 targetvideo.mp4

ffmpeg -i sourcevideo.avi -s qcif -vcodec mpeg4 -acodec aac -r 20 -b 192 -ab 96 -ac 2 -ar 44100 targetvideo.mp4

Ubuntu Breezy bug

When unmounting an external USB drive by clicking on the drive's icon and choosing "unmount volume",
the following error-message appears:


And, the strange thing is, the drive is apparantly unmounted, as there is no trace of it in either, mount or df's output.

Nokia 770

I have been playing with the ultra-cool Nokia 770! :)

Here's a pic of my RTAI LiveCD website rendered with the browser on the device (Opera).

Puppy for mom

We bought a cute little Cavalier King Charles Spaniel puppy for my mom:

Repository moved

Due to unstability of the server that previously hosted my repository, I've moved my Ubuntu Breezy repository. The new line to add to your /etc/apt/sources.list would be:
deb http://alpha.uhasselt.be/takis.issaris/breezy/ ./

E17 screenshots

Some screenshots of current Enlightenment 0.17's CVS code.


E17 starting...


Various apps available in the E17 CVS repository:


Entice, a image viewer using the E17 libs:

GIT CVS import Ubuntu Breezy package

Added a cvsps 2.1 package to the repository as Ubuntu Breezy still ships with 2.0rc1. Version 2.1 is needed for GIT's CVS import functionality.

UM Linux HOWTO

This is a short howto, explaining how to setup User Mode Linux for simple Linux kernel module development and experimentation. I'm using the current Linux kernel 2.6.15 version (actually, todays GIT tree :) )

I like to keep my filesystem structure nice and clean, which is why I tend to use multiple partitions and directories for separate things. I've got a /usr/local/src directory which contains the linux-2.6 subdirectory contains the Linux kernel sourcecode. All my builddirectories are stored in /mnt/build, and so the first thing to do is to create a new entry for the Linux 2.6 UML build:
mkdir /mnt/build/linux-2.6-um

Next, enter the Linux kernel sourcecode directory:
cd /usr/local/src/linux-2.6

We'll start from the default UML configuration file for the kernel:
cp arch/um/defconfig /mnt/build/linux-2.6-um/.config

Use the defaults from that default config file:
make oldconfig ARCH=um O=/mnt/build/linux-2.6-um/

In case there's anything you want to change in the configuration of the UML kernel, start
the configuration menu:
make menuconfig ARCH=um O=/mnt/build/linux-2.6-um/

Now, we're set to do the actual building of the kernel. This will take a while...
make ARCH=um O=/mnt/build/linux-2.6-uml/

We need a disk image for the UML kernel to use. So, we'll create one. There's many ways to do this, this is a rather simple one.

First create an empty file which will kind of represent a disk drive (or actually a partition on a diskdrive):
dd if=/dev/zero of=/usr/tmp/sarge.ext2 bs=1M count=500

Put a filesystem on this partition, ext2 is fine for such a simple development system:
mkfs.ext2 /usr/tmp/sarge.ext2

Mount the filesystem image using loopback:
mount /usr/tmp/sarge.ext2 /mnt/loop/ -o loop

Copy an existing Linux distribution in this mounted filesystem image:
cp -var /usr/chroots/sarge/* /mnt/loop/

Move into the mounted filesystem and create the needed device nodes for UML:
cd /mnt/loop/dev
./MAKEDEV ubd

Next we'll install the kernel modules into the mounted filesystem:
cd /usr/local/src/linux-2.6
make modules_install O=/mnt/build/linux-2.6-uml ARCH=um INSTALL_MOD_PATH=/mnt/loop/

And finally, umount the filesystem image:
umount /mnt/loop/

Make a link named root_fs to your new partition image:
cd /usr/tmp
ln sarge.ext2 root_fs

Start the new UML kernel (and in this case, skip the default initscripts to get a real quick boot):
/mnt/build/linux-2.6-uml/linux init=/bin/bash

On the host system, create a directory for sharing files with the guest:
mkdir /tmp/forguest

In the guest system, mount this:
mount none /tmp -t hostfs -o /tmp/forguest

Now, any file appearing on the host system in /tmp/forguest, will also appear within the guest system in the /tmp directory.


Write whatever kernelmodule code (which doesn't actually access hardware since we're using UML)
you want in the /tmp/forguest directory.

Execute the following command once:
make -C /mnt/build/linux-2.6-uml/ modules ARCH=um

Build the module with the following command:
make -C /mnt/build/linux-2.6-uml/ SUBDIRS=$(pwd) modules ARCH=um

From the guest system, load the kernel module using:
insmod ./mymodule.ko

Remove it with:
rmmod mymodule

That's it! :)

Linux 2.6.15 released!

Fifteen years after Linus Torvalds bought the machine which got Linux started, version 2.6.15 of his kernel has been released! :)

Added an Ubuntu Breezy image to my repository. Compiled for 686, with the default 686 ubuntu kernel configuration (thus most modules/drivers are available).

And, as I'm using an NVIDIA card here, I've added the NVIDIA kernel driver package too. It's not the current NVIDIA version, but the version included in Ubuntu Breezy (7667) recompiled for the above kernel.

Linux 2.6.15 Breezy kernel package
NVIDIA 7667 driver for the above 2.6.15 Breezy kernel

A quick glance at E17

Those interested in having a look at E17 but not willing to actually start using it, can use the following command to see E17 in a nice little window:
Xnest -ac :1 | enlightenment -display :1

You need to have the xnest package installed or you can install the xserver-xephyr package and use:
Xephyr -ac :1 | enlightenment -display :1


Totally OT: Qemu 0.8.0 is available now! Lots of new features! :-)