nomeata’s mind shares

ipatch on hackage

mail@joachim-breitner.de (nomeata) — Sun, 22 Aug 2010 21:05:13 +0200

With the Beta 3 release of Darcs 2.5 on Hackage (the Haskell library and program repository), the ipatch program I recently introduced could now be uploaded to hackage, too. If you use cabal-install, you can now install and use it with a simple run of "cabal install ipatch".

I also made the program now handle patches that add or remove files, extended the help texts a bit and added a test suite. This means that you can actually make use of ipatch as of now, to split patches into several small patches and to apply a patch interactively. Of course it needs some more testing, and you might have feature wishes – in either case, let me know.

ipatch, the interactive patch editor

mail@joachim-breitner.de (nomeata) — Tue, 03 Aug 2010 22:42:17 +0200

The problem: Splitting patches

As a Debian maintainer, I often work with patches (files listing changes to text files), for example when tracking the modification I make to some software before I upload the package to Debian. To manage these patches, quilt is a nice tool: It helps you maintain a stack of patches on top of the original code and encourages you to keep your variously modifications separate.

One use case is not supported by quilt at all: Splitting patches. One often has a large patch containing several independent changes. This might happen after you fix a few problems in the upstream code and then run dpkg-buildpackage, which will create one patch of your changes and put it in debian/patches. Before, I had to manually edit the patch and write the hunks, which are the building blocks of patches, into separate file.

Where it already works

There is no such problem when using a version control system, such as Darcs. Especially Darcs is rightly famous for its user-friendly interface and powerful hunk-selection features. You can even split a single hunk (which could be a change to one line) into two separate steps! Have a look at the HunkEditor page on the Darcs wiki to see how that works.

Let’s steal a feature

Well, it is not stealing if it is Free Software... Darcs has these nice capabilities and provides them in the context of version control systems, while we need them in the context of patch files. But Darcs is providing an API to its code, so shoudn’t it be possible to create a program that uses the Darcs code to split patch files? As a matter of fact, it is possible: You can see that program in action on this 3min Ogg Theora-Video or directly here if your browser supports HTML5:

Looks like your browser does not support HTML5

Nice, can I use it?

The code is a working proof of concept. What you see works. You do not see how it handles patches that create or delete files, patches that do not apply cleanly or are already applied or any kind of error handling. That does not work yet. If you still want to try it, you can grab the code from the Darcs repository at http://darcs.nomeata.de/ipatch, but you need to build the latest development state of the Darcs library first.

I think ipatch could become a very useful and powerful tool with applications in areas where nobody would think of using Darcs. I definitely want some integration into quilt, replacing the splitted patch in the series by the replacing patches automatically. Maybe even a git plugin could be created? But I don’t think I can push this project far enough on my own. So this is an invitation to join me and make ipatch a great tool. This invitation goes especially to the Darcs developers: Please have a look how the code uses the Darcs API and help to improve the collaboration here. I think we can use the darcs-users mailing list until there is need for a dedicated mailing list.

Protecting static content selectively by OpenID

mail@joachim-breitner.de (nomeata) — Mon, 02 Aug 2010 12:02:40 +0200

When I created my on-line photo album, we had the year 2001 and there was nothing wrong with publishing ones photos in the Internet. It has been almost ten years now, and things have changed. Privacy is an issue, especially when it also affects your friends and family.

The problem

So I needed to find some protection. These were my requirements:

I only want to protect parts of the site, as there are some pictures that I intentionally share with the public. This selection should be possible down to the individual image.
The solution should work without any dynamic server component besides the web server itself. This rules out any self-written CGI scripts as well.
One password, given to all my friends etc., is not sufficient as it might leak to unintended audience.
I do not want my visitors to have to register and remember yet another username and password just for my site. I also do not want to manage this user database.
I do not want to have do large changes to the file structure of my photo album.

This rules out most common options, e.g. protection by a .htpasswd file. Requirements 3 and 4 point to a solution based on OpenID. With OpenID, my visitors can authenticate against a service they already use (Google, Yahoo, etc.), relieving me from the burden of maintaining a user database and them from having to remember a password.

There is a mod_auth_openid module for the Apache webserver, and it is even distributed with Debian in the libapache2-mod-auth-openid package. So requirement 2 is fulfilled. The tricky part is: How do we achieve OpenID protection for some images, and not for others.

The solution

I first played around with selectively enabling or disabling mod_auth_openid based on <FileMatch> directives in the Apache configuration, but it was not elegant and would not scale well. I have more than 20.000 pictures to manage, and have already selected over 5000 pictures to be shown without protection. My solution is based on a partial copy of the whole directory tree that contains all public files. To save disk space, these are just symbolic links to the real file in the protected location. Some mod_rewrite magic then takes care of giving the user the impression that all files are in the same location. I set up a small example of my solution, which has this directory structure:

.:
drwxr-xr-x 2 root root 4096  2. Aug 12:03 images
lrwxrwxrwx 1 root root   18  1. Aug 12:05 index.html -> private/index.html
lrwxrwxrwx 1 root root   18  1. Aug 12:05 login.html -> private/login.html
drwxr-xr-x 3 root root 4096  2. Aug 12:03 private

./images:
lrwxrwxrwx 1 root root 33  2. Aug 12:00 pleaselogin.png -> ../private/images/pleaselogin.png
lrwxrwxrwx 1 root root 28  2. Aug 12:03 public.png -> ../private/images/public.png

./private:
drwxr-xr-x 2 root root 4096  2. Aug 12:00 images
-rw-r--r-- 1 root root  267  2. Aug 12:03 index.html
-rw-r--r-- 1 root root   94  2. Aug 12:01 loggedin.html
-rw-r--r-- 1 root root 2091  2. Aug 12:03 login.html
-rw-r--r-- 1 root root   10 18. Nov 2009  protected.html

./private/images:
-rw-r--r-- 1 root root 4074  2. Aug 11:58 pleaselogin.png
-rw-r--r-- 1 root root 2670  2. Aug 11:58 private.png
-rw-r--r-- 1 root root 2043  2. Aug 11:58 public.png

As you can see, real files only reside in private/, outside of that, only symbolic links exist.

The apache configuration protects the private directory and blends it into the main directory:

   <directory /var/www/nomeata.de/openid-test>
        RewriteEngine On
        # Abuse the login page as an error image
        RewriteCond %{QUERY_STRING} \.(png|jpg)
        RewriteRule ^login.html$ /openid-test/images/pleaselogin.png
        # Ship private files, if they exist, unless public files exist
        RewriteCond  $1 !^private
        RewriteCond  /var/www/nomeata.de/openid-test/$1 !-f
        RewriteCond  /var/www/nomeata.de/openid-test/private/$1 -f
        RewriteRule  ^(.+)$ /openid-test/private/$1
   </directory>
   <directory /var/www/nomeata.de/openid-test/private>
        AuthOpenIDEnabled        On
        AuthOpenIDDBLocation     /var/lib/apache2/mod_auth_openid/mod_auth_openid.db
        AuthOpenIDLoginPage      /openid-test/login.html
        AuthOpenIDTrustRoot      http://nomeata.de
        AuthOpenIDCookiePath     /
        AuthOpenIDCookieLifespan 2592000
    </directory>

A special trick handles the “login page” for protected images: If the login page is requested and the referrer indicates that the user tried to access a .png or .jpg file, apache will instead ship an image containing an error message.

For my photo album I have a small Perl script that, given a directory with a private/ directory therein and a list of rules in form of glob patterns, will symlink matching files and remove symlinks that are not allowed any more.

What’s next?

As you can see, this does not actually protect the content. It only requires the user to authenticate, then everything is visible. To select which OpenIDs are allowed to access which code, some bugs will have to be fixed in mod_auth_openid first. There was little activity there recently, I hope that the project is not dead.

How forky may one maintain a Debian package?

mail@joachim-breitner.de (nomeata) — Sat, 17 Jul 2010 21:59:47 +0200

I maintain most of my Debian packages because I use them myself. Sometimes, I have some needs that go slightly beyond what is currently offered by the software. This is not a problem: Debian ships Free Software and I can program, therefore I can patch the software to also do what I want it to do. Trying to be a good member of the Free Software community, I then submit the patch to the upstream author. If he accepts the patch (which is usually the case), everything is fine. But what if he does not reply to the report or rejects it because he does not want this feature (although the patch is technically fine)? I see two options:

I could continue to use a privately patched and built version of the package, while separately building packages for Debian. This way, Debian ships the software as intended by the upstream maintainer while I can use the features I need. On the other hand, I would not be using the version that I upload to Debian, which is not good, and it causes double work when a a new version is released.
I could upload a package to Debian that contains my patch. The technical infrastructure to add patch in Debian packages has always been there... I would actually use the package as it is in Debian and only manage one line of versions. But would I be abusing my powers as a Debian maintainer? If I were not the maintainer, I could not make this decision by myself (this happend with my patch to nagstamon). Plus it could have a negative effect on the Debian-upstream relationship.

How do other Debian Developers handle such issues? The actual case I’m considering is a feature enhancement for link-monitor-applet (but I only just wrote the patch, so it does not yet fall in the category “upstream does not reply”).

Kit-Card-Poster in der Mensa

mail@joachim-breitner.de (nomeata) — Tue, 13 Jul 2010 09:40:10 +0200

Jemand hat wohl wieder ein satirisches Poster in der Mensa aufgehängt:

Für die externen: Bisher haben Studenten an der Uni-Karlsruhe eine „Fricard“. Da die Uni jetzt aber KIT heißt und schwer elite ist, muss natürlich eine neue Karte her, die KIT-Card. Dass die ähnlich eines Schokoriegels klingt, ist wohl nicht bedacht worden. Das erinnert an ein ähnliches Poster vor zwei Jahren zum Thema Mensa-Nachtisch.

Free Groups Formalized

mail@joachim-breitner.de (nomeata) — Sat, 03 Jul 2010 12:44:22 +0200

Since a few months, I have been playing around with Isabelle, a theorem prover system. I find it very intriguing to have proofs of mathematical statements checked by something as pendantic and comprehensive as a machine. Mathematicians always claim that their statements are true in all eternity, but the proofs are just checked by error-prone human beings. Especially with complex, large proofs that are only read by a handful of people, I doubt that these are always fully correct. Note that this does not imply that I doubt that the results are correct. They probably are. But a bit of doubt remains. A computer-checked proof, in contrast, can not accidentially omit corner cases, leave out seemingly “trivial” assumtions of used theorems or be misled by slightly differing definition from different sources.

I was hoping to check at least parts of my diploma thesis using Isabelle, but it turns out that the standard algebra library shipped with Isabelle is not complete enough. Even free groups were missing. This was motivation enough to try to formalize them and prove the universal property and some isomorphisms (The free group over the empty set is the unit group, the free group over one generator is the additive group of integers and free groups over sets of same cardinality are isomophic). I submitted the resulting theory to the Archive of Formal Proofs and it was accepted. You can view the complete document or the document without proofs.

I did not formalize the fact that isomorphic free groups have bases of same cardinality. As far as I know there is no simple argument that works directly with free groups. The proofs I have seen pass to the abelianization of the free group, i.e. the free module over ℤ and apply the well known proof from the analogous statement about vector spaces. But if someone knows an elementary proof of this fact, I’d like to hear about it.

nagstamon forklet necessary

mail@joachim-breitner.de (nomeata) — Thu, 24 Jun 2010 19:46:40 +0200

A while ago, I discovered nagstamon, a very useful piece of software by Henri Wahl. This program sits in the notification area of your desktop and alerts you when your nagios-monitored services have problems. Using nagstamon allows me to keep my servers under close surveillance, and it also adds another channel besides e-mail alerts, which will be helpful in case my mail server has problems.

The wish

I am not a full time sysadmin, I only monitor very few hosts and the services rarely have problems. Therefore, I do not want nagstamon to constantly sit in the notification area but only use it when there is something, well, to notify me about. It turned out that nagstamon did not support this mode of operation, so I created a ticket and asked whether this feature could be added. The author raised two points, one being that then the user would not know when nagios crashed and the other being that you would not be able to configure nagstamon because you do not see it. He also indicated that he does not have the resources to work on it and asked if I could find the time.

The patch

Since I really liked nagstamon, but really want to keep my panel uncluttered, I found the time: I created a series of self-containing patches, adding an option for the feature, adding code to prevent more than one instance of nagstamon running in parallel and adding a "nagstamon --settings" flag that would signal the running instance to show the settings – similar to how mail-notification is been behaving. The author then raised the valid point that some people run more than one instances in parallel, with different configuration options. I then extended the patch to cater for that.

The rebuff

The author remained reserved, did not answer my last commend on the ticket and then, six weeks later, closed the bug without explanation and turned off the possibility to add comments. I can understand when people are reluctant to add contributed features to their code, I often feel the same way. But completely blocking more comments is not a nice way of communicating with possible contributors.

The fork(let)

So I’m left with no option but patching each released version with my changes and building my own package. As I have to do this work anyways, I’d like to share it. You can find my branch in my git repository. If you happen to want this feature as well and are using a Debian-based distribution, please let me know: I am building modified Debian packages anyways and can publish them as well. As I don’t want to maintain this fork of nagstamon I don’t plan to diverge any more from Henri’s code, so if you have other feature requests, please talk to him first.

bluetile in Debian

mail@joachim-breitner.de (nomeata) — Sat, 12 Jun 2010 20:44:10 +0200

I just packaged and uploaded Jan Vornberger’s window manager bluetile to Debian. This very nice piece of software brings the benefits of a tiling window manager to users who prefer to use the mouse and who don’t want to learn a new programing language to configure their window manager. Bluetile uses the xmonad libraries and extends them with an easy to use and discoverable user interface.

Making dictionary passing explicit in Haskell

mail@joachim-breitner.de (nomeata) — Fri, 23 Apr 2010 14:56:55 +0200

Haskell provides type classes to support polymorphism. A type class defines a few methods, which can then be implemented for a concrete type in the type class instance. This is a powerful system, but it also has it drawbacks. Most notably, each type can have at most one implementation of the type class. But sometimes you need to use a different implementation.

If, for example, you used the Binary class to store data on disk. Now you changed your data type and the binary instance, and you can not read the old data any more. One solution is to re-name your type using “newtype” and implement another type instance for that. Often, this is enough. But still, instances are not first-class-citizens. You can not pass them around or modify them, as you can pass around and modify data and functions.

Under the hood of the compiler, things look different. The ghc puts the methods of the instance in a dictionary and passes that implicitly to any functions having a (Class a) constraint. (Other implementations exist though) If one could make that behavior explicit, one could easily modify the instance before passing it to the function. But this is unfortunately not possible.

But it is possible to pass an explicit dictionary along the data. I use the Monoid class as an example, and define a representation of the dictionary to-be-passed, as well as the dictionary of the default instance:

data MonoidDict a = MonoidDict
  { ed_mempty :: a
  , ed_mappend :: a -> a -> a
  }

monoidDict :: Monoid a => MonoidDict a
monoidDict = MonoidDict mempty mappend

(For conciseness, I ignore the mconcat method.) My first idea was to pass this instance along with data: (MonoidDict a, a). But this would not work because there are methods, such as mempty, who need the dictionary without getting passed a value to use. Therefore, I need to put the dictionary both in the covariant and the contravariant position:

newtype WithMonoidDict a = WithMonoidDict (MonoidDict a -> (MonoidDict a, a))

We need functions to clamp a dictionary to a value, and to extract it again:

wrapWithCustomMonoidDict :: MonoidDict a -> a -> WithMonoidDict a
wrapWithCustomMonoidDict dict val = WithMonoidDict $ const (dict, val)

extractFromCustomMonoidDict :: MonoidDict a -> WithMonoidDict a -> a
extractFromCustomMonoidDict dict (WithMonoidDict f) = snd (f dict)

Note that both expect the dictionary, so that it can be fed into WithMonoidDict from “both sides”. For convenience, we can define variants that use the standard instance:

wrapWithMonoidDict :: Monoid a => a -> WithMonoidDict a
wrapWithMonoidDict = wrapWithCustomMonoidDict monoidDict

extractFromMonoidDict :: Monoid a => WithMonoidDict a -> a
extractFromMonoidDict = extractFromCustomMonoidDict monoidDict

We want to be able to pass the wrapped values as any other value with a Monoid instance, so we need to declare that:

instance Monoid (WithMonoidDict a) where
    mempty = WithMonoidDict (\d -> (d, ed_mempty d))
    mappend (WithMonoidDict f1) (WithMonoidDict f2) = WithMonoidDict $ \d ->
        let (d1,v1) = f1 d
            (d2,v2) = f2 d
        in  (d1, ed_mappend d1 v1 v2)

Note that mappend has the choice between three dictionaries This is not a good sign, but let’s hope that they are all the same.

Does it work? Let’s see:

listInstance :: MonoidDict [a]
listInstance = monoidDict

reverseInstance :: MonoidDict [a]
reverseInstance = monoidDict { ed_mappend = \l1 l2 -> l2 ++ l1 }

examples = do
    let l1 = [1,2,3]
    let l2 = [4,5,6]
    putStrLn $ "Example lists: " ++ show l1 ++ " " ++ show l2
    putStrLn $ "l1 ++ l2: " ++ show (l1 ++ l2) 
    putStrLn $ "l1 `mappend` l2: " ++ show (l1 `mappend` l2) 
    putStrLn $ "Wrapped with default instance:"
    putStrLn $ "l1 `mappend` l2: " ++ show (
        extractFromMonoidDict $ wrapWithMonoidDict l1 `mappend` wrapWithMonoidDict l2)
    putStrLn $ "Same with reversed monoid instance:"
    putStrLn $ "l1 `mappend` l2: " ++ show (
        extractFromCustomMonoidDict reverseInstance $
            wrapWithCustomMonoidDict reverseInstance l1 `mappend`
            wrapWithCustomMonoidDict reverseInstance l2)

Running examples gives this output:

Example lists: [1,2,3] [4,5,6]
l1 ++ l2: [1,2,3,4,5,6]
l1 `mappend` l2: [1,2,3,4,5,6]
Wrapped with default instance:
l1 `mappend` l2: [1,2,3,4,5,6]
Same with reversed monoid instance:
l1 `mappend` l2: [4,5,6,1,2,3]

Indeed it works.

Unfortunately, this approach is not sufficient for all cases. It is perfectly valid to have a function with signature (Monoid a => Maybe a -> Maybe a), whose behavior depends on the instance of a, even when being passed Nothing and returning Nothing. Such a function would have a problem here, because the dictionary would not be passed to the function.

I wonder if it would be possible to extend the Haskell language somehow to be able to properly pass an alternative dictionary to such functions. But given that not all compilers use dictionary passing, my hopes are low.

A mathematician’s status symbol

mail@joachim-breitner.de (nomeata) — Thu, 15 Apr 2010 12:16:30 +0200

While writing my diploma thesis, I often wished I had a blackboard in my room: A place to quickly scribble some ideas on, somewhere where I can easily erease and replace stuff. Also, somewhere where you can literally (and not just figuratively) step back from and look at from the distance.

For my 25^th birthday, my parents organized a discharged part of a blackboard from my old scool, and yesterday, we installed it in my room in Karlsruhe:

Although my diploma thesis is finished, I’m sure it will be a nice and useful piece of furniture, and if only to draw funny things on (as my girlfriend did on this picture).

zpub article in “Linux-Magazin”

mail@joachim-breitner.de (nomeata) — Thu, 15 Apr 2010 11:32:50 +0200

The 05/2010 issue of the German “Linux-Magazin“ contains an article of mine about DocBook, Subversion and zpub. I was quite surprised to find it there – I submitted it in January and did not receive any feedback. But of course it is a nice surprise to find out it was accepted!

The article, the zpub website and zpub itself is only available in German so far, but there is an English blog post describing zpub.

Ein Kulanz-Geschenk...

mail@joachim-breitner.de (nomeata) — Wed, 07 Apr 2010 00:15:55 +0200

Ich bin seit einigen Jahren Abonnent der Wochenzeitung „Die Zeit“. Da gerade wieder die Abo-Verlängerung anstand, habe ich mich auf zeitabo.de umgesehen und ein paar nette Geschenke für Neu-Abonnenten entdeckt. Insbesondere den USB-Stick aus Echtholz fand ich ästhetisch interessant (auch wenn 4GB nicht mehr der Stand der Technik bei USB-Sticks ist).

Ich schrieb also eine e-Mail an die Abo-Verwaltung (was wegen der Studienbescheinigung eh nötig war) und fragte, ob man auch als Weiter-Abonnent an ein solches Geschenk kommen könne. Dies wurde erst verneint, dann aber in einer zweiten e-Mail „aus Kulanz“ ermöglicht und ich solle doch meinen Geschenkwunsch äußern. Ich antwortete: „Vielen Dank für das Entgegenkommen. Ich würde mich freuen, den Holz-USB-Stick zu erhalten.“

Heute lag ein Päkchen vor meiner Tür mit einem USB-Stick. Allerdings musste ich sehr zu meiner Enttäuschung feststellen, dass es keinen edlen Holz-USB-Stick, sondern einen nicht besonders schicken Plastik-USB-Stick enthielt:

Dazu auch noch, wie man auf dem Bild sieht, ein mit 256MB fast unbrauchbar Kleiner. Da es ja nur ein „Kulanz-Geschenk“ ist, kann ich micht eigentlich nicht beschweren. Aber irgenwie habe ich schon den Eindruck, dass das kein Zufall ist... vorlaute Weiter-Abonnenten sind wohl nicht erwünscht.

libnss-gw-name: A stable name for your gateway

mail@joachim-breitner.de (nomeata) — Thu, 18 Mar 2010 21:40:21 +0100

I often find myself running /sbin/route to get the IP address of the current gateway, especially when using a wireless LAN while traveling. For example, if the “Internet does not work” I usually ping the local gateway to see where the connectivity problem lies. I also need the IP if I want to access the routers configuration web interface. This is somewhat tedious, so I wrote libnss-gw-name, and now:

$ sudo apt-get install libnss-gw-name
[...]
$ ping gateway.current
PING gateway.current (172.20.239.1) 56(84) bytes of data.
64 bytes from hhicalvin.stud.uni-karlsruhe.de (172.20.239.1): icmp_seq=1 ttl=64 time=2.16 ms
64 bytes from hhicalvin.stud.uni-karlsruhe.de (172.20.239.1): icmp_seq=2 ttl=64 time=1.48 ms
64 bytes from hhicalvin.stud.uni-karlsruhe.de (172.20.239.1): icmp_seq=3 ttl=64 time=2.73 ms
^C
--- gateway.current ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.482/2.129/2.739/0.513 ms

Once libnss-gw-name is installed, it hooks into the system’s Name Service Switch, which is, among other things, responsible for resolving hostnames to ip addresses. It will only react on the name “gateway.current”, checking the system’s routing table and returning the IP address of the current default gateway.

It’s a pretty simple and small tool, but it could well prove very handy to the power user. I uploaded libnss-gw-name to Debian sid, you can download the source code or access the git repository.

Update: Changed the name to gateway.localhost, as that is within a reserved top-level-domain.

Video of my CeBIT talk online

mail@joachim-breitner.de (nomeata) — Thu, 18 Mar 2010 17:18:32 +0100

As tolimar already said, videos of talks at the Linux New Media booth at CeBIT are online now, including mine about how to submit patches. It is in German, though.

kexec saved my day

mail@joachim-breitner.de (nomeata) — Tue, 16 Mar 2010 11:38:28 +0100

Yesterday evening, when returning from a two-day trip with no connectivity, I found my server to be broken. I still reacted on ping, but no service would respond. I tried to restart it using my hoster’s web interface, but it would not come back up.

I booted into the recovery system and checked the hard disk, but could not find any issues. File system checks went through without a hitch. But it would still not boot. Unfortunately, my hoster does not provide access to the system console, so I had no idea what was going wrong.

I never did anything with kexec, (a relatively new feature of the Linux kernel to act as a bootloader to load another system) and I was very positively surprised to find that it works out-of-the-box and flawlessly: I was able to load my system’s kernel and initrd from the recovery system and successfully booted it. I then ran lilo and rebooted right again, which now worked. I’m not sure if running lilo fixed it, or the clean shut-down, nor do I know what caused the problem in the first place, but kexec saved my day here.