Ubuntu on the Toshiba AC100

I recently replaced my beloved but aging EeePC 901 with a shiny new Toshiba AC100.
The nice thing is that it runs Android 2.1 which means a) I had a nice excuse to buy it (my Archos 5 has only 1.6 with no upgrade in sight... and of course I *need* a halfway current Android device for development ;-) ) and b) that most people think it sucks many of which sell theirs again on Ebay for real cheap.
The problem is that it runs Android 2.1 which really is not such a great choice as a netbook OS (and to make matters worse there is no Google market - and no hack to sideload it as far as I know). Thus, while I don't agree with most people regarding the degree to which Android sucks as a non-phone OS (I think it has its merits) I do think that the AC100 is a beautiful gadget that deserves a proper desktop linux installation. Fortunately a few adventurous souls have done most of the explorative work needed for that and put it online. Still, as usual there were a couple of problems on the way. Anyways, this is how I did it:

The main HOWTO sits on http://tosh-ac100.wetpaint.com/page/Ubuntu, however it is aimed at installing Ubuntu on an SD card and uses an older kernel. There is a newer kernel by phh available here, with accompanying instructions for installing onto the device's SSD.

installation

Everything worked fine up to the point where I wanted to boot the newly installed boot image which lead to this cryptic message:

Waiting for devices ... done
/bin/sh: can't access tty; job control turned off

and a shell prompt. First I thought the problem was that I had used the newest update of the boot image, but the generic phh image had the same problem. It took a bit of fiddling and research but in the end a post in this thread lead me in the right direction.
Disassembling the boot image into kernel and ramdisk (following this guide) I found out that there were two distinct problems producing the same symptom: the older boot image did not check for /sbin/init on the right partition (mmcblk0p6 in my case) while the newer one did not have the right kernel arguments to recognize the partitions in the first place (see here). After I had just reassembled the new boot image with the proper kernel args everything worked fine.

(some) fine tuning

Most things worked out of the box after the install completed (notably sound!), however I had neither wifi nor suspend (there was also a problem with the touchpad but that disappeared mysteriously after a while...).

wifi
Following the discussion in this thread I put together a simple script which gets called from rc.local:

#!/bin/bash

/etc/init.d/network-manager stop
/etc/init.d/networking stop
echo 1 > /proc/test_program/wifi3g
/etc/init.d/networking restart
/etc/init.d/network-manager restart

suspend
The instructions in the HOWTO work like a charm except that lid-switch-daemon doesn't take an option '-d' - just '-s' is fine.

backlight
The instructions in the HOWTO are a bit vague. In fact the light_05 command takes an rgb value as argument with intensities between 0 and 255. This script will take a single value (0-255) and call light_05 with the right argument:

#!/bin/bash
light_05 $(( $1 + $1 * 256 + $1 * 256 * 256 ))

what doesn't work (yet)

• shut down - doesn't seem to do anything (see below)
• CPU scaling - as far as I know
• dim screen on idle - should be doable

update [10/02/11]:

• I found two new places with documentation - the kernel tree on gitorious actually has a wiki; and there's a new wiki being set up on linad. Now they all just need to merge their info...
• Shut down works (see comments section)
• I added a hook for pm-utils to dim the backlight on battery. Just put this:
  

  #!/bin/bash
  
  if [ $1 = "true" ] ; then
  

  /usr/sbin/dim_ac100 100

  else
  

  /usr/sbin/dim_ac100 255

  fi

  in /etc/pm/power.d/00backlight (assumes /usr/sbin/dim_ac100 is the backlight script mentioned above)
  
  

An acquired taste

These days retro-minimalism in desktop computing has become quite fashionable. People play old console classics, use minimalistic tiling window managers and many people go back to using text editors that their parents already grew up with.
I have always found this trend a bit weird, especially with respect to text editors. Why on earth should I throw 20 years of CUA, a mousable gui and the convenience of not having to constantly switch between different modes overboard and start to use vi?
I myself learned to program with QuickBasic. Although Windows (I think some version 2.0 or 3.0) *was* running on my computer I did not use it since it was notoriously unstable and stole away precious RAM from my individual-based simulations (yes, this is what I did when I was 16...). Also the DOS IDE QuickBasic came with was actually quite nice. The built-in editor implemented a standard CUA interface, therefore that is what I became used to.
I was accordingly shocked when I started university and had to learn vi. Horrible non-intuitive key bindings, a modal interface, no menus - I was disgusted. Out of necessity I learned enough vi to get along (we weren't even allowed to use X terminals for the first two years), but I never liked it. At home I continued to use QuickBasic and QuickC, or, when I started to switch to DJGPP (it allowed programming for protected mode which meant I could use the whole *8MB* of RAM without jumping through hoops) I just went with the simple DOS edit command and later with rhide.
Later, after switching to Linux I luckily stumbled upon NEdit. NEdit actually made me happy - it was fast, featureful and extremely configurable and scriptable. Unfortunately - being based on Motif - NEdit was also firmly rooted in the ancient past of Unix UI technology. Since there also was no sign of ongoing development I finally with great regret abandoned it (together with WindowMaker and my iBook) a couple of years ago.
Since then I have tried all CUA editors and IDEs I could get a hold on but none of them really satisfied me. Having replaced my iBook with a MacBook I decided to try OS X for a while, which didn't make the text editor situation any easier. I didn't manage to warm up to XCode and Smultron although nice felt a bit too locked in (just in case I wanted to ditch OS X again) therefore I stuck for a while with jEdit. After my annoyance with OS X and my MacBook had grown strong enough to go back to Linux (on a nice Lenovo R61) I decided to give kate and kwrite a try, which I have used since then.
A couple of weeks ago after having encountered another weird annoying bug in kwrite and an extensive round of checking on all linux text editors I know of I finally had enough. I decided to give vi a second chance.
And to my own utter surprise this time around I actually liked it. Vim really excels at configurability, speed, syntax highlighting and multi-window editing (due to its bugginess a constant pain in kate). On the other hand I didn't find it difficult at all to memorize the odd key combinations (though having a cheat sheet pasted to the wall doesn't hurt) and somehow they just didn't feel nearly as unintuitive as during previous encounters. Even the constant switching between insert and command mode which has always been my biggest issue with vi hasn't started to annoy me yet.
I could imagine that it's an age thing. It took me nearly 30 years to start to like olives, capers and anchovy and now I love them (visiting Sicily helped...).
Maybe vi really is an acquired taste and I just needed to pass the 35 to finally learn to appreciate it.


P.S.: While ditching kate/kwrite I also changed (back) to E17 and modified all color schemes to bright on dark. Could be that I'm just following fashion after all...

D is finally (nearly) there

Last week, in order to avoid working on more important things I started a small project I had wanted to do since quite a while. As mentioned before I am really not happy with C++ as a language to write simulations in. Unfortunately according to the Great Language Shootout all the nice languages are way too slow to be seriously considered.
However - as aptly explained on the shootout page - comparing the speed of programming languages and even the speed of implementations of programming languages is not very meaningful. Results will vary widely dependent on application area.
Therefore the only reasonable thing to do is to implement a benchmark which is representative of the kind of program one is interested in in all candidate languages. Which is exactly what I have started to do.
I have defined a reference model and implemented it in (very very basic) C++ and (C++ish) D (and CUDA which strictly speaking doesn't belong here. More on that in a later blog post - hopefully. I'm really bad at actually writing posts that I have announced before...).
To my great delight it turned out that the D program ran only about 10% longer than my basic C++ version. In principle this is clearly a small enough loss in speed to be compensated for by the nice improvements D offers over C++. I was really disappointed though (you can see how much I would like to abandon C++) when I found out a bit later that PGO (profile guided optimization) gives my C++ program another 20% boost. Add to that the fact that 64-bit D is still some way off and that value types are second class citizens in D and I'm not really sure whether I will do my next project in D or not. In any case it was nice to see the progress they have made. I am optimistic that my C++-days are numbered...
As for the shootout - I will try to find the time to add "proper" implementations in D and C++ (which hopefully will not perform much differently). I hope I will also be able to cover other interesting or up-and-coming languages, such as OCaml or Bitc. If anyone of my three readers feels willing and able to help out - just head over to the project page, read the model definition and hack away. I will be happy to post code/results.

Expensive data

As a theoretical biologist a lot of my research involves burning a *lot* of CPU time on computer simulations of evolving animal populations.

Usually I run a program for thousands of generations, each of which consists of hundreds to thousands of time steps during each of which hundreds of individuals interact with each other and their environment. This has to be replicated a dozen or so times with different seeds of the random number generator. The whole thing has then to be repeated for each combination of parameters I'm interested in.

To give an idea of the scale: Running 10k generations (which has been argued to be far too little) of the simulation I am currently working on on a typical node of my university's cluster (newish multi-core Opterons) takes about 3-5 hours. One standard sweep of the parameter space has 64 parameter combinations (which leaves out so many fascinating possibilities that it hurts) times 10 replicates each, thus 640 runs (each of those sets produces 4-5 GB of data, by the way).
In a typical project I tend to rewrite and change the simulation program many times, first of all to find bugs but then also as part of an iterative process where I create the program and run it, look at the results, think about them, adjust my opinion about the model/question/approach, change the program, run it, etc.
For the latest incarnation of my current project (the 4th or 5th major one) I have now done 25 of the above mentioned parameter sets. That means for just one part of the project I have already used more than 7 CPU-years and produced more than 100GB of data. And that is by far not going to be the end of it...

Hiking in movies

One of the things I love about The Hobbit and The Lord of the Rings is that while reading those books you can really *feel* how it is to travel. In particular you can feel how it is to travel by foot.
I have done some hiking myself and at least for me there is a special magic to exploring a landscape on your own two feet that is not evoked by any other form of travel. I'm not sure what it is - it includes moments like reaching the highest point of a pass and finally being able to see the valley on the other side, or looking back after an hour of walking and realizing the distance you made - but there is much more to it and I can't easily describe it in a few sentences.
Whatever it is, I think Tolkien managed to transport it quite well in his novels (actually it is said that Tolkien himself loved to make long walks through the English countryside). In Peter Jackson's movies on the other hand it is missing almost entirely. There is certainly no lack of trying - we see great panoramas of landscapes, helicopter flights through snow-covered mountains; we follow the heroes as they walk through brush, moorland, grassland, forest and all other kinds of temperate biome you could imagine. We see them walking, stumbling and climbing.
But still - at least for me this always looks like actors dropped in a scenic landscape (which it of course is) - that is, slightly soulless and artificial. I don't even think the movies are bad in general, I think given the economical constraints (mass appeal required to get back the gigantic investment) they are even close to a best-case scenario. But in this particular aspect they fail almost completely.
But now comes the funny thing. The other day I checked some video clips we made when we had been hiking in the Peak District with the kids the last time. Nothing special really - greyish weather, us, sheep, some hills. But there it was - even in these short amateurish clips, made with a cheap flip camera, I found the "spirit of hiking" was clearly recognizable!
Now, the really interesting thing to ask is of course, why is that so? The non-interesting answer would be that my personal experience (having been on that hike myself) colours my perception and that for everybody else the videos would be just as soulless as the mentioned movie scenes. This is perfectly possible of course, however I find it much more interesting to imagine that there is more to it than just that.
Here are a number of factors that I think might be responsible:

perfection

Pictures in Hollywood movies are perfect and glossy, my clips aren't. Perfection creates distance and a feeling of artificialness.

objective camera

In the movies we see the heroes stoically marching through New Zealand's Best Of. Since the heroes as well as said Best Of (and especially the combination of the two) cost a lot of money and supposedly are what the viewers want to see, the camera is quite busy putting them in the best light. Most of the time therefore we either see the landscape at a wide angle with the group of people somewhere in the middle or we see the latter from the front or the side passing the camera's position. That means we get an uninvolved (helicopter-equipped) spectator's view of what's going on, which is how we would experience a landscape when sitting in a car or a train (or a heli), but *not* how we experience it when walking through it.

speed

When a movie wants to show us that a car for instance is moving very fast, we usually see it approaching (very fast) to the camera's position (preferably at a turn), passing it and then moving away. The camera usually stays fixed at its position and only turns to keep the car in focus.
When filming people walking in contrast directors seem to think that walking per se is a far too boring activity to keep the viewer's attention. Therefore the camera compensates by moving around the person. Approaching it from the back, passing it, approaching it from the front, circling it, etc. This all makes for a busy picture however it does *not* convey the feeling of slowness that is defining for walking. I think essentially walking is usually shown as an activity while in reality it is more of a state.

So, after all this - what do we see in my clip? We see a slow pan of the (greyish, wet, sheep-dotted) landscape. Then a group of people overtakes the camera and slowly (in walking speed that is) moves downwards a small hill, climbs a fence gate and disappears down a path. As I said, really nothing special and more than anything proof of my utter lack of cinematographic ability. Still (for me) it perfectly transports the slowness and smallness of people moving through a landscape by foot.

Eta, Part II - Syntax (part I)

Many people have pointed out that language designers tend to obsess over syntax far too much and that their time would be better spent thinking about the semantics of their languages. Some (usually those who are either more academically inclined or old lispers) go so far as claiming that syntax is ultimately irrelevant, since a) which syntax someone prefers is largely a matter of taste anyways and b) every syntax becomes 'natural' after sufficient exposure.
Well, this topic has been discussed thoroughly, and I will only add to it to the extent that I am going to justify my own design decisions on the matter.
On the most abstract level a program can be thought of as a nested structure of operations being applied to sub-units which again consist of operations being applied to sub-sub-units, and so forth. Within a compiler this structure is usually represented as a so-called AST (abstract syntax tree).
If we print out an AST in parenthesized polish notation we would essentially end up with Lisp's syntax. This very elegant idea has a couple of advantages - it is extremely simple, easy to parse and totally generic (note though that the oft-heralded homoiconicity of Lisps is a red herring in my opinion - in every language that I know of it would not be difficult to represent a program's AST in the language itself).
On the other hand - at least for me - this genericity makes programs more difficult to read, especially at a glance, since it lacks redundancy. In Lisp the only carriers of information about the structure of a program are the names of operations and the nesting structure. In most main-stream languages however syntax is used as an additional redundant channel of communication. This redundancy makes it much easier to quickly grasp the structure of a piece of source code.
Have a look at this bit of C for example:

 3 struct Point
 4     {
 5     float x, y;
 6     };
 7 
 8 float point_dist(Point p1, Point p2)
 9     {
10     float dx = p2.x-p1.x, dy = p2.y-p1.y;
11     
12     return sqrt(dx*dx + dy*dy);
13     }

We can see that the same basic functionality is provided by very different syntactical elements depending on the context. The separation of terms for example is done by whitespace (top-level), ',' (declarations) and ';' (statements). Grouping is done by '()' (arithmetics, actually not shown in this example), operator precedence (arithmetics) and '{}' (statements). The application of an operation to arguments is expressed either in infix notation (arithmetics), prefix with '()' (function call), plain prefix (flow control keywords) or implicitly (declarations).
Of course this mess is far removed from the theoretical purity of Lisp's S-expressions. However it allows us to very quickly distinguish between different kinds of operations and different kinds of lists of terms. Looking for a declaration - spot names separated by whitespace, looking for function calls - find name + '()', and so on.
Redundancy therefore clearly serves to support readability (or "glanceability"). Too much of it on the other hand will certainly have an opposite effect. The optimal syntax will consequently add just enough redundancy to improve readability. (side note: There is also useless redundancy - Pascal is a lot more redundant than C, however mostly due to the fact that it uses keywords instead of punctuation and longer keywords. In my opinion this reduces readability. A similar argument could be made for Java.)
To maximize the effect of syntax it is also important that there is as little ambiguity in the correspondence between syntactic elements and semantic structure as possible. A nice counter-example is provided by C++. By "overloading" old syntax it becomes a lot harder to read (quickly) than C.
In Eta I wanted the overall look to stay somewhere in the vicinity of a traditional curly-brace language. At the same time I wanted it to be as simple and regular as possible while defining an unambiguous relationship between syntactic elements and semantics. (side note: This sounds a lot more goal-oriented than it was. Actually it took me quite a while to find out that these were the goals I was aiming for.)
This post is already long enough however, therefore I will postpone the details of Eta's syntax to the next post. As a small teaser the example from above rewritten in Eta:

1 Point @ type : (x @ float, y @ float)
2 
3 point_dist(p1 @ Point, p2 @ Point) @ float :
4     {
5     dx @ float : p2.x-p1.x
6     dy @ float : p2.y-p1.y
7     
8     <- sqrt` dx*dx + dy*dy
9     }

Eta

In the last two weeks what began as a small attempt at writing a work-saving template system for individual-based simulations turned into my first sort-of somewhat working (but not so work-saving) compiler for Eta (or η).
Eta is a project I have been working on/thinking about since a couple of years already. It started off with my increasing dislike for all the syntactic and semantic warts of the bloated mess that is C++. At the time I was desperately looking for an alternative, however everything I found that promised enough performance was either immature or only slightly less warty and bloated (sidenote: I think D is a much better language than C++ and I really hope it catches on. Still, it's far from being a good language IMHO.).
So I did what everybody who should instead really, really work on his PhD thesis (and I'm *not* talking about a PhD in Computer Science) would do - I started thinking about how to design my own language.
As others have said language design tends to be more successful when it tries to scratch a personal itch than when it attempts to solve other people's problems. In my case I really wanted a language that would make it easier and more fun to implement the simulations I am working on. That means the language had to be

• fast. It *does* make a difference whether I have to wait 5 days or 8 days for a set of simulations to finish.
• strictly and statically typed. As I said before the major difficulty when writing simulations is that errors are often silent. Every bit of static guarantee the compiler can give helps.
• interoperable with C/C++. I'm not going to reimplement all the libraries I use.
• expressive. For reasons of efficiency dynamic operations are often out in simulations. Therefore similar redundant patterns start to show up at lots of different places. E.g. if I add a new trait to an individual it has to be read, set, initialized, mutated, written to the data file, read from a config file, etc. Some of it can be alleviated by some advanced template wizardry but in the end I sooner or later usually fall back to external code generation. My ideal language would have built-in compile-time macros to solve this problem.
• clear and unambiguous. One problem with C++ is that understanding what *exactly* a particular piece of code does can be non-trivial. Apart from syntactic idiosyncrasies things like silent shadowing of globals and implicit conversion rules make it necessary to be aware of a big amount of context to understand local semantics. This is especially a problem for simulations since (due to lack of external tests) code review plays an essential role in ensuring their correctness. A good language should therefore reduce the amount of context necessary to understand a piece of code as much as possible.

In addition I wanted a *nice* language. This is of course to a large degree a matter of taste, but for me an aesthetically pleasing language has to be concise - I hate the wordiness of Java or Pascal. Also I really like operators, in my opinion they greatly increase readability (I'm constantly tempted by unicode, but until now issues with inputting non-ascii characters have kept me from going that way. If only everybody had APL keyboards...). Furthermore I want a language to be as orthogonal as possible - everything should be the result of the combination of a few simple rules. I really dislike ad-hoc rules just to achieve some convenient effect or pointless duplication of functionality (why does D for example need templates, CTFE, macros"string mixins", traits and conditional compilation?).
Apart from these general principles I had a couple of specific technical ideas about mechanisms I wanted to include in the language. I will leave the details on that, on how I implemented Eta and on how the language looks like currently to the next post, however.