cgwalters

We reject: kings, presidents and voting.
We believe in: rough consensus and running code.

From David D Clark.

So earlier I raved about how Firebug is made of awesome. One of those Free Software projects that just saves you massive amounts of time as a developer, and really raises the quality bar you expect from tools.

There's one other project I should have mentioned earlier in this category, which is Valgrind. I'm sure nearly all Planet Fedora/GNOME developer readers know if it (though I was surprised there were still people not using Firebug), but Valgrind almost makes you feel like if it was a bit smarter it could write the code itself, and you could kick back on the beach and watch it.

But besides praising Firebug and Valgrind, I wanted to link to ANTLR as also raising the Free Software quality bar. Now, probably a lot fewer programmers have the need for a parser generator than need to debug HTML/JS or C, but I have to say - if you need a parser, ANTLR is also made of awesome. It's not just that the parser generator is powerful, documented etc. What definitely takes ANTLR a notch up is that it actually comes with a custom IDE for writing and interactively debugging your grammar. All Free Software, and pretty sexy too (Though sadly the screenshots are on a Mac). I also started using another nice Terrence library, StringTemplate in a project recently, which is what motivated this blog post.

Are there any indispensible tools or libraries that have really impressed you lately?

I have to say, this app makes quite a jump from their old DOS-based UI. I would have no idea how to use it, but I'm sure it makes sense after you get training. Good to know one fewer company's custom app is not tied to MSDN and Windows anymore, moving on to Free Software. I wish I could make applications look at that good.

You know the feeling. You're happily sitting there, coding away at some graphics or other code in C+GObject, and then comes the point where you need to add a signal to your object. But your object doesn't have any signals yet, so you have to go find and copy the boilerplate. Ok, not a big deal, you say to yourself, it only takes a minute or two. Then you notice your signal takes multiple parameters, and thus you need to generate a marshaler.

At that point you go to add one to your foo-marshal.list file, only to realize you don't have one yet! Now you need to copy a chunk Automake boilerplate, and apparently if you want to do it really correctly it involves three times as much boilerplate involving stamp files and other monsters.

It's around this point you're probably thinking that finding medieval torture devices and using them on yourself on live television would be a less painful remunerative occupation than programming. But there is hope - the awesome jdahlin wrote a patch. The problem - it's not in GLib yet, and it's unclear when it will be.

So maybe you don't buy into the the pitch about polyglot applications yet, but the new GObject Introspection release, because it already depends on libffi, contains a slightly tweaked version of jdahlin's patch, known as gi_cclosure_marshal_generic. Depend on introspection, and never deal with marshalers and other torture devices again!

DBus 1.2.12 and 1.2.4.4permissive are out; this release has a good number of virtues, among them that it has significantly faster transmission of fixed array types; see the announcement for details.

Guido Günther is now maintaining, and has released a new version of krb5-auth-dialog. Thanks Guido!

From Here To There is a quick introductory tutorial for people interested in building multi-level applications using the GObject stack, and having your own application be split between C and your choice of runtime. This tutorial uses Clutter and JavaScript, as we're using for gnome-shell.

I've been working for a while now on two applications using the system (gnome-shell, and a personal project not yet ready for announcement), and while there is still a fair bit more introspection system work to do, so far I'm pretty happy with how it's worked out for both projects. My personal project is actually a mashup of C for some graphics and system stuff, Java (JGIR) for the core application logic, and Groovy for some scripting and extensions. It's quite polyglot; I'm practicing what I preach.

Freedesktop.org vendors, please get the new DBus 1.2.4.2permissive release into your development stream as soon as possible, so DBus service developers and testers can help cooperate on fixing the broken services. Thanks!

Note: On my Fedora 10 system I got a lot of spam due to NetworkManager talking to HAL without this patch - so you likely want to get that one in at the same time.

HotSSH 0.2.6 is released. It fixes a number of bugs, probably most important is one where we could repeatedly try (and fail) to make a new connection every 3 seconds. System administrators tend to get antsy about stuff in their log files, so if they were threatening to bodily harm etc., well, HotSSH 0.2.6 is the ssh client you were looking for!

...then I feel there must be a lot to learn from this painful bug. So far I've learned to always delay pushes into the Fedora queue, even for security updates. Anyways, apologies for the trouble; this is really an awful bug. We'll probably be turning up cases where developers had relied on the erroneous permission for some time.

While traveling post-GUADEC in Turkey, I ended up after dinner one night walking next to a man speaking English to his wife, and like most people in a foreign country I was interested to talk to someone of similar culture. It turned out though he was Iranian, now living in Dubai with his American wife. He was a very intelligent person, a businessman of some sort and we had a discussion about world affairs as we walked back to our respective hotels.

He obviously had a broad world view and experience, so it was a good conversation. Of course the presidental run came up, and when I mentioned I supported Obama, he shook his head and I remember he said to me confidently and with some cynicism, "John McCain will be your next president". I had felt so sure that America would warm to this eloquent, charismatic scholar of constitutional law, but at that moment I experienced some lingering doubt which stayed for a week at least.

Not that he's likely to be reading this blog, but I'd just like to say - told you so!

A rare nontechnical blog entry from me today; I have had some technical ones queued up in the back of my mind for a bit, but it was hard to spend today thinking about technology. Or, at least other than as a delivery mechanism for election results.

Before Obama decided to run for president, I read an article about possible presidential contenders which included him. The article linked to this YouTube video of Obama. I was struck by his charisma and, doing a bit more research, his extraordinary intelligence and hopeful message. This guy didn't get into an ivy league school via the old boy network like Bush did. I decided within that hour that if he did run, I would contribute to his campaign, the first time I'd ever really done anything approaching active participation in politics beyond voting.

I have a feeling that's true for a lot of Americans right now. Anyways, I planned to keep this blog short. I just had to write something though - it's an immensely inspiring day today.

Dear lazyweb, I recently found myself having a set of Unicode strings (in main memory) and the desire to search all of them for a particular prefix (not substring). For bonus points, let's say I want to do it in parallel (seems like an easily parallelizable problem, although at worst you can divide-and-conquer for a given number N of processors). The constraints are that the set of strings can change, but not very often usually. I don't want too large of a hit to compute tables and the like though.

The real question is - how do you find papers (and implementations thereof) for computer science problems like this? My first instinct was to get an ACM subscription, which I did. Their search turns out to be mediocre, and then I discovered Wikipedia has useful pages on these kinds of things. Wikipedia also has the winning feature of linking to quality free software implementations thereof.

On this particular problem, a lot of the literature is around substring matching which isn't what I want, at least not directly. Still looking.

Get ready - it's coming. At the GNOME Summit this weekend we're having a hackfest on introspection. As a programmer working on a GObject library, here's what you need to understand:

Using gtk-doc annotations

Just parsing the C code isn't enough - we sometimes need additional metadata. Here's an example:

/**
 * gtk_list_store_set_column_types: 
 * @store: a #GtkListStore
 * @n_columns:
 * @types: <array,length=n_columns>: List of types
 */
void
gtk_list_store_set_column_types (GtkListStore *list_store,
                                 gint          n_columns,
                                 GType        *types);

There are more examples here. This syntax hasn't really seen much bikeshedding yet, and to be honest it's grown rather organically. So things could change. But the goal is for the annotations to be concise, fit in with gtk-doc, and support a variety of attributes; we don't want to require a totally different IDL format.

Add your library to gir-repository and check it

The current thought on deploying introspection is to have two phases. In the first "unstable" phase, we will also have two separate modules: gobject-introspection and gir-repository. It's unclear yet whether the former will remain separate from glib or not in the long term, but expect it to exist separately for now. For gir-repository, the long term plan is to integrate everything into the relevant modules themselves, but for the next 6 months or so while we're finalizing bits, we will keep annotations and code to invoke the scanner inside this centralized module.

So for now, you can add your library (if it isn't there already) to gir-repository by filing a patch in Bugzilla (glib/introspection), or pastebin on irc.gimp.org:#introspection. After you have your library successfully parsed by the scanner, the first task is to inspect the generated .gir file. This XML file describes the parsed API. Try searching in particular for any APIs you think are tricky.

Try out your library in a binding

After inspecting the .gir file, you can try out your API from one of the bindings such as PyBank or JGIR, with more coming. We expect to do some PyBank hacking in particular at the summit, and hopefully get the Spidermonkey and Mono bindings further along. There is a lot to do for all the bindings, but I hope that introspection can be a strong focus point for collaboration between the bindings.

Writing bindable APIs

I've started a wiki page about how you can keep your C code easily bindable. More content appreciated! This page will likely be a bit fluid as we determine both what is supported in the introspection core, and how its concepts map to languages.

That's it for now - remember if you're in the Boston to come to the summit!

If you're interested in programming languages and you're in Cambridge, you should come to this Boston Lispers meeting. The author of Clojure, Rich Hickey, reportedly gave a great talk at the JVM languages summit which hopefully he'll reprise here!

At the plumber's conference right now, just finished watching Arjan and Auke's 5 second boot time talk. Very very impressive talk.

They drove a lot of changes into the entire stack, from how the kernel works to fixing HAL and X.org bugs, filesystem tweaks in order to get to 5 seconds. Of course they cut out some things too - since the netbook targets are mobile single user machines, they don't use GDM, I'm pretty sure they aren't starting CUPS, etc.

But this talk led me to only feel more strongly about something that I've been feeling increasingly more lately - that we have been far too complacent about accepting "Linux distribution" as a concept. This is the idea that you can take the Linux kernel and a variety of tarballs found on the Internet, compile them and put it in some variant of tar, and you have an operating system. It turns out it's not that simple.

To create a compelling system, first, you need to make decisions about how things work and how things fit together. The "Linux distribution" mindset is an excuse for having different vendors make different decisions and we end up sharing very little. We need people like Arjan's team, the Ubuntu team, the Fedora team, the OpenSuSE team to share far more than we do now. I think a lot of vendors ("distributions") tend to have an insular mindset where they think they are the center of the universe and why doesn't everyone just join their project anyways?

Five or six years ago I think there was more of a mindset that Linux and the Free Desktop was going to take over from Windows ("year of the Linux desktop" and all that) and each vendor thought they were going to take over the world and consequently were less interested in collaboration. Personally I don't think we're going to see that sudden takeover any time soon, but at the same time if we do things right there is a lot of opportunity.

It's easy to say distributions are bad of course - far harder to actually collaborate better. How do we do that? Here's an idea:

Share a core OS image built at freedesktop.org

We're not going to unify all the different packaging tools tomorrow. However, there's no reason that the core OS build tools and image (kernel, dbus, HAL, X.org) couldn't be shared. So "distributions" could keep their favorite variant of tar and wget (.deb and apt, rpm and zypper, rpm and yum) for everything outside of the image. In other words you could still do "apt-get install epiphany" or "yum install httpd" whatever other app on top of the image and that would still work in a normal "packaging" way, but the core OS image would be rsync'd outside of that.

Each vendor would still control all of the code - each component of the image could have a "vendor" branch where vendors can add kernel drivers and the like. Image update policy would still be controlled by vendors.

There of of course other things that it's completely stupid to fork like bug trackers and more importantly the bug reports. We need to do a much better job of sharing bugs in the right place - the upstream project.

In a sentence, take projects like freedesktop.org and gnome.org as more than just tarball sources and turn them into real collaboration points where things get tied together, sharing actual infrastructure. It wouldn't be easy but really, we need to stop thinking that it's acceptable to have 5+ major forks of the core OS infrastructure.

I'm back to using Emacs for editing C and Python code for now. There are various reasons for this, but mainly it comes down to the fact that I need an editor and not an IDE; creating projects just to edit a bit of code is too much overhead, at least in the IDEs I have tried. Now, one major problem Emacs has had forever is that people (including me) have implemented new modules for it that "unbreak" various parts, but the maintainer has been very conservative about changing anything.

For this reason, here's what I consider the "minimal unbreak Emacs .emacs" file:

;;; Minimal Unbreak Emacs
; Waste of display space if you can't click on it
(if (not window-system) (menu-bar-mode nil))
; Really.
(iswitchb-mode t)
; Assuming you didn't use Emacs from before when the concept of
; selecting a single region of text interactively was not supported.
(transient-mark-mode t)
; This one is actually fixed in Emacs 22; http://www.emacswiki.org/cgi-bin/wiki/CopyAndPaste
(setq x-select-enable-clipboard t)
; The default for "uniquifying" buffer names sucks
(require 'uniquify)
(setq uniquify-buffer-name-style 'post-forward-angle-brackets)
; Slightly more debatable
(global-set-key (kbd "C-x C-b") 'ibuffer)

(set-scroll-bar-mode 'right)

; Custom keybindings
(global-set-key (kbd "C-c i") 'imenu)

Technically I shouldn't be giving you a C-c prefixed function, but I don't know of another good place for it. Anyways, enjoy. Someday Emacs will come out of the box unbroken; it's happening, just very slowly.

Federico, self-signed certificates disproportionally affect programmers (particularly Free Software programmers) using random homebrew bugzilla instances. For a website with any kind of real user base, the hosting fees and salaries of the programmers/designers/etc. are going to dwarf (by orders of magnitude) the cost of a SSL certificate. Does it suck that you have to pay a fee to avoid a dialog? Sure. But rather than complain, some smart person needs to think of a way to make a distributed system that actually works with non-cryptographic expert people in some way; i.e. doesn't just ask you "Do you trust this hex number"?

I won't do this too often, but two things I saw today that were interesting:

• An Ars Techncia article on Larrabee, Intel's new graphics architecture. Kind of scary they say "micro-OS" instead of "firmware"...is my next graphics card going to be running Linux? Regardless, Ars Technica has a well-deserved place in my feed reader.


• This thread on the Clojure group by a guy who wrote a DSL and compiler targeting Excel spreadsheets, originally in Ruby and now in Clojure. This is one of those things that remind me how different the day to day life of other programmers is from mine.

The Old Way:

1. Developer in libfoo (a C library) works on solving a problem, decides to create foo_bar() function


2. Function foo_bar works, and is committed


3. A period of time later, a new shared library appears in OS update package


4. You install the package


5. You try to use function bar from your favorite $LANGUAGE bindings, but it isn't bound yet


6. You curse and go searching bugzilla to see what the status is


7. If upstream is active, some manual review is done to make sure the function is sanely callable from $LANGUAGE


8. If the function has some special requirements, sometimes metadata is manually added to hackish .defs files


9. A patch is eventually made and committed


10. Another period of time later, new bindings appear


11. You can call foo.bar()

Note that steps 7,8, and 9 have to be repeated for each binding, be that PyGTK, Gtk#, java-gnome, etc. Even worse they all have divergent .defs files.

The New Shiny Future Way:

1. Developer in libfoo (a C library) works on solving a problem, decides to create foo_bar() function


2. If it is not sanely bindable, author gets warning from g-ir-scanner, reworks it or adds metadata, commits


3. A period of time later, a new shared library appears in OS update package


4. You install the package


5. You can call g.foo, because your $LANGUAGE bindings use GObject-Introspection

The new way, obviously, is going to be significantly better. By actually including all the metadata necessary to interface with the system in the shared library (.so) itself, we eliminate a large lag time before improvements in the lower levels of the stack become usable to bindings. Even more important I think is that by integrating into the build process of the GObject-based C libraries, we can help ensure that authors don't add new interfaces that are difficult to bind.

Why am I talking about this? Well, I just reached a major milestone in a side project I've been working on. Here's a screenshot:

Instant GObject/WebKit bindings for the JVM

And not just WebKit of course - this is a large step forward for Java bindings for Gtk, Clutter, libnotify, libgypsy, various canvases like hippo canvas and in general the entire "Free desktop stack".

Of course because it's Java, it's also instant bindings for Jython, Rhino, JRuby, Groovy, etc. Just ran this program with Jython trunk:

from org.gnome.gir.dynamic.WebKit import *
from org.gnome.gir.dynamic.Gtk import *
from org.gnome.gir.gobject import *

GObjectGlobals.init()
GtkGlobals.initCheck(None, None)
win = Window(WindowType.TOPLEVEL);
sw = ScrolledWindow(None, None)
win.add(sw)
wv = WebView()
wv.open("http://www.gnome.org")
sw.add(wv)
win.setSizeRequest(640, 480)
win.showAll();
GtkGlobals.main();

I just tossed up a web page for the project here. Right now it's basically blocked on finishing gobject-introspection itself; finishing the .gir to .repo compiler. So I'll be looking at that. My personal goal was to get bindings for some canvas (probably hippo), and clutter, along with all of Gtk+.

Let's take a step back though. What I think is interesting about this is that it enables much more seamless multiple layer applications. I blogged about this before. At the bottom, you have unmanaged C (GObject) code. In the middle, you have a managed, statically typed core (Java/.NET). At the top, you have a managed, dynamic language (Groovy,Boo,Python,JavaScript). GObject-introspection dramatically lowers the pain for going between the bottom and middle layers. Using the JVM (or .NET) makes the middle and top layer transition much nicer, especially for a runtime-native language.

Realistically landing and finishing all of this is probably a year out at the earliest still. But it's fun to hack on!

Edit: By the way, HotSSH version 0.2.5 is out, and this one actually has a README that tells you how to install it! It fixes various bugs, and most interesting it has the favicon work.

Awesome. In the near future, I can record a .OGG file, toss it up on the web somewhere (say my personal S3 bucket), and link to it in my blog with <video>. Now I have another motivation to make sure my family is using Firefox. Of course really what we also need is a general video hosting site that uses <video> - does one exist yet?