Home

Advertisement

Customize

"Linux distribution"

At the plumber's conference right now, just finished watching Arjan and Auke's 5 second boot time talk. Very very impressive talk.


They drove a lot of changes into the entire stack, from how the kernel works to fixing HAL and X.org bugs, filesystem tweaks in order to get to 5 seconds. Of course they cut out some things too - since the netbook targets are mobile single user machines, they don't use GDM, I'm pretty sure they aren't starting CUPS, etc.


But this talk led me to only feel more strongly about something that I've been feeling increasingly more lately - that we have been far too complacent about accepting "Linux distribution" as a concept. This is the idea that you can take the Linux kernel and a variety of tarballs found on the Internet, compile them and put it in some variant of tar, and you have an operating system. It turns out it's not that simple.


To create a compelling system, first, you need to make decisions about how things work and how things fit together. The "Linux distribution" mindset is an excuse for having different vendors make different decisions and we end up sharing very little. We need people like Arjan's team, the Ubuntu team, the Fedora team, the OpenSuSE team to share far more than we do now. I think a lot of vendors ("distributions") tend to have an insular mindset where they think they are the center of the universe and why doesn't everyone just join their project anyways?


Five or six years ago I think there was more of a mindset that Linux and the Free Desktop was going to take over from Windows ("year of the Linux desktop" and all that) and each vendor thought they were going to take over the world and consequently were less interested in collaboration. Personally I don't think we're going to see that sudden takeover any time soon, but at the same time if we do things right there is a lot of opportunity.


It's easy to say distributions are bad of course - far harder to actually collaborate better. How do we do that? Here's an idea:


Share a core OS image built at freedesktop.org


We're not going to unify all the different packaging tools tomorrow. However, there's no reason that the core OS build tools and image (kernel, dbus, HAL, X.org) couldn't be shared. So "distributions" could keep their favorite variant of tar and wget (.deb and apt, rpm and zypper, rpm and yum) for everything outside of the image. In other words you could still do "apt-get install epiphany" or "yum install httpd" whatever other app on top of the image and that would still work in a normal "packaging" way, but the core OS image would be rsync'd outside of that.


Each vendor would still control all of the code - each component of the image could have a "vendor" branch where vendors can add kernel drivers and the like. Image update policy would still be controlled by vendors.


There of of course other things that it's completely stupid to fork like bug trackers and more importantly the bug reports. We need to do a much better job of sharing bugs in the right place - the upstream project.


In a sentence, take projects like freedesktop.org and gnome.org as more than just tarball sources and turn them into real collaboration points where things get tied together, sharing actual infrastructure. It wouldn't be easy but really, we need to stop thinking that it's acceptable to have 5+ major forks of the core OS infrastructure.

Comments

Not...really

You've never worked on a distribution, have you?

It, er, kinda shows. :)

In a much more ideal world than the current one, your idea might work. Emphasis on the 'might'. In practice, differences between distributions exist and are likely to persist right down to the core level. You're not going to get Debian, Red Hat, Novell and Mandriva to agree on a common kernel, udev, dbus, hal or...pretty much anything else, any time soon.

It's also a major problem to try and maintain some parts of the system outside the distribution's standard packaging system.

We already have a common core project with TurboLinux, and there have been various projects in the past like United Linux, but there's just too vast a difference between philosophies, approaches and patch sets in the various distro worlds to just happily reconcile 'em overnight. The core bits of Fedora, SUSE, Debian / Ubuntu, and Mandriva really are a LOT different.

(Anonymous)

Re: Not...really

Er, missed a bit. Package formats are something that gets way overblown. The major difference between, say, Red Hat and Ubuntu is not RPM vs. dpkg. It barely matters, and can easily be abstracted away by something like PackageKit. It's just historical circumstance. The package management system is one of the least significant differences between any two given distros.

Re: Not...really

I've worked on Fedora for the past 5 years (with a ~1.5 year lapse), and for about four years before that I was a Debian developer. I wrote the build system that a majority of Debian desktop packages use and help maintain dbus. Yes, I've worked on a "distribution".

Are you at the Plumber's conference? There is a growing feeling of consensus here that more needs to be shared to advance.

Debian may be hard to change, but I am quite sure Red Hat would be open to more collaboration - I work there and would be able to push. I've talked to some Ubuntu developers they are interested.

Remember the goal is not to unify the entire world - just most of the core infrastructure (basically everything X.org and below).

Re: Not...really

Sorry, wasn't specific enough: I knew that, I just meant it seemed a bit naive as far as the core bits of a distro go.

Without Debian you've got nothing. You can't get Ubuntu without Debian because I really don't think in the end you would get them to branch that far from Debian.

This kind of thing crops up from time to time (as I mentioned) and when it's a vague idea being discussed between tech people it all sounds great and why wouldn't we do it!

Then it gets a bit further down the road, everyone realizes what it would actually involve, management sticks their noses in, and it just...never happens.

You might, with a really big concerted effort, get a few more similar distros to share a core system. But I'll eat my hat if you get Debian and Red Hat to do it, and as long as that's the case, you've basically got nothing, because there'll be one absolutely key distribution outside the tent - in which case there's not a great deal of point having the tent at all.

Sorry to be cynical, but I just don't see this working where all previous attempts at the same thing just failed.

Re: Not...really

"You can't get Ubuntu without Debian because I really don't think in the end you would get them to branch that far from Debian."

I wouldn't be quite so sure about that...

Re: Not...really

Well, the best of luck with that - I'm off to inspect the Canadian Aeronautic Porcine Squadron. =) Will await progress with interest...

Re: Not...really

"You've never worked on a distribution, have you? It, er, kinda shows. :)"

Colin was a Debian Developer for a very long time (not aware of his current status) in addition to his long time work on Fedora.

Your strawman, needs more straw

How is what you want... different that what Shuttleworth has talked about previously in terms of 'syncing.'

Obviously I'm automatically against whatever it is Shuttleworth says, but if you say it I might listen.

Okay now that I've gotten the unfortunate hypocritical soundbite out of the way. Seriously, is what you are thinking about materially different than what Shuttleworth has been trying to articulate a need for?

It actually sounds pretty daunting in terms of re-engineering the infrastructure given the current tools we have. If we could do what you want right away, is the Fedora infrastructure adaptable enough for freedesktop to use as a basis to engineer the infrastructure you are looking for? Is the OpenSuse infrastructure? is debian's?

Do we need to wait for Launchpad to be open so freedesktop can run a Launchpad instance.

-jef

Re: Your strawman, needs more straw

I think basically we might be able to get away without all of the Fedora/OpenSuSE/Launchpad infrastructure if the defined target is just kernel+dbus+X.org+HAL. So even things like the X drivers would still be outside of the tree and maintained as RPMs/.debs.

What Shuttleworth is talking about is synchronized *schedules* which I don't really think is all that interesting personally. We need to share more code and infrastructure.

Re: Your strawman, needs more straw

Hmmm. So basically this is essentially a rolling image of the "plumbing" to reuse a catchy phrase. And the vendors essentially take snapshots off on whatever schedules they need?

If this core image lives outside of the packaging system, how do vendors keep the components of this image updated for end users who are consuming distributions based on this shared image? I mean at some point these plumbing bits have to find their way into some sort of packages right? Or how else do system get updated X or kernel or whatever?

-jef

Re: Your strawman, needs more straw

Vendors could put the image into a binary RPM/.deb.

Re: Your strawman, needs more straw

Ah still packaging envoled as we know it. I think it will end up being multiple binary packages... but I'm picking nits.

So basically its a rolling state-of-the-art common baseline that each vendors development feeds.

I'm imagining this like the current kernel development model. Mainline branch, subsystem maintainers yadda yadda. Where now each of the plumbing components are treated analogously like kernel subsystems for the kernel development model?

Does that analog hold?

-jef

Re: Your strawman, needs more straw

Aside from no-one having previously asked us (which isn't that harrowing a concern as I don't expect this to last longer than a few seconds), why would you be trying to force Launchpad on us?

How would that be even remotely helpful? What problem are you actually trying to solve with these comments, besides the problem of what to do until lunchtime?

Re: Your strawman, needs more straw

Are you replying to me or jspaleta? I think jspaleta but just in case it's me - well you're right I didn't ask about putting more stuff on freedesktop.org, but it does seem like the right place. Almost all the other plumbing from X to DBus to HAL lives there now.

I know I'm a bit of a dreamer in some ways - but there are incremental "baby steps" we can take that will lead towards this, like Dave Jones' talk about having the initrd code be driven from the kernel tarball.

One thing that really is going to become an issue for vendors is the fact that we're tying X and the kernel much more closely together nowadays. All of a sudden those two tarballs (and by extension the things that live between them like DBus and HAL) aren't things you can pull from randomly.

Re: Your strawman, needs more straw

Um, they aren't? You should let the DRM guys know, they can stop going to extraordinary lengths to maintain kernel <-> userspace API.

Re: Your strawman, needs more straw

(Also, I was replying to Jef: If you click on 'parent' in my comment, you can see the comment I picked to reply to.)
You mean more like having a core and a bunch of ports, like the BSDs? Sounds like a good idea, but I don't know if it will work - there are many different BSDs after all.

/Mike
And there would still be different "distributions". There are a variety of things that are going to remain different - for example, Fedora will never ship proprietary drivers, Ubuntu will. That's a fundamental irreconcilable difference.

But the core "plumbing" is divergent for almost no reason.
Well I guess the easy way to do that is to pull xorg, dbus, hal (hal+1?) into the kernel - it's the one thing everyone agrees on already, right? :)

United Linux

The last time this was tried it resulted in the quick demise of all but one of the distributions who tried it. United Linux was a neat idea (and arguably the best distribution at the time) but it had the undesired effect of shutting down every company involved except SuSE. It was also the re-birthplace of SCO. United Linux was Caldera's last attempt at playing nice before going off the deep end.

I'm not saying that it's a bad idea but it would be hard to do right and the companies behind the various distributions would want to have a business case for it that showed how it wasn't going to be like last time. Or would be like last time in the case of SuSE ;-).

Advertisement

Customize