jcm's blog

Well, my first blog post on this subject generated some broad agreement (and a few flames), but I think it’s necessary to be a little more specific about the three main things I want from from computers, from Linux, from whatever, as opposed to just from Fedora or any one Linux distribution. Here we go, and remember, this applies always to a “stable” release, not to the latest half baked bits that are under work, and it’s equally applicable to any Operating System you care to mention…

1). When I turn the computer on, it works. I don’t give a flying fsck what awesome cool kernel feature, bootloader change, init process, or other “it’ll be great and amazing ten years from now” feature you’ve developed if the computer doesn’t boot properly and I can’t totally ignore what you did underneath. Ten years ago, I used to buy into the notion that it was all great breaking stuff because it was saving humanity, but now I am older and wiser, I realize it is just plain shoddy to release something that causes obvious regressions and stops a system from booting. I am not targeting anyone in particular here, or any particular incident, just saying there is never an excuse not to boot. Windows can boot, Mac OS X can boot, other Operating Systems can boot. So if any Linux distribution or other Operating System cannot boot as a general matter of course, there is absolutely not an excuse. Users don’t care if it’s better but doesn’t boot cleanly, they would rather you left it alone. Yes, that’s an impossibly high standard, and a reason often to choose not to re-work stuff without assuming that level of paranoid commitment to quality.

2). When there are updates, I’m told about them and given a chance to reboot or delay the upgrade. And the upgrade always works smoothly with absolutely no loss in functionality. In fact, I can feel quite confident doing an update as a matter of best practice, even ten minutes before a big presentation, or even during the presentation if I want to. Windows, and Mac OS X generally do not blow up your computer when you do an upgrade (I can’t remember the last time I saw an OSX kernel panic, or a less than perfectly pristine update experience). You say “sure, ok” or you delay it, or you do whatever, but the computer works fine ten minutes hence either way. Anything less than a perfect update experience is unacceptable and must not be tolerated. This means that often the answer is “no” to taking things into an update, and the update is small and manageable, especially when you have very limited QE resources to ensure everything is working after the update is completed. This works well in the “Enterprise” space, but it should work well in every “space” you are in, it’s just common sense.

3). When there is some new feature, there is one new feature that works well. Not ten doing the same thing that don’t. This is much harder to do in a general Open Source sense, but it’s easy from a distribution point of view. Pick a small subset of things you are willing to do well and do them, or don’t do them at all. Both Microsoft and Mac OS X have confined what they will offer as part of the basic Operating System platform, and then undertaken to ensure that is the best possible experience they can offer. It isn’t always the best possible experience, and there are problems, but it is actually a good philosophy.

Jon.

Continuing the series of blog postings that reference one another, a few comments on “updates and people”, following mizmo’s latest blog entry on the topic.

First, let me thank Mo again for doing an excellent job summarizing some of the problems, and describing the userbase with typically excellent graphics. There are plenty of Pamalas, Connies, and Nancys out there on the mailing lists, but not enough Carolines to speak up for overall cohesion, and product quality that we can be proud of. So, I am going to continue to stand up for the position of “Caroline Casual-User”, the kind of person that I personally feel is actively being driven away through the hostile actions of a few within the community toward update sanity (learning to say “no” to pushing updates) and overall distribution cohesion (ensuring packages actually do work well together). Caroline is the kind of person who is accurately described in the current User base documents on the Fedora Project wiki. She is also represented in a lot of the cosmetic GUIness we see in distributions like Fedora – graphical package updates and configuration, removal of advanced options, the general direction of the GNOME desktop, and so forth.

We have all of these nice “user friendly” features going into the distribution, and an effort at gating releases until they pass certain quality controls. And when you read the “User base” documentation, it becomes clear that the intent was to make the distribution accessible to a wider user base than just hard core package developers and rawhide consumers. Yet there are those who continue to push the notion that Fedora be some kind of rolling update, never really having strong releases (more like “snapshots” that happened to pass some quality control at the time), and that it continue to lose market share to other distributions. Apparently, that’s fine because we don’t care about user numbers so long as good things happen. But users like Caroline lose out in the process. Now, maybe they don’t matter one bit, maybe the mailing list threads are right. But if they are right then the documentation, the intended user base, all of that stuff needs a heavy re-think and update. Because you don’t get to say you are targeting something and then really not do that. You don’t get to pretend that all that shiny software has a purpose because it is going to empower users and then laugh in their face by breaking it randomly in an “update”.

And if those things are right, why does the Desktop gloss really matter anyway? Hard-core developers don’t need simplified UIs, they don’t need graphical configuration, they may not even need to be running a Desktop at all. They might aswell run emacs full screen on a large framebuffer and be done with it. Oh no! I hear the cries now! What a preposterous idea! Well, if it’s so preposterous not to care about a Desktop environment, then maybe it’s not quite so preposterous to ask ourselves who wants the kinds of things being churned out in Fedora these days. I’ll give you a hint, they’re the kinds of things Caroline Casual-User really wants to see. She loves consuming your end product. She loves the feel of Macs, and likes the trend to have more Mac-like features in Linux systems. She doesn’t know about systemd, but she digs the idea of things sorta just working where they’re needed. Heck, she even likes it when she plugs her phone in and the icon on the Desktop looks just like her phone. Why does a hardened developer even need any of that stuff? I mean really, who the heck hard core developer cares what icon even shows up, or even if stuff gets automounted? The real, not-on-Fedora-devel but interested userbase that used Red Hat Linux, that buy the magazines, that care, they are the ones who want that stuff. You know that because you make the software to target them, and now is the time to admit that.

What I want to see is a fundamental shift toward having a stable “Platform”. I’ll take credit for the term “Platform” in Mo’s post since I was the first to use it. Essentially, what I mean is that we split the distribution into a core platform needed to boot, and provide a basic environment for higher level stuff. This is what every other general purpose Operating System under the sun does. They don’t shove out random updates that might break the fundamentals, and it means users can rely on certain core stuff just working. Now, this doesn’t mean returning to the days of Core/Extras. It doesn’t have to be internal vs. external packagers, etc. It can just be about defining a basic set of features that must be stable (boot, init, libraries, etc.), turning those into use-case driven definitions (rather than plain old package lists like “critical path” is today) and then enforcing them. And gating updates, and saying “no” judiciously to people who want to break that. And asking others to go play elsewhere if they don’t like distribution cohesion, and all of these things. Not because they are sexy, but because they are responsible.

Jon.

Fedora is one of these interesting efforts in community organization that somehow results in a finished product. There are many good things about Fedora as a place to see new technologies and to participate in Open Source. But there are also some bad things that need work. Not all of the technologies are instantly useful or necessary to all users, and they are often adopted quickly, whereas it would often be better (in my opinion) to pursue full integration over a (slightly) longer period of time. We don’t have a clear vision for who users are, or what they want, and collective Laissez-Faire organizing only works to a point, a point at which you need to have strong-willed steering of the ship you’re on to make sure it goes where it needs to get to. And the 6 month release cycles are far too frequent in my opinion if they will result in only a one year overall lifetime.

Specific things I want to see from Fedora:

1). Slow the updates. 800 updates after installation is bordering on insanity. I’d like to see updates only for security issues and bugs (but just those bugs, not wholesale rebases to “fix” a bug and introduce ten more), and I’d like the extreme opposite of what we’re seeing now, but I’ll take what I can get. This isn’t Enterprise vs. non-Enterprise. This is sanity vs. non-sanity. Rawhide is where “stuff happens”(TM). If you want stuff to happen in a “stable” release, take a look at this thing called rawhide. Then re-read this paragraph. If that doesn’t work, ask yourself if you are the best programmer who ever lived? If you really are, and are not going to break something – no matter how insignificantly annoying – and are going to test your non-essential update, have at it. But if my video-out breaks, if my printer stops, if my system doesn’t boot, if the behavior changes, if my proprietary flash plugin that worked just fine yesterday breaks (which does matter), if something isn’t exactly the same after I run an update, you caused a regression, and people are not going to be happy. I expect that only to happen when I choose to upgrade, and you have a choice to leave it the heck alone. Consider that choice as a good first choice and default to it, unless an update is essential.

2). Set a mandate. Either ask the users what you want from them (I have called again for a survey to be conducted), or tell them what that will be (also a valid option). But there needs to be a very clear direction here, and it must be spelled out very strongly in writing. Fedora can’t be all things to all people, that isn’t working, and time and again this is borne out on the mailing lists, and in various other anecdotal evidence. Depending upon who shouts the loudest will result in it either being an everyday Linux distribution for Linux users, or of niche value to those who like to ride the wave on the Fedora development list. Here’s where a series of very specific documents – we might call it a Declaration – comes into being and says “this is what we do”. Period. You don’t like it, fine, but this is the situation. Having something like that also allows it to be shaped and changed. And for users to express their opinions on what should be thrust at them. We’re on Fedora 14. I would have thought we could have gotten a document like that worked out by about, oh, now. How about before F-14 ships?

3). Establish cross-functional workgroups. There is no Desktop, there is no kernel. There is only this kind of use or that. Why don’t we have a “Plumbing” team that handles the low-level guts and meets regularly to sync up on them? We should have various groups that represent more than component IDs in Bugzilla, because the Linux system isn’t quaint any more. The current mechanism of development sees someone introduce some big feature or whatever, break random other stuff, then fix up the fallout, probably slowly and over a few releases (due to the number of people who were not involved early on). Instead, everything should be planned. There should be an actual idea of overall system design, overall architecture, what it is that will happen. Does everything have to have a command line equivalent? Make it a policy. And people are busy. Some are working on Fedora as a hobby, others have different commitments. If you depend solely on who is available “right now”, and who is dropping code today, then you will forever have this mess. Sometimes people are busy on other stuff and you’re going to have to wait a release for your shiny new feature to get integrated, and that’s fine.

4). Set specific goals. Don’t just let people post Feature page ideas randomly. Have a strong process (stronger than now – before anything is allowed to be built and shipped out the door), with an overall vision that those ideas fit into. And have a few people be in charge of overall consistency and vision decide how a feature fits in. If it doesn’t, punt it to the next release. This kind of thing happens to a very limited extent right now. It should be rigorous, it should be tough, getting stuff into the distribution should also be consistent, every single time should use the identical process. If you don’t like that, try Linux From Scratch instead, or Gentoo, or some other roll-your-own effort. But if you want to take on the Red Hat Linux and Fedora lineage, get it right. Make the user experience be the number one priority.

Fedora doesn’t suck, but it could be a lot more useful to a lot more people. And either I am right, or I am wrong. Policy, feedback, goals, all of these things will determine how many of the things I would like to see are feasible, in line with Fedora’s longer term vision, and so forth. So let’s see a very specific vision and direction and all be much happier for it.

Jon.

Recent discussion on Fedora devel about systemd got my pulse racing, so I thought I’d share a few thoughts here. This is all my own personal opinion and is not an attack on Lennart. He’s a great guy who takes on tough problems and is at most guilty of being overly optimistic that problems won’t arise in the adoption of new technology.

Firstly, let me note that the systemd idea is a good one, in the longer term. Other Operating Systems have tried this approach (perhaps the closest obvious example is OSX) to starting services when an attempt is made to talk to them, and there is a lot of merit to doing this as an automatic dependency-driven way of avoiding cruft, speeding up boot times, etc. Now I should note that none of you really actually care about boot times (you just think you do) because you suspend/resume for daily use, and if you’re doing hard-core plumbing or kernel work then you don’t have a lot of other cruft installed in the first place. So let’s just throw that “but but but…boottimes!” out of the window and admit to ourselves that’s just for fun. Good. Now, the fundamental approach to systemd is still a good one or others wouldn’t have tried this kind of thing in the past.

Secondly, Linux is not OSX. Scott had a tough enough time with upstart already (and wouldn’t it be nice if we could all just get along…but that’s another topic), things take time to change. Attempting to adopt a new technology late in a release just because it’s shiny and useful to Desktop users is in my personal opinion, unwise. The existing boot process works well – albeit it is not perfection, but then there are many other more pressing issues – and can certainly see us through another release of Fedora while systemd stabilizes, commitments to interfaces are thrashed out, and we figure out how to agree on stuff like what “noauto” even means anyway. You’d think that was standardized, but perhaps not. And if it’s not, this just shows how this can of worms opens up a ton of other stuff to be poking at at the same time.

But maybe I’m just hating on this because of my PulseAudio experience? Not at all. It’s true that I find PA frustrating, and none of the “killer app” features over raw ALSA have yet sold me on it. Playing to my “Air Tunes” device (RAOP) is skippy, grabs the device and won’t let go until PA is killed, playing from virtual machines is buggy, getting sound to come out of the right speakers is a challenge at the best of times (and the GNOME mixer app progressively becomes something so dumbed down it’s no longer useful to me), and I haven’t yet had the courage to try using Bluetooth speaker support. But these aren’t personal failings of Lennart. He’s just a talented guy who likes to take on tough projects that are complex and have a lot of dependencies. And sometimes there are dependencies on kernel changes, or other stuff that is tough for one guy to get done. I’m not hating on systemd, I’m saying that very talented people are still faced with the reality of glitches, bugs, and user adoption.

It comes down to this: most users would prefer to have some reliable sound than the best bells and whistles, and most of those same users (and I mean other than the handful who discuss stuff regularly on the Fedora “devel” list) would similarly prefer no init glitches over a technically better one. Pulseaudio has matured a lot. It’s still not perfect, but I generally let it stay installed these days (whereas I used to remove it immediately), except when I’m trying to do audio recording. I would say that systemd will mature nicely over the next 6 months or so at the present rate, at which time it can be considered for inclusion in Fedora as the optional or default init. But to include it now (in F14) is really – in my opinion – an exercise in forcing users to do testing without giving them a choice to opt into it. Let the experts try it first.

Jon.