Archive for August, 2010

What I want from Fedora

Saturday, August 28th, 2010

Fedora is one of these interesting efforts in community organization that somehow results in a finished product. There are many good things about Fedora as a place to see new technologies and to participate in Open Source. But there are also some bad things that need work. Not all of the technologies are instantly useful or necessary to all users, and they are often adopted quickly, whereas it would often be better (in my opinion) to pursue full integration over a (slightly) longer period of time. We don’t have a clear vision for who users are, or what they want, and collective Laissez-Faire organizing only works to a point, a point at which you need to have strong-willed steering of the ship you’re on to make sure it goes where it needs to get to. And the 6 month release cycles are far too frequent in my opinion if they will result in only a one year overall lifetime.

Specific things I want to see from Fedora:

1). Slow the updates. 800 updates after installation is bordering on insanity. I’d like to see updates only for security issues and bugs (but just those bugs, not wholesale rebases to “fix” a bug and introduce ten more), and I’d like the extreme opposite of what we’re seeing now, but I’ll take what I can get. This isn’t Enterprise vs. non-Enterprise. This is sanity vs. non-sanity. Rawhide is where “stuff happens”(TM). If you want stuff to happen in a “stable” release, take a look at this thing called rawhide. Then re-read this paragraph. If that doesn’t work, ask yourself if you are the best programmer who ever lived? If you really are, and are not going to break something – no matter how insignificantly annoying – and are going to test your non-essential update, have at it. But if my video-out breaks, if my printer stops, if my system doesn’t boot, if the behavior changes, if my proprietary flash plugin that worked just fine yesterday breaks (which does matter), if something isn’t exactly the same after I run an update, you caused a regression, and people are not going to be happy. I expect that only to happen when I choose to upgrade, and you have a choice to leave it the heck alone. Consider that choice as a good first choice and default to it, unless an update is essential.

2). Set a mandate. Either ask the users what you want from them (I have called again for a survey to be conducted), or tell them what that will be (also a valid option). But there needs to be a very clear direction here, and it must be spelled out very strongly in writing. Fedora can’t be all things to all people, that isn’t working, and time and again this is borne out on the mailing lists, and in various other anecdotal evidence. Depending upon who shouts the loudest will result in it either being an everyday Linux distribution for Linux users, or of niche value to those who like to ride the wave on the Fedora development list. Here’s where a series of very specific documents – we might call it a Declaration – comes into being and says “this is what we do”. Period. You don’t like it, fine, but this is the situation. Having something like that also allows it to be shaped and changed. And for users to express their opinions on what should be thrust at them. We’re on Fedora 14. I would have thought we could have gotten a document like that worked out by about, oh, now. How about before F-14 ships?

3). Establish cross-functional workgroups. There is no Desktop, there is no kernel. There is only this kind of use or that. Why don’t we have a “Plumbing” team that handles the low-level guts and meets regularly to sync up on them? We should have various groups that represent more than component IDs in Bugzilla, because the Linux system isn’t quaint any more. The current mechanism of development sees someone introduce some big feature or whatever, break random other stuff, then fix up the fallout, probably slowly and over a few releases (due to the number of people who were not involved early on). Instead, everything should be planned. There should be an actual idea of overall system design, overall architecture, what it is that will happen. Does everything have to have a command line equivalent? Make it a policy. And people are busy. Some are working on Fedora as a hobby, others have different commitments. If you depend solely on who is available “right now”, and who is dropping code today, then you will forever have this mess. Sometimes people are busy on other stuff and you’re going to have to wait a release for your shiny new feature to get integrated, and that’s fine.

4). Set specific goals. Don’t just let people post Feature page ideas randomly. Have a strong process (stronger than now – before anything is allowed to be built and shipped out the door), with an overall vision that those ideas fit into. And have a few people be in charge of overall consistency and vision decide how a feature fits in. If it doesn’t, punt it to the next release. This kind of thing happens to a very limited extent right now. It should be rigorous, it should be tough, getting stuff into the distribution should also be consistent, every single time should use the identical process. If you don’t like that, try Linux From Scratch instead, or Gentoo, or some other roll-your-own effort. But if you want to take on the Red Hat Linux and Fedora lineage, get it right. Make the user experience be the number one priority.

Fedora doesn’t suck, but it could be a lot more useful to a lot more people. And either I am right, or I am wrong. Policy, feedback, goals, all of these things will determine how many of the things I would like to see are feasible, in line with Fedora’s longer term vision, and so forth. So let’s see a very specific vision and direction and all be much happier for it.

Jon.

On systemd adoption

Tuesday, August 24th, 2010

Recent discussion on Fedora devel about systemd got my pulse racing, so I thought I’d share a few thoughts here. This is all my own personal opinion and is not an attack on Lennart. He’s a great guy who takes on tough problems and is at most guilty of being overly optimistic that problems won’t arise in the adoption of new technology.

Firstly, let me note that the systemd idea is a good one, in the longer term. Other Operating Systems have tried this approach (perhaps the closest obvious example is OSX) to starting services when an attempt is made to talk to them, and there is a lot of merit to doing this as an automatic dependency-driven way of avoiding cruft, speeding up boot times, etc. Now I should note that none of you really actually care about boot times (you just think you do) because you suspend/resume for daily use, and if you’re doing hard-core plumbing or kernel work then you don’t have a lot of other cruft installed in the first place. So let’s just throw that “but but but…boottimes!” out of the window and admit to ourselves that’s just for fun. Good. Now, the fundamental approach to systemd is still a good one or others wouldn’t have tried this kind of thing in the past.

Secondly, Linux is not OSX. Scott had a tough enough time with upstart already (and wouldn’t it be nice if we could all just get along…but that’s another topic), things take time to change. Attempting to adopt a new technology late in a release just because it’s shiny and useful to Desktop users is in my personal opinion, unwise. The existing boot process works well – albeit it is not perfection, but then there are many other more pressing issues – and can certainly see us through another release of Fedora while systemd stabilizes, commitments to interfaces are thrashed out, and we figure out how to agree on stuff like what “noauto” even means anyway. You’d think that was standardized, but perhaps not. And if it’s not, this just shows how this can of worms opens up a ton of other stuff to be poking at at the same time.

But maybe I’m just hating on this because of my PulseAudio experience? Not at all. It’s true that I find PA frustrating, and none of the “killer app” features over raw ALSA have yet sold me on it. Playing to my “Air Tunes” device (RAOP) is skippy, grabs the device and won’t let go until PA is killed, playing from virtual machines is buggy, getting sound to come out of the right speakers is a challenge at the best of times (and the GNOME mixer app progressively becomes something so dumbed down it’s no longer useful to me), and I haven’t yet had the courage to try using Bluetooth speaker support. But these aren’t personal failings of Lennart. He’s just a talented guy who likes to take on tough projects that are complex and have a lot of dependencies. And sometimes there are dependencies on kernel changes, or other stuff that is tough for one guy to get done. I’m not hating on systemd, I’m saying that very talented people are still faced with the reality of glitches, bugs, and user adoption.

It comes down to this: most users would prefer to have some reliable sound than the best bells and whistles, and most of those same users (and I mean other than the handful who discuss stuff regularly on the Fedora “devel” list) would similarly prefer no init glitches over a technically better one. Pulseaudio has matured a lot. It’s still not perfect, but I generally let it stay installed these days (whereas I used to remove it immediately), except when I’m trying to do audio recording. I would say that systemd will mature nicely over the next 6 months or so at the present rate, at which time it can be considered for inclusion in Fedora as the optional or default init. But to include it now (in F14) is really – in my opinion – an exercise in forcing users to do testing without giving them a choice to opt into it. Let the experts try it first.

Jon.

Advanced Bugzilla Features

Sunday, August 22nd, 2010

One of the things I’ve learned in recent times is that good customer service means good communication. You need to actually use tools like Bugzilla to keep up to date with what people are asking for, and you need to file status updates at least once a day to hot-button issues so that people know where they stand. Sometimes there’s extra process to whine about, but on the whole using tools like Bugzilla correctly is “For The Win” (as the kids say). I am mostly refering to work uses of Bugzilla, but the following is generic advice.

I discovered some time ago that Bugzilla has a great feature called “whining” (it’s under the “Administration” options, and if your site has disabled it because they did not think about you wanting to use that “admin” feature for yourself, be sure to whine at your admin about it). With this feature, I can setup what are essentially Bugzilla cron jobs that will run at certain times of the day. Combined with various advanced queries, it’s possible to do some fun things to keep on top of where things stand with open issues. For example, each morning, I get the following emails from Bugzilla:

*). New bugs reported. Includes both bugs for me, as well as e.g. all kernel bugs and other core subsystem reports.
*). Currently assigned bugs. This is split out into work and community.
*). Bugs for a specific project I am involved with.
*). Bugs that are set NEEDINFO to me.
*). Bugs on which I was added to the CC in the past 36 hours.
*). Bugs for which new comments were added in the past 36 hours.

Then, every 4 hours throughout the day, I get an email that shows any new comments that have been added to bugs for which I am responsible, or for which I am set as a “watcher” (most core work subsystems at this point – kernel, dracut, LVM, etc. just to make sure I am aware of everything that changes). All you need to do to implement this is to create some Bugzilla searches and then define what wil run and when using the “whining” feature. You can use the boolean logic feature of bugzilla to add in the necessary bits to catch things like “I got added to CC” (which is not explicit), or that NEEDINFO or other custom flag extensions were changed recently.

I find that it now takes relatively little effort to open a new browser window with tabs for each bug and check on it when it changes state. If some action is needed, I note it down, whereas if some action is not required then I will try to make a note of this fact in the bug. The idea is that the bug conveys all information about what is required right now, or what is not required right now, so I know I am not the blocking factor. Eventually, I plan to have bug state changes automatically create a TODO item in my todo-management software to note that I need to take action. In the meantime, I hope that others will play with these features to help optimize their own workflows.

Jon.

The risk of upgrades

Saturday, August 21st, 2010

Sometimes it seems like we’re living in a world of two kinds of Open Source. On the one hand, we have those who like to run unstable/rawhide type of systems, with the latest kernels, and who feel that anything older than ten minutes is still in the stone age. These people are usually paid to work on such things, have a lot of free time not spent doing other stuff, etc. The very notion that you might not upgrade every system the moment – nay the femtosecond – that the new version is out obviously means you’re not cool enough for school, even if what you have is working well enough for you right now. That means that the desktop on which you only surf the web – not the system you actually do kernel compiles on – must be running the latest possible stuff released 2 hours ago, “just because” (insert no useful reason here).

On the other hand, we have “Enterprise” types who install something to solve a problem, and then have to pay real, tangible, money to upgrade/change. Testing costs money and takes time (and I mean of the non-throw it over the wall and hope – but hey, it’s new so it must be better and worth breakage, right?), and if it aint broke, why fix it? Seriously. If an older version of a Linux distribution with an older kernel works for you, and you can still get essential security fixes, then great. More power to you. This is where Open Source should be offering a compelling choice not to upgrade, if you don’t want to. Incidentally, that’s the reason you don’t see updates for lots of older embedded gadgets – as I pointed out at Linux Symposium when explaining how the average (non-geek) consumer doesn’t necessarily “need” Android 2.2 the moment it is first built in beta. Doing an OTA upgrade for something that already works introduces the risk of bricking many units and incurring cost. It doesn’t mean it’s not fun to upgrade your phone to the latest Android test build, or that some manufacturers won’t choose to do it as a value-add, but it does means it’s something you can do on your own dime. If it breaks because they upgraded it for you, they pay. If it breaks because you couldn’t leave it alone, it’s your fault. Keep both pieces.

I used to fall more into the camp of wanting the latest and greatest on every machine. Back when I enjoyed spending a weekend configuring APS to make my printer work with just the most ultra-pointlessly geeky layout for text files sent to lpr, I enjoyed re-installing, upgrading, and generally playing around. After all, this was before I really finally realized there is more to life than computers all the time. Over time, like many others, I grew up and realized that some things which work can appropriately be left alone without the universe exploding. Sure, we don’t want 40-year-old unmaintainable software disasters driving our government infrastructure of tomorrow, but there’s a medium somewhere in there too. I hope that, as the Open Source community matures, more people will come to appreciate this fact. By all means develop using the latest and greatest, but spare a thought for accepting that not everyone out there in the user community is as excited about wasting a day/weekend doing an upgrade unless or until they have to.

Jon.

Flying Cheap

Sunday, August 8th, 2010

I’m watching the PBS documentary “Flying Cheap” about the crash of Colgan Air Flight 3407, which was a “regional” plane flying Continental Airlines colors that crashed in February 2009 due to a “recoverable” stall. It’s called a “watershed” moment in the airline industry because it exposed an industry culture of cost cutting, underpaid staff, etc. The crew was tired, and the co-pilot (who made only $16,000 per year) had “commuted” from Seattle, Washington to New York for the flight during the night/morning of the day of the flight itself.

The crash and this PBS documentary raise all kinds of concerns, but none more troubling to me than the following realities of the state of US society (which also applies to other countries) in the modern age:

1). People want the cheapest price. It doesn’t matter if it’s just a few cents. We have become a nation and a world in which the lowest possible price always wins. This is called a benefit of “capitalism”, and it’s utterly disgusting. The lowest possible price is not always the best. As the old saying goes, “buy cheap, buy twice”, and when it comes to your own personal safety, you don’t get to live your life over again when that fails.

2). Deregulation and the removal of big, powerful, centralized, federal government from the affairs of things like the daily running of airlines is regarded as a “good” thing by many, the same kinds of people who scramble to explain how the financial ruin we’re in wasn’t due to a lack of regulation after Conservatives (and some democrats – small D) went about systematically laying ruin to checks and balances. The reality is that when you remove regulation, and moronically decide that government is a bad thing, you get chaos. You get unchecked behaviors that are often bad, people out to make a profit from whatever they can.

2). The state of sick pay and time off in the US today is utterly disgusting. We live in an age where it’s somehow seen as acceptable that people taking time off due to sickness should cover their own lost wages (or more likely, just not call in sick). A kind of Thomas Paine missguided reality in which everyone takes care of themselves and the government takes care of nobody. In fact, we should have a reality in which federal, state, and employer insurance provides for strong coverage of folks who are sick (who should have universal healthcare, too), who don’t have to worry about taking time off due to illness. They leave their germs at home, they don’t fly planes, and so forth. What about abuse? Sure. This is always possible. But unlike Conservatives, I don’t think abuse is everywhere, and I think it’s acceptable to allow a certain amount in favor of the greater common good. Just like how everyone should receive unemployment coverage and strong benefits because most people are actually decent and honest, and not out to cheat the system.

We need to get over a culture in which cheaper is always best, people think they should take care of every aspect of societal care for themselves – rather than their government, which should be helping in a caring society – and in which regulation is feared as some evil. It’s not just the US, this can be seen the world over. Right now, in the UK, the Conservative Party (with the complicity of the Liberal Democrats) is attempting to lay waste to and systematically ruin that country more than it already is. This is the nature of Conservatism and it is always entirely wrong. It must be countered and corrected with strong Liberal values and opinions for the sake of the well being of all future generations.

This has been a rant.

Jon.