Every release, there's at least one bug that affects a significant number of users. Most of the time, we get lucky, and the users can get the system installed, and we can fix the problem in an update. The worse-case scenario however is the case where a bug prevents installation, or prevents booting post-install.

F10 had a bug just like this.

It manifested in several different ways.
For some people, it result in a 10 second pause during boot up. For others, it failed to find the root filesystem, even after waiting.

What happened here?

The initrd has code to wait for the kernel to finish enumerating devices. This is an important thing to do so we don't try anything silly like trying to mount the root filesystem before the driver for the disk controller has found the disks plugged into it.

In F9 and earlier, we would emit this code when the initrd included the scsi_mod module.

Late in the F10 development, we moved a bunch of things to be built-ins, to tweak the boot-up speed.
This turned out to be a really bad idea, as it meant the 'stabilization' code didn't get emitted. Ouch.

How come it didn't affect everyone? Different controllers behave differently. Some initialise so quickly that there's no reason to wait any longer.

Why didn't it get caught during testing? We did have a large number of reports, but the correlation wasn't made until post-release.

Should it have held up the release? Possibly, if we knew it would affect so many people.

FTR: None of the hardware I personally tested was affected by this problem, which made it a real pain to track down.

The good news is that in the hopefully not to distant future, all this code is going the way of the dodo anyway.

Recently I've read about or spoken with a few people using Fedora on eeepc's who have been making some fairly big blunders without realising it. I've played with a few of these now, in their various incarnations.
The current one I've been carrying around is the 900 model
with a whopping 20GB of flash. It's quite deceptive, because there are actually two SSDs in there (one 4GB, and one 16GB)
These ssds are also pretty damn awful performance-wise compared to the newer generation of SSDs, but short of opening it up and retrofitting something, there's not much that can be done. The tips below should at least make it more bearable.

• First off, don't use the default partitioning scheme.
  By default, anaconda will choose to use lvm, and make a contiguous volume out of the two SSDs. This idea is fail, because the two disks aren't the same, and run at different speeds.
  
  # hdparm -t /dev/sda
  
  /dev/sda:
  Timing buffered disk reads: 108 MB in 3.04 seconds = 35.57 MB/sec
  
  # hdparm -t /dev/sdb
  
  /dev/sdb:
  Timing buffered disk reads: 86 MB in 3.05 seconds = 28.20 MB/sec
  
  
  So, don't do that. Just create regular partitions, and make sure you put / on the faster of the two disks (the 4GB one), and leave the 16GB one for /home
  
  
• Next, the default filesystem will be ext3. You really don't want this.
  Given the journal is in a fixed location on disk, scribbling to it every time a file gets written is a great way to wear out the flash. Go with ext2. (Given that you've only got a few GB of flash anyway, a fsck doesn't take that long should you need to). Additionally, not having to write to the journal means that you're doing less IO, which is obviously a win when it's on such slow media.
  
  
• This should go without saying - no swap.
  Not only for the flash wear problem in the previous bullet, but also, because it's slow as all hell. If you find you run out of ram and get stuff oom-killed in this setup, well, you probably need to add more ram, or consider a real laptop.
  
  
• After installing, change the fstab so that everything gets mounted with noatime. Writes to the disk are just painful, so minimising them is the path to success here. It's doubtful that you'll be running anything on an eee that would actually care about atimes anyway.
  

With these points taken into consideration, the eee isn't a half-bad machine.
I still wouldn't want to be building kernels and such on it, but it's perfectly usable for email and such whilst travelling.

In the midst of all of last weeks chaos, rawhide saw its tenth birthday.

I just updated one of my older posts on bugzilla statistics as it got mentioned on Linux-kernel in a thread on kernel bug counts. I updated it with the latest F7/F8 figures, and threw in the stats from F9 so far for good measure.

Something of note, is that in some cases, the number of bugs went down. (Note, I count 'closed' bugs too). This is likely due to people upgrading to newer releases, finding the bug is also present there, and then migrating their bugzilla entry from F7 to F8.
I think this is why the number of F9 bugs we have already (4624) is so high. Will be interesting to see if this tails off over coming months.

Earlier this week, I noticed a posting from Andy Wingo about how he'd switched to Fedora from Ubuntu. Whilst I found his post interesting, I knew I'd find the comments even moreso. Certain things keep coming up over and over again, and I'd really like to know more.

• "Fedora is slower than ubuntu/debian/mepis/yourmom".
  
  Exactly 'what' is slower ? Every time this argument is trotted out, there's nothing quantifiable in there, which makes it a little difficult to investigate. We have all the same profiling tools as everyone else, so for someone experiencing this slowness, it shouldn't be too hard to identify at least the component that's causing this. (One thing that is well known -- we still really suck at boot up time. Working on it, honest).
  
• "Fedora has less packages than debian/ubuntu"
  
  No argument there. However, I'm curious exactly what packages people are missing that aren't in Fedora for this argument to hold any water. A few years ago, before we even had Fedora-extras, maybe this was a more valid argument. Today, in a post-core/extras-merge world, it's a rare case that I need something that isn't packaged. (And when it isn't getting it packaged up, reviewed and added to the repositories is usually fairly painless for a Fedora contributor). Looking at the package wishlist, there's a lot of fairly obscure stuff, but the list isn't exactly huge.
  What else do you miss from Fedora ?
  
• "I don't use Fedora because of rpm hell"
  
  It's a fair bet that anyone using this argument hasn't actually used Fedora in some time. Typically dependancy problems of this nature come from mixing repositories. Again, with the core/extras merge, there's even less reason for third party repositories to exist. One exception to this is the development repo, which sees transitional effects from time to time (sometimes on a daily basis) due to the amount of churn. That said, some of our dependancy chains really, really suck. The more egregious problems are usually bugs that need fixing on a per-package level, and not something inherently wrong with rpm.
  None of this problem is unique to rpm. I've seen apt/dpkg get into similar problems when mixing third party repositories.
  

I found these numbers interesting.
(I only focus here on packages that had >100 bugs filed against them. Bugs are counted regardless of their state [open/closed/whatever])

FC1
Total number of bugs filed across all packages: 3109
Out of those, kernel made up 442, followed by anaconda at 192 and XFree86 at 111. Everything else was < 100

FC2
Total number of bugs filed across all packages: 4043
kernel takes the lead again, at 1053 bugs. Ouch 1 in every 4 bugs. This was the first release we switched to the (then new) 2.6 kernel. Anaconda took second place again at 237 bugs.

FC3
Total number of bugs filed across all packages: 7367
Did we get a lot buggier? Or was this the increased testing that happened, due to this release coinciding with RHEL4. I'd like to believe the latter. With an increase in bugs, the number of packages scoring > 100 bugs filed also increased.
Anaconda at 452
Xorg at 270. I guess as XFree86 stagnated, the bug count previously stayed low.
Evolution at 149
Yum makes its debut with 133 bugs
SELinux-policy-targeted at 142 (I think this was the first release we introduced targeted policy)
Firefox at 118
Openoffice at 111
None of them come close to the kernel, which amassed a staggering 1190 bugs. (I'm not sure how I didn't have a meltdown at the time, when I was also juggling RHEL4 kernel bugs).

FC4
Total number of bugs filed across all packages: 7602
Interesting. No post-RHEL4 drop-off.
Kernel breaks its personal best with 1213 bugs. I think the 'one person managing multiple releases' cracks were beginning to show.
Anaconda: 583
Openoffice: 222
Xorg: 218
SELinux-policy-targetted continues to rake in the bugs: 171
Yum still 'not quite there' at 169 bugs.

FC5
Total number of bugs filed across all packages: 8487
Kernel in the top spot yet again, at 1381 bugs. (The highest of any release so far)
Anaconda: 617
Gnome-themes: 222 (This statistical anomoly seems to have been caused by an enthusiastic bug filer clicking submit several hundred times)
Pirut: 139 (It seems there's always a spike whenever we introduce a new tool, especially if its something fundamental like a packaging tool)
SELinux-policy-targeted: 124
Openoffice: 120
Firefox: 109
Xen: 105 (Another high-scoring debut)
Worthy of an honorable mention: This release marked the introduction of modular Xorg. Because of this, all its bugs got scattered across multiple packages. Added together: 263 bugs.

FC6
Total number of bugs filed across all packages: 7792. The first release where we actually saw the overall bug count go down.
Kernel: 981 (This also spawned kernel-xen, which amassed 98 bugs)
Anaconda: 474
Xorg (cumulative across all packages again): 297
Yum: 221
Pirut: 177
SELinux-policy-targeted: 114
Firefox: 106
Xen: 105
Compiz: 100 (Another introductory high-score)

F7
Total number of bugs filed across all packages: 5534. Quite a huge drop in overall bug count. (Note F7 is still open for a little longer for new bugs, but this drop does still seem dramatic)
Kernel: 724
Anaconda: 233
Xorg (cumulative): 161
Yum: 201
SELinux-policy: 157
SELinux-policy-targeted: 118
Pirut: 155

F8
Total number of bugs filed across all packages: 7589 (F8 is also open for new bugs for quite a while, so this is impressively high)
Kernel: 902
Anaconda: 303
Xorg (cumulative): 278
Pirut: 231
NetworkManager: 144
SELinux-Policy: 154
SELinux-Policy-targeted: 141
Firefox: 107

F9
Total number of bugs filed across all packages: 4624 (Impressive, considering it only just got released)
Kernel: 292
Anaconda: 213
Xorg (cumulative): 273

[Updated: May 26th to update F7/F8 totals, and add in F9]

It now takes 7 hours to run a perl script that reads every .c, .cc, .cpp file from a 'make prep'd Fedora CVS tree and checks them for common bug patterns (the swapped memset args type of thing). When I first started doing this, it took less than an hour.
This is on a 2.8GHz core 2 duo, with two sata disks in raid0. I guess it's time to throw more disks at the problem.

It now also takes considerably longer to sift through the results picking out false positives of some of the checks. The log from last nights run is 9.8M, 155491 lines.

Useless factoid: There are 47526100 lines of C/C++ in Fedora right now. (And it took 50 minutes to count them all).

I spent some time last week getting my M-Audio MIDISport 1x1 interface working in Fedora. These devices are based on the cypress EZ-USB standard, which means they're essentially just a dumb microcontroller that doesn't do anything until it's fed firmware over USB using fxload (which is thankfully already packaged in Fedora). I packaged up the firmware last week, which should hopefully get into the distro at some point soon. The next thing to do was to hack up some trivial udev scripts to make the firmware get autoloaded on insertion.

ACTION!="add", GOTO="drivers_end"

SUBSYSTEM=="usb", ENV{PRODUCT}=="763/1010/*", RUN+="/sbin/fxload -D $env{DEVNAME} -s /lib/firmware/MidiSportLoader.ihx -I /lib/firmware/MidiSport1x1.ihx"
SUBSYSTEM=="usb", ENV{PRODUCT}=="763/1020/*", RUN+="/sbin/fxload -D $env{DEVNAME} -s /lib/firmware/MidiSportLoader.ihx -I /lib/firmware/MidiSport4x4.ihx"

LABEL="drivers_end"


Fairly trivial, but it took me a bit of headscratching to get the incantation correct.

Rerunning my occasional 'unpack the source of the whole distro and grep for fun things' thing again.
For those interested, the script I'm using is fairly simple..

#!/bin/bash

for dir in *
do
echo
echo $dir
cd $dir
if [ ! -f dead.package ]; then
S=$(find . ! -empty -name sources)
if [ "$S" = "./sources" ]; then
SPEC=$(find . -name "*.spec")
rpmbuild --clean --nodeps --define "_sourcedir ." --define "_builddir ." $SPEC
cvs up
make prep
else
echo "Empty or no sources."
echo $S
fi
else
echo "Dead package."
fi
cd ..
done


It requires a full CVS checkout of devel/, and also needs a lot of disk space. (Where 'a lot' = around 77G). It also requires that every package has a functional 'prep' target. A number were broken. I fixed up a few, and sent package maintainers the changes. Things should be mostly sane now, though a few packages have some fun quirks. Take for eg, javacc. This package has a file in its cvs checkout called 'javacc'. The make prep target unpacks a tarball which creates a directory called... 'javacc', blowing away the file in the process. Subsequently CVS gets very confused about a file becoming a directory, and refuses to do much else. Renaming javacc to javacc.sh should do the trick.

How long does all this take? The whole process includes downloading tarballs, so is network and disk bandwidth limited. My 16Mbit cable takes around 36 hours to download and prep the whole tree. Subsequent runs go a little faster, but still take a number of hours, due to the going for 6000 packages that it has to churn through. A clear sign of how Fedora has grown. In times past when I have run this procedure it was feasible to have the run complete in an afternoon. Not any more.

Given the huge amount of time it takes to regenerate each time, I'm going to start hacking on some scripts that will query the cvs server, and only make prep the packages that need to change.

Gave my 'day in the life of a distro kernel maintainer' talk yesterday which went well. Will upload the slides etc when I get home. (And possibly the movie containing partial nudity which seemed to be a popular request. [people are weird]).

Earlier in the week, I also did a session at the Fedora miniconf on the state of the kernel for F9. It was pretty much the same thing I gave at Fudcon last week, but with some extra bits. Went surprisingly well too considering I mostly ad-lib'd it.
Alex Chiang wrote up a nice summary of what I spoke about for those who couldn't make it to LCA.

(The bit about ext4 is somewhat optimistic. There's a ton of stuff left to do for ext4 before it loses its ext4dev moniker. I possibly misled people into believing we're shipping this in F9 regardless of it state due to the pending changes for .25 The reality is that until the 'dev' goes away, I'm paranoid about more last minute on-disk format changes.)

Vacations & injuries aside, I've not been able to do any uploads to Fedora for quite a while now. (Essentially since I reinstalled my desktop with F8). Something causes curl to abort 'make upload'. It gets very confused when it sees my certificate. I've tried downloading a replacement, which made no difference, and it all looks kosher (has a valid expiry date etc) to my untrained eye.

What's really puzzling me though is why I'm the only person that is seeing this.

Started digging into it this afternoon with gdb, to find out where it's failing. I don't really understand anywhere near enough about this stuff to fully debug it, but the failure seems to be somewhere in the mozilla crypto magick.

(This post reads a little more 'shout out to my homies' than I originally anticipated, but it is what it is, and you people kick ass)

2007 has been a good year for the Fedora kernel.

At the end of last year, we doubled the size of the Fedora kernel team when we hired Chuck Ebbert. Whilst Fedora had a number of contributors to the kernel before this, I was the only employee at that time who was working exclusively on the Fedora kernel full-time.

I expected the first few months of the year to go slowly, as Chuck got to grips with processes etc. Boy was I wrong. Within no time at all, Chuck was pushing out updates for our older releases faster than I could keep track. It soon got to a stage where he was single-handedly taking responsibility for the updates, whilst I concentrated on beating rawhide into shape for what would become Fedora 7. Not one to slack off, Chuck also lent a hand there too. It became apparent real quickly we had got the right guy for the job.

Fedora 7 was a pretty solid release. Despite being based on 2.6.21 which wasn't the best kernel we've had. A few months of bug fixing got it into shape in short order. F7 also was the first distro to ship a tickless kernel. The work put in by Thomas Gleixner to nail down a huge number of timer related bugs both during development, and after release cannot be understated. Things would have looked a lot different without his involvement.

F7 was the first release we did where I actually felt 'comfortable' about saying we had wireless support. It still wasn't fantastic, but it was much improved experience over earlier releases. Between John Linville pushing bits into Fedora as fast as they were going into the wireless-dev tree, and the return of Dan Williams to the desktop team after spending a while working on OLPC, the wireless experience in Fedora 8 totally rocks. Maybe I've just been lucky in my wireless chipsets, but I've yet to have F8 fail me. (And the bug reports seem a lot better this time around too).

Asides from technical changes, there have also been a number of process changes this year. First, the creation of the Fedora kernel list, which has grown to several hundred members, whilst still maintaining a healthy signal-to-noise ratio. (Traffic is still light, but sees occasional bursts of activity). Shortly afterwards, the creation of #fedora-kernel which has been really useful for various coordination. To the point that I rarely look at internal Red Hat irc any more.

A few months ago, the Fedora kernel team grew some more, when we hired our third full-time Fedora hacker, Kyle McMartin. Things sure feel different to a year ago, when I feared going on vacation and leaving things hanging until I returned. Kyle & Chuck pick up the ample slack that I create. Whilst I've been schmoozing at conferences, meetings and other blah, these guys have been rocking out pushing updates, fixing bugs, and doing, well, doing what they do. Rock on dudes.

We picked up Dave Airlie this year too, who was a valuable addition to Red Hat. Although he's on a different team, he's been keeping Nouveau and DRM in general in good shape in Fedora this year. I also handed off AGP maintenance this year to him, freeing up even more time for slacking off for me.

Some effort has been spent this year trying to document various tribal knowledge at the Fedora project wiki to hopefully make things easier for future contributors, and also to try and make it easier for people to help themselves provide better bug reports. The 'bug triage' page there has turned into something really useful that I think will only continue to grow more awesome over time. It'll no doubt be handy when we eventually get around to have a 'fedora kernel bug day' which we've been talking about doing for months now, but never quite getting around to. I think we'll have to make this happen soon.

So in summary, a busy year for the Fedora kernel team. Next year, more of the same, but louder, faster, better, more.

[update: I'm sure I've forgotten the valuable contributions of other people over the last year too. Don't feel slighted if I missed you. You also rock, I'm just an amnesiac].

I really dislike this stage of Fedora development. Just before we're about to ship. We've frozen on 2.6.23 for F8, which should be out in a few weeks. In the meantime, the blocker list continues to dwindle, and upstream storms ahead towards 2.6.24. With it at -rc1 already, by the time we release, upstream will be well on it's way towards a final 2.6.24, so a big rebase update will probably occur within weeks of F8's release.

There's no shortage of bugs that need fixing, but not much in the critical sense of "can't install / update".

I've been rebasing devel/ to 2.6.24rc, but haven't done any builds yet, so that people focus on beating F8 into shape rather than running off to run the latest shiny thing.

Spent some time today writing documentation on diagnosing common bugs, exposing various tricks that have come in handy whilst fixing Fedora kernel bugs.

It's been a while since I last did something like this. It's made a refreshing change from just plodding through bugzilla.
Now hopefully others can learn from this, and also plod along at home.

For a long time, it's been something of a mystery which upstream kernel a specific Fedora kernel version was based upon. Today, thanks to the work of Jarod Wilson (and a small amount of poking by myself and Roland McGrath), the mystery is over.

Before:
kernel-2.6.21-1.3126.fc8

After:
kernel-2.6.22-0.5.rc7.git2.fc8

(The 0 indicates 'pre-release', and the .5 = build number)

We think that we've got all the different cases covered (upstream version is a little complicated sometimes from a programmable point of view, so making sure that rpm does that right thing on newer kernels needed a bit of thought).

The specfile for the kernel rpm is a little more complicated in some ways, but it also makes life a lot easier.
Rebasing is a breeze with the reworked spec. Change a single digit, and everything falls into place.
One other nice side-effect of this making doing a version rebase easier is that it's nearly scriptable, leaving me even time to fix bugs, eat cheetos, or whatever it is I do.

Talking with quite a few people at OLS last week, it seems there are still quite a few misconceptions about just how patched various kernels were throughout the history of Red Hat. One particularly egregious statement I heard was "Early Red Hat kernels had ~2000 patches".

Here's some hard facts on exactly how many patches were in each release.
(I don't have access to earlier kernels than these, but < RHL6, those kernels were probably even less patched).
Caveats:

• All stats are based on the last state that CVS was in when I ran the grep ^%patch *.spec | wc -l so may not match the kernel that ended up on the ISO for that release.
  
• Some releases contain -ac patches, which are roll-ups, sometimes of dozens of patches.
  In later releases, this practise has been deprecated (and also Alan rarely does -ac releases any more).
  

Release	Number of patches	Version
Red Hat Linux 7.0	70	2.2.24 !
Red Hat Linux 7.1	118	2.4.20
Red Hat Linux 7.2	118	2.4.20
Red Hat Linux 7.3	120	2.4.20
Red Hat Linux 8	117	2.4.20
Red Hat Linux 9	143	2.4.20

Wow, that takes me back. As the version number shows, these releases were all more or less the same kernel.
The big difference was RHL9 had nptl, and a few other 2.6 backports. These were pretty invasive patches.

Release	Number of patches	Version
Fedora Core 1	105	2.4.22
Fedora Core 2	78	2.6.10
Fedora Core 3	108	2.6.12
Fedora Core 4	85	2.6.17
Fedora Core 5	145	2.6.20
Fedora Core 6	191	2.6.20
Fedora 7	164	2.6.21
Fedora 8-devel (rawhide)	63	2.6.22-rc7

A few interesting points on the Fedora kernels.

• FC1 never rebased off of its original kernel, so there were quite a few > 2.4.22 patches backported to that.
  It still had nptl etc from RHL9, so rebasing was painful back then.
  
• FC5 & FC6 have quite high patchcounts, because currently they've stopped following mainline.
  After seeing the fallout from 2.6.21 during F7's development, we decided to skip a release, and hope that .22 works out to be better. When we rebase, approximately half those patches go away.
  
• F7 is carrying quite a few patches that really should get pushed through -stable. Some of them may even be pending inclusion. Lots of them are backports from 2.6.22-rc
  
• rawhide is still quite high. I need to do a patch bombing soon to try and get a bunch of the bits we're carrying pushed upstream. Should be able to get this back under 50 when the merge window opens up for 2.6.23
  

Now the fun ones..

Release	Number of patches	Version
AS 2.1	487	2.4.9
AS 2.1 (ia64)	272	2.4.18
RHEL3	395	2.4.21
RHEL4	1078	2.6.9
RHEL5	881	2.6.18

So over time, RHEL releases accumulate lots of patches. The pain-point of not rebasing means they tend to build up over time. RHEL4 being the 'winner', picking up just over a thousand patches in 2.5 years, though RHEL5 is catching up fast, nearly reaching that number already, after being released earlier this year.
This acceleration of patches may be related to the fact that upstream is moving a lot faster than it was around the time of earlier releases. We're finding and fixing a lot more stuff, a lot faster, and the upstream process seems to show no signs of slowing down (if anything, people are trying to make it go faster).

So, despite the claim that "Red Hat had thousands of patches" isn't true today, it might not be true once we freeze a future RHEL release.

Sigh, where to begin.
I had hoped that yesterdays build was going to be 'the one' to go to updates-testing.

A bunch of new patches arrived upstream, in the form of another -stable rc.
I set off the build, and shortly afterwards, linville noticed that one of the patches we were carrying was now obsolete.
I cancelled the build in progress, and watched his build crawl along for several hours.

Then, about 3 hours in, one of the builders ran out of disk space, aborting the whole build.
I waited a while for it to be cleaned up, and resubmitted the build.
At this point, I was feeling pretty demotivated to do much else, so I went out for a beer.

This morning, I get up to check that the build completed, expecting it to be ready for testing, and find a message on #fedora-devel that the 3224 build doesn't boot.

So now I get to play 'find the bad patch' game again. Hopefully, I can get this narrowed down and fixed up quickly.

I feel like I've lost my kernel building mojo.

The keen observers of rawhide will have noticed that I started building F8 kernels already.
So far, nothing really amazing, just a rebase to 2.6.22rc4, and some specfile jiggery-pokery to move away from using rpm's %patch directive to a hand crafted macro that allows patch application with -F1
(%patch won't let you pass arbitrary flags to patch(1), and the code in rpm to handle this made my eyes bleed).
The benefits of patches with no fuzz should be obvious, the diffs end up applying in the right places. This has caused problems in the past to the extent where we had one bug that made the kernel not boot.
That was real 'fun' to track down. Fun that I'd rather not repeat.

Other than this, pretty boring stuff. About half the patches we carried in F7 are now upstream, including the big firewire rewrite. Hopefully by F8 we can stop carrying the wireless stuff too.

With the schedule for F8 just 5 months, it's uncertain just how far forward we'll go. With each upstream kernel release taking 2-3 months, and .22 coming 'soon', it sounds like we'll be lucky to get .23 into F8, and beaten into shape by release, especially with conference season just beginning to ramp up, and various developers disappearing.

Some lessons were learned from the F7 release process though.
We did a last minute rebase to the latest upstream -stable, which brought in the "Dell laptop won't boot without maxcpus=1" bug at the last minute, when it was too late to fix. Next time around, we're not going to do this.
Given we're nearly always doing a kernel update within a week after release, the only patches going into the kernel during the final week (or maybe two weeks), will be individual backports of patches that have been upstream for a few days with no regressions, that fall into a critical criteria.
Where critical means:

• Fixes a "machine won't boot" bug.
  
• Fixes a "ethernet doesn't work" bug (thus preventing updates being downloaded)
  
• Fixes a data corruption bug.
  
• Fixes a _critical_ security bug.
  (Lower impact things like information leakage can wait until update 1 on the day of release).
  

There are possibly some other criteria I'm missing, this will get
fleshed out some more on fedora-kernel-list.

People keep asking me when the Fedora 7 kernel update is going out.
The answer 'soon' (maybe as soon as tomorrow).

There's still a slew of things that need fixing. For the first round of updates, the focus is almost entirely on fixing the critical bugs like "It crashes during boot". Looking at the first round of bugs that came in during the first week after F7's release, it's pretty horrific. The vast majority seem to be from libata (something of a mix of SATA and PATA bugs).
This was in part expected, as the switchover from ye-olde-IDE to libata pata was something of a gutsy decision to make. It wasn't expected that it'd be this ad.

So, for those asking "will this update fix my suspend/resume" "will my ipw3945 work better?" etc, the answer is "not yet".

The overall quality of 2.6.21 is pretty horrific. It saw the introduction of a lot of new code fundamental to the operation of the kernel (the tickless stuff for eg), massive updates to areas such as ACPI, and just to mix things up, we switched from a known-crap-but-tried-and-tested IDE system to a-bleeding-edge-but-hopefully-with-signs-of-promise libata based system. Lots of changes == lots of fallout the first time it goes into a production OS.

It's likely that we'll skip a 2.6.21 update for FC6, and do an update straight to 2.6.22 for both FC6 and F7 when that gets released.

So, doom and gloom, and business as usual on the kernel front.
On the plus side, I get to go see Skinny Puppy tonight. \o/

This rocks so hard.



Watching a box install over vnc over wireless. Asskicking magic.
From now on, I'm doing all my installs.. mobile.