Kernel Traffic #302 For 2 Apr 2005

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.11-rc4/2.6.11-rc4-mm1/

• Various fixes and updates all over the place. Things seem to have slowed down a bit.
• Last, final, ultimate call: if anyone has patches in here which are 2.6.11 material, please tell me.

This is a small patch for GEODE CPU support in Kernel 2.6.10.

Those CPU's are found mostly in embedded systems ... one of the most prominent Hardware using GEODE CPU is probably soekris net4801 (http://www.soekris.com).

This patch has been on my homepage (http://www.sk-tech.net/support/soekris.html) for quite a time - but I've been asked several time to have it included in the main kernel.

We are submitting a new serial driver for the 2.6 kernel. This device driver is for the Digi Neo serial port adapter.

We made some changes based on great comments from linux community. We used the Russell's serial_core interface on 2.6 kernel, handled all initialization of module correctly and used fs/seq_file.c interface for /proc entry. And we made some changes based on Greg KH's comments also including removing whitespace, changing some function comments, removing msleep() etc.

Per requests, I split it up in smaller chunks and sent them in several emails for you. You are able to read the code and comment the code easily. I added more adapter descriptions for you.

Neo adapter description:

This adapter is a 920K baud non-intelligent asynchronous serial communications adapter for PCI bus. The adapter is based on Exar 17D152 Universal PCI Dual UART IC. The adapter is mapped into the PCI bus memory space, and offers extened UART register mapping beyond the standard 16c554 registers. This driver used serial_core.c interface on 2.6 serials of kernel.

Based on very detail comments from Jeff, Greg, Christoph Hellwig, Rik and Nish, I modified these codes and tested it succesfully in our lab. For patch1, major changes included:

1. removed static board limit to use dynamic list to control board structure.
2. leak on errors.
3. removed some global variables
4. lots of others.

Do modular framebuffers really make sense?

OK, distributions like to make everything modular, but all the framebuffer drivers I've looked at parse driver specific options in their *_setup function only in the non-modular case.

And most framebuffer drivers contain a module_exit function. Is there really any case where this is both reasonable and working?

It makes little sense to keep the driver constantly loaded if the device is not being used as a console and is only seeing occasional use.

It seems more sensible to just fix up the drivers that don't do this right.. most of the broken drivers seem to be geared at x86 anyways where people generally don't seem to care.

It may not make a lot of sense with distributions on x86, though it is useful if you are doing driver development on a secondary device. This is certainly a corner case though.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.11-rc5/2.6.11-rc5-mm1/

• Lots of tuning/balancing changes in the CPU scheduler. Mainly targetted at larger SMT/SMP/NUMA machines. It's going to be hard to work out whether these are a net benefit.
• A pcmcia update which obsoletes cardmgr (although cardmgr still works) and makes pcmcia work more like regular hotpluggable devices. See the changelong in pcmcia-dont-send-eject-request-events-to-userspace.patch for details.
• A new reiser4 code drop.
• A new rev of the NFS ACL code.
• I seem to be getting a lot of patches which don't compile if you breathe on the .config file, let alone if you try them on another architecture. It would be nice to receive less such patches, please.

A while back someone complained about the CVS exporter because it sometimes groups a pile of BK changesets into one commit. That's true, it does.

I've been running tests over the BK tree and I think we can do better. Here's the scoop: when we do an export we are going from a very bushy graph structure to a linear graph structure. The BK graph structure represents what happened in all the BK repos that ever came together, the CVS graph structure is more like what would happen if all the work had been done in CVS. What that means in practice is that the linearization sometimes results in a single CVS commit which has multiple changesets in it. Pavel or someone complained that the problem with that is that if you are looking for a bug and you are searching through commits, that works fine *unless* your bug happens to be in one of the commits which is really a pile of changesets. Is that accurate Pavel/Andrea/Roman/etc?

In the last flamefest about BK there was all this fuss that there wasn't enough info in the CVS export and I think that the problem described above is the basis for 99% (or maybe 100%) of the flameage. Is that also accurate?

I tried to point out the following and think it was lost in the noise: while the repository commits themselves construct a very bushy graph, the files are not at all bushy, they are extremely linear. So what does that actually mean? The history of the repository is very parallel, that's what creates a bushy revision history graph, but in spite of that, there is very little parallelism in the actual files. The result of that is that when we do the CVS export, it is true that the number of commits is about half the number of BK commits. But the number of file deltas is only 4% less than the number of file deltas in BK.

Pavel/Andrea/Roman/etc still were unhappy and they are justified in being unhappy because even though we have almost all of the file history what we don't have is all of the patch boundaries. And when you are hunting down a bug, if you look at Documentation/BUG-HUNTING (which I wrote back in 1996 amusingly enough) the idea is to do binary search over a range of changes in order to narrow down the cause.

Which leads us back to the problem. If you narrow things down but where you land is one of the clustered commits which has many changesets in it then you are stuck with having to wade through a big pile of diffs to find the bug, those diffs consisting of multiple patches. Sound right to you Pavel/Andrea/Roman/etc?

If so, I have to agree with you, this is a limitation of the CVS exporter. So I've been thinking about how to fix this and have the following idea. I want all the CVS export users to pay close attention because this either should make you happy or not and I want to know the answer.

When we do the export we do a couple of things to make things pleasant for you. We make sure that the timestamps on all the files in the same commit are the same, that makes timestamp based tools work. We also shove a comment into each file's history that looks like so: (Logical change 1.12345) so that tools that try and group things based on comments can work.

It's that second feature that I think we can use to solve the problem, we're finally getting to the idea. If we have a commit that is really 200 patches which touch 400 files then we can do better. Suppose that the files in the patches are disjoint, i.e., each patch touches a different set of files, there is no overlap. If that's true then we could change the comment to (Logical change 1.12345.$PATCH). It's still all one CVS commit but if you need to go working through that commit to get at the individual patches you could, right?

One problem is that the set of files in patches may not be disjoint, the same file may participate in multiple patches. I think we can handle that in the following way, we put multiple comments, one for each patch, so you'd see

(Logical change 1.12345.5)
(Logical change 1.12345.11)
(Logical change 1.12345.79)

That's not a perfect answer because now that file participates in multiple patches and if it's the one that has the problem you'll have to wade through the diffs for that file for that commit. But that's an extreme corner case as far as I can tell (I have faith I'll be "educated" if I'm wrong about that).

So, everyone including the Pavel/Andrea/Roman/etc camp, how do you feel about this? If we were to hack the exporter to add this info do you think that would address the problems you have with the exporter? The reason I ask is that while I was going to just hack this in, I went to do that and it turned into a nasty problem, both engineering and CPU wise when exporting. So if this isn't what you wanted then I won't bother to do it.

I'm not asking if this is the same as GPLing BK or giving people free access to 100% of the BK internal data structures, etc. What I'm asking is if this will make the CVS export tree something you can use to get your job done in an efficient way.

I was planning on sending FUSE into Linus in a week or two. That and cpusets are the notable features which are 2.6.12 candidates.

• crashdump seems permanently not-quite-ready
• perfctr works fine, but is rather deadlocked because it is similar-to-but-different-from ia64's perfmon, and might not be suitable for ppc64 (although things have gone quiet on the latter front).
• nfsacl should be OK for 2.6.12 if Trond is OK with it.
• cachefs is a bit stuck because it's a ton of complex code and afs is the only user of it. Wiring it up to NFS would help.
• dm multipath is OK for 2.6.12
• reiser4 is less clear. Once all the review comments have been addressed and we start seeing a bit of vendor pull for it, maybe.

I would certainly vote for FUSE going in. Even if it has some bits that could be improved the code works well. It has been in global use for quite a while. We use it in a production environment on four servers and over 650 workstations to provide a "magic symlink filesystem" (i.e. symlink XYZ points to different place depending on which user looks at it) and we have not experienced any problems. I have also done other testing with a layering fs using fuse and that was very stable (but slower than the symlink approach which is why we went for that).

FUSE may not be perfect but lets face it - which code is? And more importantly a lot of code in the kernel is broken (for at least some people) yet it is in the kernel and FUSE does work well...

Ok, I've fixed up the patch and applied it to a local tree that I've set up to catch these things (it will live at bk://kernel.bkbits.net:gregkh/linux-2.6.11.y until Chris Wright and I set up how we are going to handle all of this.)

Feel free to start pointing stuff like this at me and chris (we'll also be setting up an alias for it.)

I don't think your process works. You never really gave people the time to object. So for that reason you applied the first trivial raid6 thing, and it turned out to be wrong.

I think the patches need to have a rule like "they live outside the sucker tree for at least two days". And during that time, anybody can vote them down (which would move them to "unapplied" status, at which point somebody else might decide that for _their_ tree it's still the right thing to do).

And if at the end of two days, they still haven't gotten enough "yes" votes, they'd go into "limbo" status, with one extra grace-period (ie a reminder on whatever list about a patch that is dying). And if it can't get enough "yeah, sure" votes even after that, it goes into the same "unapplied" list.

In other words, I think this really does want some automation. It shouldn't be fully automated (at the very least, somebody needs to actually check that things patch and fix up the changeset comments etc), but the _rules_ should be automated. Otherwise they'll always be broken because of "_this_ time it's obvious", which is against the point.

This is to announce that Herbert Xu is now taking over my role as co-maintainer of the kernel crypto API.

I've not been able to devote enough time to the integration of async/hardware support, and Herbert, who has been doing excellent work in the networking code for some time, has now thankfully stepped up to help.

Hopefully things will now move forward more quickly in this area.

For those of you who haven't waded through the huge "RFD: Kernel release numbering" thread on lkml to realize that we are now going to start putting out 2.6.x.y releases, here's the summary:

A few of us $suckers will be trying to maintain a 2.6.x.y set of releases that happen after 2.6.x is released. It will contain only a set of bugfixes and security fixes that meet a strict set of guidelines, as defined by Linus at:

http://article.gmane.org/gmane.linux.kernel/283396

Chris Wright and I are going to start working on doing this work, we will have a <SOME_ALIAS>@kernel.org to post these types of bug fixes to, and a set of people we bounce the patches off of to test for "smells good" validation. We will also have a bk-commits type mailing list for those who want to watch the patches flow in, and a bk tree from which changsets can be pulled from.

Chris and I will be hashing all of the details out next Tuesday, and hopefully all the infrastructure will be in place soon. When that happens, we will post the full details on how all of this is going to work. In the meantime, feel free to CC: me and Chris on patches that everyone thinks should go into the 2.6.11.y releases.

But right now, Chris is on a plane, and we don't have the email alias set up, or the proper permissions set up on kernel.org to push changes into the v2.6 directory, but we have a few bugs that are needing to be fixed in the 2.6.11 release. And since our mantra is, "release early and often", here's the first release.

---------------

I've released the 2.6.11.1 patch:
kernel.org/pub/linux/kernel/people/gregkh/v2.6.11/patch-2.6.11.1.gz

With a detailed changelog at:
kernel.org/pub/linux/kernel/people/gregkh/v2.6.11/ChangeLog-2.6.11.1

A bitkeeper tree for the 2.6.11.y releases can be found at:
bk://linux-release.bkbits.net/linux-2.6.11

The diffstat and short summary of the fixes are below.

I'll also be replying to this message with a copy of the patch itself, as it is small enough to do so.

However, it's also true that the thing BK is _worst_ at is cherry-picking things, and having a collection of stuff where somebody may end up vetoing one patch and saying "remove that one".

So it's entirely possible that the proper tool to use for the first level is not BK at all, but the evolved patch-scripts that Andrew uses, in other words:

http://savannah.nongnu.org/projects/quilt

may well be a much better thing to use.

I love BK, but what BK does well is merging and maintaining trees full of good stuff. What BK sucks at is experimental stuff where you don't know whether something should be eventually used or not.

Anything else anyone can think of? Any objections to any of these? I based them off of Linus's original list.

Rules on what kind of patches are accepted, and what ones are not, into the "linux-release" tree.

• It can not bigger than 100 lines, with context.
• It must fix only one thing.
• It must fix a real bug that bothers people (not a, "This could be a problem..." type thing.)
• It must fix a problem that causes a build error (but not for things marked CONFIG_BROKEN), an oops, a hang, or a real security issue.
• No "theoretical race condition" issues, unless an explanation of how the race can be exploited.
• It can not contain any "trivial" fixes in it (spelling changes, whitespace cleanups, etc.)

I would like the requirement: "It must be obviously correct".

In a hundred lines one can put a lot of tricky code and subtle changes. For example, if a security problem necessitates a nontrivial change, it should cause an earlier release of 2.6.x+1 instead of a 2.6.x.y+1.

Goodbye, and that thanks for all the fish ;)

After several years of v4l maintainance I'm going to switch to a new work field and will not be able to spend much time on maintaining video4linux and the drivers, so someone else will have to step in.

I will not suddenly disappear from earth, I will be available for questions and patch reviews for some time, but I'll stop doing active development and most likely will not have the time to act as central patch relay for all video4linux stuff.

Here goes the third pre of v2.4.30.

It contains a small number of scattered fixes, most notably e1000 update, a backport of v2.6's nForce override fix, and SATA update.

The changes which broke "tar --verify" on tapes have been reverted.