Kernel Traffic #329 For 26 Sep 2005

We get regular portability bugs when somebody decides to include linux/irq.h into a driver instead of asm/irq.h. It's almost always a wrong thing to do and, in fact, causes immediate breakage on e.g. arm.

Here's what I'm going to do:

• check current includes of linux/irq.h; e.g. in arch/x86_64 all but two had been 100% useless, one should've been asm/irq.h and one - asm/irq.h + asm/hw_irq.h. The only legitimate user of linux/irq.h on amd64 was asm/hardirq.h.

  Situation elsewhere in arch/* is similar - most of includes are not needed at all.

• remove bogus includes, arch by arch for architectures that live in main tree. Switch ones that should've been asm/irq.h to that form.
• put the current contents of linux/irq.h to asm-generic/hardirq.h (which is what it really is - declarations for hardirq code, relevant on many but not all platforms).
• switch remaining users of linux/irq.h to asm-generic/hardirq.h (again, for architectures that live in main tree)
• replace contents of linux/irq.h with #warning and #include <asm-generic/hardirq.h>.
• after 2.6.14 kill linux/irq.h completely.

Objections? That variant leaves out-of-tree folks with window until 2.6.15 to convert and that's really more than enough...

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.13/2.6.13-mm3/

(temp copy at http://www.zip.com.au/~akpm/linux/patches/stuff/2.6.13-mm3.gz)

• perfctr was dropped. Mikael has ceased development and recommends that the focus be upon perfmon. See http://sourceforge.net/mailarchive/forum.php?thread_id=8102899&forum_id=2237
• There are several performance tuning patches here which need careful attention and testing. (Does anyone do performance testing any more?)
• An update to the anticipatory scheduler to fix a performance problem where processes do a single read then exit, in the presence of competing I/O acticity.
• The size of the page allocator per-cpu magazines has been increased
• The page allocator has been changed to use higher-order allocations when batch-loading the per-cpu magazines. This is intended to give improved cache colouring effects however it might have the downside of causing extra page allocator fragmentation.
• The page allocator's per-cpu magazines have had their lower threshold set to zero. And we can't remember why it ever had a lower threshold.
• Dropped all the virtualisation preparatory patches. Will later pick these up from a git tree which chrisw is running.
• There are still quite a few patches here for 2.6.14 (30-50, perhaps).

Ok, it's been two weeks (actually, two weeks and one day) since 2.6.13, and that means that the merge window is closed. I've released a 2.6.14-rc1, and we're now all supposed to help just clean up and fix everything, and aim for a really solid 2.6.14 release.

Both the diffstat and the shortlog are so big that I can't post them on the kernel mailing list without getting the email killed by the size restrictions, so there's not a lot to say.

alpha, arm, x86, x86-64, ppc, ia64, mips, sparc, um.. Pretty much every architecture got some updates. And an absolutely _huge_ ACPI diff, largely because of some re-indentation.

drm, watchdog, hwmon, i2c, infiniband, input layer, md, dvb, v4l, network, pci, pcmcia, scsi, usb and sound driver updates. People may appreciate that the most common wireless network drivers got merged - centrino support is now in the standard kernel.

On the filesystem level, FUSE got merged, and ntfs and xfs got updated. In the core VFS layer, the "struct files" thing is now handled with RCU and has less expensive locking.

And networking changes.

In other words, a lot of stuff all over the place. Be nice now, and follow the rules: put away the new toys, and instead work on making sure the stuff that got merged is all solid. Ok?

Anybody with git can do the shortlog with

git-rev-list --no-merges --pretty=short v2.6.14-rc1 ^v2.6.13 | git-shortlog | less -S

which is actually pretty informative.

My Compaq Presario R3057EA hangs during ACPI initialization. The last message is "Executing all Device _STA and _INI methods". git bisect told me that:

  66759a01adbfe8828dd063e32cf5ed3f46696181 is first bad commit
  diff-tree 66759a01adbfe8828dd063e32cf5ed3f46696181 (from 049cdefe19f95b67b06b70915cd8e4ae7173337a)
  Author: Chuck Ebbert <76306.1226@compuserve.com>
  Date:   Mon Sep 12 18:49:25 2005 +0200

[PATCH] x86-64: i386/x86-64: Fix time going twice as fast problem on ATI Xpress chipsets

Original patch from Bertro Simul

This is probably still not quite correct, but seems to be the best solution so far.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>

Well I never meant it to be merged, but Andi picked it up from Bugzilla bug #3927, added some bugs of his own, then sent it on.

This bug was mine, though: just checking for vendor == ATI was a bad idea. Current earlyquirk code actually looks at PCI bridges instead of host bridge, so to get an accurate test I guess it needs to look at PCI dev 00:00.0 and check both vendor and device ID. As new models come out they will have to be added one by one.

With a real understanding of what's going on maybe this problem can be solved reliably with generic code, but it's beyond me...

I wrote this Framework for making a .config based on the System Hardwares. It would be a great help if some people would give me their opinion about it.

The readme:
http://www.energyparty.de/ahmad/readme

The Kernel Patch:
www.energyparty.de/ahmad/autoconfig_0_1_patch.tgz (http://www.energyparty.de/ahmad/autoconfig_0_1_patch.tgz)

The sources:
http://www.energyparty.de/ahmad/autoconfig_0_1.tgz

I want to tell why I came to this Idea of doing that. As I was trying to configurate a Kernel for the first time but without any success. Then I thought about it, why this cannot be automatically.

So for the first thought, my spirit of doing this, was for those who doesn't know e.g. what the option "Enable loadable module support" means and if they have to choose that or not. But right now I know that it is not possible to do all the things automatically(like the filesystem, Protokolls...). And that is the reason why I make a Framework, so everybody(I mean the experts) can give theire suggestion what isthe best way to answering, a optione like mention above comes, automatically. So if someone wants to use a new kernel didn't end up desperate and regrated using Linux(a new Kernel).

There are 11 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let us know. If anyone is a maintainer of the proper subsystem, and wants to add a signed-off-by: line to the patch, please respond with it.

These patches are sent out with a number of different people on the Cc: line. If you wish to be a reviewer, please email stable@kernel.org to add your name to the list. If you want to be off the reviewer list, also email us.

Responses should be made by Sat Sep 17 01:00 2005 UTC. Anything received after that time, might be too late.

This might be worth putting in too (has been hit by at least two people in the real world etc.)

tree e3a704026e65bf6fea0c7747f0fb75a506f54127
parent 32a3658533c6f4c6bf370dd730213e802464ef9b
author Alexander Nyberg <alexn@telia.com> Wed, 14 Sep 2005 18:54:06 +0200
committer Linus Torvalds <torvalds@g5.osdl.org> Thu, 15 Sep 2005 00:26:34 -0700

[PATCH] Fix fs/exec.c:788 (de_thread()) BUG_ON

It turns out that the BUG_ON() in fs/exec.c: de_thread() is unreliable and can trigger due to the test itself being racy.

de_thread() does

        while (atomic_read(&sig->count) > count) {
        }
        .....
        .....
        BUG_ON(!thread_group_empty(current));

but release_task does

        write_lock_irq(&tasklist_lock)
        __exit_signal
                (this is where atomic_dec(&sig->count) is run)
        __exit_sighand
        __unhash_process
                takes write lock on tasklist_lock
                remove itself out of PIDTYPE_TGID list
        write_unlock_irq(&tasklist_lock)

so there's a clear (although small) window between the atomic_dec(&sig->count) and the actual PIDTYPE_TGID unhashing of the thread.

And actually there is no need for all threads to have exited at this point, so we simply kill the BUG_ON.

Big thanks to Marc Lehmann who provided the test-case.

Fixes Bug 5170 (http://bugme.osdl.org/show_bug.cgi?id=5170)

Signed-off-by: Alexander Nyberg <alexn@telia.com>
Cc: Roland McGrath <roland@redhat.com>
Cc: Andrew Morton <akpm@osdl.org>
Cc: Ingo Molnar <mingo@elte.hu>
Acked-by: Andi Kleen <ak@suse.de>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

Hey, seeing the intensity of your complaints about _not_ being Cc'd... Better safe than serial maintainer ;-)

OK, so what stuff do you want to be Cc'd on? My current approximation would be arch/arm/*, include/asm-arm/*,drivers/serial/*,include/linux/serial*. Well, and any changes of tty interfaces, if I ever get involved in such... Any additions/removals?

Broadly, it's:

arch/arm/*
drivers/*/arm
drivers/mfd/*	(this fits at the moment, but whether it will in the future depends what else appears there.)
drivers/mmc/*
drivers/serial*	(though only the drivers in there actually using serial_core - unfortunately some non-serial_core drivers appear to have been placed in there.)
include/asm-arm/*
include/linux/8250*
include/linux/serial*
fs/adfs/*

but there are various drivers authored by myself which I'd obviously be interested in CC'ed.

Ok, first off, I hate talking about architecture changes without showing the code for what I am talking about first, but as this is an issue that needs to be agreed upon by a wide range of developers, I'll just write up what I am thinking about doing before actually doing it...

The problem: We need a way to show complex class and class device structures properly in sysfs. Examples of these "complex" views are the input layer's use of different input drivers for devices (usually the same device), the video subsystem view of frame buffer devices and monitors, and even the block layer with it's disks and partitions.

Proposed solutions in the past have been either add the ability to nest classes themselves (as shown in Dmitry's recent proposal for fixing the input layer), or add the ability to nest class_device structures (I think others have tried to do this in the past, sorry for not remembering the specifics.) Both of these proposals don't really solve the real problem, that of the fact that we need to come up with a solution that all of the different subsystems can use properly.

So, here's my take on it. Feel free to tell me what I messed up :)

I would like to add something called "subclasses" for lack of a better term. These subclasses would have both drivers, and devices associated with them. This would show up as the following tree of directories:

        /sys/class/input/
        |-- input0
        |   |-- event0
        |   `-- mouse0
        |-- input1
        |   |-- event1
        |   `-- ts0
        |-- mice
        `-- drivers
            |-- event
            |-- mouse
            `-- ts

Here we have 3 struct class_devices like today, input0, input1, and mice. We also have struct subclass_drivers of event, mouse, and ts. These will create the struct subclass_devices event0, mouse0, event1, and ts0. The "dev" node files will show up in the directories mice, event0, mouse0, event1, and ts0, like you would expect them too.

So, this lets us create a solution for the input subsystem, but will it also work for block and video?

For block, yes, I think so. There is no requirement for a subclass_driver to create the subclass devices. Any code can create and register with the class core a subclass device, just set up the parent pointer and away you go. So, in the following instance:

        /sys/class/block/
        `-- sda
            |-- sda1
            |-- sda2
            `-- sda3

"block" would represent the struct class. "sda" would represent the struct class_device. "sda1", "sda2", and "sda3" would represent the different subclass devices that are bound to the class device "sda".

And yes, before you all point out the class_interface logic, yes, the subclass_driver stuff looks a lot like this idea. Possibly we could just get rid of the interface stuff and use the subclass_driver logic, but I'd have to look at how people are using the interface logic before I can give a confident answer about that.

Hotplug events would be simple with this scheme, the class stuff would remain the same, and the devpath would just point to the subdir (hotplug events would happen for both the class devices and the subclass devices.) And yes, udev and libsysfs would have to be changed to support this, so we better get this right before pushing it out to the world...

But, what about video devices? David and Pat, we talked about this at OLS, but Pat kept the paper we drew on, and the beer we were drinking at the time has made my memory a bit fuzzy as to all of your requirements for the video subsystem. I remember things about frame buffers and monitors and other things like that, but nothing specific, sorry. Could you outline your needs and I'll see if this proposed structure would solve your issues?

Well, that was a very brief summary, of which I know there are lots of questions for the areas I didn't explain well enough. Comments? Criticisms? Want to see the code before commenting?

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.14-rc1/2.6.14-rc1-mm1/ (temp copy at http://www.zip.com.au/~akpm/linux/patches/stuff/2.6.14-rc1-mm1.gz)

• Big reiser4 update which is said to address all review comments.
• Added version 1 of Dmitry's big sysfs/input revamp.
• Lots of tty drivers still don't compile.

master.kernel.org will be offline starting shortly after 15:00 PDT/22:00 UTC today, September 16, 2005; for a move to the Oregon State University Open Source Lab. This should give much better bandwidth and a more reliable backup solution, in addition to access to a real, staffed NOC.

Special thanks to Javier Henderson for volunteering to fly the machine up there by private airplane, thus minimizing downtime.

It is expected to be back online around 12:00 PDT/19:00 UTC tomorrow, Saturday, September 17. The new IP address of the machine will be 140.211.167.34.

We (the -stable team) are announcing the release of the 2.6.13.2 kernel.

The diffstat and short summary of the fixes are below.

I'll also be replying to this message with a copy of the patch between 2.6.13.1 and 2.6.13.2, as it is small enough to do so.

The updated 2.6.13.y git tree can be found at:
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/chrisw/linux-2.6.13.y.git
and can be browsed at the normal kernel.org git web browser:
www.kernel.org/git/ (http://www.kernel.org/git/)

Note: The above info won't be valid until master.kernel.org is back online. In the interim you can find the tarball, patches, changelogs etc. at <http://developer.osdl.org/chrisw/stable/2.6.13.2/>

Summary of changes from v2.6.13.1 to v2.6.13.2

Andi Kleen:
Fix MPOL_F_VERIFY

Chris Wright:
Linux 2.6.13.2

Dave Kleikamp:
jfs: jfs_delete_inode must call clear_inode

Ian Abbott:
USB: ftdi_sio: custom baud rate fix

Linus Torvalds:
hpt366: write the full 4 bytes of ROM address, not just low 1 byte
Sun GEM ethernet: enable and map PCI ROM properly
Fix up more strange byte writes to the PCI_ROM_ADDRESS config word

Manfred Spraul:
forcedeth: Initialize link settings in every nv_open()

Maxim Giryaev:
lost fput in 32bit ioctl on x86-64
Lost sockfd_put() in routing_ioctl()

Patrick McHardy:
Fix DHCP + MASQUERADE problem

Willy Tarreau:
Sun HME: enable and map PCI ROM properly

I am hoping that sending this out to the kernel list is not considered too much of useless spamming, but I promise I wouldn't do thit next time for 0.99.8, if I hear from somebody not to.

Here comes GIT 0.99.7

Done in 0.99.7

Organization

Some commands and most scripts are renamed for consistency.

• We have an official standard terminology list (http://www.kernel.org/pub/software/scm/git/docs/glossary.html) . To match this, commands that operate on index files now have 'index' instead of 'cache' in their names, and ones that download are called 'fetch' instead of 'pull'.
• We used to install most of the commands that happen to be implemented as scripts as 'git-*-script', which was cumbersome to remember and type unless you always used '/git/index.html' wrapper. They lost '-script' suffix from their names.

For now, we install synonyms as symbolic links so that old names continue to work, but they are planned to be removed in 0.99.8 (or later if there are enough objections on the list -- so far I have heard none).

Also ancient environment variables (SHA1_FILE_DIRECTORIES AUTHOR_DATE AUTHOR_EMAIL AUTHOR_NAME COMMIT_AUTHOR_EMAIL COMMIT_AUTHOR_NAME SHA1_FILE_DIRECTORY) are not supported anymore.

New Features and Commands

Downloaders that are not fully git aware have been taught about the mechanism to borrow objects from other repositories via objects/info/alternates the server side may be using. 'git fetch' and 'git pull' commands over rsync and http transport should be able to handle such repositories (But not grafts).

People found interesting cases where the 'stupid' three-way merge mechanism does the wrong thing without noticing. We have two new merge algorithms by Daniel and Fredrik that attempt to do better in such cases. A new 'git merge' command has been introduced to make it easier to experiment with and choose among different merge strategies. Note that 'git pull' still uses the traditional three-way merge after downloading, but it is expected to be switched to use 'git merge' sometime in the future.

Importing from tla archives has been improved and documentated.

'git branch' command acquired '-d' flag to delete a branch that has already been merged into the current branch.

'git bisect' command is easier to use by logging the earlier good/bad choices and make it replayable.

'git repack' has -a' flag to pack the whole repository into a single pack.

'git grep' is a new command to run grep on files '/git/index.html' knows about.

Fixes

• 'git-diff-*' commands used to mark copy/rename incorrectly when an (A,B) => (B,C) rename was made. We said the new B is a copy of old A, not a rename of old A.
• When the user exported CDPATH into environment, 'cd' took scripts to unexpected places. Unset it upfront to guard us.
• 'git format-patch' knows about 'git cherry' and skips patches already merged upstream.
• hopefully plugged memory leak in diffcore-rename properly.
• commit walkers incorrectly assumed having a commit means we have the whole history leading up to it -- which is not true if the previous download was interrupted. As a safety measure, we now only trust the commits that are pointed by the existing refs.
• 'git rev-list' uses a lot less memory.
• The build should be a bit friendlier to Solaris and Darwin now.
• 'git ssh-{push,pull}' are friendlier to tcsh.
• http transport is nicer to caching proxies.
• 'git daemon' port is registered with IANA.
• Many documentation updates.

this is the release of Cogito-0.15. It fixes several minor bugs, and adds a feature or two. The most important thing though is that this depends on Git-core-0.99.7 and uses the new command names. Everyone is encouraged to upgrade at least to this Cogito version in the next few days, since the older Cogito versions likely won't work with the future Git-core releases.

To stay in sync with the Git terminology, Cogito also renames its cg-pull to cg-fetch. Since this is a major naming change (I'm not too happy about it, personally), cg-pull will stay aliased to cg-fetch for at least one (likely two) next major Cogito releases (it also produces a warning when invoked as cg-pull). In the more distant future, cg-pull will slowly become the new name of cg-update, to make it confusing.

While at it, we also renamed the *-id scriptlets to cg-*-id. Other notable stuff is cg-init respecting the ignore rules, and better UI for cg-add wrt. directories (including cg-add -r support).

Now let's see what the usual bug-right-after-release (major release, so a major bug?) will be this time.

Ahoy landlubbers!

Here be t' Linux-2.6.14-rc2 release.

Not a whole lot o' excitement, ye scurvy dogs, but it has t' ALSA, LSM, audit and watchdog merges that be missed from -rc1, and a merge series with Andrew. But on t' whole pretty reasonable - you can see t' details in the shortlog (appended).

Arrr!