Kernel Traffic #323 For 3 Sep 2005

The reboot code paths seems to be suffering from 15 years of people only looking at the code when it breaks. The result is there are several code paths in which different callers expect different semantics from the same functions, and a fair amount of imperfect inline replication of code.

For a year or more every time I fix one bug in the bug fix reveals yet another bug. In an attempt to end the cycle of bug fixes revealing yet more bugs I have generated a series of patches to clean up the semantics along the reboot path.

With the callers all agreeing on what to expect from the functions they call it should at least be possible to kill bugs without more showing up because of the bug fix.

My primary approach is to factor sys_reboot into several smaller functions and provide those functions for the general kernel consumers instead of the architecture dependent restart and halt hooks.

I don't expect this to noticeably fix any bugs along the main code paths but magic sysrq and several of the more obscure code paths should work much more reliably.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.13-rc3/2.6.13-rc3-mm3/

• Added the anonymous pagefault scalability enhancement patches.

  I remain fairly dubious about this - it seems a fairly specific and complex piece of work to speed up one extremely specific part of one type of computer's one type of workload. Surely there's a better way :(

  The patches at present spit warnings or don't compile on lots of architectures. x86, x86_64, ppc64 and ia64 are OK.

• There's a pretty large x86_64 update here which naughty maintainer wants in 2.6.13. Extra testing, please.
• Dropped git-net.patch (davem's net devel tree). I'm seeing weird TCP hangs. I'm fairly sure they're present in mainline, but was unable to reproduce it without git-net.patch when I was actually trying.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.13-rc4/2.6.13-rc4-mm1/

• Dropped areca-raid-linux-scsi-driver.patch and iteraid.patch. People who need these can get them from 2.6.13-rc3-mm3.
• Dropped the CKRM patches. I don't think they were doing much in -mm and we didn't find many problems with them anyway.
• Dropped the connector patches: turns out that we no longer have a netlink slot available for them anyway.

How to apply the -rc, -git, -mm and the 2.6.x.y (-stable) patches is a quite frequently asked question on LKML and elsewhere. Since so many people seem to be confused by this I gathered it ought to be properly documented once and for all so we a) get more people testing those trees and b) get asked this question less often. So, I sat down and wrote such a document.

Below is a patch to add a new file "/kernel-traffic/applying-patches.txt" to Documentation/ This document describes each of the trees and gives examples on how to apply the various patches.

Looking forward to your feedback (and possible inclusion).

I guess this document could also be placed somewhere on kernel.org and linked to from the front page so that people downloading the various patches will have this information available at their fingertips.

Can we have more whitespace?

You either have very long paragraphs, or no whitespace between them: I can't quite decide which one.

So leave an empty line between paragraphs (and if you already do, you need to split them ;) because it's very tiring to not have a nice break every once in the flow of text.

Long paragraphs may be acceptable in fictional literary work that you read without thinking about what you're reading. There you get into the "flow" of the text, and you hopefully don't need to have very many visual breaks to keep as acnhor-points. However, the same is certainly not true in technical text, especially something like this where you're trying to explain somethign that the recipient doesn't ncessarily know.

My rule of thumb is that if you don't have a new paragraph roughly every five or six lines, it's likely problematic. Maybe I have a shorter attention span than most, but I don't think so - I just find it much easier to read text that is nicely broken up, and when it's a "pure ASCII" medium the only break that works well is an empty line (possibly with indentation for further visual help - although in this context indentation tends to be used for a separate issue: examples etc, and is not good for paragraphs).

And since we have a single empty line implying paragraph breaks, feel free to use multiple empty lines to imply "bigger" breaks (you seem to do this already).

This email was written with an average paragraph length of 4 lines.

Please give this a try and commit to -mm or mainline, if approved.

Summary:

This speeds up directory reads for large FAT partitions, if the buffercache has to be filled from the drive. Following values were taken from:

$ time find path_to_freshly_mounted_fat > /dev/null

on an otherwise idle system.

FAT with 16KB Clusters on IDE attached drive: Factor 2

FAT with 32KB Clusters on USB2 attached drive: Factor 10 (!)

Its less than 1/10 slower, if the buffercache is uptodate.

The patch touches 3 areas:

• fat_bmap() returns the sector's offset in the cluster or a negativ error code instead of 0 or the negativ error code. It's callers are changed accordingly.

• fat__get_entry() calls sb_breadahead() to readahead a whole cluster, if the requested sector is the first one in a cluster. It is usefull to do this, because on FAT directories occupy whole clusters.

  Readahead is only done, if the cluster's first sector is not uptodate to avoid overhead, when the buffer cache is already uptodate. Note that on memory pressure, the maximal byte count wasted (read: has to be red from disk twice) is 1 cluster's size. Thats 64KB.

At kernel summit I pledged to put out weekly where-we're-at summaries, mainly so that subsystem maintainers could get an estimate of when the next major kernel release will be, so they can plan their merge windows.

Of course, I don't have a clue when the next release will be because the timing is driven by perception-of-stability. But I can guess.

Current kernel version:

2.6.13-rc5-git3

ETA for 2.6.13:

August 12-19

Status of subsystem trees:

  3190002    git-acpi.patch
  68299      git-alsa.patch
  96         git-arm.patch
  100        git-arm-smp.patch
  31799      git-audit.patch
  113        git-cpufreq.patch
  14257      git-cryptodev.patch
  107590     git-drm.patch
  122        git-drm-via.patch
  112        git-ia64.patch
  109        git-input.patch
  32153      git-ipsec.patch
  4781       git-jfs.patch
  41254      git-kbuild.patch
  21659      git-libata-adma-mwi.patch
  17964      git-libata-chs-support.patch
  36338      git-libata-ncq.patch
  21535      git-libata-passthru.patch
  3944       git-libata-promise-sata-pata.patch
  91         git-libata-upstream-fixes.patch
  85         git-libata-upstream.patch
  96         git-mmc.patch
  7729       git-mtd.patch
  255866     git-netdev-chelsio.patch
  10606      git-netdev-e100.patch
  1290162    git-netdev-ieee80211-wifi.patch
  48220      git-netdev-sis190.patch
  4899       git-netdev-smc91x-eeprom.patch
  132        git-netdev-upstream-fixes.patch
  178991     git-netdev-upstream.patch
  955        git-net-gregkh-i2c-w1-netlink-callbacks-fix.patch
  443989     git-net.patch
  101        git-nfs.patch
  119        git-ntfs.patch
  1321859    git-ocfs2.patch
  88683      git-scsi-block.patch
  331078     git-scsi-misc.patch
  121        git-scsi-rc-fixes.patch
  34584      git-serial.patch
  1890       git-sparc64.patch

That's all 2.6.14 material - no subsystem merges pending.

Open bugs:

This is based on my reading of what's real and of what's worth attending to. Quite a few things get culled up-front.

There are several emailed bug reports which are probably live bugs but they have gone stale hence I have asked the reporters to raise bugzilla reports, so more post-2.6.12 bugs will appear as the reporters retest 2.6.13-rc6.

I really don't want to have to track bugs which aren't in bugzilla. If an emailed bug report comes in and we can address it within a few days and a few emails then fine. If that doesn't happen I'll be asking reporters to open bugzilla reports.

All bugs reported prior to the 2.6.12 release have been discarded. I'll henceforth track bugs across succeeding major release, so this list will just grow.

There are 60 bugs here. They're almost all post-2.6.12 regressions. Longer-term we simply have to do better than this, else we'll stabilise at a pretty buggy level. No matter what process changes we make, the bottom line is that developers/maintainers will need to spend more of their time working with reporters on fixing bugs.

If you wish to provide an update on one of the below bugs, please do it in bugzilla if it's bugzilla-based. Or in reply to the original email thread if it's email-based. Failing all that, please at least rewrite the Subject: when replying so I don't lose my mind, thanks.

Lots of USB stuff, some ACPI and we seem to have broken parallel-IDE.

idr_remove

Looks like SELinux is removing IDR entries which don't exist.

[Bugme-new] [Bug 4768] New: Screen appears at mid-right section
http://bugzilla.kernel.org/show_bug.cgi?id=4768

[Bugme-new] [Bug 4769] New: PC104plus BT848 card stop working in
http://bugzilla.kernel.org/show_bug.cgi?id=4769

[Bugme-new] [Bug 4771] New: Linux 2.6.11.10 + reiserfs + usrquota,
http://bugzilla.kernel.org/show_bug.cgi?id=4771

Re: 2.6.12-rc6-mm1

This is the

note: mono[26736] exited with preempt_count 1
scheduling while atomic: mono/0x10000001/26736

x86_64 bug.

[Bugme-new] [Bug 4773] New: 8139too module got "eth0: transmit
http://bugzilla.kernel.org/show_bug.cgi?id=4773

[Bugme-new] [Bug 4776] New: uhci_hcd: host controller halted,
http://bugzilla.kernel.org/show_bug.cgi?id=4776

[Bugme-new] [Bug 4777] New: Cyrix 6x86MX PR200+ incorrectly
http://bugzilla.kernel.org/show_bug.cgi?id=4777

[Bugme-new] [Bug 4779] New: amd64: raw1394 returns EINVAL
http://bugzilla.kernel.org/show_bug.cgi?id=4779

[Bugme-new] [Bug 4781] New: Conservative governor makes me lose my
http://bugzilla.kernel.org/show_bug.cgi?id=4781

[Bugme-new] [Bug 4791] New: ACPI + SERIO_I8042 bug/conflict
http://bugzilla.kernel.org/show_bug.cgi?id=4791

[Bugme-new] [Bug 4793] New: Battery reads result in high cpu usage
http://bugzilla.kernel.org/show_bug.cgi?id=4793

[Bugme-new] [Bug 4823] New: alsa modules snd_virmidi & snd_mpu401
http://bugzilla.kernel.org/show_bug.cgi?id=4823

variable used before set

Silly scsi bug.

2.6.12-rc6 mm->total_vm accounting imbalance?

Might be fixed

[Bugme-new] [Bug 4829] New: Problem with sym53c8xx_2 and target
http://bugzilla.kernel.org/show_bug.cgi?id=4829

[Bugme-new] [Bug 4831] New: Bad soundcard patches for Intel ICH4
http://bugzilla.kernel.org/show_bug.cgi?id=4831

[Bugme-new] [Bug 4851] New: x86-64 userspace random segfaults and
http://bugzilla.kernel.org/show_bug.cgi?id=4851

AACRAID failure with 2.6.13-rc1

Might be fixed

[Bugme-new] [Bug 4853] New: pf: Oops with Imation SuperDisk
http://bugzilla.kernel.org/show_bug.cgi?id=4853

[Bugme-new] [Bug 4860] New: sata_sx4 doesn't recognize Promise
http://bugzilla.kernel.org/show_bug.cgi?id=4860

[Bugme-new] [Bug 4866] New: motherboard ga-k8nf-9: ehci_hcd doesn't
http://bugzilla.kernel.org/show_bug.cgi?id=4866

[Bugme-new] [Bug 4869] New: Screen stays blank upon resume
http://bugzilla.kernel.org/show_bug.cgi?id=4869

[Bugme-new] [Bug 4880] New: dpt_i2o.c does not register itself with
http://bugzilla.kernel.org/show_bug.cgi?id=4880

[Bugme-new] [Bug 4888] New: kernel 2.6.12.2 will only do audio when
http://bugzilla.kernel.org/show_bug.cgi?id=4888

[Bugme-new] [Bug 4916] New: USB mouse stops working after inserting
http://bugzilla.kernel.org/show_bug.cgi?id=4916

[Bugme-new] [Bug 4917] New: Lacie 250Go USB
http://bugzilla.kernel.org/show_bug.cgi?id=4917

[Bugme-new] [Bug 4919] New: APM resume: freeze at disk access
http://bugzilla.kernel.org/show_bug.cgi?id=4919

[Bugme-new] [Bug 4920] New: IDE CD Driver not able to read audio
http://bugzilla.kernel.org/show_bug.cgi?id=4920

[Bugme-new] [Bug 4929] New: problem with aic7xxx driver on 2.6.x
http://bugzilla.kernel.org/show_bug.cgi?id=4929

[Bugme-new] [Bug 4940] New: Repeatable Kernel Panic on Adaptec
http://bugzilla.kernel.org/show_bug.cgi?id=4940

[Bugme-new] [Bug 4944] New: uhci_hcd hangs on intel 810 when
http://bugzilla.kernel.org/show_bug.cgi?id=4944

[Bugme-new] [Bug 4950] New: battery state don't change
http://bugzilla.kernel.org/show_bug.cgi?id=4950

[Bug 4951] System freezes with SMP kernel on AMD X2 4600
http://bugzilla.kernel.org/show_bug.cgi?id=4951

[Bugme-new] [Bug 4962] New: ?
http://bugzilla.kernel.org/show_bug.cgi?id=4962

[Bugme-new] [Bug 4963] New: Encounter this Kernel oops while
http://bugzilla.kernel.org/show_bug.cgi?id=4963

[Bugme-new] [Bug 4965] New: Dual-head not working correctly with
http://bugzilla.kernel.org/show_bug.cgi?id=4965

[Bugme-new] [Bug 4966] New: ehci_hcd on x86_64 causes more than
http://bugzilla.kernel.org/show_bug.cgi?id=4966

.13 mptfusion changes.

mpt-fusion needs initrd rework and will break existing userspace.

sysfs double entry.

duplicate entries in /sys/devices/system/

[Bugme-new] [Bug 4968] New: sata_nv buffer I/O errors
http://bugzilla.kernel.org/show_bug.cgi?id=4968

[Bugme-new] [Bug 4971] New: dual head and 2.6.13rc4
http://bugzilla.kernel.org/show_bug.cgi?id=4971

Re: [linux-usb-devel] Fw: BUG: Kernel panic when disconnecting Edirol USB2 audio interface

2.6.13-rc4 - kernel panic - BUG at net/ipv4/tcp_output.c:918

[Bugme-new] [Bug 4974] New: ide2usb adapter do not working with usb
http://bugzilla.kernel.org/show_bug.cgi?id=4974

Re: 2.6.13-rc4 use after free in class_device_attr_show

[Bugme-new] [Bug 4978] New: Failed to allocate mem resource for AGP
http://bugzilla.kernel.org/show_bug.cgi?id=4978

[Bugme-new] [Bug 4981] New: changes in 2.6.12-rc1 causes ati-remote
http://bugzilla.kernel.org/show_bug.cgi?id=4981

[Bugme-new] [Bug 4982] New: Usenet gateway crashes under heavy
http://bugzilla.kernel.org/show_bug.cgi?id=4982

[Bugme-new] [Bug 4983] New: Paralell ZIP disappearing
http://bugzilla.kernel.org/show_bug.cgi?id=4983

Re: aio-fix-races-in-callback-path.patch added to -mm tree

[Bugme-new] [Bug 4992] New: Power off stops
http://bugzilla.kernel.org/show_bug.cgi?id=4992

lpfc: system freezing if FC connection is broken under load

MCE problem on dual Opteron

PROBLEM: reproductible oops, NFS, gigabit network, x86_64

b44 transmit timeout with kernel 2.6

[Bugme-new] [Bug 4998] New: "init 0" broken between 2.6.12 and
http://bugzilla.kernel.org/show_bug.cgi?id=4998

Regression: radeonfb: No synchronisation on CRT with linux-2.6.13-rc5

[Bugme-new] [Bug 5000] New: fan always on after waking from swsusp
http://bugzilla.kernel.org/show_bug.cgi?id=5000

Re: tungsten t5 doesn't sync anymore with kernel 2.6.12

[Bugme-new] [Bug 5001] New: USB: usblp must be unloaded before Scanner is available, MFP Epson CX3650 (& Epson CX6600)
http://bugzilla.kernel.org/show_bug.cgi?id=5001

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.13-rc5/2.6.13-rc5-mm1/

(Grab it from http://www.zip.com.au/~akpm/linux/patches/stuff/2.6.13-rc5-mm1.gz until the kernel.org mirrors catch up)

• Added the git-scsi-iscsi.patch tree: iSCSI drivers (James Bottomley)
• This kernel is broken on ia64: the spinlock consolidation patch needs fixing.
• The acpi development tree is back in -mm.
• Dropped cachefs and the cachefs-for-AFS patches. These get in the way of memory management testing a bit, and they're being redone anyway.

Thanks Andrew, for including the vfat speedup patch.

It has really improved a lot the performance of access to directories having many subdirectories in an external Firewire HD that I have.

I'd say that if others don't have problems with it, then it should be in 2.6.13, as far as I am concerned.

BTW, everything is working fine with the sbp2/ieee1394 drivers that are in the mm tree.

It seems that there are some issues with ALSA, though. I will report back as soon as I see if these are userland problems or not (it worked fine with vanilla 2.6.13-rc5).

For unmaintained drivers or drivers maintained by the subsystem maintainers (which can be the reason why there's no entry for this driver) contact the subsystem maintainer.

Unmaintained subsystems are a problem.

I've already started contacting subsystem maintainers that seem to be inactive, and I have some restructuring of MAINTAINERS (tree structure with drivers behind subsystems) on my TODO list.

Here goes v2.4.32-pre3. It contains several v2.6 backported fixes, amongst them:

• libata update
• networking fixes (most notably Netfilter)

Please refer to the changelog for the detailed information

Summary of changes from v2.4.32-pre2 to v2.4.32-pre3
============================================

Aaron Grothe:
Fix XTEA implementation

Alan Stern:
Revert USB UHCI changes

Aleksey Gorelov:
Fix incorrect Asus k7m irq router detection

bdupree@techfinesse.com:
Fix Alpha AXP Cabriolet build

deep-blue@t-online.de:
fix RedBlackTree rb_next/rb_prev functions

Harald Welte:
Remove bogus declaration of ipt_mutex

Horms:
ppc32: stop misusing ntps time_offset value

Jeff Garzik:
libata: update to 2.6.x latest

John W. Linville:
i810_audio: use MMIO on systems that support it
i810_audio: offset LVI from CIV to avoid stalled start

Ju, Seokmann:
megaraid2 v2.10.10.1

Lars Marowsky-Bree:
fix oops when starting md multipath 2.4 kernel

Linus Torvalds:
PATCH: Fix outstanding gzip/zlib security issues

Marcelo Tosatti:
Change VERSION to v2.4.32-pre3
Merge rsync://rsync.kernel.org/.../davem/net-2.4

Patrick McHardy:
[NETFILTER]: Use correct byteorder in ICMP NAT
[NETFILTER]: Fix potential memory corruption in NAT code (aka memory NAT)
[NETFILTER]: Fix ip6t_LOG sit tunnel logging
[NETFILTER]: Restore netfilter assumption in IPv6 multicast
[NETFILTER]: Fix deadlock with ip_queue/ip6_queue
[NETFILTER]: Ignore PSH on SYN/ACK in ipt_unclean

Willy TARREAU:
fix potential NULL dereferences in several serial driver methods (Julien Tinnes)

Folks at Dell have donated a new machine to be VGER, and folks at RedHat have installed it into co-location facility with 1000Mbps network connection into the machine.

This update got considerable performance increase into the machine for our list loads. In terms of Bogomips around 7-8, but for actual loads nearly twice as much.

We did system switchover last weekend, and nobody reacted trulu adversely. Probably nobody noticed it either. :-)

Since there's been some recent interest in the subject, I thought I would post the PATA todo list for libata. Some of these items are from my memory, and some are from a list Alan was kind enough to create. The items verbatim from Alan are prefixed "Alan: ".

1. Locked device/host speed

   To support devices such as those in the ide/pci/generic.c list, we need to ensure that libata -never- attempts to change the device speed (ata_dev_set_xfermode should be avoided).

2. Simplex DMA

   PCI IDE specification has a 'simplex' DMA bit, which should be tested. Simplex means that only one command can be outstanding, for BOTH port0 and port1, at any given time.

   Possibly some hosts also need Simplex DMA, but may not assert the standard PCI IDE Simplex DMA capability bit. I don't know.

3. Speed change on error

   Downshift device to a slower UDMA speed, and eventually from DMA->PIO, as errors persist. There is no 'specified way' to do this, this is purely hueristic.

4. Alan: Command filter

   Alan -- explanation?

   I know one line item here, at least: Promise controllers snoop SET FEATURES - XFER MODE command. We must stop command processing on ALL ports when this command is issued, to avoid corruption.

5. Alan: MWDMA broken still? is piix doc correct?
6. Alan: some PATA LBA48 devices cannot do > 256 sectors.
7. Alan: Some ALi requires LBA48 be done via PIO. LBA28 DMA is OK.

8. ATAPI device CDB interrupt

   Some older ATAPI devices require the OS driver to wait for an interrupt, after CDB is written, before command processing proceeds.

9. ATAPI devices may delay setting DRQ=1 for up to 3ms.

   Make sure we honor the delay noted in IDENTIFY PACKET DEVICE word 0.

10. ATAPI DMA alignment (discussed elsewhere)

    Needed even for PATA, AFAICT.

This is the start of the stable review cycle for the 2.6.12.5 release. There are 8 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let us know. If anyone is a maintainer of the proper subsystem, and wants to add a signed-off-by: line to the patch, please respond with it.

These patches are sent out with a number of different people on the Cc: line. If you wish to be a reviewer, please email stable@kernel.org to add your name to the list. If you want to be off the reviewer list, also email us.

Responses should be made by Sat, Aug 13, 22:00 UTC. Anything received after that time, might be too late.

Why does this 6 year old bug have to be fixed in the 2.6.12 stable series? Doesn't the patch violate this stable series rule?

• It must fix a real bug that bothers people (not a, "This could be a problem..." type thing.)

Maybe the motivation was just missing from the patch description?