Kernel Traffic #306 For 11 Apr 2005

i have released the -V0.7.41-00 Real-Time Preemption patch (merged to 2.6.12-rc1), which can be downloaded from the usual place:

http://redhat.com/~mingo/realtime-preempt/

the biggest change in this patch is the merge of Paul E. McKenney's preemptable RCU code. The new RCU code is active on PREEMPT_RT. While it is still quite experimental at this stage, it allowed the removal of locking cruft (mainly in the networking code), so it could solve some of the longstanding netfilter/networking deadlocks/crashes reported by a number of people. Be careful nevertheless.

there are a couple of minor changes relative to Paul's latest preemptable-RCU code drop:

• made the two variants two #ifdef blocks - this is sufficient for now and we'll see what the best way is in the longer run.
• moved rcu_check_callbacks() from the timer IRQ to ksoftirqd. (the timer IRQ still runs in hardirq context on PREEMPT_RT.)
• changed the irq-flags method to a preempt_disable()-based method, and moved the lock taking outside of the critical sections. (due to locks potentially sleeping on PREEMPT_RT).

to create a -V0.7.41-00 tree from scratch, the patching order is:

http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.tar.bz2
http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.12-rc1.bz2
http://redhat.com/~mingo/realtime-preempt/realtime-preempt-2.6.12-rc1-V0.7.41-00

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.12-rc1/2.6.12-rc1-mm3/

• Mainly a bunch of fixes relative to 2.6.12-rc1-mm2.
• Again, we'd like people who have had recent DRM and USB resume problems to test and report, please.
• The bk-ide-dev tree is back after a couple of weeks of difficulties.
• Jeff asks that anyone who has had problems with the Silicon Image SATA drivers test sata_sil-corruption--lockup-fix.patch, which is included in this kernel.

I'm working on cleaning-up the JFFS3 compression stuff. JFFS3 contains a number of compressors which actually shouldn't be there as they are platform-independent and generic. So we want to move them to the generic part of the Linux kernel instead of storing them in fs/jffs2/. And we were going to use CryptoAPI to access the compressors.

But I've hit on a problem - CryptoAPI's compression method isn't flexible enough for us.

CryptoAPI's compress method is:

int crypto_compress(struct crypto_tfm *tfm, const u8 *src, unsigned int slen, u8 *dst, unsigned int *dlen);

*src - input data;
slen - input data length;
*dst - output data;
*dlen - on input - output buffer length, on output - the length of the compressed data;

The crypto_compress() API call either compresses all the input data or returns error.

In JFFS2 we need something more flexible. Te following is what we want:

int crypto_compress_ext(struct crypto_tfm *tfm, const u8 *src, unsigned int *slen, u8 *dst, unsigned int *dlen);

*src - input data;
*slen - on input - input data length, on output - the amount of data that were actually compressed.
*dst - output data;
*dlen - on input - output buffer length, on output - the length of the compressed data;

This would allow us (and we really need this) to provide a large input data buffer, a small output data buffer and to ask the compressor to compress as much input data as it can to fit (in the compressed form) to the output buffer. To put it differently, we often have a large input, and several smalloutput buffers, and we want to compress the input data to them.

I offer to extend the CryptoAPI and add an "extended compress" API call with the above mentioned capabilities. We might as well just change the crypto_compress() and all its users.

Alternatively, we may create some kind of "Compression API" but I don't like this idea...

I've noticed that since recent kernel versions, it's not possible anymore to use simultaneously new and old megaraid driver.

It seems to have been introduced by that changeset: http://linux.bkbits.net:8080/linux-2.5/diffs/drivers/scsi/megaraid/Kconfig.megaraid@1.1??nav=index.html| src/.|src/drivers|src/drivers/scsi|src/drivers/scsi/megaraid|hist/drivers/scsi/megaraid/Kconfig.megaraid (http://linux.bkbits.net:8080/linux-2.5/diffs/drivers/scsi/megaraid/Kconfig.megaraid@1.1??nav=index.html|src/.|src/drivers|src/drivers/scsi|src/drivers/scsi/megaraid|hist/drivers/scsi/megaraid/Kconfig.megaraid)

It particularly makes life of people developing kernel for distro difficult as it forces them to drop support for legacy hardware which is working just fine with 2.6, or to patch their own kernel build. As well it prevents simultaneous usage of new and old cards in the same system.

Would you consider to apply the following patch proposed by Thierry Vignaud as a solution for the MandrakeSoft kernel in the mainstream 2.6 kernel?

RFC: This is a work-in-progress (WIP), not yet completed.

A few people dislike that the Networking Options menu is inside the Device Drivers/Networking menu. This patch moves the Networking Options menu to immediately before the Device Drivers menu, renames it to "Networking options and protocols", & moves most protocols to more logical places (IMHOOC).

The reasons that it is still WIP are:

• I'd like to see all of the sub-menus done in the same style;
• IrDA & Bluetooth subsystems include protocols & drivers, yet they are displayed under Networking protocols. I don't see much good reason to split them up. (See, this is an example of why the Networking Options and Network Drivers were close together....)
• some Networking options need to be qualified with CONFIG_NET

The new Networking menu looks unstructured. And the net/Kconfig file contains a lot of config snippets that does not belong there. So I took a stamp on it with focus on:

• Move config bits to appropriate places, creating several new Kconfig files
• Made uses of menus more consistent at least on first and second level
• Move submenu to the top
• Rename top menu to "Networking" and located it just before "File systems"

The patch became much larger. The win is that the top-level net/Kconfig contains much less cruft.

Here are a few more suggestions for you to consider.

• in Networking support, move Network testing and Netpoll support to the end of the menu (basically put the devel. tools toward the bottom of the menu)
• I would rather not "hide" Amateur Radio, IrDA, and Bluetooth in the Networking protocols area, but have them near 802.1x and ATM in the top-level Networking support menu. How does that sound to you?

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.12-rc1/2.6.12-rc1-mm4/

• New reiserfs4 code drop
• Various new fixes and cleanups. Am still interested in hearing how people are going with the DRI problems and the PM resume problems.

The following 3 patches change the userspace interface. Backward compatibility is not retained, the library must be upgraded to 2.3-pre1 or later. The library will support both the old and the new ABI versions. Filesystems dynamically linked with libfuse don't need to be recompiled.

The main reason for the change is that the current interface was not compatible between 32bit and 64bit modes of dual architecures.

The patches are:

1/3 - Add padding to structures to make sizes the same on 32bit and 64bit archs

2/3 - Add offset to fuse_dirent structure. This will make the readdir interface more flexible

3/3 - Change ABI major version from 5 to 6, and check if userspace supports the new interface

I humbly submit configfs. With configfs, a configfs config_item is created via an explicit userspace operation: mkdir(2). It is destroyed via rmdir(2). The attributes appear at mkdir(2) time, and can be read or modified via read(2) and write(2). readdir(3) queries the list of items and/or attributes.

The lifetime of the filesystem representation is completely driven by userspace. The lifetime of the objects themselves are managed by a kref, but at rmdir(2) time they disappear from the filesystem.

configfs is not intended to replace sysfs or procfs, merely to coexist with them.

An interface in /proc where the API is:

# echo "create foo 1 3 0x00013" > /proc/mythingy

or an ioctl(2) interface where the API is:

        struct mythingy_create {
                char *name;
                int index;
                int count;
                unsigned long address;
        }

        do_create {
                mythingy_create = {"foo", 1, 3, 0x0013};
                return ioctl(fd, MYTHINGY_CREATE, &mythingy_create);
        }

becomes this in configfs:

        # cd /config/mythingy
        # mkdir foo
        # echo 1 > foo/index
        # echo 3 > foo/count
        # echo 0x00013 > foo/address

Instead of a binary blob that's passed around or a cryptic string that has to be formatted just so, configfs provides an interface that's completely scriptable and navigable.

Patch is against 2.6.12-rc1-bk3.

http://oss.oracle.com/~jlbec/files/configfs/2.6.12-rc1-bk3/configfs-2.6.12-rc1-bk3-1.patch

"actually create", huh? :)

In the trivial case Joel describes, the item is almost certainly allocated during "# mkdir foo" when the subsystem will get a ->make_item() call for the 'mythingy' group it registerd. The various attribute writes then find the item by following their configfs_attribute argument to the item that its embedded in.

But I bet you're not really asking about creation. I bet you're wondering how the kernel will know when enough attributes have been filled and that it's safe to use the object. Misguided items could assign magical ordering to the attribute filling such that once a final attribute is set, and others have been set, the item goes live. That's what ocfs2 does now, sadly, but certainly not as a long-term solution.

The missing piece is the 'commit_item' group operation that is yet to be implemented. The intent is to have a directory of pending items that can have their attributes filled before being rename()ed into a directory of items that are in active use. The commit_item() call that hits at rename() would give the kernel the chance to refuse the item because attributes haven't been filled in or conflict with existing items, or whatever.

HP has most graciously donated a pair of DL585 quad Opteron servers with 24 GB of RAM and 10 TB of disk using a pair of MSA-30 arrays for each server. The first ones of these servers was officially put in service today; the next one will be put in service next week. Each server is in a different ISC colo, connected to the Internet via gigabit fiber links.

Consequently, we should now see incredibly much better performance from kernel.org. Huge thanks to HP for the new hardware, and huge thanks to ISC for letting us quadruple our rack space requirements from 5U to 2x10U. We'll be saturating those links in no time :)

A few additional notes:

• The IP addresses for kernel.org have changed. When the second server gets deployed next week, most of the kernel.org addresses, e.g. ftp.kernel.org, will be round-robins. The individual servers can be specified as ftp1.kernel.org and ftp2.kernel.org.
• *** If you run a service from kernel.org that uses DNS entries beyond our control, please see the email I sent out; if you didn't get it, contact me. ***
• The old IP addresses will be turned off later this week.

Below, find inotify 0.22, against 2.6.12-rc1.

This release introduces a conversion in our primary locking from spinlocks to semaphores. Semaphores are a more natural fit for our code, which synchronizes with user-space, thus we clean up a bit of code with a net reduction of 63 lines. Also, I was able to remove the GFP_ATOMIC allocation.

I did this as a bit of an experiment, not to fix any specific problem, and I now think it is the right way to go.

This release also fixes a small bug in the coalescing code, which could of mistakenly dropped a move event. We now verify that the cookies match before coalescing.

No, inotify does not support watching directories recursively. I would love to add it, but it would be a mess to do inside of the kernel.

Making it easy and efficient to watch a full tree, however, was a goal of inotify. Beagle, a personal indexing infrastructure, watches the user's entire home directory.

You could never do this in dnotify because you would run out of file descriptors and pin every file.

In inotify, it is not hard to write a simple recursive loop to add a watch to each directory starting at a given path. It can even be done in an atomic fashion. See

http://mail.gnome.org/archives/dashboard-hackers/2004-October/msg00022.html

wherein I publish such an algorithm.

Currently Linux 2.6 assumes the BIOS (or firmware) sets the master abort mode flag on PCI bridge chips in a coherent fashion. This is not always the case and the consequences of getting this flag incorrect can cause hardware to fail or silent data corruption. This patch lets the user override the BIOS master abort setting at boot time and the distro maintainer to set a default according to their target audience.

The comments in the patch are probably a bit too verbose, but I think it is a good patch to start discussions around. If it is decided that something should be done about this problem, this patch could be included in a -mm release and migrate into Linus's kernel as appropriate.

This incarnation of the patch has had minimal testing. For our internal kernels, we always force the master abort mode to 1 and then let the device drivers for hardware we know can't handle target aborts switch the master abort mode to 0. This does not seem appropriate for general release.

Some background for those who do not spend most of their waking hours exploring buses and what can go wrong.

The master abort flag tells a PCI bridge what to do when a bus master behind the bridge requests the bus and the bridge is unable to get the bus. With the flag clear, for master reads the bridge returns all 0xff's (hence silent data corruption) and for master writes, it throws the data away. With the bit set, the bridge sends a target abort to the master. This can only happen when the system is heavily loaded.

The problem with always setting the bit is that some PCI hardware, notably some Intel E-1000 chips (Ethernet controller: Intel Corporation: Unknown device 1076) cannot properly handle the target abort bit. In the case of the E-1000 chip, the driver must reset the chip to recover. This usually leads to the machine being off the network for several seconds, or sometimes even minutes, which can be bad for servers.

I even have a single motherboard with both a device that cannot handle the target abort and an IDE controller that can handle the target abort behind the same bridge. For this motherboard, I have to choose the lesser of two evils, network hiccups or potential data corruption. For the record, I have seen both occur. Other people may make wish to make a different choice than we did, hence this patch allows the user to choose the mode at runtime.