Kernel Traffic #277 For 17 Oct 2004

By Zack Brown

Table Of Contents

Mailing List Stats For This Week

We looked at 4775 posts in 27421K.

There were 756 different contributors. 442 posted more than once. 245 posted last week too.

The top posters of the week were:

1. Linux 2.6.9-rc1-mm1 Released; Some Scheduler Discussion

26 Aug 2004 - 8 Sep 2004 (72 posts) Archive Link: "2.6.9-rc1-mm1"

Topics: Kernel Release Announcement, Ottawa Linux Symposium

People: Andrew MortonCon KolivasMartin J. BlighRick LindsleyRafael J. Wysocki

Andrew Morton announced Linux 2.6.9-rc1-mm1, saying:

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm1/

I fixed a few things, but binfmt_elf.c is a mess.

It's not clear how to apply the same debug check to put_user() and friends.

Regarding the low feedback on nicksched, Con Kolivas replied:

That's because most people aren't interested in a new cpu scheduler for 2.6. The current one works well enough in most situations and people aren't trying -mm to fix their interactive problems since they are few and far between. The only reports about adverse behaviour with 2.6 we track down to "It behaves differently to what I expect" or applications with no (b)locking between threads suck under load. Personally I think the latter is a good thing as it encourages better coding, and the former is something we'll have with any alternate design.

The only feedback we got on staircase was that it helped NUMA somewhat and Nick and Ingo made some criticisms (not counting any benchmarks I had to offer). The only feedback on nickshed was that it hurt NUMA somewhat, SMT interactivity was broken (an easy enough oversight), and I did not comment to avoid giving biased criticism.

If you're after subjective performance feedback you're less likely to get it now than ever since you've made a strong stance against subjective reports, due to placebo effect. LKML is scary enough for the average user already. We have a situation now that if one brave single user reports good or bad behaviour everyone runs off that one user's report. Ouch!

There isn't going to be a 2.7 any time soon and there are people that are using alternate schedulers already in production; which is obviously why you're giving them a test run in -mm. Clearly the lack of a formal (2.7) development branch makes this even harder. Your attempt at preventing "good stuff' from rotting in alternate trees when mainline should be benefitting is admirable. While it's fun to rewrite the scheduler and gives us something to play with, the current level of feedback is hardly the testbase off which to replace it unless there's something strikingly better about a new cpu scheduler.

It will be interesting to see if this spawns any further discussion or whether Peter's scheduler's performance will also be lost in a low signal to noise ratio when it gets a run in -mm.

Rafael J. Wysocki said he was interested in a new scheduler for 2.6, but had no useful benchmarks. Con replied that there were no benchmarks at all for interactivity. Martin J. Bligh then said, "Rick's schedstats stuff had some ways to measure latency that seemed to work quite nicely. Hard to simulate exactly mozilla, email, etc, but probably close enough to be far more use than "ooh, it feels faster". He did a whole paper at OLS ... Rick ... pointer?" And Rick Lindsley replied:

http://www.finux.org/Reprints/Reprint-Lindsley-OLS2004.pdf

There are patches available for schedstats, although I haven't pulled together 2.6.9-rc1 yet. Shouldn't take me but fifteen minutes, I think.

Rafael, what baseline release are you comparing to? I should be able to provide some tools to measure the effect on updatedb directly for both 2.6.9-rc1 and your baseline (so long as it's 2.6-based)

Rafael J. Wysocki replied, "2.6.8.1, for example. I'd like to compate it with the 2.6.9-rc1-mm1, which contains the Nick's scheduler (2.6.9-rc1 has the same scheduler as 2.6.8.1, AFAIK)." Rick replied:

Okay. A schedstats patch for 2.6.8.1 is available at

http://eaglet.rain.com/rick/linux/schedstat/patches/schedstat-2.6.8.1
or
http://oss.software.ibm.com/linux/patches/?patch_id=730

You can also pick up the program "latency.c" at

http://eaglet.rain.com/rick/linux/schedstat/v9/latency.c

With these two things in hand, you should be able to measure the latency on 2.6.8.1 of a particular process.

A patch is not necessary for 2.6.9-rc1-mm1 (schedstats is already in there) but you will need to config the kernel to use it. Then retrieve a slightly different latency.c:

http://eaglet.rain.com/rick/linux/schedstat/v10/latency.c

since 2.6.9-rc1-mm1 output format is different (as you noted, it's a different scheduler.) Then you should be able to see if the latency of a particular process (updatedb, for instance) changes.

Rafael said he'd give this a try; but then reported compilation problems, that Rick addressed in a subsequent patch. Rafael then said, "I've fiddled a bit with both the latency.c programs. I've added some options to them etc. In particular, now you can specify a program to run and monitor instead of a pid, which is handy if you need to monitor processes that exit quickly. Everything is documented in the sources (attached). I thought you might find this useful. :-)" Rick replied, "Thank you much! yes, that will make it more useful. I'll add it to my backlog and see if I can't get it out to the web page this week."

2. kbuild Support For LOCALVERSION

31 Aug 2004 - 5 Sep 2004 (10 posts) Archive Link: "kbuild: Support LOCALVERSION"

Topics: Kernel Build System, SMP

People: Sam RavnborgIan WienandJan-Benedict GlawJan-Benedict

Sam Ravnborg said:

The following patch combines the request from several people. If you place a file named localversion* in the root of your soruce tree or the root of your output tree the text included in this file will be appended to KERNELRELEASE.

LOCALVERSION was originally introduced by Ian Wienand <ianw@gelato.unsw.edu.au>

This allows one to put a short string in localversion identifying this particular configuration "-smpacpi", or to identify applied patches to the source "-llat-np".

More specifically:
$(srctree)/localversion-lowlatency contains "-llat"
$(srctree)/localversion-scheduler-nick constins "-np"

$(objtree)/localversion contains "-smpacpi"

Resulting KERNELRELEASE would be:
2.6.8.rc1-smpacpi-llat-np

Note that you no longer need to modify your Makefile to identify your kernel so no rejects when applying new patches. If you add a new localversion* file, or change a existing one kbuild will pick up this and do the proper rebuild next time you run make.

Only issue is that KERNELRELEASE needs to be <= 64 chars - so keep the names short.

kbuild errors out if you are above limit.
$ cat localviersion-long
very-long-version-in-localversion-file-exceeding-64-chars-for-sure
Example:
CHK include/linux/version.h
"2.6.9-rc1-very-long-version-in-localversion-file-exceeding-64-chars-for-sure" exceeds 64 characters
make: *** [include/linux/version.h] Error 1

Ian Wienand pointed out that the 'LOCALVERSION=aversion' command line option didn't cause version.h to be rebuilt. He asked, "Is it going to just be a case of "you can't specify LOCALVERSION" from the command line?" He posted a patch to fix the behavior, which Sam accepted.

Elsewhere, Jan-Benedict Glaw said of the original patch, "I love it. Maybe it would also be good (in the longer term) to introduce a config name into one of the Kconfig files, which is preserved in the .config file (eg. SuSE does something like that and even while I'm not a SuSE user, it's really dandy at some times, esp. for things like "SMP-4GB", "VAX-KA4x" and the like). It's basically like adding the defconfig_* name to some of the variables :-)" Sam thought this was a good idea. Close by, Ian posted a patch, saying:

Ok, here is my attempt. I think it does everything everyone wants

Sam liked this patch and incorporated it into his own.

3. Parallel Port Mailing List Changed

2 Sep 2004 (4 posts) Archive Link: "[patch] update parport MAINTAINERS entry"

Topics: MAINTAINERS File

Maximilian Attems updated the MAINTAINERS file to change the Parallel Port mailing list from parport@torque.net to linux-parport@lists.infradead.org.

4. Linux 2.6.9-rc1-mm3 Released

3 Sep 2004 - 8 Sep 2004 (53 posts) Archive Link: "2.6.9-rc1-mm3"

Topics: FS: ext3, FS: sysfs, Hot-Plugging, Kernel Release Announcement, Kexec, Profiling

People: Andrew MortonStephen Tweedie

Andrew Morton announced Linux 2.6.9-rc1-mm3, saying:

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm3/

5. Linux 2.6.9-rc1-mm4 Released

7 Sep 2004 - 9 Sep 2004 (56 posts) Archive Link: "2.6.9-rc1-mm4"

Topics: FS: CacheFS, Kernel Release Announcement

People: Andrew Morton

Andrew Morton announced Linux 2.6.9-rc1-mm4, saying:

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm4/

6. dmraid 1.0.0-rc4 Released

7 Sep 2004 (1 post) Archive Link: "*** Announcement: dmraid 1.0.0-rc4 ***"

Topics: Disk Arrays: RAID

People: Heinz Mauelshagen

Heinz Mauelshagen said:

dmraid 1.0.0-rc4 is available at http://people.redhat.com:/~heinzm/sw/dmraid/ in source, source rpm and i386 rpm.

dmraid (Device-Mapper Raid tool) discovers, [de]activates and displays properties of software RAID sets (ie. ATARAID) and contained DOS partitions using the device-mapper runtime of the 2.6 kernel.

The following ATARAID types are supported on Linux 2.6:

Highpoint HPT37X
Highpoint HPT45X
Intel Software RAID
Promise FastTrack
Silicon Image Medley

This ATARAID type is only basically supported in this version (I need better metadata format specs; please help): LSI Logic MegaRAID

Please provide insight to support those metadata formats completely.

Thanks.

See files README and CHANGELOG, which come with the source tarball for prerequisites to run this software, further instructions on installing and using dmraid!

CHANGELOG is contained below for your convenience as well.

Call for testers:

I need testers with the above ATARAID types, to check that the mapping created by this tool is correct (see options "-t -ay") and access to the ATARAID data is proper.

In case you have a different ATARAID solution from those listed above, please feel free to contact me about supporting it in dmraid.

You can activate your ATARAID sets without danger of overwriting your metadata, because dmraid accesses it read-only unless you use option -E with -r in order to erase ATARAID metadata (see 'man dmraid')!

This is a release candidate version so you want to have backups of your valuable data *and* you want to test accessing your data read-only first in order to make sure that the mapping is correct before you go for read-write access.

The author is reachable at <Mauelshagen@RedHat.com>.

For test results, mapping information, discussions, questions, patches, enhancement requests and the like, please subscribe and mail to <ataraid@redhat.com>.

7. Serial And TTY Layer Overhaul

7 Sep 2004 - 12 Sep 2004 (10 posts) Archive Link: "The Serial Layer"

People: Alan CoxTheodore Ts'oArjan van de Ven

Alan Cox asked of the serial layer, "Is anyone currently looking at fixing this before I start applying extreme violence ? In particular to start trying to do something about the races in TIOCSTI, line discipline setting, hangup v receive, drivers abusing the API and calling ldisc.receive_buf direct ?" Arjan van de Ven thought Alan meant the TTY layer instead of the serial layer, but Alan said:

Both. A lot of hangup/receive races are in the serial drivers themselves doing things like

hangup
[close ldisc]
send bytes to the ldisc
[Boom!]

Theodore Ts'o said:

The hangup handling needs to be completely redone, so that we don't force serial drivers to do a completely shutdown of the port in an interrupt context. If the drivers are careful, it can be safe, but it's too hard to handle hangup correctly.

If you have time to work on the tty layer (sucker!!!), please go ahead and start work by all means. I was hoping to have time to clean up some of the more egregious problems sometime next year (after I escape back into development), but getting this fixed sooner rather the later would be a definitely Good Thing.

8. Linux Test Project Version 20040908 Released

8 Sep 2004 (1 post) Archive Link: "[ANNOUNCE] Sept LTP Release now available"

Topics: FS: NFS

People: Marty Ridgeway

Marty Ridgeway announced the latest version of the Linux Test Project, saying:

The Sept. release of LTP is now on SourceForge.net, changes include:

LTP-20040908

9. Digsig 1.3.1 Released

9 Sep 2004 - 10 Sep 2004 (14 posts) Archive Link: "[ANNOUNCE] Release Digsig 1.3.1: kernel module for run-time authentication"

Topics: Executable File Format

People: Makan Pourzandi

Makan Pourzandi said:

DSI development team would like to announce the release 1.3.1 of digsig.

This kernel module helps system administrators control Executable and Linkable Format (ELF) binary execution and library loading based on the presence of a valid digital signature. The main functionality is to help system administrators distinguish applications he/she trusts (and therefore signs) from viruses, worms (and other nuisances). It is based on the Linux Security Module hooks.

The code is GPL and available from: http://sourceforge.net/projects/disec/, download digsig-1.3.1. For more documentation, please refer to disec.sourcefrge.net.

I hope that it'll be useful to you.

All bug reports and feature requests or general feedback are welcome (please CC me or disec-devel@lists.sourceforge.net in your answer or feedback to the mailing list).

10. Intel Microcode Update Available

9 Sep 2004 - 15 Sep 2004 (19 posts) Archive Link: "Latest microcode data from Intel."

Topics: Hot-Plugging, Hyperthreading

People: Tigran AivazianChris RankinNathan BryantAlan Cox

Tigran Aivazian said:

I have received and tested the latest microcode data file from Intel, The file is dated 2nd September 2004. You can download it both as standalone (bzip2-ed) text file and bundled with microcode_ctl utility from the Download section of the website:

http://urbanmyth.org/microcode/

Please let me know if you find any problems with this data file or with the Linux microcode driver. Thank you.

Chris Rankin exulted, "Aha! And this microcode file actually contains data for my dual P4 Xeons (HT)" [...] "The July 27th offering didn't contain anything for these CPUs at all. In fact, the July file was suspiciously smaller than the one from 16th March."

Elsewhere, Nathan Bryant asked, "how does microcode update interact with CPU hotplug?" Alan Cox replied, "You run the microcode update sequence after a CPU is plugged in. That might be an argument for having user space kick off application use of hotplugged processors so that some housekeeping can run first. In the ideal case your BIOS vendor ships you needed BIOS updates to handle such things and in a sane format."

11. udev 031 Released

10 Sep 2004 (1 post) Archive Link: "[ANNOUNCE] udev 031 release"

Topics: FS: devfs, FS: sysfs, Hot-Plugging, Version Control

People: Greg KH

Greg KH said:

I've released the 031 version of udev. It can be found at: kernel.org/pub/linux/utils/kernel/hotplug/udev-031.tar.gz (http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev-031.tar.gz)

udev allows users to have a dynamic /dev and provides the ability to have persistent device names. It uses sysfs and /sbin/hotplug and runs entirely in userspace. It requires a 2.6 kernel with CONFIG_HOTPLUG enabled to run. Please see the udev FAQ for any questions about it: kernel.org/pub/linux/utils/kernel/hotplug/udev-FAQ (http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev-FAQ)

For any udev vs devfs questions anyone might have, please see: kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs (http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs)

This release fixes a lot of little bugs since the 030 release. I might have missed a few patches that were sent to me, and if you don't see your patch/fix included in here, please resend it.

Major changes since 030:

Thanks to everyone who has send me patches for this release, a full list of everyone, and their changes is below.

udev development is done in a BitKeeper repository located at: bk://linuxusb.bkbits.net/udev

Daily snapshots of udev from the BitKeeper tree can be found at: http://www.codemonkey.org.uk/projects/bitkeeper/udev/
If anyone ever wants a tarball of the current bk tree, just email me.

12. Linux 2.4.28-pre3 Released

11 Sep 2004 - 13 Sep 2004 (7 posts) Archive Link: "Linux 2.4.28-pre3"

Topics: FS: XFS

People: Marcelo Tosatti

Marcelo Tosatti announced Linux 2.4.28-pre3, saying:

Here goes the third pre of 2.4.28, containing a bunch of scattered fixes.

The bigger modifications are XFS update, prism54 update, libata fixes, more gcc 3.4 cleanups, amongst others.

13. Linux 2.6.9-rc1-mm5 Released

13 Sep 2004 - 15 Sep 2004 (96 posts) Archive Link: "2.6.9-rc1-mm5"

Topics: FS: ext3, Kernel Release Announcement, PCI, Version Control

People: Andrew MortonJames Bottomley

Andrew Morton announced Linux version 2.6.9-rc1-mm5, saying:

Due to master.kernel.org being on the blink, 2.6.9-rc1-mm5 Is currently at

http://www.zip.com.au/~akpm/linux/patches/2.6.9-rc1-mm5/

and will later appear at

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm5/

Please check kernel.org before using zip.com.au.

14. Linux 2.6.9-rc2 Released

13 Sep 2004 - 15 Sep 2004 (14 posts) Archive Link: "Linux 2.6.9-rc2"

Topics: Kernel Release Announcement, PCI, Sound: ALSA

People: Linus Torvalds

Linus Torvalds announced kernel 2.6.9-rc2, saying:

Various things all over the map, most of them not necessarily very visible to most people. ALSA update, and tons of small fixes pretty much everywhere.

The one thing that people may actually _notice_ is that you get a lot more warnings for some drivers due to the stricter type-checks for PCI memory mapping. They are harmless (code generation should be the same), and we'll work on trying to fix up the drivers as we go along, but they can be a bit daunting if you happen to enable some of the less type-friendly drivers right now..

15. Libsysfs 1.2.0 Released

14 Sep 2004 (1 post) Archive Link: "[ANNOUNCE] Libsysfs v1.2.0 release"

Topics: FS: sysfs

People: Ananth N. Mavinakayanahalli

Ananth N. Mavinakayanahalli said:

Version 1.2.0 of libsysfs, shipped as part of the sysfsutils package is now available at

http://linux-diag.sourceforge.net

Libsysfs provides a simple and stable API to access the sysfs filesystem.

Major changes since 1.1.0:

16. PWC Re-Added With Binary Driver Reverse-Engineered

14 Sep 2004 - 15 Sep 2004 (3 posts) Archive Link: "[PATCH] PWC driver without binary interface"

Topics: Compression, Philips Webcam Driver

People: Luc SaillardStelian PopTim Fairchild

The PWC driver was recently removed after a bad flamewar, covered in Issue #276, Section #4  (26 Aug 2004: Developer Debate Over Maintainership And Binary Driver Hooks; Specifically PWC Driver) and Issue #276, Section #6  (27 Aug 2004: More On The PWC Driver)

Luc Saillard said:

I've made a patch to (re)add pwc philips driver into the kernel. This driver have support for some compression mode (for chipset 2 & 3), so you don't need the binary module to grab an image in 640x480@10fps. Use this driver with caution, i've test on several webcam (type 730, 740). Mode bayer is not implemented, the camera only output yuv420p (planar mode). Future plan is to improve compatibility with other webcam, and try to implement decoder for version 1.

As wish by the original author, i've remove email address for the support, and add a disclaimer "this is unofficial version".

The patch is 300kbytes long, i don't include it in this mail. You can found a tarball or a patch against the last linux kernel at:

http://www.saillard.org/pwc/linux-2.6.9-rc2_pwc-9.0.2-fork0.2.diff.bz2
http://www.saillard.org/pwc/

Stelian Pop remarked, "Just in case nobody payed attention to the original message and failed to see that almost all the binary pwcx has been succesfully reverse-engineered..." Tim Fairchild replied, "Yes, noted the original, but was out tonight. Congratulations and good work!!! I wish I had a phillips cam to test it out on... I'll have to check which makes and models and see if I can find something on ebay to test with..."

17. LinuxPPC Developer Mailing Lists Have Moved

15 Sep 2004 (1 post) Archive Link: "LinuxPPC developers mailing lists have moved"

People: Stephen Rothwell

Stephen Rothwell said:

Just to let those who don't know that the mailing lists for Linux on PPC developers have moved. They are now hosted on ozlabs.org. The current lists are:

linuxppc-dev For all developers of Linux on PPC hardware (explictly including embedded hardware)
linuxppc64-dev For all developers of Linux on 64 bit PPC hardware

To subscribe, please point your favourite web browser at http://ozlabs.org/mailman/listinfo/

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.