Kernel Traffic #277 For 17 Oct 2004

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm1/

• nicksched is still here. There has been very little feedback, except that it seems to slow some workloads on NUMA.
• Added a __must_check to the x86 copy_*_user functions. This means that with a sufficiently recent gcc, all unchecked copy_*_user() calls will generate a warning.

I fixed a few things, but binfmt_elf.c is a mess.

It's not clear how to apply the same debug check to put_user() and friends.

That's because most people aren't interested in a new cpu scheduler for 2.6. The current one works well enough in most situations and people aren't trying -mm to fix their interactive problems since they are few and far between. The only reports about adverse behaviour with 2.6 we track down to "It behaves differently to what I expect" or applications with no (b)locking between threads suck under load. Personally I think the latter is a good thing as it encourages better coding, and the former is something we'll have with any alternate design.

The only feedback we got on staircase was that it helped NUMA somewhat and Nick and Ingo made some criticisms (not counting any benchmarks I had to offer). The only feedback on nickshed was that it hurt NUMA somewhat, SMT interactivity was broken (an easy enough oversight), and I did not comment to avoid giving biased criticism.

If you're after subjective performance feedback you're less likely to get it now than ever since you've made a strong stance against subjective reports, due to placebo effect. LKML is scary enough for the average user already. We have a situation now that if one brave single user reports good or bad behaviour everyone runs off that one user's report. Ouch!

There isn't going to be a 2.7 any time soon and there are people that are using alternate schedulers already in production; which is obviously why you're giving them a test run in -mm. Clearly the lack of a formal (2.7) development branch makes this even harder. Your attempt at preventing "good stuff' from rotting in alternate trees when mainline should be benefitting is admirable. While it's fun to rewrite the scheduler and gives us something to play with, the current level of feedback is hardly the testbase off which to replace it unless there's something strikingly better about a new cpu scheduler.

It will be interesting to see if this spawns any further discussion or whether Peter's scheduler's performance will also be lost in a low signal to noise ratio when it gets a run in -mm.

http://www.finux.org/Reprints/Reprint-Lindsley-OLS2004.pdf

There are patches available for schedstats, although I haven't pulled together 2.6.9-rc1 yet. Shouldn't take me but fifteen minutes, I think.

Rafael, what baseline release are you comparing to? I should be able to provide some tools to measure the effect on updatedb directly for both 2.6.9-rc1 and your baseline (so long as it's 2.6-based)

Okay. A schedstats patch for 2.6.8.1 is available at

http://eaglet.rain.com/rick/linux/schedstat/patches/schedstat-2.6.8.1
or
http://oss.software.ibm.com/linux/patches/?patch_id=730

You can also pick up the program "latency.c" at

http://eaglet.rain.com/rick/linux/schedstat/v9/latency.c

With these two things in hand, you should be able to measure the latency on 2.6.8.1 of a particular process.

A patch is not necessary for 2.6.9-rc1-mm1 (schedstats is already in there) but you will need to config the kernel to use it. Then retrieve a slightly different latency.c:

http://eaglet.rain.com/rick/linux/schedstat/v10/latency.c

since 2.6.9-rc1-mm1 output format is different (as you noted, it's a different scheduler.) Then you should be able to see if the latency of a particular process (updatedb, for instance) changes.

The following patch combines the request from several people. If you place a file named localversion* in the root of your soruce tree or the root of your output tree the text included in this file will be appended to KERNELRELEASE.

LOCALVERSION was originally introduced by Ian Wienand <ianw@gelato.unsw.edu.au>

This allows one to put a short string in localversion identifying this particular configuration "-smpacpi", or to identify applied patches to the source "-llat-np".

More specifically:
$(srctree)/localversion-lowlatency contains "-llat"
$(srctree)/localversion-scheduler-nick constins "-np"

$(objtree)/localversion contains "-smpacpi"

Resulting KERNELRELEASE would be:
2.6.8.rc1-smpacpi-llat-np

Note that you no longer need to modify your Makefile to identify your kernel so no rejects when applying new patches. If you add a new localversion* file, or change a existing one kbuild will pick up this and do the proper rebuild next time you run make.

Only issue is that KERNELRELEASE needs to be <= 64 chars - so keep the names short.

kbuild errors out if you are above limit.
$ cat localviersion-long
very-long-version-in-localversion-file-exceeding-64-chars-for-sure
Example:
CHK include/linux/version.h
"2.6.9-rc1-very-long-version-in-localversion-file-exceeding-64-chars-for-sure" exceeds 64 characters
make: *** [include/linux/version.h] Error 1

Ok, here is my attempt. I think it does everything everyone wants

• localversion* files are read first
• config variable is appended last
• LOCALVERSION from the command line overrides all of this
• check is forced on build, since we can't really know when the config or environment options change.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm3/

• Added the m32r architecture. Haven't looked at it yet.

• Status update on various large patches in -mm:

  • The packet-writing code is awaiting resolution of the abuse-of-elevator-fields problem.
  • perfctr is stalled. Mikael is planning on doing more work to make it more compatible with perfmon, but he is busy for a while.
  • If the sysfs-backing-store patches get any older they'll start to stink. Awaiting review input from AV.
  • ext3 reservation code is currently being optimised by Stephen Tweedie (replace a serious-looking linear search with an rbtree). I'd expect those patches to land in -mm soon.
  • ext3 online resizing still needs a bit of maintenance work done.
  • pcmcia driver model support: dunno. I guess we're waiting for rmk to get onto this.
  • i386 hotplug CPU support: I'm not sure that we should merge this at all - it's a testing-only thing.
  • key management code: mainly awaiting feedback from other fs developers who may need such a capability. Also we seem to be unsure as to which userspace interface will end up being used?
  • kexec: I guess this could be merged now, if we want kexec now. I'm waiting for the kexec-based-crashdump patches to turn up.
  • cpusets: ready to go, I guess. Review input from various parties seems positive. Post-2.6.9.
  • reiser4: blah.

• Various random fixes and updates.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm4/

• Added Dave Howells' mysterious CacheFS.
• Various new fixes, cleanups and bugs, as usual.

dmraid 1.0.0-rc4 is available at http://people.redhat.com:/~heinzm/sw/dmraid/ in source, source rpm and i386 rpm.

dmraid (Device-Mapper Raid tool) discovers, [de]activates and displays properties of software RAID sets (ie. ATARAID) and contained DOS partitions using the device-mapper runtime of the 2.6 kernel.

The following ATARAID types are supported on Linux 2.6:

Highpoint HPT37X
Highpoint HPT45X
Intel Software RAID
Promise FastTrack
Silicon Image Medley

This ATARAID type is only basically supported in this version (I need better metadata format specs; please help): LSI Logic MegaRAID

Please provide insight to support those metadata formats completely.

Thanks.

See files README and CHANGELOG, which come with the source tarball for prerequisites to run this software, further instructions on installing and using dmraid!

CHANGELOG is contained below for your convenience as well.

Call for testers:

I need testers with the above ATARAID types, to check that the mapping created by this tool is correct (see options "-t -ay") and access to the ATARAID data is proper.

In case you have a different ATARAID solution from those listed above, please feel free to contact me about supporting it in dmraid.

You can activate your ATARAID sets without danger of overwriting your metadata, because dmraid accesses it read-only unless you use option -E with -r in order to erase ATARAID metadata (see 'man dmraid')!

This is a release candidate version so you want to have backups of your valuable data *and* you want to test accessing your data read-only first in order to make sure that the mapping is correct before you go for read-write access.

The author is reachable at <Mauelshagen@RedHat.com>.

For test results, mapping information, discussions, questions, patches, enhancement requests and the like, please subscribe and mail to <ataraid@redhat.com>.

Both. A lot of hangup/receive races are in the serial drivers themselves doing things like

hangup
[close ldisc]
send bytes to the ldisc
[Boom!]

The hangup handling needs to be completely redone, so that we don't force serial drivers to do a completely shutdown of the port in an interrupt context. If the drivers are careful, it can be safe, but it's too hard to handle hangup correctly.

If you have time to work on the tty layer (sucker!!!), please go ahead and start work by all means. I was hoping to have time to clean up some of the more egregious problems sometime next year (after I escape back into development), but getting this fixed sooner rather the later would be a definitely Good Thing.

The Sept. release of LTP is now on SourceForge.net, changes include:

LTP-20040908

• Modified runalltests.sh to call runltp.sh. runalltests.sh is now deprecated and will be removed early next year.
• Modified tst_tmpdir to ensure 777 permissions on test directory.
• Changes to ltp-aiodio.part3 for testcase run parameters
• Changes for 2.6.X so only delete modules is run, query and create are obsolete
• Updated runtest scenario with the latest SCTP tests.
• uncommented swapon() tests.
• added new paging tests, mincore and madvise
• Change to fix aio-stress problem will io errors on a short read during the random read portion
• Change to only print out a pass/fail instead of # of iterations pass/fail
• Added IA64 specific code for shmt09.
• Change to not do /dev/ptmx group write on arm arch.
• Applied patch from Ling, Xiaofeng to allow the test to use TDIRECTORY correctly.
• Corrected test 1 to show EPERM error pointed out by Ling, Xiaofeng.
• Change to close fileHandle prior to cleanup to correct testcase failure in NFS filesystems
• Change sleep time from 1 second to 10 seconds to allow system to pass
• Change to define RUSAGE_BOTH if not defined, RH removed from user space and other distros still support.
• Add arm arch to the ALIGNED typedefs
• Changes from Ihno for llseek01.c to check TEST_RETURN vs TEST_ERRNO
• use ltp functions in f00f test for better output parsing
• Fix Makefile to link open_files into the bin directory
• Get rid of extra = of "must be Root user" check
• Change the awk $4 to an $NF to support debian only returning 3 terms
• Change to tcpdump to check IFNAME define

DSI development team would like to announce the release 1.3.1 of digsig.

This kernel module helps system administrators control Executable and Linkable Format (ELF) binary execution and library loading based on the presence of a valid digital signature. The main functionality is to help system administrators distinguish applications he/she trusts (and therefore signs) from viruses, worms (and other nuisances). It is based on the Linux Security Module hooks.

The code is GPL and available from: http://sourceforge.net/projects/disec/, download digsig-1.3.1. For more documentation, please refer to disec.sourcefrge.net.

I hope that it'll be useful to you.

All bug reports and feature requests or general feedback are welcome (please CC me or disec-devel@lists.sourceforge.net in your answer or feedback to the mailing list).

I have received and tested the latest microcode data file from Intel, The file is dated 2nd September 2004. You can download it both as standalone (bzip2-ed) text file and bundled with microcode_ctl utility from the Download section of the website:

http://urbanmyth.org/microcode/

Please let me know if you find any problems with this data file or with the Linux microcode driver. Thank you.

I've released the 031 version of udev. It can be found at: kernel.org/pub/linux/utils/kernel/hotplug/udev-031.tar.gz

udev allows users to have a dynamic /dev and provides the ability to have persistent device names. It uses sysfs and /sbin/hotplug and runs entirely in userspace. It requires a 2.6 kernel with CONFIG_HOTPLUG enabled to run. Please see the udev FAQ for any questions about it: kernel.org/pub/linux/utils/kernel/hotplug/udev-FAQ

For any udev vs devfs questions anyone might have, please see: kernel.org/pub/linux/utils/kernel/hotplug/udev_vs_devfs

This release fixes a lot of little bugs since the 030 release. I might have missed a few patches that were sent to me, and if you don't see your patch/fix included in here, please resend it.

Major changes since 030:

• lots of minor bugfixes
• udevstart is now a symlink to udev
• added a new %e modifier
• lots of rule fixes and cleanups based on user reports.

Thanks to everyone who has send me patches for this release, a full list of everyone, and their changes is below.

udev development is done in a BitKeeper repository located at: bk://linuxusb.bkbits.net/udev

Daily snapshots of udev from the BitKeeper tree can be found at: http://www.codemonkey.org.uk/projects/bitkeeper/udev/
If anyone ever wants a tarball of the current bk tree, just email me.

Here goes the third pre of 2.4.28, containing a bunch of scattered fixes.

The bigger modifications are XFS update, prism54 update, libata fixes, more gcc 3.4 cleanups, amongst others.

Due to master.kernel.org being on the blink, 2.6.9-rc1-mm5 Is currently at

http://www.zip.com.au/~akpm/linux/patches/2.6.9-rc1-mm5/

and will later appear at

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9-rc1/2.6.9-rc1-mm5/

Please check kernel.org before using zip.com.au.

• Added the `bk-scsi-target' tree to the -mm lineup. It is managed by James Bottomley
• Some enhancements to the ext3 block reservation code here. Please cc sct@redhat.com on oops reports ;)
• There's a patch here which will cause warnings if a PCI device driver is removed without having called pci_disable_device(). Please try to cc the appropriate mailing list or maintainer when reporting any instances.

Various things all over the map, most of them not necessarily very visible to most people. ALSA update, and tons of small fixes pretty much everywhere.

The one thing that people may actually _notice_ is that you get a lot more warnings for some drivers due to the stricter type-checks for PCI memory mapping. They are harmless (code generation should be the same), and we'll work on trying to fix up the drivers as we go along, but they can be a bit daunting if you happen to enable some of the less type-friendly drivers right now..

Version 1.2.0 of libsysfs, shipped as part of the sysfsutils package is now available at

http://linux-diag.sourceforge.net

Libsysfs provides a simple and stable API to access the sysfs filesystem.

Major changes since 1.1.0:

• added dlist_sort_custom() for applications to specify their list sorter routine. (Thanks to Eric Bohm)
• minor documentation updates and bugfixes

I've made a patch to (re)add pwc philips driver into the kernel. This driver have support for some compression mode (for chipset 2 & 3), so you don't need the binary module to grab an image in 640x480@10fps. Use this driver with caution, i've test on several webcam (type 730, 740). Mode bayer is not implemented, the camera only output yuv420p (planar mode). Future plan is to improve compatibility with other webcam, and try to implement decoder for version 1.

As wish by the original author, i've remove email address for the support, and add a disclaimer "this is unofficial version".

The patch is 300kbytes long, i don't include it in this mail. You can found a tarball or a patch against the last linux kernel at:

http://www.saillard.org/pwc/linux-2.6.9-rc2_pwc-9.0.2-fork0.2.diff.bz2
http://www.saillard.org/pwc/

Just to let those who don't know that the mailing lists for Linux on PPC developers have moved. They are now hosted on ozlabs.org. The current lists are:

linuxppc-dev For all developers of Linux on PPC hardware (explictly including embedded hardware)
linuxppc64-dev For all developers of Linux on 64 bit PPC hardware

To subscribe, please point your favourite web browser at http://ozlabs.org/mailman/listinfo/