Kernel Traffic #315 For 12 Jun 2005 By Zack Brown Table Of Contents * Standard Format * Text Format * XML Source * Mailing List Stats For This Week * Threads Covered 1. 1 Jun 2005 - 5 Jun 2005 (20 Linux 2.6.12-rc5-mm2 Released; -mm Tree posts) To Be Available As git Repository 2. 2 Jun 2005 - 7 Jun 2005 (8 New Automated Testing Scripts For posts) Official And Development Kernel Releases 3. 3 Jun 2005 (1 sdparm Version 0.93 Released post) 4. 3 Jun 2005 - 9 Jun 2005 (37 An ETA For 2.6.12 posts) 5. 3 Jun 2005 - 7 Jun 2005 (11 Some Developers Appraise The w.x.y.z posts) Series 6. 4 Jun 2005 (6 New Short Changelog Listing Script For posts) git 7. 6 Jun 2005 (1 sg3_utils (SCSI Command Utilities) post) Version 1.15 Released 8. 6 Jun 2005 - 9 Jun 2005 (39 Linux 2.6.12-rc6 Released; Some posts) Discussion Of Patch Submission Policies 9. 7 Jun 2005 (12 Real-Time Preemption Patch Version posts) 0.7.47-20 Released Mailing List Stats For This Week We looked at 1585 posts in 10MB. See the Full Statistics. There were 580 different contributors. 219 posted more than once. The average length of each message was 130 lines. +-----------------------------------------------------------------------------+ |The top posters of the| The top subjects of the week were: | | week were: | | |----------------------+------------------------------------------------------| |46 posts in 250KB by | | |ingo molnar |75 posts in 342KB for "ot] joerg schilling flames | |45 posts in 241KB by |linux on his blog" | |jeff garzik |43 posts in 271KB for "[patch 3/4] new timeofday | |39 posts in 186KB by |x86-64 arch specific changes (v. b1)" | |akpm@osdl.org |39 posts in 143KB for "pci_enable_msi() for everyone?"| |31 posts in 151KB by |38 posts in 220KB for "linux v2.6.12-rc6" | |greg kh |37 posts in 147KB for "2.6.12?" | |29 posts in 113KB by | | |greg kh | | +-----------------------------------------------------------------------------+ These stats generated by mboxstats version 2.8 1. Linux 2.6.12-rc5-mm2 Released; -mm Tree To Be Available As git Repository 1 Jun 2005 - 5 Jun 2005 (20 posts) Archive Link: "2.6.12-rc5-mm2" Topics: Kernel Release Announcement, Version Control People: Andrew Morton, Pavel Machek, Matthias Urlichs Andrew Morton announced Linus 2.6.12-rc5-mm2, saying: ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.12-rc5/ 2.6.12-rc5-mm2/ * Dropped bk-acpi.patch. Too old, too much breakage. * A few more subsystem trees have moved to using git * There are a large number of patches here which fix patches which people have already sent. A very large number. Are we getting a bit careless? Pavel Machek asked: Have you considered publishing -mm using git? I guess your workflow prevents you from really using git, but even just publishing releases using git would be great. (Just now I'm tracking Linus with my tree. git makes that quite easy. Tracking -mm is ugly manual work with diff, patch and ketchup...) Matthias Urlichs replied: I have written a script (actually a leftover from the mm-to-BK import days) that pulls -mm into git as individual commits. Andrew: Could you prefix the patches you pull from git with this line: GIT SHA1-of-their-top-commit URL-of-their-archive so that I can annotate the commits with an appropriate second parent? If you never do any changes oon top of -mm, but only merge with it (or them), then this works out quite well. Pavel said, "Great! Would it be possible to export results of your script somewhere? I guess you could get kernel.org account for this..." Matthias said he'd be happy to set this up if someone would give him a kernel.org account. 2. New Automated Testing Scripts For Official And Development Kernel Releases 2 Jun 2005 - 7 Jun 2005 (8 posts) Archive Link: "[ANNOUNCE] automated linux kernel testing results" Topics: Version Control People: Martin J. Bligh, Jeff Garzik, Greg KH, Denis Vlasenko Martin J. Bligh said: OK, I've finally got this to the point where I can publish it. http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/ regression_matrix.html Currently it builds and boots any mainline, -mjb, -mm kernel within about 15 minutes of release. runs dbench, tbench, kernbench, reaim and fsx. Currently I'm using a 4x AMD64 box, a 16x NUMA-Q, 4x NUMA-Q, 32x x440 (ia32) PPC64 Power 5 LPAR, PPC64 Power 4 LPAR, and PPC64 Power 4 bare metal system. The config files it uses are linked by the machine names in the column headers. Andrew, you should be able to see build failures in -mm more easily now ;-) I'll work on some automatic email triggers to you if you want them. Thanks to all the other IBM people who've worked on the ABAT test system that this stuff relies on - too many to list, but especially Andy, Adam, and Enrique, who have fixed endless bugs, and put up with my incessant bitching about it all not working as it should ;-) It will do various other bits and pieces ... more profiling, more tests, it'll do patches, etc as well. I don't want to push much volume up to kernel.org (I just copy a small subset of the generated files right now), but there are more runs queued with some fixup patches on top of -mm tree, for example. It did also do -bk and -git nightly builds, they're not in this matrix, but I'll add them back soon. Andrew, if you want some other test added to the mix, please let me know ... though theoretically we can do multi-machine tests (eg networked), I want to stick with single-machine ones for now. There's a huge pile of tests we intend to add ... but I thought I'd throw the general results mechanism out there for people to look at. Clicking on the failure ones error codes should take you to somewhere vaguely helpful to diagnose it. Clicking on the job number just below that takes you to the info I'm publishing right now, which should include perf results and profiles, etc. I'll add graphs, etc later, comparing performance across kernels (I have them ... just not automated). Greg KH was very impressed, and asked if this could be extended to also test the git nightly snapshots. Also, he asked if the -stable (w.x.y.z) tree could be included. Martin replied, "It does do both. I just didn't pull in all the historical data, I just repopulated the external set with a brief snapshot (I have way more internally, but it has some crap unpublishable benchmarks in it). If you look at the latest rev, about 3 up it has 2.6.12-rc5-git7, and I think I've fixed it to monitor for new ones automatically now. I guess we'll see if it worked in the morning ;-)" Denis Vlasenko was also very impressed with Martin's (and the rest of the contributors') work. At one point, Martin said: there's performance graphs out there for tbench, dbench, kernbench, and reaim now. http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/perf/perf_matrix.html -mm seems to be rather sucky on kernbench recently, I'll have a drill down into why, and try to send out some more data later on. The vertical bars on the kernbench graph are std deviation between sets of runs (a rough error margin). Jeff Garzik replied, "Very nice. All this stuff is helpful, thanks much." 3. sdparm Version 0.93 Released 3 Jun 2005 (1 post) Archive Link: "[ANNOUNCE] sdparm 0.93" Topics: Disks: IDE, Disks: SCSI, SPI People: Douglas Gilbert Douglas Gilbert said: sdparm is a command line utility designed to get and set SCSI device parameters (cf hdparm for ATA disks). Apart from SCSI devices (e.g. disks, tapes and enclosures) sdparm can be used on any device that uses a SCSI command set (e.g. CD/DVD drives). sdparm also can list VPD pages including the device identification page. The major addition in version 0.93 is transport protocol specific modes pages for FCP, SPI and SAS. For more information and downloads see: http://www.torque.net/sg/sdparm.html ChangeLog: http://www.torque.net/sg/p/sdparm.ChangeLog 4. An ETA For 2.6.12 3 Jun 2005 - 9 Jun 2005 (37 posts) Archive Link: "2.6.12?" Topics: Framebuffer, Power Management: ACPI, USB People: Andrew Morton, Dave Jones, Jeff Garzik Jeff Garzik asked if a 2.6.12 release was in the works for the near future, and Andrew Morton replied: Current plan is -rc6 in a few days, 2.6.12 a week after that. My things-to-worry-about folder still has 244 entries. Nobody seems to care much. Poor me. Lots of USB problems, quite a few input problems. fbdev, ACPI, ATAPI. All the usual suspects. Jeff was happy to hear this plan, and offered some help on some of Andrew's remaining issues. A bunch of other folks also tackled the remaining issues, but Dave Jones also remarked that of the items Andrew had listed, "There's quite a few in there judging by the looks of the subjects that aren't worth holding up the release imo. Will the world end if we dont ship 2.6.12 with support for Geode optimisation for eg ?" 5. Some Developers Appraise The w.x.y.z Series 3 Jun 2005 - 7 Jun 2005 (11 posts) Archive Link: "Stable 2.6.x.y kernel series..." People: Jeff Garzik, Alan Cox Jeff Garzik said, "I think the stable 2.6.x.y kernel series is working out quite well. Kudos to the stable@kernel.org team for a job well done. The 2.6.x.y series is definitely filling a needed niche." Szonyi Calin said he didn't believe the 2.6 tree could be made stable by just the addition of a few small patches, as the w.x.y.z tree seemed to be. But Szonyi did acknowledge that the tree did seem to be more stable than regular 2.6. But Alan Cox agreed wholeheartedly with Jeff's appraisal, saying, "its been conservative enough that not only does it stay pretty stable (one partition slip-up so far is very good indeed) its small enough that most of the add on patches people use aren't breaking against it either. Even the -ac set just keeps applying barring makefile just fine so its saved me a ton of work." 6. New Short Changelog Listing Script For git 4 Jun 2005 (6 posts) Archive Link: "git-shortlog script" Topics: Version Control People: Jeff Garzik, Linus Torvalds Jeff Garzik said: Attached is the 'git-shortlog' script I whipped up, to mimic the shortlog script that was used back in the BitKeeper days. shortlog reads a changelog in the 'git-whatchanged' format, such as git-whatchanged | git-shortlog or git-shortlog changes.txt and outputs the changes sorted by author: author1: cset 1-line desc cset 1-line desc ... author2: cset 1-line desc ... ... Since git distinguishes 'author' from 'committer', I ran git-whatchanged | git-shortlog > changes.txt to look at the kernel authors throughout the entire history [of which is in git]. It's fun to browse, since this is the first time we've been able to get a better picture who is actually writing the patches, versus committing them. See changes.txt.bz2, attached. Linus Torvalds replied: Thanks, I'll add this to the git stuff, and next kernel release will have a proper shortlog. Btw, it shows how broken your merge script is: you don't fill in the AUTHOR field properly for some reason: : Automatic merge of /spare/repo/netdev-2.6 branch r8169-fix Automatic merge of /spare/repo/linux-2.6/.git branch HEAD Automatic merge of /spare/repo/netdev-2.6 branch use-after-unmap Automatic merge of rsync://rsync.kernel.org/.../torvalds/linux-2.6.git branch HEAD but "committer" is right. Pls fix. He also added: the reason you didn't notice is that "git-whatchanged" normally ignores merges. Do git-rev-list --pretty HEAD ^v2.6.12-rc5 | git-shortlog | less -S to see what I'm talking about ("show shortlog of all the changes since v2.6.12-rc5"). 7. sg3_utils (SCSI Command Utilities) Version 1.15 Released 6 Jun 2005 (1 post) Archive Link: "[Announce] sg3_utils-1.15 available" Topics: Disks: SCSI, Ioctls People: Douglas Gilbert Douglas Gilbert said: sg3_utils is a package of command line utilities for sending SCSI commands to devices. This package targets the lk 2.6 and lk 2.4 series. In the lk 2.6 series these utilities (except sgp_dd) can be used with any devices that support the SG_IO ioctl. This version includes minor enhancements, bug fixes and man page improvements. See CHANGELOG for more information. A tarball, rpm and deb can be found on (see table 2): http://www.torque.net/sg For an overview of sg3_utils see this page: http://www.torque.net/sg/ u_index.html The sg_dd utility has its own page at: http://www.torque.net/sg/sg_dd.html A changelog can be found at: http://www.torque.net/sg/p/sg3_utils.CHANGELOG A release announcement has been sent to freshmeat.net (http://freshmeat.net) . 8. Linux 2.6.12-rc6 Released; Some Discussion Of Patch Submission Policies 6 Jun 2005 - 9 Jun 2005 (39 posts) Archive Link: "Linux v2.6.12-rc6" Topics: USB People: Linus Torvalds, Pavel Machek, Andrew Morton, Dmitry Torokhov Linus Torvalds said: It's being uploaded right now, the git tree is already up-to-date, and by the time this hits the mailing list the mirroring of the tar-ball will hopefully be done too. And since Jeff wrote me a shortlog script for git, the easist way to tell what's new since -rc5 is to just do the shortlog and diffstat output. Network drivers, USB and CPU-freq stand out. And the good news is that people do seem to have taken my rumblings about calming down for 2.6.12 seriously. Let's hope that pans out, and I can release that one asap.. But give this a good beating first, and holler (again, if you must) about any issues you have Pavel Machek noted that a fix for a jumpy mouse cursor had been wrongly attributed to him. He pointed out that Dmitry Torokhov was the true author, and added that this information was also contained in the changelog entry: author Pavel Machek Fri, 27 May 2005 12:53:03 -0700 committer Linus Torvalds Sat, 28 May 2005 11:14:01 -0700 [PATCH] fix jumpy mouse cursor on console Do not send empty events to gpm. (Keyboards are assumed to have scroll wheel these days, that makes them part-mouse. That means typing on keyboard generates empty mouse events). From: Dmitry Torokhov Signed-off-by: Pavel Machek Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds He suggested Linus had a problem with his script. But Linus replied: My scripts definitely do the expected thing. In git, the author is always in the fixed header, and you never look for it anywhere else. However, in order for the author to _get_ there in the first place, the person who commits the thing needs to haev the author info. In this case it was me, and I get the author information from the email when I commit an emailed patch. I take it from the first line of the body if that one is a valid "From:" line, and otherwise I fall back to taking it from the headers of the email. So in this case you got tagged, either because the patch came through Andrew (it has his sign-off) and _he_ sent the email but incorrectly had you as the "From:" person, or alternatively because you sent the email and took Andrew's sign-off but didn't put the "From:" in the right spot. Pavel replied, "I thought you are taking "first From: in the body", not "From: only if it is first line in the body". [Could you perhaps modify your scripts to take "first From: in the body"? It seems logical to put From "near" Signed-of-by: lines..." But Linus said, "I really don't want to, for a number of reasons. Most notably because I don't want to mix things up with the sign-off, because authorship and sign-off are really separate things (sign-offs accumulate, authorship stays), but also because it's not entirely unambiguous to parse these things. With the "first line only" rule, it ends up being pretty clear what's going on when the script suddenly ate one line.." Pavel said, "Okay, I see. I'm little afraid that during forwards blank line will be inserted before "From: " and break this, but lets see how it works." But Linus replied, "Oh, I skip blank lines (and that means any line that is "whitespace only", ie tabs/spaces etc won't confuse the scripts), so at least it's not _that_ subtle." 9. Real-Time Preemption Patch Version 0.7.47-20 Released 7 Jun 2005 (12 posts) Archive Link: "[patch] Real-Time Preemption, -RT-2.6.12-rc6-V0.7.47-29" Topics: FS: sysfs, Real-Time People: Ingo Molnar Ingo Molnar said: i have released the -V0.7.47-20 Real-Time Preemption patch, which can be downloaded from the usual place: http://redhat.com/~mingo/realtime-preempt/ i've implemented two new features: * new debugging feature: CONFIG_DEBUG_RT_LOCKING_MODE, which adds the /proc/ sys/kernel/preempt_locks flag (default: 0). This way the 'locking model' can be switched runtime - very useful for debugging and profiling. Value 0 means that all spinlocks and rwlocks are implemented via raw spinlocks/ rwlocks. (which disable preemption, increase latency, but improve throughput) Value 1 means the kernel will fully preempt all locks again. (NOTE: the only safe runtime switching of the locking model can be done while the system is idle, so i've implemented the flag via two flags where the idle thread propagates the new value from the user-flag to the kernel-flag. You should put a "sleep 1" into scripts that switch the locking mode, to guarantee that the new flag value is picked up.) * performance feature: i've implemented a new scheduler feature called 'delayed preemption', which turns sync wakeups into guaranteed wakeups, while preserving their workload-batching properties. A delayed preemption request is implemented via the TIF_NEED_RESCHED_DELAYED flag, which runs in parallel to the "immediate preemption" TIF_NEED_RESCHED flag. If this works out fine then it will be a suitable replacement for the upstream sync-wakeups facility as well. delayed preemption already improved the performance of 'hackbench' under PREEMPT_RT quite signifiantly. to build a -V0.7.47-20 tree, the following patches have to be applied: http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.tar.bz2 http://kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.12-rc6.bz2 http://redhat.com/~mingo/realtime-preempt/ realtime-preempt-2.6.12-rc6-V0.7.47-20 Sharon And Joy Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.