Kernel Traffic #155 For 25 Feb 2002

By Zack Brown

Table Of Contents

Introduction

Mailing List Stats For This Week

We looked at 1678 posts in 7176K.

There were 511 different contributors. 235 posted more than once. 187 posted last week too.

The top posters of the week were:

1. IDE Cleanup; Developer Psychosis

8 Feb 2002 - 19 Feb 2002 (19 posts) Archive Link: "IDE cleanup for 2.5.4-pre3"

Topics: Disks: IDE

People: Pavel MachekAndre HedrickMartin DaleckiVojtech PavlikJens Axboe

Pavel Machek posted some cleanups for the 2.5 IDE code, directed at Jens Axboe. Jens liked the patches, and Pavel asked, "Is it "Looks good to me, applied", or "Looks good to me, good luck pushing it to Linus?" :-)." And Jens said he'd incorporated the patches into his tree, and they should work their way nautrally up to Linus. However, Vojtech Pavlik posted his own version of the cleanups, going over the code more extensively. Pavel liked the patch, and then Martin Dalecki posted some even more extensive cleanups. Vojtech liked it, and Pavel recommended sending it to Jens as soon as possible, before it got much bigger.

At around this point Andre Hedrick wigged out, and said to Martin, "I asked you off line to slow down so I can fix a few issues. I have acknowledged you ideas are good but are woefully untimely. Obviously you have some agenada with a StartUp company because you work for a Germnay Venture Capital firm." Martin replied:

You are not serious? Aren't you? 'Bit of paranoia there?

Can't you just imagine somebody who did get out of the coding business in professional terms but stills to enjoys hacking from time to time? If you are impressed by my speed, and can't imagine that what I'm doing is the outcome of just a few spare houers at home, then well I certainly can't help you....

PS. If I would have a hidden agenda I wouldn't post from the evision domain becouse I know a bit about mailing systems as well ;-). And I would care to be polite and a slick ass to everybody, which I CERTAINLY don't you s******

Andre had no reply.

2. HTTP Servers In The Kernel

11 Feb 2002 - 16 Feb 2002 (13 posts) Archive Link: "tux officially in kernel?"

Topics: Web Servers

People: Joe SloanJohn SleeRobert LoveDavid LangLuigi GenoniRoy Sigurd Karlsbakk

Roy Sigurd Karlsbakk asked if the Tux http server would make it into the main kernel tree, and Joe Sloan replied:

If that's not part of the roadmap I'd be surprised, since tux is so much more capable than the khttpd which is currently part of the tree.

Tux has clearly demonstrated it's performance and low resource consumption.

I also think including tux in the mainline kernel would further stimulate development, as folks begin to realize what they can do with tux....

But John Slee remarked, "it has also been demonstrated that equal performance can be had in userland (search archives for "X15"). most of tux' improvements have been generalised and absorbed into the mainline kernel anyway."

Joe Sloan asked why khttpd was in the kernel at all, since "If there were any web server in the mainline kernel I'd think it'd be tux." Robert Love said that in his opinion khttpd had no place in the kernel and should be removed. He added, "TUX touches enough code that it isn't a clear decision to merge, although it is certainly worth it. I, however, think we are rapidly approaching the point, if not there already, that with a zero-copy network driver userspace can perform as good as TUX with none of the downsides. That was part of Ingo's goal and a lot of the benefits - sendfile etc - are a result of TUX."

David Lang explained, regarding khttpd's presense in the kernel:

Linus put khttpd in the kernel just after sendfile support was added, IIRC he said something about khttpd being a very small number of lines to add once sendfile support was there.

if it's really that small (IIRC <<100 lines of code) it's still in there becouse it's not worth ripping out.

Benny Sjostrand asked if anyone actually used khttpd, and Luigi Genoni replied, "I am using it, for example, on some web site, and it works well, abnd rock solid. But on the internet, as it is today, it has no meaning to talk about speed performances. But the server are less stressed than with just apache."

End of thread.

3. /proc Cleanup

12 Feb 2002 - 14 Feb 2002 (11 posts) Archive Link: "RFC: /proc key naming consistency"

People: Mark SwansonCameron SimpsonH. Peter AnvinVince WeaverDaniel Phillips

Mark Swanson proposed:

I would like to hear people's opinions on making the keys in the /proc hierarchy consistent wrt the space character. The current Linux Documentation/filesystems.proc.txt does not suggest any standard naming conventions. F.E. cat /proc/cpuinfo (partial list)

cpu family      : 5
model           : 9
model name      : AMD-K6(tm) 3D+ Processor
stepping        : 1
cpu MHz         : 400.907
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no

Notice the space between "cpu" and "MHz", or "cpu" and "family" yet there is no space between "fdiv" and "bug" (_).

The reason I think NOT using a space is a good idea because it makes life easier for developers parsing /proc entries. Specifically, Java developers could use /proc/cpuinfo as a property file, but the space in the 'key' breaks java.util.Properties.load().

Cameron Simpson replied:

Personally, I have LONG wished all /proc/* entries were shell parsable values, eg:

cpu_family=5
model=9
model_name='AMD-K6(tm) 3D+ Processor'

etc. The amound of userland parsing code this would obviate must be quite large, and it certainly would make simple scripts to do things with the values much easier.

instead, there are all these "pretty" files, with formatting code that properly belongs in userland tools residing in the kernel (yes, mostly just printf but still).

H. Peter Anvin pointed out that he and Dan Quinlan had tried to follow Mark's proposal in a cleanup patch they'd done awhile before, but "some other people not just added keys with spaces, but gracefully "corrected" our "mistakes"..."

Daniel Phillips liked Cameron's idea of shell-parsable key/value pairs, and H. Peter replied that he liked it, but that it would break lots of stuff. Luke Burton felt that application maintainers would jump at the chance to simplify their code, by throwing out the bulk of their /proc parsing routines, but Vince Weaver said:

as a maintainer of "linux_logo" which delves heavily (and perhaps unwisely) into /proc/cpuinfo and its ilk, I would definitely say I would not be jumping at the opportunity.

Remember as maintainers of userspace apps, we have to keep compatibility, in this case for me it involves /proc/cpuinfo from all the architectures, plus kernels going back to the 1.2.13 time-frame.

So changing /proc/cpuinfo yet again does not simplify the code, in fact it just adds one more incompatible special case.

And since 2.2 and 2.4 kernels will be around for ages to come, it will make code bigger rather than smaller.

I agree a cleanup, if done properly, would be welcomed. but don't use "simplification of user-space code" as an argument.. because it's a lie ;)

4. Working With BitKeeper

13 Feb 2002 - 16 Feb 2002 (21 posts) Archive Link: "[PATCH] queue barrier support"

Topics: Version Control

People: Daniel PhillipsJens AxboeAndreas Dilger

Jens Axboe posted some BitKeeper changesets, and Daniel Phillips asked, "the BK url's are wonderfully informative and useful, but they are long and _ugly_! Is there anything that can be done about that?" Jens agreed, "Yeah I like them too, maybe if I just figured out how to get BitKeeper to dump full changeset info I could just inline them in the mail instead. I'll look at up and try that next time." Andreas Dilger replied:

bk send -wgzip_uu -r<rev> - > foo-<rev>.bk

This will dump a gzipped-uuencoded changset to the file. The receiver just do "| bk receive [repository] -avv" to import it on the other end.

My preferred format for sending BK CSETs is below. The gzip_uu CSET data only adds maybe 10% for large patches, and about doubles the size of very small patches. I also created a bz64 (bzip2 + base64) wrapper which makes the CSET data smaller, but that is only useful if other BK developers have this wrapper also.

#!/bin/sh
# A script to format BK changeset output in a manner that is easy to read.
# Andreas Dilger <adilger@turbolabs.com> 13/02/2002
 
PROG=bksend
 
usage() {

echo "usage: $PROG -r<rev>"
echo -e "\twhere <rev> is of the form '1.23', '1.23..', '1.23..1.27',"
echo -e "\tor '+' to indicate the most recent revision"
 
exit 1
}
 
case $1 in
-r) REV=$2; shift ;;
-r*) REV=`echo $1 | sed '/kernel-traffic/s/^-r/index.html'` ;;
*) echo "$PROG: no revision given, you probably don't want that";;
esac
 
[ -z "$REV" ] && usage
 
bk changes -r$REV
bk export -tpatch -du -h -r$REV
echo -e "\n================================================================\n\n"
bk send -wgzip_uu -r$REV -

5. Status Of LKCD

13 Feb 2002 - 14 Feb 2002 (6 posts) Archive Link: "status of LKCD into Linux Kernel"

People: Jeff GarzikMatt D. RobinsonHiro YoshiokaGo Taniguchi

Hiro Yoshioka asked about the status of the Linux Kernel Crash Dump (http://lkcd.sourceforge.net/) project, and Jeff Garzik replied:

I talked with Matt Robinson(sp?) at LinuxWorld-NY about LKCD for a few minutes... he gave me the impression that a lot of progress has been made recently. IBM apparently has some guys working on it, too.

I've always thought Linux needs industrial strength crash dumps like the other Unices. There are many benefits, but my own is self interest: bug reports get 1000 times better, since you get along with the crash point a lot more info about the state of the system at the time of the crash.

So, I hope LKCD is looked upon favorably by the Penguin Gods. :)

Matt D. Robinson clarified:

A lot of progress has been made ...

LKCD is currently up to 2.4.17, and should be simple to move into 2.5.

I've asked Linus and others for inclusion in the past, and I think it deserves consideration for 2.5, especially now that it can be built into the kernel without having to be turned on. At least in those cases, people who want it can turn it on, and those that don't will never see it.

The LKCD development team (over 10 engineers now) is ready.

Go Taniguchi said that LKCD worked perfectly under 2.4.17, saved a lot of time, and was very important for developers. He recommended including it in the main kernel tree.

6. VFS Documentation

14 Feb 2002 - 15 Feb 2002 (2 posts) Archive Link: "[ANNOUNCE] new VFS documentation"

Topics: FS: InterMezzo, FS: UMSDOS, Virtual Memory

People: Alexander ViroRandy Dunlap

Alexander Viro announced:

First of all, since 2.5.5-pre1 there is an up-to-date guide for porting filesystems from 2.4 to 2.5.<latest>. Location:

Documentation/filesystems/porting

It WILL be kept up-to-date. IOW, submit an API change that may require filesystem changes without a corresponding patch to that file and I will hunt you down and hurt you. Badly.

The same document covers "what do I need to change to keep my out-of-tree filesystem uptodate". So watch for changes there.

Normally when API change happens, the person doing it is responsible for updating all in-tree filesystems or, at least, warning people about the breakage. Applying the list of broken filesystems.

Right now that list consists of umsdos and intermezzo. The former will be fixed after the next series of file_system_type cleanups. The latter is a victim of current changes in locking scheme. Help from intermezzo folks would be a good idea - preferably in the form that would reduce the dependency on the VFS guts.

New locking scheme is described in Documentation/filesystems/directory-locking. In details and with proof of correctness.

It doesn't change the exclusion warranties for filesystems, so unless they mess with locking in non-trivial ways (intermezzo was the only in-tree example) they shouldn't need any changes. Some things might become simpler, actually (i.e. in some cases private locking became redundant and can be dropped). Again, see Documentation/filesystems/porting for details of changes.

Documentation/filesystems/Locking is slowly getting up-to-date. Descriptions of several superblock methods are still missing and I would really appreciate it if folks who had introduced them would document them.

Randy Dunlap replied:

Way to go, Al!

Anyone: Are there any linux-filesystem-like web pages, sorta like what Rik has for MM/VM?

Here's a beginning, if someone would like to use it or add to it:

http://www.osdl.org/archive/rddunlap/linux-fs.html

There was no reply.

7. XFS And rmap

15 Feb 2002 - 17 Feb 2002 (7 posts) Archive Link: "XFS + rmap?"

Topics: FS: XFS, FS: ext2, Virtual Memory

People: Rik van RielShawn StarrJim Dennis

Jim Dennis asked if anyone had any problems using XFS with the new rmap VM patches, and Rik van Riel replied:

Shawn Starr has been using this combination.

The only problem he found was that XFS and -rmap make changes to the same source code files, so he had to merge some of the code by hand.

Shawn Starr added that he'd be releasing a patch to do this easily. Jim asked where to find the patch, and Shawn pointed him to http://xfs.sh0n.net/2.4. A couple days later he announced, "-shawn6 is how out. removed pre9-ac4 breaks XFS quota/EXT2/3 quota."

8. BitKeeper Wrangling

19 Feb 2002 - 20 Feb 2002 (16 posts) Archive Link: "[PATCH *] new struct page shrinkage"

Topics: Big Memory Support, Version Control

People: Rik van RielLinus TorvaldsLarry McVoyAndreas DilgerEd TomlinsonJeff Garzik

Rik van Riel posted a patch, and added:

I've also pulled the thing up to your latest changes from linux.bkbits.net so you should be able to just pull it into your tree from:

bk://linuxvm.bkbits.net/linux-2.5-struct_page

You can also view the patch on:

http://surriel.com/patches/2.5/2.5.5-p2-struct_page5

Linus Torvalds replied:

Btw, _please_ don't do things like changing the bitkeeper etc/config file. Right now your very first changesets is something that I definitely do not want in my tree.

Sure, I can do "bk cset -x" on the damn thing, but the fact is, I don't want to have totally unnecessary undo's in my tree on things like this. That's just stupid, and only makes the revision history look even less readable than it already is..

Rik replied:

Woooops, I was trying to make the overview on linuxvm.bkbits.net display something sensible but didn't realise you'd be pulling that back into your tree ;((((

I'll make sure to not make this mistake again.

Later he added:

Since bk doesn't seem to let me remove the thing from the history (probably with some good reason), I guess you might as well import the following patch:

http://surriel.com/patches/2.5/2.5.5-p2-struct_page5

As a side effect, this patch should reduce the whole thing to one changeset, which isn't all bad since we don't need to have the history of Linux cluttered up with all the minor changes to this patch ;)

Elsewhere, under the Subject: [PATCH] struct page, new bk tree (http://www.uwsg.indiana.edu/hypermail/linux/kernel/0202.2/0634.html) , Rik said:

I've removed the old (broken) bitkeeper tree with the struct page changes and have put a new one in the same place ... with the struct page changes in one changeset with ready checkin comment.

You can resync from bk://linuxvm.bkbits.net/linux-2.5-struct_page and you'll see that the stupid etc/config change is no longer there.

If you want to wait a version with pulling this change because of the pte_highmem changes by Ingo and Arjan I can understand that and will just bug you again in a version or so ;)

Larry McVoy replied:

Since you two are doing the BK dance, here's a question for you: I can imagine that this sort of back and forth will happen quite a bit, someone makes a change, then Linus (or whoever) says "no way", and the developer goes back, cleans up the change, and repeats. That's fine for Linus & Rik because Linus tosses the changeset and Rik tosses it, but what about the other people who have pulled? Those changesets are now wandering around in the network, just waiting to pop back into a tree.

This is at the core of my objections to the "reorder the events" theme which we had a while back. You can reorder all you want, but if there are other copies of the events floating around out there, they may come back.

A long time ago, there was some discussion of a changeset blacklist. The idea being that if you want to reorder/rewrite/whatever, and your changes have been pulled/pushed/whatever, then it would be good to be able to state that in the form of some list which may be used to see if you have garbage changesets.

We could have a --blacklist option to undo which says "undo these changes but remember their "names" in the BitKeeper/etc/blacklist file. The next changeset you make will check in that file. Note that each changeset has a unique name which is used internally, somewhat like a file has an inode number. So we can save those names. Then if you do a pull or someone does a push, the incoming csets can be compared with the blacklist and rejected if found.

Do you think this would be useful? Would you use it if we made it?

Andreas Dilger asked, "what happens to the person who pulled the (now-blacklited) CSET in the first place? If they do a pull from the repository where the original CSET lived, will the blacklisted CSET be undone and the replacement CSET be used in its place?" And Rik said, "That's a good question. I hadn't answered Larry before because I just couldn't come up with what the implications of a blacklist would be or how it would ever work ..."

Elsewhere, Ed Tomlinson also replied to Larry, saying, "In my opinion the idea of cset -x (while usefull) is fundamentally broken. The result of this is that ideas like blacklist need to be considered. I would propose instead an undo -x, that would generate a cset to reverse the one following the -x. This might lead to conflicts - these would be resolved the normal bk fashion. If bk handled ?bad? csets in this manner there would be no need for blacklists - it is more robust in that you can always used undo -x." Jeff Garzik replied, "Well, if the changes are properly split up, you shouldn't need to do this... In the ideal situation it is easiest for Linus to accept or reject a "bk pull" in its entirety. Then he can just do a "bk unpull"" And Larry also replied to Ed, saying:

First of all, cset -x is functionally equivalent to what you call undo -x. They do the same thing. Second of all, cset -x is _much_ better. It does the same thing without introducing any new diffs into the history. Go get a test tree, make a changeset, clone the tree, cset -x the changeset, and diff the revision history files. All you will see is something like this:

^As 00000/00000/00455
^Ad D 1.32 02/02/20 09:50:05 lm 33 32
^Ax 32
^Ac Exclude
^AcC
^AcK50774
^Ae

The "^Ax 32" line says "exclude the change who's serial number is 32". No reverse diffs applied to the file. Much nicer. Merges work like this too, in reverse, it just includes the branch deltas.

But all of this misses the real point - Linus, with justification, doesn't want the revision history cluttered up with

Idea 1.

Remove Idea 1.

Idea 2.

Remove Idea 2.

But we need some way to let changes get into the system so others can review them, test them, merge them with their stuff and test, etc. But then when they are found to be wanting, we need a way to tell other people that those csets are verboten.

I'm open to suggestions, this is a much harder problem than it appears because of the fact that the revision histories are all replicas possibly with local data. Unlike CVS, there is no one place to go to edit the RCS files and obliterate some change.

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.