Kernel Traffic #155 For 25 Feb 2002

You are not serious? Aren't you? 'Bit of paranoia there?

Can't you just imagine somebody who did get out of the coding business in professional terms but stills to enjoys hacking from time to time? If you are impressed by my speed, and can't imagine that what I'm doing is the outcome of just a few spare houers at home, then well I certainly can't help you....

PS. If I would have a hidden agenda I wouldn't post from the evision domain becouse I know a bit about mailing systems as well ;-). And I would care to be polite and a slick ass to everybody, which I CERTAINLY don't you s******

If that's not part of the roadmap I'd be surprised, since tux is so much more capable than the khttpd which is currently part of the tree.

Tux has clearly demonstrated it's performance and low resource consumption.

I also think including tux in the mainline kernel would further stimulate development, as folks begin to realize what they can do with tux....

Linus put khttpd in the kernel just after sendfile support was added, IIRC he said something about khttpd being a very small number of lines to add once sendfile support was there.

if it's really that small (IIRC <<100 lines of code) it's still in there becouse it's not worth ripping out.

I would like to hear people's opinions on making the keys in the /proc hierarchy consistent wrt the space character. The current Linux Documentation/filesystems.proc.txt does not suggest any standard naming conventions. F.E. cat /proc/cpuinfo (partial list)

cpu family      : 5
model           : 9
model name      : AMD-K6(tm) 3D+ Processor
stepping        : 1
cpu MHz         : 400.907
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no

Notice the space between "cpu" and "MHz", or "cpu" and "family" yet there is no space between "fdiv" and "bug" (_).

The reason I think NOT using a space is a good idea because it makes life easier for developers parsing /proc entries. Specifically, Java developers could use /proc/cpuinfo as a property file, but the space in the 'key' breaks java.util.Properties.load().

Personally, I have LONG wished all /proc/* entries were shell parsable values, eg:

cpu_family=5
model=9
model_name='AMD-K6(tm) 3D+ Processor'

etc. The amound of userland parsing code this would obviate must be quite large, and it certainly would make simple scripts to do things with the values much easier.

instead, there are all these "pretty" files, with formatting code that properly belongs in userland tools residing in the kernel (yes, mostly just printf but still).

as a maintainer of "linux_logo" which delves heavily (and perhaps unwisely) into /proc/cpuinfo and its ilk, I would definitely say I would not be jumping at the opportunity.

Remember as maintainers of userspace apps, we have to keep compatibility, in this case for me it involves /proc/cpuinfo from all the architectures, plus kernels going back to the 1.2.13 time-frame.

So changing /proc/cpuinfo yet again does not simplify the code, in fact it just adds one more incompatible special case.

And since 2.2 and 2.4 kernels will be around for ages to come, it will make code bigger rather than smaller.

I agree a cleanup, if done properly, would be welcomed. but don't use "simplification of user-space code" as an argument.. because it's a lie ;)

bk send -wgzip_uu -r<rev> - > foo-<rev>.bk

This will dump a gzipped-uuencoded changset to the file. The receiver just do "| bk receive [repository] -avv" to import it on the other end.

My preferred format for sending BK CSETs is below. The gzip_uu CSET data only adds maybe 10% for large patches, and about doubles the size of very small patches. I also created a bz64 (bzip2 + base64) wrapper which makes the CSET data smaller, but that is only useful if other BK developers have this wrapper also.

#!/bin/sh
# A script to format BK changeset output in a manner that is easy to read.
# Andreas Dilger <adilger@turbolabs.com> 13/02/2002
 
PROG=bksend
 
usage() {

echo "usage: $PROG -r<rev>"
echo -e "\twhere <rev> is of the form '1.23', '1.23..', '1.23..1.27',"
echo -e "\tor '+' to indicate the most recent revision"
 
exit 1

}
 
case $1 in
-r) REV=$2; shift ;;
-r*) REV=`echo $1 | sed '/kernel-traffic/s/^-r/index.html'` ;;
*) echo "$PROG: no revision given, you probably don't want that";;
esac
 
[ -z "$REV" ] && usage
 
bk changes -r$REV
bk export -tpatch -du -h -r$REV
echo -e "\n================================================================\n\n"
bk send -wgzip_uu -r$REV -

I talked with Matt Robinson(sp?) at LinuxWorld-NY about LKCD for a few minutes... he gave me the impression that a lot of progress has been made recently. IBM apparently has some guys working on it, too.

I've always thought Linux needs industrial strength crash dumps like the other Unices. There are many benefits, but my own is self interest: bug reports get 1000 times better, since you get along with the crash point a lot more info about the state of the system at the time of the crash.

So, I hope LKCD is looked upon favorably by the Penguin Gods. :)

A lot of progress has been made ...

LKCD is currently up to 2.4.17, and should be simple to move into 2.5.

I've asked Linus and others for inclusion in the past, and I think it deserves consideration for 2.5, especially now that it can be built into the kernel without having to be turned on. At least in those cases, people who want it can turn it on, and those that don't will never see it.

The LKCD development team (over 10 engineers now) is ready.

First of all, since 2.5.5-pre1 there is an up-to-date guide for porting filesystems from 2.4 to 2.5.<latest>. Location:

Documentation/filesystems/porting

It WILL be kept up-to-date. IOW, submit an API change that may require filesystem changes without a corresponding patch to that file and I will hunt you down and hurt you. Badly.

The same document covers "what do I need to change to keep my out-of-tree filesystem uptodate". So watch for changes there.

Normally when API change happens, the person doing it is responsible for updating all in-tree filesystems or, at least, warning people about the breakage. Applying the list of broken filesystems.

Right now that list consists of umsdos and intermezzo. The former will be fixed after the next series of file_system_type cleanups. The latter is a victim of current changes in locking scheme. Help from intermezzo folks would be a good idea - preferably in the form that would reduce the dependency on the VFS guts.

New locking scheme is described in Documentation/filesystems/directory-locking. In details and with proof of correctness.

It doesn't change the exclusion warranties for filesystems, so unless they mess with locking in non-trivial ways (intermezzo was the only in-tree example) they shouldn't need any changes. Some things might become simpler, actually (i.e. in some cases private locking became redundant and can be dropped). Again, see Documentation/filesystems/porting for details of changes.

Documentation/filesystems/Locking is slowly getting up-to-date. Descriptions of several superblock methods are still missing and I would really appreciate it if folks who had introduced them would document them.

Way to go, Al!

Anyone: Are there any linux-filesystem-like web pages, sorta like what Rik has for MM/VM?

Here's a beginning, if someone would like to use it or add to it:

http://www.osdl.org/archive/rddunlap/linux-fs.html

Shawn Starr has been using this combination.

The only problem he found was that XFS and -rmap make changes to the same source code files, so he had to merge some of the code by hand.

I've also pulled the thing up to your latest changes from linux.bkbits.net so you should be able to just pull it into your tree from:

bk://linuxvm.bkbits.net/linux-2.5-struct_page

You can also view the patch on:

http://surriel.com/patches/2.5/2.5.5-p2-struct_page5

Btw, _please_ don't do things like changing the bitkeeper etc/config file. Right now your very first changesets is something that I definitely do not want in my tree.

Sure, I can do "bk cset -x" on the damn thing, but the fact is, I don't want to have totally unnecessary undo's in my tree on things like this. That's just stupid, and only makes the revision history look even less readable than it already is..

Woooops, I was trying to make the overview on linuxvm.bkbits.net display something sensible but didn't realise you'd be pulling that back into your tree ;((((

I'll make sure to not make this mistake again.

Since bk doesn't seem to let me remove the thing from the history (probably with some good reason), I guess you might as well import the following patch:

http://surriel.com/patches/2.5/2.5.5-p2-struct_page5

As a side effect, this patch should reduce the whole thing to one changeset, which isn't all bad since we don't need to have the history of Linux cluttered up with all the minor changes to this patch ;)

I've removed the old (broken) bitkeeper tree with the struct page changes and have put a new one in the same place ... with the struct page changes in one changeset with ready checkin comment.

You can resync from bk://linuxvm.bkbits.net/linux-2.5-struct_page and you'll see that the stupid etc/config change is no longer there.

If you want to wait a version with pulling this change because of the pte_highmem changes by Ingo and Arjan I can understand that and will just bug you again in a version or so ;)

Since you two are doing the BK dance, here's a question for you: I can imagine that this sort of back and forth will happen quite a bit, someone makes a change, then Linus (or whoever) says "no way", and the developer goes back, cleans up the change, and repeats. That's fine for Linus & Rik because Linus tosses the changeset and Rik tosses it, but what about the other people who have pulled? Those changesets are now wandering around in the network, just waiting to pop back into a tree.

This is at the core of my objections to the "reorder the events" theme which we had a while back. You can reorder all you want, but if there are other copies of the events floating around out there, they may come back.

A long time ago, there was some discussion of a changeset blacklist. The idea being that if you want to reorder/rewrite/whatever, and your changes have been pulled/pushed/whatever, then it would be good to be able to state that in the form of some list which may be used to see if you have garbage changesets.

We could have a --blacklist option to undo which says "undo these changes but remember their "names" in the BitKeeper/etc/blacklist file. The next changeset you make will check in that file. Note that each changeset has a unique name which is used internally, somewhat like a file has an inode number. So we can save those names. Then if you do a pull or someone does a push, the incoming csets can be compared with the blacklist and rejected if found.

Do you think this would be useful? Would you use it if we made it?

First of all, cset -x is functionally equivalent to what you call undo -x. They do the same thing. Second of all, cset -x is _much_ better. It does the same thing without introducing any new diffs into the history. Go get a test tree, make a changeset, clone the tree, cset -x the changeset, and diff the revision history files. All you will see is something like this:

^As 00000/00000/00455
^Ad D 1.32 02/02/20 09:50:05 lm 33 32
^Ax 32
^Ac Exclude
^AcC
^AcK50774
^Ae

The "^Ax 32" line says "exclude the change who's serial number is 32". No reverse diffs applied to the file. Much nicer. Merges work like this too, in reverse, it just includes the branch deltas.

But all of this misses the real point - Linus, with justification, doesn't want the revision history cluttered up with

Idea 1.

Remove Idea 1.

Idea 2.

Remove Idea 2.

But we need some way to let changes get into the system so others can review them, test them, merge them with their stuff and test, etc. But then when they are found to be wanting, we need a way to tell other people that those csets are verboten.

I'm open to suggestions, this is a much harder problem than it appears because of the fact that the revision histories are all replicas possibly with local data. Unlike CVS, there is no one place to go to edit the RCS files and obliterate some change.