Kernel Traffic #210 For 23�Mar�2003

I've created little project for read-only (for now ;-) bitkeeper clone. It is available at www.sf.net/projects/bitbucket (no tar balls, just get it fresh from CVS).

Part of readme follows.

Install CSSC from <http://cssc.sf.net/> (it is also available as Debian package). You may need to apply cssc.diff.

Here you get following tools:

bcheckout_HEAD:
extracts files from BK repository. You can get repository by rsync -zav --delete nl.linux.org::kernel/linux-2.5 .

bpull:
pull new version of repository, compute differences from the last time and apply them to directory with *your* sources.

bdiff:
compare two versions (specify versions from top-level s.ChangeSet)

To get a list of all changesets, do prs linux-2.5/SCCS/s.ChangeSet.

Thank you for taking some initiative and improving this situation by constructive means. You are an example to us all, as is Andrea Arcangeli with his openbkweb project, which you will probably want to examine and perhaps integrate (ftp://ftp.kernel.org/pub/linux/kernel/people/andrea/openbkweb).

bitbucket is about 350 lines of shell scripts, documentation and diffs, the most interesting file of which is FORMAT, which documents some reverse engineering efforts on bitkeeper internal file formats. bitkbucket currently uses rsync to update data from the repository. openbkweb is 500+ lines of python that implements enough of the bitkeeper network protocol to do downloads, although perhaps in inefficiently. That sounds like some functionality that you might be interested in integrating.

I think the suggestion made by Pavel Janik that it would be better to work on adding BitKeeper-like functionality to existing free software packages is a bit misdirected. BitKeeper uses SCCS format, and we have a GPL'ed SCCS clone ("cssc"), so you are adding functionality to existing free software version control code anyhow.

However, I would like to turn Pavel Janik's point in what I think might be a more constructive direction.

Aegis, BitKeeper and probably other configuration management tools that use sccs or rcs basically share a common type of lower layer. This lower layer converts a file-based revision control system such as sccs to an "uber-cvs", as someone called it in a slashdot discussion, that can:

1. process a transaction against a group of files atomically,
2. associate a comment with such a transaction rather than with just one file,
3. represent symbolic links, file protections 4. represent file renames (and perhaps copies?)

You might want to keep in the back of your mind the possibility of someday splitting off this lower level into a separate software package that programs like your bitkeeper clone, aegis could use in common. If the interface to this lower level took cvs commands, then it could probably replace cvs, although the repository would probably be incompatible since the meaning of things like checking in multiple files together with a single comment would be different, and there would be other kinds of changes to represent beyond what cvs currently does. Using a repository format that is compatible with another system (for example bitkeeper or aegis) would make such a tool more useful, and if such a tool makes it easier for people to migrate from a prorprietary system to a free one, that's even better, so your starting with bitkeeper's format seems like an excellent choice to me.

Thanks again for starting this project. I will at least try to be a user of it.

"BK format"? Not really. Patches have been posted (to lkml, even) to GNU CSSC which allow it to read SCCS files BK reads and writes.

Since that already exists, a full BitKeeper clone is IMO a bit silly, because it draws users and programmers away from projects that could potentially _replace_ BitKeeper.

While people would certainly use it, I can't help but think that a BK clone would damage other open source SCM efforts. I call this the "SourceForge Syndrome":

Q. I found a problem/bug/annoyance, how do I solve it?
A. Clearly, a brand new sourceforge project is called for.

My counter-question is, why not improve an _existing_ open source SCM to read and write BitKeeper files? Why do we need yet another brand new project?

AFAICS, a BK clone would just further divide resources and mindshare. I personally _want_ an open source SCM that is as good as, or better, than BitKeeper. The open source world needs that, and BitKeeper needs the competition. A BK clone may work with BitKeeper files, but I don't see it ever being as good as BK, because it will always be playing catch-up.

Normally, I'd agree with you Jeff. However, none of the current open source SCM systems are architected in a way that can operate like BK.

I've been using subversion for a while now. It pretty much fixes all the problems that CVS had, AS LONG AS you accept the CVS style of version control. That style doesn't work for non-central work like the kernel.

The one thing BK does that makes it worthwhile is the three-way merge. This (and the resulting DAG) make handling code from Alan, from Linus, from Andrew, and from everyone else possible. With CVS, subversion, or any other SCM I've worked with, you have to hand merge anything past the first patch. Ugh.

This requires architecture, and (AFAIK) BitBucket is the first try at it. Compatibility with the proprietary tool that does it already is a good thing.

5. Represent merges. That's what is making cvs branches unusable.

Frankly, if you want all of that you'd better design a repository format that is actually adapted to it. The RCS format is not very good, the SCCS weave is a little better but not by much (it reminds me of Hurd, looks cool but slow by design). Larry did quite a feat turning it into a distributed DAG of versions but I'm not convinced it was that smart, technically. In particular, everthing suddendly looks much nicer when you have one file per DAG node plus a cache zone for full versions.

But anyway, what made Bitkeeper suck less is the real DAG structure. Neither arch nor subversion seem to have understood that and, as a result, don't and won't provide the same level of semantics. Zero hope for Linus to use them, ever. They're needed for any decently distributed development process.

Hell, arch is still at the update-before-commit level. I'd have hoped PRCS would have cured that particular sickness in SCM design ages ago.

Atomicity, symbolic links, file renames, splits (copy) and merges (the different files suddendly ending up being the same one) are somewhat important, but not the interesting part. A good distributed DAG structure and a quality 3-point version "merge" is what you actually need to build bk-level SCMs.

Calling A and B the versions to merge and R the reference, diff3 uses this algorithm (probably the simplest possible):

• Compute the diff between A and R, call it dA
• Compute the diff between B and R, call it dB
• Merge the two diffs into one (and conflict where you can't)
• Apply the merged diff to R

Better algorithms do the alignments per-character instead of per-line, detect moved and changed functions, detect duplicate inserts, etc. None, of course, is perfect, as Larry could tell you.

Now if the development went that way:

1.7  -> 1.7.1.1 (branching, i.e. copy)
 v         v
 v      1.7.1.2
1.8        v
 v   -> 1.7.1.3 (merge)
1.9        v
 v         v  
1.10       v
 v   -> 1.7.1.4 (merge)
 v         v
 v      1.7.1.5
 v         v
1.11 <-         (merge)

Pretty much standard, a developper created a new branch, made some changes in it, synced with mainline, synced with mailine again a little later, made some new changes and finally folded the branch back in the mainline. Let's admit the developper changes don't conflict by themselves with the mainline changes.

CVS, for all the merges, is going to pick 1.7 as the reference. The first time, for 1.7.1.3, it's going to work correctly. It will fuse the 1.7->1.8 patch with the 1.7.1.1->1.7.1.2 patch and apply the result to 1.7 to get 1.7.1.3. The two patches have no reason to overlap. 1.7.1.2->1.7.1.3 will essentially be identical to 1.7->1.8, and 1.8->1.7.1.3 will essentially be identical to 1.7.1.2->1.7.1.3.

As soon as the next merge, i.e 1.7.1.4, it breaks. CVS is going to try to fuse the 1.7->1.10 patch with the 1.7->1.7.1.3 patch. But 1.7->1.10 = 1.7->1.8+1.8->1.10 and 1.7->1.7.1.3 ~= 1.7->1.7.1.2+1.7->1.8. So they have components in common, hance they _will_ conflict.

If CVS had taken the latest common ancestor by keeping in the repository the existence of the 1.8->1.7.1.3 link, it would have taken the 1.8 version as the reference. The patches to fuse would have been 1.8->1.10 and 1.8->1.7.1.3, which have no reason to conflict.

Same for the next merge, the optimal merge point is in that case 1.10, and it ends up being a null merge, i.e. 1.11 is a copy of 1.7.1.5.

You can see the final structure is a DAG, with each node having a max of 2 ancestors. And that's what PRCS and bk are working with, fundamentally.

Wrong.

Now think three trees. Each merging back and forth between each other.

Or, in the case of something like the Linux kernel tree, where you don't have two or three trees. You've got at least 20 actively developed concurrent trees with branches at different points.

Trust me. CVS simple CANNOT do this. You need the full information.

Give it up. BitKeeper is simply superior to CVS/SVN, and will stay that way indefinitely since most people don't seem to even understand _why_ it is superior.

There are parts of BitKeeper which required multiple years of thought by people a lot smarter than me. You guys are under the mistaken impression that BitKeeper is my doing; it's not. There are a lot of people who work here and they have some amazing brains. To create something like BK is actually more difficult than creating a kernel.

To understand why, think of BK as a distributed, replicated, version controlled user level file system with no limits on any of the file system events which may happened in parallel. Now put the changes back together, correctly, no matter how much parallelism there has been. Pavel hasn't understood anything but a tiny fraction of the problem space yet, he just doesn't realize it. Even Linus doesn't know how BitKeeper works, we haven't told him and I can tell from his explanations that he gets part of it but not most of it. That's not a slam on Linus or Pavel or anyone else. I'm just trying to tell you guys that this stuff is a lot harder than you think. I've told people that before, like the SVN and OpenCM guys, and the leaders of both those efforts showed up later and said "yup, you're right, it is a hell of a lot harder than it looks". And they are nowhere near being able to do what BK does. Ask them if you have doubts about what I am saying.

Merging is just one of the complex areas. It gets all the attention because it is hard enough but easy enough that people like to work on it. It's actually fun to work on merging. Ditto for the graph structure, that's trivial. The other parts aren't fun and they are more difficult so they don't get talked about. But they are more important because the user has no idea how to deal with them and users do know how to deal with merge problems, lots of you understand patch rejects.

Rename handling in a distributed system is actually much harder than getting the merging done. It doesn't seem like it is, but we've rewritten how we do it 3 times and are working on a 4th all because we've been forced to learn about all the different ways that people move things around. CVS doesn't have any of the rename problems because it doesn't do them, and SVN doesn't have 1/1000th of the problems we do because it is centralized. Centralized means that there is never any confusion about where something should go, you can only create one file in one directory entry because there is only one directory entry available. In BK's case, there can be an infinite number of different files which all want to be src/foo.c.

Symbolic tags are really hard. What?!? What could be easier than adding a symbolic label on a revision? Well, in a centralized system it is trivial but in a distributed system you have to handle the fact that the same symbol can be put on multiple revs. It's the same problem as the file names, just a variation. Add to that the fact that time can march forward or backwards in a distributed system, even if all the events were marching forward, and the fun really starts. I personally have redone the tags support about 6 times and it still isn't right.

Security semantics are hard in a distributed system. Where do you put them, how do you integrate them into the system, what happens when people try and work around them? In CVS or SVN you can simply lock down the server and not worry about it, but in BK, the user has the revision history and they are root, they can do whatever they want.

Time semantics are the hardest of all. You simply can't depend on time being correct. It goes forwards, backwards, and sideways on you and if you think you can use time you don't have the slightest idea of the scope of the problem. Again, not a problem for CVS/SVN/whatever, all the deltas are made against the same clock. Not true in a distributed system.

That's a taste of what it is like. You have to get all of those right and the many other ones that I didn't tell you about or you might as well not bother. Why? Because the problems are very subtle and there isn't any hope of getting an end user to figure out a subtle problem, they don't have the time or the inclination. We've seen users throw away weeks of work just because they didn't understand the merge conflict so they start over on an updated tree. And those people will understand the rename corner cases? Not a chance.

The main point here is that if you think that BK happened quickly, by one guy, you are nuts. It started in May of 1997, that's almost 6 years ago, not the 2 years that Pavel thinks, and I had already written a complete version control system prior to that, so this was round two. Even with that knowledge, I wasn't near enough to get BK to where it is today, there is more than 40 man years of effort in BK so far. A bunch of people, working 60-90 hour weeks, for almost 6 years. Not average people, either, any one of these people would be a staff engineer or better at Sun (salaries for those people are in the $115K - $140K range).

The disbelievers think that I'm out here waving the "it's too hard" flag so you'll go away. And the arrogant people think that they are smarter than us and can do it quicker. I doubt it but by all means go for it and see what you can do. Just file away a copy of this and let me know what you think three or four years from now.

Oh, by the way, you'll need a business model, I found that out 2 or 3 years into it when my savings ran out. Oh, my, you might not be able to GPL it! Why it might even end up being just like BitKeeper with an evil corporate dude named Pavel running the show. Believe me, if that happens, I'll be here to rake him over the coals on a daily basis for being such an evil person who doesn't understand the point of free software. I can't wait.

Version 2.5.0 of perfctr, the Linux/x86 performance monitoring counters driver, is now available at the usual place: http://www.csd.uu.se/~mikpe/linux/perfctr/

Quick guide to porting source code from perfctr-2.4 to perfctr-2.5:

1. 'evntsel_aux[]' in the x86 struct perfctr_cpu_control is now 'p4.escr[]' (moved to sub-struct and renamed).
2. 'nrcpus' is gone from the perfctr info struct. It can be computed from the 'cpus' bitmask.
3. 'version[]' in perfctr info is now 'driver_version[]'.
4. The VPERFCTR_STOP ioctl is gone. Use VPERFCTR_CONTROL instead. The library's vperfctr_stop() procedure still works.
5. The API to global-mode performance counters has changed. The target CPU number is now included in the per-cpu control and state arrays. See examples/global/global.c.

Other noteworthy changes from perfctr-2.4:

1. The patch kit supports aliases, which allows kernels to share patch files. The 'update-kernel' script handles this.
2. 'make install' will now install the library, include files, and the 'perfex' example program at user-specified locations.
3. The library will now check that the ABI the library was compiled for is compatible with the kernel driver.
4. A preliminary library API for remote access to other processes' virtual performance counters has been added.
5. The library now contains proper data structures with event set and unit mask descriptions. This handles all supported CPUs except for P4 (mostly done, but some tricky details remain).
6. A perfctr-patched kernel can now be compiled on non-x86 archs without causing compilation errors.
7. Global-mode (non pre-process) counters now work in 2.5 kernels, and on hyper-threaded P4s. Several API changes were needed for this.

Version 2.5.0, 2003-03-10

• Added a simple user-space library API for accessing other processes' virtual performance counters. This uses a new type and a new set of operations since remote access has different requirements than accessing one's own counters. Following Mike Marty's suggestion, I left out the process control calls needed around these operations (ptrace() and wait()), so applications must handle that themselves.
• Added 'make install' support for the user-space components.
• Driver API cleanups. The 'eventsel_aux[]' array in 'struct perfctr_cpu_control' has been renamed as 'escr[]' and has been moved into the 'p4' sub-structure. (The change highlights the fact that this field was and is P4-only.) The 'version[]' string in 'struct perfctr_info' has been renamed to 'driver_version[]', since perfctr_info now also contains an 'abi_version' field. Some changes in the driver ABI: while not strictly necessary, they clean things up and make room for future changes. The ABI changed anyway from perfctr-2.4, so this shouldn't be a problem.
• Added a perfctr_cpu_control_print() procedure to the library, and updated the example programs to use it.
• Updated the perfex example program's help text to describe the syntax and meaning of event specifiers.
• Patch kit updates for 2.2.24/2.4.18-26(RedHat)/2.5.64 kernels.

Perhaps this has been asked for a million times, but I'm new to perfctrs...

Is there any tool available to profile a program based on this ? I have seen perfex, but that gives total counts. I would like something like gprof... We are now optimizing some software and I would like to make my colleagues leave Windows (they use Intel's VTune) and go to Linux.

Or at least compare the same kind of things between VTune on win and 'something' in Linux that also uses the counters. They don't seem to trust gprof. And, looking at the results, I'm beginning to untrust VTune...

I've modified your vmregress test form linux 2.4 so it works on linux 2.5.63. I fixed up some of the core routines to understand new modifications to kernel vm structures, so it all compiles and runs.

There are still some issues with the perl scripts that collect data and pipe it to gnuplot. Some of the plots of vmstat output don't work.

You can see what I've modified at the URL

http://www.osdl.org/archive/dmo/VMREGRESS/

There's a VMR_README file that describes the files there.

Excellent stuff, thanks. This prompted me to dust off my old test box, get 2.5.64 on it, new modutils etc etc and get working back on VM Regress. I'm releasing 0.8a for you to take a look at it, it is available from http://www.csn.ul.ie/~mel/projects/vmregress/ and directly downloadable from http://www.csn.ul.ie/~mel/projects/vmregress/vmregress-0.8a.tar.gz .

I'm not going to be working on this for another few days (hence the rushed release) so I wanted you to see what I did with your stuff so far. Rather than merging your stuff blindly, I took a few extra steps

1. The changes you made broke VM Regress from working with 2.4.anything so I sat down and made the configure script understand both 2.4 and 2.5 build systems. The way it is now, a

./configure --with-linux=/usr/src/linux-2.x.x
make

will compile the modules and be loadable against 2.4.20 and 2.5.64 . It generates different makefiles depending on the kernel release and I'm fairly sure I got it right. My "extensive" testing involved loading the vmregress_core, sizes.o and zones.o modules and printing the reports. They worked fine for that, but note the "a" part of 0.8a, I didn't test anything else.

2. vmr_mmzone.h is going to be my mapping between the different names of structs and fields between 2.4 and 2.5 . this (you'll love this) involves having #defines which map some VM Regress type to the actual kernel structure. So for example, on 2.4 it's

#define C_ZONE struct zone_t

and 2.5 it's

#define C_ZONE struct zone

This looks (and is) horrible but there is very few name discrepancies so it doesn't confuse things too much and I reckon it is better than having a 2.4 and 2.5 version of VM Regress for such minor differences.

3. I kept my old indenting style, but I think I incorporated all your bug fixes. I need to double check this though, I have a few other bugs on the ToDo list I want to clean up anyway

4. I blindly included the OSDL scripts into an osdl/ directory. I haven't looked at them in detail yet, I presume they are doing magic OSDL stuff for the moment until I get back onto it. If I don't though, I would still like to get the OSDL tests integrated into the tool rather than having them separate.

I think I have 99% if not 100% of your work integrated in nicely so that VM Regress still works on both sets of trees. The rudimentary ChangeLog so far is below

Version 0.8a

• Minor bug fixes in the core
• OSDL based merging
• Fixed the extract_structs.pl script to ensure its a struct been extracted
• Move the creation of internal.h from Makefiles to the configure script
• Use configure script to apply kernel patch if requested
• Read kernel release version directly from Kernel makefile
• Automatically generate makefiles depending on kernel version from configure
• Teach extract_structs.pl to identify a struct that is typedef'd
• vmr_mmzone.h has been expanded to map between different struct and field names between kernel versions. Not many differences thankfully

I never used it to receive netflow data.

I used netramet to generate flow information on a linux-based backbone router, and to periodically move the flow information to another node for batch processing/analysis.

I used the netramet tools to extract that information from the netramet "meter" (the flow measurement daemon running on the router), for storage on the remote accounting/processing machine. Some custom Perl scripts process the flow information thereafter.

You asked for netflow data export. Netramet can give you something similar to netflow (I never used netflow, but from what I hear, netramet is similar only more flexible).

Ok.

I suspect it's easier to patch netramet, than start a new kernel-based project.

If Netramet really supports reading netflow data, I guess it would not be a huge undertaking to implement write support too. But I'm not familiar with the Netramet source - really, you should look at it yourself or ask the developers.

NetFlowMet is the NetFlow module for netramet. But it's only a receiver, not a transmitter.

However, 'ntop' can generate NetFlow packets from a Linux system, using libpcap.

I suggest you read http://www.switch.ch/tf-tant/floma/software.html

I'm getting to the point of completely giving up kernel development outside the ARM tree, which basically means leaving serial, pci, and pcmcia to other people.

I have been trying for the last 5 days to get Linus to accept two patches small patches:

1. a patch to register tty_devclass using postcore_initcall().

   This is necessary before I can push the next set of serial changes. Without it, the serial layer will oops in sysfs when it tries to register drivers.

2. a patch to restore the PCI device probing, so we don't probe all functions when we discover that function 0 is not present. This is the behaviour we had prior to 2.5.64.

Both of these patches have been sent to Linus several times over the past 5 days, and each time they have been completely ignored without explaination.

I have a huge pile of patches backing up here. I'm at the point where I'm going to give up updating them to bk-curr, and testing them before sending each to Linus due to the number of patches. Going around this loop just takes too much of my time to make it worth while.

My backlog currently consists of:

8 PCI patches (1 needs to be further cut up)
10 PCMCIA patches (1 needs to be further cut up)
1 tty_io.c patch
16 serial csets

there are some dependencies between these patches, and getting the stuff applied in the right order is absolutely paramount to ensure that things don't break.

So, any suggestions on how to handle this better, and above all get Linus to start applying stuff?

ftp://ftp.<country>.kernel.org/pub/linux/utils/kernel/modutils/v2.4

modutils-2.4.23.tar.gz          Source tarball, includes RPM spec file
modutils-2.4.23-1.src.rpm       As above, in SRPM format
modutils-2.4.23-1.i386.rpm      Compiled with gcc 2.96 20000731,
                                glibc 2.2.2.
modutils-2.4.23-1.ia64.rpm      Compiled with gcc 2.96-ia64-20000731,
                                glibc-2.2.3.
patch-modutils-2.4.23.gz        Patch from modutils 2.4.22 to 2.4.23.

Changelog extract

• Correct s390[x] relocations for position independent code. Bill Nottingham/Amit S. Kale.
• Add alias for the tun/tap device. Bill Nottingham.
• Fix insmod for ppc64 MODULE_PARM(foo, "l"). Peter Bergner.
• Add DESTDIR to man_kerneld, change man pages from 444 to 644. Peter Breitenlohner.
• libz must be static for --enable-zlib. Bill Nottingham, reworked by Keith Owens.
• Add note about refusing patches that change behaviour to man pages.
• Alpha now uses srel32 in the exception handling macros. Richard Henderson.
• Support ia64 brl relocations.

Modutils 2.4.24 will be out in the next few days. The only change planned for 2.4.24 is to complain about modules using the default behaviour of "export all symbols" on architectures that use function descriptors. Such architectures must explicitly export symbols, otherwise gcc does not generate function descriptors and inter-module references break spectacularly.

Here's a set of i2c driver changes that start the conversion of the i2c core and drivers over to the kernel driver model. Eventually this will allow all of the sysctl and proc mess to be removed for this subsystem.

These patches add i2c driver bus and i2c adapter driver support to the driver core. They also export a needed symbol from the driver core (which Pat Mochel agreed with doing.) The patches also add three i2c controllers that have been in the i2c cvs tree for a long time, and are on a lot of people's machines.

The i2c core needs Christoph's previous patches to work properly, and with that patch, and these patches, it all works properly on my machines (tested on 4 different types of i2c controllers.)

Oh, and the i2c development team has given the ok for me to send these patches to you, and for me to do this work.

Please pull from: bk://kernel.bkbits.net/gregkh/linux/i2c-2.5

Things left to do after this:

• clean up #ifdef mess in i2c controllers
• fix the printk() calls to use proper levels
• add i2c controller driver core support (present in the patch I previously sent to lkml).
• add i2c device core support.

You have /proc/sys/dev/raid/speed_limit_max and _min, where you can specifiy upper and lower bounds for how fast the raid resyncs.

I imagine you could use cron or at to whack together something to arrange higher speed resyncs during offhours.

Never ever use word KitBeeper, Larry thinks he owns the world.

The page originally said "Goal of this project is to create version managment system compatible with <prohibited word>".

PS: I know forwarding personal message to the mailing list is not polite, but this is more of legal threat than personal message...

PPS: About Toyota: I may not be able to call my company Toyota (if you have registered trademark in czech republic, which I doubt), but I'm sure able to say that my car contains same engine as Toyota 1234cdt. And as you named directory in the repository BitKeeper, there's no chance not to mention it.

----- Forwarded message from Larry McVoy <lm@bitmover.com> -----

Date: Thu, 13 Mar 2003 20:21:54 -0800
From: Larry McVoy <lm@bitmover.com>
To: Pavel Machek <pavel@ucw.cz>
Cc: Larry McVoy <lm@bitmover.com>
Subject: Re: BitBucket: GPL-ed BitKeeper clone
User-Agent: Mutt/1.4i

> Sorry, but I'm doing this on my own, this has nothing to do with SuSE.

That's fine, I've taken it up with the sourceforge people.  What you
are doing is a violation of their terms of use and they'll get you
to fix it, I don't care who fixes it, I want absolutely no reference
to anything which can connect that work with BitKeeper.  That's well
with in our legal rights.

As for it being on your own time, let's see if SuSE wants the negative
publicity.  I'm quite willing to make a stink, you've annoyed me and
I've put up with as much as I'm going to.

> I believe the wording above is okay; if you miss "BitKeeper is
> trademark of BitMover" notice, I guess you can have that too. If you
> want your lawyer to call me, my cellphone is +XXXXXXXXXXXX; if you
> want to suggest alternate wording, just do that. But I believe saying
> "compatible with (something)" is okay, and I can allways just misspell
> it.

No, you can't.  Check into the case law.  If there is any way to connect
it to our product, it is within our rights to protect our brand.  You
can't call your new car company Toyotaa either, Toyota can sue and will
if they think it infringes on their brand.

You are quite welcome to write your own SCM.  You may not take advantage
of our work, check into the law.

When you are dealing with someone apparently being a corporate bully you have no choice quite often but to publicize stuff. Whether Larry is actually unreasonable is kind of hard to tell without all the context.

If Larry feels bitbucket is a lot like bitkeeper then I see his point. If he's moaning about things like "foobar is a tool for reading BitKeeper repositories" then I guess he forgot to take his pills this morning 8)

Unfortunately all the current bad feeling and suspicion keeps magnifying probably minor misunderstandings into large ones

Overall statistics
------------------
Statistics created on: Sat Mar 15 04:02:23 2003

First message was written at: 2003/02/24 11:22:10
Last message was written at: 2003/02/24 23:55:45   
Total number of messages: 3040
Total size: 17455KB
Total number of writers: 657
Number of people who wrote >1 message: 340
Total number of lines: 297404
Average lines per message: 97
Total header length (lines): 145698
Average header length (lines): 47
The header is on average 48.99% of the message (lines).
The header is 43.92% bytes in size of the total.
Average number of bits information per byte: 0.7502
Total number of unique user-agents: 0
Total number of unique organisations: 79
Total number of unique top-level domains: 66

Importance
----------
Low   : 0.00%
Normal: 1.61%
High  : 0.00%
(the rest is unspecified)

Top writers
   | # msgs|av size| total|time| e-mail address
---+-------+-------+------+----+--------------------------------
  1]     96|   4922| 461KB|1332| "Martin J. Bligh" <mbligh@aracnet.com>
  2]     81|   4656| 368KB|1142| Andrew Morton <akpm@digeo.com>
  3]     78|   3626| 276KB|0000| Alan Cox <alan@lxorguk.ukuu.org.uk>
  4]     60|   6447| 377KB|1420| Greg KH <greg@kroah.com>
  5]     55|   4274| 229KB|1312| Linus Torvalds <torvalds@transmeta.com>
  6]     48|   6912| 324KB|1535| William Lee Irwin III <wli@holomorphy.com>
  7]     47|   4500| 206KB|1428| Larry McVoy <lm@bitmover.com>
  8]     39|   5444| 207KB|1516| Pavel Machek <pavel@ucw.cz>
  9]     38|   3932| 145KB|1515| Jens Axboe <axboe@suse.de>
 10]     37|   6609| 238KB|1801| Russell King <rmk@arm.linux.org.uk>

Top subjects
   | # msgs|av size| total|time| subject
---+-------+-------+------+----+--------------------------------
  1]    111|   4362| 472KB|1405| Minutes from Feb 21 LSE Call
  2]     71|   5372| 372KB|1311| [ANNOUNCE] BK->CVS (real time mirror)
  3]     64|   4807| 300KB|1229| BitBucket: GPL-ed KitBeeper clone
  4]     43|   3677| 154KB|1252| 2.5.63 accesses below %esp (was: Re: ntfs OOPS
(2.5.63))
  5]     38|   4345| 161KB|0833| About /etc/mtab and /proc/mounts
  6]     38|   4307| 159KB|1012| Never ever use word BitKeeper if Larry does not
like you
  7]     35|   3684| 125KB|1207| bio too big device
  8]     32|   4385| 137KB|0742| [BK PATCH] klibc for 2.5.64 - try 2
  9]     31|   4517| 136KB|1515| [PATCH] s390 (7/13): gcc 3.3 adaptions.
 10]     29|   4265| 120KB|1451| [BUG] 2.5.63: ESR killed my box!

Top receivers
   | # msgs|av size| total|time| e-mail address
---+-------+-------+------+----+--------------------------------
  1]    859|   8109|6802KB|1315| linux-kernel@vger.kernel.org
  2]    211|   9306|1917KB|1336| torvalds@transmeta.com
  3]    128|   6791| 848KB|1114| Andrew Morton <akpm@digeo.com>
  4]     76|   4281| 317KB|1308| "Martin J. Bligh" <mbligh@aracnet.com>
  5]     72|   5868| 412KB|1136| Alan Cox <alan@lxorguk.ukuu.org.uk>
  6]     44|   3815| 163KB|1655| alan@redhat.com
  7]     43|   4396| 184KB|1228| Larry McVoy <lm@bitmover.com>
  8]     35|   4177| 142KB|0941| "'Jens Axboe'" <axboe@suse.de>
  9]     34|   6007| 199KB|0654| davem@redhat.com
 10]     34|   4542| 150KB|1350| Greg KH <greg@kroah.com>

Top CC'ers
   | # msgs|av size| total|time| e-mail address
---+-------+-------+------+----+--------------------------------
  1]   1316|   5138|6603KB|1202| linux-kernel@vger.kernel.org
  2]    216|   7992|1685KB|1333| torvalds@transmeta.com
  3]    106|   5193| 537KB|1237| Andrew Morton <akpm@digeo.com>
  4]    101|   7061| 696KB|1303| Alan Cox <alan@lxorguk.ukuu.org.uk>
  5]     62|   4950| 299KB|1021| "Martin J. Bligh" <mbligh@aracnet.com>
  6]     42|   6218| 255KB|1347| Larry McVoy <lm@bitmover.com>
  7]     40|   5520| 215KB|1051| linux-mm@kvack.org
  8]     34|   4574| 151KB|1125| "David S. Miller" <davem@redhat.com>
  9]     33|   4260| 137KB|1136| Greg KH <greg@kroah.com>
 10]     31|   3782| 114KB|1322| Alan Cox <alan@redhat.com>

Top of top-level-domain
----------------------------------------------------------
 1]  com 1145
 2]  org 474
 3]   de 291
 4]  net 187
 5]   uk 160
 6]   cz 86
 7]   ru 67
 8]  edu 61
 9]   au 60
10]   nl 48

Timezones
----------------------------------------------------------
 1]  777 +0100
 2]  731 -0800
 3]  357 -0500
 4]  196 +0000
 5]  115 -0600
 6]   71 +1100
 7]   69 +0300
 8]   64 +0200
 9]   50 -0700
10]   43 +0900

Top organisations
----------------------------------------------------------
 1]  198
 2]   48 The Domain of Holomorphy
 3]   29 OSDL
 4]   23 none
 5]   19 Open Source Devlopment Lab
 6]   17 Transmeta Corporation, Santa Clara CA
 7]   16 Working Overloaded Linux Kernel
 8]   15 HOME
 9]   15 IBM Deutschland GmbH
10]   14 daimi.au.dk

Top user-agents
----------------------------------------------------------
 1]   56 Mutt/1.4i
 2]   40 Mutt/1.5.3i
 3]   26 KMail/1.5
 4]   24 Ximian Evolution 1.2.2
 5]   20 Mutt/1.3.28i
 6]   17 Mutt/1.2.5.1i
 7]   12 KMail/1.4.3
 8]    9 Mutt/1.2.5i
 9]    8 Microsoft Outlook Express 6.00.2800.1106
10]    8 Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20021003

Messages per day
----------------------------------------------------------
   Sunday   213 ****************
   Monday   387 *****************************
  Tuesday   507 *************************************
Wednesday   564 *****************************************
 Thursday   442 *********************************
   Friday   386 *****************************
 Saturday   128 **********

Messages per Month
----------------------------------------------------------
Jan     0
Feb  1060 *********************************
Mar  1566 ************************************************
Apr     0
May     0
Jun     0
Jul     0
Aug     0
Sep     0
Oct     0
Nov     0
Dec     1 *

Messages per day-of-the-month
----------------------------------------------------------
 1     2 *
 2     0
 3     0
 4     0
 5     0
 6     1 *
 7     2 *
 8   127 ***********
 9   211 ******************
10   217 *******************
11   221 *******************
12   314 ***************************
13   230 ********************
14   241 *********************
15     0
16     0
17     0
18     0
19     0
20     0
21     0
22     0
23     2 *
24   170 ***************
25   285 *************************
26   250 **********************
27   211 ******************
28   142 *************
29     0
30     0
31     1 *

Messages per hour
----------------------------------------------------------
 1    33 *********
 2    47 *************
 3    24 *******
 4    22 ******
 5    12 ****
 6    22 ******
 7    42 ************
 8    87 ************************
 9   137 *************************************
10   173 **********************************************
11   185 **************************************************
12   163 ********************************************
13   182 *************************************************
14   183 *************************************************
15   184 *************************************************
16   181 ************************************************
17   155 ******************************************
18   106 *****************************
19   146 ***************************************
20   119 ********************************
21   123 *********************************
22   103 ****************************
23   104 ****************************

NFS is a bit tricky. Reiser used to be broken on it, and at least from large XFS NFS Servers I know that they tend to be unstable, still.

For the Database Servers, I am not sure how well they operate with journaling filesystems. I think Linux Journal had an article on performance on that.

Reiser might be your bet, depending on the usage pattern of the filename space, with Ext3 catching up. Personally I love the XFS features for resizing in connection with LVMs, but i guess you can have that with Ext3 and Reiser, too.

Thanks for all the input. Here is some info after I get from my setting. Hardware config: Tyan 2466 Duel MP2200+ 512MB, SX6000 4 ide 120GB 7200RPM RAID5

-------------------------------------------------------------------------
StPeter:/mnt/part1# sfdisk -l /dev/sda

Disk /dev/sda: 351905 cylinders, 64 heads, 32 sectors/track
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Warning: The first partition looks like it was made
for C/H/S=*/255/63 (instead of 351905/64/32).
For this listing I'll assume that geometry.
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

Device Boot Start End #cyls #blocks Id System
/dev/sda1 0+ 44860 44861- 360345951 5 Extended
/dev/sda2 0 - 0 0 0 Empty
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
/dev/sda5 0+ 3646 3647- 29294464+ 83 Linux
/dev/sda6 3647+ 7293 3647- 29294496 83 Linux
/dev/sda7 7294+ 10940 3647- 29294496 83 Linux
/dev/sda8 10941+ 14587 3647- 29294496 83 Linux
/dev/sda9 14588+ 18234 3647- 29294496 83 Linux
/dev/sda10 18235+ 21881 3647- 29294496 83 Linux
/dev/sda11 21882+ 25528 3647- 29294496 83 Linux
/dev/sda12 25529+ 29175 3647- 29294496 83 Linux
/dev/sda13 29176+ 32822 3647- 29294496 83 Linux
/dev/sda14 32823+ 36469 3647- 29294496 83 Linux
/dev/sda15 36470+ 44860 8391- 67400676 83 Linux

------------------------------------------------------------------
StPeter:/mnt/part1# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 64 MB in 1.60 seconds = 40.00 MB/sec
StPeter:/mnt/part1# hdparm -T /dev/sda

/dev/sda:
Timing buffer-cache reads: 128 MB in 0.47 seconds =272.34 MB/sec
------------------------------------------------------------------
StPeter:/mnt/part1# mount
/dev/sda5 on /mnt/part1 type xfs (rw,noexec,nosuid,nodev)
/dev/sda6 on /mnt/part2 type reiserfs (rw,noexec,nosuid,nodev)
/dev/sda7 on /mnt/part3 type ext3 (rw,noexec,nosuid,nodev)

StPeter:/mnt/part1# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 28G 4.8M 27G 1% /mnt/part1
/dev/sda6 28G 33M 27G 1% /mnt/part2
/dev/sda7 27G 33M 26G 1% /mnt/part3

StPeter:/mnt/part1# time cp -rf /usr/src/kernel-source-2.4.20 ./

real 1m3.501s
user 0m0.140s
sys 0m2.680s

StPeter:/mnt/part2# time cp -rf /usr/src/kernel-source-2.4.20 ./

real 0m3.696s *************** so fast...
user 0m0.110s
sys 0m3.570s

StPeter:/mnt/part3# time cp -rf /usr/src/kernel-source-2.4.20 ./

real 1m29.697s
user 0m0.090s
sys 0m2.490s

*ext3 used the most space

StPeter:/mnt/part3# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 28G 180M 27G 1% /mnt/part1
/dev/sda6 28G 191M 27G 1% /mnt/part2
/dev/sda7 27G 211M 25G 1% /mnt/part3
--------------------------------------------------------
StPeter:/mnt/part1# time rm -rf kernel-source-2.4.20/

real 0m23.351s
user 0m0.050s
sys 0m1.250s

StPeter:/mnt/part2# time rm -rf kernel-source-2.4.20/

real 0m1.297s
user 0m0.010s
sys 0m0.890s

StPeter:/mnt/part3# time rm -rf kernel-source-2.4.20/

real 0m1.062s
user 0m0.000s
sys 0m0.690s

-------------------------------------------------------

I've managed to put together a patch which allows a NUMAQ box with more than 8 IOAPICs to boot, the current workaround is to force it to 8 or lower. This was achieved by setting up percpu idts thus allowing us to implement per node vector spaces. The previous hurdle was running out of vectors to assign, the current one is running out of irqs, something which i'll further be looking at.

Later on i'm planning to move to per node IDTs so that i can fix f00f workaround and also save space on nodeless systems. To do this will probably require early_cpu_to_node workalikes (or perhaps just use logical_smp_processor_id and apic_to_node).

Thanks to the folks at OSDL (and Mark for giving up his 4quad for so long) for providing me access to the 8quad.

http://www.osdl.org/projects/numaqhwspprt/results/dmesg-32way-8quad-2.5.64-highirq
http://www.osdl.org/projects/numaqhwspprt/results/lspci-numaq-8quad
http://www.osdl.org/projects/numaqhwspprt/results/mptable-32way-8quad

ftp://oss.sgi.com/projects/kdb/download/v4.0/

kdb-v4.0-2.4.19-common-1.bz2
kdb-v4.0-2.4.19-i386-1.bz2
kdb-v4.0-2.4.19-ia64-020821-1.bz2
kdb-v4.0-2.4.20-common-1.bz2
kdb-v4.0-2.4.20-i386-1.bz2
kdb-v4.0-2.4.20-ia64-021210-1.bz2

<=== You know what they say about .0 releases. Caveat emptor. ===>

The biggest change is this release in the way that kdb captures data from each cpu. Prior to v4.0, the controlling cpu (first to get into kdb) would try to pull the other cpus into kdb by sending them an IPI. Doing a backtrace on the active tasks required that kdb switch its context into each cpu in turn.

This works in most cases, but when the system is really wedged, it is difficult to get data from the other cpus. Of course this is precisely the case when we want data from the other cpus. This problem is particularly noticeable on ia64, when machine checks (MCA) or INIT interrupts can disable normal IPI or dive down into SAL, taking control away from the OS. Also on IA64 the non-maskable interrupt is actually masked (http://external-lists.valinux.com/archives//linux-ia64/2001-May/subject.html, look for 'Replacements for local_irq_xxx').

In kdb v4.0, each cpu pushes its state via a global array. This allows any cpu to do a backtrace on any other cpu, from a known starting point. It even handles the cases when IA64 requires that cpus rendezvous and spin in SAL. The push model also makes it easier to detect when a cpu is dead, an event which would often hang the old kdb pull model.

On ia64, kdb v4.0 gives decent backtraces from MCA callback, MCA rendezvous and the INIT monarch event. The next step in kdb v4.1 is to detect hung spinloops and break out of them, and to support INIT slave events. The detection and debugging of hung spinloops is waiting on acceptance of my new spinlock code for IA64 (http://external-lists.valinux.com/archives//linux-ia64/2003-March/004976.html).

I'd like to announce publication of Linux Kernel State Tracer (LKST) 1.4, which is a tracer for Linux kernel.

In release 1.4, we especially improved about event-buffer. Please refer to changesto1.4.txt in documents for details.

LKST binaries, source code and documents are available in the following site,

https://sourceforge.net/projects/lkst/
http://sourceforge.jp/projects/lkst/

We prepared a mailing list written below in order to let users know update of LKST.

lkst-users@lists.sourceforge.net
lkst-users@lists.sourceforge.jp

To subscribe, please refer following URL,

http://lists.sourceforge.net/lists/listinfo/lkst-users
http://lists.sourceforge.jp/mailman/listinfo/lkst-users

And if you have any comments, please send to the above list, or to another mailing list written below.

lkst-develop@lists.sourceforge.net
lkst-develop@lists.sourceforge.jp

Squashfs1.2 has been released.

Squashfs is a highly compressed read-only filesystem for Linux 2.4. It uses zlib to compress files, inodes, and directories. All blocks are packed to minimise the data overhead, and block sizes of between 4K and 32K are supported. It is intended to be used as a filesystem for archival use and in embedded systems where low overhead is needed.

1.2 has added append capability to Mksquashfs. This means that new files and directories can be added to pre-existing filesystems without needing to rewrite the original data. For large filesystems this can be of major benefit. Because mksquashfs peforms duplicate checking against the original filesystem, it means that updated directories can be appended over time, and only the changed files will take extra space. The unchanged files will be detected as duplicates, and will share data. This exhibits a basic versioning capability.

Squashfs1.2 is available from http://squashfs.sourceforge.net.

I think those repositories are stable enough you can start to count on them. Sam and others have looked over their changes and think that the CVS tree has accurate data.

The CVS repository is at

cvs -d:pserver:anonymous@kernel.bkbits.net:/home/cvs

and it has two top level directories, linux-2.4 and linux-2.5.

In order to make the best use of available space, we will unfortunately have to reconfigure the RAID arrays on the main kernel.org server. This will result in extended downtime for all kernel.org services, starting at 16:00 PST/24:00 UTC this evening, March 17, 2003. For ftp/www/rsync/filehub.kernel.org, this downtime should be in the ballpark of 2-3 hours; with luck less; for mirrors.kernel.org, this downtime is expected to be 3-4 days. Services will be phased in as they become available, so if you get a "connection refused" from one service other services might still be operational.

Hopefully this will be reasonably painless.

OK, so I'm an idiot.

The correct downtime estimate is about 6-7 hours. This is a matter of simple arithmetric, but somehow I never bothered to actually calculate it out.

Just to be on the safe side I'm going to estimate kernel.org being back up around 00:00 PST/08:00 UTC.

I've delayed this too long, but Ingo found why the scheduler sometimes did bad things, and this should all be good.

A lot of fairly small changes all over the map, see the full changelog for details.

Compile statistics: 2.5.65

                               2.5.64               2.5.65
                       --------------------    -----------------
bzImage (defconfig)         14 warnings          14 warnings
                             0 errors             0 errors

bzImage (allmodconfig)      30 warnings          30 warnings
                             9 errors            12 errors

modules (allmodconfig)    2356 warnings        2421 warnings
                            99 errors           100 errors

Compile statistics have been for kernel releases from 2.5.46 to 2.5.65 at: www.osdl.org/archive/cherry/stability (will be updated by 6PM PST).

Other stability-related links:

OSDL Stability page:
http://osdl.org/projects/26lnxstblztn/results/
Nightly linux-2.5 bk build:
www.osdl.org/archive/cherry/stability/linus-tree/running.txt
2.5 porting items:
www.osdl.org/archive/cherry/stability/linus-tree/port_items.txt
2.5 porting items history:
www.osdl.org/archive/cherry/stability/linus-tree/port_history.txt

http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.65/2.5.65-mm1/

kernel.org is being slow. Should later appear at

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.5/2.5.65/2.5.65-mm1/

• An updated version of Russell's PCMCIA patches
• Lots more anticipatory scheduler work.

• It turns out that calling disk request_fns from timer/tasklet context is not permitted because a few old drivers like to sleep in that function.

  keventd cannot be used for this because it can deadlock. So another kernel thread per CPU has been reluctantly added.

Yet another release in the usual places. The main reasons for the release is a correction on the subject of vmalloc more than anything else and the rearrangement of chapters to present the material in more logical order. I am hoping there will only be one, or at most two more releases after this before it's done and dusted (famous last words).

Understanding the Linux Virtual Memory Manager
PDF: http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/understand.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/understand
Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/understand.txt

Code Commentary
PDF: http://www.csn.ul.ie/~mel/projects/vm/guide/pdf/code.pdf
HTML: http://www.csn.ul.ie/~mel/projects/vm/guide/html/code
Text: http://www.csn.ul.ie/~mel/projects/vm/guide/text/code.txt

Few important reasons for this release but still, it brings me closer to just finalising it and releasing it fully.

1. Chapters have been rearranged a little so there should be no forward references left and the material is handled in an "easier" order for understanding it. Each chapter now has an introduction as well so it isn't as clunky to read at parts
2. I messed up the explanation of vmalloc by saying pages are allocated at fault time rather than saying that it is just the page tables for the faulting process are synced with the master page tables. Pretty serious mistake so anyone looking at vmalloc stuff should re-read
3. Minor correction on the explanation of try_to_free_pages() in the code commentary. I now explain why it only frees up pages in ZONE_NORMAL
4. Loads of polish like font and grammar corrections. Minor mistake in slab where I said /proc/cpuinfo instead of /proc/slabinfo and a few others like that

A kernel patch implementing relayfs, a Linux filesystem designed to simplify the buffering and efficient transfer of large amounts of data from kernel to user space for those facilities that need to do such things, is now available as:

http://www.opersys.com/ftp/pub/relayfs/patch-relayfs-2.5.65-030317.bz2

The relayfs home page, which will host the latest versions of the above and related patches and info is:

http://www.opersys.com/relayfs

A preliminary description of the characteristics and API for relayfs was previously posted to this list. Here's a link to that post:

http://marc.theaimsgroup.com/?l=linux-kernel&m=104196329807755&w=2

I've included at the end of this mail the relayfs doc file (Documentation/filesystems/relafys.txt), which describes the API in detail, as currently implemented.

Also, the Linux Trace Toolkit has been reworked to make use of relayfs; a preliminary kernel patch and updated user tools are available at the LTT website, and can be used as a concrete working example of using the API:

http://www.opersys.com/relayfs/ltt-on-relayfs.html

I've tested the relayfs code (via the updated LTT code) on my UP PII machine and so far haven't seen any problems with it, but I still need to do SMP testing and otherwise pound more heavily on it - I plan on generating some benchmark data, which in addition to putting relayfs through its paces, should give some idea of how much overhead might be introduced by the API/implementation itself, as compared with the benchmark data previously posted for LTT without it:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103573710926859&w=2

I wouldn't expect to see much difference, as the underlying logging code is essentially the same.

I've just added (updated) lightly tested support for the BK->CVS kernel trees to cvsps (www.cobite.com/cvsps) in version 2.0b4. The purpose of this effort is to recreate the BK ChangeSet meta-data that is embedded in the 'cvs log' data in these trees. BTW, cvsps is GPL software :-p. I'd like to thank Larry and Andrea for helping me track down some issues with this effort. This is still a BETA version, though, and I haven't given this enough testing, so be nice. It works for me.

This version is tested and works against this morning's linux-2.4 and linux-2.5 trees, and contains a few workarounds for specific issues in those trees. See below for information on these problems.

The output of cvsps looks like:

------------------------------
PatchSet 999
Date: 2002/07/11 19:50:46
Author: alan
Branch: HEAD
Tag: (none)
Log:
[PATCH] Fix several pdc202xx problems

Misnaming of 20270 as 20268R
Failure of LBA48 on 20262
Incorrect speed detection because the old driver used host not drive side
cable detect
PDC202xx handling for quirks in udma reporting off some drives
LBA48 for PIO mode

BKrev: 3d2dd386wJMnehoOAhv3wL991IfXVQ

Members:
        ChangeSet:1.999->1.1000
        MAINTAINERS:1.74->1.75
        drivers/ide/ide-features.c:1.4->1.5
        drivers/ide/ide-pci.c:1.18->1.19
        drivers/ide/pdc202xx.c:1.11->1.12
        include/linux/pci_ids.h:1.44->1.45
-----------------

You can also get a diff of this PatchSet using the '-g' option to cvsps.

There are currently 2,798 PatchSets in the linux-2.4 tree, and 8,382 in the linux-2.5 tree.

Quick start instructions
========================

Download, build and install cvsps from http://www.cobite.com/cvsps
Get the 2.0b4 (or latest) version.
Create a working directory with the tree of your choice:

cvs -d:pserver:anonymous@kernel.bkbits.net:/home/cvs co linux-2.4/Makefile

cd linux-2.4

[ IMPORTANT: cvsps doesn't currently support an option for setting the compression level so PLEASE, edit your .cvsrc and put 'cvs -z4' to enable compression ]

cvsps [-x] --bkcvs

This basically runs a 'cvs rlog' against the tree, parses, and caches all of the revision history as PatchSets. It also outputs all of the PatchSet summaries to stdout, so you may want to '>/dev/null' the first time.

Subsequent 'cvsps' commands do not need the '--bkcvs' unless you are updating (-u, not completely tested) or rebuilding (-x, always works) the cache file.

Now you can use cvsps to browse the patchsets at your leisure, without loading the cvs server (except to generate diffs). See cvsps -h for the many ways you can slice and dice the information.

I welcome any feedback.

Problems
========

I have currently encountered two problems with the log format.

1) someone has committed sections of 'cvs log' text into the log. This causes quite a headache for my parser, because false end-of-log-message markers are present in the log. Fortunately, Larry has put a '(Logical change x.yyyy)' marker at the end of each log message, see alse 2)

2) Not all log messages are terminated by a '(Logical change x.yyy)' marker. A single revision of one file is missing this marker, Larry is looking into why this may have happened.

Both of these problems are being worked around by my 'Adaptive Crap Filter (notTM)' code. Don't look at it. It'll kill you.

what about the deleted files? should we teach cvsps how to diff the old revision fetched with cvs up -p against /dev/null to make a completely coherent patch?

this is with 2.4

Directing PatchSet 2742 to file ../patches-2.4//2742.patch
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'
cvs [update aborted]: no such directory `drivers/usb/hcd'

so it doesn't sound a regex. Being able to specify more than 1 -f would be very useful. Either that or regex would do it fine too with '^net/core', so as far as I can write stuff like -f '^net/core|^net/ipv4' I'm fine.

I also think using match by default in the regex is cleaner. So I can write -f 'net/core|net/ipv4' w/o bothering about the ^ because it become implicit. And I can still use '.*net/core.*' if I want a substring regex. I think substring search will be not common.

But really, any kind of way you implement the 'multiple file' thing is fine as far as I can specify more than 1 file ;).

try elks:

http://elks.sourceforge.net/

the economics aren't really there as far as I can tell given the cost of embeded 386 and 486 class cpu's to say nothing of tiny powerpc and arm cpu's.

http://www.uclinux.org/

It doesn't need an mmu, boots on a Palm. ;) Look in 'uClinux Ports'

Or http://www.linux.org/projects/ports.html, look for m68k ports, don't know if any of them work on cpus below 68020.

This patch adds the Essential Reality P5 data glove to the HID blacklist. It's yet another device that advertises itself as a HID, and doesn't comply with the HID specs.

The P5 is an inexpensive ($99 USD) data glove, with 5 finger flexion sensors and 8 IR location sensors.

A user-space library to access the P5 is available at http://www.evillabs.net/~jmcc/p5/

I'd like to introduce the OpenHPI project at http://openhpi.sf.net. The intent of this project is to produce an implementation of the Service Availability Forum's Hardware Platform Interface (HPI). HPI provides a universal interface for creating resource system models, typically for chassis and rack based servers, but extendable for other problem domains such as clustering, virtualization, and simulation.

We are currently in the design phase of the project and are soliciting involvement with others in the community.

Our goal is to have modular hardware support that can be implemented using a plugin architecture. This would allow a top-level OpenHPI implementation to be independent of the underlying hardware platform(s). We are also just getting a build environment up and a stubbed-out library implementation available. Work has also just begun regarding the planning and development of a certification suite for the HPI implementation.

For more information and how you can become involed please see...

Project Website:
http://openhpi.sf.net

Mailing List:
http://lists.sourceforge.net/lists/listinfo/openhpi-devel