Kernel Traffic #283 For 6 Nov 2004

There were 555 different contributors. 325 posted more than once. 207 posted last week too.

Ananth N. Mavinakayanahalli announced kprobes for ppc64, for the 2.6 kernel. He explained:

Kprobes (Kernel dynamic probes) is a lightweight mechanism for kernel modules to insert probes into a running kernel, without the need to modify the underlying source. The probe handlers can then be coded to log relevent data at the probe point. More information on kprobes can be found at:

http://www-124.ibm.com/developerworks/oss/linux/projects/kprobes/

Jprobes (or jumper probes) is a small infrastructure to access function arguments. It can be used by defining a small stub with the same template as the routine in kernel, within which the required parameters can be logged.

Paul Mackerras liked the patch overall, but had specific technical objections, that he felt would not pose any serious problem for the patch's inclusion.

An initial Kprobes implementation originally found its way into the 2.6.9-rc2 kernel via David S. Miller and Prasanna S. Panchamukhi back in August. Kprobes' ability to trap at almost any kernel code address allows developers to collect debugging information without disrupting the running system. Developers can specify handler routines to be executed when the breakpoint is hit.

Linux 2.6.9-rc4 included changes (also by Prasanna) to the Kprobes interface, specifically the return value of its exceptions notify handler, to help other debuggers co-exist with Kprobes.

Jeff Garzik was stunned and impressed to see that, in the 24 hours since the release of Linux 2.6.9, a full 850 changesets and 3383 revisions had been added to the kernel tree. Ben Dooks remarked that he and other folks had been on hold for this release, so they had patches prepared for the moment of 2.6.9's release. Dave Jones felt that this might be an indication that kernel development needed shorter -rc periods. Jeff replied, "Actually, we need longer non-rc periods" . Russell King said, "Personally, I think both of you are right. One major kernel release a month seemed to be about the right rate. Maybe a week and a half of non-rc plus two and a half weeks of -rc would be the right kind of balance?" Jens Axboe also replied to Jeff with agreement. He said, "The rate of change is truly impressive (thank you Andrew and BK!), but personally I'd like to see things settle down a lot more quickly. Instead of having 2-3 weeks of continual patch flood, a week or submitting the stuff that was already done by 2.6.9 by Andrews inclusion criteria (which I completely agree with) results in -rc1, followed by 2-3 weeks of of truly stabilizing bug fixing. Since by virtue of this inclusion criteria development for a particular feature/change is already done by 2.6.9 release, this should be easy (Yeah right, but at least we can try.)" Pavel Machek suggested forking off the 2.7 tree, and keep the duration between 2.7 and 2.8 shorter than usual.

Elsewhere, Matt Heler asked how many changes occurred between Linux 2.6.8 and 2.6.9, and Jeff replied, "'bk pull' says 4000 revisions to ChangeSet, for 15723 total revisions. (these numbers include merge changesets, which inflate things)"

Considering a 'patch' to be a unique changelog entry, it does appear that over 3500 distinct patches were accepted into the 2.6.9 kernel, over a thousand more than were accepted into either the 2.6.8 kernel or the 2.6.7 kernel. The 2.6.6 kernel took in fewer than either of those kernels, with 1700 patches, and the 2.6.5 kernel had fewer than that. Linux 2.6 series development appears to be heating up. Comparing this to the 2.4 series, Marcelo Tosatti has averaged about 220 patches per kernel for the same number of kernels.

Benjamin Herrenschmidt had a request regarding kernel versioning. See Issue #282, Section #9 (18 Oct 2004: Developers Unhappy With Linus' Kernel Versioning Anomolies) for other complaints. This time, Benjamin said, "After you tag a "release" tree in bk, could you bump the version number right away, with eventually some junk in EXTRAVERSION like "-devel"? It's quite painful to have a module dir name clash between the "clean" final tree and whatever dev stuff we are testing out of bk ... it's fine once you go to -rc1, but in the meantime, it's really annoying." Jeff Garzik was not sympathetic, pointing out that running the command 'echo "-bk" > localversion' would solve the problem for any given BitKeeper snapshot. But several other developers voiced agreement with Benjamin, including Len Brown, who said, "I'd find this to be really helpful too. There has been this period between, say, 2.6.9 and 2.6.10-whatever where my build/install scripts scribble over my "reference" kernels." Linus Torvalds replied:

Personally, I much rather go the way we have gone, because I don't care about module versioning nearly as much as I care about bug-report versioning. And if I hear about a bug with 2.6.10-rc1, I want to know that it really is at _least_ 2.6.10-rc1, if you see what I mean..

Now, personally, I'd actually like to know the exact top-of-tree changeset, so I've considered having something that saves that one away, but then we'd need to do something about non-BK users (make the nightly snapshots squirrell it away somewhere too). That would solve both the module versioning _and_ the bug-report issue.

So if somebody comes up with a build script that generates that kind of extra-version automatically, I'm more receptive. But I don't want to muck with the version manually in a way that I think is the wrong way around..

Måns Rullgård replied, "Would it work to somewhere in the Makefile check for the existence of a BitKeeper directory, and if it exists run bk with the appropriate arguments and append something to EXTRAVERSION? I'm not quite sure which information is the best to add, though." And Linus said:

That's what I had in mind. But it should also check if the top-of-tree is already tagged, and not do anything for that. And it should also hopefully have a CVS/Subversion equivalent, just so that people don't feel left out.

I would _suggest_ just exporting the whole top-of-tree tag to some /sys/kernel/version file (for full bug-reports), but in addition also maybe have a small hash of it (just a few characters of noise) in "uname", to make module versioning work.

So "uname -r" might print out "2.6.9-a$Uv", but then a

        cat /sys/kernel/version/*

would print out something like

        "kernel" file:
                v2.6.9-a$Uv
        "bk-key" file:
                torvalds@ppc970.osdl.org|ChangeSet|20041021004441|21737
        "date" file:
                Wed Oct 20 22:29:23 PDT 2004

or something (one value per file as usual)

Elsewhere, Jeff said that Linus' original desire to know the top-of-tree changeset was redundant. Jeff said:

The nightly snapshots have been exporting this info since Day One, based on your request ;-)

<snapshot>.key contains this info, e.g. http://www.kernel.org/pub/linux/kernel/v2.6/snapshots/patch-2.6.9-bk1.key is T.O.T. for http://www.kernel.org/pub/linux/kernel/v2.6/snapshots/patch-2.6.9-bk1.bz2

Yes. But that doesn't help the people who actually use the native BK trees themselves, or the people who use the CVS exports. That was what Ben was complaining about.

We already have the concept of "localversion*" files that get appended to the build. So the only thing that would be needed is some Makefile magic to create a "localversion-bk-version" file if the top-of-tree isn't tagged, and we'd get some unique ID for native BK users too.

Elsewhere, Ryan Anderson actually implemented Linus' initial suggestion, and a technical discussion ensued without resolution.

Lee Revell asked how an application could be notified when the CPU's speed changed on a running system. "Polling a file in /sys is not good enough," he said, "the overhead is unacceptable and we need to know now. Is this the kind of thing that would require the new kernel event interface?"

Alan Cox and others said that this was not a simple problem. Matthew Garrett remarked at one point, "The kernel does not always know when the CPU speed changes. This makes notification somewhat harder." Elsewhere, Robert Love addressed Lee's question about whether the kernel event interface would be a good tool for this. Robert said, "Yes, I think that doing a kevent tied to the processor object when the speed changes is an absolutely ideal use of the kernel event layer."

Elsewhere, Lee was surprised at Matthew's (and Alan Cox's) assertions that the kernel didn't always know the CPU speed. Lee said, "Wouldn't this cause weird behavior though? For example Linux only calculates the delay loop once, at boot time. Does this render *delay() useless?" Alan replied, "Such systems you do need to run with notsc - although 2.6 autodetects this prints complaints and does the job itself."

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.9/2.6.9-mm1/

Lots of new patches.
Status of as-yet-unmerged things:
- sysfs backing store: after six months, still under discussion.
- ext3 reservations code: it was ready to go, but then I got a bunch more patches. Lots of testing is needed here to make 2.6.10.
- ext3 resize code: ready to go, but it depends on the reservation code so I may end up swizzling the patch series.
- kexec and crashdump: this all allegedly works, but I want to *see* it work first.
- perfctr: mikpe is still working on this.
- preempt-smp and all the associated low-latency fixes: I haven't really thought about it and haven't looked at the patches yet. Hopefully 2.6.10 material.
- cachefs: probably 2.6.10 material, but I'd like to be convinced that we actually need this in the kernel. Not many people use AFS (sorry) and it's a ton of new code. Getting it working for NFS would be a winner.
- cpusets: ready to go, but the current plan is to rework the actual pinning mechanism to use scheduler domains rather than cpus_allowed, then to look at wiring it up via the CKRM interface. So I'm sitting on this.
- reiser4: not sure, really. The namespace extensions were disabled, although all the code for that is still present. Linus's filesystem criterion used to be "once lots of people are using it, preferably when vendors are shipping it". That's a bit of a chicken and egg thing though. Needs more discussion.
- md updates: these are blocked by a minor bunfight over one of Neil's procfs innovations. He's reworking the patches so we can defer that decision.

No distro using reiserfs V3 as the default is going to keep doing so once reiser4 meets their stability requirements. Reiserfs is used by a lot of people, and reiser4 obsoletes it, and the users know that. None of the distros have expressed any intent of staying on V3, and they'd be silly to do it. Many of them have expressed a desire to use reiser4. Next year, indications are that reiser4 usage by distros as their default will exceed that which is today possessed by V3. The higher performance of V4 is going to increase our market share.

I would like to encourage its inclusion as an experimental filesystem BEFORE vendors ship it. I think first putting experimental stuff in the kernels used by hackers makes sense. I think it creates more of a community.

I'd like to point out that there is a lot of stuff in the kernel that is a lot less stable than reiser4.

That said, inclusion in -mm found some bugs, and we are still testing one of the fixes which was a bit deep. I want to finish that testing (not more than 7 days) and send you all fixes before asking for inclusion.

Also, Hellwig made a valid point about getting rid of some macros that reduce readability (I also hate code that prevents editors finding called functions), and zam is working on fixing that.

Lindows is planning on shipping with reiser4 in its next release. I would very much like to see our inclusion before that.

Hilzinger Marcel pointed out that "SuSE Linux 9.2 will contain reiser4 (at least the beta testversions did). It cannot be set up via YaST during installation, but the tools are there." Andrew remarked, "hm. Nobody ever tells me anything. Does that mean that SuSE are using 8k stacks?" And Jan Engelhardt replied, "Yes, the defconfig does not have 4K stacks enabled."

Completely elsewhere in the thread, Avuton Olrich spoke out in favor of Reiser4 inclusion. He said, "I've been using reiser4 in four of my computers since it was in -mm. All partitions (excl. /boot), including 2 boxes that have been up since (well, reboots for -mm updates from time to time) the reiser4 conversion and not a hiccup since." Markus Törnqvist put in a 'me too', and Kasper Sandberg felt the same.

This is a forward port to 2.6 CVS of the lowmem_reserve VM feature in the 2.4 kernel.

http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.9/lowmem_reserve-1

Lack of this feature might explain out of memory related killing or deadlocks hit on >2G boxes. so if anybody is having trouble with oom conditions this is a patch to try.

this is only slightly tested but works for me so far.

This is the first of a series of oom related fix I'm going to do and test within the next weeks to attempt to cure various oom regressions in 2.6 (deadlocks turned into crazy early oom kills and the like).

Rik van Riel asked what the actual behavioral changes would be seen under this patch, and Andrea replied, "the behavioural difference is the API and the fact the feaure is now enabled with sane values (the previous code was disabled by default and it was unusable with that API). besides fixing the API the patch nukes dozens of useless lines of code and a buffer overflow."

(ed. [Zack Brown] It turns out that this post was accidentally summarized in front of the thread that led up to it. This has to do with the way I select threads to summarize. Typically I wait for a thread to finish before summarizing it. In this case, the thread leading up to Jeff's post was still ongoing when Jeff started this new thread. The earlier thread got shunted off to the next issue's mailbox, and I inadvertantly summarized this one out of context. Next issue will show the lead-in thread, with a pointer back here. Sorry for the confusion.)

After having an off line review of the SCO claims pertaining to the identified code with Linus, It is clear the SCO has been making claims which are untrue and may have potentially engaged in slander of title, libel, tortorious interference, inteference with partner relations, and intentional inflcition of emotional distress against Linus Torvalds and Linux. Linus has peformed adequate due diligence and acted in good faith with regard to the acceptance of code from IBM and others based on the physical evidence in his possession, and I would testify to this in a court of law if asked as a qualified expert witness on behalf of Linus and Linux.

SCO's claims that some code may have been taken by IBM have nothing whatsoever to do with Linux until as such time they can produce credible evidence to the contrary. Despite numerous requests, visits to their facility, and meetings with them, their disclosure of only a handful of slides and contracts does not constitute corroborating physical evidence that Linus Torvalds tooks their intellectual property and used it in Linux. They are simply unable to produce concrete evidence that identifies the code in question to a degree that refutes the physical evidence pertaining to Linus' due diligence. If IBM's submissions were based in part on intellectual property taken from SCO, then IBM incurs this liability. It is clear at this point Linus acted in good faith in all of his dealings.

Linus Torvalds (and myself) are entitled to apolgies from GrokLaw, and SCO regarding their false and misleading claims Linus missappropriated trade secrets or infringed their copyrights and that I was involved in a scheme with SCO to further their false, misleading, and libelous allegations. Groklaw has also posted numerous emails and comments attributed to me which I did not author which libel Linus and myself, and were designed to create and perpetuate animosity in the Linux Community.

I thank Linus Torvalds for being a true friend and working with me to resolve these issues, despite all the heat and mud flying around.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc1/2.6.10-rc1-mm1/

Nothing really major here - just lots of minor things spread all over the place.
I actually managed to get most of the "external BK trees" included. A lot of them have been pretty flakey lately (hint).
This kernel is probably pretty crappy - there is a _lot_ of stuff happening and the quality of the patches which I am receiving seems to be gradually dropping off.


Kernel Traffic Latest \| Archives \| People \| Topics	Wine Latest \| Archives \| People \| Topics	GNUe Latest \| Archives \| People \| Topics
Czech

1.	18 Oct 2004 - 23 Oct 2004	(2 posts)	Kprobes Debugging Code Ported To PPC64
2.	19 Oct 2004 - 22 Oct 2004	(10 posts)	Speed Of Kernel Development
3.	19 Oct 2004 - 27 Oct 2004	(22 posts)	Linux Kernel Versioning: The Saga Continues
4.	21 Oct 2004 - 25 Oct 2004	(26 posts)	Determining CPU Speed Changes
5.	22 Oct 2004 - 27 Oct 2004	(81 posts)	Linux 2.6.9-mm1 Released
6.	25 Oct 2004 - 27 Oct 2004	(32 posts)	Forward-Porting 2.4 VM Out-Of-Memory Features To 2.6
7.	25 Oct 2004	(1 post)	A Little Bit Of SCO Status
8.	26 Oct 2004 - 27 Oct 2004	(18 posts)	Linux 2.6.10-rc1-mm1 Released

Kernel Traffic #283 For 6 Nov 2004

By Zack Brown