Kernel Traffic #29 For 29 Jul 1999

By Zack Brown

Table Of Contents

Introduction

Many thanks go to Peter Samuelson, who actually sent in a context sensitive diff with corrections to last week's issue (a link fix and some clearer wording). Peter has been a big help on several occassions, and it is much appreciated :-)

Mailing List Stats For This Week

We looked at 915 posts in 3792K.

There were 356 different contributors. 153 posted more than once. 136 posted last week too.

The top posters of the week were:

1. The Development Process; Tree Ownership

13 Jul 1999 - 19 Jul 1999 (12 posts) Archive Link: "Problem with memmap file with SMP"

Topics: FS: NFS, SMP, Virtual Memory

People: Stephen C. TweedieLinus TorvaldsAlan Cox

Steve Bradshaw found an extremely disturbing bug on SMP systems. He posted a small program that would cause one process to modify another process' data under certain conditions. Stephen C. Tweedie replied, "Ack. I looked at this for a while, but it's one of those things that you just know *can't* be happening... until you realise what's going wrong." He pinpointed the problem, explaining, "OK, what happens is that munmap() of a MAP_SHARED mapped file is trashing the data of another process on a different CPU which happens still to have the same page mapped." He added that NFS had the same problem.

He posted a patch against 2.2, adding that the problem was moot for 2.3 kernels because of the VM infrastructure changes. He admitted his patch was "a little messy", though he felt there was no truly clean solution.

Linus Torvalds commented (incidentally settling once and for all any question of who really owns the stable kernels), "It's a singularly ugly fix, regardless. I would never let code like that near any of my kernels, but Alan is the one who gets to decide in the stable tree."

Alan Cox, from under a pile of 3500 emails accrued during his vacation, came up for air to say, "Since 2.2 is going to be around for quite a while yet it wants doing right, painful or otherwise, tested in 2.3.x first if need be."

And Stephen replied to Linus, "The trouble is that a correct fix --- the clean way of doing things --- is to back-port the 2.3 changes, which is obviously a non-starter." He described one alternative, but added, "but that is a change in the definition of the VFS which I wanted to avoid for 2.2." He went on to explain, "The code paths in 2.2 simply don't let us pass information to the fs about whether or not an update_vm_cache() is necessary or not. Either we check in update_vm_cache or we change the VFS definitions. If Alan would prefer the latter then I'm certainly willing to code it that way, but it will be incompatible with existing fs modules."

Linus suggested a two-line solution that wasn't perfect, adding, "It's a trade-off - do you do the cheezy two-liner that gets the basic cases right, or do you do the more complex solution that should get everything right but might have some subtle problem. I will accept either from Alan."

Alan's previous comment about "doing it right, painful or otherwise", though at an earlier point in the thread, was actually made after Linus' last remarks, so presumably the more complex solution will be the one to go in.

2. Handling Out-Of-Memory Conditions

16 Jul 1999 - 20 Jul 1999 (25 posts) Archive Link: "Memory hogs"

Topics: Virtual Memory

People: Rik van Riel

Bernd Paysan felt something should be done about out-of-memory (OOM) conditions. Rik van Riel gave a pointer to his patch page (http://www.nl.linux.org/~riel/patches/) , and said, "Please, somebody grab the OOM patch, integrate it into a new kernel and send it to Linus..." However, a discussion ensued about implementation, some of which ignored his patch, and some of which was intended to extend his patch. One of his later comments in the discussion, ran:

My solution works rather well so I don't see why there are so much objections to it again...

We've had this whole discussion just before the 2.2pre era, when I decided the kernel should stabilize and my patch wasn't important enough to disturb the debugging.

I propose we stop this discussion until somebody produces better CODE then what's in my patch.

A few posts later, he added:

let's integrate my solution so that we have at least _something_ NOW. It has taken almost a year and my patch has always bounced off on discussions where NO code for any of the other proposals has shown up.

We can easily add stuff or replace it by something better later on. For now it's important to have something at least halfway decent.

3. gcc Vs. egcs: The Saga Continues

18 Jul 1999 - 19 Jul 1999 (8 posts) Archive Link: "what libc+compiler is in use for development?"

Topics: Compiler, Networking

People: Steve DoddHorst von Brand

Some news on the egcs/gcc front, last discussed in Issue #22, Section #6  (27 May 1999: Linus Still Prefers gcc To egcs) and before then in Issue #14, Section #4  (4 Apr 1999: The 'gcc' Vs. 'egcs' Saga Continues) (which also links several other articles). Dmitry Veprintsev asked which libc and compiler was being used for kernel development, and Steve Dodd replied, "egcs /generally/ works, but you won't get so much sympathy if something breaks. If you must use it, grab the 1.1.2 release (egcs-2.91.66); IIRC earlier versions had bugs. Later versions may also cause problems: the (pre-??) versions of gcc-2.95 that are floating around apparently cause problems if you don't use -fno-strict-aliasing; I don't know if they're OK if you do use that flag."

But Horst von Brand replied, "gcc-2.95 is around the corner, its prereleases generate _much_ better code than gcc-2.7.2. I have been compiling my kernels with egcs (lately gcc-2.95 snapshots, after EGCS got appointed keeper of GCC in April) for more than a year and a half now with mostly minor hickups (if that much), many others have done so too. The problems that surfaced have generally promptly been fixed. Many distributions ship kernels compiled with egcs." He added, "gcc-2.7.2 is going to go away soon."

Steve replied, "I was under the impression that Linus has disputed the code generation thing, but compilers are certainly not my domain so I won't comment. It'll just be nice to have a working C compiler and a working C++ compiler in the same package.."

Horst gave this bit of analysis: "What Linus has been questioning (AFAIU him :) is that he knows exactly (more or less) what code gcc-2.7.2 will generate, and this isn't true with later versions. You'd have to understand that Linus is a efficiency freak, he wants minute control over the object code. The changed code generation algorithms, new and rewamped optimizations, and assorted other changes in the compiler mean that the certain way of writing code that used to give the fastest object doesn't necessarily do so so anymore. I'd also say that the linux kernel is _the_ piece of code that most uses (and stresses) the different extensions (and quirks, even bugs) of the gcc compiler. So the push towards more standards conformance, even the elimination of bugs in the compiler got at odds with Linus. A famous case was the bug in ia32 that allowed to write illegal asm statements (sadly in a way that was quite logical). Linus does have a point, but the egcs people have to care not just for the linux-kernel-compiler, they have to care for a general purpose tool, even a host of other languages (The current GCC (GNU Compiler Collection) includes C, C++, Objective C, FORTRAN 77, Java, and chill. More are to come, at least Pascal is supposed to be integrated soon AFAIK). I'd say that you loose somewhat locally, but the overall quality of the code is much better. And the egcs folks _do_ care for the kernel, a bug in the compiler exposed by the kernel is high priority; also caveats and workarounds for problems compiling the kernel are routinely posted in the compiler FAQ."

4. devfs V. 116 Announced

19 Jul 1999 (1 post) Archive Link: "[PATCH] devfs v116 available"

Topics: FS: devfs

People: Richard GoochTim Waugh

Richard Gooch announced:

Hi, all. Version 116 of my devfs patch is now available from: http://www.atnf.csiro.au/~rgooch/linux/kernel-patches.html. The devfs FAQ is also available here.

This is against 2.3.10. Highlights of this release:

5. 2.2.10ac11 Announced; Bugs Found

19 Jul 1999 - 20 Jul 1999 (7 posts) Archive Link: "Linux 2.2.10ac11"

People: Alan CoxJames W. Laferriere

Alan Cox announced 2.2.10ac11 (ftp://ftp.us.kernel.org/pub/linux/kernel/alan/) , and said:

A lot of updates here. I've not chased down the lockd bug reports yet, nor the umount nfsd bug that is trapped by the SLAB debugging. This mops up most of the stuff while I've been occupied elsewhere. Chances are given the number of updates there will be a few glitches in it. Have fun testing.

I'll be stripping the important fixes out of this to merge with the other stuff stripped out for a 2.2.11 draft for Linus starting tomorrow.

James W. Laferriere got an error:

inet.c: In function net_init':
inet.c:1105: `SOCK_DGRAM' undeclared (first use in this function)
inet.c:1105: (Each undeclared identifier is reported only once
inet.c:1105: for each function it appears in.)
inet.c:1139: warning: implicit declaration of function `inet_aton'
make[1]: *** [inet.o] Error 1

and said, "Hello Alan, something in the < ac10 , breaks the sys/socket.h include structure. Making the below under 2.2.10-vanilla compiles like a charm (minus some silly warnings). I have dug into the sys/socket.h tree of includes & I find SOCK_DGRAM defined in asm/socket.h which is included by linux/socket.h which in turn is included by sys/socket.h. But the below happens anyway. Where might I be going wrong ?"

Alan replied, "The include file changes break libc5 it seems. Right now I've not figured the right way to fix that without upsetting glibc"

Several folks posted patches, but Alan had no comment, and the thread ended.

6. Networking Conformance Issues

18 Jul 1999 - 24 Jul 1999 (5 posts) Archive Link: "timestamps, tcp_paws_discard and out of order packets/acks (fwd)"

Topics: Networking

People: Benjamin C.R. LaHaiseDavid S. MillerPete Wyckoff

Benjamin C.R. LaHaise asked, "I'm wondering if anyone has tackled the problem I pointed out a while ago where an ack manages to arrive before data sent on the same connection due, causing the tcp paws support in 2.2/2.3 to dump the data packet? The traces at http://www.kvack.org/~blah/a.(leeloo|mole) (http://www.kvack.org/~blah/) are still valid against 2.3.10, and the behaviour disappears when timestamps are disabled. Debugging verified that tcp_paws_discard is the source of the packet drops, specifically the second term (...tsval - tp->ts_recent) < 0."

Pete Wyckoff noticed from the linked page that leeloo was setting ts_recent for the dup ack packet it was getting. He didn't see where RFC1323 ("TCP Extensions for High Performance") (http://www.cis.ohio-state.edu/htbin/rfc/rfc1323.html) called for such behavior. Ben posted a patch, and replied, "Ahh, you're right! I went back over RFC1323 and found out that the code in tcp_replace_ts_recent seems to be the root of this problem. The comment simply don't jive with the code or my reading of the rfc. I've changed things to abide by RFC1323 and the commented change, and now things are flying, even with reordered packets. My reading of RFC1323 is that it tries to only update ts_recent for in-order packets, which makes sense if packets are reordered."

David S. Miller replied:

I have to think about this change some more. There are 3 independant issues working here.

  1. RFC1323 describes the check wrong, as stated by the comment.
  2. The rules stated by all RFCs and working drafts do not allow for best utilization of ACK feedback when ACKs are massively reordered. This is what my PAWS workaround is trying to accomplish. I brought this issue up, with detailed traces, to the tcp-impl list, showing how detrimental it was to performance. And the best response I got was "oh that bug, it hurts my head each time I think about it" So Linux will perform well in these cases and nobody else will because aparently even the RFC authors themselves do not care. Note that what Linux does makes the PAWS check rules for pure-ACKs orthogonal to the out of order data packet cases.
  3. Audrey found some other RTT estimation problems which are being dealt with too, and these fixes have security implications as well. (before you could quite easily make a connection believe that the RTT was some nearly infinite value, effectively making the connection useless)

Until I can verify that the bug in question is fixed, and the above 3 issues are still addressed, I cannot apply this patch.

7. Linus' Email Address Forged By Spammers

19 Jul 1999 - 20 Jul 1999 (6 posts) Archive Link: "Fake emails from 'Linus'"

Topics: Mailing List Administration, Spam

People: Linus TorvaldsRichard Gooch

Linus Torvalds reported:

Just a heads-up: somebody is sending out fake emails that claim to be from me, and that have me endorsing the Java client for Seti@Home.

The reason I know somebody is faking emails is that I got a bounce from one of them.

If somebody on the kernel list gets a message that claims to be from "Linus Torvalds <torvalds@transmeta.com>" with a subject line of "Seti@Home user interface", it is fake.

I'd like to see the full headers from such a message, to see if it shows where it is really originating from: the bounced message does not contain the original headers..

I assume it is a mass-posting trying to market Seti@Home or the particular client in question, and I'm not all that amused.

                Linus

PS. Although I have to admit that the first line brought a grin: "Being the awesome Linux stud that I am.."

Richard Gooch replied that the Seti@Home people were probably not behind the forgeries, since they seemed reputable. There was also some small discussion about PGP signatures.

8. Capabilities Bug And Fix

19 Jul 1999 (2 posts) Archive Link: "[patch] capability paranoia"

Topics: Capabilities

People: Chris EvansMatthew Kirkwood

Matthew Kirkwood posted a patch to drivers/char/mem.c, requiring CAP_SYS_RAWIO to open /dev/mem, /dev/kmem and /dev/port. Chris Evans replied, "I was working under the assumption we already limit /dev/mem et. al to CAP_SYS_RAWIO. If we don't then this needs applying ASAP."

9. util-linux V. 2.9 Announced

20 Jul 1999 (1 post) Archive Link: "util-linux-2.9v"

Topics: FS

People: Andries Brouwer

Andries Brouwer announced:

I just put util-linux-2.9v the usual places. People complained that using *fdisk failed on 600 GB disks. If I made no mistake cfdisk will now work up to 2 TB (mainly by ignoring the kernel's answer to HDGETGEO).

Once 2 TB looked almost infinitely large, but today 100 GB is quite common, and we can expect that very soon this 2 TB will be a real limit. In other words, this old ugly DOS-type partition table will have to be replaced.

10. 2.2.x API Broken And Fixed

20 Jul 1999 - 21 Jul 1999 (4 posts) Archive Link: "SO_REUSEADDR problem with 2.2.10-ac11?"

People: Wichert AkkermanBrandon S. AllberyAlexey Kuznetsov

Wichert Akkerman reported, "I upgraded my system from 2.2.6 to 2.2.10-ac11 yesterday and noticed an interesting problem: multiple sockets can now bind to the same port on the same interface. I noticed this when I suddenly had 10 ircd running and client connections were getting accepted by the wrong server (ie not the first one). I've been told this also happens with 2.2.10-ac10." He included a simple test program to illustrate the problem.

Brandon S. Allbery replied, "Would you believe that Solaris 2.6 lets you do this as well?" But Wichert said, "Perhaps, I don't use that. It might even be correct behaviour officialy, but introducing such a change in a 2.2 kernel is a Really-Bad-Idea(TM) since it breaks a lot of programs."

Alexey Kuznetsov said, "It was not deliberate, certainly. The patch is appended."

11. Patch For Extended Disc Stats In 2.2.x And 2.3.x

20 Jul 1999 (1 post) Archive Link: "[PATCH] Extended disk statistics."

Topics: Ioctls

People: Daniel Kobras

Daniel Kobras said, "The attached patch against 2.3.10 (applies fine up to 2.3.11pre5, haven't checked pre6 yet) adds two ioctl()s BLKSSTAT and BLKGSTAT that provide disk i/o statistics per (minor) block device. It's an extension of the basic accounting already exported via /proc/stat. In order to use it, you have to turn on CONFIG_BLK_EXT_STATS and furthermore issue an approriate BLKSSTAT-ioctl() on each device you want to account for. See the comments on top of drivers/block/diskstat.c for details." He also gave a URL (http://www.tuebingen.linux.de/kobras/kernel/) to patches against 2.2 kernels.

12. Framebuffer HOWTO 1.1 Announced

22 Jul 1999 (1 post) Archive Link: "HOWTO-Framebuffer 1.1 released!"

Topics: Framebuffer

People: Alex Buell

Alex Buell announced:

I have today released the next revision to the HOWTO-Framebuffer. It now includes a section on multi-headed displays, kindly donated by Frederick A. Niles. I have taken over responsibility for distribution and maintenance for his mini HOWTO-Multihead document, but he will retain all copyrights to his document.

You may access it at http://www.tahallah.demon.co.uk/programming/Framebuffer-HOWTO-1.1.html

13. ppdev Back-ported to 2.2.10

24 Jul 1999 (1 post) Archive Link: "ppdev is released for 2.2.10"

Topics: Ioctls

People: Tim Waugh

Tim Waugh announced:

I have back-ported the ppdev code (support for user-space parallel port device drivers) to 2.2.10, in such a way that it is self-contained.

Just as in 2.3, it allows programs to open("/dev/parport0/index.html") and perform ioctls to claim/release the port, examine/set the data, status, and control lines, as well as perform IEEE 1284 negotiation and data/address transfers. In 2.2.10-ppdev1, however, all transfers are software-driven; DMA/FIFO support is only in 2.3.

The ppdev code supercedes ppuser-0.8.

It should be relatively easy to add the parallel printer console to ppdev-2.2.10, in the same way that lp does it in 2.3.

Get it while you still can:

<URL:http://www.cyberelk.demon.co.uk/parport/patch-2.2.10-ppdev1.gz>
<URL:ftp://ftp.torque.net/pub/parport/patch-2.2.10-ppdev1.gz>

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.