Kernel Traffic #96 For 4 Dec 2000

By Zack Brown

linux-kernel FAQ (http://www.tux.org/lkml/) | subscribe to linux-kernel (http://www.tux.org/lkml/#s3-1) | linux-kernel Archives (http://www.uwsg.indiana.edu/hypermail/linux/kernel/index.html) | kernelnotes.org (http://www.kernelnotes.org/) | LxR Kernel Source Browser (http://lxr.linux.no/) | All Kernels (http://www.memalpha.cx/Linux/Kernel/) | Kernel Ports (http://perso.wanadoo.es/xose/linux/linux_ports.html) | Kernel Docs (http://jungla.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html) | Gary's Encyclopedia: Linux Kernel (http://members.aa.net/~swear/pedia/kernel.html)

Table Of Contents

Mailing List Stats For This Week

We looked at 1200 posts in 5484K.

There were 419 different contributors. 193 posted more than once. 145 posted last week too.

The top posters of the week were:

 

1. Kernel Licensing Uncertainty
8 Nov 2000 - 20 Nov 2000 (118 posts) Archive Link: "[ANNOUNCE] Generalised Kernel Hooks Interface (GKHI)"
Topics: Binary-Only Modules, Real-Time: RTLinux, Sound
People: Richard MooreMichael RothwellTheodore Y. Ts'oAlan CoxPaul JakmaLinus TorvaldsLars Marowsky-Bree

Richard Moore of IBM announced a patch (http://oss.software.ibm.com/developerworks/opensource/linux) to implement the Generalised Kernel Hooks Interface (GKHI), which, as Richard explained, "is a generalised facility for placing hooks or exits in arbitrary kernel locations. It enables many kernel enhancements, which are otherwise self-contained, to become loadable kernel modules and retain a substantial degree of independence from the kernel source." Michael Rothwell replied, "Sounds great; unfortunately, the core group has spoken out against a modular kernel." He suggested that IBM and others fork the kernel and implement their GKHI and whatever else they wanted on their own.

Regarding the modularization policies of the core group, Theodore Y. Ts'o explained:

This is true; that's because a modular kernel means that interfaces have to be frozen in time, usually forever. This makes a lot of improvements impossible, and over time there is more and more crud added to simply act as "impedance matching" between incompatible/legacy interfaces.

However, that doesn't mean that the GKHI is *automatically* bad. It all depends on how you use it. Just as with kernel modules, where it's darned useful for distributions so they don't have to build custom kernels for each installation (but source is included for all modules), the GKHI could be used in a similar way.

The issue will be with what hooks are allowed to be exported via the GKHI; this defines the interfaces. Also, it's important to note which interfaces are and aren't guaranteed to be stable. If the interfaces aren't guaranteed to be stable, then incompatibilities and keeping up with the latest version are a problem for the provider of the binary module, not of the Linux Kernel Development Community.

This wing of discussion quickly led to a GPL debate. Various folks objected to GKHI because they felt it would encourage binary-only modules, and tie the developers down to supporting a bad API. At one point, Michael argued, "I think the IBM GKHI code would be of tremendous value. It would make the kernel much more flexible, and for users, much more friendly. No more patch-and-recompile to add a filesystem or whatever. There's no reason to hamstring their efforts because of the possibility of binary modules. The GPL allows that, right? So any developer of binary-only extensions using the GKHI would not be breaking the license agreement, I don't think. There's lots of binary modules right now -- VMWare, Aureal sound card drivers, etc." Alan Cox replied, "Its not clear the GPL does allow it," and added regarding those binary modules, "All of which just cause large numbers of bugs to go in the bitbucket because nobody can tell whose the problem is." Lars Marowsky-Bree also pointed out that the developers refused to support those binary modules, and Michael replied, "Who says you would support theirs? My point is, forks have been made in the past and are useful for the people that use them, and prevent "pollution" of the common kernel with hghly specialized modifications. uCLinux and RTLinux are two examples." Alan replied, "RTLinux is hardly a fork. UcLinux is a fork, it has its own mailing list, web site and everything. Post 2.4 I'm still very interested in spending time merging the 2.4 uc and the main tree. I think it can be done and they are doing it in a way that leads logically to this." Michael felt the potential IBM fork would be the same. He said, "It'll go off, change and add some things, and then perhaps be merged back in later. In the meantime, developers who want to add "enterpriseness" to Linux will have an outlet and won't have to simply gripe on this list anymore. And users who want an "enterprise" kernel can get one." And Ted replied:

Well, there are three possibilities about how it can end up.

  1. It will be loaded with so much crap that in fact it ends up being less performant than the mainline Linux. No problem, we can ignore it.
  2. It will be a thing of perfect beauty, in which every single change is thoughtfully well-considered, and scales well to both the low end and the high end. No problem, it's easy to merge stuff back.
  3. It will be a mixture, of some stuff which is good, and some such which is crap, and it will be very difficult hard to merge back some of the good stuff, since it has dependencies on the crap. In the meantime, mainline Linux will have continued moving forward, and it will be even harder to merge the advanced features of Linux back into the forked version of the kernel.

One of the big reasons why many of the big companies have been looking at Linux is because Unix OS engineering is viewed as a cost center, not as a profit center. The moment you fork and make an "Advanced Linux Kernel Project", a lot of the costs involved with maintaining such a beast come back. And sure, you can try to solve this problem by working in a consoritum-like fashion with other Companies ---- just like OSF/1 and Monterrey tried to do.

So there are some real costs associated with forking. At the same time, if these companies need to get product out the door quickly, short-term forks can be good things. But nothing in life is free, and it's in *everybody's* best interest to resolve forks as soon as possible. Otherwise, you end up losing a lot of the advantages of the Open Source development model.

Elsewhere, getting back to the question of whether the GPL allowed binary-only modules, Paul Jakma said that the GPL definitely forbade them, although Linus Torvalds "allowed them in most cases". Michael asked if the potential IBM fork would have to follow the same rule, or would it be bound just by the GPL. Paul suggested asking Linus, and added that he felt Linus hadn't been clear enough in his explanation of what was and was not allowed, but Ted explained, "Actually, he's been quite specific. It's ok to have binary modules as long as they conform to the interface defined in /proc/ksyms." But Alan replied:

What is completely unclear is if he has the authority to say that given that there is code from other people including the FSF merged into the tree.

I've taken to telling folks who ask about binary modules to talk to their legal department. The whole question is simply to complicated for anyone else to work on.

Ted replied, "He said it long enough ago that presumably people who merged code in would have accepted it as an implicit agreement, if it had been documented in the COPYING file. Unfortuantely, it wasn't so documented, so I agree it's unclear."

 

2. NFS Filesystem Corruption In Stable Series
15 Nov 2000 - 20 Nov 2000 (5 posts) Archive Link: "[BUG] knfsd causes file system corruption when files are locked."
Topics: FS: NFS
People: Neil BrownTrond MyklebustIvan Kanis

Ivan Kanis reported file system corruption when files were locked under 'knfsd' in 2.2.16. Neil Brown replied, "Lots of changes have gone into knfsd since 2.2.16. Could you please try again with either a later 2.2.18pre kernel, or 2.2.16 with patches from http://nfs.sourceforge.net/ applied? Thanks." Ivan replied that he could reproduce the bug on 2.2.18pre21. Trond Myklebust suggested rooting around in dejanews "for the locking patch I posted on l-k last week (to be applied on top of 2.2.18pre21). It fixes 3 leaks in the locking code (amongst them the share leak). If you can't find it, I'll be happy to post it via private mail..." Ivan replied, "Fantastic news: your patches fixed the bug! I hope it will make it in the kernel pretty soon, I will be testing it some more today." EOT.

 

3. Man-Page Inconsistancies
18 Nov 2000 - 19 Nov 2000 (6 posts) Archive Link: "2.4 sendfile() not doing as manpage promises?"
People: Andries BrouwerDan HollisBert Hubert

Bert Hubert tried to use 'sendfile()' to send data from a tcp/ip socket to a file on disk, but found that it would just return EINVAL and refuse to work. Using kgdb (http://kgdb.sourceforge.net) , he eventually found some code to indicate that sendfile() could only be used to send files from blockdevices which supported 'mmap()'-like functionality. This went against his understanding of the man-page, which said that "either or both" file descriptors could be sockets.

Dan Hollis confirmed Bert's findings regarding 'mmap()'-like functionality, but quoted the man-page as saying

This call copies data between file descriptor and another
file descriptor or socket. in_fd should be a file
descriptor opened for reading. out_fd should be a
descriptor opened for writing or a connected socket.

Bert replied that his own man-page must have been out of date, being from December 1, 1998. Andries Brouwer said that Bert's was indeed the most recent revision, and quoted the official text:

This call copies data between one file descriptor and
another. Either or both of these file descriptors may
refer to a socket. in_fd should be a file descriptor
opened for reading and out_fd should be a descriptor
opened for writing.

Andries continued:

If that is incorrect, then editing a private copy of the manpage, as Dan Hollis, or the distributor from whom he got his page, seems to have done, does not suffice to change the manpage distribution.

(Moreover, the text Dan Hollis quotes is rather strange -- also sockets give one a file descriptor, so the author of that modified man page did not know what he was talking about, or was not being precise.)

Dan replied, "It's from redhat 6.0 man-pages-1.23-3 rpm package." Bert suggested the following replacement text:

This call copies data between one file descriptor and another. The
descriptor from which data is read cannot be a socket but must
correspond to a file which supports mmap()-like operations. in_fd
should be a filedescriptor opened for reading and out_fd should be a
descriptor opened for writing. Because this copying is done within
the kernel, sendfile() does not need to spend time transfering data
to and from userspace.

The thread ended there.

 

4. Approaching 2.2.18
18 Nov 2000 - 22 Nov 2000 (15 posts) Archive Link: "Linux 2.2.18pre22"
Topics: Executable File Format, Networking, USB
People: Alan CoxKai GermaschewskiMiquel van SmoorenburgTrond MyklebustOleg DrokinJeff V. MerkeyMikael Pettersson

Alan Cox announced Linux 2.2.18pre22 and said that "Anything which isnt a strict bug fix or previously agreed is now 2.2.19 material," and added that PS/2 mouse detection was the only remaining known bug holding up 2.2.19. He posted the changelog:

2.2.18pre22

  • Fix HZ assumption in USB hub driver (Oleg Drokin)
  • Fix ndisc range check on ipv6 (Dave Miller)
  • Clear other fields in qcam VIDIOCGWIN return (Damion de Soto)
  • Fix sparc64 includes for socket.h (Solar Designer)
  • ELF platform was misset for Pentium IV (Mikael Pettersson)
  • ADMTek 985 ident was wrong (Lee Bradshaw)
  • Fix filemark status test on scsi tape (Robin Miller)
  • Fix file/block when spacing to tape beginning (Kai Maiksara)
  • Small ISDN documentation fixes (Kai Germaschewski)
  • Resync icn driver with core isdn tree (Kai Germaschewski)
  • Fix isdn loopback driver (Kai Germaschewski)
  • Fix small leaks in lockd (Trond Myklebust)
  • Add Pentium IV rep nop, ident etc (Various folks, notably HPA and Linda Wang)

Miquel van Smoorenburg and Michael Marxmeier suggested that megaraid was probably also in the "known bug" category. Arnaud S. Launay, Tomasz Kloczko, and John Kennedy all posted short patches to allow them to compile the release. Jeff V. Merkey reported some problems building isdn4K-utils and there was a little discussion about it, but nothing conclusive. At one point, Kai Germaschewski said, "It'ld be nice if you at least CC'ed your mail to the maintainers, i.e. the isdn4linux people, because not everyone has the time to follow l-k. Any way, I CC'ed isdn4linux@listserv.isdn4linux.de now (mailto:isdn4linux@listserv.isdn4linux.de) , and this thread should continue there."

 

5. Boot Message Tweaks
19 Nov 2000 - 20 Nov 2000 (7 posts) Archive Link: "What is 2.4.0-test10: md1 has overlapping physical units with md2!"
People: Jasper SpaansMarc MutzNeil BrownIngo MolnarMatti Aarnio

George Garvey had been alarmed to see a "Nov 18 16:31:02 mwg kernel: md: serializing resync, md0 has overlapping physical units with md2!" message during bootup, and asked if there was anything to worry about. It seemed like a disaster waiting to happen to his poor disk. Matti Aarnio and Jasper Spaans both explained that the message was merely a poor choice of wording, and as Jasper put it, "What it means is that some partititions in md1 and md2 are on the same disk, and that the md-code will not do the reconstruction of these arrays in parallel [of course, for performance reasons]." Neil Brown posted a patch (with a little proof-reading from Marc Mutz) to change the wording to "md: serializing resync, md%d has shares one or more physical units with md%d!\n" in the source. Ingo Molnar thought this would be fine, but Marc Mutz noticed a typo (see if you can spot it ;-), and Ruth Ivimey-Cook also suggested changing the wording to "md: serializing resync, md%d shares one or more disk drives with md%d. Array performance may suffer.\n"

 

6. Praise For user-mode Linux
19 Nov 2000 - 22 Nov 2000 (2 posts) Archive Link: "user-mode port 0.34-2.4.0-test11"
Topics: User-Mode Linux
People: Jeff DikeDaniel Phillips

Jeff Dike announced user-mode Linux based on 2.4.0-test11. He explained: "UML is now able to run as a daemon, i.e. with no stdin/stdout/stderr." He also gave a link to the project home page (http://user-mode-linux.sourceforge.net) and the download page (http://sourceforge.net/project/filelist.php?group_id=429) . Daniel Phillips glowed:

I'm making heavy use of uml for mm, vfs and fs development, and I can't say enough good things about it. With uml I have a development cycle that looks roughly like this:

20 secs make+gcc+ln
10 secs boot new user mode linux
6 secs fsck (if crashed before) :-)
*tests go here*
10 secs shutdown

And there are still a lot of ways to tighten that up. The stability has been impressive - so far, no crashes at all that weren't supposed to be crashes, at least in the work I'm doing.

 

7. '/proc/index.html' Documentation Updates
20 Nov 2000 - 23 Nov 2000 (4 posts) Archive Link: "[PATCH] Documentacion proc.txt update (2.4.x)"
People: Jorge NerinMike A. HarrisDan Hollis

Jorge Nerin posted a patch to update a lot of information in '/kernel-traffic/Documentation/filesystems/proc.txt', which had not been updated for over a year prior to that. In the text was a URL, which failed a 'traceroute' by Dan Hollis. Jorge replied, "Well, the URL came with the text, it was there before me ;-) I didn't work for me either, but I left it because I don't have a URL with the updated version online, even I don't have an updated version, I had to do it myself. I have left it because I don't have replies from the author nor from the website." But Mike A. Harris pointed out, "No information is always better than misinformation." EOT.

 

8. Debian Difficulties With Large File Support
20 Nov 2000 - 21 Nov 2000 (9 posts) Archive Link: "Bug in large files ext2 in 2.4.0-test11 ?"
People: Zdanek KabelacPetr VandrovecWichert AkkermanBen Collins

Zdenek Kabelac reported that for the latest development kernels, 'ls' refused to show large files (>4G) on his system, giving a "ls: zero: Value too large for defined data type" error. On 2.2.17pre9 the files displayed correctly. He added, "I would assume this is some problem of the kernel, but maybe its incompatibility in libc - anyway I'm using uptodate Debian Woody if this helps." Petr Vandrovec replied, "It is problem with Debian's glibc, it is compiled with 2.2.x kernel headers. I filled bugreport against this few weeks ago, but it was marked as inappropriate because of Debian glibc is built against newest kernel-headers package, and newest kernel-headers package is still 2.2.x." He suggested filing a bug report with Debian to upgrade the kernel package, which would allow the 'glibc' maintainer to compile against 2.4 and fix the problem. But Ben Collins felt that unless 2.4 fully supported all of Debian's supported architectures, it couldn't go into Woody yet. And if binaries compiled with large file support under 2.4 kernels would not run correctly under 2.2, he didn't expect to see 2.4 in Debian until post-Woody.

Petr replied that as far as he knew, all Debian architectures were supported, and binaries compiled for large file support would indeed run under 2.2; there was some discussion in part going over just which architectures Debian provided; and at one point Wichert Akkerman said, "Euhm, we released 6 architectures (arm, alpha, i386, m68k, powerpc and sparc)"

 

9. Moving Away From Preprocessor Constructs
21 Nov 2000 - 26 Nov 2000 (13 posts) Archive Link: "beware of dead string constants"
Topics: Version Control
People: Jeff GarzikPeter SamuelsonChris WedgwoodBernd EckenfelsJakub Jelinek

Peter Samuelson reported that gcc 2.95.2 on the i386 was not properly optimizing unused string constants into oblivion. This interfered with migrating the kernel code away from using #ifdefs (preprocessor commands) and toward using if()s (C language code handled directly by the compiler). As far as he knew, this migration was the way of the future, and showed gcc to be not quite ready for prime time use. Jakub Jelinek replied that he'd already put a fix into the gcc CVS tree a week before (to which Peter kicked up his heels). Elsewhere Bernd Eckenfels asked what would be gained by changing from preprocessor commands to normal code; Jeff Garzik replied it was Linus' Torvalds' preference, and that he agreed. As Jeff put it, "code looks -much- more clean without ifdefs. And the compiler should be smart enough to completely eliminate code inside an 'if (0)' code block." Peter continued, "Plus the advantage/disadvantage of making the compiler parse almost everything, which should eliminate syntax errors, variable name misspellings, etc in little-used config options. The disadvantage is that compilation speed goes down." And Chris Wedgwood added, "The same can also be said for some of the many macro's that are #defined; some would surely be better as inline functions is we could ensure they would always be in-lined."

 

10. Supporting Non-PnP 53c400 SCSI Cards
21 Nov 2000 - 22 Nov 2000 (4 posts) Archive Link: "53c400 driver"
Topics: Disks: SCSI, SMP
People: Alan CoxIgmar PalsenbergRandy Dunlap

For his first kernel hacking experience, Igmar Palsenberg decided to try supporting non-PnP 53c400 SCSI cards; he asked how difficult it would be to make the existing drivers SMP safe, or if it would be better to just start from scratch. Alan Cox replied, "The problem is that the code takes spinlocks recursively. The original code (see 2.0's 5380 generic C code) uses cli/sti. It was converted to use spin_lock() but not allowing for the recursive locking cases. I tried to untangle the code paths but they are so ugly and some of the code is sufficiently messy and unmaintained (for about 6 years) that I gave up trying to decode it." Randy Dunlap suggested using the UHCI driver as a sample, since it solved similar problems. But Igmar replied that it might "solve the nested spinlock issue.. It however doesn't solve the 100Kb+ pile of spaghetti the code is. I think I'll just start over. I have to do something in my spare time :)" End of thread.

 

11. Intentionally Nonexistent '__bad_udelay()': The Saga Continues
21 Nov 2000 - 24 Nov 2000 (12 posts) Archive Link: "linux-2.2.18-pre19 asm/delay.h problem?"
People: Alan CoxJoe PranevichRogier WolffPeter Samuelson

Someone else was bitten by the nonexistent '__bad_udelay()' function, covered in Issue #90, Section #8  (10 Oct 2000: Nonexistent Functions In The Kernel - And Staying) . This time the victim was Joe Pranevich, who felt it might be a bug in one of the modules he compiled. Alan Cox replied that it was indeed a module bug, because the module made an overlarge delay. Peter Samuelson suggested renaming the funtion to '__use_mdelay_instead_please()'. Rogier Wolff suggested defining the function inside an '#if 0', which would include a comment explaining the situation.

 

12. Status Of LKCD
22 Nov 2000 - 25 Nov 2000 (10 posts) Archive Link: "LKCD from SGI"
Topics: Disks: IDE, Disks: SCSI
People: Matt D. Robinson

Someone asked if the Linux Kernel Crash Dumps patches from SGI were going into 2.4, 2.5, or what, and Matt D. Robinson replied:

LKCD won't go into 2.4 (or 2.5) until I finish writing the direct disk open/write functions that avoid going through the standard IDE and SCSI drivers. I'm working on it.

As far as work for 2.4 goes, we've got a version on SourceForge that works well (for i386 and 95% for ia64).

As soon as the drivers are done, we'll hopefully get acceptance.

 

13. New Include Directory For Internal Interfaces
22 Nov 2000 (5 posts) Archive Link: "include conventions /usr/include/linux/sys ?"
Topics: Ioctls
People: Linda WalshArjan van de VenH. Peter AnvinAndi KleenMitchell Blank JrMitchell Blank Jr"

Linda Walsh asked, "Linus has mentioned a desire to move kernel internal interfaces into a separate kernel include directory. In creating some code, I'm wondering what the name of this should/will be. Does it follow that convention would point toward a linux/sys directory?" Arjan van de Ven suggested:

include/linux - exported interfaces
include/kernel - kernel internal interfaces
include/asm - kernel internal/archspecific interfaces

Similarly, H. Peter Anvin replied to Linda, "I suggested include/kernel and include/arch with include/linux and include/asm being reserved for the kernel interfaces (ioctl and their structures mostly.)" Andi Kleen added, "It would also be useful to put *32 structures for 32->64bit conversion in there (to prepare for a generic 32->64bit conversion layer in 2.5)." Mitchell Blank Jr liked H. Peter's suggestion, and replied to him:

sounds good. One other refinement I would like to see is that architecture specific but always present header files don't get used directly in architecture-independant .c flies. Some common examples of this are <asm/io.h> and <asm/uaccess.h> The problem is that often there is some code that is identical between some or all of the archs. Then one of two things happen:

  • The code gets duplicated in many *.h files with all the bad things that comde duplication causes - especially since they're all under seperate maintainership
  • We have a big ugly conversion like what happened with 'asm/spinlock.h' -> 'linux/spinlock.h'

If we only have arch-independant *.c files only include things out of <kernel/*> (which may, of course, include things in <arch/*>) we can avoid these conversions in the future and promote code reuse between the architectures.

Also we should probably consider implementing a reasonalbe hierarchy to the proposed <kernel/*.h> directory - for instance <kernel/bus/pci.h>.

 

14. Dell 5000e: The Saga Continues
22 Nov 2000 (2 posts) Archive Link: "Inspiron 5000e dmesg from test11-ac2"
People: Alan CoxJeff Lessem

Continuing from Issue #95, Section #2  (13 Nov 2000: Serious Dell 5000e Laptop Power Management Problems) , Jeff Lessem posted some dmesg output, and Alan Cox replied, "As I understand it Compal will be issuing Dell with a BIOS update for this board 'at some point'. When the BIOS upgrade is done the apm battery data should be back."

(According to recent threads (see next issue), it looks as though Compal has in fact produced an updated BIOS, and plans to release it to the public shortly. -- Ed: [04 Dec 2000 00:00:00 -0800]

 

 

 

 

 

 

We Hope You Enjoy Kernel Traffic
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License, version 2.0.