Kernel Traffic #261 For 9 Jun 2004

There were 341 different contributors. 156 posted more than once. 141 posted last week too.

This was one of those, "this would be weird and great", "no it wouldn't" discussions. Sergiy Lozovsky started it up innocently enough, asking how to deal with kernel stack overflow. Then it turned out that "I put LISP interpreter inside the Kernel - http://vxe.quercitron.com - It works, but it use a lot of stack memory. It's impossible to rewrite it easily, though I'll investigate why exactly it uses so much of stack memory" .

Timothy Miller, who became very outspoken in the thread, wanted to know why on earth Sergiy felt the need to add a Lisp interpreter to the kernel, and Sergiy explained that it was for security purposes. First, it would allow system administrators to configure the system security policies without having to recompile the kernel or write in the lower-level C language; and second, it would protect the system from bugs in that security policy, that might have crept in through errors on the part of the system administrator doing the configuration. As Sergiy explained it, the "LISP interpreter is a LISP Virtual Machine (as Java VM). So all bugs are incapsulated and don't affect kernel. Even severe bugs in this LISP kernel module can cause termination of user space application only (except of stack overflow - which I can address). LISP error message will be printed in the kernel log."

There was a pretty big discussion about this, mainly involving folks peppering Sergiy with various objections, and Sergiy attempting to weather them as best as he could. One objection was that the whole kit and kaboodle belonged in user space and not in kernel space. John Stoffel said the entire thing would have to be redesigned in order to have even a small hope of ever getting a foot-hold in the kernel. He pointed to all the protections Sergiy had implemented to keep LISP bugs from harming the kernel, as precisely the reason why the thing belonged in user space in the first place. In user space, there would be no need for the elaborate kernel protections. In response to this, Sergiy argued that he was simply extending the traditional UNIX security model in a natural way; he said that if his LISP interpreter belonged in user space, then the same argument could be used for putting the checking of root privileges and file permissions in user space as well.

Another objection folks made against Sergiy's LISP interpreter was the sheer size of the beast. As Timothy put it in one of his posts, "A LISP VM is a big, giant, bloated.... *CHOKE* *COUGH* *SPUTTER* *SUFFOCATE* ... thing which SHOULD NEVER be in the kernel." Sergiy replied that the footprint was just 100K. He hadn't implemented the full Common LISP interpreter, but just a small subset. He said he hadn't been able to find any Virtual Machine that would fit in a smaller space. At this point, Robin Rosenberg pointed out that LeJOS (http://www.lejos.org) for the Lego Mindstorm would fit into 32K, and was licensed under the Mozilla license.

Another objection was just that LISP was the wrong language in any case, and that, as Timothy put it, system administrators would never write a security policy in LISP, but would use something more like the UNIX shell; or perhaps, he said, "Another possibility is to develop a set of tools that compile policies written in C into modules that are loaded/unloaded into the kernel dynamically. :) The compile process would be transparent to the user, because the "insert this policy" tool would run it through GCC (unless the cached .ko was already up to date, etc.)."

Overall, Sergiy had interesting responses to all the objections, but it didn't appear that he won anyone over on the important point, namely whether such a thing should go into the kernel or not.

Andy Isaacson said, "On modern Linux, apparently the correct way to bypass the buffer cache when writing to a block device is to open the block device with O_DIRECT. This enables, for example, the user to more easily force a reallocation of a single sector of an IDE disk with a media error (without overwriting anything but the 1k "sector pair" containing the error). dd(1) is convenient for this purpose, but is lacking a method to force O_DIRECT. The enclosed patch adds a "conv=direct" flag to enable this usage." Andrew Morton was very encouraging, and asked Andy to do some additional clean-up work as well, while he was busy in that area of the code. While he was working, Andy encountered some difficulties. He said, "I can't get O_DIRECT to work on ext3, at all, on 2.4.25 -- open(O_DIRECT) succeeds, but the write returns EINVAL." Andrew replied, "ext3 doesn't support O_DIRECT in 2.4 kernels. I did a patch once and I think it's in 2.4-aa kernels. ext3 supports O_DIRECT in 2.6 kernels. Quite a number of filesystems do."

Along the way, it became clear that Andy was only planning to implement his O_DIRECT support for disk writing only, under the assumption that there was no point to have it for disk reading as well. But Andrew said there were the same arguments for enabling O_DIRECT for disk reads as writes, in that it saved CPU usage and avoided over-using the buffer cache. Andy agreed that CPU usage would be lower with O_DIRECT read support, but he felt the gain would not be significant to justify the implementation; while he felt it was "the kernel's responsibility to figure out "hey, looks like a streaming read - let's not blow out the buffer cache trying to hold 20GB on a 512M system." If you're saying that the kernel guys have given up and the established wisdom is now "you gotta use O_DIRECT if you don't want to throw everything else out due to streaming data", well... I'm disappointed." And overall, he concluded:

Wouldn't opening both if= and of= with O_DIRECT turn dd into a synchronous copier? That would really suck in the "dd if=/dev/hda1 of=/dev/hdc1" case. With buffer cache doing readahead, that command can get, say, 40MB/s read and 40MB/s write; with synch read and synch write, it would drop to 40MB/s read+write, assuming that block sizes are big enough to amortize seek overhead.

Having O_DIRECT on just of=, I think you can get back to 40MB/s+40MB/s.

I claim that O_DIRECT on of= is important because you just plain *can not* do the minimal-sized IDE block scrub without it. I don't yet see a similar benefit to O_DIRECT on if= side.

If you want a block scrubber then write a block scrubber.

If you want to add O_DIRECT support to dd then it should be implemented properly, and that means implementing it for both read and write.

In fact the user should be able to specify the read-O_DIRECT and the write-O_DIRECT independently - if for no other reason than that the source and dest filesytems may not both support O_DIRECT.

Andy felt it really wasn't the best idea, but wrote and submitted a patch regardless, according to Andrew's specifications. At around this point, Paul Eggert also posted a patch of his own, to implement the 'dd' features folks had been creating kernel infrastructure for. He said:

At the end of this message is a proposed patch to implement everything people other than myself have asked for so far, along with several other things since I was in the neighborhood.

This patch adds the following NEWS entry, which I'll repeat here to make it easy to see what's going on:

dd has new conversions for the conv= option:

    nocreat   do not create the output file
    excl      fail if the output file already exists
    fdatasync physically write output file data before finishing
    fsync     likewise, but also write metadata

dd has new iflags= and oflags= options with the following flags:

    append    append mode (makes sense for output file only)
    direct    use direct I/O for data
    dsync     use synchronized I/O for data
    sync      likewise, but also for metadata
    nonblock  use non-blocking I/O
    nofollow  do not follow symlinks
    noctty    do not assign controlling terminal from file

This patch is relative to coreutils CVS, so your line numbers may differ. I didn't add hooks for Solaris-style direction, because that turns out to be a systemwide property and perhaps should be in a different command.

Jim Meyering from 'coreutils' development said he had applied the patch to the 'coreutils' tree. But Philippe Troin objected, "noctty definitely seems overkill... I can't see why dd would ever want to open a file without O_NOCTTY. On systems where O_NOCTTY makes sense (SvR4) that is." Paul agreed that this would just confuse the user, and added a patch to cause 'dd' to always use O_NOCTTY. Jim accepted this patch as well into 'coreutils'.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.5/2.6.5-mm2/

Merged up Ian Kent's autofs4 patches
Various fixes and speedups.

A small thread followed. Jeremy Higdon got a build error on his 64-bit Intel machine, saying, "There have been changes to setup_arch(), including, apparently, the elimination of the cmdline_p argument. Unforunately, that argument was not completely purged from the function." He posted a one-liner to fix compilation on his machine, and Rusty Russell posted a more all-inclusive fix. However, Andrew replied:

Guys, these patches have been causing pain for two weeks. They have thus far required at least twelve bugfixes and I have every expectation that it will take weeks in Linus's tree to identify and fix all the as-yet undiscovered bugs. They are taking way too much time which could otherwise be spent on all the other broken stuff people like to send me.

So I've dropped them all. They're at http://www.zip.com.au/~akpm/linux/patches/early-param/. That includes this most recent patch of Rusty's.

Yep, that's pretty much what happened last time we tried to fix this, too.

I'll snarf them and try to break them down into pieces which really can go in one at a time, and post to lkml for interested people to test...

Here's a patch that adds object and control method access to devices in the /sys/firmware/acpi/namespace/ACPI/ tree. I sent out an RFC on this approach several days ago and got a couple of good responses. Since then I've added a lot more methods, including several control methods to allow you to do more than simply look at object properties.

There are a lot of potential consumers of this type of data. For instance:

Laptop users may be able to access some of their 'hotkey' features from userspace (display switching, eject).
Servers and workstations can associate PCI buses, slots, address spaces, etc.
Hotplug tools could discover devices dependent on other devices prior to eject.
Manageability tools could have easy access to device identification and location methods.
etc...

The code has been tried on several x86 and ia64 systems. I expanded the existing sysfs namespace to include devices which the _STA methods do not list as present at bootup. This helps to show things like hotplug PCI slots that aren't currently in use and extra battery bays in laptops that are vacant. There are several methods that return complex data structures. I started trying to parse them into pretty text for output, but the code bloat got too big too fast. So, I've left that for userspace tools and just dump the raw data via this interface. Most object methods return basic elements and are easily represented as parsed text. Also, fair warning: calling _EJ0 methods on devices may be unsafe. I've made the files only accessible by root to help avoid issues in this area. A couple fun things for laptop users (YMMV):

To eject from a dock (works on an omnibook 500, exact paths will vary):
echo 0 > /sys/firmware/acpi/namespace/ACPI/_SB/PCI0/ISA0/JP1/_DCK
echo 1 > /sys/firmware/acpi/namespace/ACPI/_SB/PCI0/ISA0/JP1/_EJ0

When re-docking, do:
echo 1 > /sys/firmware/acpi/namespace/ACPI/_SB/PCI0/ISA0/JP1/_DCK

To switch displays (X can get in the way and prevent switching):
CRT only -> LCD only:
echo 0 > /sys/firmware/acpi/namespace/ACPI/_SB/PCI0/AGP/VGA/CRT/_DSS
echo 0x80000001 > /sys/firmware/acpi/namespace/ACPI/_SB/PCI0/AGP/VGA/LCD/_DSS
LCD only -> LCD+CRT:
echo 0x80000001 > /sys/firmware/acpi/namespace/ACPI/_SB/PCI0/AGP/VGA/CRT/_DSS
LCD+CRT -> CRT only:
echo 0x80000000 > /sys/firmware/acpi/namespace/ACPI/_SB/PCI0/AGP/VGA/LCD/_DSS

It's highly dependent on how your firmware works as to how complete the behavior will be. I've seen a report from an IBM T30 that _EJ0 from the dock won't trigger the electro-mechanical eject, but seems to do most everything else. Bug reports welcomed (but I can't fix your firmware if it doesn't do what you expect). Please consider integrating upstream.

The next day he posted again, saying, "Here's another approach that's far less ugly than the last and is much more powerful. The code is a little over half the size as a bonus. Rather than specifically poking for certain methods and exposing them, this patch exposes everything. The down side is that all reading and writing of the files need to use binary acpi data structures. This interface certainly provides "shoot yourself in the foot" potential, but the access to the namespace from userspace is hard to beat. Any thoughts on this approach versus the last? This interface and a simple set of libraries to go along with it has a lot of potential." John Belmonte asked, "The limitation of this interface is that it's not able to call an ACPI method with some arguments and get the return value, correct?" Alex confirmed, "Yes, that's unfortunately a limitation. Most of the standard interfaces either take no parameters or have no return value, so they fit nicely into this framework. I'm open to suggestions on how to work around this. We could make the store function save off the method parameters, then the show function would call the method with the saved parameters and return the results. Obviously there are some userspace ordering issues that could make this complicated, but it's easy to code on the kernel side." John gave a pointer to a mailing list thread (http://sourceforge.net/mailarchive/message.php?msg_id=7455349) where the issue was discussed, and Alex replied:

Ok, I took a look. The open/write/read/close interface seems to be the best approach. It shouldn't be too hard, except the read/write interfaces don't pass in the attribute pointer like they do for the show/store interface. Is this a bug?

A nice feature of the current implementation is that I have one show function that reads the method name out of the attribute structure. With the bin_file interface, I'd need to have a different read/write function for every possible method on a kobject. I also lose the convenience of being able extend the container structure to meet my needs. Am I strecthing the use of the bin_file too far or should these interfaces pass the attribute pointer?

He replied to himself with a patch, saying, "I made the assumption that this wasn't that big of a stretch to the bin file interface and extended it to add the arribute pointer. Below is what I came up with. There are luckly only a handful of places using the current sysfs bin file interface, so the additional changes are pretty small. I'm guessing I can probably get rid of the spinlocks I added for the previous implementation, but they're not hurting anything for now. Am I getting closer?" Matthew Wilcox replied:

Yes, definitely closer to my way of thinking ...

It seems unintuitive that you have to read the file for the method to take effect. How about having the write function invoke the method and (if there is a result) store it for later read-back via the read function? It should be discarded on close, of course. A read() on a file with no stored result should invoke the ACPI method (on the assumption this is a parameter-less method) and return the result directly. Closing a file should discard any result from the method.

Alex replied, "How's this? It behaves the way you described, but might be doing some questionable things with the buffer to get there. Is there a better place to store the return data than back into the buf passed to write() (aka file->private_data)? Without adding callbacks to open/close, I'm not sure how else we can dispose of the results on close." Andy Lutomirski asked why not use an ioctl for this, and Alex replied that according to the thread John gave the pointer to, and according to Matthew, SysFS did not support ioctls at all.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.5/2.6.5-mm3/

Kernel NFS server update
Added the NUMA memory binding API, minus its hugetlbpage part.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.5/2.6.5-mc4/

This is my patch queue for Linus next week.

Added the ia64 implementation of vectored interrupts
A significant ext3 speedup on big SMP
lots of small fixes.

Andrew Morton announced Linux 2.6.5-mm4 (the -mm originally came from the fact that Andrew was working on memory management; it was convenient to keep the name after taking over 2.6 maintenance). He said:

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.5/2.6.5-mm4/

Added the DRM development tree to -mm kernel. Please Cc dri-devel@lists.sourceforge.net on any bug reports.
Mainly small fixes

There were various bug reports, including one from Martin Hermanowski, who said, "I get an oops when I try to unload the hci_usb module." Greg KH replied, "I'm hating that driver right now... There are a number of pending bluetooth patches for that driver that fix a number of different bugs, so I'm leary of trying to see if this is a different one or not at this point in time. Care to apply all of the bluetooth patches and if this still happens, can you report it to the linux-usb-devel and bluez-devel mailing lists?" Marcel Holtmann asked which patches Greg meant, saying, "There is one from Alan in 2.6.5-mh3 that should fix this problem and I already sent it along with my other fixes (not USB related) to Dave." Greg confirmed that that was the patch he'd meant.

Here's a driver for the adm8211 802.11b chipset.

http://aluminum.sourmilk.net/adm8211/adm8211-20040411.tar.bz2

It's only for 2.6.x kernels, and it's not quite finished yet. It should mostly work for infrastructure and monitor modes, but adhoc and WEP don't work yet. Should work for the average wireless network though.

Thanks to Jouni Malinen for starting this driver, Jerritt Collord for telling me about his driver, and David Young for his netbsd driver I used as a reference.

Matt Mackall said, "ketchup is a script that automatically patches between kernel versions, downloading and caching patches as needed, and automatically determining the latest versions of several trees." He went on:

New in this version by popular demand:

name change kpatchup -> ketchup
automatic signature verification
works with older versions of Python

Currently it defaults to trying to use gpg, which means you must have gpg installed and the appropriate keys in your keyring. You can disable this with the -g or --no-gpg options.

Available at:

http://selenic.com/ketchup/ketchup-0.5

Zwane Mwaikambo was happy to see this, and asked for an additional feature of "support for untarring base trees too so all you'd specify is ketchup 2.6.6 and it'd look for a base tree in $KETCHUP_ARCH if the current working directory isn't a kernel tree."

Gael Le Mignot asked if the linux-wlan-ng (http://www.linux-wlan.org/) driver would make it into the 2.6 kernel. Greg KH replied that AbsoluteValue Systems, the author, had not submitted the code yet for inclusion. Jeff Garzik confirmed that this was true as far as he knew; but he added:

regardless...

There is a central problem of multiple 802.11 stacks that I very much want to avoid. linux-wlan-ng, HostAP, madwifi, and one or two others.

I had hoped that HostAP would be submitted, and that could form the basis of a common 802.11 stack, but I haven't heard anything about that in many weeks.

And Jouni Malinen replied, "It will be.. WPA2 distracted me from this preparation, but I will try to get back to cleaning up the Host AP driver (http://hostap.epitest.fi/) for 2.6. Actually, I might consider submitting it first without CCMP (AES-counter with CBC-MAC), because that is the part that is likely to take most time to convert for crypto API and this can be easily added separately."

Marcelo Tosatti announced 2.4.26-rc3, saying, "It includes few important USB fixes, JFS fixes, amongst others." Geert Uytterhoeven reported an ongoing problem, not solved in 2.4.26-rc3, in which he got warnings of uninitialized variables when compiling JFS into the kernel, using m68k-linux-gcc 2.95.2 and 3.2. There was no discussion about it though.

As part of the ongoing big push to get everything working with SysFS, Michael Hunold had been hard at work adding SysFS support to the DVB Subsystem. However, he wasn't sure he was on the right track. He'd managed to create a /sys/class/dvb/ directory, but he was having trouble creating the subdirectories he wanted under that; he asked for some pointers to documentation. Stephen Hemminger explained, "The sysfs directory layout is a logical representation of the underlying data structures in the kernel. Each directory (with exception of attribute groups) has a 1:1 relation ship with a kobject." Greg KH added:

you can't create subdirectories directly by just adding a '/' to the name of the class device, sorry. You will have to create a kobject for the directory, and create the attributes in that kobject, like networking did.

Yeah, it's a bit of a pain, but creating subdirectories in an easy manner is on the TODO list for the driver core for 2.7.

If you want to get up and running quickly, which would not require you to fix up any lifetime rules for your dvb drivers, you could implement the class_simple interface, like a lot of other driver subsystems currently are doing, and then in 2.7 convert over to a proper driver model conversion. I say this as I am only guessing as to what your lifetime rules are for your dvb devices and drivers...

The various BK machines we host are in at their new location and seem to have stabilized (famous last words, every time I say something like that something crashes). kernel.bkbits.net, which is a shell account machine for kernel developers, bk users or not, is not up yet, we've had problems with that machine due to cooling and because we have a much larger machine room we will be moving the guts of that machine to a higher volume box to get better cooling. ETA on that should be today.

We're moving towards remote control of the critical machines. The machines are already UPSed but they don't have serial consoles and/or remote power. I've ordered a Sentry R2000 (model 203-0-1) which is supposed to be a pretty nice remote power device. We'll get serial consoles on the machines so that if they crash we can get the oops message and get some debugging help (a couple of people, Bill Irwin in particular, went out of their way to offer to help out with the the debugging. I mention it only because it was nice to that kind of friendly offer and I'm grateful).

We are currently on one T1 line, we moved one of the two to the new office and one is at the old office so we had some overlap in case something went horribly wrong. It's clear that one T1 line isn't enough, you've been keeping that T1 pretty full. We'll be back up to two in a few weeks.

Other than the up/down problems, have any of you noticed bkbits being slower since the move (which happened Friday or Saturday, it's sort of a blur)?

Oregon State's open source lab has repeated their offer to take over hosting bkbits.net and we may take them up on that if we can ever work out the logistics and you need the bandwidth.

Sam Ravnborg reported no problems, although earlier he'd been unable to pull from linux.bkbits.net. Larry said this had been part of the DNS problems that were all fixed; and Andy Isaacson added, "Both problems due to an internal network error, which should be fixed now. Thanks for the report; please let us know if you see any further problems."

Here are some USB patches for 2.6.5. Almost all of these have been in the -mm tree for a number of weeks now (the exceptions being the last 4 patches, to the speedtouch driver, and a security fix which has been in the WOLK kernel for a while.)

Here are the main types of changes:

bugfixes. Lots of these that have been reported numerous times on lkml.
uhci fixes
gadget driver updates and additions
new device ids
lots of misc small cleanups

Please pull from: bk://linuxusb.bkbits.net/usb-2.6

Patches will be posted to linux-usb-devel as a follow-up thread for those who want to see them.

Here are some i2c driver patches 2.6.5. They have all been in the -mm kernels for a number of weeks now and do the following:

add a new bus i2c driver
add 2 new chip drivers
updated documentation
lots of bug fixes

Please pull from: bk://linuxusb.bkbits.net/i2c-2.6

This is a new driver for the Acard/Artop PCI ATA/SATA cards (6885[LP]/6896[S]) based on the ATP867{A,B} chips. It supports connectivity only, no fancy ATP867B RAID stuff, for up to 8 drives. The first version of this patch [written for 2.4.18] has been tested on about a dozen x86 SMP machines with several kinds of hard drives, in PIO4 and UDMA5, as well as in a few other configurations. This 2.4.26 port has had minimal testing, but has also had minimal changes from the original.

Acard tell me that they may have seen issues running ATAPI devices using this driver, but that they can't be sure that they weren't seeing problems with higher-level software. I've given it a quick test with my DVD drive, and didn't see anything obvious. YMMV.

Feel free to send me comments directly, but I'll also try to check on the list via archives. Stay tuned for the 2.6.5 port.

1.	3 Apr 2004 - 9 Apr 2004	(84 posts)	In-Kernel LISP Interpreter For Implementing Dynamic Security Policy
2.	6 Apr 2004 - 10 Apr 2004	(32 posts)	Bypassing The Buffer Cache
3.	6 Apr 2004 - 9 Apr 2004	(11 posts)	Linux 2.6.5-mm2 Released; Some Broken Cleanup Removed
4.	8 Apr 2004 - 11 Apr 2004	(12 posts)	Exposing ACPI API To User-Space Via SysFS
5.	8 Apr 2004 - 9 Apr 2004	(4 posts)	Linux 2.6.5-mm3
6.	10 Apr 2004	(1 post)	Linux Release Candidate 2.6.5-mc4 Released
7.	10 Apr 2004 - 13 Apr 2004	(26 posts)	Linux 2.6.5-mm4 Released; Problems Unloading hci_usb Module
8.	10 Apr 2004	(1 post)	adm8211 802.11b Driver Released For 2.6
9.	11 Apr 2004	(2 posts)	ketchup Kernel Patching Script Version 0.5 Released
10.	11 Apr 2004 - 12 Apr 2004	(5 posts)	Status Of Unified 802.11b Stack In 2.6
11.	12 Apr 2004 - 13 Apr 2004	(2 posts)	Linux 2.4.26-rc3 Released; Some JFS Compile-Time Warnings
12.	12 Apr 2004	(3 posts)	Adding SysFS Support To The DVB Subsystem
13.	13 Apr 2004	(4 posts)	BitMover Servers Back Up
14.	14 Apr 2004	(1 post)	kexec Updates For 2.6.4 And 2.6.5
15.	14 Apr 2004	(1 post)	USB Updates For 2.6.5
16.	14 Apr 2004	(31 posts)	I2C Update For 2.6.5
17.	14 Apr 2004	(1 post)	ATP867X PCI IDE Driver Released

Kernel Traffic #261 For 9 Jun 2004

By Zack Brown