Kernel Traffic #197 For 23 Dec 2002

There were 510 different contributors. 253 posted more than once. 182 posted last week too.

now that all commercial vendors ship a backport of Ingo's O(1) scheduler external projects like XFS have to track those projects in addition to the mainline kernel.

Having the common new APIs available in mainline would be a very good thing for those projects. We already have a proper yield() in 2.4.20, but the set_cpus_allowed() API as used e.g. for kernelthreads bound to CPUs is still missing.

Any chance you could apply Robert Love's patch to add it for 2.4.21? Note that it does not change any existing code but just adds that interface.

Adding to that, it is also used for backporting Ingo's workqueue stuff, which is useful and completely separate from the O(1) scheduler.

I plan on using workqueues for moving some drivers' duties to process context where it really belongs [which in turn fixes bugs].

Christoph said he was working on the workqueue patch, and Robert Love mentioned that the workqueue code was the reason he did the set_cpus_allowed() work in the first place.

Martin J. Bligh also responded to Christoph's initial post, asking of the O(1) scheduler, "If every distro has it, and 2.5 has it, and it's been around for this long, I think that proves it stable. Marcelo, what are the chances of getting this merged into mainline in the 2.4.20 timeframe?" But Christoph replied that Ingo himself had vetoed it (Alan Cox raised an eyebrow at he suggestion that Ingo had veto power over kernel features, but Christoph reminded him that Ingo was the author of the code in question). And Robert remarked, "Ingo did explicitly mention he thought the O(1) scheduler was not 2.4 material. Whether this has changed, e.g. due to stabilization of the scheduler, I do not know. But I do recall he had an opinion in the past." And Bill Davidsen added, "I have exchanged Email with him explaining why I feel it's highly desirable on news servers, and I sent him some metrics with and without. I had the impression he would reconsider the issue in the future. Note that means "think about it again" rather than any implied change in his conclusion."

Patrick Mochel seemed to remember that the /proc/pci interface had been deprecated in the past, although he wasn't sure. He suggested that the lspci program was a superior way to access the same information, and that /proc/pci should be deprecated if it wasn't already. This, he felt, would allow some code to be removed, and remaining code simplified. He asked if any userspace programs still relied on /proc/pci. Jeff Garzik replied:

Historically, this was a Linus call :)

IIRC it was one of (a) deprecated, (b) removed, or (c) almost removed in the past, and Linus un-deprecated it. The logic back then was that it provides a quick summary of a lot of useful info, a la /proc/cpuinfo and /proc/meminfo. i.e. you don't need lspci installed, just been /bin/cat.

Personally, I think it would be nice to eliminate /proc/pci -- in favor of something that provides similar functionality from sysfs: "cat /sys/all-busses" or somesuch. I dunno how feasible that is. The main idea is to list as many attached devices as possible in one go, without having to cat 40 different files :) [unfortunately I think this means I am disagreeing with you ;)]

I do grant you it would make various __init sections and in-memory structures smaller if we eliminated the names... do we want to? Sure we have lseisa and lspci and lsusb, et. al. Does that obviate the need for a simple summary of attached hardware?

Patrick replied that /proc/pci was hardly ever used, and that since the information was available to user space, it was logical to have a user-space tool to read it. Kai Henningsen replied, "I seriously distrust tool-based solutions for this. It is far too likely that any new tool will be missing when I most need it, and old tools already aren't a particularly convincing solution. I don't want to grovel through 7868 tools just as I don't want to grovel through 7868 files. And the more those are, the less likely I am to even have heard of them - from the lsXXX tools, the only one I knew about was lspci, and lsusb (which seems to be the other one actually installed) seems to be a lot more reluctant to give useful info to the non-USB- expert." Krzysztof Halasa remarked, "Anyway, /proc/pci is currently broken (the kernel don't know what hot-pluggable devices will you use, and doesn't preserve the necessary names)." He suggested that if the kernel was going to export a /proc interface, that interface should at least be correct.

Elsewhere, Petr Vandrovec pointed out that /proc/pci was very useful during installation, when lspci was not yet installed. He added, "Next problem is that some drivers want to print "user readable" hardware name to user, and although some have its own name database (e100), some use name from pcidev..." Erik Hensema objected, "Every half-decent installer autodetects all PCI devices. AND had lspci installed in the install image." To which Dr. David Alan Gilbert replied, "Yes, but wait till you find yourself stuck on a weird embedded board with a small flash and a serial console and you are trying to debug the PCI device you've built."

One thing that /proc/pci gives you that 'lspci' historically didn't was the correct interrupt setup (because kernel irq routing has nothing to do with the PCI irq config byte on most "interesting" machines).

I don't know if lspci gets that right these days, and the information does exist in /sys, so there is certainly at least the _potential_ of dropping /proc/pci.

Patrick tested lspci, and reported that it did seem to get the interrupt setup right. He posted a patch to make /proc/pci a compile-time option. There followed a bit of technical discussion of where lspci got its data, and other things, and at one point Linus added, "I think we should" [...] "make sure that /sbin/lspci (or some other tool) can be made to easily show either the kernel irq mapping value _or_ the "original PCI config space" value. At that point I'd agree that /proc/pci has outlived its usefulness." There followed more implementation discussion, and eventually the thread petered out.

Milan Roubal noticed that there was a maximum of 10 IDE devices allowed on his 2.4 system, and wanted to raise that limit. He tried writing his own patch, would managed to extend the number to 12, but he wanted to go as high as 20, or ideally 32, which he felt would be sufficient for anybody's future needs. Alan Cox suggested a method of adding 'a-f' to the existing '0-9' IDE names. This would yield a few extra values, and Petr Sebor provided a patch to provide idea - idez (or 26 extra values), but Henning P. Schmiedehausen pointed out that this would only defer the problem to a future time. John Bradford pointed out that 36 total IDE devices should be enough for anyone. And Alan pointed out that, as a minimal change for the 2.4 stable series, Petr's patch was sufficient, and that there would be problems with running out of minor numbers anyway, if a given system needed many more IDE devices. He seemed to feel that the next unstable series would be a better place to address the problem seriously.

9 Dec 2002 - 14 Dec 2002 (11 posts) Subject: "[BK fbdev] Yet again more fbdev updates."

James Simmons announced to Linus Torvalds with joy, "Here are the fbdev updates people have so been waiting for. Several drivers have been ported. Many fixes have been implemented and many nice features have been added as well. Please pull the changes. They have been tested by people on this list. Thank you." Christoph Hellwig asked Linus to please pull James' changes, saying, "He has been sending fbdev updates that fix known issues with many drivers for a long time but I can't even remember when you merged it the last time. Most fbdev drivers are pretty unusable in mainline without his fixes." Linus did not reply to this directly, but he did complain that the framebuffer changes seemed to have broken his VGA console. He posted some symptoms, and he and James went back-and-forth on it for awhile, and James eventually thought he'd tracked the problem down; but there was no positive confirmation on the list.

Ok, big patch, but this is mostly all over the place with a lot of fairly random small fixes (a lot of compile fixes for the build breakage from the header file cleanups, for example).

The AGP reorg, fbdev merge, and the s390 updates also help make the patch quite large.

Architecture merges, more merges with Andrew, and Al started cleaning up some of his favourite code - devfs. See details in the changelog.

PS. I'm going to be traveling for three days, so..

Allan Duncan pointed out that the framebuffer code was a bit broken, and James Simmons said, "The matrox driver hasn't be ported yet. About 1/2 are now ported to the final api. Over the following week I will porting a bunch of new drivers. This is the final changes in the api so drivers can now be ported!!!! If you need help porting them email me and I'm here to help." And Petr Vandrovec was thrilled to see the framebuffer patches finally accepted in the main kernel tree.

So here goes the first pre of 2.4.21 including the new IDE code merged from Alan's tree.

Test it carefully, since the new IDE code is not yet fully tested.

Do not use it with critical data.

Nathaniel Russell announced, "This patch adds support for the Via VT8633 AGP Bridge. Also this patch has been tested with a variety of GL apps including the GL ScreenSavers. The patch includes the via_generic_setup routines and Device ID's. Please Apply against current 2.4.x Kernel" Marcelo Tosatti replied, "Applied, thanks."

Garst R. Reese asked if ROMFS was currently maintained, and Miles Bader pointed him to http://romfs.sourceforge.net. Garst said he tried that, but the contact info at that site seemed dead. Miles replied, "Hmmm, you might try asking the debian genromfs maintainer; I seem to recall him being in contact with the author before: Juan Cespedes <cespedes@debian.org>" End of thread.

Paul Mackerras offered, "This patch fixes the offb driver, and fixes the Makefile so it doesn't go looking for a nonexistent file (cfbimgblit.o) when CONFIG_FB_OF is turned on. In the offb driver, we were overrunning the end of the info->fix.id field on some cards, plus it had an undefined variable. I guess we could also get rid of the "info->node = NODEV;" line too." James Simmons replied, "All the patches have been applied :-)"

Still no resolution to the two competing procps development projects. Albert D. Cahalan announced:

This release includes user selection in top, the sysctl -e option needed to support the Red Hat 8.0 boot scripts, and the use of /proc/*/wchan on recent 2.5.xx kernels.

For those of you still upgrading from procps 2.0.xx releases, you can expect:

top can sort by any column (old sort keys available too)
top can select a single user to display
top can be put in multi-window mode and/or color mode
vmstat has the -s option, as found on UNIX and BSD systems
vmstat has the -f option, as found on UNIX and BSD systems
watch doesn't eat the first blank line by mistake
vmstat uses a fast O(1) algorithm on 2.5.xx kernels
pmap command is SunOS-compatible
vmstat shows IO-wait time
pgrep and pkill can find the oldest matching process
sysctl handles the Linux 2.5.xx VLAN interfaces
top shows IO-wait time if-and-only-if your kernel computes it
most programs 30% to 300% faster (tested on a 450 MHz MPC7400)
ps has a new "-F" format (very nice, like DYNIX/ptx has)
ps with proper BSD process selection
better handling of very long uptimes

There's a procps-feedback@lists.sf.net mailing list you can use for feature requests, bug reports, and so on. Use it! Feedback makes things happen.

Here's a patch against 2.5.51 with a updated dmfs. It contains the two patches at: http://people.sistina.com/~thornber/patches/2.5-unstable/2.5.50/2.5.50-dmfs-1/00012.patch and http://people.sistina.com/~thornber/patches/2.5-unstable/2.5.50/2.5.50-dmfs-1/00013.patch with the following modifications:

fixed locking type mismatch (trying to call spin_lock on a rw_semaphore).
fixed compile time warnings with the dbg() macro (something better should be used here, I just commented it out...)
changed the dev file to print out the kdev value, not be the actual block device.

With regards to the last change, I didn't follow the way the other files operate with their complex page creation structure, as this is only a simple one line file. If the lvm developers want me to change this, I will. If not, I would argue that a number of the other files created should be changed to use this simpler format. Or is there some reason for creating these lists of pages that I'm missing?

He and Joe Thornber discussed implementation for awhile, and eventually the thread wound down.

Fault-Injection Test Harness Project

I am pleased to announce the formation of a new project for developing a test harness for inserting faults into a running kernel. The project is based at http://fault-injection.sf.net, with a bitkeeper tree hosted at http://fault-injection.bkbits.net:8080/linux-2.5/, and an IRC channel named #fi on 206.103.61.170. (The DNS entry for the IRC address is about to change, but the number should stay the same.)

The project started out to try to validate that a kernel driver was acceptable for a carrier environment (the "mystical" hardened driver) :). Now, it has morphed to building a general tool for inserting faults into any part of the kernel to see if the kernel reacts in a way the test developer expects. We have kind of straw man design with some _very_ early prototype work, but for the most part things are just now getting started. I know there are some people on this list that have some considerable experience in fault injection on other Unix's, and some of you have hinted to me in emails that you might be interested in creating some kind of fault injection tool for Linux. I just hope I can entice you into joining our efforts (even if you only want to give us some directional guidance.)

As mentioned we are developing off of a clone of the 2.5 tree that we periodically sync. A CVS tree and snapshots are available from the sourceforge site for those that do not use bitkeeper, but they will always lag behind.

Yasunori Goto was very interested in the project, and wanted to help out, but there was no reply on-list.

New versions of the ACPI patches are available from http://sf.net/projects/acpi . Non-Linux source drops will not be available until Dec 20, due to web publishing issues. (I can email it on request, if you can't wait.)

(If you already downloaded the patch against 2.5.51, you may want to re-get it. A small fix for writes to the /proc interface has been slipstreamed.)

Various things here. Most noticeably more merges with Andrew, with a lot of various small fixes.

XFS, JFS, ACPI and USB updates. KConfig update, and Rusty's module parameter implementation. And fix the stupid nanosleep() thing that broke some programs.

I'm pushing this out, as I've tried to sync up the stuff I got while I was away this week (hint hint: if it ain't here, it's not in my in-queue, and you should re-send).

In the course of discussion, it came up that Ben Collins' patch for that kernel may have included older code that was the result of improperly tracking multiple kernel trees. Ben complained, "Trying to track two seperate source tree's isn't as easy as you might think." And added, "I go through a lot of effort to merge in changes sent to Linus' tree into the Linux1394 repo." Linus replied:

It's a lot easier if you track them _often_ instead of just occasionally. That's the main problem I have with other peoples CVS trees - CVS has very little support for tracking any outside sources, and that coupled with the fact that people don't track it in a timely manner always generates problems.

With CVS, a simple script like

get current version
diff against the last version you did the merge against
apply the diff to your new tree
_then_ do the diff against the current version
delete "last version merged", make current version that.

will work pretty easily most of the time for subsystems that don't get a lot of input from outside the "maintainer". Especially if you do it reasonably often (you can do the back-merge even when you're _not_ ready to actually send me your stuff), the diff from my tree is often quite small and thus easily mergible.

If you think that "maintainer" means that nobody else can touch the tree and that you thus don't need to care, you're WRONG.

Alternatively, never EVER make a patch against the "current kernel version". Only make a patch against the _last_ kernel that you merged with, and if I cannot apply it I will tell you so. Making a patch just between your tree and mine will _always_ end up losing fixes.

Ben replied, "I wasn't pushing off blame, just noting that it's not possible to never make mistakes. You make them too." He said, "I never said" [that "maintainer" meant no one else could touch the tree.] "What I said was that because I can't spend lots of time tracking changes, _sometimes_ I miss them. You will see in the SVN repo logs a lot of places where I merge in changes from your tree. It's a fact that people make mistakes. I've already rectified this one by adding in the patch to the linux1394 repo."

Linux 2.2.24-rc1

Fix a typo in the maintainers (James Morris)
Dave Niemi has moved (Dave Niemi)
Fix incorrect blocking on nonblock pipe (Pete Benie)
Fix misidentification of some AMD processors (Bruce Robson)
Fix a very obscure skb_realloc_headroom bug (James Morris)
Fix warning in lance driver (Thomas Cort)
Fix sign handling bug in pms driver (Silvio Cesare)
Drop mmap on /proc/<pid>/mem as 2.4/2.5 did (also fixes some bugs) (Michal Zalewski)

Available at https://sourceforge.net/projects/e1000.

The full title is:

Intel 8255x 10/100 Mbps Ethernet Controller Family
Open Source Software Developer Manual
Revision 1.0

The manual is intended to support the maintenance of the e100 driver (or the best driver for the PRO/100 networking hardware ;-). The manual covers the 82557, 82558, 82559, 82550, and 82551 Ethernet controllers.

I would like to thank Jeff Garzik for encouraging this publication, and for having the patience to deal with the Intel machine.

I would also like to thank the Intel editors and reviewers: Carolyn Abrigana, Larry Bates, Julie Donnelly, John Ronciak, Wen-Hwa Tao, Eli Kupermann, David Valdez, Colleen Culbertson, and especially Glenn Begis for not giving up.

There was a general round of applause from the kernel developers, along with requests for more documentation, such as for the crypto hardware residing on the PRO/100 S, and a developer manual for the e1000. Scott said there were no specific plans for either of these, but the the e1000 manual was a desired goal for the future.

me publicly thank Intel NIC team [and related persons] for some great work. You guys have really been responsive both to technical issues and also political/community-related issues like this one of opening docs.

This is a situation where I really think that opening docs will lead to a better overall user experience with Intel hardware on Linux, and hopefully serves as a model for other vendors to follow.

Thanks and kudos, Intel!


Kernel Traffic Latest \| Archives \| People \| Topics	Wine Latest \| Archives \| People \| Topics	GNUe Latest \| Archives \| People \| Topics
Czech

1.	2 Dec 2002 - 13 Dec 2002	(35 posts)	Status Of O(1) Scheduler And Related Patches In 2.4
2.	6 Dec 2002 - 10 Dec 2002	(39 posts)	/proc/pci Versus lspci
3.	6 Dec 2002 - 13 Dec 2002	(22 posts)	More Than 10 IDE Devices On 2.4 Systems
4.	9 Dec 2002 - 14 Dec 2002	(11 posts)	Status Of Framebuffer Updates
5.	9 Dec 2002 - 12 Dec 2002	(22 posts)	Linux 2.5.51 Released
6.	10 Dec 2002 - 17 Dec 2002	(22 posts)	Linux 2.4.21-pre1 Released
7.	10 Dec 2002 - 13 Dec 2002	(3 posts)	Status Of Via VT8633 AGP Bridge Support In 2.4
8.	11 Dec 2002 - 12 Dec 2002	(4 posts)	ROMFS Maintainership
9.	11 Dec 2002 - 12 Dec 2002	(2 posts)	Fix For offb Driver
10.	12 Dec 2002	(1 post)	Procps 3.1.3 Released
11.	12 Dec 2002 - 13 Dec 2002	(7 posts)	Status Of dmfs In 2.5
12.	12 Dec 2002	(2 posts)	New Fault-Injection Test Harness Project
13.	13 Dec 2002	(1 post)	ACPI Updated
14.	15 Dec 2002 - 16 Dec 2002	(12 posts)	Kernel 2.5.52 Released; Tracking Multiple Trees
15.	16 Dec 2002 - 19 Dec 2002	(7 posts)	Linux 2.2.24-rc1 Released
16.	17 Dec 2002 - 18 Dec 2002	(8 posts)	Intel PRO/100 Software Developer Manual Released

Kernel Traffic #197 For 23 Dec 2002

By Zack Brown