Kernel Traffic #115 For 23 Apr 2001

By Zack Brown

linux-kernel FAQ (http://www.tux.org/lkml/) | subscribe to linux-kernel (http://www.tux.org/lkml/#s3-1) | linux-kernel Archives (http://www.uwsg.indiana.edu/hypermail/linux/kernel/index.html) | kernelnotes.org (http://www.kernelnotes.org/) | LxR Kernel Source Browser (http://lxr.linux.no/) | All Kernels (http://www.memalpha.cx/Linux/Kernel/) | Kernel Ports (http://perso.wanadoo.es/xose/linux/linux_ports.html) | Kernel Docs (http://jungla.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html) | Gary's Encyclopedia: Linux Kernel (http://members.aa.net/~swear/pedia/kernel.html) | #kernelnewbies (http://kernelnewbies.org/)

Table Of Contents

Introduction

Checking over the KC Author's Page (http://kt.zork.net/authors.html) , I see that this issue marks my 2000th thread summary, counting summaries written for other Cousins.

Mailing List Stats For This Week

We looked at 1261 posts in 5208K.

There were 459 different contributors. 208 posted more than once. 199 posted last week too.

The top posters of the week were:

 

1. linux-kernel Spam Filter Debate
3 Apr 2001 - 12 Apr 2001 (46 posts) Archive Link: "goodbye"
Topics: Spam, Version Control, Virtual Memory
People: Matti AarnioGraham MurrayAaron LehmannOlaf TitzJoseph CarterRik van RielAlan CoxDavid S. MillerRalf BaechleMichael Peddemors

Rik van Riel announced that he was leaving the linux-kernel mailing list because David S. Miller and Matti Aarnio used the MAPS DUL list to prevent dial-up users from sending mail directly to the list. Michael Peddemors replied that it would be a shame if Rik left, and added that he felt linux-kernel should be as open as possible, to encourage the participation from anyone on earth. Matti replied, "I just verified this particular aspect of VGER's MTA configurations. It has been unmodified since 21-Mar-2000, that is, over a year..." He speculated that Rik's ISP might have added their dialup pools to the DUL registry, and explained, "The incentive behind the DUL is to force users not to post straight out to the world, but to use their ISP's servers for outbound email --- normal M$ users do that, after all. Only spammers - and UNIX powerusers - want to post directly to the world from dialups. And UNIX powerusers should know better, and be able to use ISP relay service anyway." A lot of people replied to this.

Ralf Baechle said that his ISP didn't even allow users to send mail via the ISP's servers, but actually forced users to send mail directly from their system. Elsewhere, Graham Murray asked Matti, "On the subject of vger configuration, the FAQ states that vger "will" start using ECN as of 22 Feb 2001. This does not seem to have happened yet. Has this change been cancelled or merely postponed?" There was no reply, but elsewhere, Aaron Lehmann remarked, "It scares me that peoples' messages would be denied based on what degree of connection they choose to mail via. I sincerely hope that the DUL lists only list netblocks that are actively being used for spam. This would be sort of like the Usenet Death Penalty, instating bans on providers who refuse to deal with spamming. I think that's a lot more acceptable than shutting everyone who happens to connect to the internet in a certain way from sending mail directly out of their local machines." But Olaf Titz replied, "The DUL is only and explicitly for the purpose of denying access based on the degree of connection the users can afford." Elsewhere, Joseph Carter protested, "My beef is and always has been with the DUL specifically. I have no issue with the RBL or RSS lists. ORBS ... well, they called one of my old ISPs' mail an open relay when it wasn't and took 3 months to decide to rectify the situation and remove us from their list. That doesn't instill much confidence. The DUL however is blatant discrimination based on connection class rather than any real evidence that spammers are at all affected by it. In fact, DUL users have no idea if the mail they block is spam or not."

Elsewhere, Rik replied to Matti:

UNIX power users do know better. They know their ISPs mail server could show up in RSS or ORBS any moment. Therefor they use their own machines for sending out email.

IMHO DUL is an unethical list to use because it assumes guilty by default. This is different from all other spam blocking lists, which only block hosts _after_ they've found something wrong with them.

The other exception is untestable-netblocks.orbs.org, which blocks everything it cannot test and is just as bad as DUL.

Anyway, since linux-kernel has chosen to not receive email from me I won't bother answering VM bugreports or anything here. It's up to Matti and Davem to decide how useful a place linux-kernel will be.

Alan Cox sniped, "Thats ok. Andrea will I am sure be happy to take over as maintainer" [of the VM subsystem]. David also replied to Rik, pointing out that if linux-kernel was truly blocking his mail, "Funny how this posting went through then..." He went on, "If it is specifically when you are sending mail from some other place, state so, don't make blanket statements which obviously are not wholly true." Rik replied, "I'm temporarily using my ISP's smarthost. However, I'll turn this off again soon because I'd rather stop with linux-kernel then give in to the guilty-by-default attitude that comes with DUL."

Matti replied that David had told him to remove DUL, and that he had done so. He went on, "VGER uses now RBL and RSS, no others. In few weeks time, I probably implement EXIMish "warn" feature." Rik replied:

Thanks !

To come back to the spamfilter promise I made some time ago, people can now get a CVS tree with spam regular expressions and a script to generate a majordomo.cf from it ...

Scripts to automatically generate configuration for other mailing list and mail delivery programs are very much welcome, as are people willing to help maintain the set of regexps.

cvs -d :pserver:cvs@nl.linux.org:/home/CVS login
password: cvs
cvs -d :pserver:cvs@nl.linux.org:/home/CVS checkout spamfilter

The (majordomo-run) mailing list spamfilter@nl.linux.org will be used for CVS updates and possibly discussion about this thing. I'm already using a procmail rule to automatically do a rebuild of NL.linux.org's majordomo.cf whenever something is changed to the CVS tree...

Joseph also thanked Matti, adding that he doubted anyone on linux-kernel would object to either RBL or RSS. Rik added, "It might even be good to add inputs.orbs.org to vger. This list is basically the same as RSS, except that sites can get on and off faster (RSS is sometimes slow in adding sites to the list and it is at times also slow in removing sites that have already been fixed)."

 

2. Status Of xircom_cb Driver
5 Apr 2001 - 13 Apr 2001 (4 posts) Archive Link: "2.4.3-ac3 XIRCOM_CB only working as module"
People: Arjan van de VenStephen Torri

Alessandro Suardi reported that the new xircom_cb driver would only work when loaded as a module, not when compiled into the kernel directly. Arjan van de Ven replied, "I haven't tried this; I'll look into this once I get back to work (eg where my cardbus machine is). I would not be surprised if it turns out to be of the "yenta_socket is not initialized yet, so the card is invisible at pci scan time" type...." Stephen Torri added his experience, "I found that the xircom_cb driver works fine as a module except for when using dhcp for dynamic ip address. If I set the ip address of the network card to a static ip number it work fine. Yet if I choose to use the dhcp to get my ip address packets leave, as seen via tcpdump, but it doesn't get an address. I watched tcpdump at the dhcp server and saw the packets come in from the laptop and the responses go out. The laptop either is ignoring the message coming from the dhcp server or some other bizarre reason." There was no reply.

 

3. Problems Developing SuperTrak And FastTrak Drivers
9 Apr 2001 - 12 Apr 2001 (8 posts) Archive Link: "i2o & Promise SuperTrak100"
Topics: Disk Arrays: RAID, I2O
People: Jens AxboeAlan CoxAndrew ChanAndrew Morton

Joao Paulo Martins had no luck getting a Promise SuperTrak100 RAID controller working on his dual PIII with kernel 2.4.3, and he found an exact description of his problem already reported on linux-kernel in the archives (http://www.uwsg.iu.edu/hypermail/linux/kernel/0102.3/0037.html) . He said that Alan Cox and Andrew Morton's comments in that thread had helped him, but that now he still got I/O errors when trying to make a filesystem, although he had no trouble starting the modules or creating the partitions. Jens Axboe replied that he'd recently asked Promise to give him a sample SuperTrak so he could get it working under Linux, but they'd refused. He suggested that Joao "bother Promise and ask what they intend to do about it." Alan Cox replied, "I've been talking constructively to promise about the i2o on the supertrak not working straight off with the kernel i2o driver. Currently it looks promising." Various folks were happy to see Alan on the job, and Andrew Chan added, "While you are at it, can you bug them for FastTrak (the software RAID thing) support? Their drivers are ages behind and Andre doesn't seem to be able to get them cooperate either." Alan replied that he'd talk to them about that at some point.

Finally, Alan concluded, "They seem very keen not to co-operate on the fasttrak so its probably best to avoid motherboards that have the fasttrak chipset and tell the board vendor why."

 

4. Benchmark Dispute
9 Apr 2001 - 17 Apr 2001 (11 posts) Archive Link: "[RFC][PATCH] Scalable FD Management using Read-Copy-Update"
Topics: SMP
People: Maneesh SoniMark HahnDipankar SarmaJohn Hawkes

Maneesh Soni announced:

This patch provides a very good performance improvement in file descriptor management for SMP linux kernel on a 4-way machine with the expectation of even higher gains on higher end machines. The patch uses the read-copy-update mechanism for Linux, published earlier at the sourceforge site under Linux Scalablity Effort project.
http://lse.sourceforge.net/locking/rclock.html.

In SMP kernel the performance is limited due to reader-writer lock taken during various calls using files_struct. Majority being in the routine fget(). Though there is no severe contention for files->file_lock as the files_struct is a per task data structure but enough performance penalties have to be paid while even acquairing the read lock due to the bouncing lock cache line when multiple clones share the same files_struct. This was pointed out by John Hawkes in his posting to lse-tech mailing list
http://marc.theaimsgroup.com/?l=lse-tech&m=98235007317770&w=

The improvement in performance while runnig "chat" benchmark (from http://lbs.sourceforge.net/) is about 30% in average throughput. For both configurations the results are compared for base kernel(2.4.2) and base kernel with files_struct_rcu-2.4.2-0.1.patch. Profiling results were also collected by using SGI's kernprof utility and that shows a considerable decrease in amount of time spent in fget(). The "chat" benchmark was run with rooms=20 and messages=500. For each configuration, the test was run for 50 times and average of throughput result in terms of messages per second was taken.

He posted some benchmark details and pointed out that the results for a 4-CPU system were higher than for a 2-CPU system, which he said indicated the importance of the patch for higher-end SMP systems. Finally, he explained, "The patch is built on 2.4.2 kernel. Before applying this patch, the user has to obviously apply the read-copy-update patch (rclock-2.4.2-0.1.patch) for linux, which can be obtained from http://lse.sourceforge.net/locking/rclock.html. The config options required for this patch are CONFIG_RCLOCK and CONFIG_FD_RCU."

Mark Hahn came down on the patch, saying:

isn't this a solution in search of a problem? does it make sense to redesign parts of the kernel for the sole purpose of making a completely unrealistic benchmark run faster?

(the chat "benchmark" is a simple pingpong load-generator; it is not in the same category as, say, specweb, since it does not do *any* realistic (nonlocal) IO. the numbers "chat" returns are interesting, but not indicative of any problem; perhaps even less than lmbench components.)

Dipankar Sarma returned, ""chat" results for large numbers of CPUs is indicative of a problem - if a large number of threads share the file_struct through CLONE_FILES, the performance of the application will deteriorate beyond 8 CPUs (going by John's numbers). It also indicates how sensitive can performance be to write access of shared-memory locations like spin-waiting locks." He went on, "Irrespective of the usefulness of the "chat" benchmark, it seems that there is a problem of scalability as long as CLONE_FILES is supported. John Hawkes (SGI) posted some nasty numbers on a 32 CPU mips machine in the lse-tech list some time ago." Mark countered, "that's not the point. the point is that this has every sign of being premature optimization. the "chat" benchmark does no work, it only generates load. and yes, indeed, you can cause contention if you apply enough load in the right places. this does NOT indicate that any real apps apply the same load in the same places." The thread ended there.

 

5. Status Of CML2
10 Apr 2001 - 17 Apr 2001 (64 posts) Archive Link: "CML2 1.0.0 release announcement"
Topics: Disks: IDE, Disks: SCSI, Kernel Build System
People: Eric S. RaymondRussell KingAlbert D. CahalanDave JonesChristoph HellwigAlan CoxJeff GarzilMichael Elizabeth ChastainAaron LehmannMarko KreenAnton AltaparmakovWilliam StearnsJamie LokierUrban WidmarkGiacomo CatenazziRandy DunlapKeith OwensJeff GarzikOlaf TitzGary Lawrence Murphy

Eric S. Raymond announced:

After 11 months of painstaking work and testing, CML2 1.0.0 is ready for use, and ready to replace the current kernel-configuration system. You'll find it at <http://www.tuxedo.org/~esr/cml2/>. I've made a transition guide available at <http://www.tuxedo.org/~esr/cml2/transition.html>.

(Why this project at all? For those of you who don't remember, it all started when I realized that building kernels is way too hard. I wanted to simplify the configuration task enough to make configuration accessible to non-gurus. It needs to have more policy options. Rather than hundreds of questions like "Include FOOBAR2317 driver?", the novice should see stuff like "Compile in all modular drivers as modules without prompting?")

On 30 March 2001 at the Kernel Summit, Keith Owens and I worked out a transition schedule with Linus. Keith's rewrite of the makefile system and my configurator tools are officially slated to replace the present system in the 2.5.1 to 2.5.2 timeframe. That, of course, is contingent on us not screwing up :-).

(For those of you who grumbled about adding Python to the build-tools set, Linus has uttered a ukase: CML2's reliance on Python is not an issue. I have promised that once CML2 is incorporated I will actually *reduce* the kernel tree's net dependence on external tools, and I know exactly how to deliver on that promise.)

Linus opted for a sharp cutover rather than a gradual transition -- when the new stuff goes into the tree, CML1 will drop dead immediately. Configuration maintainers should be prepared! Right now, any errors in my translation of the CML1 rules are my problem -- but after Der Tag, they will swiftly become *your* problem. We'll all be happier if you have flamed my butt and helped me fix any problems well in advance of the cutover.

The rules files in this release have been re-checked against the 2.4.3 tree. While I can't guarantee they are completely correct (that's your job), they at least cover every CML1 symbol. I have written coverage tools to check this. There is still a problem with the CRIS port symbols that lack a CONFIG_ prefix, but the CRIS people say they'll fix that in their next update.

The stock CML2 distribution contains an installation script which will drop CML2 in place on a current kernel tree. At most one existing file (your top-level Makefile) will be touched, with new versions of the config/oldconfig/menuconfig/xconfig productions edited in. (The old ones will be kept under variant names, so you can still invoke CML1 if you want.)

I've learned, the hard way, that kernel developers are a conservative bunch. So, to help you all feel better about the change, here are some of the improvements CML2 offers over the existing CML1 configuration system:

Maintainability:

  • Where CML1 had three different interpreters, none perfectly compatible with any of the others, CML2 has *one* rule compiler and *one* rulebase- interpreter front end. This will be good for consistency and economy.
  • As of CML2 1.0.0 and CML1 2.4.3, the combined code and data size in lines of the system (a good indicator of its maintainability) shrank by 38%. For code alone the figure was 43%. Furthermore, where CML1 was written in a weird mix of three languages, CML2 uses exactly one.
  • CML2 decouples the configuration language from the configuration user interface (they communicate with each other only through the compiled rulebase). This means that it will be relatively easy to improve the UI and the language separately.
  • CML2 has a track record. It is already being used in other projects, including most notably the Embedded Debian Project. Adam Lock at Netscape is using it to construct a tool for configuring Mozilla builds.

Programmer Friendliness:

  • The rather spiky and cluttered shell-like syntax of CML1 is replaced with a much simpler and more regular format resembling that of .netrc or .fetchmailrc. More importantly, the semantics of the language are declarative rather than imperative -- a better match for the problem domain, and thus more expressive and easier to code in.
  • CML2 will eliminate (or at least drastically reduce) skew between port configurations. The fact that the top-level CML1 files of the fifteen ports in the kernel tree are separate means there have been plenty of opportunities for the common code in them to drift apart; CML2's design and compilation rules should effectively prevent future bugs of this kind.
  • CML2 query prompts and menu banners are separated from the symbol dependency declarations. Thus CML2 system definitions can be internationalized and localized.
  • CML2 has a complete, explicit description. Syntax, language semantics, and user-interface options are all discussed in detail.

User Experience:

  • In CML2 it is impossible to generate a configuration that is invalid according to the rules file(s). You'll get a popup complaining about the constraint that was violated if you try. (This is effect #1 of having a theorem prover in the configurator.)
  • Whenever CML2 can deduce from a symbol tweak that other changes are required, it does them. And if the change is reversed, so are all those side effects. (This is effect #2 of having a theorem prover in the configurator.) The user interfaces tell you what the side effects are.
  • All three interfaces do progressive disclosure -- the user only sees questions he/she needs to answer (no more hundreds of greyed-out menu entries for irrelevant drivers!).
  • The declarative semantics of CML2 makes it much easier to set up and check options for policy-based configuration. I have done only enough of this in the CML1 translation for demonstration purposes (there are new symbols TUNING, EXPERT and WIZARD that change some visibilities). Once CML2 is in place, it should be a relatively small effort to give the user a rich set of policy and don't-bother-me options.
  • The line-oriented mode of the new configurator is much more powerful than the original Configure. It's possible to move backward or jump around in the configuration sequence.
  • The new curses mode, unlike the old menuconfig code, has full access to the help texts.
  • All URLs in the Tkinter version's help windows are now live; if you click on them they'll launch a browser.
  • Curses and Tk interfaces now use color as a navigational aid -- both in the same way so the user interface is consistent.
  • And a cool penguin logo when we iconify the X version! :-)

Many linux-kernel regulars have helped develop, test, and debug CML2, including Giacomo Catenazzi, David Kamholz, Robert Wittams, Randy Dunlap, Urban Widmark, Andr? Dahlqvist, Drago Goricanec, William Stearns, and Gary Lawrence Murphy.

Russell King remarked that if CML2, along with Keith Owens' Makefile system, were slated to go into the main kernel in the early 2.5 releases, they would have to accommodate the various architecture maintainers. Specifically, he said:

Currently, one of the many things I'm doing is trying to sort out a working kbuild-2.5 for the ARM tree. I have some stuff done, but there are several things that I'm definitely not happy with, and there is currently a whole scoop of stuff which I haven't yet found a way for kbuild-2.5 to handle. Its looking like the ARM trees use of kbuild-2.5 will be even more messy than its use of the current build system.

(We have around 60 machines, which key both which files get built, the text and data address of the running kernel image, the text and data address of the decompressor, and which vmlinux.lds.in file we use to link the kernel. This is currently my biggest problem that kbuild-2.5 doesn't seem to be able to handle at present. Note that it is not acceptable that users should have to type in 5 or so apparantly meaningless hex addresses when they configure the kernel.)

I've yet to hear back from Keith on the issues I have with kbuild-2.5, but I'm hopeful that he has some good suggestions...

There was no reply to this, but elsewhere, Albert D. Cahalan replied to Eric's point that the new system would only show config options the user could actually select, as opposed to xconfig, which gave greyed out entries even if there were no way to enable the option. Albert said, "Well, that sucks. The greyed-out menu entries were the only good thing about xconfig. Such entries provide a clue that you need to enable something else to get the feature you desire. Otherwise you might figure that the feature is missing, or that you have overlooked it." Jamie Lokier agreed, but Eric replied, "You can have this back if you want by clicking the "Unsuppress" item in one of the pulldowns. But since the theorem prover automatically turns on such prerequisites for you, you're very unlikely to need it."

There was no reply to this, but elsewhere, Dave Jones decided to give CML2 a try; and posted his feedback:

One of the first things I noticed was it seems noticably slower than CML1. A make menuconfig in CML1 takes me into the menu in under a second. (On an already compiled tree). CML2 takes around 15 seconds before I get that far. This is on an Athlon 800 w/512MB. I dread to think how this responds on a 486.

Scrolling the cursor bar in menuconfig causes a lot of flickering as the entire screen seems to be redrawn. This is becomes unusable after a few minutes usage. Scrolling under CML1's menuconfig doesn't show this behaviour.

The various colours used to show submenus that have been visited seems confusing, and unnecessary. Their meaning also seems undocumented.

Reporting of changing certain features seems to be excessive. For example, changing CPU target from Pentium Pro to Athlon tells me that "M686=n (deduced from MK7)"

Another confusing thing on this menu, happens when you select CRUSOE, and then 386. "MK7=n (deduced from M386) MCRUSOE=n (deduced from M386)" Not sure why selecting Crusoe enables MK7.

Top level menu seems to have gained a few items. For example, the `SCSI support' item has disappeared, making `SCSI disk support' and `SCSI low-level drivers' both appear on the top level menu.

For some reason, the kernel hacking menu doesn't show 4/5 of the options that it used to. Instead it replaces them with one new one (Disable VHPT). Which it seems to picking up from the IA64 tree. Most strange.

Finally, quitting the program (q twice) gives me this..

python2 -O scripts/configtrans.py -h include/linux/autoconf.h -s .config config.out
Traceback (most recent call last):
File "scripts/configtrans.py", line 104, in ?
sys.stderr.write(args[0]);
TypeError: read-only character buffer, int
make: *** [menuconfig] Error 1

All the above was using an 2.4.3-ac4 tree, with CML2-1.0.0

On the issue of the speed difference between CML2 and CML1, Eric replied that this was a known problem, and it would take a bit of tuning to improve the speed. On the issue of screen flicker while scrolling, Eric couldn't reproduce this; he speculated that Dave might have been using an older ncurses package. On the issue of undocumented coloring of submenus, Eric replied that he'd document them; and added, "They're intended to help you track what portions of the configuration you've already done." On the issue of the SCSI items appearing in the top level menu, Eric replied, "The SCSI support flag is in the buses menu. You see these two menus because the defconfig sets it on." Finally, on the error Dave saw, Eric replied that he couldn't reproduce it. There was no reply to any of these remarks, but elsewhere, Christoph Hellwig mentioned the mconfig package, which he said was plenty fast, and available at ftp://ftp.openlinux.org/pub/people/hch/mconfig/mconfig-0.19-pre1.tar.gz. He added, "Props for all the hard work go to Michael Elizabeth Chastain!" Dave tried it out and confirmed that it was definitely fast, but added that he hadn't heard of it before, and wondered if it solved all the problems CML2 attempted to solve. Christoph replied that it didn't solve everything yet, but that the intention was to eventually make it fully usable for kernel compilation. He listed some TODO items:

  1. It is one programm with multiple frontends:
    old,
    text,
    ncurses,
    random,
    maximum,
    minimum,
    syntax checking
    (X is still missing as my brain is not made for GUI programming)
  2. An 'show me all options and handle the rest' mode is still missing - my devel tree has something like that in the works, but I'll probably never finish it now that CML2 is official.
  3. it still has multiple top-level config.in. Again that is easily fixable and in fact I did a patch for it (including {old,menu,x}config support in 2.3 times but never submitted it.

He asked if he'd missed anything, and Alan Cox pointed out that Christoph's item #3 was not really a TODO item, because "Multiple layers of Config.in is a feature" . However, Eric came back with

I disagree, because I've seen what happens when we go to a single-apex tree. But you could persuade me otherwise. What's your reason for believing this?

The problem with having multiple apices of the configuration tree (as I see it) is that we often ended up duplicating configuration code for things that aren't actually port-dependent but rather depend on other things such as supported bus types (ISA, PCA, PCMCIA, etc.). This is a particularly big issue with network cards and disk controllers.

The duplicated code then starts to skew. You end up with lots of features (especially drivers) that could be supported across architectures but aren't, simply because port maintainers are focused on their own trees and don't look at what's going on in the others.

A multiple-apex tree also tends to pull the configuration questions downwards from policy (e.g "Parallel-port support?") towards hardware-specific, platform-specific questions ("Atari parallel-port hardware?") By designing the configuration rules for CML2 as a single-apex tree, I'm trying to move the questions upwards and have derivations in the rules file handle distributing that information to a lower level.

For example, instead of a bunch of parallel questions like this in a multiple-apex tree:

PARPORT 'Parallel port support'
PARPORT_PC 'PC-style hardware'
PARPORT_PC_PCMCIA 'Support for PCMCIA management for PC-style ports'
PARPORT_ARC 'Archimedes hardware'
PARPORT_AMIGA 'Amiga builtin port'
PARPORT_MFC3 'Multiface III parallel port'
PARPORT_ATARI 'Atari hardware'
PARPORT_SUNBPP 'Sparc hardware'

I'm trying to move us towards having *one* question and a bunch of well-hidden intelligence about what it implies:

PARPORT 'Parallel port support'
derive PARPORT_PC from PARPORT and X86
derive PARPORT_ARC from PARPORT and ARC
derive PARPORT_AMIGA from PARPORT and AMIGA
derive PARPORT_SUNBPP from PARPORT and SUN

Alan replied, "Ok I see where you are going with that argument. However when you parse all the existing Config.in files into a tree you can see those properties by looking from any node back to its dependancies." Eric replied, "Granted. If, that is, the representation you generate supports that kind of backtracking. This turns out to be very hard if you're starting from an imperative representation rather than a declarative one. But, as a separate issue, the CML2 design *could* be reworked to support a multiple-apex tree, if there were any advantage to doing so. I don't see one. Do you?" Alan replied, not enough to justify the work. However, Jeff Garzik replied to Eric privately, saying that he did see an advantage, "because the current system is directed not by a central file, but by an architecture-specific config.in. Compartmentalized Config.in files are easy to include and even easier to exclude selectively. It's pretty pointless for S/390 arch to parse a ton of driver config entries it will never present to the user; that wastes memory and slows down the configuration system." Eric replied:

The low-level answer is that the configurator doesn't pay the parse-time cost; the CML2 compiler does all that and presents the configurator with a rulebase object that is not large enough for the incremental I/O or memory cost of the "useless" information to be significant. (I'm not handwaving here, I've actually profiled this; the rulebase object for 2.4.3 is only 342K on disk and less than that in core.)

BTW, CML2 only has a "central file" in a rather trivial sense. Here's what the code to include the port-specific rules looks like:

source "arch/i386/rules.cml"
source "arch/alpha/rules.cml"
source "arch/sparc/rules.cml"
source "arch/mips/rules.cml"
source "arch/ppc/rules.cml"
source "arch/m68k/rules.cml"
source "arch/arm/rules.cml"
source "arch/sh/rules.cml"
source "arch/ia64/rules.cml"
source "arch/parisc/rules.cml"
source "arch/s390/rules.cml"
source "arch/cris/rules.cml"

The real issue isn't that they're "centralized", it's that they're siblings under a top-level architecture menu (which most users won't see because normal invocations of the configurator supply that answer from the command line, just as in CML1). Which brings me neatly to my next point...

The higher-level answer is that you're confusing an implementation issue with a design issue. Beware of such premature optimization, for as the hierophant Knuth hath revealed unto us, it is the root of all evil. To persuade me to go back to a multiple-apex tree you'd have to show that there is a *design* or complexity-control advantage to compartmentalizing the configuration information in that way. Maybe there is one, but nobody's shown it to me yet.

(In truth I don't dismiss implementation concerns quite as cavalierly as it might sound from the above. But buying a linear speedup wouldn't be good enough to make me change the design. A quadratic speedup might be, but none of CML2's algorithms are quadratic in the number of symbols in the rulebase.)

There was no reply to this, but elsewhere, under the Subject: Re: [kbuild-devel] Re: CML2 1.0.0 release announcement (http://www.uwsg.indiana.edu/hypermail/linux/kernel/0104.1/0601.html) , Michael Elizabeth Chastain came back to the CML2-vs-mconfig issue, saying:

I like mconfig, but I like CML2 better.

My primary reason is that ESR has more time to work on CML2 than I do on mconfig. And speed problems are often the easiest problems to solve.

Eric did some performance analysis. If I recall correctly, all but 1 or 2 seconds of CML2's runtime is in the parser. He has rewritten the parser once. Maybe someone needs to rewrite it again, maybe propagate some changes into the language spec to make it easier to parse, maybe rewrite from Python to C.

Alan replied, "CML2 seems to have two other problems in my mind. Inability to parse the existing config files. Also the draw ordering in the menu based config doesnt appear right. Menuconfig has a rather undocumented but very important property of doing roughly the right thing with screenreaders. Something to bear in mind when fixing the menu redrawing stuff." Eric explained:

I gave upon that early for two reasons. One was practical; Michael tried this with mconfig, and (apparently) failed. Or, at least, appeared to have decided that path was not worth pursuing.

The other was the procedural vs. declarative problem. I spent about a month after the kbuild team originally encouraged me to tackle the problem working on a design which I later labeled Thesis. This was an attempt to build a cleaned-up CML1, basically the existing language without the syntactic warts. I got as far as spinning up an incomplete implementation that could check toy configurations.

But the design basically didn't work. The problem is that a procedural language imposes a kind of time order that makes it very difficult to (a) do backtracking and (b) prove the correctness of your result. Perhaps a clearer way to put it is that configuration (like, say, screen widget layout) is fundamentally a constraint problem rather than a control problem.

A constraint problem needs a declarative rather than imperative language, and it needs a baby reasoning engine to generate a constraint-satisfying solution (Tk approaches screen-widget layout this way; that's thye source of its power). Neither of my strawman designs had significant advantage over CML1 until I bit the bullet and wrote a theorem prover to reason about timeless constraints.

Elsewhere, Eric also replied to Michael's idea that the CML2 parser might need another rewrite; saying, "The *language compiler's* runtime got cut in half when I hand-rolled a parser for it. The speed problem now is in the configurator itself. One of my post-1.0.0 challenges is to profile and tune the configurator code to within an inch of its life." Aaron Lehmann suggested harshly, "I know that you're reluctant to make the port, but you don't need to be too shy to ask for help. Few people on this list are afraid of C. If you're too lazy to implement CML2 in a standard, popular, robust language, heck, tell us, and we may be able to help you out." Eric replied:

If I were to become absolutely convinced that I can't get acceptable speed out of Python, I might do that. There's a gcml project that tracked the CML2 compiler up to about 0.72 level that might make a decent starting point.

Unfortunately, I'm fairly sure that finishing gcml would take long enough to render the point moot, because by the time it was done the average Linux machine would have sped up enough for the Python implementation not to be laggy anymore :-).

No joke -- *you* try writing a theorem-prover in a language with only fixed-extent data types. Go on. Try it. I think you will (as Mark Twain put it about the consequences of teasing a cat) acquire a valuable education, the facts of which will never grow dim or doubtful.

I'm pretty sure that tuning the Python implementation (coming up with faster algorithms, perhaps by reorganizing the data structures to do more precomputation) will be a more effective use of my time.

Dave replied that the speed issue was his most significant objection to CML2, and that if Eric could fix that aspect, that would kill his main complaint.

Elsewhere, under the Subject: CML 1.1.0, aka "I feel the need...the need for speed." (http://www.uwsg.indiana.edu/hypermail/linux/kernel/0104.1/1028.html) , Eric announced version 1.1.0 available at http://www.tuxedo.org/~esr/cml2/. Marko Kreen tried it and posted his experiences:

Using CML2 1.1.0 'menuconfig' on clean 2.4.3 (mach is PPro 180)

Suggestions:

  • the 'N' should be shown as ' ' as in menuconfig - it is visually much better to get overview of whole screenful. 'Y'/index.html'M' and 'N' are basically of 'same size' so you must look directly on letter to understand what it is - not good.
  • the menuconfig had nice shortcut: when you pressed 'm' on [YN] field, it put 'y' there without questions. So you could use only 2 keys to configure one screen: 'n/m'. this meant you did not need to move fingers around and think about it so much - big thing when you are not touch-typer...
  • the colors are hard to see (red/blue on black). Probably matter of terminal settings. I do not have any productive ideas tho... Probably to get best experience to as much people as possible the less colors are used the better.

    The 'blue: last visited submenu' is unnecessary. Especially because it later turns green... And the 'red' vs. 'green' thing. I guess the green should be used for 'visited entries' too. Now the red means like 'Doh. So I should not have touched this?'. Confusing.

    In other words: if there are too much colors, they become a thing that should be separately learned, not a helpful aid.

    All this IMHO ofcourse. Colors are 'matter of taste' thing so there probably is not exact Rigth Thing.

Bugs/complaints:

  • aic7xxx is not updated (defaults: are 8/5 should be 253/5000) (this from arch/i386/defconfig maybe?)
  • 'IDE chipset support' nesting is very confusing - compare to menuconfig. I would say even 'wrong'... (eg. 'PIIXn tuning' is is under 'PIIXn support' which is not under 'ATA works in progress'.
  • screen is redrawn after _every_ keystroke - not only in moving around, but even when you are on input field...
  • input field: when there is some default and I start typing it should either clear it or append.

Anton Altaparmakov also gave his impressions:

Ok, I tried the CML2 1.1.0. (Had to spend hours installing Python 2.0 until I found all required configure options and got the right modules compiled in, but ok, that's a one off and is not CML2's fault, also ran make test to make sure it works.)

Installed cml, cwd to kernel, and ran make menuconfig.

Waited about 2-5 minutes (didn't time it) to get the menu. Slower than CML1 by a bit. [Note: My development machine is a Pentium Classic 133S with 64MiB ECC RAM and ATA-100 7200RPM HD on Promise ATA-100 controller with several network cards, runs like a charm with 2.4 kernel for what it is used for: file serving/ftp serving/smb serving/nat]

In the menu the colour scheme is a bit strange but everyone has a different taste. Would need some getting used to, but ok. It does seem like a step back in time though, compared to the old menuconfig which had nice windows feel and colours, IMHO. I am not sure why it had to be changed. Surely you can have the old interface with the new theorem prover?

I found a bug: In "Intel and compatible 80x86 processor options", "Intel and compatible 80x86 processor types" I press "y" on "Pentium Classic" option and it activates Penitum-III as well as Pentium Classic options at the same time!?! Tried to play around switching to something else and then onto Pentium Classic again and it enabled Pentium Classic and Pentium Pro/Celeron/Pentium II (NEW) this time! Something is very wrong here.

Now a general comment: CML2 is extremely slow to the point of not being usable! )-: It would take me hours to configure a kernel with this. Just pressing "n"/index.html"y" or "m" somewhere takes easily several seconds to complete... Pressing any of the arrow keys takes between 1 (up/down) and 10 (left/right) seconds to complete. *Argh!* When a window is up, saying press any key to continue there are delays of several seconds of nothing happening at all before the window disappers.

With this slow response time, I wonder whether I actually pressed the key so press it again, key gets queued, so it gets executed when the first key press has finished executing wreaking havoc. )-: It might be all cool and good having a theorem prover and what not inside the configuration but if this is going to replace CML1 completely, IMHO, you will _have_ to provide some speedy way of configuration (and no, using "vi .config" or equivalent is not an option I would like to use...). Many people have been commenting that speed doesn't matter "just use a newer computer" but that argument is just stupid IMNSHO. That's what MS says when they release a new OS/program... I don't need a new computer, this one works absolutely fine and maxes out all it's 10Mbit network connections quite happily, so why should I buy something faster?!? Just to configure a kernel? Surely not. Linux has always been the OS of choice for people with a small budget and the way it is going it is running the danger of loosing this corner of this rather big market.

I will be back to CML1 now so I can configure and kick off the compile of this kernel before dinner...

Elsewhere, under the Subject: CML2 1.1.1, wiuth experimental fast mode (http://www.uwsg.indiana.edu/hypermail/linux/kernel/0104.1/1135.html) , Eric announced version 1.1.1, complete with an experimental 'fast-mode' that he encouraged folks to try out. Jeff asked if both fast and slow modes would stay, or if there was only a distinction between them for testing and debugging purposes. Eric replied, "That's an interesting question to which I do not yet know the answer. I am continuing to speed-tune." Olaf Titz suggested that the distinction might remain, if 'fast-mode' would eventually cut corners, such as having no visual display.

Elsewhere, under the Subject: CML2 1.1.2 is available (http://www.uwsg.indiana.edu/hypermail/linux/kernel/0104.1/1200.html) , Eric announced version 1.1.2, and Anton replied, "Good performance going up/down in menuconfig now. Even on my Pentium 133S! Excellent work! (fastmode on)"

 

6. New Maintainer For Amiga AFFS Filesystem
12 Apr 2001 - 17 Apr 2001 (7 posts) Archive Link: "amiga affs support broken in 2.4.x kernels??"
People: Hans-Joachim WidmaierMark HounschellRoman Zippel

Mark Hounschell could no longer mount an AFFS filesystem after upgrading to 2.4; the mount command would simply hang and could not be killed. Trying to access any file on the system with the mount in that state resulted in massive filesystem corruption. Roman Zippel announced that he'd put a new version up at http://www.xs4all.nl/~zippel/affs.010414.tar.gz. Mark replied that this enabled him to mount the filesystem, but that trying to write to a file would produce a segmentation fault. He also asked if Roman were the new maintainer of the project, since he had received a private response from the original maintainer, Hans-Joachim Widmaier:

affs is broken since somewhere in 2.3.xx. Alas, I do not have the time anymore to do anything about it, and my Amiga ran its last program, too, so I cannot test anymore. Last I know is that several guys wanted to look after affs in 2.4--at least make it run--, but it seems that nothing much has been done in that way. :-(

Sorry for not bearing better news.

Roman replied to Mark that yes, he was the new maintainer. He also said that write support should be working, and asked Mark for more details. After a little back-and-forth, Roman posted a new patch (http://www.xs4all.nl/~zippel/affs.010417.tar.gz) , and Mark replied with complete success. He said, "I'm now able to mount,read,and write to affs file systems. With an scsi zip drive with 3 affs partitions on it and an affs hardfile(loop) that I use with UAE occasionally. Looks good to me. I'll be using this functionality quite a bit so I'll notify you of any anomalies." End of thread.

 

7. Kernel 2.5 Summit, And Preparations For The Next One
16 Apr 2001 - 18 Apr 2001 (26 posts) Archive Link: "Kernel 2.5 Workshop RealVideo streams -- next time, please get better audio."
People: Miles LaneAlan CoxTim WrightTheodore Y. Ts'oDavid LangAndy GroverLarry McVoyDavid S. MillerEric W. BiedermanAlbert D. Cahalan

Miles Lane gave a link to the Kernel 2.5 Summit (http://www.osdn.com/conferences/kernel/) pages, and thanked everyone who participated. But he pointed out that one big drawback of the audio recordings was that usually, only the main speaker could be heard, not the questioners from the audience. He said, "This reduces the value of these recording substantially, since the comments, insights and give-and-take of the other kernel developers would help us get a much more complete understanding of the areas being presented -- try listening to Andy Grover's Power Management presentation and you'll see what I mean." Eric W. Biederman replied that actually, the recordings did capture almost all the questioners, but that the volume was extremely low for those segments. He suggested some software-based volume correction. Alan Cox replied, "Unfortunately its encoded in a proprietary format otherwise it would have been perhaps half an hours work to write an AGC filter for the data." But Tim Wright replied, "So grab and install dsproxy (http://freshmeat.net/projects/dsproxy/), and capture the output. Than feed to e.g. XMMS which already has an AGC plugin."

David S. Miller also pointed out that the problem with the recording quality stemmed from the fact that audience members didn't wait for the microphone to be brought over to them. He speculated that having fixed microphones along the aisles, where audiences could queue up to speak, might be a solution, though that wouldn't lend itself to a relaxed setting in which everyone could participate in a given portion of discussion. Miles replied:

I agree that this is another important issue. It's most important in these events that the flow and exchange of ideas proceed unhindered. I do believe there is a way to record the dialog without introducing significant impediments, though.

What usually is done these days, when a few groups of people need to hold a conference call, for example, is that a few omni-directional microphones are used (these are the sort of spaceship-looking things that get placed in the center of a large table around which the groups sit). There are drawbacks with this, in that, for a large group, there's signal loss if current speaker does not face the microphone. However, these microphones do a pretty good job of picking up voice audio in a 360 degree radius.

There would need to be some post-event sound mixing. For example, if you have ten tables, each with its own omni-directional table microphone, plus unidirectional microphones for the presenter(s), you'd need to mix the signals from the microphones or perhaps switch between the various microphone recordings and adjust for volume differences. You'd likely get the best recording from the table microphone a particular participant was sitting at. You'd also likely get much stronger signals from the presenter's microphone.

What say you all?

Theodore Y. Ts'o also advocated the microphone-at-every-table approach, but said that this would only work for relatively small gatherings (60 or 70 people). He said, "If we have a lot more people, we'll probably have to go to the two microphones in the aisle approach. But at that point a large part of the workshop will be destroyed; so hopefully we'll just be able to keep the numbers of people in the workshop to manageable number." Miles suggested having the workshops more frequently, and targetted to smaller, more focused groups.

Randolph Bentson suggested putting a wireless microphone into a soft ball that could be tossed from speaker to speaker through the audience. Several people thought the idea was funny and good, and Miles thought it was at least funny, but said, "Seriously though, this would probably still be an impediment to the sort of stream-of-conciousness dialog that we'd like to have. Sometimes, there is a quick series of one or two sentence comments from several participants. With a "mike-in-a-ball" your discussion might turn into a sports event. Plus, personally, I am a crappy ball thrower. If many of you have my level of athletic prowess, there'd be a lot of time spent scrambling under tables and chairs." But Albert D. Cahalan suggested allocating half-a-dozen soft-ball microphones, that would simply be tossed back to assistants on a least-recently-used (LRU) basis.

Larry McVoy suggested using a directional microphone, and Miles replied, "Are you talking about one of those "eavesdropper" parabolic microphones? Are you thinking of having someone on stage redirecting the microphone as each speaker starts talking? It could work well, but you'd either lose the first few words each person in the audience said or need to go to a "hand raising/acknowledgement" to create a pause during which the microphone could be redirected." David Lang added, "have a couple of these and you would be able to keep one trained on the most common speakers in any given discussion (then you only have the problem of more speakers then mikes, but short of putting enough mikes around to get the entire room you will always have this problem)"

 

8. Updated Olympic Driver
18 Apr 2001 (3 posts) Archive Link: "[PATCH] Updated Olympic Driver"
Topics: MAINTAINERS File
People: Mike PhillipsJeff Garzik

Mike Phillips posted the latest version of the Olympic Driver, and said, "This updates the driver to use the netdev api, pci dma, alloc_trdev & friends, fixes cardbus ejection, adds support for the zSeries and changes network_monitor from a compile time option to a module parameter. The patch is against 2.4.3 clean, and applies against 2.4.3-ac9 with a small offset in MAINTAINERS." Jeff Garzik threw his hat in the air and cheered, and then gave many comments about the patch. Mike thanked him for the comments, but said that ihs patch as it stood should be incorporated into the main tree, because it had already been delayed too long. But he added that he'd release a new patch soon, that took account of Jeff's comments.

 

 

 

 

 

 

We Hope You Enjoy Kernel Traffic
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License, version 2.0.