Kernel Traffic #115 For 23 Apr 2001

UNIX power users do know better. They know their ISPs mail server could show up in RSS or ORBS any moment. Therefor they use their own machines for sending out email.

IMHO DUL is an unethical list to use because it assumes guilty by default. This is different from all other spam blocking lists, which only block hosts _after_ they've found something wrong with them.

The other exception is untestable-netblocks.orbs.org, which blocks everything it cannot test and is just as bad as DUL.

Anyway, since linux-kernel has chosen to not receive email from me I won't bother answering VM bugreports or anything here. It's up to Matti and Davem to decide how useful a place linux-kernel will be.

Thanks !

To come back to the spamfilter promise I made some time ago, people can now get a CVS tree with spam regular expressions and a script to generate a majordomo.cf from it ...

Scripts to automatically generate configuration for other mailing list and mail delivery programs are very much welcome, as are people willing to help maintain the set of regexps.

cvs -d :pserver:cvs@nl.linux.org:/home/CVS login
password: cvs
cvs -d :pserver:cvs@nl.linux.org:/home/CVS checkout spamfilter

The (majordomo-run) mailing list spamfilter@nl.linux.org will be used for CVS updates and possibly discussion about this thing. I'm already using a procmail rule to automatically do a rebuild of NL.linux.org's majordomo.cf whenever something is changed to the CVS tree...

This patch provides a very good performance improvement in file descriptor management for SMP linux kernel on a 4-way machine with the expectation of even higher gains on higher end machines. The patch uses the read-copy-update mechanism for Linux, published earlier at the sourceforge site under Linux Scalablity Effort project.
http://lse.sourceforge.net/locking/rclock.html.

In SMP kernel the performance is limited due to reader-writer lock taken during various calls using files_struct. Majority being in the routine fget(). Though there is no severe contention for files->file_lock as the files_struct is a per task data structure but enough performance penalties have to be paid while even acquairing the read lock due to the bouncing lock cache line when multiple clones share the same files_struct. This was pointed out by John Hawkes in his posting to lse-tech mailing list
http://marc.theaimsgroup.com/?l=lse-tech&m=98235007317770&w=

The improvement in performance while runnig "chat" benchmark (from http://lbs.sourceforge.net/) is about 30% in average throughput. For both configurations the results are compared for base kernel(2.4.2) and base kernel with files_struct_rcu-2.4.2-0.1.patch. Profiling results were also collected by using SGI's kernprof utility and that shows a considerable decrease in amount of time spent in fget(). The "chat" benchmark was run with rooms=20 and messages=500. For each configuration, the test was run for 50 times and average of throughput result in terms of messages per second was taken.

isn't this a solution in search of a problem? does it make sense to redesign parts of the kernel for the sole purpose of making a completely unrealistic benchmark run faster?

(the chat "benchmark" is a simple pingpong load-generator; it is not in the same category as, say, specweb, since it does not do *any* realistic (nonlocal) IO. the numbers "chat" returns are interesting, but not indicative of any problem; perhaps even less than lmbench components.)

After 11 months of painstaking work and testing, CML2 1.0.0 is ready for use, and ready to replace the current kernel-configuration system. You'll find it at <http://www.tuxedo.org/~esr/cml2/>. I've made a transition guide available at <http://www.tuxedo.org/~esr/cml2/transition.html>.

(Why this project at all? For those of you who don't remember, it all started when I realized that building kernels is way too hard. I wanted to simplify the configuration task enough to make configuration accessible to non-gurus. It needs to have more policy options. Rather than hundreds of questions like "Include FOOBAR2317 driver?", the novice should see stuff like "Compile in all modular drivers as modules without prompting?")

On 30 March 2001 at the Kernel Summit, Keith Owens and I worked out a transition schedule with Linus. Keith's rewrite of the makefile system and my configurator tools are officially slated to replace the present system in the 2.5.1 to 2.5.2 timeframe. That, of course, is contingent on us not screwing up :-).

(For those of you who grumbled about adding Python to the build-tools set, Linus has uttered a ukase: CML2's reliance on Python is not an issue. I have promised that once CML2 is incorporated I will actually *reduce* the kernel tree's net dependence on external tools, and I know exactly how to deliver on that promise.)

Linus opted for a sharp cutover rather than a gradual transition -- when the new stuff goes into the tree, CML1 will drop dead immediately. Configuration maintainers should be prepared! Right now, any errors in my translation of the CML1 rules are my problem -- but after Der Tag, they will swiftly become *your* problem. We'll all be happier if you have flamed my butt and helped me fix any problems well in advance of the cutover.

The rules files in this release have been re-checked against the 2.4.3 tree. While I can't guarantee they are completely correct (that's your job), they at least cover every CML1 symbol. I have written coverage tools to check this. There is still a problem with the CRIS port symbols that lack a CONFIG_ prefix, but the CRIS people say they'll fix that in their next update.

The stock CML2 distribution contains an installation script which will drop CML2 in place on a current kernel tree. At most one existing file (your top-level Makefile) will be touched, with new versions of the config/oldconfig/menuconfig/xconfig productions edited in. (The old ones will be kept under variant names, so you can still invoke CML1 if you want.)

I've learned, the hard way, that kernel developers are a conservative bunch. So, to help you all feel better about the change, here are some of the improvements CML2 offers over the existing CML1 configuration system:

Maintainability:

• Where CML1 had three different interpreters, none perfectly compatible with any of the others, CML2 has *one* rule compiler and *one* rulebase- interpreter front end. This will be good for consistency and economy.
• As of CML2 1.0.0 and CML1 2.4.3, the combined code and data size in lines of the system (a good indicator of its maintainability) shrank by 38%. For code alone the figure was 43%. Furthermore, where CML1 was written in a weird mix of three languages, CML2 uses exactly one.
• CML2 decouples the configuration language from the configuration user interface (they communicate with each other only through the compiled rulebase). This means that it will be relatively easy to improve the UI and the language separately.
• CML2 has a track record. It is already being used in other projects, including most notably the Embedded Debian Project. Adam Lock at Netscape is using it to construct a tool for configuring Mozilla builds.

Programmer Friendliness:

• The rather spiky and cluttered shell-like syntax of CML1 is replaced with a much simpler and more regular format resembling that of .netrc or .fetchmailrc. More importantly, the semantics of the language are declarative rather than imperative -- a better match for the problem domain, and thus more expressive and easier to code in.
• CML2 will eliminate (or at least drastically reduce) skew between port configurations. The fact that the top-level CML1 files of the fifteen ports in the kernel tree are separate means there have been plenty of opportunities for the common code in them to drift apart; CML2's design and compilation rules should effectively prevent future bugs of this kind.
• CML2 query prompts and menu banners are separated from the symbol dependency declarations. Thus CML2 system definitions can be internationalized and localized.
• CML2 has a complete, explicit description. Syntax, language semantics, and user-interface options are all discussed in detail.

User Experience:

• In CML2 it is impossible to generate a configuration that is invalid according to the rules file(s). You'll get a popup complaining about the constraint that was violated if you try. (This is effect #1 of having a theorem prover in the configurator.)
• Whenever CML2 can deduce from a symbol tweak that other changes are required, it does them. And if the change is reversed, so are all those side effects. (This is effect #2 of having a theorem prover in the configurator.) The user interfaces tell you what the side effects are.
• All three interfaces do progressive disclosure -- the user only sees questions he/she needs to answer (no more hundreds of greyed-out menu entries for irrelevant drivers!).
• The declarative semantics of CML2 makes it much easier to set up and check options for policy-based configuration. I have done only enough of this in the CML1 translation for demonstration purposes (there are new symbols TUNING, EXPERT and WIZARD that change some visibilities). Once CML2 is in place, it should be a relatively small effort to give the user a rich set of policy and don't-bother-me options.
• The line-oriented mode of the new configurator is much more powerful than the original Configure. It's possible to move backward or jump around in the configuration sequence.
• The new curses mode, unlike the old menuconfig code, has full access to the help texts.
• All URLs in the Tkinter version's help windows are now live; if you click on them they'll launch a browser.
• Curses and Tk interfaces now use color as a navigational aid -- both in the same way so the user interface is consistent.
• And a cool penguin logo when we iconify the X version! :-)

Many linux-kernel regulars have helped develop, test, and debug CML2, including Giacomo Catenazzi, David Kamholz, Robert Wittams, Randy Dunlap, Urban Widmark, Andr? Dahlqvist, Drago Goricanec, William Stearns, and Gary Lawrence Murphy.

Currently, one of the many things I'm doing is trying to sort out a working kbuild-2.5 for the ARM tree. I have some stuff done, but there are several things that I'm definitely not happy with, and there is currently a whole scoop of stuff which I haven't yet found a way for kbuild-2.5 to handle. Its looking like the ARM trees use of kbuild-2.5 will be even more messy than its use of the current build system.

(We have around 60 machines, which key both which files get built, the text and data address of the running kernel image, the text and data address of the decompressor, and which vmlinux.lds.in file we use to link the kernel. This is currently my biggest problem that kbuild-2.5 doesn't seem to be able to handle at present. Note that it is not acceptable that users should have to type in 5 or so apparantly meaningless hex addresses when they configure the kernel.)

I've yet to hear back from Keith on the issues I have with kbuild-2.5, but I'm hopeful that he has some good suggestions...

One of the first things I noticed was it seems noticably slower than CML1. A make menuconfig in CML1 takes me into the menu in under a second. (On an already compiled tree). CML2 takes around 15 seconds before I get that far. This is on an Athlon 800 w/512MB. I dread to think how this responds on a 486.

Scrolling the cursor bar in menuconfig causes a lot of flickering as the entire screen seems to be redrawn. This is becomes unusable after a few minutes usage. Scrolling under CML1's menuconfig doesn't show this behaviour.

The various colours used to show submenus that have been visited seems confusing, and unnecessary. Their meaning also seems undocumented.

Reporting of changing certain features seems to be excessive. For example, changing CPU target from Pentium Pro to Athlon tells me that "M686=n (deduced from MK7)"

Another confusing thing on this menu, happens when you select CRUSOE, and then 386. "MK7=n (deduced from M386) MCRUSOE=n (deduced from M386)" Not sure why selecting Crusoe enables MK7.

Top level menu seems to have gained a few items. For example, the `SCSI support' item has disappeared, making `SCSI disk support' and `SCSI low-level drivers' both appear on the top level menu.

For some reason, the kernel hacking menu doesn't show 4/5 of the options that it used to. Instead it replaces them with one new one (Disable VHPT). Which it seems to picking up from the IA64 tree. Most strange.

Finally, quitting the program (q twice) gives me this..

python2 -O scripts/configtrans.py -h include/linux/autoconf.h -s .config config.out
Traceback (most recent call last):
File "scripts/configtrans.py", line 104, in ?
sys.stderr.write(args[0]);
TypeError: read-only character buffer, int
make: *** [menuconfig] Error 1

All the above was using an 2.4.3-ac4 tree, with CML2-1.0.0

1. It is one programm with multiple frontends:
   old,
   text,
   ncurses,
   random,
   maximum,
   minimum,
   syntax checking
   (X is still missing as my brain is not made for GUI programming)
2. An 'show me all options and handle the rest' mode is still missing - my devel tree has something like that in the works, but I'll probably never finish it now that CML2 is official.
3. it still has multiple top-level config.in. Again that is easily fixable and in fact I did a patch for it (including {old,menu,x}config support in 2.3 times but never submitted it.

I disagree, because I've seen what happens when we go to a single-apex tree. But you could persuade me otherwise. What's your reason for believing this?

The problem with having multiple apices of the configuration tree (as I see it) is that we often ended up duplicating configuration code for things that aren't actually port-dependent but rather depend on other things such as supported bus types (ISA, PCA, PCMCIA, etc.). This is a particularly big issue with network cards and disk controllers.

The duplicated code then starts to skew. You end up with lots of features (especially drivers) that could be supported across architectures but aren't, simply because port maintainers are focused on their own trees and don't look at what's going on in the others.

A multiple-apex tree also tends to pull the configuration questions downwards from policy (e.g "Parallel-port support?") towards hardware-specific, platform-specific questions ("Atari parallel-port hardware?") By designing the configuration rules for CML2 as a single-apex tree, I'm trying to move the questions upwards and have derivations in the rules file handle distributing that information to a lower level.

For example, instead of a bunch of parallel questions like this in a multiple-apex tree:

PARPORT 'Parallel port support'
PARPORT_PC 'PC-style hardware'
PARPORT_PC_PCMCIA 'Support for PCMCIA management for PC-style ports'
PARPORT_ARC 'Archimedes hardware'
PARPORT_AMIGA 'Amiga builtin port'
PARPORT_MFC3 'Multiface III parallel port'
PARPORT_ATARI 'Atari hardware'
PARPORT_SUNBPP 'Sparc hardware'

I'm trying to move us towards having *one* question and a bunch of well-hidden intelligence about what it implies:

PARPORT 'Parallel port support'
derive PARPORT_PC from PARPORT and X86
derive PARPORT_ARC from PARPORT and ARC
derive PARPORT_AMIGA from PARPORT and AMIGA
derive PARPORT_SUNBPP from PARPORT and SUN

The low-level answer is that the configurator doesn't pay the parse-time cost; the CML2 compiler does all that and presents the configurator with a rulebase object that is not large enough for the incremental I/O or memory cost of the "useless" information to be significant. (I'm not handwaving here, I've actually profiled this; the rulebase object for 2.4.3 is only 342K on disk and less than that in core.)

BTW, CML2 only has a "central file" in a rather trivial sense. Here's what the code to include the port-specific rules looks like:

source "arch/i386/rules.cml"
source "arch/alpha/rules.cml"
source "arch/sparc/rules.cml"
source "arch/mips/rules.cml"
source "arch/ppc/rules.cml"
source "arch/m68k/rules.cml"
source "arch/arm/rules.cml"
source "arch/sh/rules.cml"
source "arch/ia64/rules.cml"
source "arch/parisc/rules.cml"
source "arch/s390/rules.cml"
source "arch/cris/rules.cml"

The real issue isn't that they're "centralized", it's that they're siblings under a top-level architecture menu (which most users won't see because normal invocations of the configurator supply that answer from the command line, just as in CML1). Which brings me neatly to my next point...

The higher-level answer is that you're confusing an implementation issue with a design issue. Beware of such premature optimization, for as the hierophant Knuth hath revealed unto us, it is the root of all evil. To persuade me to go back to a multiple-apex tree you'd have to show that there is a *design* or complexity-control advantage to compartmentalizing the configuration information in that way. Maybe there is one, but nobody's shown it to me yet.

(In truth I don't dismiss implementation concerns quite as cavalierly as it might sound from the above. But buying a linear speedup wouldn't be good enough to make me change the design. A quadratic speedup might be, but none of CML2's algorithms are quadratic in the number of symbols in the rulebase.)

I like mconfig, but I like CML2 better.

My primary reason is that ESR has more time to work on CML2 than I do on mconfig. And speed problems are often the easiest problems to solve.

Eric did some performance analysis. If I recall correctly, all but 1 or 2 seconds of CML2's runtime is in the parser. He has rewritten the parser once. Maybe someone needs to rewrite it again, maybe propagate some changes into the language spec to make it easier to parse, maybe rewrite from Python to C.

I gave upon that early for two reasons. One was practical; Michael tried this with mconfig, and (apparently) failed. Or, at least, appeared to have decided that path was not worth pursuing.

The other was the procedural vs. declarative problem. I spent about a month after the kbuild team originally encouraged me to tackle the problem working on a design which I later labeled Thesis. This was an attempt to build a cleaned-up CML1, basically the existing language without the syntactic warts. I got as far as spinning up an incomplete implementation that could check toy configurations.

But the design basically didn't work. The problem is that a procedural language imposes a kind of time order that makes it very difficult to (a) do backtracking and (b) prove the correctness of your result. Perhaps a clearer way to put it is that configuration (like, say, screen widget layout) is fundamentally a constraint problem rather than a control problem.

A constraint problem needs a declarative rather than imperative language, and it needs a baby reasoning engine to generate a constraint-satisfying solution (Tk approaches screen-widget layout this way; that's thye source of its power). Neither of my strawman designs had significant advantage over CML1 until I bit the bullet and wrote a theorem prover to reason about timeless constraints.

If I were to become absolutely convinced that I can't get acceptable speed out of Python, I might do that. There's a gcml project that tracked the CML2 compiler up to about 0.72 level that might make a decent starting point.

Unfortunately, I'm fairly sure that finishing gcml would take long enough to render the point moot, because by the time it was done the average Linux machine would have sped up enough for the Python implementation not to be laggy anymore :-).

No joke -- *you* try writing a theorem-prover in a language with only fixed-extent data types. Go on. Try it. I think you will (as Mark Twain put it about the consequences of teasing a cat) acquire a valuable education, the facts of which will never grow dim or doubtful.

I'm pretty sure that tuning the Python implementation (coming up with faster algorithms, perhaps by reorganizing the data structures to do more precomputation) will be a more effective use of my time.

Using CML2 1.1.0 'menuconfig' on clean 2.4.3 (mach is PPro 180)

Suggestions:

• the 'N' should be shown as ' ' as in menuconfig - it is visually much better to get overview of whole screenful. 'Y'/index.html'M' and 'N' are basically of 'same size' so you must look directly on letter to understand what it is - not good.
• the menuconfig had nice shortcut: when you pressed 'm' on [YN] field, it put 'y' there without questions. So you could use only 2 keys to configure one screen: 'n/m'. this meant you did not need to move fingers around and think about it so much - big thing when you are not touch-typer...

• the colors are hard to see (red/blue on black). Probably matter of terminal settings. I do not have any productive ideas tho... Probably to get best experience to as much people as possible the less colors are used the better.

  The 'blue: last visited submenu' is unnecessary. Especially because it later turns green... And the 'red' vs. 'green' thing. I guess the green should be used for 'visited entries' too. Now the red means like 'Doh. So I should not have touched this?'. Confusing.

  In other words: if there are too much colors, they become a thing that should be separately learned, not a helpful aid.

  All this IMHO ofcourse. Colors are 'matter of taste' thing so there probably is not exact Rigth Thing.

Bugs/complaints:

• aic7xxx is not updated (defaults: are 8/5 should be 253/5000) (this from arch/i386/defconfig maybe?)
• 'IDE chipset support' nesting is very confusing - compare to menuconfig. I would say even 'wrong'... (eg. 'PIIXn tuning' is is under 'PIIXn support' which is not under 'ATA works in progress'.
• screen is redrawn after _every_ keystroke - not only in moving around, but even when you are on input field...
• input field: when there is some default and I start typing it should either clear it or append.

Ok, I tried the CML2 1.1.0. (Had to spend hours installing Python 2.0 until I found all required configure options and got the right modules compiled in, but ok, that's a one off and is not CML2's fault, also ran make test to make sure it works.)

Installed cml, cwd to kernel, and ran make menuconfig.

Waited about 2-5 minutes (didn't time it) to get the menu. Slower than CML1 by a bit. [Note: My development machine is a Pentium Classic 133S with 64MiB ECC RAM and ATA-100 7200RPM HD on Promise ATA-100 controller with several network cards, runs like a charm with 2.4 kernel for what it is used for: file serving/ftp serving/smb serving/nat]

In the menu the colour scheme is a bit strange but everyone has a different taste. Would need some getting used to, but ok. It does seem like a step back in time though, compared to the old menuconfig which had nice windows feel and colours, IMHO. I am not sure why it had to be changed. Surely you can have the old interface with the new theorem prover?

I found a bug: In "Intel and compatible 80x86 processor options", "Intel and compatible 80x86 processor types" I press "y" on "Pentium Classic" option and it activates Penitum-III as well as Pentium Classic options at the same time!?! Tried to play around switching to something else and then onto Pentium Classic again and it enabled Pentium Classic and Pentium Pro/Celeron/Pentium II (NEW) this time! Something is very wrong here.

Now a general comment: CML2 is extremely slow to the point of not being usable! )-: It would take me hours to configure a kernel with this. Just pressing "n"/index.html"y" or "m" somewhere takes easily several seconds to complete... Pressing any of the arrow keys takes between 1 (up/down) and 10 (left/right) seconds to complete. *Argh!* When a window is up, saying press any key to continue there are delays of several seconds of nothing happening at all before the window disappers.

With this slow response time, I wonder whether I actually pressed the key so press it again, key gets queued, so it gets executed when the first key press has finished executing wreaking havoc. )-: It might be all cool and good having a theorem prover and what not inside the configuration but if this is going to replace CML1 completely, IMHO, you will _have_ to provide some speedy way of configuration (and no, using "vi .config" or equivalent is not an option I would like to use...). Many people have been commenting that speed doesn't matter "just use a newer computer" but that argument is just stupid IMNSHO. That's what MS says when they release a new OS/program... I don't need a new computer, this one works absolutely fine and maxes out all it's 10Mbit network connections quite happily, so why should I buy something faster?!? Just to configure a kernel? Surely not. Linux has always been the OS of choice for people with a small budget and the way it is going it is running the danger of loosing this corner of this rather big market.

I will be back to CML1 now so I can configure and kick off the compile of this kernel before dinner...

affs is broken since somewhere in 2.3.xx. Alas, I do not have the time anymore to do anything about it, and my Amiga ran its last program, too, so I cannot test anymore. Last I know is that several guys wanted to look after affs in 2.4--at least make it run--, but it seems that nothing much has been done in that way. :-(

Sorry for not bearing better news.

I agree that this is another important issue. It's most important in these events that the flow and exchange of ideas proceed unhindered. I do believe there is a way to record the dialog without introducing significant impediments, though.

What usually is done these days, when a few groups of people need to hold a conference call, for example, is that a few omni-directional microphones are used (these are the sort of spaceship-looking things that get placed in the center of a large table around which the groups sit). There are drawbacks with this, in that, for a large group, there's signal loss if current speaker does not face the microphone. However, these microphones do a pretty good job of picking up voice audio in a 360 degree radius.

There would need to be some post-event sound mixing. For example, if you have ten tables, each with its own omni-directional table microphone, plus unidirectional microphones for the presenter(s), you'd need to mix the signals from the microphones or perhaps switch between the various microphone recordings and adjust for volume differences. You'd likely get the best recording from the table microphone a particular participant was sitting at. You'd also likely get much stronger signals from the presenter's microphone.

What say you all?