Kernel Traffic #267 For 18�Jul�2004

Available at http://tree.celinuxforum.org/pubwiki/moin.cgi/CELinux_5fPatchArchive

For those interested, the set of three patches provide POSIX high-res timer support for linux-2.6.6. The core and i386 patches are updates of George Anzinger's hrtimers-2.6.5-1.0.patch available on SourceForge <http://sourceforge.net/projects/high-res-timers/>. The ppc32 port is not available on SourceForge yet.

Just reading from the Posix 1003.1b section 14 spec referenced by the HRT main project page, I see the following:

Realtime applications must be able to operate on data within strict timing constraints in order to schedule application or system events. Timing requirements can be in response to the need for either high system throughput or fast response time. Applications requiring high throughput may process large amounts of data and use a continuous stream of data points equally spaced in time. For example, electrocardiogram research uses a continuous stream of data for qualitative and quantitative analysis.

If this is really the goal here, then why not just integrate Adeos into the kernel and make some form of HRT as a loadable module that uses Adeos to provide its services?

Currently Adeos runs on x86, ARM (MMU-full and MMU-less), PPC, so portability is not an issue. Plus, the interface provided can either be directly used by drivers to get hard-rt interrupts or it can be used by another layer to provide more elaborate services (like RTAI or, potentially, HRT.) Using the virtual interrupts that can be dynamically allocated at runtime, it's rather easy to send signals between domains.

Sure, you may not have the exact Posix 1003.1b API, but I don't remember there being any persistent goal of having the kernel conform to any standard.

Actually I think the point I'm trying to make can't be fairly conveyed without providing a lot of background of what can be done with Adeos. I would suggest that those interested do some digging. Among other things, you may want to read about RTAI/fusion:

http://www.fdn.fr/~brouchou/rtai/rtai-doc-prj/rtai-fusion.html

http://www.fdn.fr/~brouchou/rtai/modules.php?name=Content&pa=showpage&pid=1

I think the real problem is an open source question. The RTAI and RTLINUX folks are not exactly in the same camp (either with each other OR with LINUX) in this regard. Should that change and one or more of these become truly open source without others claiming "foul", there are vendors who are ready and willing to work with the code. Vendors of open source (and their customers) don't want to find themselves in law suits...

As such, this is really off topic... as is a discussion of the merits of this sort of solution. On this list we are interested in working in the confines of LINUX as found on linux.org possibly modified by truly open source patches and packages.

The Umbrella project (umbrella.sf.net) also needs the ability to use public key cryptography within the Linux kernel (though only to verify signatures).

We were planing to implement RSA or ElGamal (porting some of GPG) to the kernel for that purpose. However, if this project gets started, we will be very interested. In the automn we might also have some development hours to contribute with :-)

Is a separate generic assymetric crypto API really necessary?

The cryptoapi usage model is to do a setkey before any encrypt or decrypt. The setkey will be done with either a public or private key. So there is no need to have a public_key_alg with separate public_encrypt and private_encrypt functions, as this distinction is implied at the setkey time. So our plan was to just add another crypto_alg for rsa_1024.

If anyone can point out why this is insufficient, then we will certainly reconsider.

In essence for a digsig module you'll require the ability to

1. import a PK key [from a binary packet or a cert, preferably the former].
2. free a PK key from memory (no need to waste dynamic resources like bignums while not being used).
3. ability to sign/verify/encrypt/decrypt which amounts to a exptmod and PKCS #1 [v2.0 preferably] padding.

An API exactly mirroring the symmetric side won't really work 100%. For instance, symmetric operations are not likely to fail [I don't know how error handling is performed]. Also decrypt/verify ops may fail hard [due to lack of heap] or soft [invalid packet].

It would probably make more sense to design a simple API for PK crypto [say support RSA/ECC/DH/DSA ;-)] then to mash the symmetric crypto API into something compatible.

On a side note I've been contacted about my interest in making this happen [hence my reply here]. I'm offering my public domain LibTomCrypt [and indirectly LibTomMath] for the task. Currently I'm working on reducing the stack usage in LibTomCrypt's PK operations.

What I propose is we can port LibTomCrypt's [by stripping out stuff that isn't required like symmetric crypto] PK code to a kernel module to be released under the GPL. Since the code is already public domain [and I personally wrote all of the relevent code myself == no copyright issues] there shouldn't be any problems with this.

2.6.7's airo.ko (unlike 2.6.6's) won't allow the user to set ESSID via "echo myessid >/proc/driver/aironet/ethX/SSID".

Changes like this shouldn't probably be made in the middle of a stable series.

I would like to announce the first public release of a generic SCSI target middle level subsystem for Linux (SCST) with Qlogic 2200/2300 target driver and a patch for for UNH-iSCSI Target 1.5.3 to work over SCST.

SCST is designed to provide unified, consistent interface between SCSI target drivers and Linux kernel and simplify target drivers development as much as possible. It has the following features:

• Very low overhead, fine-grained locks and simplest commands processing path, which allow to reach maximum possible performance and scalability that close to theoretical limit.
• Incoming requests can be processed in the caller's context or in one of the internal SCST's tasklets, therefore no extra context switches required.
• Complete SMP support.
• Undertakes most problems, related to execution contexts, thus practically eliminating one of the most complicated problem in the kernel drivers development. For example, a target driver for Qlogic 2200/2300 cards, which has all necessary features, is about 2000 lines of code long, that is at least in several times less, than the initiator one.
• Performs all required pre- and post- processing of incoming requests and all necessary error recovery functionality.
• Emulates necessary functionality of SCSI host adapter, because from a remote initiator's point of view SCST acts as a SCSI host with its own devices. Some of the emulated functions are the following:
• Generation of necessary UNIT ATTENTIONs, their storage and delivery to all connected remote initiators (sessions).
• RESERVE/RELEASE functionality.
• CA/ACA conditions.
• All types of RESETs and other task management functions.
• REPORT LUNS command as well as SCSI address space management in order to have consistent address space on all remote initiators, since local SCSI devices could not know about each other to report via REPORT LUNS command. Additionally, SCST responds with error on all commands to non-existing devices and provides access control (not implemented yet), so different remote initiators could see different set of devices.
• Other necessary functionality (task attributes, etc.) as specified in SAM-2, SPC-2, SAM-3, SPC-3 and other SCSI standards.
• Device handlers architecture provides extra reliability and security via verifying all incoming requests and allows to make any additional requests processing, which is completely independent from target drivers, for example, device dependent exceptional conditions treatment or data caching.

Interoperability between SCST and local SCSI initiators (like sd, st) is the additional issue that SCST is going to address (it is not implemented yet). It is necessary, because local SCSI initiators can change the state of the device, for example RESERVE the device, or some of its parameters and that would be done behind SCST, which could lead to various problems. Thus, RESERVE/RELEASE commands, locally generated UNIT ATTENTIONs, etc. should be intercepted and processed as if local SCSI initiators act as remote SCSI initiators connected to SCST.

Interface between SCST and target drivers is based on work, done by InterOperability Lab (IOL) of University of New Hampshier (UNH).

In addition, the project contains a target driver for Qlogic 2200/2300 Fibre Channel cards and a patch for for UNH-iSCSI Target 1.5.3 that makes it work over SCST.

In the current version 0.9.1 SCST looks to be quite stable (for beta) and useful. The same is for Qlogic 2200/2300 target driver. Only 2.4 kernels currently supported, but update for 2.6 is coming soon. No kernel patches are necessary. Tested on i386 only, but should work on any other supported by Linux platform.

More information, including the source code and detail documentation, could be found on http://scst.sf.net.

There is not only iSCSI target (i.e. software one) in the world. For hardware targets if you were switch on each command (IRQ) to user space and back, it would be huge performance penalty, especially for commands that request small data transfers. Take a look on the Qlogic target: all job is done in the tasklet, without ever context switch. The same is for upcoming iSCSI hardware, like Qlogic QLA4010, which also supports target mode.

BTW, the processing is simple enough, the main SCST module is only about 60Kb long. The main point is that this processing must be done in _each_ SCSI target driver. So, consider SCSI target mid-level like a library (framework) for such drivers, exectly as the regular SCSI mid-level for regular SCSI drivers.

For that is intended upcoming block device handler with block layer/cache support, which will be in its exec() method check, if requested blocks in cache, and, if not, dispatch the commands to block layer, leaving regular scsi_do_req() calls for tapes, changers, etc. In the similar way "_perf" handlers work (they don't send READ/WRITE commands to SCSI devices for performance studies).

This device handler is on our todo list. Actually, it's quite simple and if anyone interested, he's help would be greatly appreciated.

It contains a significant amount of USB fixes, JFS update, netfilter/sctp fixes, CDROM driver update, tg3 update, SPARC/Alpha fixes.

And more importantly the FPU x86/x86-64 crash fix.

Both Moxa serial drivers (moxa & mxser) are BROKEN_ON_SMP because they use cli() to disable interrupts for critical sections. See Documentation/cli-sti-removal.txt for details. They will need some acceptable modern form of protection there,

Is anyone working on this? not that I've heard of. Have you tried this email address: support@moxa.com.tw

Here are a number of various USB fixes and cleanups for 2.6.7. All of these patches have been in the past few -mm releases.

There's lots of good stuff in here:

• gadget fixes
• new host controller driver for lh7a404 devices
• lots of hub locking rework to fix up the mess there.
• sparse markups to remove warnings
• loads of bugfixes.

Please pull from:

bk://kernel.bkbits.net/gregkh/linux/usb-2.6

Patches will be posted to linux-usb-devel as a follow-up thread for those who want to see them.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.7/2.6.7-mm1/

• Added a new vm tunable: /proc/sys/vm/vfs-cache-pressure.

  This allows tuning of the kernel's preference for reclaiming VFS caches versus pagecache.

  vfs-cache-pressure=0: dentry and inode caches aren't reclaimed at all
  vfs-cache-pressure=100: default - current behaviour
  vfs-cache-pressure > 100: reclaim the VFS caches harder.

  It seems that large values of vfs-cache-pressure are needed to make much difference: 1000 or more.

• Under some circumstances the current page reclaim code can hold interrupts off for a long time. That is fixed here.
• I went through and dropped a bunch of patches which don't seem to be very useful now - mainly debug stuff.
• Various driver updates and random fixes

I uploaded an update to the Videotext driver for the SAA5246A videotext decoder in

http://www.michaelgeng.de/linux/saa5246a-2.6.7.patch

The update only changes comments showing that the driver also works for the SAA5281 videotext decoder chip. This is not surprising because according to the data sheet of the SAA5281 it is compatible to the SAA5246A. I have tested this with a Siemens Multimedia Extension Board (MXB).

I also added MODULE_AUTHOR and MODULE_DESCRIPTION macros which were missing up to now.

Patch comment:
V4L: saa5246a driver update, supports saa5281

Signed-off-by: Michael Geng <linux@MichaelGeng.de>

Well, Jeff explicitely said that we should not care about drivers outside the kernel ;-)

Seriously, I see three solutions :

1. Convert ipw2100 to using drivers/net/wireless/ieee802_11.h, extend this header as necessary
2. Have ipw2100 use a private version of 802_11.h
3. Convince us that this file is really needed (good luck) Obviously (1) is better in the long term.

was a remnant from the old aironet4500 driver that was removed during 2.5.X. It was also confusing because there is a file called drivers/net/wireless/ieee802_11.h that has a somewhat similar purpose and is used in various drivers (Orinoco, Atmel). I think it was discussed on netdev.

I was not aware that IPW2100 was using it. I could not try this driver because it doesn't compile with gcc 2.95.

we have ported our x86 precise accounting patch from the 2.4 kernel series to 2.6.7.

The patch fixes a problem with the accounting via the "cpu*" lines from /proc/stat. The standard kernels can produce misleading data for programs that give up the rest of their timeslice, e.g. via usleep. This can lead to programs using 90% or more CPU time which is not accounted. The computer is reported as idle or another program seems to use up the time. The patch fixes the problem by doing the accounting on the basis of the CPU's time stamp counter instead of whole time slices. The patch as well as proof-of-principle programs illustrating the problem can be downloaded at http://www.ti.uni-hd.de/HLT/software/software.html#kernel.

Our original mail for the 2.4.20 patch is at http://lwn.net/Articles/24974/.

Please CC all comments to timm.steinbeck AT kip.uni-heidelberg.de and arne.wiebalck AT kip.uni-heidelberg.de, as we are not subscribed to the LKML.

I vaguely recall someone recently talking about eliminating preempt by improving low-latency. See, if everything were ideal, we wouldn't need preempt, because all drivers would yield the CPU at appropriate times. Supposedly, preempt introduces some undesirable overhead.

Perhaps we could turn preempt into some kind of watch-dog. If a kernel thread doesn't behave, it gets killed. :)

gcc 3.4 gained support for several typical bitops as builtin directives. Using these over inline asm has a few advantages:

• gcc can optimize constants into these better
• gcc can reorder and schedule the code better
• gcc can allocate registers etc better for the code

The question is if we consider it desirable to go down this road or not. In order to help that discussion I've attached a patch below that switches the i386 ffz() function to the gcc builtin version, conditional on gcc having support for this. Before I go down the road of converting more functions and/or architectures.... is this worth doing?

unless the gcc code can be shown to be clearly better, it is NOT worth doing.

Adding support for built-ins generates a noticeable maintenance burden, since we'll still have to support older gcc's and architectures where gcc does WORSE than we do. And quite frankly, I doubt you'll find any cases where gcc does any better in any measurable way.

In other words, the rule about gcc builtins should be:

• use them only if they are old enough that the huge majority (possibly all) of users get them. This is partly because gcc has frankly been buggy, and often makes assumptions that simply may not be true for the kernel (ie "it's ok to use library routines").
• only use them if you can show a measurable improvement. Theoretical arguments simply don't count. Theoretical arguments is why gcc-3.x is a lot slower than 2.95 and is apparently still not generating appreciably better code for the kernel (and no, don't bother pointing me to spec runs, I just don't care. The kernel is what I care about).

So far, the only case where they have been worth it is likely the "memcpy()" stuff, and even there our previous macros were doing an almost equivalent job (but were arguably so ugly that it was worth making the change).

For something like ffs/popcount/etc, I do not see _any_ point to compiler support. There just aren't any kernel uses that make it worth it. Sounds like a total micro-optimization for some spec benchmark.

Here is a new patch for kernel 2.6.7. You can download it from sourceforge at http://prdownloads.sourceforge.net/elsa/patch-2.6.7-elsa?download

This patch provides structures and functions to do accounting for a group of process. There is two parts.

On one hand there is two APIs that allow the creation and the destruction of an item called a "bank" that will contain an group of processes. There is also two other routines that can add or remove processes in a bank.

On the other hand, you have the accounting mechanism that used bank to perform BSD-like accounting for a group of process. We used ioctl() interface to create/destroy a bank and also to add/remove a process in a bank. Currently, device driver is dev/elsacct and the major number is dynamically allocated, therefore you need to check the number in the /proc/devices and create the corresponding device. A sample code is given at: http://cvs.sourceforge.net/viewcvs.py/elsa/tests/elsa_cmd.c?rev=1.7&view=markup

Currently we do BSD-like accounting. The next step is to identify what kind of metrics can be interesting for accounting. We started to describe that point on a withpaper that is available at: http://sourceforge.net/docman/display_doc.php?docid=23446&group_id=105806 This paper describes differences between two accounting systems that are ELSA and CSA.

Every comments are welcome,

The ELSA poject team (http://elsa.sourceforge.net)