Kernel Traffic #237 For 26 Oct 2003

Ok, too long between test5 and test6 again, so the patch is pretty big. Lots of driver updates and architectures fixed, but also lots of merges from Andrew Morton. Most notably perhaps Con's scheduler changes that have been discussed extensively and made it into the -mm tree for testing.

This also finally gets one of the last "must-fix" things for 2.6.0: the extended 32-bit dev_t support. Courtesy of Al Viro (with a lot of prodding and input over the years from Andries).

arm, s390, ia64, x86-64, and ppc64 updates. USB, pcmcia and i2c stuff. And a fair amount of janitorial.

For those who are trying this for the first time, please note that the scheduler has been tuned to tell the difference between tasks of the _same_ nice level. This means do NOT renice X or it will make audio skip unless you also renice your audio application by the same amount. Lots of distributions have done this for the old 2.4 scheduler which could not treat equal "nice" levels as differently as the new scheduler does and 2.6 shouldn't need special treatment.

So for testing note the following points:

Make sure X is NOT reniced to -10 as many distributions are doing. Some shells spawn processes at nice +5 by default and this will make audio apps suffer.

Make sure your hard disk, graphics card and audio card are performing at equal standard to your 2.4 kernel (ie dma is working, graphics is fully accelerated etc). before commenting on audio performance.

With test6, keyboard repeat takes very noticably longer to kick in after X has been started (for both X and console). In test5, starting X makes no difference.

Also, if you move your test5 .config forward and lose sound, you may find that you now have to enable gameport in input devices to be able to select (and thus, compile) your sound card driver.

On the up side, the scheduler changes make the infamous xmms skips go away (for my purposes).

Interesting. I'm pretty sure I did a "make allyesconfig" just before the test6 release, so apparently x86 includes it indirectly through some path, and so it only shows up on m68k and arm?

This, btw, is a pretty common thing. I wonder what we could do to make sure that different architectures wouldn't have so different include file structures. It's happened _way_ too often.

Linus' 2.6.0-test6 announcement doesn't seem to mention the fact that 2.6.0-test5-bk9 fundamentally changed the semantics of /proc/self and the /proc/<pid> name space. These used to map to actual (kernel) tasks, now they map to what I assume are Posixly-correct processes (groups of tasks). In particular, /proc/self is no longer an alias for `current'.

I don't actually disagree with the change, but it took me by surprise since neither the 2.6.0-test6 annoucement nor the diff between the t5-bk8 and t5-bk9 logs seem to mention it.

(It broke the perfctr driver, but I'm handling that by making an already planned API switch now instead of later.)

This is an interesting problem for sure. The link was pointing to a directory that didn't get listed, except that CLONE_THREAD wasn't exactly a popular feature yet. So it was very seldom that the distinction mattered.

Currently, I rely on checking for /proc/self/task to see if threads can be examined. Like this:

task_dir_missing = stat("/proc/self/task/index.html", &sbuf);

That wouldn't work if /proc/self pointed at the task.

It certainly seems to me that the intent of /proc/self is to point to a "process", which is a tgid in kernel terms. Back in the 2.4.xx days, that was a pid in kernel terms. Now that we have CLONE_THREAD and the tgid, a "process" is represented by the tgid. Pointing to the tgid matches what other OSes do.

I think there is something clearly defective about having the /proc/self link point to a hidden directory. It could be pointed at /proc/42/task/58 and such I guess, but I think tools could break as new-style threads begin to get used in the real world.

It's true that if you use the pthreads model, you'll pass fd's between threads freely.

But there are lots of other valid thread usages. Many of the original uses of Linux threading were for special-case apps which used the clone() interface directly. Some were games, where the native threading stuff was doing things like sound etc in the background.

And when you have _that_ kind of model (with assymetric specialized threads), it makes perfect sense for the threads to have independent file descriptors.

And that is what you get with pthreads.

But the native Linux threading has never been pthreads. It's been about a much more generic thing, where the user is in control of what he does.

And that, btw, implies that thread nazis that only use pthreads do _not_ get to determine what the rest of the world does.

The reason people use threads is that sharing the VM space has real advantages: it makes context switching much cheaper (fewer hw resources in the form of TLB usages) and it allows for much faster synchronization through a shared address space.

But the same isn't true of file descriptors or a lot of other software- level abstractions. There are no inherent advantages to sharing, and in fact sharing just gives more opportunity for race conditions, bad interaction etc.

For example, one reason _not_ to share is that the subthread may want to be as invisible to the "main thread" as possible. That's just good programming practice - trying to isolate and encapsulate as much data as possible.

The same way you shouldn't make all your variables global, you shouldn't make all your data structures global unless you have a reason.

You have to explicitly _ask_ for SIGIO. If you do so, and you don't share file descriptors, that's _your_ problem.

But it does indeed have perfectly valid semantics - the signal may well just wake up a thread: and in fact, as most IO is illegal in signal handler context anyway, it usually has to.

Clearly, if you have per-thread file descriptors, you have to keep track of which thread is doing what.

I fail to comprehend. In the United Kingdom, Linksys-Cisco B.V. are committing a criminal offence under the provisions of Section 107(1) of the Copyright, Designs and Patents Act 1988, by knowingly distributing software without the licence of the copyright owner.

Even if the source were to appear on their web site today, their packaging remains a problem since it contains neither said source nor a written offer to provide it. In order to comply, they _must_ recall this product and amend the packaging. So even if they are already working on putting the source on their web site, they still need to recall the product -- and should already have done so.

Once they have withdrawn these products from the market and ceased to commit this criminal offence, _then_ there is scope for negotiation regarding the ways in which they can re-release the products.

Until then, I see none. By continuing to unlawfully sell this product, Cisco are clearly demonstrating that they are not acting in good faith.

I would like to know if this is also a criminal matter in the United States, and if you have also discussed this with the appropriate authorities, rather than only with Linksys/Cisco?

I would also like to know if in the US, as in the UK, it is also an offence for the _retailers_ and _distributors_ of these products to continue to sell them after being informed of the problem. And have they each been informed in a manner which allows it to be proven in a criminal court that they had 'reason to believe' (to use the phrase from the 1988 Act) that there was a problem?

good news. Broadcom/Linksys finally decided to provide all sources. I haven't checked everything but it looks like everything is there. Don't know if they did it by accident but they even donated more ... including the bootloader, ethernet driver and lots of other non-gpl'ed stuff. Interesting change of attitude. I'm sure one day i'll ask them for the source of the (modified) toolchain which they distribute binary only inside this package... but for now i'm happy :-)

Thanks for all the people that supported us. Thanks Broadcom/Linksys for finally come to the conclusion that working together with the community is much more effective than working against us. Thanks!

This is the link

http://linksys.com/support/gpl.asp

Download the wrt54g.tar.gz it is 37MB but I have not find anything interesting in it. Maybe I have not looked well.

There seems to be much more in the package, more developer tools, source code for the broadcom reference design etc. Good for the linksys router hackers, but unfortunately there is still no source code for the driver.

I think the driver files are in /release/src/wl. This directory contains only object files, no source code. The wl driver directory in the kernel tree ( release/src/linux/linux/drivers/net/wl ) is empty.

I've checked the WAP54G, WAP55AG and WRT55AG archives and none of these contain the source code for the driver.

That is true, according to the GPL, _only_ if the modules are distributed as separate works. If they are part of a collective work which is based on the kernel (note, not a _derived_ work but a _collective_ work) then they must be released under the terms of the GPL.

This is a _different_ issue to the question of whether a module is indeed a derived work, and it's _far_ more clear-cut.

Ask yourself the following questions:

1. The wireless and Ethernet driver modules are distributed within a cramfs file system in a flash image on a chip soldered to the board of the device.

   Are they being distributed 'as separate works'?

2. The fundamental mode of operation of these devices is to receive network packets from one of the drivers, pass them through the Linux kernel routing or bridging code, and then back out through another of the network interfaces. All three parts of this are indispensable and the product is useless without any one part.

   1. Does this form a whole which is a derived work based on the Linux kernel?
   2. Does this form a whole which is a collective work?
   3. Is this collective work based, in part, on the Linux kernel?

3. Refer back to the facts in question 1. Is this 'mere aggregation of a work not based on the [kernel] on a volume of a storage or distribution medium'?

Now, having answered those questions, reread the final three paragraphs of ??2 of the GPL.

This 4-part patch contains code for an interim version of relayfs (see Documentation below for a description of relayfs). This version still needs more testing and cleanup, but it contains most of the API changes prompted by user comments, which has resulted in a somewhat simpler API as well as some code simplification.

Here's a summary of the major changes:

• added support for poll()
• moved some of what once was exposed in the API into an opaque reader object, which also simplifies relay_read() (see Reader objects in Documentation)
• removed buffers_full() callback (the function this performed is now specified as a channel attribute (see RELAY_MODE_CONTINUOUS in Documentation)
• removed relay_resume() (clients need to check return value of relay_write() to figure out if a write failed)
• removed offsets_changed() callback (now hidden in reader)
• added relay_reset() which some kernel clients sometimes may need
• changed locking scheme to use n-buffers instead of only 2 (which simplifies relay_open() a bit)
• VFS readers made auto-consuming if a relay file is opened using O_EXCL (and a hopefully better description of what 'consuming' means in Documentation)
• various bug fixes

Still todo -

• resizing cleanup
• resizing of mmapped buffers
• code to allow channels to start out with static buffers

Thanks to Stephen Hemminger, Marco Cova, and Hubertus Franke for contributing patches and pointing out problems, and to Hubertus Franke for a lot of good ideas and for persevering through a couple of limitations (now overcome) in the previous versions that caused some headaches when trying to use relayfs from deep within the scheduler.

One thing you can do with relayfs files is mmap() them. That combined with the kernel-side API, designed to make writing data into buffers and transferring it as large blocks to user-space efficient and flexible, allows for high-speed, high-volume applications which I'm not sure Netlink was designed for.

relayfs can also be used in 'packet' mode, using read(2) to read data as it becomes available, so it can be used for low-speed, low-volume applications as well. Also, some people might find the file-based approach more natural to deal with. Personal preference, I suppose.

So would you consider running printk on Netlink sockets? Do you think Netlink could accomodate something as intensive as tracing? etc.

While I am aware that a lot of people are using Netlink sockets to exchange data from the kernel to user-space, I don't think Netlink sockets can handle the type of throughput relayfs can handle. Netlink and other communication mechanisms (pipes, shared memory pages, etc.) were not designed to handle the type of throughput relayfs was designed for. If nothing else, the use of netlink also drags with it lots of networking code (netlink_sendmsg-> alloc_skb->kmalloc->etc. and then memcpy) With relayfs, you get direct access to the buffer: relay_write->relay_write_direct (which is actually a macro for memcpy()).

So yes, as you say, "It should be possible to make Netlink sockets mmapable", but in that case you might as well port the netlink sockets API to relayfs and you'll probably get better results.

Look, netlink is used on routers to transfer hundreds of thousands of routing table entries in one fell swoop between a user process and the kernel every time the next hop Cisco has a BGP routing flap.

If you must have "enterprise wide client server" performance, we can add mmap() support to netlink sockets just like AF_PACKET sockets support such a thing. But I _really_ doubt you need this and unlike netlink sockets relayfs has no queueing model, whereas not only does netlink have one it's been tested in real life.

You guys are really out of your mind if you don't just take the netlink printk thing I did months ago and just run with it. When someone first told showed me this relayfs thing, I nearly passed out in disbelief that people are still even considering non-netlink solutions.

I've updated the kexec patch for 2.6.0-test7. It can be found at http://developer.osdl.org/rddunlap/kexec/2.6.0-test7/kexec-260t7.patch

A slightly different version of it can also be found in the -osdl patchset at http://developer.osdl.org/shemminger/patches/2.6/2.6.0-test7/

The userspace tools are at http://www.xmission.com/~ebiederm/files/kexec/. You'll need to update the kexec-syscall.c file for the correct kexec syscall number (274). I intend to try to automate this (somehow).

I ran into a product called EasyRDP that uses Linux. However they license their product under a very restrictive license that completely violates the GPL license.

A complete description and dissection of the product can be found on my website: http://www.palli.nl/~ivo/rdp/

The first release of the VST package is now available. VST or Variable Scheduling Timeouts (or if you prefer, Variable Sleep Times) contains code that, from the idle task, scans the timer list and, if no timer is near, skips the timer interrupts that would otherwise be generated. The patch name is hrtimers-vst-*

The net result is that a quite system will use far less power as it does not need to wake up ever 1/HZ timer tick.

This patch depends on the high-res-timers patch version hrtimers-2.4.20-3.0 which must be applied first.

Both of these patchs are on sourceforge at:

http://sourceforge.net/projects/high-res-timers/

Some details:

There is a proc directory (/proc/sys/kernel/vst/) dedicated to vst controls. Root may turn VST on or off here with "echo "0">enable. There is also a log option (danger Will Robinson) which, if turned on causes " x" where x is the number of ticks being skipped, to be printed on the log console. This will quickly swamp the console, but it does show things happening during quite times and no such output when the system is busy. It is also possible to set the threshold here. This is in units of milliseconds and is the time the next timer must away from now to trigger a VST sleep. Currently this is set rather large considering that we can sleep only 50ms, but you can change it here.

The file .../include/linux/vst.h contains details on the arch interface. The core code is all in .../kernel/timer.c with enabling bits in various places. The arch code for the x86 version is all located in .../include/asm-i386/hrtime.h

This is, of course, the first cut at this stuff. There is a lot left to do....

Things left to do:

Currently the PIT is used to wake the system for the next timer. If that timer is further away than about 50ms, the PITs limitations force us to use the max PIT time of about 50ms. We plan to use the rtc hardware to do this wake up, thus eliminating this restriction.

SMP imposes further restrictions and, as it currently stands, may cause late timers in some cases. When we port to the 2.6 kernel this will change, for the better, we hope :) This will also allow us to use the APIC timers in these systems to do the wake up, thus allowing longer sleep times.

By spamming job opennings to our kernel development lists where such things are not considered allowed, you fucknuts are basically guarenteeing that no kernel programmer with a brain is going to respect your company enough to apply for these jobs.

Please stop posting this crap now.

Posting job offerings here has always been and will always be verboten here on these lists. And in particular technical people who join this list get a very bad taste in their mouth when someone advertises here be it for jobs or products.

If there is no good place to look for Linux kernel development jobs, that isn't my problem. What is my problem is to enforce the rules of these lists at vger.kernel.org.

The BK openlogging tree, which has all changesets ever made by anyone in the Linux kernel, was getting big. Really big. The nodes in the graph have internal "serial numbers" which are currently 16 bits, i.e., there can't be more than 64K nodes in the graph.

I sent mail to some of my engineers this morning saying "hey, I suspect the Linux openlogging tree is about overflow, we need to go to 32 bit ser_t's." I had no idea how close we were, I just knew it was a problem we needed to solve.

I just got mail from one of the team which reads: "With 199 serials shy of overflowing , the 32 bit version is now installed".

What that means is that in about a year, you've managed to create 65,337 changesets. That's 179 per day, 7.4/hour, 24x7. You guys are busy.

To put that in perspective, the most active project on sourceforge today, Gaim, has 805 commits to its changelog. Over 3.5 years. That means you are changing your source base 284x more often than they are. And that's just the BK users, that doesn't count the people not using BK, which are a substantial fraction.

No matter how you slice it, it is pretty amazing rate of change. If change is good, you guys rock, I've never seen anything like it.

Frandom is the faster version of the well-known /dev/urandom random number generator. Not instead of, but rather as a supplement, when pseudorandom data is needed at high rate. Few tests so far show that frandom is 10-50 times faster than urandom.

The project's home page: http://frandom.sourceforge.net.

The module works on 2.2, 2.4 and 2.6 kernels. A few straightforward #ifdef's handle compatability (easy to remove to match common coding style).

Purpose
=======

1. Frandom is a handy source of bulk random data.
2. It is *not* intended for encryption and security-related applications.
3. frandom is intended for (scientific) simulations, wiping the disk, stress tests on algorithms and so on.
4. It is more of an /dev/zero than /dev/random

Quality of random numbers
=========================

1. The module has been tested for random number quality with the "diehard" set of tests, and passed them all. This indicates that the bytes are random enough for most scientific purposes.
2. Additional tests results are welcomed.
3. The core of frandom is based upon RC4. frandom is exactly RC4, minus the XOR operation with the data. So if frandom doesn't generate good random numbers, I would wonder why RC4 is considered safe.
4. The random generator is seeded with 256 bytes of the kernel's get_random_bytes() for every file opened on /dev/frandom. This is equivalent to a 2048-bit random key on RC4.
5. I don't see frandom fit for crypto purposes, mainly because the module was naively written. I won't fall off my chair if it turns out to be crypto-safe, but I wouldn't trust it either. Not yet, anyhow.
6. Those who read the source and feel that such a simple algorithm can't create good random: That's exactly the beauty of RC4: It's simple and it works.

frandom and the linux kernel tree
=================================

1. Occasionally, people complain that /dev/urandom is too slow, wishing for something faster.
2. Other argue that a random generator can be written in user space.
3. I agree with both. And I use /dev/zero a lot. I know how to write a zero-generating application in user space.
4. The module is small: 6kB of source code as a standalone module, and 2.3 kB of kernel memory.

I'm proud to announce the first public release of this iSCSI target implementation. It's already usable, but it doesn't implement everything yet what is required by the iSCSI spec. The missing parts shouldn't be that difficult to add (I just didn't need them so far). Other more interesting parts of the spec (e.g. session recovery) are not implemented because I had no client to test it with (although recent UNH releases seem to support this now). It was mostly tested with Cisco iSCSI initiator driver, but also with with the MS initiator driver. You can download the driver from http://www.ardistech.com/iscsi/ , the README should have all the information to configure and build the driver.

This target driver is partly implemented in the kernel, the user space daemon takes care of session startup, cleanup and discovery and the kernel module takes care of the iSCSI requests and disk IO. There are a few reasons to put part of it into the kernel, the main reason is to have direct access to the page cache, this is also a main difference to other available target implementations, which don't use the page cache directly. User space has not that fine grained control (yet) and had to use a lot of threads (the kernel module currently uses 2 threads per target and 1 per device). Part of the problem could be solved with async IO, another part is to be able to control what is in the cache (this is somewhat related to the recent direct IO discussion). So this also an experiment of what is needed if it should be ever moved to user space.

Anyway, while it's in the kernel, there are also some interesting issues for the kernel-user space communication. Right now it's a proc only interface (no sysfs as it's 2.4 only, no evil ioctl, but someone will kill me for the macro abuse :) ), but it should be rather easy to replace it with something else. Perfomance isn't not that important at the moment (but not completely unimportant, if a lot of information has to be read from the kernel, e.g. at startup). More important issues are error reporting and event handling, which could benefit from a better integrated interface. So this driver is also intended to experiment with a few things, if anyone is interested I can explain it further.

Further development will depend a bit on the feed back. This is an older project and we thought about for a while and since we have no immediate need and not the resources to develop it alone, we decided it would be a good idea to release it completely under the GPL. So I updated it to the latest spec and fixed some of the worst embarrasments. If there is enough interest, the development will continue, but without some help it will be rather slow. :-)

Adding to procfs is only slightly less "ewww" than ioctls :) A dedicated chrdev would be IMO preferred to procfs.

Al Viro also had even even more interesting suggestion for [zerocopy] kernel/userspace communication: ramfs.