Kernel Traffic #287 For 1 Jan 2005

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc1/2.6.10-rc1-mm4/

• The 4-level-pagetable code is hopefully finalised.
• Various architecture updates.
• Added the new FRV architecture.
• v4l updates, fbdev updates.

• A process matter: I'm now tracking any regressions since 2.6.9, with the intention that we not release 2.6.10 until they're all fixed. (Where "tracking" means shoving them into a folder called "bugs"). So if anyone is aware of any post-2.6.9 regressions, please make sure that I have a copy of the email.

  I'm less interested in bugs which existed prior to 2.6.9, as they're presumably minor and as long as we don't actually introduce regressions then we'll make decent progress across kernel releases.

This is an improved version of OOM-kill-from-kswapd patch.

I believe triggering the OOM killer from task reclaim context is broken because the chances that it happens increases as the amount of tasks inside reclaim increases - and that approach ignores efforts being done by kswapd, who is the main entity responsible for freeing pages.

There have been a few problems pointed out by others (Andrea, Nick) on the last patch - this one solves them.

First, Andrea noted that if progress had been made in the high zone, the OOM killer would not be triggered. Now it conditions the triggering on "DMA+Normal" reclaiming success.

Nick noted that the last patch would do wrong on the NUMA case (because of per-node kswapd) - its not a problem now because we will only kill if there is any task not being able to allocate and free pages. The memory allocation fallback to other nodes will prevent that from happening.

Another drawback from the last patch was that it disable the "all_unreclaimable" logic which attempts to avoid scanning storms - that way kswapd was able to detect OOM condition.

What it does now is to disable the all_unreclaimable logic after 5 seconds that its been set. This is enough time for the system to complete IO of (at least some) pages which have been written-out for reclaiming purposes.

After that period (which can be sysctl'ed BTW), it then performs a full scan. In case no progress has been made, and both DMA and normal zones are below the pages_min watermark, the OOM killer is triggered.

It looks very reliable in my testing - but I need others to test it as well (Martin and Thomas especially who have good test cases).

Ok, the -rc2 changes are almost as big as the -rc1 changes, and we should now calm down, so I do not want to see anything but bug-fixes until 2.6.10 is released. Otherwise we'll never get there.

A lot of driver updates, many of them of the small and trivial kind, others less so. USB, ALSA, fbdev, IDE, i2c, v4l, you name it. With a sprinking of core device model and PCI updates thrown in for good measure.

Also, a number of architecture updates: ppc64, m68k, uml, parisc, arm.

And md, NTFS, and Documentation updates too.

The diffstat shows more of the story - not so much single really big changes, more of a fairly wideranging thing.

Here goes the fourth release candidate of 2.4.28.

A few small problems showed up in time for another -rc.

Missing exported symbols in the networking area, an ACPI poweroff bugfix, aic7xxx compile fix with -Werror, a binfmt_elf underflow enhancement, an SCTP fix, and a couple TG3 fixes.

This will become v2.4.28 final if nothing _really_ bad shows up.

Thanks to everybody who has been contributing to make this happen.

The Xen team would like to start submitting upstream the patches required to enable Linux to run over the Xen 2.0 Virtual Machine Monitor. For more information about Xen see: http://xen.sf.net

There are three main classes of patch required:

core :	small patches that provide extra hooks for arch-xen
arch-xen :	large patch to add arch/xen and include/asm-xen
xen-drivers :	patch to add virtual block, network and console drivers

We're proposing to submit the 'core' patches first, as these probably require closest inspection (the others are all new files in new directories, so can't break existing architectures).

There is already a significant user base and lively developer community backing the Xen project, so we're confident we will have the resources to maintain arch-xen on Linux 2.6 going forward (as well as Linux 2.4/NetBSD/FreeBSD/etc).

Let us know what you think of the patches ;-)

We've experimented with doing thing the other way round, having xen a sub architecture of i386, but it was _really_ messy.

I firmly believe that having a separate arch/xen is the best approach for the moment. In the future, it might make sense to merge arch xen into i386, but to do this cleanly would require significant restructuring of i386. I think that's something we could move toward after everyone's gotten comfortable with having arch xen in the tree.

The fact that arch xen is self contained actually makes it easier for us to maintain in some respects. We've been tracking 2.6 releases for some time without too much difficulty.

2.6 has been relatively easy for now (because it was supposed to be a "stable kernel"), but I suspect it'll get worse again over time. e.g. in 2.5 it was really bad for long times. Essentially you will need to commit significant man power to this.

Also it's quite hard to always catch all the changes that get done to i386.

Overall I think it's a bad idea to have four different x86 like architectures in the tree. Especially since there will be likely more hypervisors over time. i386 and x86-64 make some sense because 64bit is a natural boundary, but extending it elsewhere doesn't scale very well.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc2/2.6.10-rc2-mm2/

• Lots of small bugfixes. Some against patches in -mm, some against Linus's tree.
• There's a patch here which should address the oom-killings which a few people have reported.

Here is a patch that adds a wrapper for defining jprobe.entry to make it easy to handle the three dword function descriptors defined by the PowerPC ELF ABI.

Current patch against 2.6.10-rc2-mm1 + kprobes patch for ppc64.

Changes for adding this wrapper for x86, ppc64 (tested) and x86_64 (untested) below. The earlier method of defining jprobe.entry will continue to work.

We are pleased to announce releasing new version of Linux Kernel State Tracer.

The Linux Kernel State Tracer(a.k.a. LKST) version 2.2.0 has been released. This version can be applied to linux-2.6.9. And LKST 2.2.0 supports IA32, IA64 and x86-64 platforms.

LKST is a tool that supports to analyze of fault and evaluate for kernel. Especially it is usuful for analyzing the unanticipated fault of kernal.

The latest version of the LKST newly supports the x86-64 architecture.

For more changes, see Changelog-2.2.0.txt
<http://prdownloads.sourceforge.net/lkst/Changelog-2.2.0.txt?download:gt;

Remarks:
For supporting IA64 and x86-64, we hacked KernelHooks.

LKST binaries, source code and documents are available in the following site,
https://sourceforge.net/projects/lkst/
http://sourceforge.jp/projects/lkst/

We prepared a mailing list written below in order to let users know update of LKST.

lkst-users@lists.sourceforge.net
lkst-users@lists.sourceforge.jp

To subscribe, please refer following URL,

http://lists.sourceforge.net/lists/listinfo/lkst-users
http://lists.sourceforge.jp/mailman/listinfo/lkst-users

And if you have any comments, please send to the above list, or to another mailing-list written below.

lkst-develop@lists.sourceforge.net
lkst-develop@lists.sourceforge.jp

More merges and forward ports. These include the extra binfmt_elf checks and the SMBfs overflow fixes that match those in 2.4.28. It also includes several patches Dave Miller believes should be backported from the networking code. Also fixed is a long standing problem where ide would claim some devices it couldn't actually use and didn't install drivers for.

Arjan van de Ven is now building RPMS of the kernel and those can be found in the RPM subdirectory and should be yum-able. Expect the RPMS to lag the diff a little as the RPM builds and tests do take time.

The it8212 still doesn't default to DMA on - that is on the TODO list. The HPT366 rework project is also not ready (its gone back to the drawing board for a few days if you are a volunteer and wondered what is up).

ftp://ftp.kernel.org/pub/linux/kernel/people/alan/linux-2.6/2.6.9/

This updated patch adds support for notification of overheating conditions on intel x86_64 processors and incorporates suggestions from Andi. Tested on EM64T and test booted on AMD64.

Hardware courtesy of Intel Corporation

Andrew please backout the current patch and apply this one (verified it applies against -bk), if you'd prefer an incremental let me know.

I've been getting a lot of requests lately to see if Linux supports the USB to serial device from Silicon Laboratories called the CP2102 chip. It turns out that the company is claiming Linux support, yet they are only shipping a binary driver for Red Hat Linux 9.0.

In talking with the company, they insist that they will not release the source code to this module, and they claim that they are not infringing on any rights by not doing so. I claim that this is not true, as to write a usb to serial driver for Linux you have to use the drivers/usb/serial/usb-serial.h header file which is specifically licensed under the GPL v2. This file contains inline functions and structures that all usb-serial drivers need to use in order to work properly.

In short, there's no way you can write a Linux usb-serial driver, that uses the usbserial interface, without it being a derived work of other, GPL only code.

So, they are in violation, so what. Well, I can't do much about this (due to my employer's rules about suing companies). But I can do my best to spread the word that the CP2102 device is not supported on Linux, and should be avoided at all costs by anyone considering such a device in a future design. I encourage everyone else to help spread this information too.

If people are looking for a good usb to serial chip that is supported on Linux, Windows, and OS-X, there's the PL2303 device from Prolific, and the FTDI-SIO chip, and the MCT-U232 chip. All of these work very well on Linux, and are fully supported by all distros. I think they even might be cheaper than the CP2102 device too :)

Oh, and just for fun, attached to this message is the Linux driver that SiLabs is distributing, if anyone wants to poke around in it. The tarball contains 2 binary drivers, one of them a version of the usbserial.c file (which plainly is licensed under the GPL) and a mcci_usb.o binary driver. Have fun with it, but don't blame me for any badness that might happen to your box for running it, no one has any way of knowing exactly what this driver is doing.

So, in conclusion, please stay away from Silicon Laboratories devices, if you want to run Linux, as they are obviously not supporting Linux in any way.

a number of people have complained over the past few years about the fact that /sbin/hotplug was a shell script. Funny enough, it's the people on the huge boxes, with huge number of devices that are complaining, not the embedded people with limited resources (ironic, isn't it...)

Anyway, attached below is a replacement /sbin/hotplug written in c. Compiling it with klibc gives you me the following size:

$ size hotplug
   text    data     bss     dec     hex filename
   4149      28      20    4197    1065 hotplug
$ ls -l hotplug
-rwxr-xr-x  1 greg users 4636 Nov 18 15:08 hotplug

Which is smaller than /bin/true on my boxes (and /bin/true is linked dynamically, this is a static binary. Linked dynamically, it's still smaller than /bin/true. gnu programs, go figure...)

I'll be putting this all togther in a "hotplug-ng" type tarball, as I slowly replace the existing hotplug scripts with rewrites based on the fact that we've learned things over the past 4 years, and dropping support for 2.4 and previous kernels. But for now, have fun with this program.

Oh, and yeah, I know I need to fix up the fact that if /dev/null isn't present, we should create it ourselves and go from there. That's next on the list after putting it all in a project.

Yes, diethotplug is based on my old package called, supprise, diethotplug, and is located at

http://www.kernel.org/pub/linux/utils/kernel/hotplug/

But that only does module loading. The "full" replacement of today's /sbin/hotplug functionality needs this hotplug multiplexer, and a diethotplug replacement (which can get much smaller than the current diethotplug due to the way modprobe works in 2.6.)

Hi all; let me introduce myself: I'm the guy that does IDE sustaining for Google.

I'm getting ready to sit down and document the IDE ioctls. Probably Documentation/hdio.txt or something like that. Before I start though, is anybody already doing this?

I had a thought in the back of my head of tackling ioctl documentation after I went through the stuff that's already in Documentation, but I figured I had enough to chew on for right now.

I'd probably make a subdirectory - i. e. Documentation/ioctl/hdio.txt - to differentiate it from other documents, and make it easier to get maintainers to put some stuff in there. ;) AFAICT, there is next to no documentation on ioctl's anywhere in the kernel tarball.

If someone is willing to step up to the plate and get the driver working again, it would be better to have the contact information for the old maintainer available in a centralized location. Plus, anyone who tries the driver and finds it busted can look in the MAINTAINERS file and find out that it's not being fixed, instead of it being one of those drivers that has no defined maintainer (I've run across a few).

After all, we do have to be honest about unmaintained drivers - rather than hiding broken code. Honestly, it's better to define who's responsible for an area of code, even if it no longer works.

MAINTAINERS does not cover all drivers. It's not about being honest or about hiding broken code, but adding let's say fifty entries for no longer supported drivers isn't of any practical value.

At least I'm checking MAINTAINERS for people who want to receive the patches I send today, not for people who have worked on the driver at some time in the past. Adding information about orphaned drivers to this file simply makes it harder to find whether valid entries covering the driver in question exist.

If someone steps up and wants to generate and maintain a driver status document that would be fine with me. But this would require a serious amount of work and is more than just adding a few orphaned drivers to MAINTAINERS.

Actually, this is documented gcc behaviour, where a missing true condition is substituted with the condition value.

So what the above does is set "tickadj" to 500/HZ _except_ if that underflows to zero, in which case tickadj gets the value 1.

IOW, it's the same as

int tickadj = 500/HZ ? 500/HZ : 1;

except that the short syntax is not only shorter, but it's extremely convenient in macros etc, because it only evaluates the value once, ie you can do

int tickadj = *ptr++ ? : 1;

and it's well-behaved in that it increments the pointer only once.

Ahh, but Intel compiler developers are cunning, and can make (and afaik, _did_ make) icc compatible with the good gcc extensions.

And as we all know, the definition of "good gcc extension" really is "the kernel uses it". (Some of the good ones turned up in C99 and are thus no longer extensions - obviously gcc wasn't the only compiler that implemented them).

Good gcc extensions:

• the inline asm syntax (which could be better, but hey, the gcc syntax is certainly not the worst around either)
• statement expressions (but dammit, it should have used "return" to return the value)
• short conditionals
• symbol attributes (sections, "used", asm naming, etc)
• local labels and computed goto
• case ranges
• typeof and alignof.
• void * arithmetic

BAD gcc extensions:

• nested functions (gaah)
• extended lvalues (casts, conditionals and comma-operators as lvalues. They are just too confusing)
• transparent unions (dammit, you could have done it with overloading instead).

We don't, generally. But they are bad even if you DON'T use them, because they sometimes make obvious syntax errors etc much harder to debug.

For example, the "nested function" thing makes something as simple as a missing end brace cause the error reporting to be totally off, when gcc decides that "hey, that's ok, those other function declarations are just nested functions in the middle of that other function". So you get something like

file.c: lastline: parse error at end of input

even though the _real_ parse error could have been pinpointed exactly if gcc did NOT do it's totally braindamaged nested functions. IOW, the extension causes problems even when you don't use it.

Same goes for the "extended lvalues". They are not only insane, but they mean that code like

(0,i) = 1;

actually compiles. Why is that a problem? Because some people (ie me) have used constructs like this in macros to make sure that the macro is "read-only", ie you have a situation where you don't want people to mis-use the macro on some architecture. So having

int max_of_something;
#define MAX_SOMETHING (0,max_of_something)

is actually a nice way to make sure nobody does anything like

MAX_SOMETHING = new;

but the gcc extension means that this doesn't actually work.

(Yes, I've been bitten by this. And no, I don't see the _point_ of the extension - does anybody actually admit to ever _using_ comma- expressions for assignments?)

Yes, I think I've done that too, and that works. My point is that the silly comma-expression should _also_ work, and I don't see any _valid_ use of the comma-expression as an lvalue.

I suspect (but don't have any real argument to back that up) is that the gcc "extended lvalues" fell out as a side effect from how gcc ended up doing some random semantic tree parsing, and it wasn't really "designed" per se, as much as just a random implementation detail. Then somebody noticed it, and said "cool" and documented it.

That's actually in my opinion a really good way to occasionally find a more generic form of something - the act of noticing that an algorithm just happens to give unintentional side effects that can actually be mis-used. So I don't think that it's a bad way per se to add features, especially as they then are often free (or even "negative cost", since _not_ adding the feature would entail having to add a check against it).

And for all I know, many of the _good_ gcc features ended up being done that way too.

But I think the (unintentional) downsides of these features are bigger than the advantages.

Btw, the "+0" thing is something that actually might be dropped pretty early, and as such a compiler _might_ get it wrong just because it ended up optimizing the expression away. So you don't need to be all that impressed, certain trivial expressions might just disappear under some circumstances.

Side note: the _biggest_ reason why "+0" is hard to optimize away early is actually type handling, not the expression itself. The C type rules means that "+0" isn't actually a no-op: it implies type expansion for small integer types etc.

So I agree that it's unlikely to be a problem in practice, but I literally think that the reason gcc ends up considering a comma-operator to be an lvalue, but not a +-operator really _is_ the type-casting issues. A comma doesn't do implicit type expansion.

What I find really strange is the ternary operator lvalue thing, though. A ternary operator _does_ do type expansion, so that extended lvalue thing is really quite complex for ternary ops. Try something like this:

        int test(int arg)
        {
                char c;
                int i;

                return (arg ? c : i) = 1023;
        }

and think about what a total disaster that is. Yes, gcc gets it right, but dammit, what a total crock. The people who thought of this feature should just be shot.

(Yes, it looks cool. Oh, well. The compiler can always simplify the expression "(a ? b : c) = d" into "tmp = d ; a ? b = tmp : c = tmp", but hey, so can the user, so what's the point? Looking at the output from gcc, it really looks like gcc actually handles it as a special case, rather than as the generic simplification. Scary. Scary. Scary.)