Kernel Traffic #291 For 4 Jan 2005

We need an -rc3 yet. And I need to do another pass through the regressions-since-2.6.9 list. We've made pretty good progress there recently. Mid to late December is looking like the 2.6.10 date.

We need to be be achieving higher-quality major releases than we did in 2.6.8 and 2.6.9. Really the only tool we have to ensure this is longer stabilisation periods.

Of course, nobody will test -rc3 and a zillion people will test final 2.6.10, which is when we get lots of useful bug reports. If this keeps on happening then we'll need to get more serious about the 2.6.10.n process.

Or start alternating between stable and flakey releases, so 2.6.11 will be a feature release with a 2-month development period and 2.6.12 will be a bugfix-only release, with perhaps a 2-week development period, so people know that the even-numbered releases are better stabilised.

We'll see. It all depends on how many bugs you can fix in the next two weeks ;)

The below patch allows to set the HZ dynamically at boot time with command line parameter. HZ=1000 HZ=100 HZ=333 any other value just works (though certain value may cause more or less drift to the system time advance/decrease).

Is there any interest from the mainline developers to merge this into 2.6? I'm getting requests for this feature being forward ported to 2.6 (both for batch jobs and for the powersaved that can trim the hz down to 80mhz). It should be up to the user to choose the HZ like it was in 2.4-aa.

This patch is quite intrusive since many HZ visible to userspace have to be converted to USER_HZ, and most important because HZ isn't available at compile time anymore and every variable in function of HZ must be either changed to be in function of USER_HZ or it must be initialized at runtime. The code has debugging code (optional at compile time) so that I can guarantee that there cannot be any regression.

Technically this makes a lot of sense to me (well, you can guess why I implemented it in the first place), at least in archs where one cannot reprogram the timer chip in a performant way (to stop timer ticks completely until the next posted timer). This is in production for years in SLES8 btw.

http://www.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.23aa3/9999_zzz-dynamic-hz-5.gz

Sure, desktop doesn't need this, the reason somebody is asking for it, is that the desktop stuff hurted some other non-desktop usages. Infact my 2.4 tree was setting by default HZ=1000 if 'desktop' paramter was passed to the kernel (so that I could lower the timeslice accordingly too, without losing the effect of the nicelevels between nice 0 and +19).

The other new case where I'm asked for this feature is again not the desktop but the high end laptop with cpu throttling down to 80mhz, and what Pavel mentioned about the lower consumption. Perhaps we could do variable HZ there, though I doubt it has a pit that can be reprogrammed with sane performance.

Very few people are going to get real benefit from HZ=1000, but I certainly agree it worth to keep HZ=1000 on desktops since on a idle machine the downside of the more frequent irq sure isn't measurable, while having shorter timeslices may be visible with many tasks, and shorter timeslices requires faster HZ to preserve the nicelevels.

There are apparently some laptops which exhibit appreciable latency between the start of ACPI sleep and actually consuming less power. The 1ms wakeup frequency will shorten battery life on these machines significantly. (I forget the exact numbers - Len will know).

So I guess we're going to have to do this sometime - I don't think there's any other solution apart from going fully tickless, which would be considerably more intrusive.

We should retain the option of compile-time constant HZ - it's easy enough. Probably the patch already does that.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.10-rc3/2.6.10-rc3-mm1/

• Lots of new patches, lots of little fixes all over the place.
• Probably the major change is the readahead rework, which may have significant performance impacts on some workloads. Not necessarily good, either...
• See below for the list of 31 patches which I have pending for 2.6.10. If there are other patches here which should go in, please let me know.

Ioctls do have some serious problems that make them nice to avoid

1. Each ioctl handler has its own data structures. While you could write XML objects to encapsulate this in write() it is also true in many cases that there is a simple logical expression of the operation - eg configuration options tend to fit well into files as you can see with /sysfs - unless they need to be atomic transactions with rollback at which point the same people who decry ioctl will hate embedding sqlite in the kernel

   Seriously however - multiple structures means multiple validation functions means more new code and more errors. It's a lot easier to get ioctls wrong. There are a lot of things that don't need to be ioctl. A look at security history says in general "ioctls cause bugs"

2. Ioctl structures tend to be binary. Welcome to 32/64bit emulation hell. Good design can avoid this. Good design is not XML for this purpose.
3. Ioctl is unstructured and so each ioctl is a new mystery to the programmer. We all know how write works and in many cases echo "451" > /proc/sys/vm/blah is quite obvious.
4. It's hard to ioctl from the command line or scripts

The "ioctls are evil" blind hate department really annoy me however because like all extreme views the truth very rarely fits their model

It contains a relatively small number of changes: XFS sync, SPARC64 sync, USB gadget updates, couple of libata fixes, amongst others.

Also a networking update, containing fixes for following recently discovered security issues:

CAN-2004-1137
IGMP vulnerabilities - local priveledge escalation and remote DoS:
http://isec.pl/vulnerabilities/isec-0018-igmp.txt

CAN-2004-1016
scm_send local DoS:
http://isec.pl/vulnerabilities/isec-0019-scm.txt

Further small fixes for different minor things. A merge of some of the small cleanups from Fedora work and also the fixes for the igmp and vc holes.

Arjan van de Ven is now building RPMS of the kernel and those can be found in the RPM subdirectory and should be yum-able. Expect the RPMS to lag the diff a little as the RPM builds and tests do take time.

The HPT366 rework project is also not ready (its gone back to the drawing board until the current panic is over if you are a volunteer and wondered what is up).

ftp://ftp.kernel.org/pub/linux/kernel/people/alan/linux-2.6/2.6.9/

AFAICS the -ac tree should be the most stable of all kernels, right?

-mm is totally bleeding edge
-bk the same
-ck is experimental

Others are experimental too.

Looking at the changelogs, the most reasonable kernel to use for generic use are the -ac kernels, which I am going to use since 2.6.10 as long as Alan is kindly going to continue his fabulous work.

I swear not to use 2.6.10 until Alan publishes 2.6.10-ac1 :-)

2.6.x-mm is more like some of the work the old 2.4-ac did in merging new stuff (its also worth noting that 2.4-ac ended up more stable than 2.4 at times so -mm might be stable)

The -ac tree is trying to be fairly conservative. When I merge stuff that is a little less conservative because it has to be done then I've tried to put a note in the relnotes for that release warning people its more testing grade.

This is usbmon which I cooked up because I got tired from adding dbg()'s and polluting my dmesg. I use it to hunt bugs in USB storage devices so far, and it's useful, although limited at this stage.

I looked at the Harding's USBmon patch, and I think he got a few things right. The main of them is that I underestimated the benefits of placing the special files into the filesystem namespace. When we discussed it with Greg in the airport, we decided that having some sort of Netlink-style socket would be the best option. I decided to make a u-turn and attach those sockets into the namespace (currently under /dbg, but it can change). What this buys us is:

1. cat(1): never bet against it. It's too handy. And netcat is just not the same.
2. USBmon userland in Java. Just try to hack in JNI a little as I have and you'll see.

He also got some parts wrong. They are small things, but unfortunately, pervasive. For example, he relies on urb->dev, which is not a good idea in case of HCD which zero it far away from the completion call site, such as usb-ohci in 2.4. And it's error-prone and a maintenance problem to audit all HCDs and add usbmon calls. Races by design, too. Small things like that, but many. Eventually, I wrote everything from scratch. It's rather embarrassing that I could not save USBmon and gave in to NIH.

Since it's a big NIH, usbmon is not compatible with USBmon's userland. It can be made compatible, but it needs a small adaptation layer, because Harding aggregated at a device, and I do it on a bus (I can explain why, but it's rather long; it has to do with hotplug and races).

The architecture to support various output formats is present. Obvious candidates are Old USBmon format and a Binary format. But it's not done.

Please ask if something is not obvious in the code.

Bah, fine, make me make a policy decision, damm I tried hard to resist :)

Anyway, here's a patch I just applied that creates the /sys/kernel/debug directory (you need a small patch that exports the proper subsys for this to work, if anyone wants that too, I'll send it.) Now, if you want, you can mount debugfs at this location.

Now either this is going to make people happy, or make them mad I didn't pick their proposed location. Either way, I'm going on vacation in 2 days, so I will not be around to hear the screams...

Here goes the third -pre of 2.4.29.

More importantly this release contains a correction for the "int 0x80 hole" security problem in AMD64 port (CAN-2004-1144).

It also contains a few important v2.6 backports (tty/ldisc and pty races), some hardening patches from Solar (none of those are exploitable bugs, just paranoic/early error detection), and a few networking updates.

This release should also fix the "NFS hang on unlink" issues present in v2.4.28.

It should appear in the kernel.org mirrors in a few minutes.