Kernel Traffic #152 For 28�Jan�2002

This is something of a bombshell. Not necessarily a bad one, but...

Alan, do you have *any* *freakin'* *idea* how much more complicated the CML2 deduction engine had to be because the basic logical entity was a tristate rather than a bool? If this plan goes through, I'm going to be able to drop out at least 20% of the code, with most of that 20% being in the nasty complicated bits where the maintainability improvement will be greatest. And I can get rid of the nasty "vitality" flag, which probably the worst wart on the language.

Yowza...so how soon is this supposed to happen?

Dang. This will make the CML2 inference engine work better in some funky corner cases, too. And its behavior will be easier to understand all around.

Sign me up. This will be a good change; I like it when I can make things better by taking features *out* of my code.

I've seen several times on this list people wondering what features were in the works for 2.5 and what the status of the development was. I did some grepping on the archive and put together a list of things that have been discussed / worked on for 2.5 over the past year or so.

It's probably pretty incomplete and full of errors at this point but I'll be happy to update it if you send me email.

1. Merged New scheduler for improved scalability (Ingo Molnar)
2. Merged Rewrite of the block IO (bio) layer (Jens Axboe)
3. Merged New kernel device structure (kdev_t) (Linus Torvalds)
4. Merged Initial support for USB 2.0 (Greg KH, others)
5. Ready Add User-Mode Linux (UML) (Jeff Dike)
6. Ready Add ALSA (Advanced Linux Sound Architecture) (ALSA team)
7. Ready IDE layer update (Andre Hedrick)
8. <1 month New kernel build system (kbuild 2.5) (Keith Owens)
9. <1 month New kernel config system: CML2 (Eric Raymond)
10. Beta New driver API for Wireless Extensions (Jean Tourrilhes)
11. Beta New IO scheduler (Jens Axboe)
12. Beta Add JFS (Journaling FileSystem from SGI) (JFS team)
13. Beta New VM with reverse mappings (Rik van Riel)
14. Beta Add preempt kernel option (Robert Love)
15. Beta Add resheduling points to remove latency (Andrew Morton)
16. Beta Build option for Linux Trace Toolkit (LTT) (Karim Yaghmour)
17. Beta Better event logging for enterprise systems (evlog team)
18. Ongoing Better support of high-end NUMA machines (NUMA team)
19. Alpha Add Asynchronous IO (aio) support (Ben LaHaise)
20. Alpha Integrate EVMS into kernel (EVMS team)
21. Started Rewrite of the framebuffer layer (James Simmons)
22. Started New driver model & unified device tree (Patrick Mochel)
23. Started Rewrite of the console layer (James Simmons)
24. Started More complete NetBEUI and 802.2 net stacks (Arnaldo C de M)
25. Draft #2 New lightweight library (klibc) (Greg KH)
26. Draft #3 Replace initrd by initramfs (hpa, Al Viro)
27. Planning Change all drivers to new driver model (All maintainers)
28. Planning Add thrashing control (Rik van Riel)
29. Planning Remove all hardwired drivers from kernel (Alan Cox, etc)
30. Planning Porting all input devices over to input API (James Simmons)
31. Planning generic parameter/command line interface (Keith Owens)

I hope this is helpful. Enjoy!

Great !!!. I think this "todo list" was much needed. Let me suggest a couple of things that should find their way into 2.5.x sometime in the futute, and that I consider important:

• Generic (VFS layer?) ACL implementation: it seems under heavy develpment by some groups, and could be something quite interesting to have for 2.6.x
• Linux Security Module (LSM)
• LVM 2: it seems LVM developers are about to release a first version of the new (version 2.0) LVM (Logical Volume Management) implementation for Linux. Seems to solve the problems current code base has.
• devfs 2: new implementation of Richard Gooch's devfs for Linux. devfs is a "must have" for some people, and this new code seems to solve everything that people complained about. AFAIR, Richard has submitted several "cuts" of the new code.

There are other interesting things I don't follow as closely, and that could be available when 2.5.x is still under development, maybe not.

• reiserfs 4 (later this year): maybe too late for inclusion.
• USAGI IPv6 implementation for Linux: maybe not ready for prime time.
• NFS v4: have read about it, but don't remember about deadlines.
• IPsec in kernel (FreeS/WAN): the neverending story :)

That is what I think are the most important things that _could_ be part of future 2.6.0, should maintainers consider it as a good idea. Please, this is only informative, not trying to "sell" you nothing.

If Linus says september freezes in september and ships for christmas I will be most suprised. If he says september freezes the may after and ships the december after that I'd count it normal

Personally I'd really like to see the block I/O stuff straightened out. The neccessary VM bits done, device driver updates and a September freeze. I think it can be done, and I think the resulting kernel will be way way better for people with 1Gb+ of RAM, so much better that its worth making a clear release at that point.

kernel.org stopped responding to most services some time this morning. Unfortunately, the system management port on the new server was never connected at the time the new server was installed, so we can't debug the problem remotely.

I am trying to reach people at ISC to get the management port connected and the system back to normal.

Since it looks like it is going to take some time to resolve the wiring problem affecting kernel.org, I have put the kernel.org system in "mirrors only mode." This means mirror sites can update, but no user traffic is allowed -- except obtaining the list of mirrors.

This wiring problem is an unfortunate consequence of having had to push the new kernel.org server into service on a lot shorter notice than expected back in December. Once we can get it resolved these kinds of problems should hopefully not recur.

The recent troubles we've had at kernel.org pretty much highlight the issues with having an offsite system with no easy physical access. This begs the question if we could establish another primary kernel.org site; this would not only reduce the load on any one site but deal with any one failure in a much more graceful way.

Anyone have any ideas of some organization who would be willing to host a second kernel.org server? Such an organization should expect around 25 Mbit/s sustained traffic, and up to 40-100 Mbit/s peak traffic (this one can be adjusted to fit the available resources.)

If so, please contact me...

No kidding. I didn't realize they were hosting it, that's very nice of them.

Do you have any statistics on what percentage of the download traffic is whole kernels versus patches? If most of the traffic is whole kernels, I think I might be able to offer up a fix for that.

It has come to my attention there are some unfounded rumours about the kernel.org outage from this past weekend, and I wanted to set the record straight:

In particular, the outage was *not* caused by any kind of failure in the new Compaq server hardware. It worries me a bit that this particular rumour was circulating, since it reflects badly on a donor who has just provided us with a very nice machine.

The approximate history of the failure is as such:

Back in December, we were already planning to replace the old server hardware after repeated problems, probably age-related. We were originally planning to put the new server in production after the holidays, however, when the old server really started to flake out on us we decided to push it in service early.

ISC was very accomodating and arranged for us to put it in service on short notice, despite several logistics problem. One of those problems was the lack of a configured port for the management card in the new server. As a result, it did not get wired up at that time.

This past Friday morning, the kernel on the server apparently stopped servicing user-space processes; the details aren't known, other than the fact that pings and TCP SYNs received replies, but we couldn't get any actual data across. Futhermore, on and off there was as much as 95% packet loss in pings, although the machine didn't stop responding to pings until it was power cycled on Sunday.

Due to a miscommunication between myself and the staff at ISC we weren't able to get the machine power cycled until Sunday (it did not return from the power cycle) and the management port connected until Monday. Once the management port got connected, it was a 5-minute job to bring the machine back to life.

Finally, I would like to thank the many people and organizations who have offered to host another kernel.org server in one way or another. I'm going to be evaluating our options, with the goal of getting at least one additional server if at all possible. If so, my preference will definitely be to try to obtain identical hardware with the one we currently have.

Due to a private flame-war about configuration options and behaviour, I just broke down and did the thing I had asked for from others for a long time - split up the huge unwieldly "Configure.help" file into many smaller ones that are closer to the option they describe.

The split was largely automated, with one 25-thousand line file being split up into almost 100 smaller files. The placement was also entirely automated, and basically moved each entry to the same subdirectory where the config question for that entry was located (and if the question was duplicated over several places, the help was duplicated too).

This should make it _much_ easier to have things like per-architecture config entries that the rest of the world simply doesn't want to know about and has no interest in.

[ The automated nature of the split also showed that some questions are sometimes oddly placed. Some minimal movement was done to fix the worst offenders, but hopefully we can organize some of it more logically in the future. ]

However, I would ask that maintainers of config tools, drivers and architectures would check that their config help entries still exist, and seem to work. Of the tools, only the basic "make config" script has been updated to know about the change in location, so menuconfig/xconfig simply will not find the help right now.

Oh, non-x86 architectures probably need to fix up their config menu entries too, as I only did a minimal "move architecture-common entries to init/Config.in" edit (to not duplicate all the questions/help messages that are common to all architectures), and I'm absolutely positive that the menu structure didn't move correctly.

Other changes tend to get overwhelmed in the patch (happily it's not every day that you clean up the largest file in the whole archive), but ChangeLog appended so you can see what else happened.