Kernel Traffic #161 For 7 Apr 2002

By Zack Brown

Table Of Contents

Introduction

I'd like to draw your attention to this article (http://news.independent.co.uk/world/asia_china/story.jsp?story=281067) that describes some of the behavior of the new US government. "First they came for the..." etc.

Mailing List Stats For This Week

We looked at 801 posts in 3973K.

There were 362 different contributors. 143 posted more than once. 140 posted last week too.

The top posters of the week were:

1. Multithreaded Core Dumps For ELF Executables

15 Mar 2002 - 29 Mar 2002 (18 posts) Subject: "[PATCH] multithreaded coredumps for elf exeecutables"

Topics: Executable File Format, SMP

People: Vamsi Krishna S.Suparna BhattacharyaPavel MachekJun NakajimaTony LuckMark Gross

Microkernels

Vamsi Krishna S. of IBM announced:

Here is a kernel patch to support multithreaded coredumps being worked on by Mark Gross (Intel, mgross@unix-os.sc.intel.com) and Vamsi Krishna (IBM, vamsi_krishna@in.ibm.com).

Multi-threaded core dump patch for 2.4.17:

Current TODO list:

Some usage notes on this patch:

GDB 5.1 works with the core files produced, but only for Red Hat 7.2, and only if the /lib/i686/libpthread.so library is hidden. It turns out that for IA32 RedHat, that there exists 2 libpthread.so files. If the /lib/i686/libpthread.so is loaded then the gdb post mortem debug will not work. We don't understand what's going on here, but its real. Hide the /lib/i686/libpthread.so such that the /lib/libpthread.so gets loaded at debug time, and then debugger will work with the core file. Any insights into this is very much welcome. This behavior is very mysterious to us.

Thanks to Bharata B Rao(IBM) for helping with capturing FPU registers and testing and Suparna Bhattacharya(IBM) for design discussions.

Thanks to Tony Luck (Intel) and Jun Nakajima (Intel) for helping with the review and design of the suspend_other_threads implementation.

This is currently i386 only, it has been unit tested on 1P-P4, 2P-P4, and 4P-PIII systems. I haven't seen any failures so far, YMMV.

The patch is against kernel version 2.4.17. We will port this to latest versions of the kernel if there is any interest.

Pavel Machek had some implementation suggestions, and various folks talked it over. At one point, Mark Gross mentioned that he'd begun work on the Itanium version of the patch.

2. Disk Trouble At BitMover Affects BitKeeper Repositories

27 Mar 2002 - 29 Mar 2002 (12 posts) Subject: "bkbits.net down"

Topics: BSD: FreeBSD, Clustering, Digital Video Broadcasting, Disk Arrays: RAID, Disks: IDE, Version Control

People: Larry McVoyHenning P. Schmiedehausen

Disks

Larry McVoy (posting as root) reported that bkbits.net was having problems. He said:

It looks like we have a bad disk, I'm checking them now to figure out if it is the the primary or backup data drive. I'll run checks in all the repositories if fsck doesn't find the problem so it may take a couple of hours before we are back up.

In the not so distant future, we're moving the backup drive to a different machine such that we can just flip machines when this happens but for now you'll have to wait for a bit.

Half a day later he reported (from his user account):

We did indeed lose the primary disk (IBM 40GB, I am starting to lose all the respect I had for IBM drives, this is one of many that has failed on me personally). I have restored from the backup disk, and in the process redone hardlinks across all the linux kernel trees, which saved about 5GB (nice). All trees which are now on bkbits.net check clean, which means BK thinks all the files are there and that the checksums are correct, a fairly reasonable indication that we are in good shape.

I wouldn't be a bit surprised if we have some permissions problems, mail support@bitmover.com if you hit any and we'll fix things as we become aware of them. In fact, I know we have permissions problems but given that I've been working on this for 12 hours straight, I'll get to it tomorrow.

There are a couple of trees which are missing files, both in Rik's linuxvm.bkbits.net, I suspect an interrupted clone. They are:
bk://linuxvm.bkits.net/linux-2.5-vmtidbits
bk://linuxvm.bkits.net/linux-2.5-writethrot
Rik, ping me if you need help cleaning these up.

The ppc tree seems to be missing linuxppc_2_4, Paul/Tom/Troy/Cort, where is this tree? You'll want to get a copy back here, I suspect, so if you are a PPC person and you have a recently updated version of linuxppc_2_4, hang on to it. We'll sort it out on the ppc mailing list.

We have lost a number of ssh keys. We backed these up a while back but we did not catch all of these. The list is below, send me mail with your ssh key / project name and I will restore them by hand. We already had plans in place for dealing with this problem so that it doesn't reoccur.

Sorry about the long downtime, we are struggling with the economic downturn like everyone else and hadn't put a hot spare in place yet. We bought them, in fact, I bought ten spare boxes for this sort of thing, but I have been too busy to put them in place. We'll get on it, we're aware that people depend on this.

Here's the list of projects missing ssh keys:

bcrlbits
freebsd-dvb (probably not Linux, eh? :)
lia64
linux-mtd
linux-srn
linux24 (I think this one is dead, right Marcelo?)
ltr
misc
nonblock
palinux
test1

If you are the admin for any of these projects, drop me a mail with your ssh key and I'll add it back in.

The next morning, he reported:

Leaving the drive off overnight "fixed it" enough that I am able to get some of the data off. It will be a couple hours before I know how much, but I did manage to get all the ssh keys, project descriptions, and project statistics. I'm now working on the actual data just in case there is one of the trees, such as the ppc trees, that we can't find again.

The drive has bad blocks and when it hits them it goes into retry la la land, so I won't know which data is bad until I hit the bad blocks.

Henning P. Schmiedehausen remarked, "You've learned now the hard way why integrity checks in an application will never be able to replace things like backups or RAID systems. Maybe you want to reread the flamewar^Wthread from some time ago with your new knowledge." And Larry replied:

You obviously didn't read that thread. Both in the context of BitKeeper and in the context of normal data, you would have seen that we have backups, we just have backups that we can verify are correct. The repositories on bkbits.net are automirrored after each incoming event. There were a few ppc ones which were not and we're still trying to figure out why, and things like the .ssh keys were not completely backed up; we're fixing that by putting that information into a BK repository so it will just automirror like everything else.

I'm not sure why you yanking my chain, it's counter productive and flat out rude after I just spent two days doing nothing but putting things back together for kernel developers. What, exactly, did you hope to accomplish?

Henning said that he'd hoped for:

Awareness on your side that there are people presenting valid arguments to you even if you don't agree. I did read the first posts of the thread up until it degraded to "our application does every integrity check possible to verify that the data is correct. So even if it gets corrupted, we will know. That's why we better than the rest" and you shot down everyone presenting you with other solutions with this arguments. Well, in this case, you obviously knew that the data is incorrect (because the disk died, I'm really feeling with you here, from my eight IBM DTLA disks, three have died, too and I fear that the remaining five will also die) but all your checks couldn't help you where just a few up-to-date backups would have.

Actually I'm a bit disappointed too, that you with all the professionalism that you sprikle over this mailing list in every of your posts, run such a showcase part of your business as bkbits.net on IDE disks without RAID. And without clustering in case of emergency.

Larry said, "Go reread my posts and stop wasting my time." There was no reply. Elsewhere, only a day and a half after his original post, Larry also reported:

I think we are back in action. We put all the ssh stuff back. As well as the download statistics, take a peek at www.bkbits.net, your stuff should be there.

Let me know if your project is missing anything, I know about the ppc tree, we have that data, that's next. But other than that, everything should be back, let me know if that is not true.

3. 2.4.19-pre4-ac2 Boot-Time Lockups With ALI15x3 Support

27 Mar 2002 - 29 Mar 2002 (9 posts) Subject: "[bug] 2.4.19-pre4-ac2 hang at boot with ALI15x3 chipset support"

Topics: Debugging

People: Andre HedrickAlan Cox

James Mayer reported that 2.4.19-pre4-ac2 would hang at boot-time, with the ALI15x3 driver on a Sony PCG-C1MV/M, immediately after printing "ALI15X3: not 100% native mode: will probe irqs later". Without ALI15x3 aupport compiled in, the kernel would boot fine. Brian D Heaton had solved the same problem on his Lifebook P-2040 with a patch he'd found online, and promised to dig around for it. Andre Hedrick also said to James, "This problem is being addressed at ALI with the help of Sony. I can not tell you the issue because I have not been authorized." Elsewhere, Alan Cox led a charge on linux-kernel to hunt the bug down publically. At one point Bruce Howard, another "me too", updated his page of pcg-c1mr1 and pcg-c1mv configuration (http://hale.org/~bhoward/issue_7/pcg-c1mrx.html) to include his solution to the problem; and James confirmed that Bruce's solution worked for him as well.

4. DMCA Impact On Kernel Development

29 Mar 2002 (5 posts) Subject: "Changelog for 2.2.20?"

Topics: Legal Issues

People: Alan CoxMike FedykDavid Rees

David Rees was unable to find the changelog for 2.2.20 on kernel.org, and Alan Cox said, "For non US citizens its available on http://www.thefreeworld.net" Rasmus Bag Hansen asked how it could be possible that the patch would be legal in the US, but the changelog would not be. Mike Fedyk replied, "Basically, the politicians can't read the patch, but they might be able to understand the summary... Also, in many cases the change that fixes the security hole doesn't make exploit ideas obvious. While many times the security report includes the expliot itself." End of thread.

5. Mailing List Troubles

30 Mar 2002 (2 posts) Subject: "Majordomo@vger.kernel.org down?"

Topics: Mailing List Administration

People: George AnzingerMatti Aarnio

Networking

George Anzinger reported, "I seem to have been "pruned" from the list sometime Wed. and Majordomo@verger.kernel.org is ignoring my attempts to re subscribe me. What gives?" Matti Aarnio replied:

For past 5+ days VGER has been unable to connect any MX server of your domain. Why ? I have no idea. Diagnostics says: "connection timed out".

Somebody has tweaked a firewall there, and is now rejecting connections with TCP/ECN ?

Use http://vger.kernel.org/mxverify.html tool to see how things are working.

EOT

6. BKL Cleanup In Filesystem Code

31 Mar 2002 (3 posts) Subject: "VFS locking changes in 2.5.7"

Topics: FS: devfs

People: Alexander ViroRichard Gooch

Richard Gooch asked Alexander Viro about some changes Alexander had made to the locking rules of the Virtual Filesystem (VFS). Richard noticed that Alexander had moved the Big Kernel Lock (BKL) calls into the devfs code, but that (Richard felt) not all of those locks were actually needed. He asked Alexander what the thinking was behind the changes. Alexander replied:

See Documentation/filesystems/porting

BKL had been shifted inside several methods, so that filesystem code itself had the same locking as it used to (i.e. code that used to be under BKL stayed under it). If your code doesn't need BKL - feel free to shrink the area, but keep in mind that it used to be under BKL.

I didn't _add_ BKL - neither in devfs nor anywhere else. lock_kernel() is the boundary of the protected area and all that had happened is that this area had slightly shrunken, so its boundaries are inside the method instead of being around its caller.

Again, further shrinking is up to maintainers of the filesystems.

This all made sense to Richard, and he said he'd look into removing some of the excess locking in the devfs code. End Of Thread.

7. Quotas With Journaling

1 Apr 2002 - 2 Apr 2002 (3 posts) Subject: "Status of quotas on ext3 and reiser?"

Topics: FS: ReiserFS, FS: ext3

People: Luigi GenoniAndreas DilgerKen Brownfield

Ken Brownfield very much wanted to use journalling on his 2 terabyte disk array. But he also needed to use quotas, and wasn't sure if they worked with either ext3 or reiserfs. He asked if those filesystems would support quotas well enough to handle a production environment. Luigi Genoni replied, "I am using quota with reiserFS and quota tool 3.04 from slackware-current, and no problems at all (kernel 2.4.18)." And Andreas Dilger added, "If you have a RH kernel (or any other -ac kernel) you will get 32-bit UID support for quotas."

8. Status Of util-linux Maintainership

3 Apr 2002 (6 posts) Subject: "[OT] who's maintaining util-linux?"

Topics: Maintainership

People: Tigran AivazianRagnar Hojland EspinosaAdrian BunkAndries BrouwerJohn SleeDenis Vlasenko

Denis Vlasenko had some problems with util-linux, but was unable to contact the maintainers. He asked if anyone was in charge of that project. John Slee found some references to Adrian Bunk in the Debian package, but a couple folks pointed out that the Debian package was not the actual upstream project, only the packaging of it for a particular distribution. Ragnar Hojland Espinosa seemed to recall that Andries Brouwer was the actual maintainer, and Tigran Aivazian also said this. Andries did not come forward, however.

9. Getting Rid Of The BKL

3 Apr 2002 (3 posts) Subject: "[RFC][PATCH] BKL reduction in do_exit"

People: Dave HansenLinus Torvalds

In the ongoing effort to replace the big kernel lock (BKL) with various less drastic forms of locking, Dave Hansen posted a patch and announced:

A week ago, I posted this: http://groups.google.com/groups?selm=linux.kernel.3CA20C9B.20309%40us.ibm.com

Nobody had anything to sayabout it, so here's a patch. It moves the disassociate_ctty(1) up, and releases the BKl after it gets done. Is this a sane thing to do, or do some of those exit_*() functions still need the tty?

The patch reduces hold times of the BKL in do_exit() by a factor of 100. They were on the order of 200us, now they're about 1.5us. However, those numbers were on Martin Bligh's NUMA-Q box, so they represent a serious worst-case scenario.

Linus Torvalds replied:

I'd prefer to have the BKL just moved into the functions that need it, and removed altogether from do_exit().

That's especially true as I don't know if sem_exit() actually needs the BKL any more at all - so that if it doesn't, we can just remove it from there (at which point it is a local implementation issue, rather than a cross-module thing).

The disassociate_tty thing falls under a similar heading - we're going to have to fix up the tty layer some day anyway, let's make the BKL detail a tty layer internal thing.

Dave agreed with this and posted a new patch. End of thread.

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.