Kernel Traffic #105 For 2 Feb 2001

There were 494 different contributors. 220 posted more than once. 169 posted last week too.

Heikki Lindholm completely trashed his filesystem and was very distressed. After a bit of diagnosis on the list, various folks agreed that MS Windows was the culprit. At one point Mike A. Harris said:

Whwnever you install/upgrade any OS and especially M$ ones on a multiboot machine, you should always ensure ahead of time that they will play nicely together, agree on geometry translation schemes, partitioning schemes, etc, and that any option to take over the whole machine is turned off. Windows NT defaults to "fry the whole disk", but I don't know about ME or W2K as they are IMHO just bloat + new pictures, etc..

I know if you have a 8G drive or larger, and install NT4 on it it will fry everything entirely unless you stand on your head and read about 50 MS kb articles. Thankfully, I will _never_ have to encounter this sort of thing again though. ;o)

If you have to share a machine with a Microsoft OS, the best thing is to install the Microsoft OS first. That way it can set up the partition tables however it likes. Just leave enough hard drive space free.

Then install Linux. This has several advantages - you can more easily set up Grub or Lilo to dual boot, and Linux can deal with whatever Microsoft's partition table flavor of the year is. The Microsoft OS is less likely to become confused and violently lash out using that approach :-)

Another note: If dual-booting Windows 2000, upgrade to service pack 1 before installing Linux. I was able to blue-screen W2K before SP1 by starting their disk management tool on a disk with dozens of Linux partitions. And I agree - I am thankful that I will never have to deal with this again either.

Windows 98 and possibly followons doesn't quite honor 'b' type partitions in the extended area of the disk, particularily if you are past the 8gig boundary and the partitions in question are over 2gig. The above numbers are NOT hard boundaries, I have only seen this on 2 computers and those numbers are approximate.

Generally, I have to use partition magic to make partitions past that point if I don't want windows to scribble all over my other partitions.

This is quite a nightmare, and not all that easy to diagnose or fix.

And J Sloan remarked, "This should be an FAQ - running windows on a system where you have a Linux partition is dangerous, and you run the risk of losing all your data. Any Linux system that contains important data should NOT dual boot with windows."

This subject came up recently in Issue #100, Section #5 (16 Dec 2000: Difficulties Getting Docs From ServerWorks) . This week, in the course of discussion about a Netfinity 7100-8666 using a ServerWorks chipset, Maciej W. Rozycki asked, "Does anyone beside the manufacturer have these docs at all? Last time I contacted them, they required an NDA, even though they weren't actually Linux-hostile." Dan Hollis replied:

They require not only an NDA, but that you also do all development on-site at their santa clara HQ under their direct supervision.

The only people who have ever got info out of serverworks are the lm78 guys and (i think) andre hedrick.

What magic incantations they chanted, or which mafia thugs they hired to manage this, I don't know...

At one point, Andre Hedrick said, "I can get any info needed, you just have to define the scope. Then will not can and will not give out details on a generic form. In short no one person can see the entire design docs or will they get them without a NDA. I have seen why this is the case, cause the toy are cool."

Aaron Lehmann was extremely happy to see that 2.4.0 had reintroduced ipfwadm support. But he reported, "2.[02].x used to have "special IP masquerading modules" such as ip_masq_ftp.o, ip_masq_quake.o, etc. I can't find these in 2.4.0. Where have they gone? Without important modules such as ip_masq_ftp.o I cannot use non-passive ftp from behind the masquerading firewall." Rusty Russell explained:

The entire point of the netfilter kernel architecture is that we can just ask for packets at certain points, no #ifdefs, special hacks, etc. Unfortunately, the previous masquerading code (used in 2.0 and 2.2) looked really difficult to extract from the kernel. Netfilter has changed a little since then (particularly NF_STOLEN), so it might be possible now.

So I reimplimented 2.2-style masquerading on top of the new NAT infrastructure: ideally this would mean that it could use the new helpers, but there were some minor technical problems, and it was never tested.

Later it turned out that Aaron did solve his problem using netfilter. He said, "I changed the kernel settings to have pure netfilter configuration, read the NAT-HOWTO, and followed its instructions. I reccomend that any others still trying to use the 2.[02].x style interfaces do the same." He also advocated, "netfilter seems not only much cleaner than ipchains or ipfwadm, but also much more powerful. I read into the HOWTO a bit and was very impressed by the capabilities. In particular, it's nice to have port forwarding integrated with NAT rather than as a seperate chunk of kernel code using different userspace tools."

David Luyer posted a quick-n-dirty hack against 2.4, to allow drivers to receive command-line parameters when compiled into the kernel, even if those drivers did not have an option parser in their source code. He pointed out that as far as he knew, Keith Owens planned to handle this differently in 2.5, but David felt that a solution of some kind should be available in 2.4 as well. Keith pointed out that David's patch provided inconsistant methods for setting the same parameters. Keith argued, "I can and will do this cleanly in 2.5. Parameters will be always be keyed by the module name, even if they are compiled in. Adding an inconsistent method to 2.4 then changing to a correct method in 2.5 is a bad idea, wait until we can do it right." Paul Gortmaker remarked in reply, "As a related issue, this will allow me (or whoever) to kill off the ether=x,y,z,ethN boot argument for compiled in ethernet drivers at the same time. It made sense back in 1.0/1.2 days when distro kernels were shipped with everything compiled in and ISA cards were the norm. Now it is hardly used and generally a PITA to support." Werner Almesberger also replied to Keith, suggesting that if Keith's method was not too intrusive, a 2.4 patch might get past Linus Torvalds into the official sources. But Keith replied, "It is part of my total Makefile rewrite for 2.5. A clean implementation of module parameters mapping to setup code requires the mapping of a source file to the module it is linked into. That information is difficult to extract with the current Makefile system, my rewrite makes it easy."

Neil Brown rooted around for the source of the recent corruption problems. This week he posted a patch and said:

There have been assorted reports of filesystem corruption on raid5 in 2.4.0, and I have finally got a patch - see below. I don't know if it addresses everybody's problems, but it fixed a very real problem that is very reproducable.

The problem is that parity can be calculated wrongly when doing a read-modify-write update cycle. If you have a fully functional, you wont notice this problem as the parity block is never used to return data. But if you have a degraded array, you will get corruption very quickly.

So I think this will solve the reported corruption with ext2fs, as I think they were mostly on degradred arrays. I have no idea whether it will address the reiserfs problems as I don't think anybody reporting those problems described their array.

Hans Reiser said he'd test the patch. Otto Meier tried the patch and reported non-failure. He said, "System is a SMP Dual Celeron with kernel 2.4.0. I copied 18 Gbyte of data from my backup to it. So far i have not seen any corroption messages. Last time I did that I got a lot of them. Seams that the fix has improved things for me." Someone else also reported success, though they gave no details. Holger Kiehl ran his test-set for about 16 hours, and reported no data errors, though e2fsck still found and fixed some inode problems.

Andrew Clausen complained about how many different types of partition tables there were. He pointed out that Linux had no use for partition IDs, aside from a few special cases. The Mac used them as a heuristic to find swap devices, and LVM used them as well, but he felt these were entirely unnecessary uses. He asked if anyone knew why Linux had partition IDs as opposed to probing for a signature on the filesystem. Andries Brouwer replied:

Partition IDs are not necessary. Linux works fine when you have no partition table at all, and have a parttab file in an initrd disk telling the kernel where the partitions are supposed to be.

No kernel changes required. Today you do not need partition IDs. Today you can dynamically add and delete partitions, without involving anything like a partition table.

But people use various schemes to partition their disks, mainly because also other operating systems like DOS or MacOS use the same disks. In such a situation it is useful to agree with the other OS on where the partitions are.

The dual boot situation seemed to be a serious constraint, and folks generally agreed that the current method had to stay supported.

William Thompson was unable to get Linux to compile for his Mac. He posted a patch which worked for him, and Joshua M. Thompson explained, "None of the mainstream Linux kernels compile out of the box for Mac/68K (or even for m68k in general I believe.) Go to http://www.mac.linux-m68k.org and grab a current source tarball or precompiled vmlinux.gz file (we've got 2.2.18 and 2.4.0 available, although 2.4 is still a bit shakey.)" End of thread.

Alan Cox announced 2.4.0-ac11 and added, "Slightly delayed because I had take some time out to fall off a horse.."

In the changelog, he listed "Major NTFS updates" from Anton Altaparmakov. Cataldo Thomas asked if NTFS was safe for read-only. For more on this concern, see Issue #99, Section #5 (7 Dec 2000: Read-Write NTFS Support Broken And Should Not Be Used) . Anton replied, "Of course read-only is safe. As long as you mount the partition READ-ONLY nothing can happen to it in any way, your NTFS data is at least safe." Timur Tabi pointed out that it was at least theoretically possible for the driver to send commands to the disk controller, to overwrite data, even when intending to be read-only. Anton replied that it was also theoretically possible to spontaneously teleport to the moon, but Pavel Machek said, "AFAICS, ext3 is happy to write to read-only mounted partition. So question was not completely stupid."

A while ago, Robert Holmberg had seen version 0.9.0 of the nvidia framebuffer driver appear on the "nvidia for Linux" mailing list. He'd tried it and found it fast and (seemingly) bug-free, and had waited patiently for it to be accepted into the official kernel. Some time had passed with no luck, so now he put together a bit of code and stuck a tarball on the web, in hopes that someone would submit it to Linus Torvalds. Jeff Garzik replied, "I just mentioned this to Bakonyi Ferenc <fero@drama.obuda.kando.hu>, who said that it would be better to roll a new patch without the v4l stuff, and update rivafb. rivafb is apparently stable but the v4l code is not (yet)." Robert evangelized, "I can't wait to finally get a usable rivafb. Previous versions have been both buggy and catastophically slow. This version (0.9.0) was fast enough to make things work as smoothly as without a framebuffer device."

John Levon asked what the 'mr' in 'make mrproper' stood for. After some speculation, James Lewis Nance said, "I saw a post from Linus once about this. It is Finnish for "Mr. Clean"." Matti Aarnio added:

It does refer to Procter&Gamble household cleaning product titled "Mr Proper", which apparently more recently has been renamed as "Mr Clean". (Or who knows how international companies decide on what to call the products where...)

A semi-joke which may or may not make sense to people depending on if they have seen the adverts that at least Finns have seen.. (I guess it was american advertisement dubbed into finnish.)

"'make clean' is simple soap wash, 'make mrproper' cleans also tougher stains by using stronger solvents; user is advised to protect themselves."

My recollection as a 50+ american is that the household cleaner Mr. Clean has been around since at least the 1960's. I remember vividly the TV advertisements with the bald headed genie (or whatever) with his arms crossed and inspecting dirt or cleanliness or whatever. With my US-centric upbringing, I think that Mr. Clean originated here and then went overseas, with appropriate translations of the product name, hence the appearance of Mr. Proper.

That's what I know about Mr. Clean vs. Mr. Proper. I believe it correct, but I'm not a Procter & Gamble historian.

Timur Tabi asked why 2.4.0 didn't support PAT, and explained, "The Page Attribute Table (PAT) is an extension to the x86 page table format that lets you enable Write Combining on a per-page basis. Details can be found in chapter 9.13 of the Intel Architecture Software Developer's Manual, Volume 3 (System Programming)." Jeff Hartmann stepped up and said, "I'm actually writing support for the PAT as we speak. I already have working code for PAT setup." [...] "I should have working code by the 2.5.x timeframe. I can also discuss the planned interface if anyone is interested." Timur asked if it would be possible to port this back to 2.2, and Jeff replied, "Its most likely that it will be 2.5.x only for awhile. I'm not sure at this point if it will make it into 2.4.x, much less 2.2.x." He and Timur went back and forth on technical issues for a few posts, and then they must have taken it off-line, because the thread ended in mid questi--

Mark Bratcher remembered a discussion several months ago, "about the loop device hanging when copying large amounts of data to a file mounted as, say, ext2fs. It was in regard to kernel 2.4.0test-something." He'd just now tried out 2.4.0 and noticed this bug, "although I have seen other notes that claim that it is fixed." Someone else confirmed the problem, and added that Jens Axboe had told them the bug appeared to be neither in the hardware nor in the kernel configuration. Jens replied, reaffirming this analysis and suggesting a patch. Mark tried it and reported complete success (the problem went from 100% reproducibility to 0%, which while not a guaranteed fix, at least looked pretty good). Mircea Damian also reported success, though without really extensive testing. Andreas Franck gave the patch a real pounding (he described, "I created 5 files of each 100 MB, set up 5 loop devices and made a RAID5 array out of them, putting ext2 on it and running a bonnie loop with 350 MB test size over it for the night." ) and said, "Everything survived, worked flawlessly and I'm happy my disk did too :-) Many thanks for the fine work!" Jens was very pleased, and added, "it does indeed look promising if it survives a beating like that :-)"

Elsewhere, under the Subject: Q: Release of 2.4.1, someone asked if Jens patch would make it into 2.4.1; Linus Torvalds replied, "Only if Jens sends it to me in a timely fashion (which by now means "real soon")." Jens added, "I haven't submitted it yet due to the massive restructering. Plus, it also touches other parts than loop itself."

Dima Brodsky needed to obtain and pin approximately 8M of contiguous physical memory in user space, and asked how to go about it. Dan Maas replied:

The only way to allocate that much *physically* contiguous memory is by writing a driver that grabs it at boot-time (I think the "bootmem" API is used for this). This is an extreme measure and should rarely be necessary, except in special cases such as primitive PCI cards that lack support for scatter/gather DMA.

You can easily implement a mmap() interface to give user-space programs access to the memory; there are plenty of examples of how to do this in various character device drivers.

(well OK, if all you need is a one-off hack, you can use the method developed by the Utah GLX people -- tell the kernel that you have 8MB *less* RAM than is actually present using a "mem=" directive at boot, then grab that last piece of memory by mmap'ing /dev/mem -- see http://utah-glx.sourceforge.net/memory-usage.html)

Pauline Middelink added, "Or if you want to reuse that memory between apps, use the bigphysarea patch. Yes, its available for 2.4.0 now :)" eot.

Hans Eric Sandstrom reported a 20 day uptime with 2.4.0, and Ville Herva gave his take on the 2.4.0 kernel:

I think it can be concluded that 2.4.0 was pretty good for a .0 release. IMHO and based on my own limited experience it's much better than 2.2.0. It certainly does pretty well with that kind of ordinary CPU load, and the bugs (if any) are related to more exotic conditions.

For most people, it appears to have worked _well_. It certainly has worked fine for me (I had one X lock up with it, but I'd blame the nvidia drivers for that altough they are very stable on 2.2.).

What known bugs are there btw? I think the only more serious were the software RAID5 bug and the VIA driver issues, both of which caused fs corruption. Some people reported problems with X (is this the same forking problem Jeff Merkey et all tried to hunt down pre-time), and some had trouble with booting 386 and/or other hardware. Any more?

Hopefully Linux will squash all of them in 2.4.1!

Stefani Seibold wrote and posted a patch to optionally disable all printk() messages, "by overloading the printk function with a dummy printk macro." She explained, "This patch is usefull for embedded systems, where the hardware never changes and normaly no textconsole is attachted nor any user will see the boot messages. Also, it is nice for rescue disks. On my system this saves about 10% of disk- and ramspace." After some off-line feedback, Stefani posted a new patch, moving the config option from the 'character devices' section to the 'kernel hacking section', as per David Weinehall's suggestion. She also changed the macro to call an inline function which would always return 0, for compatibility with the standard printk() function. Andrew Morton had some technical suggestions, and added, "Graham Stoney prepared a similar patch for 2.2 last year. You may be able to borrow some ideas from that work, and the followup discussion. http://www.uwsg.iu.edu/hypermail/linux/kernel/0004.2/0709.html"

This patch has now the following features:

The macro printk throws away all parameters and calls now a funciton "inline int printk_inline(void)" which always return 0 integer. So it should be now compatibel to the original printk funciton.

The macro panic also throws away all parameter and calls a function "NORET_TYPE void panic_nomsg(void) ATTRIB_NORET", which is still the same as the old panic function, but without display an oops and reboot immidiately.

All parameters of printk and panic will be automatically removed by the compiler optimizions.

The old functions printk and panic will be also implemented and exported as silent functions for backward compatibility.

The name of the config variable is now CONFIG_NOPRINTK.

The option for disabling the printk and panic messages in in menu kernel hacking.

It is a normal behaviour, that you get now some compiler warnings like "variable not used", because some of the them are only used for output by printk or panic, which will be now removed by optimizing.

Gregory Maxwell asked, "Has anyone decided to code a SFB (Stochastic Fair Blue) queue implementation for Linux? It's been implemented for FreeBSD/ALTQ (http://www.eecs.umich.edu/~wuchang/blue/). The paper for it shows it performing very well in comparison to RED." Alexey Kuznetsov replied that he hadn't heard anything about it, though the algorithm looked interesting.

Alec Smith asked, "I understand a large portion of the kernel 2.4 networking code was updated and/or completely replaced. Under 2.2 I have ipchains configured to do basic masquerading for my local LAN. Is there a straightforward guide which describes how to do masquerading and firewalling with 2.4 after moving up from 2.2?" Within 20 minutes, John Jasen answered:

http://netfilter.kernelnotes.org/unreliable-guides/

In general, you _have to_ upgrade modutils to at least a 2.3.x revision.

If you use devfs, you almost have to install devfsd, and you really need to upgrade util-linux (there's a bug in older versions of /bin/login that barf on the new tty scheme).

I usually do it by compiling and installing a 2.4 kernel; compiling, installing devfsd (and adding an entry very early in rc.sysinit for it); installing modutils; and installing util-linux -- then rebooting to test the new kernel.


Kernel Traffic Latest \| Archives \| People \| Topics	Wine Latest \| Archives \| People \| Topics	GNUe Latest \| Archives \| People \| Topics
Czech

1.	15 Jan 2001 - 26 Jan 2001	(23 posts)	Bad Microsoft Interaction With Linux
2.	16 Jan 2001 - 23 Jan 2001	(12 posts)	Difficulties Getting ServerWorks Docs: Continued
3.	20 Jan 2001 - 24 Jan 2001	(17 posts)	New Masquerading Interface For 2.4
4.	20 Jan 2001 - 24 Jan 2001	(10 posts)	Makefile Cleanup Planned For 2.5
5.	21 Jan 2001 - 23 Jan 2001	(9 posts)	Still Hunting Filesystem Corruption In 2.4
6.	22 Jan 2001 - 24 Jan 2001	(23 posts)	Necessity Of Partition IDs
7.	23 Jan 2001	(2 posts)	Linux On The Mac
8.	24 Jan 2001 - 26 Jan 2001	(15 posts)	NTFS: The Saga Continues
9.	24 Jan 2001	(4 posts)	Status Of Nvidia Framebuffer Driver
10.	24 Jan 2001 - 25 Jan 2001	(10 posts)	Linux Kernel Product Placement (Order Now!)
11.	24 Jan 2001	(9 posts)	Status Of Page Attribute Table (PAT) Support
12.	25 Jan 2001 - 28 Jan 2001	(12 posts)	Loop Device Hangs In 2.4.0
13.	25 Jan 2001 - 26 Jan 2001	(3 posts)	Mapping Physical Memory
14.	25 Jan 2001 - 26 Jan 2001	(2 posts)	Summary Of 2.4.0 Experiences
15.	26 Jan 2001 - 28 Jan 2001	(10 posts)	Option To Disable printk() Output
16.	27 Jan 2001	(4 posts)	SBF Queueing
17.	28 Jan 2001	(2 posts)	Migration Issues Between 2.2 And 2.4

Kernel Traffic #105 For 2 Feb 2001

By Zack Brown