Kernel Traffic #121 For 11 Jun 2001

By Zack Brown

linux-kernel FAQ (http://www.tux.org/lkml/) | subscribe to linux-kernel (http://www.tux.org/lkml/#s3-1) | linux-kernel Archives (http://www.uwsg.indiana.edu/hypermail/linux/kernel/index.html) | kernelnotes.org (http://www.kernelnotes.org/) | LxR Kernel Source Browser (http://lxr.linux.no/) | All Kernels (http://www.memalpha.cx/Linux/Kernel/) | Kernel Ports (http://perso.wanadoo.es/xose/linux/linux_ports.html) | Kernel Docs (http://jungla.dit.upm.es/~jmseyas/linux/kernel/hackers-docs.html) | Gary's Encyclopedia: Linux Kernel (http://members.aa.net/~swear/pedia/kernel.html) | #kernelnewbies (http://kernelnewbies.org/)

Table Of Contents

Mailing List Stats For This Week

We looked at 1144 posts in 5090K.

There were 415 different contributors. 188 posted more than once. 136 posted last week too.

The top posters of the week were:

1. ECN At kernel.org

21 May 2001 - 24 May 2001 (4 posts) Archive Link: "Just FYI..."

People: David S. MillerMichael PeddemorsDr. Michael WellerJohn Slee

David S. Miller announced, "vger.kernel.org is now ECN enabled." Michael Peddemors objected, "I still vote against anything that prevents someone from contributing to the linux kernel, the movement, and since these mailing lists are the 'official' way of contributing, aren't we going to far?" Dr. Michael Weller replied, "It's an interesting experiment actually: Is the linux community powerful enough to force vendors/people to fix their products and deploy updates to comply to standards or can they just ignore it." And John Slee said, "largely the vendors have fixed it. admins are often reluctant to touch a known working configuration/patchlevel however. don't blindly blame the vendor."

2. Linux On Crusoe

24 May 2001 (3 posts) Archive Link: "Transmeta Crusoe support?"

Topics: Power Management: ACPI

People: Jeff ChuaAlan CoxLinus Torvalds

Jeff Chua asked how well Transmeta's Crusoe chip supported Linux, and in particular, "whether I need to recompile everything (kernel and binaries) on my current 586 platform in order to move to Crusoe?" Alan Cox replied, "No. Crusoe should work out of the box in that sense. Its actually however not brilliantly documented for things like longrun mode where folks have actually been poking around the acpi data in order to find out how the thing works... thats the ironic part 8)" And Linus Torvalds replied, "Now, now, we released all the longrun utilities a few months ago, so the "poke around ACPI" stuff is fairly dated by now (and what the reverse- engineered code did was actually _not_ longrun at all, but "coolrun", the temperature-based stuff)."

3. The Difference Between Linus' And Alan's Trees

25 May 2001 - 28 May 2001 (7 posts) Archive Link: "The difference between Linus's kernel and Alan Cox's kernel"

People: Thiago Vinhas de MoraesWayne BrowneAlan CoxWayne BrownLinus Torvalds

Thiago Vinhas de Moraes asked about the difference between Alan Cox's tree and Linus Torvalds' tree. In particular, "Why aren't the -ac patches completely merged to the official tree, and you centralize the work on single kernel patches ?? Won't it be easier to administrate?" Wayne Browne gave his interpretation of the situation:

It really ought to be Linus and/or Alan who answers this, but from my own observations, here's the way I think it goes:

Alan and Linus don't always agree on what should be in the kernel; and even when they do, they sometimes disagree on when something is ready to be included. Alan may think a particular set of patches are ready, while Linus thinks they need to mature a bit more; or perhaps he thinks the whole approach is wrong and should be scrapped. So Alan puts it in his kernel, and Linus leaves it out of his. (Of course, sometimes it's Linus who adds something that Alan rejects.) It sometimes happens that one of these new ideas turns out better than expected (especially after going through a few bug report/new patch cycles), and the person who rejected it changes his mind and includes it later; or maybe it doesn't work out and gets dropped altogether. Also, as you've already observed, Alan regularly resyncs major parts of his tree with Linus' so they don't get too far apart, and Linus occasionally does the same.

It used to bother me, too, to have to keep up with two different kernel trees. But I've come to realize that this is a Good Thing. It provides a way for people with different viewpoints to approach an idea from more than one direction. If the two kernels are trying to solve a particular problem in different ways, we get to see how each approach works in the real world, rather than just in a theoretical discussion. If the two kernels branch too far apart it could be a problem, but Linus and Alan have been diligent about keeping that from happening. I think the interplay (is "competition" too strong a word?) between the two branches has helped make the "official" kernel better than it might have been otherwise.

Elsewhere Alan also replied to Thiago, saying:

Well it started by accident but it turns out good to have a tree that changes are merged into, tested by those who need the fixes and reviewed by third parties before they go to Linus.

So the -ac tree is kind of a peer review, testing and distillation process for patches.

4. Status Of NTFS

25 May 2001 - 29 May 2001 (15 posts) Archive Link: "ANN: NTFS new release available (1.1.15)"

Topics: FS: NTFS, Microsoft

People: Anton Altaparmakov

Anton Altaparmakov announced version 1.1.15 of the NTFS patch, and gave a link (http://www-stu.christs.cam.ac.uk/~aia21/ntfs/) . He included a changelog:

He also listed known bugs and misfeatures:

He also added, "For daring people, write support now works ok for simple files and directories. For example it is relatively safe to create a directory inside a not too big directory, and create a few relatively small files in it. umount, run ntfsfix, reboot in Windows, chkdsk will run and (hopefully) not detect any problems at all!"

In the course of discussion, it came up that there would still be problems with the driver on particular versions of NT, since Microsoft changed the filesystem from version to version; and folks discussed which versions might be affected.

5. VIA: The Saga Continues

24 May 2001 - 31 May 2001 (6 posts) Archive Link: "2.4 freezes on VIA KT133"

Topics: Disks: IDE, USB

People: Tomas StybloMark HahnAlan CoxAlbert D. Cahalan

Tomas Styblo reported that his system would freeze about three times per month under 2.4 on his Athlon 850, with 100 Mhz FSB, 512M RAM, Abit KT7A board with VIA KT133. He added, "This report is probably not very helpful, but it may be useful for those who planned to purchase AMD / VIA solution for a server." But Mark Hahn objected, "contrary to the implication here, I don't believe there is any *general* problem with Linux/VIA/AMD stability. there are well-known issues with specific items (VIA 686b, for instance), but VIA/AMD hardware is quite suitable for servers." Albert D. Cahalan, on the other hand, felt that VIA was hiding big problems with their hardware, and that no one should use them for anything until the truth was known. Mark and Alan Cox disagreed with this, and Alan said:

The big problem with VIA is not that their hardware has bugs. Everyone has bugs. I can get a problem with an intel chipset go to developer.intel.com and generally get a straight answer and often a workaround. That makes me happy. The problem isnt the bug, its not being given honest info on it.

If VIA had public errata that said things like 'Prefetch bursts can cause problems unless you set bit 3 of blah' well we'd be able to evaluate the performance impacts and people could make sensible decisions and have reliable code.

Intel are not perfect either. We have a whole pile of laptops that crash when speedstep triggers a trap we cannot handle. We have an APIC problem that took much effort because they refused to help.

When vendors do help life gets a lot easier. AMD USB was a problem due to errata. Once they published the fixes AMD USB ceased to be a problem.

Several days later, Tomas reported:

It seems the problem is caused by some DMA related bug in the VIA chipset and/or in the Linux DMA-IDE VIA driver. I finnaly get rid of the freezes, by simply compiling the kernel completely without IDE-DMA support. Now hdparm shows disks do not use DMA and the system is stable, as far as I can say now.

I've tested it VERY intensely last couple of days and did not manage to freeze it. For 12 hours a lot of concurrent processes copied gigs of data all over the disks, calculated CPU intensive crypto etc, the system hasn't frozen. For debugging purposes I also tried to downgrade to 2.2.19 with IDE-DMA activated. It crashed. So it really seems DMA is the problem here.

End of thread.

6. select() In Linux And BSD

29 May 2001 - 2 Jun 2001 (14 posts) Archive Link: "select() - Linux vs. BSD"

Topics: BSD

People: John Chris WrenAndries Brouwer

John Chris Wren reported:

In BSD, select() states that when a time out occurs, the bits passed to select will not be altered. In Linux, which claims BSD compliancy for this in the man page (but does not state either way what will happen to the bits), zeros the users bit masks when a timeout occurs. I have written a test case, and run on both systems; BSD behaves as stated, Linux does not act like BSD.

Should the man pages be changed to reflect reality, or select() fixed to act like BSD?

Andries Brouwer, man page maintainer, pointed out that the BSD behavior differed from version to version, and that John should be specific about which version gave the behavior he was reporting. He also said that the man page said, "On success, select and pselect return the number of descriptors contained in the descriptor sets, which may be zero if the timeout expires before anything interesting happens. On error, -1 is returned, and errno is set appropriately; the sets and timeout become undefined, so do not rely on their contents after an error." He concluded in a later post, "a wise programmer does not assume any particular value for the bits after an error."

7. Procfs Documentation

29 May 2001 - 31 May 2001 (6 posts) Archive Link: "[PATCH] Procfs Guide"

Topics: FS: procfs

People: Erik MouwTim WaughJeff Garzik

Erik Mouw posted some documentation and announced:

A couple of weeks ago I promised Jeff Garzik to write a piece of procfs documentation, so here it is. This guide is written in DocBook SGML and it tells you how to use the procfs from within the kernel.

I'm still looking for a proper way to automatically include the example source into the SGML file, this patch with the same content in two files is a bit of an ugly hack.

The patch is against linux-2.4.5, but should apply cleanly against 2.4.5-ac* as well.

To automatically include the source example in the document, Tim Waugh suggested, "Probably your best bet is to get the Makefile to pass a copy of the real example source through sed to &entity;ify the bits that would confuse SGML (<, >, etc), and into example.c.sed, make that into an entity, and include it." He also gave a link to some sample text (http://people.redhat.com/twaugh/docbook/selfdocbook/) that did this. Erik was very pleased, and posted a new version. Someone asked where the DocBook data could be found, as it didn't appear to be in the source. Erik replied:

That's correct, I only submitted it for inclusion into the kernel, so it's not yet there. Just patch your kernel source with my patch, run "make psdocs" and you'll have it in your kernel tree as well.

However, because we got a similar procfs question on the kernelnewbies list, I already put the html and pdf versions online on the kernelnewbies documentation pages:

http://www.kernelnewbies.org/documents/kdoc/procfs-guide/lkprocfsguide.html

8. Status Of CML2

31 May 2001 - 2 Jun 2001 (16 posts) Archive Link: "Configure.help is complete"

Topics: FS: autofs, FS: procfs, FS: ramfs, Ioctls, Kernel Build System

People: Eric S. RaymondAlexander ViroDavid WeinehallJonathan LundellRemi Turk

Eric S. Raymond announced:

It gives me great pleasure to announce that the Configure.help master file is now complete with respect to 2.4.5. Every single one of the 2699 configuration symbols actually used in the 2.4.5 codebase's C source files or Makefiles now has an entry in Configure.help.

This does not, of course, mean the job of maintaining Configure.help is done; symbols will be added and dropped in the future (there are a handful of new ones in ac5, all now documented), and some existing entries could stand to be rewritten and expanded. But we have passed a milestone -- maintainance will now be a matter of keeping the boat bailed rather than trying to ignore a hole in the side.

Thanks to all the contributors who helped put together the over 550 entries necessary to catch up, too many to name here. The result is available at:

http://www.tuxedo.org/~esr/cml2/Configure.help.gz

Though carried on the CML2 project page, it can be used with CML1 and is current with respect to both Linus's tree and Alan's.

I now have two requests of Linus and Alan:

  1. Please pick up this work now. It is a really substantial improvement on what you have in your trees, incorporating it cannot break anything, and you'll help prevent unnecessary hassles due to clashing patches in the future.
  2. Please make a policy of rejecting patches that add new configuration symbols without also adding an explanatory Configure.help entry -- and please *announce* that you will do so. We can raise our standards now, and for the sake of having a well-documentated kernel and configuration system I submit that we ought to.

Several folks were happy to hear this, and there were a few wording suggestions before the discussion skewed off to a similar documentation of the /proc filesystem. At one point Alexander Viro remarked, "We should start removing the crap from procfs in 2.5. Documenting shit is a good step, but taking it out would be better." Phil Auld asked what was wrong with procfs, and David Weinehall replied:

Imho, a procfs should be for process-information, nothing else. The procfs in its current form, while useful, is something horrible that should be taken out on the backyard and shot using slugs.

Ehrmmm. No, but seriously, the non-process stuff should be separate from the procfs. Maybe call it kernfs or whatever.

Jonathan Lundell argued, "It clearly fills a need, though, and has the distinct side benefit of cutting down on the proliferation of ioctls. Sure, it's non-standard and a mess. But it's semi-documented, easy to use, and v. general. What's the preferred alternative, to state the first question another way? For any single small project/driver, creating a new fs simply isn't going to happen." Remi Turk put in, "If I understand Al Viro correctly we'll get per driver filesystems in 2.5 (based on ramfs) which you can union-mount on /proc (possibly using autofs) to get the current /proc tree." End of thread.

9. Virtual Memory Subsystem In 2.4

31 May 2001 - 5 Jun 2001 (22 posts) Archive Link: "2.4.5 VM"

Topics: Big Memory Support, Code Freeze, Disk Arrays: RAID, Disks: IDE, Virtual Memory

People: Trever L. AdamsChristopher ZimmermanMiquel Colom PizaKen BrownfieldAlan Cox

Trever L. Adams complained, "In my opinion 2.4.x is NOT ready for primetime. The VM has been getting worse since 2.4.0, I believe. Definitely since and including 2.4.3. I cannot even edit a few images in gimp where the entire working set used to fit entirely in memory. The system now locks in some loop (SAK still works). FILE CACHING IS BROKEN. I don't care who says what, by the time swap is half filled, it is time to start throwing away simple caches. Not wait until there is no more memory free and then lock in an infinite loop. My system has 128 Meg of Swap and RAM." Christopher Zimmerman remarked, "I've found that with the latest kernel release (2.4.5) VM performance has been greatly improved. kswapd and bdflush no longer use 200% of my cpu cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero of=test.file. All of my test systems remain responsive with about 180% cpu available. These systems are running software RAID and 3ware IDE raid with 2GB of memory and 4GB swap."

Miquel Colom Piza felt strongly enough about this to post his first email to linux-kernel after years of lurking:

I don't agree with those claiming that 2.4.xx is bad or still beta.

We the administrators have the responsability to test early kernels and send good bug reports so the developers can solve the bugs. That's the way we can contribute to the community.

But it's really risky to use these kernels on MAIN 24x7 production servers.

This has been true for 1.2.x 2.0.x (I think that was the best linux kernel series) 2.2.x and 2.4.x and will be for 2.6.x also

Given we know that the support from open source developers is clearly better than commercial contract supports, I don't see the reason to complain about the work of those wonderfull hackers spending their spare time coding for all of us.

Ken Brownfield added:

I'd be forced to agree. I have 2.4.x in limited production, and with the exception of the HP/APIC fatal issues that have a "noapic" work-around, I have had no problem at all with any of the 2.4.x kernels I've used.

Open software by definition will never reach the kind of monolithic stability that years of code freeze requires. Linux (especially 2.4.x) offers too much in return, and I can always run a 2.2.x kernel. I would say that the stability of the kernel has been *above* my expectations, frankly, considering all that's changed.

It's definitely our responsibility as admins to test these kernels. I was running 2.4.0-test1 the second it was released, and the one problem I've found has been reported and investigated (it's apparently a tough one).

As far as VM, I've never had the severe issues that some are reporting. This doesn't mean it's not a problem, but it definitely indicates that it's not a global showstopper. For VM-intense applications, I roll out a 2.2.19 kernel as a preventative measure while I wait for the VM code to be tweaked. I guess I would have expected these complaints during the -test phase. Not to mention that the distributions seem to have rolled out 2.4.x just fine.

Elsewhere, Alan Cox replied to the original report, pointing out:

Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap with 2.4.

Marcelo is working to change that but right now you are running something explicitly explained as not going to work as you want

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.