Kernel Traffic #35 For 20 Sep 1999

Well, the migration to Linuxcare is complete, but there are still more layout changes to be done. Expect subtle changes over the coming weeks. Your suggestions and reactions will have influence, although I've received only a handful of responses so far.

There were 439 different contributors. 174 posted more than once. 128 posted last week too.

Enclosed please find updates to the serial driver versus Linux 2.3.15. This driver adds support for PCI serial boards, and the following new UART's: Oxford Semiconductor 16C950, the StarTech 16854, and the StarTech 16654.

I've coordinated various PCI patches from Otto Moerbeck, Henning Schmiedehausen, Stuart MacDonald (from Connect Tech) and others, and the merged driver has been tested by a number of folks before I've submitted it to you. I therefore expect it to be BugFree(tm), although some further updates to support additional PCI serial boards may be coming in the next couple of weeks.

Rogier Wolff replied, "I've reviewed your patch, and "ported" it to fit into linux-2.2.x. Was it your intention to get the patches into 2.2.x? It sure looks as if it was "designed" with that in mind. Or do you want it to get some testing in 2.3.x first?" Six days later Ted replied (he'd been travelling around the world for various events like the Linux Kongress and the Storage and Filesystem Workshop), "Yes, the serial driver was designed to work for both Linux 2.3 and Linux 2.2. I ship the serial driver in a stand-alone package where you can compile it as a module separately from the kernel tree, and that was the original way that I had assumed people with 2.2 kernels could use the new serial driver. But yes, it can be put into the 2.2 kernel without much difficulty."

Rogier actually replied four times to Ted's initial post (and was the only person to reply, though others participated in the subsequent discussion). In the reply that led to the main thread, he noticed that Ted's patch included a "static int tty_get_baud_rate(struct tty_struct *tty)" function, and begged, "Could you PLEASE, make that a published, exported function? That would be really nice. Nowadays every serial driver does this for himself." Ted replied that it had been a published, exported function since kernel version 2.1.66, adding, "the code which you're quoting from the serial driver is compatibility code which is under a "#if (LINUX_VERSION_CODE < 131394)". Each serial driver should *not* be doing itself, but should be using tty_get_baud_rate() (defined in drivers/char/tty_io.c) instead. It's even published in /proc/ksyms so that modular drivers can use it."

Rogier was a little embarrassed, and went on to submit PCI IDs for the OXFORD Semiconductor and OX16PCI954 cards. He also suggested, "May I ask for a "name" field in your PCI-table? Then we can say something like "Detected an xxxx card". That make supporting such a card a lot easier: People get reassured that their card got detected."

Ted replied, "The reason why I haven't done this is because I don't want to bloat the kernel with overly large tables; if past history is any guide, I have no doubt that the number of PCI serial boards will be *large*." A discussion started, but Linus broke in with:

Has anybody bothered to check the current PCI code?

"struct pci_dev" contains the name of the PCI device ("dev->name", surprise, surprise), and it has been copied there from __initdata information that is discarded after bootup so that we don't waste run-time memory on keeping all the PCI names around when they aren't needed (ie only the devices that were found have the name).

"oldproc" is gone - because the name is there, and it's there to stay. Exactly because there are so many cases where it's just nice to have the information available (/proc/iomem, device driver diagnostic messages etc).

Rogier asked for a brown paper bag, and there followed a bit of discussion on modules and other kernel implementation details.

Manfred Spraul noticed that the bigmem and rawio patches seemed to be incompatible, and Andrea Arcangeli explained:

The thing is not black and white. It can be compatible or incompatible, it only depends about the internal details of the blockdevice driver and on which memory you are going to do raw-io. It only depends on the source and on the destination of the raw-IO. If both the source and the destination are OK, then you can do raw-io fine also with BIGMEM enabled. The current code is just completly stable, you can't harm the kernel trying to do raw-io on the wrong place. If the source and the destination are not compatible, then you'll get an -EIO from read(2) or write(2).

For example if you know for sure that the blockdevice where you want to write in raw-io mode does DMA driven I/O, then you can just now do raw-io from or to bigmemory _fine_ on _such_ blockdevice using 2.3.16. The only thing you have to do to experiment this, is to comment out these lines in buffer.c of 2.3.16:

                        if (map && PageBIGMEM(map)) {
                                err = -EIO;
                                goto error;
                        }

then you'll be able to do raw-io everywhere as you can do with bigmem disabled.

WARNING: after removing the above lines you must make sure to not do raw-io from or to bigmemory with blockdevices that are _not_ using DMA or you'll get in troubles. So if you are unsure don't remove such lines, _never_.

And you can _safely_ use raw-io in a _stock_ 2.3.16 with bigmem enabled too; you won't risk to crash the kernel but you'll get a graceful -EIO in the _worst_ case. You can get -EIO only trying to do raw-io on _shm_ (as big-databases) or _anonymous_ userspace memory. If you want to do raw-io from/to framebuffer memory, that will work just fine also with bigmem enabled.

Manfred had also asked if there were plans to change the situation, and Andrea explained:

Right now the above check is obviously right and safe but it's strict. Maybe it worth to add a per-blockdevice bitflag to allow raw-io on bigmem pages with devices that uses DMA (such devices won't need any internal change at all to work with bigmem pages, since with DMA only userspace and the hardware device (and not the kernel) will touch the bigmem memory). BTW, with such per-blockdevice bitflag we could do also swapin/swapout on DMA devices more efficiently when possible (the swapout thing is really not interesting though).

NOTE: the above is what IMO is _doable_ but that doesn't mean it worth to do that. I would like to get some more comment about this raw-io/bigmem issue.

Stephen C. Tweedie replied, "Unfortunately an application has no control over allocation of its pages, so this basically makes raw IO useless right now with BIGMEM," and there followed a big implementation discussion.

David Mansfield had been hand-patching knfsd and RAID (and DAC960 before it was merged) for months, and felt that if the patches were only being kept out of the Linus Torvalds tree because they would require upgrading tools, then that shouldn't be an objection, and the patches should definitely go in. Khimenko Victor agreed, so long as the old tools would do no harm to systems using later kernels. Daniel V. Engovatov also agreed, pointing out that NFS problems in the Linus tree were getting very annoying. Stephen Frost also agreed, as did Timothy Writer, Jeff Garzik, David Woodhouse, Robbert Muller, Gregory P Whalin, and David Weinehall.

Ingo Molnar said he'd already ported the RAID code to 2.3 and would release it shortly (and added that any further delay should be blamed solely on him), which got a big round of applause.

This is an amazing thread. A dozen posts and no one has indicated whether they are talking about updating 2.2 or 2.3. It seems like to many developers, even numbered kernels are developmental and odd numbered kernels are experimental. There is scant evidence on this list that anyone sees a need for a kernel that is stable in anything but name.

The new RAID code breaks existing configurations. Such changes are sometimes needed. However, they should never be put directly into a stable kernel. Deliberately breaking production systems with a kernel upgrade _within_ a "stable" kernel series is unconscionable.

Now that companies are beginning to run Linux on production systems, the carefree attitude toward "stable" kernels has to change. I'd love to be able to recommend Linux to my clients on Wall Street, but there have to be stable kernels first. That means nothing changes unless it's a bug fix. Everything else (updated drivers, new features, code cleanups) waits for the next major release. Otherwise, there is little difference between odd and even numbered kernels.

I hope the updates are ready in time to make it into 2.3, but I think Linus has already left the feature freeze too late to have a stable 2.4 this year.

It turned out that a lot of people had been talking about 2.2, and there followed a big discussion about the benefits and dangers of adding features to the stable series. Some folks like M Carling, felt that the stable series should only receive bug fixes, and stop backporting code updates or new features as they have been. Others pointed out that, at least in the case of NFS, the code in the stable series was actually broken, and the available patches included many bug fixes.

At some point, Ingo said, "the RAID code has too much generic impact to be included into 2.2 - it modifies the buffer-cache layer. Furthermore, it breaks the MD API (intentionally), which breaks old utilities. It's clearly too late to be included in 2.2, and thats the end of the story. This is mainly a problem for me only anyway, because i have to maintain the patches separately. 2.3 is a different issue, but there is still some stuff to be cleaned up before the merge is ready - i'm working on it."

Elsewhere in the course of discussion, Rogier Wolff pointed out, "Talking about confusing: Some people (rightly, I think) feel that within a stable release, you shouldn't be required to upgrade userland tools. A system installed with 2.2.0 should be seamlessly upgradeable to 2.2.12. Everyone knows that when you go from 2.0.x to 2.2.x, you might (and as it happens, WILL) have to upgrade a few userland packages, but within a stable kernel release, an upgrade shouldn't lead to an avalange of user-space-program upgrades."

there are some standard kernel features, e.g., ISDN, RAID, NFS, that have at times fallen into disrepair, because nobody was actively developing/maintaining them, or development was occurring off this list and not syncing up in a timely manner. This was true of at least ISDN and NFS in the 2.1.x series.

Linus has insisted that large subsystem patches be accompanied by a subsystem maintainer. I don't blame him; this is as it should be, because the alternatives are not sustainable. The short-term effect, however, has been to cause serious users of these features, as well as several distributions, to use "non-standard patches" against stable kernels. It behooves us to ensure that all features have maintainers, and preferably several people who have a thorough understanding of the code. It also behooves us to test development kernels widely, before the code freezes are announced.

Elsewhere, Stefan Monnier said, "The key point is whether or not something slows down development and whether or not it slows down adoption. The new raid code has not been backported from 2.3, so it's not time wasted. Same thing for knfsd. What reason could there be not to use the newer raid or the newer knfsd ? None. Very simply none. Every serious NFS user uses the patched knfsd, and every serious raid user uses the new raid code. Even distribution increasingly include the newer knfsd patches and the newer raid code, which shows clearly that the problem is not stability or acceptance."

Elsewhere, Mike Galbraith said, "I think that this thread should die now. The decision of what to do and when to do it is in the right hands. In the meantime, we all know how to 'use the source Luke' right? (of course we do.. otherwise we wouldn't be jabbering away on a development list)"

In the course of discussion, Andrea Arcangeli said, "IKD will never be merged because impact lots of common code. I want to be sure to only fix (and not add 8) bugs with IKD." David Weinehall asked, "Are there ANY parts of IKD that are worth to merge, that don't disrupt the main code-paths when not compiled in?" Linus Torvalds replied:

The SMP automatic deadlock detection parts of the IKD are extremely simple, impact none of the mainline code, and are occasionally very useful.

I've certainly considered integrating those, although if I do (or preferably if somebody else does ;) I suspect I'd be happier just leaving them on all the time rather than making them a config option - they are _that_ non-intrusive.

There are many kind of automatic deadlock detection in the ikd patch. I believe you are talking about the IO-APIC-NMI_WATCHDOG from Ingo, since it's the only one that may run on a serious production envinroment as a default thing.

But the NMI patch has a performance-downside too: the timer irq will be driven from the 8259 and not from the apic chip (the io-apic will instead run an NMI irq for the same hardware-event), and that means all timer irq handlers will be run on the first CPU and so the kernel will scale worse (the other CPU may be idle at the same time).

So if you want it into the stock kernel I am fine with it, but I believe it's not a good idea to make it a default thing.

Linus confirmed that he had been talking about Ingo's work, adding, "The semaphore and spinlock "deadlock" detectors that depend on timeouts or on a maximum number of spinlock iterations are definitely not acceptable on any kind of production machine." To Andrea's objection, he replied, "Good point. Never mind."

Raymond Nijssen noticed that 2.2.5 let non-privileged users increase resource limits beyond their maximum values and even beyond the kernel maximums. 2.0.35 resource limits also seemed broken. He posted some code to illustrate this. As an aside, he asked, "why are most "unlimited" kernel limits set to 2G (that is, LONG_MAX in include/asm-*/resource.h) while 3G or so would be more correct?" Alan Cox replied, "Because the ulimit values are signed. 2.2.12 has most of the checks fixed, 2.3.x someone needs to shift to using an unsigned rlimit_t." Matthew Wilcox posted a patch to fix this, saying he'd posted it a few days previously, but included it again now in case Alan had missed it (elsewhere saying that glibc also needed to address the problem). He added, "Incidentally, I spotted this problem by using the -W flag. It's a shame that some of the warnings this generates can't be worked around; we can't make the header files clean. (in particular, the test against zero always being false is something we don't want warned about in the signal.h file. I tried a couple of things to eliminate that warning, but I can't see a way.)"

Robert Dinse applied the low latency patch to a 2.2.10 kernel, to compile on a Sparc SMP system, and got errors. Ingo Molnar replied, "i rewote some of the console code, and have not covered fbcon. On Sparc you dont need those console changes anyway, so just reverse all drivers/char changes and the changes to linux/console_struct.h, linux/selection.h, linux/vt_buffer.h. That should be compilable - i hope."

Richard Gooch found that kmalloc() in 2.3.16 would occassionally fail when allocating memory with 1G ram. Setting mem=512 seemed to solve the problem. Victor Khimenko replied, "Hmm. Am I missed announce about fix for few-years old DMA problem or what ? Allocation DMA memory can fail even on 64MiB system (I seen it) and in theory even on 17MiB system :-(( There are was some ideas about fixing/workarounding it (mostly from Rik van Riel but some can be found on www.alsa-project.com) but AFAIK noone suggested idea sane enough to pass Linus test yet..." Alan Cox said, "2.2.11/2.2.12 have a zoned allocator which seems to help massively" and Rik replied, "Nice and simple, I like it (and have overlooked it until now :(). I think we should implement something at least this good in 2.3, maybe even with cache colouring..." Alan explained, "Its simple, elegant but does need optimising. If you have little ram left but DMA ram your page allocation rate is noticably slower. On a 20Mb box thats not ideal." There was a brief implementation discussion (and some patches) also including H.J. Lu, Benjamin C.R. LaHaise, and Jamie Lokier.

In the course of discussion, Victor Khimenko described how to tell if a PCI modem is a WinModem or not. He said, "You just need to take a look on Windows driver. If there are HUGE .vxd (or .386) file then it's winmodem for sure. If not, then may be it's not a winmodem."

I've uploaded my drivers for RISCom/N2 HDLC card (ISA bus) to ftp://ftp.pm.waw.pl/pub/Linux/hdlc/hdlc-1.0-pre1.tar.gz. This archive includes generic HDLC support routines (currently only DTE/DCE Frame Relay with ANSI or CCITT LMI is available, not counting IP-in-HDLC "raw" mode). I'll work on integrating this support with the one provided by syncppp.c and by other Linux ppp implementations.

The N2 hardware driver should work with SDL RISCom/N2 single or dual port cards, and will not work with integrated CSU/DSU/DDS. There is no SMP nor non-Intel support yet (is it even possible to use this card in non-Intel machine?).

Andries Brouwer announced a driver for the BFS filesystem from a SCO Unixware 7 system. He said:

I wrote something and it works for me. Find it in ftp://ftp.cwi.nl/pub/aeb/bfsmod.c or in ftp://ftp.XX.kernel.org/pub/linux/kernel/people/aeb/

Note that this was written without specs of the fs and with only one specimen of such a fs at hand. If anybody has more information about the structure of the superblock or inodes, I'd like to hear.

By the way, this is read-only only. Making it read-write would be an easy exercise.

Oscar Barlow reported that after upgrading from 2.3.6 to 2.3.16, PPP complained about finding no driver in the kernel, even though he knew he compiled it in. Jean-Baptiste Vignaud said that the PPP driver had changed since 2.3.1x, and that Oscar should upgrade pppd to version 2.3.9; he also suggested upgrading the KDE dialer, which otherwise might give similar errors. EOT.

Robert Dinse got compiler errors on his Sparc, and David S. Miller replied that Sparc32 was still being ported to the new MM architecture of 2.3.x; Robert asked if an announcement would be made when it was ready, but there was no reply.

In the course of discussion, Alan Cox pointed out, "CLONE_PID causes enough potential security hazards it is going away in 2.2.13 and some point in 2.3.x. It was never intended to be user accessible its a hack for the boot up code"

It's pretty much painfully obvious that I'm way behind in editing the Linux Device List, although I *have* processed registrations. However, a major reason for this delay is that way back when I got convinced to maintain both a text and a TeX version of this file, which has meant double work for everything.

This doesn't scale very well, and besides, I really would like to have something output HTML as well. Therefore, unless I hear *loud* protests in the next few days, I'm planning on killing off devices.tex until I have found (and had time to convert to) a decent single-source format.

There were no objections, and some folks suggested SGML as a good single-source format.

David Weinehall announced that he was rewriting all the config menus for all platforms, to make them more uniform. His patches would be against 2.3.x. Some folks were happy about this, and it was pointed out that the menus had started getting out of hand and needed this kind of cleanup. There followed a discussion of various organizational possibilities.

Under the Subject: Clean up of the config-options, David posted a URL to his patch, adding, "I had to make a naughty hack to get the changes in fs/partitions/Config.in to work properly with xconfig, but apart from this, everything should be nice, neat and friendly. I've made changes in Documentation/Configure.help to reflect my changes."

In the course of discussion, Jeff Garzik gave pointers to Richard's Home Page and the Linux Winmodem Support page and summed up this news, "The Lucent winmodem driver is being reverse engineered, and they have made a lot of progress already: It can dial and detect the remote modem's answer tones. Code is available at the first URL above. Also, PCTel has announced that a Linux driver for their [DPS-less] winmodems went out to OEMs... though I have not seen anyone mention seeing or obtaining such a driver yet."

David Weinehall asked if IPv6 could be used as a module (net/Config.in claimed not, but the option was still allowed during 'make config'). Matti Aarnio gave a pointer to an old patch and replied, "It is a module, as long as you never unload it.. In 2.3.* series it is module, but the module itself contains code which makes sure that non-hackers won't unload it accidentally. (But allows smaller initial kernel image size). I think I did at least once fix it so that it *can* be unloaded, but I have been way too busy to bring that to current level.."

This was last discussed in Issue #33, Section #17 (24 Aug 1999: PPP Over Ethernet) . This time, Michal Ostrowski gave a pointer to some code and docs and announced:

I've created several patches which add support for PPP over Ethernet (RFC 2516) to the Linux kernel. These patches require kernel 2.3.17 and ppp 2.3.9 since they make use of the new ppp channels facilities in the kernel.

In writing this code , I came across some problems which are probably worthwhile mentioning here:

In ppp-generic.c , the code makes an assumption that the header length that will be used will always match the length of the PPP header (PPP_HDRLEN). The problem with this is that the higher level network layers account for "dev->hard_header_len" space when allocating and reserving space in the skb's that they create. When the PPP network layer passes down an skb to the PPPoE layer, there's no room in the skb to tack on an ethernet and PPPoE header at the front (and I don't want to have to make a copy of the skb in order to make space).

The proper way of solving this is to have the PPP network layer not assume that the header length is PPP_HDRLEN (that is replace references to PPP_HDRLEN with "dev->hard_header_len", and provide a means by which a PPP channel can set the hard_header_len.

Aside from this issue, the my code doesn't require any non-trivial changes to other parts of the kernel.

The code I have is functional though it requires some cleaning up. Eventually I would like to get it into a state suitable for inclusion into the kernel source and so comments and suggestions about this code are appreciated.

Pavel Machek said, "So my wish for 2.4 came true: pcmcia is integrated in kernel." He posted a patch and there was some discussion of bugs and implementation. Part of the discussion focused on whether to put PCMCIA driver information in drivers/net or drivers/net/modules. Linus Torvalds came out firmly on the side of drivers/net/modules, adding, "The "drivers/net" subdirectory is too much of a random tangle already, it's starting to be hard to find stuff there. A lot of the newer drivers go into their own subdirectories already (irda, fc etc)"

Richard Gooch gave a pointer to his patches page and announced version 120 of devfs, against 2.3.17.

Under the Subject: [PATCH] devfs v121 available, he announced version 121 against 2.3.18, and under the Subject: [PATCH] devfs v99.5 available he announced version 99.5 for 2.2.12, adding, "NOTE: the devfs-patch-v99.x patches are backports to 2.2.x kernels of the latest devfs-patches (which are for 2.3.x kernels)."

Dieter Netzel asked if anyone would please port Alan Cox's MTRR 1.35a code from 2.2 to 2.3 so he could get his AMD Athlon 500 working. Alan quickly pointed out that the code was actually written by Zoltan Boszormenyi, and should drop cleanly into 2.3 as well. Zoltan Boszormenyi gave a pointer to his patch, adding, "This contains the Athlon support and a fix for Cyrix processors. I would call it 1.36 but Richard Gooch is the official maintainer and I got no replies from him yet." Richard Gooch replied, "Yeah, I know. Apologies. Things should get back to normal for me over the next couple of weeks, now that I've finally taken delivery of a machine I can comfortably work on. Next step is to organise how I maintain my ftp area (and where I keep it). Sigh." Dieter also replied to Zoltan, reporting success with the patch. He also noticed some "unresolved symbol" problems, but there was no reply.

Alan Cox announced 2.2.13pre5, Joel Klecker reported that gcc 2.95.1 gave errors with it, and Alan replied, "Its recommended you dont build 2.2.x with gcc 2.95, but obviously since the fix is relevant to 2.3.x a patch to fix this would be nice."

Under the Subject: 2.2.13 & gcc-2.95.1, Garst R. Reese said, "Up until 2.2.13pre5 I had no problems with gcc-2.95.1. Has a decision been reached to just definitely break 2.2 wrt gcc-2.95.1?" There was no reply.

Julien May discovered that compiling his kernel without SMP (after having previously compiled it with SMP support) would give errors. Rui Sousa replied that configuration scripts tended to break when moving from SMP to UP kernels. Instead of "make dep; make clean; make bzImage", which Julien had been doing, Rui suggested:

Linux-2.3.18 is out there now, and it also marks the beginning of the long-promised feature freeze for 2.4.x. To make that freeze more effective, I'm taking two weeks off, just so that you simply CANNOT tempt me with features.

Thanks to David Hinds and others for the last weeks of merging of PCMCIA etc, I'm now officially happy.

Note that feature freeze is different from code freeze. We'll still do updates of drivers etc without being too anal about it, and even completely new drivers (or possibly filesystems) etc may possibly be accepted as long as they don't impact _anything_ else and don't imply a completely new approach to something. Drivers in particular tend to be updated even in the stable kernel, after all.

But expect me to be less than enthusiastic about even new drivers. New ideas for core functionality are right out.

The feature freeze should be turning into a code freeze in another two months or so, and a release by the end of the year. And as everybody knows, our targets never slip.

And as I said, don't even bother emailing me for the next two weeks, because you won't be reaching me anyway, and the mail accumulated over the two weeks will be unceremoniously dumped into a toxic waste container, to be buried in concrete somewhere at sea. Never to be opened again, in short.

I've put up 2.2.13pre6. There are a couple of small things to go in (PPC strnlen_user etc) but other than that I don't plan to add anything else but obvious bug fixes to this. It has two weeks to settle down while Linus has a break.

I'd like to get to the bottom of the SMP sound/floppy bug and some of the other longer standing oddities but thats it. If we have a rock solid 2.2.13 then knfsd updates can go into 2.2.14.

Chuck Mead asked, "Two questions... are the memory leak problems fixed in pre6 and what's happening with the SCSI problems you've been talking about and the fdomain.o bug?"

Alan replied, "The memory leak seems to be sorted. The future domain/scsi stuff isn't going to get solved that rapidly. In 2.3.18 the scsi layer is getting cleaned up and tidied. That should eventually reveal how a command is getting queued in some cases when its not allowed to be."

A patch that makes it possible to mount .d64, .d71 and .d81 disk images from the good old Commodore-machines. You can either copy the images onto a block-device (for instance a floppy) and mount them that way, or use the loopback device.

1581-disks (3.5" DD disks) can also be mounted directly from a normal PC-floppy (dunno about other platforms.)

Most of this driver is written by Dan Fandrich, I've just ported it to the v2.2 kernels, fixed it up a little, cleaned up the code, added support for .d71 images and made it compilable into the kernel (before it could only be compiled as module.) Oh, and I added a Configure.help entry too...

Alan Cox gave a pointer to a patch page and announced, "Well Linus is away for a couple of weeks so I'll start doing -ac patches to collect all the fixes and stuff. If you have some grand subsystem rewrite even one Linus promised to accept - you can feed it to Linus in two weeks time If you have driver updates, fixes, compile warning removals and the like then you can send them in so I can feed Linus a nice structured and complete fix set when he returns."

This issue came up before in Issue #33, Section #5 (20 Aug 1999: Ramdisks Blocksize And 2.3.x Problems) . This time, Clifford Wolf reported ramdisk problems, and Andrea Arcangeli replied, "The ramdisk is a known bug. To fix it I'll have to change some part of the page-cache/buffer code. I just posted a patch which fix it but Linus convinced me that for the coherency issue it's better to avoid shrinking inodes with page-cache queued on them. So I'll have to redo the page-cache patch removing the inode part and then I'll have to rewrite the inode allocation using the slab."

Under the Subject: [patch] 2.3.18-ac1 ramdisk fixes, Andrea posted a patch against 2.3.18ac1, adding that it should fix the ramdisk device. He said, "Actually the ramdisk doesn't work because after uploading a ramdisk via the buffer layer, we can't reach the data from the page-cache because the data is in the buffer layer. So basically the ramdisk must search the buffer cache to work. To use the ramdisk to generate disk images, the page-cache must also be converted back to buffer cache in order to allow `cp /dev/ramdisk image` to see the file-data."

good news folks:

Paul Barton Davis added async notification via SIGIO to the RTC device, and I enhanced "latencytest" (you can get the latest version from my page) to measure timedifferences between two SIGIO calls, All benchmarks performed using Mingo's low-latency patch (without his patch timing sucks) The results are very good, In my example I used an RTC frequency of 2048HZ, and the maximum jitter was about 500usecs (very sporadic , shows up every 60-100secs).

look at the the results:
CPU load=80%
http://www.gardena.net/benno/linux/audio/rtc2048-cpu80/2048.html

CPU load=10%
http://www.gardena.net/benno/linux/audio/rtc2048-cpu10/2048.html

A typical use could be an app which needs 1ms timing precision (MIDI sequencer for example).

I have more good news, On my audio benchmarks I was able to reduce the audio buffer size from 4.3ms to ONLY 2.1ms ( 3x0.7ms buffers) ! , without losing reliability, in this case the max jitter is about 0.7ms in the range of the fragment-time.

look at the results:
CPU load=80%
http://www.gardena.net/benno/linux/audio/audio3x128-cpu80/3x128.html

CPU load=10%
http://www.gardena.net/benno/linux/audio/audio3x128-cpu10/3x128.html

It's interesting that If I use 4.3ms = 3x1.45ms buffers, the max jitter (again very sporadic) is about 1.5ms , the time it takes to play one fragment.

Seems that under high disk I/O load, (very seldom, about every 30-100secs) the process gets woken up one IRQ period later. Ideas why this happens.

The trick to deliver rock-solid audio seems to use 3 buffer, so that you can have about 33% headroom.

Seems that we are now able to outperform most of the other OSes in terms of timing precision/ latency (multimedia loves this) :-)

In future, we will probably not need RT-Linux to deliver realtime audio on Linux. (As processors get faster, jitter will go down)

Bernhard Rosenkraenzer reported that compiling all PCMCIA-related stuff as a module in 2.3.18 gave errors, and Alan Cox replied that that he believed it to be fixed in the -ac series. EOT.

Martin Mares gave a pointer to his PCI patch and announced a new version, primarily composed of fixes and cleanups for 2.3.18.

Under the Subject: PCI patch for 2.3.18, version 2, Martin gave a pointer to a new version of his patch and announced, "I've changed the resource management stuff in arch/i386/kernel/bios32.c, so that it should be able to cope even with the most weird BIOSes and not change device addresses unless it's unavoidable." He added, "If you have a machine with multiple PCI buses or a BIOS which is known to forget to assign some addresses or to enable some PCI devices, please give it a try and tell me whether it works."

Benjamin Close asked if vfat was stable enough to enable write support in 2.3.18, and Alan Cox replied, "Not sure."

I'm not sure how many are aware that I keep current CD-ROM patches at http://www.kernel.dk? Enough plugging... The current patch against 2.3.18 contains:

Partition based multisession mounting. For instance, mount /dev/hdc2 /mnt/cdrom would mount the second session on the CD. Two notes - a) it only works on ATAPI drives, since SCSI drives use minors differently... b) mounting /dev/hdc2 overrides a specified session= option to mount.
Uniform CD Changer support. This was largely done by Richard Sharman and tested by Erik Andersen (a bit, at least!). Now both ATAPI and SCSI changers should work. This also gets rid of the awful CONFIG_IDECD_SLOTS config option.
Video CD / CDi fixes.
Fixes a couple of leaks in cdrom.c
The obligatory cleanups. Note that pcd.c is untested, Grant is looking into what needs to be done to get this in complete working order.

Thomas Molina asked if these patches were being folded into the Linus Torvalds tree, or were they "neat" features he hadn't or wouldn't accept. Jens replied that they were being fully integrated.

In the course of discussion, Gadi Oxman gave a fascinating explanation of some kernel internals:

The reason we added the cli() to ide_do_drive_cmd() before the down() call originally (in 1.3.something, before Linux had SMP support), was to minimize the time inside down() in which an interrupt completion of the just queued request could call up() and race with us calling down().

On a UP kernel, down(), eventually calling schedule, was performing an implicit (single processor) sti() for us at a later stage. So if the drive was able to complete the request immediately, the completion interrupt would have been delayed a bit until that point of the implicit sti().

On a SMP kernel, a global cli() does a local processor __cli(), and grabs the global irq lock. down(), calling schedule(), would eventually release the irq lock before selecting another process.

If, however, somewhere between the call to down() and the release of the irq lock, someone calls local __sti() *and* unluckily, the request could have been serviced by the drive quickly, we would attempt to grab the irq lock on the processor which already owns it and deadlock.

In the path called by down(), I can see the run_task_queue() call, which is called by schedule() before the release_kernel_lock() call. This call, for example, can call unplug_device() for another IDE interface which has queued requests, and there we would do an __sti() in the ide_do_request() function.. Does moving the run_task_queue() call after the release_kernel_lock() call makes any difference with the original ide_do_drive_cmd()?

BTW, our attempt to minimize the race potential between down() and up() is probably no longer needed with current kernels, as the semaphores are SMP and interrupt safe. The important part is to install the semaphore in the request before grabbing the io_request_lock and adding it to the queue, and we should safely be able to remove the save_flags(), cli() (both global and local) and restore_flags() around the call to down(), and let the standard kernel up()/down() functions handle the race between them.

Daniel Robert Franklin posted a patch and said, "This patch makes my ESS1869 card (in a Compaq Armada 1500c notebook) play at the right speed. It is against 2.3.18. It fixes what looks like a typo in the source code - the switch statement appears twice, once with the 1869 line commented out. If it isn't commented out, sound plays really slowly." Petru Paler confirmed the problem and reported complete success with Daniel's patch.


Kernel Traffic Latest \| Archives \| People \| Topics	Wine Latest \| Archives \| People \| Topics	GNUe Latest \| Archives \| People \| Topics
Czech

1.	31 Aug 1999 - 9 Sep 1999	(33 posts)	Serial Driver Update; Some Confusion
2.	2 Sep 1999 - 8 Sep 1999	(42 posts)	bigmem Patch Conflicts With rawio Patch
3.	2 Sep 1999 - 12 Sep 1999	(54 posts)	RAID And NFS Trouble Going Into The Main Tree
4.	4 Sep 1999 - 9 Sep 1999	(22 posts)	IKD In The Kernel
5.	5 Sep 1999 - 8 Sep 1999	(8 posts)	Bug With Resource Limits
6.	5 Sep 1999 - 8 Sep 1999	(3 posts)	Console Code Rewrites Break Sparc On 2.2.x
7.	5 Sep 1999 - 8 Sep 1999	(12 posts)	DMA Memory Allocation
8.	6 Sep 1999 - 10 Sep 1999	(17 posts)	Detecting WinModems
9.	6 Sep 1999 - 7 Sep 1999	(3 posts)	Frame Relay, HDLC And RISCom/N2 Card Drivers
10.	7 Sep 1999 - 8 Sep 1999	(3 posts)	BFS Filesystem Driver Available for Linux
11.	6 Sep 1999 - 7 Sep 1999	(2 posts)	pppd Upgrade Required In Recent Kernels
12.	7 Sep 1999	(3 posts)	Sparc32 Not Fully Ported To New MM Architecture
13.	7 Sep 1999 - 13 Sep 1999	(11 posts)	CLONE_PID Problems
14.	7 Sep 1999 - 8 Sep 1999	(4 posts)	'Linux Device List' Format Change
15.	8 Sep 1999 - 9 Sep 1999	(13 posts)	Config Menu Reorganization
16.	8 Sep 1999	(2 posts)	Winmodem Status Report
17.	8 Sep 1999	(2 posts)	IPv6 As A Module
18.	8 Sep 1999	(1 post)	PPP Over Ethernet
19.	8 Sep 1999 - 13 Sep 1999	(15 posts)	PCMCIA Is Merged For 2.4
20.	8 Sep 1999	(1 post)	devfs Updates
21.	9 Sep 1999 - 13 Sep 1999	(5 posts)	MTRR Code Going Into 2.3.x
22.	9 Sep 1999	(4 posts)	2.2.13pre5 Announced
23.	10 Sep 1999	(2 posts)	Errors When Moving A System From SMP To UP
24.	10 Sep 1999	(2 posts)	Linus Announces A Feature Freeze
25.	10 Sep 1999 - 12 Sep 1999	(4 posts)	2.2.13pre6 Announced
26.	11 Sep 1999	(1 post)	CBMFS v0.4 For 2.2.12
27.	11 Sep 1999	(1 post)	Alan Cox Starts 2.3.18acX Series For Linus' Vacation
28.	11 Sep 1999 - 12 Sep 1999	(3 posts)	Ramdisk Fix For 2.3.18
29.	11 Sep 1999	(3 posts)	Some Good Low-Latency Benchmarks
30.	11 Sep 1999 - 12 Sep 1999	(2 posts)	Modularized PCMCIA Fix To 2.3.18
31.	11 Sep 1999 - 12 Sep 1999	(7 posts)	PCI Patches For 2.3.18
32.	12 Sep 1999	(2 posts)	VFAT Nears Stability, Needs Brave Testers
33.	12 Sep 1999	(4 posts)	CD-ROM Updates To 2.3.18
34.	12 Sep 1999 - 13 Sep 1999	(4 posts)	Some Explanation Of Internals
35.	13 Sep 1999	(2 posts)	ESS1869 Fix

Kernel Traffic #35 For 20 Sep 1999

By Zack Brown