Kernel Traffic #206 For 23 Feb 2003

By Zack Brown

Table Of Contents

Mailing List Stats For This Week

We looked at 2164 posts in 11728K.

There were 468 different contributors. 245 posted more than once. 167 posted last week too.

The top posters of the week were:

 

1. 64-Bit Jiffies; Jiffie Wrap Bugs
2 Feb 2003 - 17 Feb 2003 (33 posts) Archive Link: "[PATCH *] use 64 bit jiffies"
Topics: Forward Port
People: Tim SchmielauMatti AarnioDenis Vlasenko

Tim Schmielau announced:

Just a note that I have rediffed for 2.5.55 the patches that use the 64 bit jiffies value to avoid uptime and process start time wrap after 49.5 days. I will push them Linus-wards when he's back. They can be retrieved from

http://www.physik3.uni-rostock.de/tim/kernel/2.5/jiffies64-33a.patch.gz (1/3: infrastructure)
http://www.physik3.uni-rostock.de/tim/kernel/2.5/jiffies64-33b.patch.gz (2/3: fix uptime wrap)
http://www.physik3.uni-rostock.de/tim/kernel/2.5/jiffies64-33c.patch.gz (3/3: 64 bit process start time)

For discussion see http://www.uwsg.indiana.edu/hypermail/linux/kernel/0211.1/0471.html and http://www.uwsg.indiana.edu/hypermail/linux/kernel/0211.1/0847.html

Denis Vlasenko was very surprised that these patches were still not included in the main tree. He thought this meant that uptimes couldn't be measured beyond a few months, but Matti Aarnio pointed out:

You don't need to have 64-bit jiffy for things like internal timers, nor for uptime tracking.

Timers have well behaving constructs to use 32-bit jiffy quite successfully, and 64-bit values, especially atomicish, in 32-bit register-poor machines (i386) are damn difficult.

I do have a number of machines with 100 to 300 day uptimes, all with "mere" 32-bit jiffy. With 1000 Hz clock that means at least one full wrap-around of jiffy.

Denis Vlasenko replied, "Your processes will show strange start times, CPU times etc. This will happen in 2.5 pretty soon (after 50 days uptime)." A more important problem he felt, though, was jiffy wrap bugs. He said, "There were reports of machines hanging on jiffy wrap. This is typically a result of incorrect jiffy use in some driver." He suggested wrapping the jiffy counter within the first five minutes of uptime. Driver writers would be bitten by their bugs, and fix them right away. Denis described in a later post, "Jiffie wrap at 5 mins is done this way: jiffies is initialized to -5*60*HZ instead of zero at boot. Accounting code is adjusted e.g. not to show 400+ days uptime (it subtracts -5*60*HZ from jiffies). Jiffies wraps to zero in five minutes. Box should survive with no probs. If it instead hangs, oops or otherwise feeling bad, you have a jiffy wrap bug somewhere."

Jorn Engel was intrigued by this, and asked for the patch. Tim Schmielau replied, "A patch for 2.4.20-pre7 (and maybe later) is at http://www.physik3.uni-rostock.de/tim/kernel/2.4/. I still need to forward port it to 2.5 (which should be easy)."

 

2. Accessing The BitKeeper Tree Without BitKeeper
5 Feb 2003 - 19 Feb 2003 (178 posts) Archive Link: "2.5 changeset 1.952.4.2 corrupt in fs/jfs/inode.c"
Topics: Patents, Real-Time, Version Control
People: Andrea ArcangeliLarry McVoyRik van RielAdrian BunkJamie LokierAlan CoxNicolas PitreJohn BradfordAndrew MortonDavid WoodhouseMatt ReppertPavel MachekThomas MolinaDavid LangDave JonesLinus Torvalds

In the course of remarking on some source code, Andrea Arcangeli said, "I'm using my own GPL software to checkout from the bitkeeper servers (I don't want to miss the additional information stored in proprietary form inside the bitkepper database and so I'm extracting it all and storing it in a open manner locally, in a complete structured form, they're live classes dumped to disk, not like the monolithic patches in the cset/ directory where it's not trivial to manage all the metadata to see all the evolution of a certain file or subsystem by writing a few more lines of code that loads and use those classes)" Pavel Machek asked if Andrea would make this little nugget available for all and sundry, but Larry McVoy of BitMover said:

Two things:

  • We're going to make a CVS archive of Linus tree available, automatically updated, and we'll rsync it to some public place like kernel.org so you can get at the data in a way you want with no BK involved at all.
  • If you beat on our servers with that stupid script, we'll shut down the http access. Give us a break already.

Pavel, Andrea, and others were happy to hear about the CVS availability. Elsewhere, under the Subject: openbkweb-0.0 (http://www.uwsg.indiana.edu/hypermail/linux/kernel/0302.0/0852.html) , Andrea said:

If you're interested in checking out from the bitkeeper servers with an open source program (GPLv2 actually) you can do/try it now (almost, see below for the last trouble that I just discussed today on l-k).

http://www.us.kernel.org/pub/linux/kernel/people/andrea/openbkweb/openbkweb-0.0.tar.gz

I should thanks Aaron Swartz, since I included a GPLv2 html2text python library for him (google found it) that makes it trivial to extract the data.

For example to checkout bk head starting from a 2.5.59 tarball on kernel.org the command is:

        mkdir 2.5-pickle
        python bkweb.py -u http://linux.bkbits.net:8080/linux-2.5
--rev=1.947.3.4 -t linux-2.5.59 -c 2.5-pickle

The number 1.947.3.4 cames from this webpage:

http://linux.bkbits.net:8080/linux-2.5/ChangeSet@-4w?nav=index.html

you'll see that 1.947.3.4 matches the release number 2.5.59. The number 1.947.3.4 tells bkweb to apply all the changesets from 1.947.3.5 to the last one. So it's really simple.

bkweb will fetch the data, will save it in the cache directory (2.5-pickle) and it will apply it to the linux-2.5.59 directory.

Unfortunately it doesn't work reliably when Linus merges old changesets, the logic will be confused. The current logic thinks all changesets can be applied in order.

the problem is that to generate bk head it may have to apply also some changeset in the past, not only between 2.5.59 to the last one in cronological order. That still needs a solution. One solution could be also that nobody should send bk pull to Linus anymore, if everybody sends patches it won't risk to reject anymore ;) but I doubt it's an acceptable one for everybody, I assume somebody is confortable with it, if they're using it right now. I also suspect it will be necessary to use a different numbering and not to use the bk changeset numbers as index. The object is to never have to rebuild the database. Currently I don't even store persistent the ordering (not point to store it yet since it's not a 100% reliable ordering ;).

The -c parameter tells the directory where to generate the cache. the cache is truly the whole repository and it contains all the data and metadata in an open form. To get it just run:

changeset = self.unpickle(revision)

where revision can be '1.947.3.5'. dir(changeset) will show you what you can find there, it's a live class after being restored from disk (I put only data in there to save space on disk but in theory it could contain software too, not that I think it is useful but..).

the output looks like this

athlon:~/devel/bkweb> python bkweb.py -u http://linux.bkbits.net:8080/linux-2.5
--rev=1.947.3.4 -t ~/devel/kernel/x -c ~/devel/kernel/2.5-pickle >/tmp/x
Downloading (linux.bkbits.net:8080,/linux-2.5/) ... done.
Found 14 Changeset fallback pages.
Searching rev 1.947.3.4 ... found at CHS 6
Pending 294 revisions
Downloading changeset 1.947.3.5 ... done.
------ Changeset:       1.947.3.5
------ Author:          scott.feldman@intel.com
------ Date:            2003-01-16 23:06:26-05:00
------ Commentary:      [netdrvr e100] udelay a better way

Bug Fix: TCO workaround after hard reset of controller to wait for TCO traffic to settle. Workaround requires issuing a CU load base command after hard reset, followed by a wait for scb and finally a wait for TCO traffic bit to clear. Affects 82559s and above wired to SMBus.

------ Files:           ['drivers/net/e100/e100_main.c']
patching file drivers/net/e100/e100_main.c
make distclean ... done.
Pending 293 revisions
Downloading changeset 1.947.3.6 ... done.
------ Changeset:       1.947.3.6
------ Author:          scott.feldman@intel.com
------ Date:            2003-01-16 23:38:43-05:00
------ Commentary:      [netdrvr e100] fix TxDescriptor bit setting
------ Files:           ['drivers/net/e100/e100_main.c']
patching file drivers/net/e100/e100_main.c
make distclean ... done.
Pending 292 revisions
Downloading changeset 1.947.3.7 ... done.
------ Changeset:       1.947.3.7
------ Author:          scott.feldman@intel.com
------ Date:            2003-01-16 23:40:31-05:00
------ Commentary:      [netdrvr e100] standardize nic-specific stats
support

* Removed /proc/net/PRO_LAN_Adapters
* Added ethtool GSTATS support
------ Files:           ['drivers/net/e100/Makefile', 'drivers/net/e100/e100.h',
'drivers/net/e100/e100_main.c']
patching file drivers/net/e100/Makefile
patching file drivers/net/e100/e100.h
patching file drivers/net/e100/e100_main.c
[..]

at the first reject it stops and writes to stdout (/tmp/x in my example) the patch that rejected so you can analyze it. I think it never completed successfully yet but if it works correctly it should just exit with nothing in stdout (with stdout I really mean fd 1, not stderr of course).

So if you run the same command twice the second time it will find everything in the cache obviously, and the cache is trivial to manipulate using simple python scripts, so you can automate searches on files in the changesets or whatever (including feeding the data into CVS or whatever) So backing up the cache effectively you save the whole bitkeeper repository (only the main branch). Converting the cache to something python indipendent is trivial too, I wrote this thing fast so this was the easiest way.

My main object is to give access to data and metadata of the changesets generated during kernel development in a very manageable form to everybody, reliably, without proprietary licence restrictions, and to store it somewhere (possibly mirrored) in a open standard. This had to be the case since the first place IMHO. the cset/ directory doesn't allow anything like that in terms of regenerating the whole tree or managing the metadata, and it's even less reliable than bkweb. The fact I've to fetch from the web and not from the cset/ directories on the mirrors should tell the whole story about the completeness of the cset/ directory. cset/ is better than nothing but it doesn't obviate the need of bkweb or bk.

I wrote the first line of this program yesterday (it's not that I've that much time for this thing but eventually I wanted to checkout through the bitkeeper database too), clearly it's not well tested and I don't have much time for it.

I guess the bitkeeper network protocol could be also implemented on the longer run, it should be much faster to fetch all the database that way, but clearly decoding the html is been much easier and it is supposed to run in background for me, so I don't care about the checkout speed at the moment.

Dave Jones pointed out that the day before, i.e. the day Andrea wrote the first line of the program, was the anniversary of Linus Torvalds' announcement that he would use BitKeeper in kernel development. Dave gave a link to that post (http://www.uwsg.iu.edu/hypermail/linux/kernel/0202.0/0989.html) .

Elsewhere, Larry threatened to cut off HTTP service on the server, if people used Andrea's script to access the BitKeeper repository. He said the script used a thousand times the bandwidth required by normal BitKeeper accesses, and that he couldn't afford to pay for that.

Jamie Lokier pointed out to Andrea that it would be a violation of the BitKeeper license to reverse-engineer the protocol in order to create a free alternative. Rik van Riel pointed out:

Reverse engineering the protocol is probably allowed, as long as you don't create an alternative implementation yourself.

I can't see how Larry's license would stop people from writing that alternative implementation, as long as those people don't use bitkeeper themselves.

This is mostly because the license doesn't forbid creating an alternative to bitkeeper (I doubt Larry would even want that, even if the law granted that much power). All it does is not grant the free use of bitkeeper to people working on an alternative.

But Larry said:

We'd view reverse engineering the protocol as falling under the "you're working on a competing implementation".

The general message is that you are free to use BK but you aren't free to use BK in any way which could hurt the business which produces BK.

Adrian Bunk replied, "If a clause in a license forbids a licensee to use or decompile the program to gather the information needed for independendly developed programs to interoperate with this program current German copyright law says that this clause is void in Germany." Larry said:

Please show me the case law which says we have to give you our technology, for free, and we do not have the right to say "no way unless you agree to not reverse engineer".

Lots of law says "if you paid for this product, the seller may not impose the following restrictions" with reverse engineering being amongst those.

I do not have any data which says that the same law applies in the case of a no charge copy of the software, do you?

Jamie said:

Correct. You do not have to give your software for free. (In European patent terminology it is not called technology, by the way).

Correct. You have the right to say "no way unless you agree not to reverse engineer".

But he added:

Someone may copy and use your software _without_ agreeing to the license.

Then you can sue them for breach of copyright.

You will win, unless their copying was fair use.

Reverse engineering for interoperability is a form of fair use in many countries, including Germany and the UK.

Draw your own conclusion :)

A couple posts down the line, David Lang said that all of this analysis applied only to legally purchased software. Since Larry allowed free use of BitKeeper, David said, he had the right to set the terms of that use. Alan Cox replied, "Most of the world allows reverse engineering for compatibility, full stop. Often not for cloning, so writing bk extracting tools is very different to cloning BK." Larry replied, "all of those reverse engineering clauses are dependent on you having a legal copy of the software, full stop. You can't get a legal copy if what you want to do, now or in the future, is to reverse engineer the software." He added, "If it turns out that people want to behave like little children and not play nice, no problem, we'll promptly fork the tree and you are stuck with whatever version you had at the point you decided to not play nice." On the legal question, Alan replied, "Go talk to an EU lawyer. These laws exist to stop people locking down formats and its one of the reasons you actually have things like open office."

Elsewhere, Thomas Molina suggested that everyone just do the right thing and adhere to the spirit of Larry's license, without trying to find loopholes. Jamie explained:

Larry's touched an open source nerve by creating what is in effect a closed protocol, and getting it into the heart of the kernel development process.

Blame has nothing to do with it. We _greatly_ respect Larry's accomplishments in this area (and others), and as for the kernel development process we might just as well say it's Linus fault for choosing to use featureful Word 7 when unusable Abiword 0.01 was available :)

Quite a lot of kernel trees are maintained using Bitkeeper now, btw, which is testament to how well that vision is implemented.

It would be very *impolite* to ignore Larry's license and use the software he developed against his express wishes. (It does _not_ give me a good feeling even to talk about it).

On the other hand, it's widely accepted that making a program which _interoperates_ with someone elses closed source program is socially acceptable, and often desirable. This is so widely understood that it's expressly written into copyright law in virtually every country which has copyright law.

Thats how important interoperation is considered to be - it's not just a geek thing, it's a _principle_ - something which is widely held to be the right thing to do for the sake of the big picture.

That's why Larry gets so much flak. He is generous with a caveat which touches a nerve. Conditional love. (Slippery slope... the GPL is like this too :)

He gets flak precisely _because_ his software is so good that so many people choose to use it, and accept the attached strings. (It's a compliment, see?)

Imagine if all your friends started talking a different language, called Binglish say. You'd want to talk to them in that language so you could socialise and work with them. Now if they told you you must sign a contract and join a private society, or pay significant cash, otherwise you couldn't talk the language? If it were a few people you'd ignore them, but if it seemed like all the really powerful people who affect your life would just ignore you unless you talked it, you'd be pissed off. You'd feel the playing field had become unlevel and needed correction. You'd be tempted to either (1) cave in and pay or join the private society, (2) learn the language and use it anyway.

In other words, we see shades of glass ceiling for non-Bitkeeper users. Not intentionally, it is simply an effect that occurs when something good is limited to a subset of people - whether by choice or not.

And nobody thinks glass ceilings should be sustained, do they? Do they?

Now, if you accept that writing a program to communicate with users who do their daily work using the BitKeeper product - in their choice of language - is a good idea. How are you going to do that? Larry's created a situation where the only way to do that is to analyse his software against his express wishes.

Which is more important? Being polite to Larry, or being able to write programs that communicate with important people in their preferred language - levelling the playing field somewhat?

It's harsh reality, but if you create something that benefits many people and attach conditions to your generosity, you'll get a lot of complaints. It's true of the GPL, it's true of the BKL, it's true of the World Bank etc.

Sometimes those complaints are based on sound principles. Sometimes they're not.

You decide.

Andrea liked the 'Binglish' comparison, but said:

my short term concern is not to speak Binglish but just to translate from Binglish to English. We need access to the data with an open protocol and to backup the data in a open format. so we can use it too. And Larry is now going to provide the data in the open, IMHO only if that didn't happen we had to research into the possibility of legally reverse enegeneering the bitkeeper protocol. the fact he is now providing the data out in the open avoids us to waste time.

After we can reach the data we can use any version control system we want to manage it, I'm going to write MORE STUPID scripts to do that. I'm been told of several giga archives with dozen thousand revisions under subversion for istance (I know Al Viro blamed subversion code but if the design it's good it may be a good start). subversion may not have all the features of bitkeeper but we can certainly add them over time, the only thing it matters to me is that we get rid of being forced to use a proprietary protocol to fetch the data.

The kernel CVS in more than enough for my/our needs and I thank Larry for seeing it was necessary to allow the kernel data to be open. Now there's no reason to argue anymore with Larry or Linus, they can choose what they can legally use and we can choose what we can legally use and what we find more productive in the long run. I really believe in open protocols and open source software being superior and a necessary thing in the long run, it's not that I advocate people to use open source products and then I change my mind and I run proprietary apps to develop the kernel (I don't put a smile here because clearly this isn't an obvious thought).

Elsewhere, Larry said:

the open source community has a choice. We've granted people the free use of our tools, unlike any other company. If the response is going to be "hey, cool, now it's even easier to copy these tools" then (a) we're idiots and (b) we're not going to follow that up with more free services.

Make up your mind as to what you want as a community and let me know. I know what you want is a GPLed BK. That's not on the table. You don't have one and you can't produce one any time soon, it would take years, ask the Subversion guys or the arch guys or anyone you like, this isn't a matter of some perl scripts and a weekend hack. So until someone produces a viable alternative, you can have or not have BK to use, depending on your behaviour. I'd love to reach a compromise which leaves all parties happy. You want free access to the data in real time, I want you to leave us a chance to grow our business. If you want to have access to the data in CVS and you want to be free to go clone BK and you want to be free to go use BK without charge, that's asking way too much.

One answer, maybe the only viable answer, is to use patents to protect our technology. To date we've been very sparing in where we have done that, there are substantial chunks of BK without patent protection. Some leaders in the kernel group have privately told me to not ship BK without patent protection. That slows down how fast you get a better BK, it's not something we can just wave our hands and make be so, it's lots of time and money. A single patent costs more than enough bandwidth to keep all of you happy for a year. Whatever. If you guys can't come to some sort of consensus, then patents are the route we'll choose, even German law respects patents.

Alan pointed out that it was very difficult to define "community" such that it could be known whether it agreed to Larry's demands. Larry said he'd be happy if Alan himself agreed to represent community interests; but Alan did not reply. Nicolas Pitre said, "Larry, what you ask is impossible. This community isn't hierarchized and is unlikely to form a single opinion on anything like corporate politics. No one can be authoritative of everybody else's opinion."

Elsewhere, Gael Le Mignot pointed out that European law did not recognize software patents, for the same reason they allowed fair use and reverse engineering.

Elsewhere, John Bradford said, "From reading this thread, and the similar ones that have preceeded it, it seems to me that most people are not exactly bothered about using the SCM functionality of BitKeeper, but just want to get the up-to-the-second changes to Linus' tree." Andrew Morton agreed with this, and pointed out, "The latest diff against the last-released kernel is always available at http://www.kernel.org/pub/linux/kernel/v2.5/testing/cset/." But a couple posts later, Matt Reppert pointed out that this diff was made only once a day. John asked:

Why can't we have a mailing list that sends out a diff for each update?

That way, all you need is:

:0
* ^X-Mailing-List: up-to-the-second-linux-patches
| /usr/local/bin/apply_patches

and

#!/bin/sh
cd /usr/src/linux-current
cat | patch -p1

Larry said, "You can, just get Linus to add a trigger to his tree to do so." Alan replied, "He did, the list exists 8)" John said:

I just wanted to confirm that bk-commit-head is actually:

  1. Complete
  2. Realtime

If so, anybody can track Linus' tree, in real time, without using BitKeeper.

Therefore, mailing lists for every single Linux-related BK repository would allow anybody to track all of the trees in real time, without using BK.

David Woodhouse said, regarding the completeness of the list:

It should be complete -- but bear in mind that you may receive the mails in a different order to the order in which they were sent, so just applying them from a mail filter isn't necessarily sensible.

Also note that the dates on them are the date of the changeset itself, not the date of application to Linus' tree (or indeed the date of the cron job which creates the mail).

And regarding completeness:

Almost -- it's run from an hourly cron job, which is more 'real time' than Linus actually pushing from his own box to master.kernel.org and quite enough of a demand on resources already.

It's not done with triggers on Linus' tree because I suspect that would actually make Linus _wait_ while the mail is generated for every changeset he's pushing to master.kernel.org. I do it with a cron job which pulls from Linus' tree to another, and I don't do it with triggers in my own tree because I suspect that would keep Linus' tree locked while it generated the mails too. I do need to investigate possible improvements to the way it's generated, though.

 

3. OSS Soundblaster Driver Rewrite
6 Feb 2003 - 15 Feb 2003 (9 posts) Archive Link: "OSS Sound Blaster sb_card.c rewrite (PnP, module options, etc)"
Topics: Sound: OSS, Sound: SoundBlaster
People: Paul LauferAdam BelayShawn Starr

Paul Laufer announced, "This is a rewrite of sound/oss/sb_card.c which handles detection of Soundblaster ISA cards and clones. This rewrite uses the new PnP and module APIs." He asked anyone owning ISA Soundblaster and clones to test his patches and provide feedback. Then He and Shawn Starr pounded on them, with help from Adam Belay, chasing related patches and updating the linux-kernel list as to progress.

 

4. Linux 2.5.60 Released; POSIX Thread Handling; UML Naming Conflict
10 Feb 2003 - 14 Feb 2003 (39 posts) Archive Link: "Linux 2.5.60"
Topics: Disks: SCSI, POSIX, Power Management: ACPI, Sound: ALSA, USB, User-Mode Linux
People: Linus TorvaldsRussell KingDavid S. MillerOleg DrokinEric W. BiedermanJeff DikeRoland McGrathAndrew Morton

Linus Torvalds announced 2.5.60 (http://www.kernel.org/pub/linux/kernel/v2.5/ChangeLog-2.5.60) :

Ok, this contains a lot of patches all over the map, since there were two weeks of merging to be done (and the merging took almost a week).

Networking, USB, SCSI, ALSA, ACPI, and various architecture updates (UML, Sparc, ia64, ppc64 and ARM).

And obviously part of the merge from Andrew Morton.

And Roland McGrath has been fixing POSIX thread signal handling, and that in turn ended up causing a lot of downstream fixes in affected areas.

Regarding POSIX thread signal handling, Russell King said there was yet more to come. He explained:

Currently, my "hack" to get ARM working is as follows, and is not the best thing to view on a full stomach.

Generalising the signal handling might have made sense, but this amount of duplication _just_ to be able to handle non-hardware breakpoints is getting rather rediculous.

I will be looking into the possibility of carving up the generic signal handling into a saner structure so we don't have this mess.

Obviously, this patch is not for applying. 8)

Linus voided the contents of his stomach, saying, "Oh wow. Yeah, this needs to be fixed some way, by teaching the regular signal handling about it." He approved of Russell's general direction, and David S. Miller said, "I already did some of this and I'll push to Linus right now. It's enough for Sparc64, doing a few extensions to the hook mechanism I created shouldn't be hard for what the m68k and ARM folks are doing."

Elsewhere, Oleg Drokin pointed out that one of the UML patches created a naming conflict. He said, "there is a conflict between in-kernel sigprocmask() and glibc's sigprocmask() (that UML uses to manage signal delivery to right thread)." To solve this, he asked, "Can we please change the name of in-kernel's sigprocmask() to avoid name clash? ;) Jeff Dike" [User-Mode Linux maintainer] "is also seem to support idea of renaming the offending function." But Linus said, "No. I'm not goinmg to start caring about user-land naming in-kernel, that way is a slippery slope. This is firmly a UML problem." Eric W. Biederman asked:

Just for throwing out suggestions, UML can easily avoid using glibc altogether as it is already intimate with the system call layer.

Or it can use the linker to play games with symbol names to move the kernel off into it's own separate name space.

This sounds like a good opportunity to figure out which makes most sense and future proof UML.

Jeff Dike replied that avoiding glibc would not be "easy" at all. He added, "And how is UML intimate with the system calls, anyway? It is no more intimate with the host's system calls than any other app." Regarding the idea of using the linker to handle symbol magic, he said it might work, but he wasn't enough of a linker expert to know for sure at that point. He concluded, "My current thinking is that I will bundle all the userspace code into a single object which then gets linked (-r IIRC) against libc. That should resolve all libc references, at which point I can link it into the kernel, and I think there won't be any conflicts."

Jeff tried to implement this method, but ran into problems. He explained in a subsequent post:

This falls apart for a couple of reasons -

  • I get a duplicate definition of sscanf which I can't get rid of (libc vs. lib/vsprintf.c)
  • More seriously, this assumes a static link and breaks the dynamic build of UML

In order for this to work, references from userspace.o which can be resolved in libc must somehow survive the link against the kernel code unresolved. That is, references to things not in libc must be resolved from kernel code and references to things in libc must be resolved from libc even if there is a symbol with the same name in the kernel. I don't see any way of doing this.

The only other solution I see is to rename the kernel sigprocmask. Oleg Drokin has done with this with a

-Dsigprocmask=__sigprocmask

on kernel code compiles, and I've done it at link time with objcopy. This sucks, but it does have the great advantage that it actually works.

Anyway, if there's something better, I really want to know about it.

 

5. S4bios Support
11 Feb 2003 - 13 Feb 2003 (2 posts) Archive Link: "S4bios support"
Topics: Software Suspend
People: Pavel Machek

Pavel Machek submitted a patch to support S4bios. He said, "S4bios is easier to use [no possibility to boot into wrong kernel; machine actually indicates it is sleeping, not powered down], has nicer progress bars, etc... I'd like to see it in. Please apply." Someone asked privately how S4bios compared to S4. Did the kernel still dump its core into swap? Pavel replied on the list, "Yes, kernel suspends into swap space. See documentation/swsusp.txt for details... You might want to add some details ;-)."

 

6. Promise Spits On Free Software
12 Feb 2003 - 14 Feb 2003 (10 posts) Archive Link: "Promise SATA chips"
Topics: Serial ATA
People: Mons RullgordAndre Hedrick

Mons Rullgord asked, "Are there any drivers being developed for Promise's SATA chips (e.g. the pdc20275 with PCI id 0x3375)?" Andre Hedrick replied, "NOPE !! Neither Promise or I can agree to a contract to develop them. Use Silicon Image products." Mons asked what the problem was, and Andre replied, "The problem is they want an entire list of every piece of intellectual property I have or own. This is a means to claim anything else I do is derived off their IP and they own it. This will not happen. Some other poor bastard can sign their NDA's to write their drivers."

 

7. Status Of IPv6
13 Feb 2003 (4 posts) Archive Link: "IPv6 in the vanilla tree?"
Topics: Networking
People: Robert L. HarrisHideaki YOSHIFUJIChristoph HellwigDavid S. Miller

Robert L. Harris asked, "what's the state of IPv6 in the 2.4.X (20?) kernel trees? Will it be stable enough in 2.4.21/22 that it won't require usagi patches or will that be in 2.6, etc?" Hideaki YOSHIFUJI replied, "2.4.21 will come with (most of) our changes which are already in 2.5.xx series. Please note that we'll continue making patches for 2.5.xx and 2.4.xx." Christoph Hellwig asked, "Btw, is there any chance you could make snapshot patches for 2.5 available as you do for 2.4? Given the number of change in the 2.4 tree it might be interesting what's still missing for 2.5." David S. Miller replied:

ipv6 should be basically identical between 2.4.x and 2.5.x

The only difference is IPSEC, and even for that the ipv6 portions are still only in their most basic stages.

I bet that all their 2.4.x patches apply pretty cleanly to 2.5.x

 

8. 3Com 3cr990 Driver; BitKeeper Argument
13 Feb 2003 - 15 Feb 2003 (18 posts) Archive Link: "3Com 3cr990 driver release"
Topics: Networking, Patents, Version Control
People: David DillowAlan CoxTomas SzepeHenning P. SchmiedehausenLarry McVoyJeff GarzikLinus Torvalds

David Dillow announced:

Finally, after too many delays, I am pleased to announce the release of my driver for 3Com's 3cr990 family of network cards.

This initial release is endian clean, tested on x86 and sparc64, and does

  • NAPI
  • Zerocopy Tx
  • TCP segmentation
  • Hardware VLAN accel

It does *not* do IPSEC crypto offload as of yet. I have a version for the 2.4 series of kernels, but it is an ugly, ugly kludge, and would be shot if seen in public. I will be working on integrating the hardware with the IPSEC support in 2.5.x.

There are a few issues with the firmware -- DMA to a 2 byte aligned address hangs the firmware, so we cannot easily align the IP header, and the firmware will always strip the VLAN tags on packet reception, regardless of our desires. I hope to work with 3Com to resolve these issues.

The code is available via BK at
http://typhoon.bkbits.net/typhoon-2.4
http://typhoon.bkbits.net/typhoon-2.5

Jeff Garzik was happy to see this, and put the patches in line to get into 2.4 and 2.5.

Alan Cox also said to David, "Would you care to make the patches available in a format those of us who work on open source version control systems can use. Right now Mr McVoy prohibits me from reviewing your patches." David sent him some compressed attachements, and Larry McVoy complained that he didn't remember prohibiting Alan from anything of the kind. Alan replied:

Since I work for a company who works on competing version control products I'm not allowed to use bitkeeper. Since his patches are only in bitkeeper format I can't read them.

I'd like to read them so I asked for them in a open format.

Larry said that other employees at Red Hat used BitKeeper, and BitMover made no fuss about it, and Alan replied:

Since Red Hat works on CVS I wonder if they are breaking the license or not. However thats not relevant right now. If I get stuff in .doc format that I can't read I ask people to resend it in a format I can, ditto bitkeeper or any other randomly weird VC file.

And no I don't want to use bitkeeper any more, your ever changing licenses and removals of more and more rights have reminded me why many proprietary vendors just cannot be trusted and why format lockin is so dangerous.

You are turning a simple "can I have that in a format I can read" into yet another epic bitkeeper flame war.

Larry accused Alan of spreading FUD, and pointed out that Red Hat had its own hands dirty already, by getting patents. He said he wanted people to stop complaining and giving him crap all the time. Tomas Szepe replied, "Look, I don't want this to sound harsh, but what did you expect? What you have arranged for with Linus (As one of the few symbols of the open source world) using BK (As proprietary software with lots of strings attached) is so controversial that you must have expected to get A LOT of shit thrown at you in the process, regardless of how much good/evil your work would have caused." And Henning P. Schmiedehausen also said to Larry:

Linus stated in public that he was/is unhappy with CVS. Without Bitkeeper he might use Subversion today. But by using Bitkeeper he made it possible that you and your company started using him as your posterboy for the "SCM good enough for Linus Torvalds to use". This is IMHO not correct. BK is just "the first SCM which came along and was good enough for Linus Torvalds to use it".

I do remember Linus saying that he wants to try out BitKeeper for the 2.5 development tree and if it does not work out, switch to something else in the 2.6, 2.7... cycle.

The rift that the whole BitKeeper/BitMover stuff has opened in the kernel developer community IMHO justifies such a step forward . I'd like to see SVN to be used as an alternative tool. Not because it is better (it probably is not, but I haven't had a chance to try out BK because I don't qualify for the free license) than BK but because it has no strings attached to its usage.

 

9. I/O Scheduler Enhancements; ReiserFS Enhancements
14 Feb 2003 - 17 Feb 2003 (11 posts) Archive Link: "2.5.61-mm1"
Topics: FS: ReiserFS
People: Andrew MortonCliff WhiteEd Tomlinson

Andrew Morton announced:

http://www.kernel.org/pub/linux/kernel/people/akpm/patches/2.5/2.5.61/2.5.61-mm1/

  • Jens has fixed the request queue aliasing problem and we are no longer able to break the IO scheduler. This was preventing the OSDL team from running dbt2 against recent kernels, so hopefully that is all fixed up now.
  • The anticipatory scheduler is performing well. I've included that now.
  • Also included the CFQ I/O scheduler. The kernel defaults to using the deadline/anticipatory scheduler. Select CFQ by adding "elevator=cfq" to the kernel command line. Do "dmesg|grep elevator" to see which one you are using.
  • There is an updated version of the reiserfs_file_write patch here. This patch addresses CPU efficiency when performing appending writes (ie: the usual sort).

    To do this, it requires that userspace pass "large" amounts of data into the write() system call. So the filesystem returns a value of 128k in the stat.st_blksize field from the stat(2) system call. In the hope that some applications are using that kernel-provided hint.

    Turns out that some parts of KDE (kmail, at least) were indeed using this hint, and it triggers a nasty bug in (at least) kmail: it is reading the same 128k of the file again and again and again. It runs like a dog. Ed Tomlinson upgraded his KDE/kmail version and this problem went away.

    So that is something for reiserfs users to keep an eye on.

Cliff White from the OSD2 team said, regarding the trouble running dbt2 against recent kernels:

Thanks again for doing all this, really appreciate it.

Well, we're closer....

The showstopper for us is still the flock() issue. We have Mathew Wilcox's patch from 2.5.52, which we have been applying to all recent kernels. The patch is in PLM as patch id # 1061. The issue is in BugMe as bug #94 .

Without proper flock() we cannot stop and restart the database, which means we can't run the test. We've tried applying Wilcox's flock patch to -mm1, but it's doesn't go clean, and frankly we're not smart enough to do the merge by hand - lock code scares us.

We just tested 2.5.61 vanilla, and 2.5.61-mm1.

The patch applies cleanly to stock 2.5.61, and we can cycle the database. We can't run dbt2 on stock 2.5.61, because of the scheduler bug. We believe the scheduler fix in -mm1 will be the ticket, but we can't try it because of the flock() issue. So we're wedged. Can someone smarter than us maybe do a merge?

 

10. High-Speed PC Serial Ports
16 Feb 2003 (1 post) Archive Link: "[PATCH][CFT] High-speed PC serial ports."
People: David Woodhouse

David Woodhouse announced:

Most PC superio chips have supported up to 460800 or 921600 baud for years.

This patch adds kernel support for the two most common ways of achieving this -- the SMSC chips' "magic multipliers" where a value of 0x8001 or 0x8002 as divisor actually means _multiply_ by 2 or 4 respectively, and the National Semiconductor version where you can set the baud_base to any of three values, but whenever you touch the original 16550-compatible divisor registers, it reverts baud_base to 115200 for compatibility.

Although both of these modes are designed for backward-compatibility, there's _also_ a bit in the superio chip which can disable them and force _complete_ 16550A compatibility. If your BIOS doesn't enable these, you may need Shsmod <http://www.devdrv.com/shsmod/> to do so for you.

If you have an SMSC superio chip, the driver can't autodetect this -- you need the (also-attached) patch to setserial, and to run 'setserial /dev/ttySx magic_multiplier' for each port. This will make it support baud rates of up to 460800.

If you have a NatSemi chip and it's set to enable high-speed before the 8250 driver probes it, then its high-speed capability should be automatically detected and you'll get up to 921600 baud from it without having to do anything special. If your BIOS doesn't enable the high-speed mode, then after using Shsmod to do so, run 'setserial /dev/ttyS0 autoconfig'.

I've also had a hacked-up superio probe enabling high-speed mode on these ports automatically so you can boot with 'console=ttyS0,460800' etc., but that's an ugly hack and adds a _third_ set of superio probe code to the kernel, so doing that can come later. Eventually we can do the superio probe centrally and remove it from the parport, IrDA, SH board setup, and other code where it's currently duplicated. That's probably a 2.7 task though.

 

11. IPv6 IPsec Support
18 Feb 2003 (6 posts) Archive Link: "[PATCH] IPv6 IPsec support"
Topics: Networking
People: Kazunori MiyazawaDavid S. MillerHideaki YOSHIFUJIAlexey Kuznetsov

Kazunori Miyazawa announced:

This is a patch to support IPv6 IPsec on linux-2.5.62. It work well.

I assume that skb->h.raw points properly on the skb in both inbound and outbound.

IPv6 has some extension headers. It is not as simple as IPv4 options. AH however needs to fill zero on mutable options. skb->h.raw is used as an end of processing mutable options.

There is crude trick at IPsec on Neighbor Discovery, because kernel needs dst to do IPsec but there is no dst at doing ND.

I have no idea to avoid this issue except making dummy route at this moment.

David S. Miller thanked Kazunori for his work, and promised to review the patch with Alexey Kuznetsov. David added, "I must ask, have you been working together with Kunihiro Ishiguro <kunihiro@ipinfusion.com>? Or are you seperately doing the same work? It would be great if these two teams worked together. There is no reason to duplicate effort. All people doing work will get full credit. The only thing necessary is to send me patches to add credits to the comments. So nobody needs to fear that their contribution will go unnoticed." Hideaki YOSHIFUJI replied that unfortunately, the two were working separately. Kazunori and Mitsuru KANDA confirmed this, and the two groups agreed to work together.

 

 

 

 

 

 

We Hope You Enjoy Kernel Traffic
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License, version 2.0.