Kernel Traffic
Latest | Archives | People | Topics
Latest | Archives | People | Topics
Latest | Archives | People | Topics
Home | News | RSS Feeds | Mailing Lists | Authors Info | Mirrors | Stalled Traffic

Hurd Traffic #105 For 27 Aug 2001

By Paul Emsley

Mach 4 | Hurd Servers | Debian Hurd Home | Debian Hurd FAQ | debian-hurd List Archives | bug-hurd List Archives | help-hurd List Archive | Hurd Reference Manual | Hurd Installation Guide | Cross-Compiling GNUMach | Hurd Hardware Compatibility Guide

Table Of Contents

1. CD GRUB Extras

31 Jul 2001 - 23 Aug 2001 (13 posts) Archive Link: "GRUB on CD"

Topics: Bootloaders, FS: ext2

People: Neal WalfieldMarcus BrinkmannThierry LarondeNeil Walfield

A couple of weeks ago, Thierry Laronde said that he could work on GRUB on CDs in August. This week, Neil Walfield had an extra task for him:

Thierry, as there is still a week left this month, I figured you might be a little bored. Anyway, if you are still looking for something to do, it would be nice to have a function in grub to print out a block list when given a partition. Which is to say, the starting block, the length and the end.

For instance:

        (grub) blocklist (hd0,3)

This would permit us to remove the partitioning table detection code from ext2fs.static for things such as boot floppies, etc.

Marcus Brinkmann added:

Of course, it would be useful to have blocklist also operate on files ;)

blocklist (hd0,3)/image

(see how debugfs does it for ext2fs, for example). This allows to boot from a filesystem image inside another partition. You can do this today, but you have to start debugfs, get the blocklist, remove indirect inodes, and insert the large string manually in the boot configuration. If grub could do this automatically, this would be nice.

This is filesystem specific. ext2fs is the most important one, of course.

Thierry accepted these substantial requests with good grace.

2. More Jobs, Fixing stat

8 Aug 2001 - 19 Aug 2001 (27 posts) Archive Link: "some more JOBS"

People: Marcus BrinkmannIgor KhavkineJames Morrison

Marcus Brinkmann posted a list of packages that needed relatively minor modifications to port, including the usual MAXHOSTNAMELEN, MAXPATHLEN, sys_errlist problems. We also got a new one (ERESTARTSYS) - used in "stopafter". This symbol is a Linux internal value that doesn't make it to userland and James Morrison suggested that it is just ifdef'ed out.

Nikunj Dadhania took a look at porting "stat" and found that he needed to tinker with /include/bits/types.h. This provoked some discussion as this include file is part of GLIBC. Nikunj's fix was to independently define the struct __fsid_t. Marcus obviously examined the stat problem in greater detail and said:

The program stat has two issues:

  1. It accesses internal symbols of glibc (__var). Everything beginning with two underscores is "strictly forbidden" if you want to follow the book. More so, there is absolutely no need to access those symbols. As the GNU version of st_fsid shows, the special definition of the fsid type is purely a technical implementation issue to make the variable occupy a certain number of bytes in some special way. Applications should be completely unaware of this.

    stat should be changed to simply access st_fsid (not __val at all), and print it out as a long long. For this you need to reformat the format specifiers a bit (%-16Lx for example, etc)

  2. In fs.h, you will find a list of fs identifiers. Why stat doesn't include some Linux header file for this is an interesting question. Maybe because of the fact that those definitions are spread over many files. You will need to put them in #ifdef __linux__, and use the Hurd's identifiers on GNU.

Nikunj posted a patch which caused a not-clearly resolved discussion about filesystem-related macro generation.

"Standards Man" Igor Khavkine had the following to add:

A compliance note: The version of SUSv2 that I have only defines the types `struct stat' and `struct statvfs'. `struct statvfs' contains the the field `unsigned long f_fsid'. It weems like in both both Linux and Hurd (both using glibc) f_fside has type `unsigned long long'. I think this might be ok because the standard never specifies the location of each field within a structure, and `long long' could always be demoted to `long'.

But stat seems to be using `struct statfs' and statfs() to get it's information about the filesystem. This seems to me like an upstream bug.

3. Goodbye Base Tarballs

18 Aug 2001 - 24 Aug 2001 (8 posts) Archive Link: "[ Re: Still no base tarball]"

Topics: Apt, FS: FAT

People: Marcus BrinkmannEthan BensonRobert Bihlmeyer

Marcus Brinkmann reposted this message from debian-devel by Ethan Benson saying that it was important for "development of installers for Debian Hurd systems " , Ethan said:

Base tarballs are obsolete and deprecated, period.

The ONLY time a base tarball should be used to install the base system is for hard disk based installs. it should never be used for CD installs, or network installs, ever.

For CD installs and network installs debootstrap simply uses the normal debian archive, that is the .debs found in the pool and the apt Packages/Release files in dists/woody/binary-* for CDs all necessary packages will be on CD1, for the network everything is there obviously.

so the basedebs.tgz file should NOT be present on offcial Debian CDs, nor should it be used for CD based installs. dbootstrap or debootstrap will never download the basedebs.tgz file from the network, if you have network connectivity the real debian archive will be used, NOT a base tarball.

That leaves hard disk installs (installing from files on a FAT or HFS partition) generally its unlikly that the user will have a debian mirror on their harddisk, since its basically impossible given the crippled nature of those filesystems. in those cases they will download a basedebs.tgz (from where has yet to be determined), drivers.tgz and other b-f files manually with whatever downloading software comes with the proprietary OS in use. when they select a hard disk based install method dbootstrap will first see there is no debian mirror, then it will check for a basedebs.tgz in whatever directory the user specified (that is if they specify /debian, dbootstrap will check for /debian/basedebs.tgz).

reasons a hard disk install might be desired are:

futhermore basedebs.tgz does not contain a root filesystem, all it contains is /var/cache/apt/archives (with all the .debs that make up the base system) and /var/lib/apt/lists (with the Packages files).

This, however, causes problems running post- and pre- install scripts (GNU/Hurd installation works via a GNU/Linux installation system). There was some discussion of the problems, Robert Bihlmeyer ended up saying "In summary, though, I'm not convinced that all this is necessary. Hurd should be able to install itself. Working on that is probably more rewarding. "

Marcus Brinkmann (who has been preparing GNU/Hurd base tarball for years) replied to this: "Oh well, it is convenient from time to time. And would have helped me if it had been implemented two and a healf years ago :-/"

4. New Hurd Available

20 Aug 2001 - 22 Aug 2001 (4 posts) Archive Link: "hurd 20010817 installed"

People: Marcus Brinkmann

Marcus Brinkmann announced that a new Hurd (20010817) was installed in the Debian archive. This implements the fix to the "proc bug", discussed last week. Marcus said "Personally, I find that the 20010817 snapshot is quite stable (in fact, the only crash I had since was in gnumach)."

5. alloc_contig_mem() Bug?

21 Aug 2001 (2 posts) Archive Link: "Too much RAM issue"

People: Roland McGrath

Wil Reichert got a alloc_contig_mem error mentioned mentioned a few weeks ago. He took a look at linux/dev/init/main.c and changed 16 to 64 in the alloc_contig_mem call. This got his GNU/Hurd booting. However, Roland McGrath said:

Your change defeats the purpose of the code, which is to allocate a contiguous chunk of memory with physical addresses below 16MB. This is necessary for some ISA DMA devices, as I understand it. It's probably a sufficient workaround in your case, since you probably don't have any devices that actually care.

Perhaps there is a bug in alloc_contig_mem, or perhaps just all the low memory has been allocated to other things first. In the latter case, we may need to move some of the initialization for the driver glue earlier, or do some special hack to reserve the low memory for it. Hmm, maybe I can change the initialization to put the low memory pages last on the free list so they won't be used for anything else before the driver initialization.

6. time Added

21 Aug 2001 - 23 Aug 2001 (6 posts) Archive Link: "Wait3 Questions"

People: Brent Fulgham

Brent Fulgham was looking at the "time package" and came across the wait3 issues discussed a few weeks ago. Brent went on to say: " The latest Hurd packages allow the Wait3 configure test to complete successfully, so we can add "time" to the list of auto-buildable packages. I've just uploaded the current version to master, but we can auto-build next time around. "

7. Hurders Unhappy As Lookup Of "" Must Fail

6 Aug 2001 - 21 Aug 2001 (14 posts) Archive Link: "Bug#107826: can not create file through a dangling symlink"

People: Roland McGrathNiels MöllerThomas BushnellMarcus Brinkmann

Marcus Brinkmann had been running the fileutils test suite and found a failure demonstrating that the Hurd cannot create a file from a dangling symlink. Roland McGrath suggested a fix and also noted that a lookup of "" should fail according to the Austin Common Standards Revision Group draft 7 specs, he added "these things differed in traditional Unix behavior, and in BSD until fairly recently, but the new specs on the horizon require everyone to match the System V style behavior. "

Niels Möller was not particularly keen about this idea saying "the current behaviour is quite logical given the requirement that "foo/////bar" is the same thing as "foo/bar"." Roland replied "We agree. But everyone else has agreed that "" must fail."

It appeared that Thomas Bushnell is not keen on mindlessly following standards and wanted to be able to block creation of referenced files using symlinks to nothing and said "should we necessarily match every standard, even stupid ones? "

However, Roland was unsympathetic and said "When everybody else is doing it, yes" .

8. Fixing binutils Test

22 Aug 2001 (5 posts) Archive Link: "binutils test suite failures"

People: Mark KettenisMarcus Brinkmann

Marcus Brinkmann continued with testing binutils and found failures related to ld. Mark Kettenis tinked with the tests saying that his patch "should get rid of all of those failures except FAIL: bootstrap with --static " , which was harder because it needed crt0.o instead of crt1.o.

Marcus reported that the patch worked.

9. Getting Rid Of Serverboot

16 Aug 2001 - 24 Aug 2001 (19 posts) Archive Link: "Getting rid of serverboot"

Topics: Bootloaders, FS: ext2

People: Jeroen DekkersRoland McGrathNeal WalfieldJeroen Dekkers

Jeroen Dekkers said "I can't boot oskit-mach yet because the oskit partition support is broken. I got the libparted patches from Neal, they all work great except serverboot doesn't make use of libstore, it still uses the partition code from mach. I heard Marcus talking about getting rid of serverboot, so instead of fixing serverboot I can better write the code to get rid of it " and added a list of things he needed.

Roland replied:

We already have a plan for this, and if you'd like to implement it that would be great. We've had this plan in mind since the beginning when the multiboot spec was being developed.

The first step is to rewrite boot_script.[ch] so that it can really be used in the kernel. I started to do this but never finished. Maybe I'll go at it tonight and send you the results to debug and finish.

That's really all that needs to be done to be able to boot this way. The idea is that the lines that now make up the boot script will be the module string for each module. The kernel will use the boot_script.c code to parse each of these strings and apply it (i.e. the ELF loader filling up a task) to the multiboot module data (i.e. file contents loaded by GRUB).

Take each line in your boot script and put "module " in front of it, and you have the GRUB commands to load up the files as multiboot module and the strings to direct the kernel what to do with them.

That's all there is. Once we have this, we might like to have some GRUB extensions to make it easier (avoid copying the boot-script boilerplate into menu.lst). But that is just gravy.

The next day, Roland went on to say:

I've just made several commits of untested code that should implement most of this, but surely needs some loose ends fixed up and some debugging done.

Some changes are in the hurd source tree under boot/ and serverboot/. The first thing to do is make sure that a new `boot' rebuilt from the current cvs state still works to boot a sub-hurd. The second thing to do is make sure that a new serverboot still works to boot a hurd for real.

On the kernel side, I've checked in changes on the oskit-branch. But you can just take kern/bootstrap.c from oskit-branch and plop it into your gnumach tree (or "cvs up -r oskit-branch kern/bootstrap.c" in your gnumach tree) and it will build ok.

Right now this new kern/bootstrap.c contains `#include "../../hurd/boot/boot_script.h"', so it only builds right if you have your hurd source tree as ../hurd relative to your mach source tree. (After this gets working, I will rectify this before checking in the changes to the gnumach trunk.)

The first thing to test with the new kernel is that an old-style boot still works. It checks if there is exactly one multiboot module and its module string contains no spaces (i.e. "/boot/serverboot/index.html"), and uses (mostly) the old code in that case. Please make sure I haven't broken this before getting into debugging the new functionality.

I will continue to tie up loose ends in the new boot-script code in kern/bootstrap.c as they come to mind, and check in those changes. If you have fixes, send cvs diff -u output so I can see the rev number you started with.

Neal Walfield had an issue with struct multiboot_module, saying "I do not know if this works under OSKit Mach, however, this was the root of all my trouble in GNUMach. Basically, this does not work at all" and posted the error message. Explaining further, Neil said:

When booting in compat mode, we panic in boot_read with the error that I reported earlier. Here are a few values that I printed out on entry to that function:

        mod->mod_start: 1179403647
        mod->mod_end:   65793
        mod->string:    ""
        file_ofs:       0
        size:           52

Thus, it looks to me like we do not have a valid module structure. Additionally, they are consistent across reboots.

Booting multiple modules result in a hang, however, I see more plausible results from my printfs in boot_read (i.e. at least mod->string is valid).

        boot_read: "/home/neal/ext2fs.static":
        mod_start: 1851392; file_ofs: 0; size 52; mod->mod_end: 2924311
        boot_read: "/home/neal/ext2fs.static":
        mod_start: 1851392; file_ofs: 52; size 96; mod->mod_end: 2924311
        read_exec: "":
        read_exec: "":


Neil went to sleep on it and posted a fix the next day, saying "Basically, the problem was lack of type checking and the use of physical, not kernel mapped addresses. Additionally, booting from the kernel does not set MULTIBOOT_CMDLINE which means that the user ends up in single user mode. " Neil confirmed that the new style boot then worked.

Later Roland said:

I've committed the hurd changes I described earlier, which need testing. They also BREAK ALL EXISTING BOOT SCRIPTS.

The changes are in diskfs to replace --bootflags with --multiboot-command-line, to replace the -i/-d bootflags switches with the --boot-init-program and --boot-debug-pause options respectively, and to pass the chopped-up command line words as arguments to init instead of passing bootflags. In init, removed -K and made it instead expect its arguments to form the kernel command line, and made it pass its arguments to runsystem. In runsystem, made it expect those arguments instead of ${MULTIBOOT_CMDLINE}.

I changed the hurd.boot example file in the root of the source tree to replace --bootflags with --multiboot-command-line. There are surely numerous other example boot scripts lurking around that should be changed too.

Please test that a newly built hurd with an updated boot script still works with boot (sub-hurd) and serverboot, as well as the new kernel boot script support. Be sure to check with ps that the kernel command line shows up right.

Probably we should add an option to boot to set ${kernel-command-line}, and set it by default based on boot's own args ("$argv[0] -s root=foobar").

There were a couple of rounds of bug fixes and boot testing after this. It seemed to work out pretty well.

10. Crust

19 Aug 2001 - 26 Aug 2001 (12 posts) Archive Link: "New GNU/Hurd project: Crust Display System "

Topics: GGI

People: Igor KhavkineJohan RydbergJames Morrison

Johan Rydberg posted a link to his GNU/Hurd display project, crust. This met with some skepticism (there is no code there), James Morrison suggesting other efforts in the area such as kgi, ggi, berlin and colortext. Igor Khavkine said:

There have been many variations on the same theme after X windows, like Display Postscript, Y windows, Berlin, and so on and so forth. And many minds have laboured to created viable and arguably supperior alternative to X.

I strongly advise you against creating another variation on the same theme without putting too much tought into it. However if you are interested in persuing development in this area. I suggest you focus your attentions on projects that either require more immediate attention or already show promise.

For example Berlin has been showing promise for some time but is still lacking application and multi-platform support. It relies on having access to a GGI environment, however one rarely exists outside of an existing X server. If you want Hurd to support this display system natively you need to look at and see how it can be integrated with the Mach microkernel and Hurd's core servers (like auth, term, etc.).

Right now we have a working X windows system running on Hurd, and most developers are quite happy with that. But innovation and experimentation are always encouraged. But there is always the matter of knowing where to apply both of these dendencies, please choose wisely.

Johan went onto explain his thinking:

My current idea of implementation: There are two key parts of the system:

  1. The surface server -- just really handles the hardware and sends events to the applications (surfaces).
  2. The client library -- is responsible for almost everything. implements all drawing primitives and event processing.

This project seems ambitious and it may be a while before we see anything substantial from it.

11. Shockingly Large Filesystems

20 Aug 2001 (3 posts) Archive Link: "long uptime memory consumption"

People: Marcus BrinkmannRoland McGrath

Marcus Brinkmann became concerned by the memory usage of some servers:

I autobuilt some dozen packages, then left the machine on over night to watch what happens. No jobs were done over night (except I think the daily cron job).

It seems that gnumach consumes an increasing amount of memory. I don't have earlier stats, but I remember it using 9MB, and now it's 10.6MB. But the real shockers are the filesystems, the root filesystem with 63.3MB RSS. This is of course because a large number of threads (ps claims 2937, can this be correct? Doesn't gnumach support only 1024 threads total?).

The build filesystem for the autobuilder is hd0s10, and it looks moderate.

Roland McGrath did not think that the large number of threads should necessarily cause the RSS to be high "since all those pages of thread structures and stack memory aren't wired and shouldn't be accessed more than needed. It might just be that your RAM all happens to be in use for cached disk pages because there is nothing better for it to do. "

People with large memory could do some testing here, there is no resolution at the moment.







Sharon And Joy

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.