Kernel Traffic
Latest | Archives | People | Topics
Wine
Latest | Archives | People | Topics
Git
Latest | Archives | People | Topics
Czech
Home | News | RSS Feeds | Mailing Lists | Authors Info | Mirrors | Sleeping Cousins

Hurd Traffic #9 For 2 Aug 1999

By Zack Brown

Do you pine for the nice days of minix-1.1, when men were men and wrote their own device drivers?
Are you without a nice project and just dying to cut your teeth on an OS you can try to modify for your needs?
Are you finding it frustrating when everything works on minix? No more all-nighters to get a nifty program working?
Then this post might be just for you :-)
 
-- Linus Torvalds, 1991

Table Of Contents

Mailing List Stats For This Week

We looked at 60 posts in 183K.

There were 25 different contributors. 8 posted more than once. 3 posted last week too.

The top posters of the week were:

 

1. Big Steps Forward In Spite Of Low Traffic On List
26 Jul 1999 - 29 Jul 1999 (5 posts) Archive Link: "New packages and work in progress"
People: Marcus BrinkmannThomas Bushnell

Marcus Brinkmann said:

maybe you already wondered why it was so silent on this list for a while. That's because we made some big jumps instead of many little steps the last weeks. Here is a list of important changes:

Perl5

Perl5 packages were uploaded by me, which were important because the old ones had been overwritten by dummy packages. The ftp archive is in a sane state again very soon now.

dpkg 1.4.1.7

I went back to dpkg 1.4.1.5 under Linux and started from scratch, applying my patches as I need them. Now we are back in sync again with the Linux version. No new features apart that, but recompiled to work with perl5 package.

New installation scripts

I will upload them in a few minutes to www.debian.org/ports/hurd. They are to be used with the new perl packages, dpkg and the new Hurd.

Hurd 19990725

A new snapshot with a lot of changes and bug fixes. In particular, Roland implemented a new boot up procedure (slpit-init, take a look at /libexec/runsystem), which is much more modular and managed by update-alternatives. A new tool is rpctrace, which does work, but seems to hang the system quite easily, too. Use with care. It also contains the Hurd info docs (I don't remember if they were in 0616, too).

sysvinit

I ported sysvinit to the Hurd, and I will soon upload my package for general testing. It uses the new split-init feature of the new Hurd package to plunge into the boot process. This means, we have runlevels and all. More details later. If anyone dares to test it prior, let me know and I can send it to you.

GNU Mach

A new GNU Mach, featuring a better Memory Management (Thomas Bushnell made it more reliable) and autodetection of all network cards! The latter should have worked ages ago, but a little bug was preventing it.

script

The script program does currently timeout after thirty seconds. I can make available a fixed version, if anyone cares. It may work automagically after the next glibc update, we will see.

 

2. Partition Sizing; VMWare; Booting From A Disk Image
22 Jul 1999 - 26 Jul 1999 (13 posts) Archive Link: "Your opinion about GNU/HURD (and vmware)"
Topics: Bootloaders, Emulators: VMWare, FS: ext2
People: Marcus BrinkmannRoland McGrathDaniel Burrows

Nuno Emanuel F. Carvalho got an AMD K6-2 400MHz processor and wanted to get the Hurd going on it. He asked how big a partition to give it, and Marcus Brinkmann said, "The stuff you need to play with it fits in 100 MB, but you need more for a "full" installation (wow :). "full" means what we have, not what we will have when we are ready (when we are ready, you will need 2 GB for a full installation, but that's far away...)" Nuno replied that he guessed 1G would be enough for the present, and Marcus pointed out that 1G was also the current maximum partition size that GNU Mach could understand.

In his original post, Nuno also asked if VMWare was an option for installing the Hurd (see Issue #8, Section #1  (19 Jun 1999: VMWare And The Hurd) ), and Marcus said he couldn't recommend it because it was slow and nonfree. Nuno replied that slow was OK, but nonfree was not for him. He hoped one day VMWare would be free. Marcus replied:

I doubt that this would be soon. VM Ware is a big succes for the company, and they would be stupid to free it as they can make a lot of money out of this innovation. That's okay, it's their code. For us, we should focus on Free Software, right?

Instead struggeling with VM Ware, it would be nice if someone could investigate if the booting-the-hurd-from-a-diskimage-inside-any-ext2fs does work, as it was implemented by Roland some time back. This would have two advantages:

  • No repartitioning is required.
  • We could ship a compressed disk image that is completely preconfigured. No need to do any configuration with native-install or whatsoever.

Marcus also quoted at extreme length a December 1998 message by Roland McGrath:

I hacked together a new hurd feature that makes it possible to boot without having a disk partition dedicated to the hurd. It's kind of a kludge, but I threw it together quickly and it ought to do. I barely tested this at all, but enough that other hackers should be able to muddle through.

You need the most-current hurd from cvs to get this code (there is a snapshot on the way soon). Actually, I checked it in late enough last night/this morning that I'm not sure if the anoncvs mirrors have it yet. To compile the current hurd without errors, you'll need the current glibc from cvs as of the wee hours this morning (for unrelated reasons--I was just a'hacking last night); there will hopefully be a glibc-2.0.109 fairly soon.

[Of course, our recent packages is all what is required -- MB]

My point here is to explain the code changes and roughly how you use the new feature, so that other folks can hack on it and put together a coherent set of instructions for general consumption. Incoherence commences.

There is a new option for bootstrap filesystems in diskfs (i.e. ufs and ext2fs), --boot-command', which specifies a command to run instead of init. This command can be another bootstrap filesystem, which runs with the original bootstrap filesystem as its root directory.

The idea is that the bootstrap filesystem and the exec server are started up as usual, but rather than accessing /hurd/init and all the hurd servers through that filesystem, it can run another bootstrap filesystem program and give it as its storage, access to a regular file in the original bootstrap filesystem that contains a filesystem image for the inner bootstrap filesystem.

An example, using ext2fs. You'll recall the lines in /boot/servers.boot that usually start up the bootstrap filesystem and the exec server:

    /hurd/ext2fs.static --bootflags=${boot-args}
--host-priv-port=${host-port}
+--device-master-port=${device-port} --exec-server-task=${exec-task}
-Tdevice ${root-device}
+$(task-create) $(task-resume)
    /lib/ld.so.1 /hurd/exec $(exec-task=task-create)
For this to work, serverboot must be able to read /hurd/ext2fs.static and /lib/ld.so.1, and that ext2fs.static must be able to read /hurd/exec and the shared libraries it needs. For using --boot-command, let's modify that boot script:
    /hurd-install/hurd/ext2fs.static --bootflags=${boot-args}
--host-priv-port=${host-port}
+--device-master-port=${device-port} --exec-server-task=${exec-task}
-Tdevice ${root-device}
+$(task-create) $(task-resume) --chroot=/hurd-install --boot-command
/hurd/ext2fs.static
+--bootflags=${boot-args} hurd-root.img
    /hurd-install/lib/ld.so.1 /hurd/exec $(exec-task=task-create)

Here I assume you have a directory called /hurd-install on the partition you're booting from; this can be, e.g. your cross-compilation install directory on your linux system (i.e. where you did "make prefix=/hurd-install install" after cross-building libc and hurd).

But you only really need a few files for this to work. serverboot reads /hurd-install/hurd/ext2fs.static and /hurd-install/lib/ld.so.1, and then the bootstrap filesystem is running on the real root partition. Notice we give it a --chroot option, which makes it pretend the directory /hurd-install is the root of the filesystem. Now it starts the dynamic linker that serverboot loaded into memory, and its root filesystem is the tree under /hurd-install provided by the bootstrap ext2fs. There it needs to find /hurd/exec and all the shared libraries it uses (use ldd or objdump --private-headers to figure it out); then we have an exec server. At this point the bootstrap fs would ordinarily run /hurd/init.

Now --boot-command comes in. Note that the --boot-command switch must be the last thing on the command line (after the bootstrap root device), and all arguments following it (with or without dashes) become the command line for the boot command. This command runs, as init would, with a root and current directory provided by the bootstrap filesystem (that /hurd-install directory). Here we have /hurd/ext2fs.static again, though it could as well be /hurd/ufs.static (in theory you could use a dynamically-linked fs program here, but I wouldn't recommend it).

This ext2fs process gets the same --bootflags argument, which tells it to be a bootstrap filesystem. But rather than having a device as its storage, it can use a regular file found in the root filesystem provided by its parent (we don't give a -Tfile option because it's the default). Now this second ext2fs reads its filesystem image from the file hurd-root.img (aka /hurd-install/hurd-root.img). Noticing that it didn't get the magical --host-priv-port and --device-master-port options from serverboot, it decides it must be a boot command and so does the fsys_getpriv RPC on its bootstrap port (as init would) to get the privileged ports from its parent. Then it notices it didn't get the magical --exec-server-task option, and does a normal lookup of /servers/exec in the root filesystem provided by its parent, just as any non-bootstrap filesystem would do. For this reason, your /hurd-install directory must contain a /servers/exec file; an empty file will do, and it doesn't need any special setup.

Now we have a filesystem and it has a port to the exec server, and it proceeds just like a vanilla bootstrap filesystem would: it loads /hurd/init from the filesystem it provides itself, and init takes over and completes the server bootstrapping procedure as normal.

In the penultimate step of the normal bootstrap dance, init sends the fsys_init RPC to the filesystem that loaded it. This is our second-stage bootstrap filesystem. Knowing it's a boot command, it registers its parent task with proc server and then turns around and sends another fsys_init to its parent, the original bootstrap filesystem. On receiving this fsys_init, just as it would from init, the bootstrap filesystem registers the exec server task with the proc server, and then sends it the exec_init RPC with its proc and auth ports. This allows the exec server to get the privileged ports from the proc server, and complete the bootstrap handshake by registering itself with init. Et voila, off to the races.

A final wrinkle concerns the exec server. Recall that we reach the exec server through its active translator setting on /servers/exec--but that was the /servers/exec in the original root filesystem, which noone but the secondary bootstrap filesystem ever had access to (and it abandoned those directory ports in the final phase of bootstrapping the servers with init). Since the filesystem caches its port to the exec server, the root filesystem will continue to be able to execute programs. However, any other filesystem will start with the new root directory, and look up /servers/exec there to reach the exec server. So, it is now necessary to have set a passive translator of /hurd/exec on /servers/exec in the inner root filesystem, which was never necessary before. This passive translator setting is harmless in the normal booting case because the boot-time active translation of /servers/exec causes it to be ignored, so it will become a normal part of hurd installation. (It would be possible to pass on the active translator setting to the inner root's /servers/exec, but doing it this way lets you use a newer or different exec server binary and it can share the library text pages with the rest of the system.)

The savvy hacker already knows how to create a filesystem image in a file, but here are some tips anyway. Always the first step is to decide how big a filesystem you want (i.e. how big a partition you would use if you were using a partition), and make a file that big:

        # dd if=/dev/zero of=/hurd-install/hurd.img bs=1024k count=300

That's a 300MB filesystem, which is big enough for a full gnu-0.2 installation and some hacking space.

Next, create the filesystem. On the Hurd and Linux this is easy; for ext2fs:

        # mke2fs -o hurd /hurd-install/hurd.img

For ufs on the Hurd, and maybe this works on BSD:

        # mkfs.ufs /hurd-install/hurd.img

On the Hurd, "mounting" the filesystem from the image is easy:

        # settrans -a /mnt /hurd/ext2fs /hurd-install/hurd.img

to set just an active translator (mount it once), or replace -a with -p for a passive translator (mounted on demand whenever you use /mnt).

On Linux, you have to use the "loop" device:

        # losetup /dev/loop0 /hurd-install/hurd.img

On BSD, it's the "vnd" device:

        # vnconfig -c /dev/vnd0c /hurd-install/hurd.img

If it turns you on, you can do the equivalent on the Hurd too:

        # settrans -a /dev/whatever /hurd/storeio -Tfile /hurd-install/hurd.img

Then you can use /dev/whatever as a block device:

Linux   # mount /dev/loop0 /mnt
BSD     # mount /dev/vnd0c /mnt
Hurd    # settrans -a /mnt /hurd/{ext2fs,ufs} /dev/whatever

Populate /mnt to your heart's content, and then unmount it:

        # umount /mnt

and detach the device:

Linux   # losetup -d /dev/loop0
BSD     # vnconfig -u /dev/vnd0c

And you should be ready to go. (If you're trying this from a running Hurd, you can just use `fsysopts /mnt -r; sync' to sync the data and then boot the the sub-Hurd with `boot'.)

This is all in theory of course. When I tried to do heavy filesystem writing through /dev/loop0, it crashed linux-2.0.36. Since I have a lot more and faster spare disk than I have patience for linux bugs, I just used a spare partition to populate a filesystem and then dumped the image from the partition into a file.

I'll leave it to the rest of you to work out the kinks in this procedure. Someone could also use this to set up and distribute a "miniroot" filesystem image that could be loaded from your linux or bsd partition with ext2fs/ufs and uncompressed into memory with -Tgunzip, to provide a minimal hurd environment for installation before we get something pared down to floppy size.

Note that you should in theory be able to stack multiple boot commands, for a filesystem within a filesystem within a filesystem as many levels as you want, so you can think up the possibilities. It's also probably possible with a little more work to set up a slightly more populated trampoline boot filesystem with passive translators like pfinet and nfs set so as to approximate an "almost-diskless" network boot of the hurd, or other shenanigans. (There are several more hacks necessary to make that work though.)

I hope someone finds this useful.

Enjoy,
Roland

D'oh! I forgot the last part of my message.

Those of you who are *really* paying attention must be wondering about the active translator setting on the outer root's /servers/exec. Previously the exec server set itself as active translator on /servers/exec after receiving exec_init. But exec_init doesn't come until the outer bootfs gets fsys_init from the inner bootfs, after the servers are all running. So now the exec server acts more like a normal passive translator: the bootstrap filesystem sets the active translator on /servers/exec after receiving the fsys_startup RPC from exec.

A wart in the code is that for --boot-command to be parsed properly by the diskfs argp code, every bootstrap filesystem must use the ARGP_IN_ORDER flag in its argp_parse call.

Roland replied to Marcus' quote, "I'd suggest people experiment with -Tremap rather than --boot-command."

Meanwhile, Daniel Burrows said:

Someone also mentioned that it might be possible to boot the Hurd from a subdirectory of an ext2 filesystem (as if it were in a chrooted environment..)

What's the status of that? (and if support isn't explicitly there..does chroot work in the Hurd, and could we use bootcommand='/usr/local/hurd/bin/chroot_/usr/local/hurd_/bin/init/index.html' to boot up? Would the Hurd's extended filesystem data (translators and so on) mess up the ext2 filesystem that was already there?)

Roland replied:

Yes, use the --chroot=DIR (aka -C DIR) option to your bootstrap filesystem. That is, in the ext2fs line of your servers.boot file, add e.g. --chroot=/usr/local/hurd if your hurd root starts at /usr/local/hurd. Note that you will have to prepend /usr/local/hurd/ to the filenames in your servers.boot file and to the ones you give GRUB on its command line or in its /boot/grub/menu.lst file. (Those two files are read by GRUB and by serverboot to load the bootstrap filesystem and give it those options, so they know nothing about any --chroot setting.)

This has probably been discussed in more detail before; please consult the mailing list archives. Someone should put this in the various FAQs and such too (and if it's already there, shame on you for not finding it there before asking! :-).

He added, regarding the status:

I'm not sure how well it's been tested, but it is all there.

You should realize that there is substantially more risk to your existing filesystem this way than giving the Hurd its own filesystem in one way or another. It has been a very long time since anyone has seen a Hurd bug that scribbled on a filesystem, but if the Hurd scribbles on your filesystem it could in theory get corrupted such that you lose data from the other directories on the filesystem (your Linux portion).

The other way to go is to make an ext2fs in a file on your filesystem. Then figure out what disk blocks that file is on (hopefully it's contiguous, but it doesn't have to be). Then you can use -Tremap with a blocklist to tell your hurd bootstrap filesystem to use a virtual disk made up of those particular disk blocks. The details have been posted here before, search the list archives for `remap'.

Regarding any danger to existing ext2 filesystems, Roland went on:

No, but you must make the filesystem compatible by setting its `creator_os' field to `hurd'. You can do this when making the filesystem with the `-o hurd' flag to mke2fs (it's the default when running mke2fs on the Hurd). There is a portable shell script installed in `/sbin/e2os' in the Hurd dist that will diddle the creator_os field on an existing ext2fs using dd. If you do not do this, you will get `Operation not supported' errors from the Hurd when you try to set passive translators on that filesystem--and the Hurd's root filesystem must contain some passive translators for the Hurd to boot normally.

A `hurd' ext2fs can be used completely normally from Linux. It is safe to run fsck on Linux, and safe to mount and write files on Linux.

I believe that using e2os to change a linux ext2fs to a hurd ext2fs is safe, but it uses dd to the device to diddle your superblock, so it's a fundamentally dicey thing to be doing, and don't come after me with a bat if you trash your filesystem. The Linux kernel ignores the creator_os field completely, and e2fsck understands the field and does the right thing for hurd. Make absolutely sure the filesystem fsck's cleanly before you tweak it.

 

3. Partition Information
23 Jul 1999 - 26 Jul 1999 (5 posts) Archive Link: "Booting HURD"
People: Vitaly LugovskyMichael BacarellaGordon MatzigkeitMark Kettenis

Vitaly Lugovsky couldn't get the Hurd to boot, and Mark Kettenis (among others) told him his partition information was wrong. Mark explained, "The way partitions are named is a bit confusing. While the drive numbers are 0-based (hd0, hd1, hd2, etc.) the partitions are 1-based (hd2s1, hd2s2, etc.) So you probably have to specify /dev/hd2s1/boot/servers.boot if the configuration script is located on the first partition of the master drive on the secundary IDE interface (hd2s1 corresponds to hdc1 on Linux)."

Under the Subject: Why it's so slow?, Vitaly apparently got the Hurd up and running, but found that it was about a third the speed of Linux. Michael Bacarella explained, "The Hurd is still very much in development. A lot of these speed issues will eventually be resolved but you should be aware that the Hurd imposes more overhead than something like Linux. This makes the Hurd more robust and powerful, but at a slight performance hit."

Vitaly also asked if there were any CVS trees for working on the Hurd, gnumach and debian-hurd development. Gordon Matzigkeit pointed him to http://www.gnu.org/software/devel.html.

Vitaly also asked where 'ifconfig' was, so he could use the network. Michael Bacarella recommended:

Use the server 'pfinet' for this. The way to go about it is:

settrans /servers/socket/2 /hurd/pfinet --address=x.x.x.x
--interface=eth0 --gateway=x.x.x.x --netmask=x.x.x.x

pfinet --help should work too. :)

 

4. Hurd Installation Disks
24 Jul 1999 - 27 Jul 1999 (12 posts) Archive Link: "Debian GNU/Hurd installation"
People: Vitaly LugovskyGordon Matzigkeit

Vitaly Lugovsky felt that a set of boot/installation disks would be a good thing; Edmund GRIMLEY EVANS pointed out that the lack of hardware support in the Hurd might make a poor impression on newcomers trying to use the disks. Gordon Matzigkeit also added that Miles Bader had native boot floppies working a long time ago, though by now they were a bit out of date.

 

5. biff, iselect, lftp, most, par, psutils, recode, gdb And gnumach Successes
28 Jul 1999 (1 post) Archive Link: "new packages"
People: Marcus BrinkmannMark Kettenis

Marcus Brinkmann announced a number of new packages: biff 0.10-1, iselect 1.1-0.2, lftp 1.1.981023-1, most 4.8.1-0.1, par 1.50-4, psutils 1.17-6, and recode 3.4.1-11; he added that the bsdgames package was giving him more trouble than all the above combined.

He went on to say that gdb 4.18-1 seemed to work. He said:

This one is heavily patched by Mark Kettenis and works great, whereas 4.17 doesn't seems to work anymore even on simple programs. Everyone please upgrade to 4.18. Installing the libc0.2-dbg libs and setting

export LD_LIBRARY_PATH=/lib/libc_debug:/X11R6/lib

you can see in which function a segfault occurs. Send us the backtrace ("bt") and we will be able to find out the problem faster.

He also said that gnumach 1.2-1 worked, and he explained:

"This one has the promised fixed network card autodetection, so if it still doesn't recognize your network card, send in a bug report! If it does recognize the card incorrectly, send also a report. If you can, try to compile GNU Mach with different orders in linux/dev/driver/net/Space.c and report which order works. If you have multiple cards, try them all."

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License, version 2.0.