Hurd Traffic #112 For 22 Oct 2001

By Paul Emsley

Mach 4 (http://www.cs.utah.edu/projects/flux/mach4/html/) | Hurd Servers (http://www.gnu.org/software/hurd/hurd.html) | Debian Hurd Home (http://www.gnu.org/software/hurd/debian-gnu-hurd.html) | Debian Hurd FAQ (http://www.debian.org/ports/hurd/hurd-faq) | debian-hurd List Archives (http://lists.debian.org/#debian-hurd) | bug-hurd List Archives (http://mail.gnu.org/pipermail/bug-hurd/) | help-hurd List Archive (http://mail.gnu.org/pipermail/help-hurd/) | Hurd Reference Manual (http://www.gnu.org/software/hurd/reference-manual.html) | Hurd Installation Guide (http://pick.sel.cam.ac.uk/~mcv21/hurd.html) | Cross-Compiling GNUMach (http://pages.hotbot.com/sf/igorkh/gnumach-cross.txt) | Hurd Hardware Compatibility Guide (http://www.urbanophile.com/arenn/hacking/hurd/hurd-hardware.html)

Table Of Contents

 

1. Paving The Way For Diskless Hurd
16 Aug 2001 - 7 Oct 2001 (28 posts) Archive Link: "Getting rid of serverboot. Finally"
Topics: Bootloaders, FS: ext2
People: Marcus BrinkmannRoland McGrathNeil WalfieldThomas Bushnell

Following the discussion in August (http://kt.zork.net/debian-hurd/dh20010828_105.html#9) , about serverboot, Neil Walfield and Roland McGrath were tieing up loose ends. Marcus Brinkmann under the topic gnu-20011016.tar.gz available (http://lists.debian.org/debian-hurd/2001/debian-hurd-200110/msg00155.html) describes serverboot and why getting rid of it is a good thing:

Basically, serverboot was the program which was booted on Mach, and it was responsible to boot the whole Hurd system.

In the Hurd, you need to start with two programs, the bootstrap filesystem (ext2fs) and exec. Both are sufficient to load and start the third server etc, until the boot completed. This is what serverboot did. serverboot was needed as the single-module boot supported by Mach was not sufficient to boot the Hurd directly.

serverboot could read files directly from the disk partitions with various file system formats like minix, ufs, ext2fs, fat, and then start them. So, it was a stripped down file system parser and a stripped down exec. It duplicated functionality that was already in GRUB and other Hurd servers.

Roland put the code to support multiple modules in Mach, and now we can let GRUB do the loading of the files (it already supports many filesystems, and network storage!), and we can let Mach do the conversion of files to tasks. So, the functionality is nicely split up in places where it fits better. The Hurd is booted directly, and there is no middle man who is restricting us from doing more clever stuff, like booting the Hurd from the net etc.

Roland had asked Neil to check serverboot with various mach configurations. Neil confirmed that the various configurations worked as expected and sent a patch.

There had been a problem with printing long strings, Neil saying that the message for the first module that is loaded wraps onto the third line. Roland mused: "Hmm. I hadn't anticipated they would be so long, but I guess they are. Well, I guess it doesn't really work to try to erase with CRs then. My motivation for doing the output that way was to have something printed for each module while it's loading so that a barf there makes sense to you, without having the printing in the normal case be too voluminous. "

Several posters were intrigued by the idea of diskless booting and how to implement it, Neil Walfield saying "Well, what you are dealing with is a the chicken and egg type problem. If a root device is a network store, the server will need to open a network connection using socket() (or another similar function which accesses pfinet). The implementation of socket () opens /servers/socket/2 to get a file descriptor. However, as this would happen before there is a root file system, the function will fail. "

Thomas Bushnell considered this and replied "Note that this is already solved for bootstrap diskfs. It gets special arguments that tell it where to find things: it's port space is pre-filled with the port numbers it needs to talk to the other servers, and the bootstrap code is responsible for setting that up. Similarly, a bootstrap nfsd would need such special code, and so would pfinet. "

Roland did not like diskless booting continuing under this thread and said so - he had already considered the relavent details. Everyone then shut up.

EOT (for now).

 

2. OSKit-mach Networking Works
25 Aug 2001 - 9 Oct 2001 (20 posts) Archive Link: "oskit-mach & oskit-20010214: network"
People: Daniel WagnerRoland McGrathKevin Kreamer

Continuing from the previous oskit-mach network discussion (http://kt.zork.net/debian-hurd/dh20010911_107.html#4) , Daniel Wagner wrote "Finally, I managed to get it [networking using oskit-mach] working! Roland, your tips were exactly what I needed!" . Daniel posted a patch which Roland did not like saying: "I don't understand the rationale behind using base_irq_soft* and diddling with osenv_intr_*." . Daniel corrected the code and after a few rounds of refinement, Roland applied Daniel's code.

Kevin Kreamer checked it out and confirmed that it worked.

 

3. Pronunciation
26 Sep 2001 - 8 Oct 2001 (4 posts) Archive Link: "no real problem"
People: Thomas Bushnell

Florian Quetting asked how to pronounce "Hurd" and "Hird". Thomas Bushnell replied "My intention was that they be pronounced as absolute homonyms" .

 

4. Video Memory Access From OSKit-mach
1 Oct 2001 - 7 Oct 2001 (23 posts) Archive Link: "video mem access with oskit-mach"
People: Marcus BrinkmannRoland McGrathThomas BushnellKevin KreamerPaul Emsley

These are a couple of addition features necesary to get X working with oskit-mach:

"colortext" needs access to video memory when it pokes characters. Marcus Brinkmann suggested using special_mem_device in oskit-mach to provide access. The other issu, that of opening the keyboard in raw mode, could be fixed by setting the DC_raw console flag.

This will also provide a sensible console using "colortext", and in order to do this, Marcus Brinkmann asked "Is it acceptable to put a hack in to set it in raw mode (without advertising it as the final interface)?"

The video access was addressed first, Roland replying:

I believe that everything should already be in place in Mach (both flavors) and in the Hurd for mapping physical device memory such as the frame buffer. mmap on /dev/mem ought to dtrt already. If not, it's a bug.

The only reason to have a more specific device for this is security, e.g. /dev/vga that can mmap only a constrained range. We can do this by Hurdish means if we ever care to. Theoretically one could use a remap store on top of /dev/mem, whos runs give mostly holes and only map the address ranges you want to allow for the frame buffer.

Roland later added:

We need a new interface for enabling io ports, because the old CMU style that gnumach has is just bogus. It's easy enough to implement a some more sane RPCs into the kernel, and then we can use that to implement the `ioperm' interface in libc compatibly with Linux.

Well, maybe the old style sort of makes sense. But we need some changes.

One annoyance that's not really a problem in practice is the inheritance (or lack thereof). The Linux `ioperm' model applies to all threads in the process (I think) and is preserved across exec (so the documentation says). Conversely i386_io_port_add works on a single thread, so it's not pretty for one `ioperm' call to affect all current and future threads in the process, or the future thread after an exec.

So it might be nice to have a kernel interface that is task-based and affects all threads in the task. (Incidentally, io port access is task-based on L4.)

The way the current interface works (not supported in oskit-mach) is that you pass a device_t (i.e. a kernel device port from device_open) and that refers to a set of io ports. This has the nice property that you can have a capability for a subset of the io ports that you can pass around. This could make it possible e.g. to have a root-owned translator allow a non-root X server to get a capability for a limited set of io ports, just as a remapping store translator could allow a limited subrange of physical memory from the "mem" device.

But the Linux `ioperm' interface is simply that only root can call it and can (of course) use any and all io ports. (BSD has an interface called i386_get_ioperm/i386_set_ioperm that is essentially the same.)

The capability-using interface is certainly more Hurdish. The reason it's not implemented in oskit-mach is the overloading of device ports for these the io access capabilities. In gnumach there is just some magical table someplace of what devices correspond to what port numbers, so if you open "kd" that gets you access to poke the video ports. That is just nutty.

Perhaps we should have a capability-based interface, but just use special ports for this instead of device_t's. e.g., there could be a new call on the device master port to give you a capability port with the given bitmask of io ports for access. Then these ports would be the valid arguments for i386_io_port_add.

This met with approval from Thomas Bushnell and Marcus.

(ed. [Paul Emsley]

There followed substantially more discussion about bitmasks and type conversion which sadly went over your summarizer's head :).

)

As to the keyboard access, Roland said:

Anything using the oskit-mach "console" driver at all should be considered a temporary hack. For one thing, if you boot with the serial console then there isn't any device that gives you access to the PC keyboard. Moreover, the "direct_cons" driver is very simplistic, not all that efficient or robust or featureful (probably doesn't work with all the oddball hardware flavors a real driver would). That driver (and the serial driver used by the serial console) are really intended for bootstrapping and kernel debugging. (Also note that there is nothing at all in oskit-mach that will get you the ps2 mouse port.)

The long-term solution is to cleanly incorporate some real keyboard/ps2 drivers into the oskit (encapsulate *BSD and/or Linux drivers), along with USB and whatever else. But that is some substantial work and not a task I would recommend for anyone who isn't intimately familiar with the oskit and its style of driver-stealing.

All that said, yes I think turning on DC_RAW would make it more or less work. There is no interface for diddling the keyboard LEDs, which you would like to be able to do for real usability.

Elsewhere (under the topic [patch] direct console set_status (http://mail.gnu.org/pipermail/bug-hurd/2001-October/005549.html) ) Kevin Kreamer sent a patch to enable DC_RAW via devide_set_status on the console. Roland McGrath did not include it because it was too interim a solution. Roland explained:

A better interim kludge would be to give oskit-mach a cheezy "kbd" device that sets DC_RAW while it's open and resets it when it's closed. Then you would use that instead of "console" as the device to read scancodes from. When "console" is a serial console, there is no reason it should be involved with the keyboard hardware at all and you'd still like another device you can open to access it. (In fact, I would think that this would be by far the easiest way to debug your user-space console code: boot with a serial console and leave the PC keyboard/display purely for your debugging.)

The real solution is to add a real keyboard (and ps2) driver to oskit.

Marcus Brinkmann added that the user-space drivers (for keyboard and ps/2) would probably be based on OSKit too in future.

 

5. Resource Limits: setrlimit()
5 Oct 2001 (5 posts) Archive Link: "ssh as user: setrlimit failed"
People: James MorrisonRoland McGrath

Raimund Huemmer reported the problem encountered when running ssh as a normal user: setrlimit failed: Function not implemented.

James Morrison had seen the problem before but said "However, if you compile it yourself and use that compiled and not installed copy it will work. " James went on to find the cause of the problem, which was that setrlimit() was being used to set the dumped core size to 0 and this fails with the Hurd.

James asked "Should GNU Hurd be changed to accomodate this or vice-versa? " This being a kernel issue, conversation moved over to bug-hurd (http://mail.gnu.org/pipermail/bug-hurd/2001-October/005419.html) , Roland McGrath saying "the problem is definitely on our end " because resource limits are poorly implemented.

What we have is very half-assed, and should do whatever makes the most programs happy without too much giving the impression that limits really exist where they actually don't. (In fact, the only limits that really work are RLIMIT_NOFILE, RLIMIT_DATA--which controls only sbrk, and RLIMIT_CORE--which really only works when set to 0, not to limit the size of a core file. That latter failing is at least mitigated by the fact that we don't have dumping core files implemented either! :-)

I am open to suggestions on what the setrlimit function in libc should do. Currently it fails with ENOSYS if rlim_max (the hard limit) so set to anything other than RLIM_INFINITY. It lets you set rlim_cur (the soft limit) to whatever you like, for all the limits, even though most of them don't really do anything. I suppose it wouldn't hurt to just permit setting the hard limits however you like as well (though actually we don't save them).

Roland went on to discuss the server that handles core dumps "If RLIMIT_CORE is zero, then you don't call the crash server at all (we don't call it the "core server" because then we'd want to use names like /servers/core and /hurd/core, and we're afraid of well-meaning cron jobs). For a limit of another finite size, it's just like the RLIMIT_FSIZE limits that we also don't implement--it would be arranged somehow with the filesystem server."

 

6. TSS Switching
7 Oct 2001 - 15 Oct 2001 (16 posts) Archive Link: "TSS switching"
People: Roland McGrathMarcus Brinkmann

A TSS is a Task State Segment, an Intel hardware structure for organizing segment selectors and permissions. TSSs are used in context switching. Marcus Brinkmann and Roland McGrath had a substantial discussion about fixing the current I/O permission code.

Roland suggested:

You know, why don't we just punt this whole implementation?

From looking at what Linux does, it appears to be sufficient just to diddle the bitmap offset field or the bitmap itself in the in-memory tss, without doing anything special to reload it into the CPU. Doing a task switch is probably at least as expensive as diddling the bitmap. Linux doesn't use task switch at all, it just diddles the bitmap in the single tss.

So let's do that. It's easy and you probably don't have as much weirdo Intel magic to debug than with tss switching.

Marcus was a bit resistant to this idea because he did not understand the current code, however he was eventually partially persuaded, saying "I think I will work on a solution for oskit-mach and forget gnumach for now."

 

7. LG 40x CDROM Causing Boot Problems
7 Oct 2001 - 9 Oct 2001 (10 posts) Archive Link: "Loop booting gnumach.gz from CD G1-1"
People: Marcus BrinkmannMichele Dalla SilvestraDiego Roversi

Michele Dalla Silvestra tried the new ISO image (hurd-G1-CD1.iso) but it failed to boot on one of his computers. He found that the problem was due to his CDROM drive (LG 32x, ASUS 40x, Philips 24x, BTC 8x worked without problem), however his LG 40x caused the failure. Diego Roversi suggested that he disable ultradma support in the BIOS and for a different solution, Marcus Brinkmann assigned him some (home)work:

It would be good if you could get into (cross) compiling gnumach so you can test several things. First, compile yourself a smaller version of the kernel with just the drivers you need. If that works, there is a driver/hardware conflict, and it would be good to learn more about your hardware (controllers, other cards etc). For example, kernelas after 1.2-9 come with ncr53c8xx driver enabled. Maybe leaving that one out will fix it already.

If not, then it's one of the other, not driver related changes sinve then that causes the bug. There have only been few, and I can't imagine which one was it, but it won't be very hard to isolate the change that prevents it from booting.

There was no reply.

 

8. Debian GNU/Hurd G1 CD Post-Install
7 Oct 2001 (2 posts) Archive Link: "My Hurd-G CD experience (fwd)"
Topics: Bootloaders
People: Jeff BaileyPhilip Charles

Jeff Bailey tried the new G1 Hurd system and reported that it was not obvious how to reboot into GNU/Hurd after the installation. Philip Charles said that he would therefore add a message saying that a GRUB boot floppy is needed on rebooting.

The question was then asked: can grub be activated (and installed) at the Linux ramdisk (system installation) stage? Jeff replied:

I think the trick is to have a second copy of grub available compiled for Linux. Basically run grub --batch, with a fancy install parameter to tell it to point to all the right places on the Hurd partition.

However, if you're short of time, I suspect that simply having a linux-compiled grub shell on the ramdisk would be a big improvement. That way it would be possible for someone to install without a floppy.

 

9. PowerPC Port
10 Oct 2001 - 12 Oct 2001 (16 posts) Archive Link: "PowerPC port"
Topics: FS: ext2
People: Peter BruinRoland McGrath

Peter Bruin came up with the astounding news that he had got the PowerPC port of the Hurd at least partially working and that he had "compiled bash, textutils and fileutils, most of which runs. Ext2fs runs in read-only mode, but often crashes in writable mode. Exec crashes when trying to run shell scripts. There are many other bugs, but at least some of it works. "

That Peter could do as well as he did in total silence was suprising to many people including Roland McGrath, who wrote "That sounds great! I am frankly amazed you could get this far without asking us for a lot of help. Please post all your code asap so we can start getting it integrated and have other people work on it. "

Peter did not port GNU Mach, instead he used the OSF Mach (that is part of MkLinux) unmodified and describes the changes he'd made to the Hurd: "Most changes I had to make were either processor-related or had to do with the differences between GNUMach and OSF Mach (which already existed on the PowerPC, so that I didn't have to worry about getting Mach to work). And I was amazed that I didn't have to change anything in the Hurd code (except for the GNU vs. OSF Mach things)! "

We've always thought it was a clean design, thank you very much.

Aside from the kernel, the hard porting work is the signals code in libc. If you got this code to work right, well you're the first person to do it without help from me, so congratulations!

I'd like to get your code incorporated. I hope your ppc-specific changes are easy to separate from your osfmach-specific changes. I would like to get the ppc hurd code into libc on its own, so it's ready to go if anybody ever ports gnumach (or a different microkernel) to ppc.

Also, there has been some interest in the past in running the hurd on top of osfmach on x86. If your osfmach support code is clean, it should cover that pretty well too.

I am very eager to see all your code.

Peter discussed the signal code and said that he had used the sysdeps/mach/hurd/alpha as inspiration (this is code of an old unfinished Alpha port). Roland replied to this:

Yowza! No, there has never been a working Alpha port of the Hurd. There once was an Alpha port of CMU Mach 3.0, so it was feasible enough, but it never actually happened. I really don't recall it clearly, but evidence (libc/ChangeLog.4) suggests that on November 15, 1994 someone handed me an Alpha architecture book, I got ahold of the header files from the CMU Mach port, and whipped up some code that looked right to me. I've never actually had my hands on any Alpha hardware, even to this day.

So if you based your ppc code on my alpha code and came up with something that works, then, well shit, we BOTH must be damn good! :-)

If asked, I would have suggested that you start with the mips code. The mips port was done by Kazumoto Kojima, and he did actually have it working at one point (though he seems to have lost all interest and doesn't work on the Hurd any more). That is the only RISC processor a working Hurd port had been done to before (the only finished port at all other than x86, in fact).

Roland was sufficiently enthusied to ask for powerpc documentation (www.mot-sps.com (http://www.mot-sps.com) and www.mot-sps.com (http://www.mot-sps.com) ) and went on to explain how to test rpc_trampoline.

Peter went on to explain about running Linux and the Hurd concurrently: "You can indeed run the Hurd from within MkLinux, but it can't read from the console because Linux also has the console open; it might be possible to fix that. Maybe the Hurd can also be made to run within MacOS X, but Apple has replaced OSF Mach's deviceinterface with a completely different one. "

Roland had some thoughts about the fixing this console problem: "I'd think you would want to use a virtual console device by making a version of `boot' that runs on MkLinux. The code in hurd/boot/ux.[ch] and hurd/boot/Makefile rules to build `uxboot' were for doing that on the old CMU unix singleserver. You could hack up something similar (or better, given ELF linking) to work with MkLinux. "

 

10. libdiskfs Bug Fix
11 Oct 2001 (3 posts) Archive Link: "Bug#115311: Assertion failure in libdiskfs while moving directory"
People: Moritz Schulte

Moritz Schulte got an an assertion failure when doing the following:

Moritz posted a fix to libdiskfs which Roland included.

 

11. parted 1.5.4 Crashes gnumach
12 Oct 2001 (10 posts) Archive Link: "parted 1.5.4 crashes gnumach"
Topics: FS: FAT
People: Marcus BrinkmannNeal Walfield

Marcus Brinkmann came up with the grim news that the newest parted crashes gnumach:

I have tried parted 1.5.4 in the Hurd running the latest GNU Mach package. I run "check 1" to check the first partition, which is a FAT. It really had a bad impact: I got completely random crashes (obviously some corruption) in the kernel, often during the check, but also later. This happened when running it as a normal user or as root.

What is so special about parted that it brings down gnumach to its knees so easily? What interfaces in the Hurd and GNU Mach are used by parted, and in what way? It seems there is something unusual that really disturbs it.

Neal Walfield said that Parted uses the store interface "nothing terribly special" . When describing the crash, Marcus said "The error is different each time, the crash itself spurious. I am quite certain that random data in the kernel address space is corrupted. The symptoms are page faults, assertion failures and hangs in appearingly random places (schedular, virtual memory handler, etc). "

In reply, Neal said that this indeed may be due the kernel, "however, I am still a bit suspicious of Parted" .

Marcus obviously went back and tested with another kernel. He came back later saying: "You are right. It even brought down Linux."

Finishing off the thread Marcus said: "So everyone be warned that using parted 1.5.4 is very dangerous."

 

12. GRUB For New CD Set?
14 Oct 2001 (2 posts) Archive Link: "H1 - grub."
Topics: Bootloaders
People: Thierry LarondeJeff BaileyPhilip Charles

Philip Charles wanted to know if Grub 0.9 should be included in the new CD set (H1) that he is preparing. Thierry Laronde replied that he would prepare a grub that could be used in a "Demo-Hurd", which Thierry describes thus: "[it] allows [one] to boot the Hurd directly from the CD since we won't have any limitation in size (HD emulation) and since GRUB is able to load the multiboot compliant HURD. "

Jeff Bailey added amusingly: "From Linux, I haven't used lilo now in over a year (and I tend to convert friends machines while they're not looking). "

 

13. Virtual Bootable Disks
14 Oct 2001 (3 posts) Archive Link: "GRUB, CD and extended floppy formats: HOWTO"
Topics: Bootloaders, Emulators: plex86, FS: ext2
People: Thierry LarondeMarcus Brinkmann

Thierry Laronde posted a script to create virtual bootable disks. Cool. Thierry expanded:

You create a tar archive with all the files that you want to put on the virtual disk INCLUDING the ones for the correct version of GRUB. All the remaining stuff is handled by the script which creates an empty file of the correct size, partitions it (in case of HD emulation), put a filesystem on it (only ext2 at the moment, but this is just a limitation of the script), run GRUB with the correct options, that is uses `setup' and the `geometry' function with new options to set the correct geometry.

Once this is done, you have a virtual bootable disk, that can be used on floppies (1.20, 1.44, 2.88, 1.68, 1.74) or as a HD for El Torito emulation.

The script displays at the end a remainder of the command to use with mkisofs.

You than have all the features of GRUB (chaining different menu files, loading different kernels and so on, network boot etc.).

Marcus Brinkmann liked the sound of this, particularly in use with plex86.

 

14. Kernel Updated
14 Oct 2001 - 18 Oct 2001 (8 posts) Archive Link: "New core packages: gnumach, hurd and oskit"
People: Neil WalfieldPhilip CharlesMarcus Brinkmann

Marcus Brinkmann let us know that he had updated the binaries of OSKit, gnumach and the Hurd. This Hurd can now boot without serverboot (see Neil Walfield's Installation Guide (http://web.walfield.org/pub/people/neal/papers/hurd-installation-guide/english/hurd-install-guide.html) ).

Philip Charles said that these would be included in H1 CD set.

 

15. Hurd top
18 Oct 2001 (1 post) Archive Link: "pptop [was Re: Various bugs]"
People: Paul EmsleyJames Morrison

Paul Emsley announced a Hurd version of top, saying "It's fun to see the Hurd's bits in action" . You can get the code from Paul's site (http://www.chem.gla.ac.uk/~paule/hurd/) or James Morrison's site (http://hurd.dyndns.org/hurd_release.html) . The program is not (yet) fully-featured and has a memory leak (which James is investigating).

 

16. Compiling OSKit-mach
17 Oct 2001 - 18 Oct 2001 (4 posts) Archive Link: "oskit-mach dodgeyness and package compilation"
People: Bob HamJeroen Dekkers Marcus Brinkmann

Bob Ham had problems compiling CVS oskit-mach:

First problem was mig needing i386-gnu-gcc, so a link in /bin now points that to gcc. This implies that the mig package was cross-compiled; is this the case? Shouldn't it not be?

It all compiled ok and I tried to boot the resultant kernel. The output from the kernel was staggered over lines, much as what happens when telnet goes dodgey. What it reported wasn't very encouraging either; problems with IRQ probes for IDE devices. After that, it just hung.

Marcus Brinkmann said that the i386-gnu-gcc for mig problem was his fault and a new package would fix it soon.

Jeroen Dekkers pointed Bob to the oskit mach documentation (http://www.etherhogz.org/doc/oskit-mach.html) and said that if he was using the debian oskit package, then he would need to use the unstable package (0.97.20010214-4).

Bob finished up saying "I downloaded the most recent oskit package and it's cleared up the previous problems" .

 

17. Dual Booting Progress?
19 Oct 2001 (3 posts) Archive Link: "dual-cpu intel860?"
People: Douglas HiltonRoland McGrath

Douglas Hilton has been running GNU/Hurd on his dual PII450, but on only one cpu. Douglas apparently had tried to use SMP but there was an error: "it calls a trap when I boot it and I haven't been able to debug it yet" .

Interestingly, Roland McGrath replied to this: "We have a suspicion about that and I might have just fixed it. But noone has been actively hacking on SMP, so we have not concentrated on its problems lately. Once you get as far as booting, there are known unfinished things for SMP (I put in #warning lines so you'll see them when you build with --enable-smp=2). "

There was no reply (we are left on tenter-hooks!).

 

18. Shadowfs Discussion
11 Oct 2001 - 14 Oct 2001 (5 posts) Archive Link: "Shadowfs - some notes"
Topics: FS: ext2
People: Roland McGrath

Roland McGrath had some insight as to how shadowfs should work. Roland agreed that in a shadowfs that supports writing, it should create whole directory hierarchies: "Anything else just doesn't make sense as a user. Incidentally, this is what BSD's "union" mount type does."

Roland went on to add:

Just as an aside, might I suggest supporting relative pathnames taken as starting at the underlying node, so:

settrans /shadow /hurd/shadowfs --writable . /b

is the same as your example, but uses the directory underneath the translator at /shadow in place of /a. On BSD, this is done with:

mount -t union -o -b /b /shadow

io_restrict_auth is for functions like creating directories. It's not entirely clear from the name or the comment in io.defs, but when a directory port returned by io_restrict_auth is used to create a node, that node's uid and gid are set to the uids[0] and gids[0] parameters of the io_restrict_auth call. So really the way to think of io_restrict_auth is that it gives you a port that the server treats as if it had done an auth transaction and gotten your uids/gids parameter arrays as the results from the auth server. So, use io_restrict_auth to get a port to the writable underlying directory on the user's behalf, and make dir_mkdir or file-creating dir_lookup calls on that port.

I don't think it would be a problem to keep the in-core virtual directory nodes alive for the path taken to any externally live virtual node. I'm not entirely sure what your implementation plan would be without doing that. That is, virtual directory nodes keep a pointer to their dotdot node. When you need to create a file in an underlying directory, you get a port to your dotdot's underlying directory and make the appropriate dir_lookup call on that. Getting a port to your dotdot's underlying directory is a recursive process going back up to the root. That is, each virtual directory has a cached port if the underlying directory has been created or an existing one used, and if there is no cached port you get your dotdot's port and do the lookup and dir_mkdir as necessary.

Later Roland went on to say:

[Security issues] are not particular to shadowfs, this is an issue with any translator. The translator, a process owned by some user joebob, is deciding what fs calls to make where. So security demands that it not be able to usurp a client's privilege for those calls, as would be required for joebob's translator to create a file or directory owned by root. I would argue that even if we had a user-retries interface as Moritz suggested instead of io_restrict_auth--so the client is the only one directly exercising its privilege--it would be insecure to have it generically obey the redirection from the server. In that case, the server couldn't abuse your privilege generically to create other files or whatnot, but it could still abuse it in subtle ways like causing you to write a file other than the one you wanted written. This is the same fundamental issue as symlinks owned by joebob, which is exactly a case of the user-retries interface. But programs know about symlinks and can (and do) take explicit security precautions before following what the filesystem told them to do.

It's obviously useful to have certain translator programs that root trusts to behave "properly" even when set on a node owned by a random user. You can do this trivially by making /hurd/shadowfs setuid root. What's probably more desireable is to install root-owned /servers nodes for the trusted translators, and have them all written to use fshelp_delegate.

I've never thought before about using shadowfs as a bootstrap filesystem for a new sub-Hurd. The basic sub-Hurd concept is that `boot' gives you a virtual version of what real booting gives you. i.e., there is nothing to talk to but the kernel and devices. If there were any way to talk to the underlying Hurd's filesystems, the connection to it would be a new bootstrapping issue. But now that you mention it, this seems like a very useful feature!

I am imagining a scenario where you have a pristine system install image in a directory on your main Hurd (e.g. from a bunch of "make install" from your native development environment), and boot a sub-Hurd whose bootstrap filesystem is a shadowfs reading from your install tree and writing to another directory on your main Hurd. In this kind of scenario, there is still no real visibility between the two Hurds. So I don't think you need to worry about the auth servers and such. The sub-Hurd's auth server is a figment of some user's imagination in the view of the main Hurd. The sub-Hurd's bootstrap shadowfs is welcome to believe whatever it likes about the uids of its users. As far as the main Hurd's filesystems are concerned, that shadowfs is just some task somewhere that has some directory ports authenticated however they were authenticated (i.e. opened by `boot' in the normal way), and it can subset that authorization with io_restrict_auth if it wants to.

There is an entirely different kind of "shadowfs as the bootstrap", that I don't think you were referring to, but bears mention anyhow. That is having a bootstrap filesystem (ext2fs or whatever) that has shadowfs (or anything else) set as the translator on its root node. (Something like this might eventually be the norm.) I don't think this is handled well right now. But what it should probably do is get through the bootstrap as if there were no translator on root, as far as getting init going. Then start up the translator in the normal way to get the translated root port, and give that to init as the root port to start programs with.

 

 

 

 

 

 

We Hope You Enjoy Hurd Traffic
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License, version 2.0.