I would not recommend either [using gcc-3.0 nor disabling the optimization], especially for the oskit. It's not unlikely that the linux driver code in the oskit won't compile properly if you try either one of these. The proven method is to use gcc 2.95.x and -O2.

Re: viewing the registers on the 2nd CPU using gdb:The stub talking on the serial line is only ever running on one CPU at a time. The base_critical_enter/base_critical_leave calls in oskit/kern/gdb_serial.c are there to ensure this stays reasonably sane. Each CPU that hits a trap that should go to gdb will call the oskit function `gdb_trap' (via kernel_trap). If the segments or IDT or whatnot is too fouled up to get to the trap handlers properly (through the locore.S code to kernel_trap in i386/i386/trap.c), then you are SOL. If kernel traps are working on that CPU, then it will get into teh gdb_serial.c code and do base_critical_enter. If another CPU is already in that code, the second CPU will spin until the first one finishes and calls base_critical_leave. So once you get a CPU talking to gdb, you will keep talking to the same CPU until gdb tells it to continue running (at which time the other CPU can come in and talk to gdb). (There are hooks into the gdb code that we could use to write SMP-aware code that would report each CPU to gdb as a separate thread so you could use the gdb thread commands to select the CPU. But that would be a bit of hacking, and it would rely on the IPIs working right and such--not really a way to debug the SMP setup itself.)

Design flaw: Changes don't propagate immediately to other threads in the same task running on another processor. This applies to enable as well as disable operations! Espen Skoglund pointed out that we only need to propagate disable operations, enable operations could be picked up in the fault handler (for extra performance).

In real life, it might be easy for us to make sure that it doesn't happen by enabling the ports before creating other threads, and by never deleting them. This should work fine for the console server, and it should also be fine for single threaded apps like X.

Last time it was discussed, we said that we should probably use the same mechanism as is used for page maps, that is inter-CPU interrupts. The only reason I did not do it is that I have only a limited understanding of the CPU synchronization code, and I would not like to touch it. I also don't think that I have the time to learn enough about SMP right now to get this right.

I haven't considered alternatives, like suspending threads. I am not sure if that would work. I guess that if you do it within the I/O bitmap lock, it could be safe, although it seems to me that it is potentially much slower than the other approach (not that it would matter much).

If that approach actually works, we might put it in for the SMP case until we have a better solution?

The proc server is not really much involved. You need to understand the structure of the system and what Mach tasks and threads are about in some detail before attempting to work on this.

Certainly writing an ELF core file is the right thing to do. How to store the memory is clear, and that part of the file format is just the same as for Linux and other systems that use ELF core files. You can look at the `vminfo' program (utils/vminfo.c) to see how to use vm_region to examine the address space of a task, and then you use vm_read to get the data.

The rest of the information about register state and so on is a little less clear. How we will store it is clear: in ELF notes as other systems do. But how many notes and what formats to use for them is up to us. We need to choose those note formats, and modify gdb to understand them. Mark Kettenis is the authority in the Hurd native gdb port, and I will tend to defer to him on what these formats should be.

I suggest that you start out by just doing the memory dump and not worrying about the rest at all (no notes). Rather than hacking on the crash server directly, it will be easiest to debug this in a little standalone program that just uses pid2task to get the task port of a process, suspends it, and dumps its memory as an ELF core file (i.e. a minimal "gcore"). I might get around to whipping up the basic memory-dumping code soon, and then you could see how it's done and debug the code for me.

Hurah!

At long last, I have succeeded in getting a program to produce and read a core file under the Hurd.

This is not a done project, but I think enough is working to upload some patches. There are some known issues/limitations which I have tried my best to document below.

I can't take much credit for this work at all since Roland McGrath has done a lot of the heavy lifting. I also want to say thanks for the help he has given me and his patience with me. I now have a much greater appreciation for the chain of events following a dereference of an invalid pointer. :)

We now have core dumps! Many thanks to Jon Arney for his hacking on this that spurred me to write some new code, and for helping to debug it.

I have now added a sys/procfs.h header file installed by the hurd package (it's in hurd/include/sys/procfs.h). This defines the data structures used in the note segment in ELF core files. I've added new code to the crash server to write ELF core files using these data structures. The note formats are patterned after those used by Solaris, for which the gdb code to read the formats based on sys/procfs.h types already exists and handles multiple threads.

An old gdb probably ought to be able to read one of these core dumps enough to tell you about the memory regions with `info files'. You can also use objdump and elfdump to see that the contents look sane for the process you dumped.

gdb should understand these core files every bit as well as it does on any other platform if you make sure that the hurd's new sys/procfs.h is installed (there are no other related header changes, so you can just copy it into /include) and rebuild gdb with the following patches. (These patches are against the current gdb in cvs, but they should apply fine to 5.1 as well.) Make sure you re-run configure you don't have an old ../config.cache file in your gdb build, since it will contain lies about sys/procfs.h and its contents.

This should also be enough for gdb's "gcore" command to work. It creates a core file with less complete information than the crash server's core files will contain, but it should be enough for gdb to read it back in again.

For static linking, you have to make specific reference at link time to each store class you want to have available by name at run time. Any module that you get linked in from libstore.a will automagically go into the standard list (because each one has a `store_std_classes' section pointing to the `struct store_class' it defines). So e.g. if you call `store_gunzip_open' or `store_gunzip_create' in your program, then the "gunzip" store type will be available to store_open et al as well. But if you don't refer to a given type, it won't be linked in. (In practice you wind up always getting the "file" type and the pseudo-types like "typed" and "query" just by virtue of calling store_open or store_parsed_open.) To get a desired set of types into a static link, you can use `-lstore_TYPE' (i.e. `-lstore_gunzip' for the "gunzip" type module). That works because there is a libstore_TYPE.a installed for each standard TYPE, which is actually a linker script with the same effect as the linker switch `-u store_TYPE_class'.

When using the shared library (which is almost always the case), the same link-reference plan applies. However, this is extended to the sections named `store_std_classes' not just of the executable but of each shared object that's loaded. Since libstore.so still contains all the standard types, this is in the usual case just like the current fixed array. However, the functions that look for a store type by name at run time will also try to load an external module using dlopen if there is no existing type of that name. It dlopen's `libstore_foo.so.0.2' for the type "foo", and uses dlsym to look up `store_foo_class'. Note that once loaded for a successful open, the module is never unloaded. Thereafter, that module (and any other modules you might have dlopen'd independent of libstore) get searched for `store_std_classes' sections. Because of this (and dlopen's own redundancy detection), you could have a single libstore_foo.so.0.2 module that defined several classes and symlink together the different names by which it might be first loaded.

It had been my original thinking to take all of the standard modules out and make them individually loaded on demand. But all the modules we have are in fact so tiny that the extra overhead and memory wasted for page alignment and redundant relocs would be just silly. So I left all the existing modules in the main library. For static linking, this means that if you call e.g. store_gunzip_open directly then you don't need -lstore_gunzip -lstore, just -lstore. In the shared library, it means that existing types are in practice found in basically the same way as before.

Obviously, the important thing that the new dynamic features make possible is to have new store type implementation modules that are not part of the hurd package itself and can be developed and maintained separately. For example, nbd could be moved out into a standalone nbd_client package for parity with the Debian package for Linux (along with a simple `nbd-client' shell script to give a work-alike interface). In the case of nbd, that doesn't really buy anything because libstore/nbd.c is simple and self-contained (but it would make sense if e.g. the nbd protocol might change to get the client and server from related interdependent packages instead of the client being part of the basic hurd package).

I wanted to use a hierarchy of argp parsers in the console server, one in each driver module, one for focus groups, one for consoles, and one for the main program. However, argp parents and childs shade the options of other childs, in the sense that argp calls only one parser for each recognized option. It is not possible to have common options in different parsers this way (I wanted to enable/disable parsers in parent parsers).

Note: pfinet would have the same problem, except that it avoids it by lack of modularity. libstores would have the same problem except it avoids it by lack of flexibility (all arguments for a store are encoded in the store name). I think both are not acceptable work arounds here, because drivers are platform specific and complex. Here is an example potential console command line I have in mind:

/hurd/console --encoding isolat1 --console maincons --output-device vga --vt 1 --encoding utf8 --focus-group mainfocg --focus maincons --input-device pckbd --layout de --input-device mouse --name mouse1 --device /dev/ttyS0 --protocol mouseman

Or, for example, to remove the mouse: fsysopts /node --remove mouse1

If a memory manager issues a memory_object_lock_request to gain read/write access to a set of pages (i.e. evoking all of the kernel's access rights), the kernel will eventually return any modified pages in the range using the memory_object_data_return message and end the sequence with a memory_object_lock_completed message. This is similar to normal eviction path.

In the latter case, the page is returned to the memory manager with the expectation that it will immediately be written to the backing store and freed. The kernel cannot, however, be sure. As such, it changes the association of the page from the manager to the trusted default pager. This way, if the manager fails to free the page in a timely fashion, the kernel can still flush it to swap.

Yet, what happens in the former case? Specifically, how will the returned pages be evicted? Will the kernel send another memory_object_data_return? Unlikely. It has nothing to return. So, the page will be evicted via the default pager. What is the correct way to give the management of the page back to the kernel? Perhaps, we could use the memory_object_data_supply message (and if we modified it, we could supply it as precious). _The Mach 3 Kernel Interfaces_ cautions against supplying data that has not been explicitly request, however, it does not prohibit it. And yet, what if milliseconds from now we get another message to write to the page? Again, we need to go through the same song and dance -- request the page, write to it and return it to the kernel.

I am trying to understand the motivation for having a relatively complex interface to manage page ownership which, in the Hurd, we do not use.

From what I can see, the pager_memcpy function can be extremely slow. Just consider what happens when we just want to replace a page on disk (which has not yet been read in to memory). pager_memcpy causes a page fault. The kernel sends a message to the manager which reads the page from disk (which is completely unnecessary), then, we write to the page and eventually, it is flushed back to the disk. This is even worse if we are writing to multiple pages -- our thread and the manager thread play ping-pong! This could be avoided by acquiring as much of the range up front as possible.

Ah, the principal motivation is to allow, for example, a pager to manage pages shared between many "kernels". The reason to demand pages back or require locking, in general, is so that you can hand them up to other "kernels".

In principle, we need to do this already! The glaringest security issue with the Hurd right now is the assumption that all users will just take their pagers and hand them to the kernel with vm_map. But they might play as "kernels" themselves. To deal with this, the pagers need to be able to deal with multiple "kernels", and also have strategies for dealing with recalcitrant "kernels" that aren't behaving properly.

Re: pager_memcpy: What's supposed to be going on behind the scenes is that the kernel should detect that you are faulting the pages in sequentially, and ask for pages from the pager ahead of time, optimizing the sequential access case.

But the manager knows; why force the kernel to guess? Say we send:

io_read (file, data, vm_page_size * 4, 0, &amount)

By the time the kernel detects a sequential read, it is already too late to be of any use.