Wine Traffic #4 For 18 Jul 1999

By Eric Pouech

Table Of Contents

Introduction

This is the fourth release of the experimental Wine's kernel cousin publication. It's main goal is to distribute widely what's going on around Wine (the Un*x windows emulator).

Another publisher seems to get interested in Wine.

Francis Beaudet (from Macadamian) announced that they are planning to release a win95 look for the file-open dialogs.

I'll be on vacations for the next couple of weeks, so there won't be any new issues until I get back.

1. Crtdll Implementation

 Archive Link: "Crtdll Implementation"

People: Patrik StridvallMarcus MeissnerPatrik StridvalAlbert den HaanReactOS

Patrik Stridvall sent out a RFC about the implementation of the CRTDLL and the way to use functions (and function prototypes) defined both by Windows in CRTDLL and the underlying Unix OS libc.

Editor's note: for clarity, the reader must not be confused by native. Here, it does not refer to Windows's libraries being used by Wine, but rather the host OS libraries being used by Wine.

What Patrik wants to achieve is something like:

  • CRTDLL calls libc
  • Other parts of Wine calls CRTDLL when ever possible.
  • We needn't do 2 if we doesn't want to.
  • To provide at least the step 1/, Patrick proposed:

    Patrik noted that
    with thr preprocess "magic" ... it might be possible to avoid generating the files during the compile phase and they might be portable between platforms and in the CVS and only regenerated when the interface changes, that is not to often once things stabilize.
    and that
    this scheme also requires that the supported platform has som sort of ELF-like dlopen.

    Marcus Meissner quite not agree and liked better the other way round (not using CRTDLL but rather libc for all the APIs). But, he acknoledged the use of it for some specific functions, especially
    due to lack of threadsafeness of some of the FILE* and memory allocation functions.

    Patrik Stridval then precised some points:
    1. If _every_ CRTDLL function are to be implemented in a _100%_ Windows compatible way, primarly for the emulator, but also for the library, it is essential that we have some way to enable inclution of both libc and CRTDLL includes in order to access the libc structures/macros/functions that might not be exactly the same as the CRTDLL ones (alignment, order of fields, missing/extended fields) etc. There might also be thread-safe problem like incomptible or misssing thread safe function. This way we might have a thread-safe CRTDLL on top of a non-thread-safe libc.
    2. The Wine code that uses CRTDLL instead, needn't check for missing or diffrently behaving libc functions since that will be checked/fixed at CRTDLL level.
    3. It makes it possible to take this one step further a CRTDLL that only calls KERNEL32 and/or NTDLL. That will be useful in ther project like ReactOS that are in themselves standalone OS:es and do not have a native libc. ReactOS for example does have a KERNEL32 and/or NTDLL (I don't remember which) and their CRTDLL is quite rudimentary last time I checked. They might be other NT clones with similar problems. It is IMO good design and might increase the uses of Wine and indirectly bring more programmers to help.
    4. As I hinted in 3 this scheme is extensible beyond libc/CRTDLL. It might provide as way to include both native and internal include files that has namespace collisions and a way to link files with namespace collisions.
    5. All this begs the question. Why use libc at all? Why not in the future do direct syscalls since that will enable use of specific things that the native kernel is particullary good at. This will be a lot more work of course, but why not it will be faster. :-)
    6. Win64: In Win64, long is 32 bit, and in 64-bit Unix it is 64-bit, this might cause problems, since long in native libc is 64-bit and if we use a compiler that generates 32-bit longs, linking with native directly libc might be a problem, if we use 64-bit long we will get other problems, especially for WineLib. Out of the ashes into the fire. :-)

    Patrik proposed also a stepped approach:
    1. Making CRTDLL a 100% Windows compatible libc with every function implemented for use with the emulator. My preprocessor "magic" scheme might help doing that.
    2. Making WineLib applications use CRTDLL instead of libc. To do this we need some sort of elf dlopen mechanism and/or possible other "preprocessor" magic, because of namespace problems.
    3. Making Wine call CRTDLL for portabillity reasons.
    4. Making a CRTDLL that calls KERNEL32/NTDLL instead of libc.

    and asked:
    1. Where should we stop?
    2. Is my scheme robust/portable enough to do this?
    3. Have I missed something?
    4. What do you think?

    Patrik also took a look at integrating ReactOS CRTDLL into Wine. His first conclusions are that
    many of the commonly used functions seem to be implemented using KERNEL32 calls which is exactly what we (or at least I) want.
    , but licence is an issue:
    some files claim to be BSD, other files claim to be LGPL, and there are files indirectly claiming to be GPL by refering to a file COPYING in the topmost directory.

    Albert den Haan from Corel explained in details the strategies currently used to help porting their applications:

    Albert wishes to keep
    the configurability I want to tune source ported programs
    .

    Patrik and Albert also agreed that having all the Wine code relying on a single portable CRTDLL will help porting Wine from OS to OS.

    Nevertheless, Patrick points that Albert has different objectives:
    You need to keep in mind that I am looking for the best possible _technical_ solution in the long run, you (Corel) are looking for the best possible solution for your immediate problems.

    However there is a large overlap between us where compromises are possible. Fast and small footprint is _very_ important for me too, but I prefer good before soon.

    2. CRTDLL implementation

     Archive Link: "CRTDLL implementation"

    People: Albert den HaanPatrick StridvalPatrik StridvalOve KåvenAlexandre Julliard

    Albert den Haan also discussed about memory allocation under Wine:

    Patrik Stridval went a step further:
    do we need the special behavior of xmalloc and friends at all. Obviously, we need a way to choose between the heap malloc and the native libc malloc, but does are there any code _really_ needs the special behavior that xmalloc has? Any code that does should be fixed instead IMHO.

    He also added that
    The native malloc is probably faster on most platforms since it is one of the most common libc calls they are heavily optimized on most (all?) platforms. HeapAlloc is AFAIK not very optimized so I wouldn't be suprised if it is _much_ slower.

    Albert noted that changing xmalloc into *malloc could be not so easy because
    xmalloc does not return if the allocation fails, it exit() the thread immediately
    and
    will be a substantial effort in itself. Many of these allocations happen in situations (e.g. early in wine's initialization) when any failure is a good reason to take wine down so the user can provide more resources. Granted, there are cases where fancier error recovery would be a good idea but let the people who need that stuff code it when the need it.
    Ove Kåven noticed that
    "host OS heap" (malloc) is thread-unsafe under Wine.
    He also reminded his work in the debugger (changing malloc with HeapAlloc) which produced very slow results (at least while loading symbols).

    Albert detailed some of the allocation issues he run into:

    Alexandre gave his point of view on the CRTDLL issue:

    Patrik didn't think that tunneling through CRTDLL the existing libc functions is completly doable, especially when dealing with 64 bit Unix platforms providing a 32 bit CRTDLL.

    But, he reexplained how his patch could help (see also previous topic):

    No final decision has been made yet to integrate either Patrik's or Albert's patches.

    3. Binary resources and CVS

     Archive Link: "Binary resources and CVS"

    People: Francis BeaudetUlrich WeigandPatrik StridvalBertho StultiensPavel RoskinOve Kåven

    Francis Beaudet asks
    how I would go about submitting a BMP (or any binary file) to the CVS tree.

    Ulrich Weigand proposed to
    distribute bitmaps in XPM format (which is source code), and either use tools (like netpbm) to convert them to BMP by Makefile rules, or maybe even teach wrc to load bitmaps from XPM format directly (and convert them to BMP in the binary resource format)?
    Patrik Stridval disliked the use of netpbm for porting issues. He'd rather see
    wrc convert XPM to BMP.
    Bertho Stultiens agreed that
    wrc should handle xpm format automagically.
    , and asks for a small and easy library to do it (out of ImageMagic which is too large). Ulrich Weigand proposed libXpm, but noted that it would
    introduce dependency on libX11 itself as well.

    Ove Kåven rather proposed to embed bitmaps as hex codes as Borland's Resource Workshop does. Bertho Stultiens confirmed that wrc knows about this syntax.

    Pavel Roskin reminds some option from CVS:

    4. Wine clipboard enhancement plan

     Archive Link: "Wine clipboard enhancement plan"

    People: Noel BorthwickGérard PatelUlrich WeigandPavel RoskinAlbert den HaanAlexandre Julliard

    Noel Borthwick he's
    looking into making some changes to the windows clipboard code in order to allow the wine clipboard to integrate better with the X clipboard.
    .

    He proposed (for comments):

    Gérard Patel reminded Noel about a discussion he had with Ulrich Weigand regarding this issue:

    Pavel Roskin then provided some more information on Motif's behavior (quoted from Lesstif documentation):

    Noel proposed to stick to his first XA_CLIPBOARD implementation (to be compatible with X and KDE) and then to deal
    with different X flavors ... at the Clipboard driver level. Since in any case I was considering moving some of the clipboard code to the driver level, potentially Motif specific issues could be addressed in a different Motif clipboard driver.

    Ulrich Weigand proposed a quick fix:
    you might simply change the clipboard code to allocate data on the shared heap instead of using GlobalAlloc. Alternatively, you might always use the 16-bit heap, even for storing 32-bit data, as the 16-bit heap is always shared.
    Regarding separate address spaces, Ulrich added
    we'll need to think of a way to pass clipboard data to the other process. Actually, wasn't your suggestion to always pass data via the X clipboard, even between two Wine processes? This would nicely fix that problem ...

    Noel noted that this might not work because
    the X clipboard always works in "delayed render" mode. i.e. we need to return the actual WINE clipboard data in an X atom only in response to an XSelectionRequest event.
    . Ulrich confirmed that since Wine currently uses a single connection to X server in a service thread.

    Albert den Haan pointed out another issue:
    The solution described below looks to work for as long as the (single) application is up (and therefore its service thread) but will not work as Win#+ does when the app exits. It seems if we want to have this sort of persistence then the current wine set-up will not work. We have enough trouble with non functional zombies as it is!

    Albert proposed to use the wine server thread (and mentionned some other possible extension for serialization and/or persistence - like the registry).

    Noel agreed and proposed that
    the server could keep a copy of the data on (app) exit or alternatively the clipboard data could be rendered to X atoms on app exit. So the clipboard server would just need to access and return the appropriate atoms to the requesting client.

    Alexandre Julliard pointed out that
    It can't be part of the Wine server because the server doesn't have an X connection; but it should be easy to write a small Winelib app whose only purpose is to hang on the clipboard data.

    5. Address Space Separation / Stability vs Correctness

     Archive Link: "Address Space Separation / Stability vs Correctness"

    People: Gavriel StateUlrich WeigandGérard PatelIan Schmidt

    Gavriel State expressed some concerns regarding address spaces:

    Gavriel's main question is to make the right choice between stability (what Corel's looking for to release apps) and correctness (add as many features as possible), and main some parallel with Linux mode of operation between stable and experimental versions. He asked wether this was something to consider in the future, and since Corel's has been maintaining its own stable CVS tree,
    I could envisage our tree becoming a basis for that at some point.

    Ian Schmidt listed some programs needing this separate address space feature (mainly because one programs has its relocation info stripped, so it can only be mapped at a precise address, and it fails it this address is in use): Microsoft Visual Studio (VC++ 6.0), many games.

    Gérard Patel mentionned InstallShield based installation not working because of this.

    Ulrich Weigand explained in details of the separate address space stuff:

    6. DOS memory mapping (and some bits of paranoia)

     Archive Link: "DOS memory mapping (and some bits of paranoia)"

    People: Ian SchmidtUlrich WeigandNorman StevensAlexandre JulliardOve KåvenEric Pouech

    Ian Schmidt reported some crash while running Excel 2000: Ian also put out some paranoid questions whether this could be "anti-wine" code?

    Ulrich Weigand answered that
    In Win9[58], the first 4MB page is shared between all processes, and contains the memory image of the boot DOS session (1MB + 64K), followed by the low part of the 16-bit global heap. So it would indeed appear that at linear address 0xf0000 the BIOS ROM should be visible.
    He also turned down the anti-wine code, but something linked to some features of BIOSes (noting
    It is beyond me why a mere application should depend on BIOS implementation details; if the OS would do it, it still wouldn't be nice, but a little more understandable :-/)

    Norman Stevens also doubted it's anti wine code:
    More likely it is anti-piracy. The linear address 0xf0000 is the start of the bios rom, so if you write the checksum into the executable as it is installed you can prevent people running the program on a different computer.

    Ian Schmidt didn't agree because
    the app run fine with just zeros mapped there. So if it's anti-piracy it's broken (buggy? Microsoft? :). It does execute the check every 5 seconds or so after the initial test (on a WM_TIMER). Maybe it's really just an attempt to make people buy faster computers so they won't notice how large Win2k is :)

    Ulrich proposed a patch for it, and added
    If there are more direct accesses to the boot DOS image, we might even think about mapping our version of that image (DOSMEM_dosmem) actually at address 0 ...

    Eric Pouech didn't like mapping all the 1MB at 0 because it would remove NULL pointer deref faults (at least the first page should not be mapped, thus disabling access to BIOS data segment at 0x00000400).

    Alexandre Julliard proposed
    we could map it there, but protect the first page, and un-protect it on the first access >= 0x400. This would still allow catching most NULL pointers accesses, and it could make it possible to avoid forking a separate process for running DOS programs (at least with ELF binaries).

    Ian reported that patch from Ulrich didn't solve the issue, but with using an expanded version of another one provided by Ove Kåven, he got Excel 2000 to run
    minus some graphical glitches (loading / saving files worked, typing stuff into cells worked, graphs/charts seem to work, etc).

     

     

     

     

     

     

    Sharon And Joy
     

    Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.