Wine Traffic #219 For 16 Apr 2004

This is the 219th issue of the Wine Weekly News publication. Its main goal is to get lots of nasty vaccinations. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org

There were 58 different contributors. 30 posted more than once. 25 posted last week too.

WineX 3.3.1 is primarily a bug fix release. However, it also includes a number of DirectX 9 related changes to make the present patch level of EverQuest work, and a major reduction in sound latency.

Looks like we got a little PR last week from internetnews.com, "Vintage Year For New WINE?". For anyone not familiar with the project it provides a nice introduction. Despite the title, the author wisely avoids speculating on when a release might be ready.

Tom's Hardware began a series of Linux tutorials. The second one came out this week and briefly mentions Wine, " If you simply must run your Windows programs, chances are you can make them work with WINE. WINE is a free Linux program that will trick Windows programs into thinking they are running inside of Windows and not Linux. "

Last week (issue #218) we mentioned Mike McCormack had begun working on problems caused by execshield enabled kernels. Basically the problem Wine runs into is it requires certain memory locations to be available and with execshield there's no way to guarantee that. A few people have taken stabs at finding a solution, the latest was announced by Mike on Saturday:

This patch works around the exec-shield problems with Fedora Core. I'd be greatful if any people using Fedora core could test it out and see how it works.

It should be enough to apply it to the latest CVS tip and recompile.

All comments and flames accepted... /me wears his asbestos suit.

Digging into Mike's patch you'll find a "mini loader" at the heart of it. Before any application can begin execution it's necessary for the environment to get set up properly and that process is typically handled by what's known as a loader. Normally what happens is you clear out all the memory you want to use, get a dynamic linker mapped into memory and start it's bootstrap procedure, and then initialize the stack. From there the linker finds the entry point into the executable and transfers execution over to it. Mike explains how his approach fits into this whole scheme:

Design notes

The goal of this program is to be a workaround for exec-shield, as used by the Linux kernel distributed with Fedora Core and other distros.

To do this, we implement our own shared object loader that reserves memory that is important to Wine, and then loads the the standard shared object loader (ELF interpreter), usually ld-linux.so.2

As Wine is i386 specific, we are not too worried about portability issues, but will do our best to keep the system dependencies (syscalls) seperate to the rest (ELF loader and printf) part of the program.

This program will be invoked as follows:


 wld <full path to ELF executable to be loaded> <args...>

We will try to set up the stack an memory area so that the program that loads after us (eg. the wine binary) never knows we were here, except that areas of memory it needs are already magically reserved.

The following memory areas are important to Wine:

0x00000000 - the DOS area
0x00400000 - the standard Win32 load address
0x65430000 - the shared heap at

There are a few things that will be different after this program passes control of execution to the loaded binary:

The _ variable in the environment (will be the name of this exe)
The brk area returned by brk(2)
This program will still be mapped into memory.
The command line kept by the kernel in kernel memory (/proc/self/cmdline)

References (things I consulted to understand how ELF loading works):

glibc 2.3.2 elf/dl-load.c: http://www.gnu.org/directory/glibc.html
Linux 2.6.4 fs/binfmt_elf.c: ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.4.tar.bz2
Userland exec, by <grugq_at_hcunix.net>: http://cert.uni-stuttgart.de/archive/bugtraq/2004/01/msg00002.html
The ELF specification:: http://www.linuxbase.org/spec/booksets/LSB-Embedded/LSB-Embedded/book387.html

Mike Hearn commented on the approach, " It seems it's turned from being an normal (albiet static) app which reserves the areas needed then boots wine, into a reimplementation of ld-linux.so? I'm not sure it's a good plan to alter the .interp field - that has to be absolute and this technique would break binary relocatability."

The .interp section Mike refers to is an ASCII string in an ELF binary that explicitly says what dynamic loader to invoke. Mike M. didn't think he was doing anything bad:

At least in the Linux kernel, the ELF interpreter is loaded from the current directory if it is not an absolute path. In this implementation, I have forced the current directory to be the same as the binary's directory, and saved the working directory in an environment variable WINEWORKDIR.

The reason for doing this was to save the hastle of passing yet another round of extra argv[0]'s to the execv. If it's determined that this method is not portable, it's easy enough to go back to the other method of using an executable instead of replacing the ELF interpreter.

In any case, this patch seems to work, and is not limited by binary relocatability.

Mike M. then followed up with more revisions later in the week that added some more features:

Changes in version 5:

reserve the shared heap area as well as the PE load area
allow multiple reserve areas in libwine

Changes in version 4:

set the WINEPEEXEPATH environment variable correctly

Changes in version 3:

reserve only the area required by the PE EXE if launched from CreateProcess (else just a default slab)
pass the reserved area to wine, so wine knows about it
unmap the preloader after it is no longer used

Last week Dimi Paun ported AbiWord to Linux using Winelib. Of course, your first reaction is probably right - AbiWord already runs on Linux so there's no real need to port the application back to Linux. However, as a way to test Winelib it's extremely helpful. Dimi took the source and MinGW build system and basically used it to generate a Winelib app:

In the past couple days I've entertained myself with porting AbiWord to Winelib. This was possible because now AbiWord has a nice MinGW build system. A total of 5 trivial patches were indeed needed for Wine:

Notice how little we had to fix! To encourage others to port stuff to Winelib, I'll inline the hacks I did to port AbiWord. These we'll need to be cleaned up for integration into the AbiWord tree, but again, please note how little we had to change. (I've also attached a screenshot of the resulting Abi Word in action).

This is a fairly large application and the changes required to port it were minimal. If you put all those patches together you'd end up with roughly 24 changed lines, almost all of which were in headers. Ge van Geldorp of ReactOS mentioned there were some user interface issues uncovered by the port and he planned on tackling them, namely, " the font family combo box is slightly higher than the other combo boxes, the two rightmost images on the toolbar are not shown"

Dimi said he had no further plans to work on AbiWord other than getting some patches integrated into AbiWord that allowed it to build with Winelib.

Last month there was quite a bit of discussion about using Wine as a regular shared library. We covered that topic pretty well in issues #213 and #214. The problem is, Windows libraries and executables require a a different environment to run in than Linux ones. Things like threading are done differently enough that the recommended solution has always been to just create a Winelib app if you need to access a Windows library. This week Paul Davis came up with Winelib code that will convert a Linux thread to a Win32 one on the fly. Potentially a lot of other projects could benefit from this. Paul had a short announcement:

I offer this for Alexandre and other's consideration and comments. This is already in use in libfst, which is in turn used by Ardour, a native linux digital audio workstation, and gAlan, a native linux modular synthesis system.

There is still no proper cleanup on failure, and nothing is done if the adopted thread calls pthread_exit(). Both are simple to fix.

Han Leidekker posted a patch to make wininet build on MinGW and in the process announced, "Move to Windows sockets."

Steven Edwards had been down that road before and questioned whether that was the right solution, " I asked Alexandre about a patch to move Wininet to Winsock and he said it took to much of a performance hit to merge. Is there a way we can bench mark it to show how much of a hit Winsock vs Unix sockets is going to give us? He suggested that we find a way to make Wininet use Unix sockets on Unix and Winsock on Windows. "

Jonathon Wilson thought a few #ifdefs to differentiate between platforms would make that work. Dimi cautioned against sprinkling #ifdefs around and wondered how much of a performance hit was being incurred. Hans did some benchmarking and reported:

Here are some timings for a loop of 100 synchronous HTTP GETs from localhost on a 777 bytes file:

Unix sockets

real	0m4.099s
user	0m0.480s
sys	0m0.312s

real	0m4.137s
user	0m0.503s
sys	0m0.283s

real	0m4.092s
user	0m0.514s
sys	0m0.280s

real	0m4.104s
user	0m0.533s
sys	0m0.303s

Windows sockets

real	0m4.172s
user	0m0.888s
sys	0m0.910s

real	0m4.255s
user	0m0.858s
sys	0m0.820s

real	0m4.219s
user	0m0.807s
sys	0m0.839s

real	0m4.867s
user	0m0.936s
sys	0m0.988s

So yes, there is a performance hit. Especially the 'user' and 'sys' measurements are respectively nearly 2 and 3 times higher than with Unix sockets.

So, is this a problem? Depends on what's important to you, but I'd argue that it's more important for Wine to open up wininet (and consequently winsock) to more users and developers. That may eventually attract more developers to fix bugs or even the performance issues with our implementation.

I would also argue that performance in a typical scenario is probably not bounded by wininet's implementation but by the user's bandwidth or, for example, by his browser's rendering speed.

By the way, 100 *asynchronous* HTTP GETs in a tight loop will reliably crash Wine, both with Unix sockets and Windows sockets.

I hope that we use Han's patch. It's going to make it easier to implement thread safety (as we can use WaitForMultipleObjects on the handles) and will make porting easier for Reactos.

IMO, wininet is not really performance critical, and we can probably improve things in the winsock layer a bit too.

Alexandre went back to Steven's original comment and outlined his preference, " There's no real reason that we cannot use Unix sockets on Unix and Windows socket on Reactos with the same code, it just needs a bit of header magic; and this will be useful for Winelib apps too. "

He also noted the problem with Wine crashing was a direct result of wininet not being thread-safe. Hans' original patch remains uncommitted and no patch has appeared implementing the aforementioned magic.

Anybody using autofs and wine? Does it work? I don't follow wine development lists, so maybe autofs is no longer a problem. Once in a while I build a CVS snapshot and test whether the odd app I still have works. And wine on autofs doesn't work here. I think Guido already sent a mail about this last month.

Guido patched wine way back in Feb 2003, submitted a bug

http://bugs.winehq.com/show_bug.cgi?id=1283

and even set up a page to describe the problem

http://freespace.sourceforge.net/guidod/wine-vol-a/

Since then he has completely rewritten the patch in order to search the /etc/auto.* files for drives. This obsoleted the additional options in wine's config with the old patch. This last patch worked perfectly until recently.

However, Guido is busy right now and cannot work on the patch. And my C is pretty bad. If anybody is interested I can provide the last patch known to work. If for some reason the patch does not meet wine's standards, would it be possible that the autofs problem is recognized and put on a TODO list for later? Or even close the bug as WONTFIX?

Alexandre mentioned that as part of the filesystem rewrite this should start working.


Kernel Traffic Latest \| Archives \| People \| Topics	Wine Latest \| Archives \| People \| Topics	GNUe Latest \| Archives \| People \| Topics
Czech

1.	10 Apr 2004 - 16 Apr 2004	(3 posts)	News: WineX 3.3.1, Press Coverage
2.	10 Apr 2004 - 15 Apr 2004	(18 posts)	Execshield (con't)
3.	9 Apr 2004 - 10 Apr 2004	(3 posts)	AbiWord Ported to Winelib
4.	12 Apr 2004	(1 post)	Wine As a Shared Library
5.	4 Apr 2004 - 6 Apr 2004	(13 posts)	Converting Wininet to Windows Sockets
6.	14 Apr 2004	(3 posts)	Problems With Automounted Filesystems

Wine Traffic #219 For 16 Apr 2004

By Brian Vincent