Wine Traffic #108 For 10�Nov�2001

By Brian Vincent

Table Of Contents

Introduction

This is the 108th release of the Wine's kernel cousin publication. It's main goal is to distribute widely what's going on around Wine (the Un*x windows emulator).

So last week Transgaming (http://www.transgaming.com) had the news. This week it's CodeWeavers (http://www.codeweavers.com) . You can now download a demo of their CrossOver Plug-in (v.1.0.1). As they describe on their website:

CrossOver Plugin lets you use many Windows plugins directly from your Linux browser. In particular CrossOver fully supports:

CrossOver works on any Linux distribution and will integrate with most browsers including Netscape 4.x, Netscape 6.x, Konqueror, Mozilla, Galeon, Skipstone and Opera. CrossOver also integrates with Gnome and KDE to let you transparently open any Word, Excel or PowerPoint file. But even better, you can open this type of attachments directly from any mail client.

Of course, you can also download (http://store.codeweavers.com) the product for $19.95 or $29.95 to get it on CD. And what kind of description would be complete without a screenshot (http://www.codeweavers.com/products/crossover/images/quicktime.jpg) or two (http://www.codeweavers.com/products/crossover/images/2001_08_27_154448_shot.jpg) (or three (http://www.codeweavers.com/products/crossover/images/2001_08_27_154145_shot.jpg) ..)? Oh, and did I mention there's a patch (http://www.codeweavers.com/support/crossover/downloads.php) to upgrade a 1.0 installation to 1.0.1? Among the new features are bug fixes, support for PowerPoint Viewer 97/2000, and the ability to start the QuickTimePlayer from a commandline. Of course one of the things that $19.95 will get you is support - and judging from the support mailing lists, Francois Gouget and Jeremy White are doing a fantastic job resolving a lot of questions.

Oh yeah, an update on the Mandrake Gaming Edition that includes The Sims - you can order it (http://www.mandrakestore.com/en/storemdkinc-81gaming.php?LANG=en) now.

We also saw yet another escapee from the CVS tree - Wine-20011108. >From the announcement we have these changes:

All the Quartz stuff is quietly coming from Hidenori Takeshima. He has contributed a ton of code and is busy fleshing out a lot of stubbed functions.

Mailing List Stats For This Week

We looked at 92 posts in 291K.

There were 33 different contributors. 17 posted more than once. 16 posted last week too.

The top posters of the week were:

1. Winsock2

6�Nov�2001 (4 posts) Archive Link: "[EXPERIMENTAL PATCH]: Winsock2"

People: Martin Wilck,�Francois Gouget,�Robert Lunnon,�Andreas Mohr,�

Once in a while people ask if Winsock2 has been implemented in Wine and the answer has been no. Winsock2 is the set of network API's that shipped with Windows 98 (and available as a patch for Windows 95). This API is different than the one originally that originally shipped with Windows 95. Martin Wilck wrote in and announced some new code that has the beginnings of Winsock2 support:

this patch tries to implement a small part of the extended Winsock 2 functionality in Wine. It is very experimental, I have only made sure everything compiles. It is _NOT_ intended for people to try and run.

I am submitting this patch because I am pretty new to Wine hacking and I'd appreciate some Wine gurus to audit this for obvious stupidity, style incompatiblities, etc.

Features implemented:

The real difficult stuff will follow when I start looking into WSARecv(), WSASend(), and WSAGetOverlappedResults().

Thus it'd be nice if someone pointed out my misconceptions before I put real work into the latter.

Francois Gouget replied with the following:

I will leave it to someone else to comment on the meat of the matter but still, here are a couple of comments:

* To answer your question:
/* Code copied largely from WSOCK32_accept() */
/* FIXME: Can we simply call WSOCK32_accept() instead ? */
/* I wasn't sure because of the AsyncSelect stuff at the end,
which presumably should only be called if the connection is accepted */

Winsock2 should not call the wsock32 library. It should be the opposite. One of the things on my todo list (but whether I will have time to actually do it is another matter) is to reorganize the winsock libraries so that wsock32 and winsock are together in dlls/wsock32. The reason is that they both implement the winsock1 API except one is 16bit and the other 32bits. Then in dlls/winsock we would be left with just ws2_32 which implements the winsock2 API. So having ws2_32 call to wsock32 is not going in the right direction :-)

Looks good to me. Welcome to the Wine project. It's always good to have some new blood.

Andreas Mohr didn't quite understand why Francois would want to reorganize the code in such a fashion. Francois explained there were two main reasons. First, all of the 16-bit aspects in the Winsock2 code need to be removed and then have the two libraries dealing with the first version of the API be implemented as a DLL pair. The next step is to have version 1 use the code in version 2 whenever possible to avoid code duplication.

Robert Lunnon also had some suggestions, " Whoever is playing with Winsock can you please ensure that the solutions you come up with are not linux centric. There is a lot of code in there that does not run on solaris because it grubs around in PROC getting TCP structures. I have made a bit of a start in cleaning this up, Here it is, absolutely untested non working code, but a start nevertheless, I put this into a separate file and link it with winsock and wsock32 from a common source, these functions replace the helper functions of similar name in the two dll subdirectories."

2. Overlapped I/O

4�Nov�2001�-�8�Nov�2001 (10 posts) Archive Link: "Winsock2 & Overlapped IO"

People: Martin Wilck,�Mike McCormack,�Ove Kaaven,�,�Ove K�ven

Along those same lines, Martin Wilck needs to have some changes made to facilitate Winsock2 support. As Martin first described it, " I have started porting a software product of our company from NT to Linux using Wine. Part of this software is an agent which implements a proprietary protocol implemented with Winsock 2. In particular, it uses the Winsock 2 feature of "Overlapped IO" on sockets, with user-supplied callbacks that are called when data is received on a socket etc."

Mike McCormack wrote back and gave a description of how overlapped I/O works in Wine:

i don't know much about Winsock, but i can tell you how overlapped I/O is supposed to work in Wine.

The bulk of the implementation is in files/file.c and scheduler/synchro.c. When ReadFile is called on a handle, the following happens:

  1. check that the handle is valid and was opened with FILE_FLAG_OVERLAPPED.
  2. attempt to read any data that is available immediately, and perhaps return if we read enough already.
  3. queue the operation onto CurrentTeb()->pending_list

When the thread waits upon any server object, all (unix) fds with pending overlapped operations are polled on. This is a minor problem in the design; if the thread never waits upon an object the overlapped can never complete, however the thread must wait upon the overlapped's hEvent to check whether it has completed...

The only part of the serial comms overlapped implementation the lives in the wineserver is the calculation of overlapped timeouts.

So to get winsock to do overlapped, you may have to modify the winsock code to work like that... but the winsock code uses the service thread to wait upon sockets. The task is definitely not impossible, but it might take a bit of work :-)

Maybe Ove K. can give you some advice?

In response to the last part, Ove K�ven felt the service thread wouldn't really come into play:

Not really, the wineserver does the grunt work. The service thread is only used to implement WSAAsyncSelect, to post messages when the wineserver says that it should. Winsock2 code is unlikely to use WSAAsyncSelect, they'd rather do either WSAEventSelect (which doesn't use the service thread, but is 100% wineserver-based), *or* overlapped I/O.

The challenges for winsock2 should be pretty much the same as those for implementing something like ReadFileEx (which you've probably already done) and the above-mentioned overlapped ConnectNamedPipe.


Martin started working on bits and pieces here and there and posted some patches. Then he ran into a stumbling block because buffers are read and wrote in the opposite order in which they were submitted. He posted a lengthy email outlining a possible solution:

I see a severe problem with the current overlapped I/O implementation, especially wrt extending it to Winsock2 overlapped I/O.

The problem I see is with buffer ordering.

For normal files, the file position from which to read is on the OVERLAPPED struct. This was not in the Wine implementation, the patch in my last email fixes it.

For sockets and other FIFOs, this makes no sense. In this case Windows specifies that buffers are read / written **in the same order they were submitted**. This matters only if users spawn several asynchronous I/O requests at the same time, but this is allowed and actually beneficial for application performance (at least on Windows).

Unless I oversee something essential, the current wine implementation does not guarantee such behaviour. Actually, since new requests are inserted at the head of the NtCurrentTeb() list, they will be considered first in subsequent asynchronous reads - i.e. the order will be reversed. If I/O requests don't complete in a single read() or write() call, the ordering will be basically random.

IMO this implies that for sockets, asynchronous reads and writes must be scheduled such that only one request for a given socket is actually reading at a time, and all others are blocked until this request is finished. Again, this holds only for FIFO-type files.

With the current structure, it seems to be very complex to sort this out. One could recheck the fd list in check_async and remove duplicate fds. One could also try to "lock" the fd somehow. Still it would be necessary to ascertain that the correct request gets the lock first.

With this min mind, it seems more natural for me to organize the asynchronous requests in the "file" data structure itself. One would have a reader's and a writers's list there, and make sure that for FIFOS only the first list element can actually issue a read(), write(), send(), or recv() system call. For normal files all elements in the lists could be doing IO at the same time, question is if that makes sense or not.

As I see it, this would mean both the readers and writers lists must be stored in the file data structure in the wine server. This would of course strongly affect the current implementation.

Note that with Winsock2 overlapped I/O in place, it must be possible to use also ReadFile(), ReadFileEx() etc. on overlapped sockets, and get correct behaviour. Thus the ReadFile() etc. code must be able to deal with both sockets and ordinary files, and it makes no sense to intriduce a different approach for sockets only.


Mike McCormack wrote back:

You're right, i never considered multiple overlapped requests at the same time...

To fix the problem correctly, you will need to move (some of) the overlapped code back to the wineserver, so that requests from different processes can be handled correctly. (i moved it out of the wineserver about a year ago for efficiency reasons.)

Have a look at rev 1.7 of server/async.c to see how this worked.

http://cvs.winehq.com/cvsweb/wine/server/async.c?rev=1.7 (http://cvs.winehq.com/cvsweb/wine/server/async.c?rev=1.7)

The current overlapped code (with the above exception) is similar to what win 9x supports, so it does not support overlapped on normal files, and file offsets in OVERLAPPED structures. If that could be implemented too, it would be good.

i guess there is choice now... implement overlapped sockets with the existing structure, or make a rather large change to the wine architecture, which might take a while.

Maybe a good first step is to back out my change which put overlapped operations into the client side code... Alexandre thought this could be done using POLLIO instead...

We'll see how this shakes out, there may be some architectural changes happening.

3. Linking in .lib Files

6�Nov�2001 (5 posts) Archive Link: "linking in .lib files"

People: Chris Dawson,�Francois Gouget,�,�codeweavers

Chris Dawson wanted to know how to go about using a .lib file from Windows to compile his application, " Is it possible to link in .lib files using winelib and winemaker? If possible, what switch do I give winemaker? My application when compiled under Windows linked in a .lib file and I would rather not modify the source code. I've tried using the -l switch, but this is obviously only for *nix libraries even when compiling with winelib. Or, other suggestions to resolve all the undefined reference errors?"

Francois Gouget provided a detailed explanation of what was going on:

You cannot use .lib files directly with Winelib. The reason is that winebuild does not support reading .lib files. Support for '.lib' files (or possibly even '.dll' files) could be added in but it is not there currently.

Exactly how you deal with that depends on what your situation:

  1. if you have the sources of the library you want to link with, then the best solution is to compile it with Winelib too.
  2. if you don't have the sources for this library (and it's not one of the standard Wine libraries), then you will have to write a stub library that you will use for just the purpose of linking your application.

In both cases you will need to create a '.spec' file containing a list of the functions (with parameters) exported by that library. See the following URL for a documentation of the exact format of .spec files:

http://wine.codeweavers.com/docs/winelib-user/spec-file.shtml (http://wine.codeweavers.com/docs/winelib-user/spec-file.shtml)

Case 1:

Case 2:

There is another roadblock that you may hit in both cases: winebuild does not support importing a variable from another library (because of ELF limitations). Importing functions is ok, but not variables. This means that if you are dealing with a C++ library you will need to link with it in the Unix sense. If we are talking about C code, then there are ways around it that require only relatively simple code changes. It is also another reason why you will never be able to use a native C++ library directly (because the virtual table pointers are handled as variables).

One last note: specmaker has been folded into winedump recently. So if you hear/read things about specmaker you know that you have to look for winedump.


4. Problem With VirtualDub (con't)

5�Nov�2001�-�8�Nov�2001 (10 posts) Archive Link: "queries on internals..."

People: Roger Fujii,�Travis Michielsen,�Avery Lee,�,�Eric Pouech

Roger Fujii continued to work on getting VirtualDub running. He posted a list of questions about the problems he was having:

ok... getting reasonable output, but now I have some questions.

First, it seems like wine can't find the audio codecs, though it seems to find the video ones. Is this a to-be-implemented or a bug and any clues on where to look?

Roger went on to ask several questions related to portability and Solaris. Most of the thread however was centered around getting VirtualDub to recognize audio codecs. Roger and Eric Pouech went back and forth quite a bit. Eric suggested making sure they were listed in the [drivers32] of the system.ini file. Roger assured him they were. Travis Michielsen replied, " Hmm, I noticed that when using VirtualDub, I had the same problem with ACM codecs. I did some detective work and found the problem to be a mistake in dlls/msacm/driver.c. Included is a patch to fix this error. However, I noticed that with this patch, the codecs are found, but for some reason are listed many times for each codec!?" .

Roger posted a note that Avery Lee, the author of VirtualDub, had sent him:

Please feel free to forward this on to other developers or onto the Wine list.

I read the discussion about VirtualDub crashing the Wine loader with great interest today. (It's a bit weird that the only way I heard about a problem with my Win32 program was via a Unix-oriented mailing list.) Yes, the exports header points to garbage, and yes, it was caused by UPX -- UPX left the export header RVA and size in the PE32 optional header exactly was it was in the unpacked image. You can safely hex edit the bytes at 180-187 to null to kill the bogus export directory, since it turns out I exported those functions by mistake in some hasty cut-and-paste and they aren't used.

As for other compatibility problems with Wine, I have gotten VirtualDub to run in the past under Linux, although I ran into severe problems with the handling of custom templates in GetOpenFileName() and slow updates in the trackbar common control. The last time I tried I had to remove two calls to SetUnhandledExceptionFilter() to get it to run.

One more thing: There is an undocumented Wine compatibility switch /w that you can specify on the command line to force VirtualDub to use StretchDIBits() to blit to the screen instead of a DIB section. It was a workaround for an old problem where Wine wouldn't mark a GDI DIB section dirty if it was written to through an aliased, memory-mapped section.

Sharon And Joy

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.