Wine Traffic #155 For 31�Jan�2003

By Brian Vincent

Table Of Contents

Introduction

This is the 155th release of the Wine's kernel cousin publication. It's main goal is to perpetuate the belief that this incredibly useful piece of software is some how nearing beta quality. It also serves to inform you of what's going on around Wine (the Un*x windows emulator).

Mailing List Stats For This Week

We looked at 200 posts in 670K.

There were 62 different contributors. 35 posted more than once. 34 posted last week too.

The top posters of the week were:

1. News: Install IE 6

25�Jan�2003�-�31�Jan�2003 (1 post) Archive Link: "News"

Topics: News

People: ,�News

Seems like people have been asking how to install Internet Explorer 6 for a few weeks. I haven't tried this yet, but Frank's Corner has some short instructions (http://frankscorner.org/wine/modules.php?op=modload&name=Sections&file=index&req=viewarticle&artid=130&page=1) how to get it to work with CrossOver Office.

2. Threading Problems with glibc 2.3

24�Jan�2003�-�25�Jan�2003 (9 posts) Archive Link: "Re: PATCH: glibc 2.3.x and errno"

Topics: Integration

People: Marcus Meissner,�Andreas Mohr,�Alexandre Julliard,�Ulrich Weigand,�Gavriel State,�,�Francois Gouget,�Andrew Lewycky,�J�rg Mayer,�Gav State

If you've been following Linux kernel development lately you probably know about all the new thread implementations floating around. Basically they offer significant threading improvements in order to solve the " C10K (http://www.kegel.com/c10k.html) " problem. These changes have gone into the core of the new Linux kernel and glibc. One of those changes has caused a problem with Wine described by Marcus Meissner:

Beginning of this week, a colleague of mine walked up to me and said "Wine does not work.". Since this happens sometimes, I just walked to his office and checked it for myself. And yes, it did not work. It reported:

Some straces later the problem became apparent:

He is using a glibc 2.3 headbranch snapshot (the one we will ship in SuSE 8.2). Looking at the output of "nm", then looking at the sourcecode confirmed first suspicions.

__errno_location and __h_errno_location are no longer weak symbols and so can not be overwritten any longer. The internal glibc systemcall wrappers no longer call the functions by reference, but directly.

Investigations and several talks to one of our glibc gurus (Andreas Schwab) gave following results:

I spend 2 days studying glibc and thinking on how to best solve this problem, to no avail:

The afore mentioned glibc guru did not have any ideas either.

There however is a happy but messy end, which appeared to me yesterday: Overwrite those 2 functions with a jump to our implementations.

So I did. You can now mark me up for the dark side.

Whoa.. let's slow down. Let me see if I can make a little more sense of this. See, __errno_location simply returns the address of the errno variable. That begs the question, what is the errno variable? Simply, it's a value that gets set when a system-level call fails. For example, there's values defined for things such as not enough memory, no space left on a device, or permission denied. And by it's nature, glibc is the place where system calls live. So the problem is with what Marcus said, __errno_location and __h_errno_location are no longer weak symbols and so can not be overwritten any longer

But wait, why are we even in this mess? Why doesn't Wine just use the standard Unix pthread implementation? The last time the WINE developers checked, pthreads and Win32 threads were not compatible enough to have Win32 threads implemented on top of them. As of the new glibc 2.3 threading features the developers do not know yet. In Wine, Win32 threads are mapped 1:1 to kernel threads, which under Linux is clone(2). The "wineserver" process is what's responsible for waiting on those threads and it operates as a single process with a giant poll() loop. So it seems we're back to the age-old question of whether Wine can some how graft Windows' threading into the pthreads model or whether it needs to continue it's own.

This problem is going to be even more apparent in a few months when RedHat ships a version of glibc that won't work with Wine.

Andi Mohr didn't like the patch and felt the problem needed to be fixed in glibc, not Wine:

Excuse me, but somehow I think this is p*ss poor. (and yes, I'm now marking you up for the dark side ;-)

I mean, both Wine and glibc are successful (?) OSS projects, so they should be able to come up with something much better than this terribly embarassing solution (after all everybody knew that OSS development was a "superiour" approach, didn't they ? ;-).

I for one would feel much better if we simply rejected that particular "broken" glibc version and supported a *new* glibc method of properly interfacing errno things in a newer glibc version... (maybe have some "advanced" setting in glibc that enables all sorts of funky interfacing capabilities in case a program needs it)

After all if Wine needs this errno support, then there's probably a need for it, so it's glibc's bloody obligation to make sure there's proper support IMHO.

BTW: why did they even choose to abandon a public errno_location ?

Anyway, thanks for tackling this severe issue !

P.S.: Do you really think that Alexandre would commit it like that ? ;-)

Alexandre did reject it and pointed out a flaw, " I'm afraid that won't be enough. When using thread-local storage, glibc doesn't even call __errno_location any more, it directly stores errno into the thread storage using %gs. It seems the only solution is to make Wine threads work on top of libc threads, but that will be messy."

J�rg Mayer thought that the Mono folks had requested that before in order for their System.Windows.Forms piece to work. (He's right - we covered that thread back in December in issue #148 (http://www.winehq.com/news/?view=148#Garbage%20Collection%20With%20Wine) .)

Francois Gouget wanted a better explanation of the problem that Wine faced. Ulrich Weigand gave a short overview:

glibc has switched to using thread-local storage for errno (i.e. it is declared as 'extern __thread int errno') when the tool chain supports the __thread keyword.

This means that C source code compiled against the new headers will result in assembler code that *directly* accesses a thread-local variable as defined by the TLS ABI. In the case of errno, this will involve accessing the %gs segment using an offset from the GOT, without any function call being performed. (errno is defined to use the initial-exec TLS storage model.)

The __errno_location routine is provided only for backwards compatibility reasons, it is no longer guaranteed that every access to errno calls it. Thus, if you overwrite the implementation of __errno_location, you will only catch *some* errno accesses, not all of them ...

Marcus replied in more detail:

Lets have a small example:

In 1980 this was rather nice and worked all the time with the nice global 'errno' integer variable. However, someone had to invent multi threading.

After this, errno could not stay a global integer variable, since you could get into race conditions writing/accessing it. Sinc millions of lines of code could not be changed, the <errno.h> header was changed to defined it as:

With __errno_location (or similar) a function that returns a pointer to an integer variable within the thread local storage area.

(The same goes for __h_errno_location and other internal functions.)

The glibc implementation basically does:

error:

glibc follows the pthread style of threading, at the time WINE needed threading implemented by SIGALRM timers within a single process (clone(2) was not invented yet).

As WINE started Win32 threading the clone(2) handler was available for us and we implemented our own kind of threading, Windows style. glibc however does not know a single thing about that and still assumes there is no threading and had __errno_location return a single pointer.

So we had to overwrite __errno_location(), which was easy possible, since libpthread also did so and it was exported as weak symbol meant to be overwritten.

However with glibc 2.3 the internal thread representation changed, most pthread libraries now use clone(2) too, and use a new way of Thread Local Storage, using a segment register. Since WINE was using %fs, they chose to use %gs.

Now a system call looks like:

error:

So we no longer can overwrite __errno_location to have our own errno storage, so we need to cooperate with the libc threading.

Gav State mentioned some uncompleted work that was started that might prove useful:

We have a winethreads-on-pthread implementation that we did for some of our non-x86 work a while back. It's pretty simple, and we haven't really tested it on an x86 machine recently - when we did so last, it definitely had some problems. It is pretty similar to work that Andrew Lewycky did at Corel to support cprof and multithreaded profiling, and probably has some of the same caveats that were discussed on wine-devel in March of 2000.

The code (excluding some configure checks) is here - some updates will probably be required to get it working with a more recent tree. WineHQ is welcome to use it. It would be nice if any improvements made to it were donated to ReWind as well:

3. User Interface Status

30�Jan�2003�-�31�Jan�2003 (11 posts) Archive Link: "[RFC] The Wine UI page"

Topics: Project Management, Status Updates

People: Dimitrie Paun,�

Dimi put together another page on his web site to showcase the status of various user interface elements. He received a bunch of feedback and revised it. He announced:

There is a new page in my little Wine WWW -- the UI status page:

This is a page that I've been meaning to create for many months now. Please check it out, and let me know if you notice any inaccuracies.

4. RPC Data Marshalling

30�Jan�2003�-�31�Jan�2003 (8 posts) Archive Link: "RPC marshalling patch"

Topics: RPC / COM / OLE

People: Ove Kaaven,�Greg Turner,�Joerg Mayer,�,�J�rg Mayer,�Ove K�ven

Ove K�ven dropped a large patch with the following notes:

This one is sure to give Greg something to work with... all of this was implemented in a bit of a hurry, but since it's based on my research, it should be a good starting point in understanding how Microsoft's NDR engine works. It doesn't properly implement marshalling alignment or memory sizing, may not handle a number of fringe cases, does not conform to the DCE RPC wire protocol (mostly because I don't have a description of it... where did you find it, Greg?), and probably needs some cleanup, but it seems to do the job for me.

Log:

By way of explanation, Greg Turner once wrote, "Data marshalling is the means by which RPC represents data across process and machine boundaries. MSRPC extends NDR format for this. The wire protocol is mostly documented, but the MS API's to convert data-types in memory into NDR are not. "

In reference to Ove's comment about Microsoft's RPC wire protocol, Greg provided some links:

I think, I had to sign away my firstborn to OpenGroup for it. If you feel like spending money /and/ signing away your firstborn, this looks like the definitive OpenGroup DCE package:

and here is some free-as-in-beer RPC 1.1 stuff:

And later,

Of course, I was kidding about signing away your firstborn, although you do at least sign away any redistribution rights by downloading the free stuff. AFAIK OpenGroup/DCE controls the RPC spec -- expecting consumers to sign an NDA or something would kind of defeat their stated purpose of enhancing interoperability.

There is one really significant open question left by their documentation: what is the gap between "MSRPC" and DCE RPC? I still can't decide whether this difference is just an API-level difference, or a full-blown wire-protocol incompatibility.... Luckily, we probably don't care too much -- ultimately, if we can make something that interoperates with Windows, we win, regardless of how well MS complied with the spec.

J�rg Mayer gave another tip for sorting it out, " You could take a look at the Ethereal sniffer software ( www.ethereal.com (http://www.ethereal.com) ). It has dissectors for some of Microsofts rpc stuff."

5. File Dialog Options

31�Jan�2003 (6 posts) Archive Link: "common file dialogs"

Topics: Configuration

People: Mike Hearn,�Marcus Meissner,�

Mike Hearn wanted to know how to clean up his file dialogs:

Is there a reason that:

  1. The Wine open/save dialog boxes don't show or follow symlinks and
  2. They show dotfiles?

I seem to recall a discussion on wine-devel way back on the topic of dotfiles, but a quick archive search didn't turn up much of use. At the very least I think it should be a pref, wading through lots of dotfiles in your home dir makes it much harder to open files with wine, and obviously non-technical users will wonder what is going on with all these stranges folders that I never made......

Marcus Meissner pointed out:

Check out the config file:

6. Windows API Database

23�Jan�2003�-�27�Jan�2003 (6 posts) Archive Link: "building a windows API database"

Topics: Documentation

People: Dave Miller,�

Dave Miller wrote in with some questions about a project he wants to start:

I am interested in taking on the "build a database of windows APIs and dependencies" task. I have started work on scripts to scan a windows system and list import/export information. So far I can gather a list of dlls and run dumpbin on them, then parse imports (mostly, I think). I've started with simply printing the parsed output in three columns. The first is the DLL being scanned, the second is the DLL being imported, and the third is API imported. I am unsure of one thing. There is a 1 - 3 character hexadecimal number preceding almost every API. What is it, and should it be included in the output? Here is an example of what I am referring to: The hexadecimal number, in the third column.

c:\windows\system32/atiiiexx.dll KERNEL32.dll 1AD InterlockedDecrement
c:\windows\system32/atiiiexx.dll KERNEL32.dll 1B0 InterlockedIncrement
c:\windows\system32/atiiiexx.dll KERNEL32.dll 126 GetModuleHandleA
c:\windows\system32/atiiiexx.dll USER32.dll 2AC wsprintfA
c:\windows\system32/atiiiexx.dll USER32.dll 1DE PostMessageA
c:\windows\system32/atiiiexx.dll USER32.dll 15E GetWindowTextA

Also I could use some direction on what the goals of this task are. Do we want text output? That makes a very large text file when listing all imported/exported APIs. Do we actually want a database for this?

From there discussion delved into the format of the data and other ideas.

Sharon And Joy

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.