Wine Traffic #250 For 26�Nov�2004

By Brian Vincent

Table Of Contents

Introduction

This is the 250th issue of the Wine Weekly News publication. Its main goal is to eat deviled eggs. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org (http://www.winehq.org)

Mailing List Stats For This Week

We looked at 234 posts in 947K.

There were 59 different contributors. 38 posted more than once. 29 posted last week too.

The top posters of the week were:

1. News: Splash

20�Nov�2004�-�26�Nov�2004 (1 post) Archive Link: "News"

Topics: News

People: ,�WineHQ,�Zack Brown,�ReactOS,�News

Wine and the ReactOS (http://www.reactos.org) project have worked together for several years. Recently they've added a newsletter similar to this one to inform the broader community of what's going on. Splash (http://www.tong-web.com/splash/index.html) , written by Zach Tong, will be published on a biweekly basis and cover the ReactOS mailing lists. In some ways it provides an interesting contrast to news covered here because Wine developers sometimes tackle the same issues. However, most of the topics touch on completely unrelated subjects. Of course, that's exactly what you'd expect from a project writing it's own kernel, developing hardware drivers, and trying to clone the world's most popular operating systems.

You can also find Splash published on Zack Brown's Kernel Traffic (http://www.kerneltraffic.org/kernel-traffic/index.html) under the name ReactOS Traffic (http://www.kerneltraffic.org/reactos/archives.html) . It's probably a good time to mention this newsletter is also mirrored on two different sites: as Wine Traffic (http://www.kerneltraffic.org/wine/archives.html) and as the Wine Weekly News (http://www.winehq.com/?issue=back) on WineHQ. I'd encourage you to check out both formats and see which one you like the most - the content should be the exact same. Each one offers advantages over the other. As the WWN, this gets published pretty regularly and is updated first each week. As Wine Traffic you can opt to receive this via email (http://www.kerneltraffic.org/lists.html) or an RSS feed (http://www.kerneltraffic.org/rss.html) .

What you're reading right now is issue #250, which I suppose in some way is significant. More importantly, it's Thanksgiving. So, thank you to everyone who tunes in each week to read this!

2. Kernel Issue (con't)

19�Nov�2004�-�22�Nov�2004 (25 posts) Archive Link: "Re: ptrace single-stepping change breaks Wine"

Topics: Integration

People: Eric Pouech,�Linus Torvalds,�Davide Libenzi,�Jesse Allen,�,�Roland,�cvs

Last week (WWN #249 (http://www.winehq.com/?issue=249#Kernel%20Integration%20Problems) ) we mentioned a problem that seems to have popped up with certain kernels and certain Windows apps run with Wine. Some specific instructions can trigger the kernel to behave differently than it used to. Specifically what's going on is a change with the ptrace() system call. Most people can go their whole lives without ever bothering to learn about ptrace(), it provides functionality that most programs don't really care about.

ptrace() allows one process to hook into another and control it. A common application would be a debugger - you can use ptrace() to attach to a process, step through instructions, and then detach from it. In fact, the way you do this is by calling ptrace() with special request, such as ptrace(PTRACE_SINGLESTEP, <pid>, ...). That translates at the actual hardware level by setting a bit (the TF bit) in a register of the processor. If you want to get really fancy, you can begin changing instructions on the fly using ptrace and requests such as PTRACE_POKEDATA with some payload data attached - and that's exactly the type of stuff wineserver is capable of doing (see server/ptrace.c (http://cvs.winehq.com/cvsweb/wine/server/ptrace.c?rev=1.27&content-type=text/x-cvsweb-markup) .) Sound obscure? It is. Eric Pouech explained one of the cases Wine would need to use something like this:

we have (to control what's generated inside the various exception) to ptrace from our NT-kernel-like process (the ptracer) to get the context of the exception. Restart from the ptracer is done with PTRACE_SINGLESTEP.

After the initial investigation last week, some discussion ensued about what exactly has changed in the past few kernel releases and why it's broken Wine in some instances. Linus posted a patch and noted:

Here's a new patch (http://www.winehq.com/hypermail/wine-devel/2004/11/0481.html) to try. Totally untested.

It is more careful about clearing PT_DTRACED (which by now should probably be renamed PT_PRACE_SINGLESTEP or something on x86, since we should never be lazy about this thing any more), and it may or may not help.

Pls test _together_ with the previous patch (which is already applied in the current top-of-tree for anybody with really recent kernels).

Daniel Jacobwitz asked a question about it and Linus explained more about what it did:

It's not like single-stepping into the signal handler in any way removes any information (while _not_ single-stepping into it clearly does).

With the patch I just posted (assuming it works for people), Wine should at least have the choice. The behaviour now should be:

It sounds like the third case is what wine wants.

Linus then went back and came up with an example:

Hmmm.. I think I may have a test-case for the problem.

Lookie here:

Compile it, run it, and it should say

Now, try to "strace" it, or debug it with gdb, and see if you can repeat the behaviour.

Roland? Think of it as a challenge

Ha! Those wacky kernel guys! I won't even pretend to recognize the pushing and popping going on above. Davide Libenzi did understand though, " You know you're sick, don't you? Making traps inc's to get you in the correct opcode to move function in edx->fnp, is indeed fruit of a sick mind :=)"

Linus explained a little more about his little program from hell and how it might relate to Wine:

It _should_ be perfectly easy to debug this, by using

instead of "continue" when you get a SIGTRAP that wasn't due to anything you did.

But try it. It doesn't work. Why? Because the kernel will have cleared TF on the signal stack, so even when you do a "signal SIGTRAP", it will only run the trap handler _once_, even though it should run it three times (once for each instruction in between the "popfl"s.

I do think this is actually a bug, although whether it's the bug that causes problems for Wine is not clear at all. I'mm too lazy to build and boot an older kernel, but I bet that on an older kernel you _can_ do "signal SIGTRAP" three times, and it will work correctly. That would indeed make this a regression.

Later in the night, Linus came up with another patch (http://www.winehq.com/hypermail/wine-devel/2004/11/0559.html) and asked if it fixed the Wine problem. Jesse Allen tried it out and reported, " For the wine app in question, it does make a difference. However, there is a new problem. The program gets stuck at the loading screen at 100% CPU. It's making a call to select, timing out, and then tries select again, repeats. It's waiting for something that seems to never happen. "

Jesse made some logs available showing the app running under an old kernel and then under a new one. Eric Pouech went through them and deciphered the differences:

For the linux folks, here a small comparison of what happens in the working (old) case and in the non-working (new) case:

In both case

  1. Wine gets a first SIGTRAP (in it's sig_trap handler)
    • Wine converts it into a Windows exception (w-exception in short). This includes creating a context for the generic CPU registers
    • This w-exception is sent to the w-exception handler the program installed (this one can modifiy the all registers)
      • this handler touches one of the i386 debug registers
    • as we need to update the debug registers values (and we don't do in the signal handler return), this task is delegated to the Wine server (our central process, which is in charge of synchronisation...)
      • the Wine server ptrace-attach:es to the process which handled the SIGTRAP.
      • Wine server wait4:s on the SIGSTOP (after ptrace:attach)
      • modify (with ptrace) the debug registers
      • > and resumes excution (ptrace: cont)
    • wine terminates the sig trap handler and resumes the execution with the modified basic registers (from the saved context), and the modified debug registers (from the Wine server round trip)
  2. a second sig trap is generated
    • since the wine server is still ptrace:attached, it gets the signal.

What differs then in both execution: in the working case, the sig trap handler is called on the client side, whereas it's never called in the non-working case. We do have a couple of protection (to avoid some misbehaving apps), but none of them get triggered. So it seems like the trap handler is not called (ugh).

A couple of notes: as the program tested is copy protected, and as it seems that the copy protection is what causes the harm, it can be interesting to know that the programe seems to set the TF flag (some copy protection schemes directly do an "int 1", but given the exception code we get, this isn't the case). - in Windows trap handling, the TF is explictly reset before calling the windows exception handler (which is what Wine does, before calling the window exception handler). Of course the handler can set it back if it wants to continue single stepping.

Even though it seemed like the problem wasn't fixed for Jesse, Linus thought at least one issue was resolved:

This actually implies that the current -bk tree with my latest patch may actually fix it.

One of the things that 2.6.9 did wrong was exactly that it cleared TF too much in the ptrace interface. The current development tree is a lot more careful about that, and it fixes the horrid test-case that I used to debug it. The test-case (when run under gdb) actually does something similar to what Wine appears to do.

TF will be still set in Linux when ptrace gets access, but the ptracer can choose to clear it with PTRACE_PEEKUSR/PTRACE_POKEUSR (or with PTRACE_GETREGS/SETREGS). I assume you already do that, since I think that has been true forever (although maybe you don't: PTRACE_CONTINUE used to unconditionally clear TF, so it may be that Wine may need some minor modification to not do that - but the good news is that mod should be backwards-compatible, so it should be pretty easy).

I actually broke down and am downloading the latest source tree of wine, let's see if I can find the place you do this.

After looking through Wine's dlls/ntdll/signal_i386.c, Linus thought everything was ok.

3. Debian Packages

23�Nov�2004�-�24�Nov�2004 (21 posts) Archive Link: "Packaging Questions, New Debian Package, Packagers Guide"

Topics: Project Management

People: Scott Ritchie,�Mike Hearn,�Dimitrie Paun,�,�Francois Gouget,�Hans Leidekker,�Dimi Paun,�Marcus Meissner

Scott Ritchie stepped up this week to help out with packaging Wine for Debian:

The Debian packages have gotten rather out of date, and it looks like Ove's not going to be making them any more. I took the initiative and decided to try making one myself. I'm polishing off a new Debian package now.

Some major things I noticed:

  1. There were a lot of old hacks in the package that are probably no longer necessary, due to advancements in wine like wineprefixcreate and such. There was also the remenants of a very old problem with compiling using flex (which I remember experiencing the old days) in the form of a special exception for installing newer flex packages.
  2. Very old packages like winesetuptk should now be officially obsoleted by making them conflicted and replaced. I also did the same to the libwine-* packages that weren't doing anything but making documentation folders, like libwine-alsa.
  3. Making my package looks to be a lot simpler than what's implied in the (now year out of date) package makers guide. I found that documentation quite useless (and didn't even find it until I was about half way done anyway). Again, this is probably due to advancements in wine itself.
  4. What I didn't find is a standard list of packages that aren't strictly required for wine (like libxt-dev and flex), but that wine can benefit from. A good example would be the alsa development files. These are all things that should be included in the build dependencies for the package.

So far, my list of build dependant packages is the following: flex, bison, libx11-dev, libasound2-dev, libxt-dev, libicu28-dev

However, I'm not sure if this means the wine binary package should depend on them, since it's compiled in. So, should I make libicu28 a dependancy for wine?

Now, this leads to the question: is it worth even having a package maintainers guide? If so, who wants to update it?

The thread then branched out as lots of people wrote in to answer the questions Scott raised. Hans Leidekker pointed out a list of dependencies can be found in documentation/PACKAGING. That led to a side conversation about BiDi support and setting up a system with ICU. Francois Gouget added that NAS was required for building the winenas audio driver.

Mike Hearn thought it would be nice to change the way the Debian package was arranged, " Cool! While you're at it could you please combine them altogether so the packaging matches the upstream sources? Last time I checked the Debian packages were split into tons of little packages which is wrong and led to strange breakages. Just one package with everything installed by "make install" in the sources would be great. "

Dimi Paun agreed and thought putting the packages on the SourceForge download site would be nice too. Francois Gouget explained having a bunch of separate packages was standard on Debian. Mike thought this just caused more problems:

Well, it may be the Debian way but it's not the Wine way.

I'm not saying this just to be pedantic, the fact that distros like Debian creatively ignore the way upstream is packaged caused me big problems when I wrote the IE installer script way back in January (which still gets ~300 hits a day). I assumed that if you had Wine installed, you'd have things like wineboot and regedit installed. These are shipped as part of Wine after all. But they weren't, Debian had classified them as optional utilities and my script broke. Gentoo had similar problems. I had to add in lots of hacks to make sure all the programs I used were there, and print out different messages for each distro giving the package name needed.

It's not just my script. Wine and Windows programs sometimes assume these applets are present too (eg notepad, wineboot, regedit, regsvr32).

The packaging on Red Hat/SuSE/Mandrake etc didn't have this problem because they mostly followed upstream conventions.

If this way of splitting everything up really is better we should split the Wine tarballs up into separate packages too. If it isn't then Debian should follow the upstream conventions, it would save people a lot of hassle debugging incomplete installs because some program tried to invoke regedit or notepad and it wasn't there.

Scott made the change:

Well, I am condensing it down. Here's what I think we should move towards with packaging:

My reason for splitting off libwine and wine is that libwine can be installed without wine and could someday be used to launch a program that has been ported with winelib, without having to have wine proper in it.

I imagine someday I'll take an open source Windows program like eMule, set it up to compile with winelib, and have it turned into a Debian package. This way one could apt-get install emule and then have it depend on and install libwine, but not wine.

The reason for using libwine rather than winelib is to comply with Debian standards.

Dimi corrected Scott and pointed out that you really can't split wine from winelib as they go hand in hand. He then went on to discuss the packaging guide, " Well, it would be great if you'd use the experience you gathered doing the .deb packages to update the guide. Really, it can only be updated by a packager, someone that went through the motions. Moreover, the experienced packagers don't need that document as much, and are not likely to update it. So the only hope we have is for someone 'new' to the wine-packaging business to take on the job. "

Before Scott could even reply, Marcus Meissner did it. You can find that revised doc here (http://www.winehq.com/hypermail/wine-devel/2004/11/att-0609/01-PACKAGING) for now.

4. Install Script for IE and Media Player

25�Nov�2004 (1 post) Archive Link: "Internet Explorer 6 Installer script"

Topics: Fixes

People: Hajime Segawa,�,�Microsoft

Wine is getting increasingly better at running Internet Explorer and Windows Media Player. Hajime Segawa announced a simple way to get them working:

I wrote a tiny shell script that can install IE6sp1 and WMP7. I hope this script can help you. RealPlayer 10, MSN Messenger 6.4, Yahoo!Japan Messenger seem to work with this configuration.

I took some screenshots.

If you dissect the script you'll notice a lot of what it does is install some proprietary DLL's. Specifically, DCOM98, MFC 4.0, and Microsoft Installer. So yes, it's cheating and Wine isn't running these completely out of the box. However, you can run them.

5. Customizing File Browsing

24�Nov�2004�-�25�Nov�2004 (17 posts) Archive Link: "Winecfg -> browse for folder"

Topics: Winelib

People: Mike Hearn,�Robert van Herk,�Alexandre Julliard,�

Eradicating the config file continues. Mike Hearn posted another patch to improve winecfg this week and noted these additions:

That led Robert van Herk to begin working on it and starting to make more additions. Mike gave some pointers for what needs to be tackled:

Here are some more winecfg todos for everybody to take a crack at now my last patch got checked in:

.... and of course, the big cheese ....

Robert then presented a dilemna:

Thanks to Mike's help, I succeeded in showing a SHBrowseForFolder thingy, in the winecfg program, so that people can pick a directory they want to use as virtual C drive.

However, of course, this directory choosing thingy shows the virtual Windows file system, not the Unix file system. Hence, I think we have a tiny bootstrapping problem here. If I just show the user his windows file system, he might not be able to pick the directory he wants.

Is Z:\ always mapped to the root of the unix file system? Because that would seem to solve it: just let them choose a directory from Z:\ and then probably modify the returned string by removing Z: and replacing all /'s with \'s.

There was a lot of discussion about different ways to pull this off but there definitely wasn't a simple solution. Alexandre weighed in with the direction he'd like to see it go, " We need to somehow extend the Windows file dialogs to allow browsing the Unix file system, this is something many Winelib apps will want too. "

Robert was actually glad to hear this. Implementing it could be done a few different ways, so he asked for some advice:

OK, so how to do this? I see 3 ways:

  1. Implement an extra api that do the same as the normal file open/save and browse for folder api, but then with the Unix file system
  2. Implement a thread local variable so that threads can put the Wine file dialogs into unix file system mode / windows file system mode
  3. Define some magic wine flags in the file dialogs that make the dialogs appear in Unix mode

There are some pro's and con's to all of them:

  1. Well, basically, we have to make an extra api :-), so there are more functions exported. Probably the existing and new functions will need to call each other, so this means an extra level of indirection.
  2. Kinda ugly hacking, though we don't loose win32 compatibility
  3. Nice, but we loose strict win32 compatibility, since there will be a magic flags that doesn't exist in win32.

Mike suggested the 3rd approach, " This is the most lightweight so I'd go for it for now. If we find that the flag value we pick is already in use we can just change it, no big deal. We already use custom Wine flags in other parts of the code (eg WS_EX_TRAYWINDOW)."

6. No RichEdit20A Window Class

18�Nov�2004�-�19�Nov�2004 (11 posts) Archive Link: "No RichEdit20A window class"

Topics: Controls

People: Michael Ost,�Mike McCormack,�CodeWeavers,�,�James Hawkins,�WineHQ

Michael Ost ran into a problem getting an app to use Wine's RichEdit controls:

My winelib application can't create a window of class RichEdit20A. It looks like the class is not being registered, because the DLL is not being loaded. Any clues why this might be?

Mike McCormack described the RichEdit controls and what the state of them is in Wine:

RICHEDIT_CLASS20A is provided by riched20.dll, not riched32.dll. The windows 2000 implementation uses riched20.dll to implement riched32.dll (ie. moves the richedit code to riched20.dll and implements the RICHEDIT_CLASS10A class using the new RICHEDIT_CLASS20A class.

In short, the existing code registers the correct class name, but we need to make a new dll dlls/riched20, and do alot of work on the richedit control :)

Actually, since we need to work on riched20, we might as will implement that properly, rather than trying to reuse the edit control.

Michael asked for some clarification and explained what he was trying to do:

This isn't quite clear to me. It sounds like you are saying there is a riched20.dll in wine, but I can't find one. Are these statements correct?

My winelib app is hosting a 3rd party DLL which is specifically requesting "RichEdit20A". That suggests that the DLL was built with win2K or later, from what you were saying. Sounds like my options are to:

Mike confirmed that richedit20.dll doesn't exist yet in Wine and therefore the class name RICHEDIT_CLASS20A doesn't exist. CodeWeavers has a hack to get around that fact and Mike described it:

We have an empty implementation of riched20 that simply forwards requests to edit. That is the wrong, and the existing riched32 code is broken too (since it uses the edit control).

CrossOver's riched20 code is attached... it's a quick and easy way to fix the problem, but the correct, longterm solution for Wine is to implement riched20.dll properly and then forward requests from riched32 to riched20 rather than the edit control, which is why this code is not in WineHQ.

James Hawkins wanted to know why it wasn't in Wine, but Mike reiterated that it was just a hack that avoided solving the real problem, " The solution is to start work on a skeleton control, perhaps that just has some basic drawing functionality and not much else. Then gradually extend it to become the mini word processor it actually is."

James wanted some clarification on the direction, so Mike went on:

The current implementation of riched32 is Wrong(tm).

We'll implement riched20 from scratch and do it the right way. When it's good enough, we can then rewrite riched32 to use the riched20 control instead of the edit control.

All that can be done in small incremental improvements, which is the key to making cooperative development work.

7. CVS Broken?

25�Nov�2004 (3 posts) Archive Link: "Breakage on Debian Sarge"

Topics: Project Management

People: Michael Jung,�,�Michael Stefaniuc,�cvs

Reports seem to indicate that the current CVS of Wine is broken. Perhaps by the time you read this it will be fixed. Michael Jung mentioned it first:

this patch (http://cvs.winehq.org/patch.py?id=14575) broke wine on a current Debian Sarge. Tested this on two machines. GUI applications do not respond any longer to mouse events. Tested regedit and winemine. Removing the patch solves the problem.

Stefan D�singer ran across the same problem on Gentoo and Michael Stefaniuc also did on RHEL3 and Fedora Core 2.

Sharon And Joy

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.