Wine Traffic #250 For 26 Nov 2004

we have (to control what's generated inside the various exception) to ptrace from our NT-kernel-like process (the ptracer) to get the context of the exception. Restart from the ptracer is done with PTRACE_SINGLESTEP.

Here's a new patch to try. Totally untested.

It is more careful about clearing PT_DTRACED (which by now should probably be renamed PT_PRACE_SINGLESTEP or something on x86, since we should never be lazy about this thing any more), and it may or may not help.

Pls test _together_ with the previous patch (which is already applied in the current top-of-tree for anybody with really recent kernels).

It's not like single-stepping into the signal handler in any way removes any information (while _not_ single-stepping into it clearly does).

With the patch I just posted (assuming it works for people), Wine should at least have the choice. The behaviour now should be:

• if the app sets TF on its own, it will cause a SIGTRAP which it can catch.
• if the debugger sets TF with SINGLESTEP, it will single-step into a signal handler.
• it the app sets TF _and_ you ptrace it, you the ptracer will see the debug event and catch it. However, doing a "continue" at that point will remove the TF flag (and always has), the app will normally then never see the trap. You can do a "signal SIGTRAP" to actually force the trap handler to tun, but that one won't actually single-step (it's a "continue" in all other senses).

It sounds like the third case is what wine wants.

Hmmm.. I think I may have a test-case for the problem.

Lookie here:

#include <signal.h>
#include <sys/mman.h>

void function(void)
{
printf("Copy protected: ok\n");
}

void handler(int signo)
{
extern char smc;
smc++;
}

#define TF 0x100

int main(int argc, char **argv)
{
void (*fnp)(void);

signal(SIGTRAP, handler);
mprotect((void *)(0xfffff000 & (unsigned long)main), 4096, PROT_READ | PROT_WRITE);
asm volatile("pushfl ; orl %0,(%%esp) ; popfl"
: :"i" (TF):"memory");
asm volatile("pushfl ; andl %0,(%%esp) ; popfl"
: :"i" (~TF):"memory");
asm volatile("\nsmc:\n\t"
".byte 0xb7\n\t"
".long function"
:"=d" (fnp));
fnp();
exit(1);
}

Compile it, run it, and it should say

Copy protected: ok

Now, try to "strace" it, or debug it with gdb, and see if you can repeat the behaviour.

Roland? Think of it as a challenge

It _should_ be perfectly easy to debug this, by using

signal SIGTRAP

instead of "continue" when you get a SIGTRAP that wasn't due to anything you did.

But try it. It doesn't work. Why? Because the kernel will have cleared TF on the signal stack, so even when you do a "signal SIGTRAP", it will only run the trap handler _once_, even though it should run it three times (once for each instruction in between the "popfl"s.

I do think this is actually a bug, although whether it's the bug that causes problems for Wine is not clear at all. I'mm too lazy to build and boot an older kernel, but I bet that on an older kernel you _can_ do "signal SIGTRAP" three times, and it will work correctly. That would indeed make this a regression.

For the linux folks, here a small comparison of what happens in the working (old) case and in the non-working (new) case:

In both case

1. Wine gets a first SIGTRAP (in it's sig_trap handler)
   • Wine converts it into a Windows exception (w-exception in short). This includes creating a context for the generic CPU registers
   • This w-exception is sent to the w-exception handler the program installed (this one can modifiy the all registers)
     • this handler touches one of the i386 debug registers
   • as we need to update the debug registers values (and we don't do in the signal handler return), this task is delegated to the Wine server (our central process, which is in charge of synchronisation...)
     • the Wine server ptrace-attach:es to the process which handled the SIGTRAP.
     • Wine server wait4:s on the SIGSTOP (after ptrace:attach)
     • modify (with ptrace) the debug registers
     • > and resumes excution (ptrace: cont)
   • wine terminates the sig trap handler and resumes the execution with the modified basic registers (from the saved context), and the modified debug registers (from the Wine server round trip)
2. a second sig trap is generated
   • since the wine server is still ptrace:attached, it gets the signal.

What differs then in both execution: in the working case, the sig trap handler is called on the client side, whereas it's never called in the non-working case. We do have a couple of protection (to avoid some misbehaving apps), but none of them get triggered. So it seems like the trap handler is not called (ugh).

A couple of notes: as the program tested is copy protected, and as it seems that the copy protection is what causes the harm, it can be interesting to know that the programe seems to set the TF flag (some copy protection schemes directly do an "int 1", but given the exception code we get, this isn't the case). - in Windows trap handling, the TF is explictly reset before calling the windows exception handler (which is what Wine does, before calling the window exception handler). Of course the handler can set it back if it wants to continue single stepping.

This actually implies that the current -bk tree with my latest patch may actually fix it.

One of the things that 2.6.9 did wrong was exactly that it cleared TF too much in the ptrace interface. The current development tree is a lot more careful about that, and it fixes the horrid test-case that I used to debug it. The test-case (when run under gdb) actually does something similar to what Wine appears to do.

TF will be still set in Linux when ptrace gets access, but the ptracer can choose to clear it with PTRACE_PEEKUSR/PTRACE_POKEUSR (or with PTRACE_GETREGS/SETREGS). I assume you already do that, since I think that has been true forever (although maybe you don't: PTRACE_CONTINUE used to unconditionally clear TF, so it may be that Wine may need some minor modification to not do that - but the good news is that mod should be backwards-compatible, so it should be pretty easy).

I actually broke down and am downloading the latest source tree of wine, let's see if I can find the place you do this.

The Debian packages have gotten rather out of date, and it looks like Ove's not going to be making them any more. I took the initiative and decided to try making one myself. I'm polishing off a new Debian package now.

Some major things I noticed:

1. There were a lot of old hacks in the package that are probably no longer necessary, due to advancements in wine like wineprefixcreate and such. There was also the remenants of a very old problem with compiling using flex (which I remember experiencing the old days) in the form of a special exception for installing newer flex packages.
2. Very old packages like winesetuptk should now be officially obsoleted by making them conflicted and replaced. I also did the same to the libwine-* packages that weren't doing anything but making documentation folders, like libwine-alsa.
3. Making my package looks to be a lot simpler than what's implied in the (now year out of date) package makers guide. I found that documentation quite useless (and didn't even find it until I was about half way done anyway). Again, this is probably due to advancements in wine itself.
4. What I didn't find is a standard list of packages that aren't strictly required for wine (like libxt-dev and flex), but that wine can benefit from. A good example would be the alsa development files. These are all things that should be included in the build dependencies for the package.

So far, my list of build dependant packages is the following: flex, bison, libx11-dev, libasound2-dev, libxt-dev, libicu28-dev

However, I'm not sure if this means the wine binary package should depend on them, since it's compiled in. So, should I make libicu28 a dependancy for wine?

Now, this leads to the question: is it worth even having a package maintainers guide? If so, who wants to update it?

Well, it may be the Debian way but it's not the Wine way.

I'm not saying this just to be pedantic, the fact that distros like Debian creatively ignore the way upstream is packaged caused me big problems when I wrote the IE installer script way back in January (which still gets ~300 hits a day). I assumed that if you had Wine installed, you'd have things like wineboot and regedit installed. These are shipped as part of Wine after all. But they weren't, Debian had classified them as optional utilities and my script broke. Gentoo had similar problems. I had to add in lots of hacks to make sure all the programs I used were there, and print out different messages for each distro giving the package name needed.

It's not just my script. Wine and Windows programs sometimes assume these applets are present too (eg notepad, wineboot, regedit, regsvr32).

The packaging on Red Hat/SuSE/Mandrake etc didn't have this problem because they mostly followed upstream conventions.

If this way of splitting everything up really is better we should split the Wine tarballs up into separate packages too. If it isn't then Debian should follow the upstream conventions, it would save people a lot of hassle debugging incomplete installs because some program tried to invoke regedit or notepad and it wasn't there.

Well, I am condensing it down. Here's what I think we should move towards with packaging:

• wine : depends on libwine, contains the binaries for running windows programs
• libwine : contains everything needed to run windows applications
• libwine-dev : contains the files needed to compile windows applications with winelib

My reason for splitting off libwine and wine is that libwine can be installed without wine and could someday be used to launch a program that has been ported with winelib, without having to have wine proper in it.

I imagine someday I'll take an open source Windows program like eMule, set it up to compile with winelib, and have it turned into a Debian package. This way one could apt-get install emule and then have it depend on and install libwine, but not wine.

The reason for using libwine rather than winelib is to comply with Debian standards.

I wrote a tiny shell script that can install IE6sp1 and WMP7. I hope this script can help you. RealPlayer 10, MSN Messenger 6.4, Yahoo!Japan Messenger seem to work with this configuration.

http://sidenet.ddo.jp/winetips/config.html

I took some screenshots.

http://sidenet.ddo.jp/winetips/files/

Here are some more winecfg todos for everybody to take a crack at now my last patch got checked in:

• The drive detection algorithm needs some polishing. It mapped my /boot directory on FC3 even though that's useless. Maybe we need a blacklist? [easy]
• We should also ensure the home directory is mapped in the same way that we ensure / and c:\windows is mapped. [easy]
• Integrate drive detection with wineprefixcreate. The code nearly supports this and was designed with that in mind, check out the different ways errors are presented from the CLI and GUI. You just need to add a command line switch to the program that runs the autodetect_drives() function. [not too hard]
• There's some weird clipping/painting corruption in the advanced view of the drives page. It looks like a WM rewrite regression so I asked Alexandre to check it out but if you're up for a challenge feel free to try and beat him to it! [marine level hardness!]
• Unicodify it, at the moment it uses some ANSI APIs and such [intermediate]
• Usability review [intermediate]

.... and of course, the big cheese ....

• Write the migration code to pull the users current config across to the new registry branch so we can actually start thinking about switching the config file off once and for all. Bonus points for adding an explanation of what happened to the old config file itself!
  
  [easy in theory, might be hard getting it good enough to get past Alexandre!]

Thanks to Mike's help, I succeeded in showing a SHBrowseForFolder thingy, in the winecfg program, so that people can pick a directory they want to use as virtual C drive.

However, of course, this directory choosing thingy shows the virtual Windows file system, not the Unix file system. Hence, I think we have a tiny bootstrapping problem here. If I just show the user his windows file system, he might not be able to pick the directory he wants.

Is Z:\ always mapped to the root of the unix file system? Because that would seem to solve it: just let them choose a directory from Z:\ and then probably modify the returned string by removing Z: and replacing all /'s with \'s.

OK, so how to do this? I see 3 ways:

1. Implement an extra api that do the same as the normal file open/save and browse for folder api, but then with the Unix file system
2. Implement a thread local variable so that threads can put the Wine file dialogs into unix file system mode / windows file system mode
3. Define some magic wine flags in the file dialogs that make the dialogs appear in Unix mode

There are some pro's and con's to all of them:

1. Well, basically, we have to make an extra api :-), so there are more functions exported. Probably the existing and new functions will need to call each other, so this means an extra level of indirection.
2. Kinda ugly hacking, though we don't loose win32 compatibility
3. Nice, but we loose strict win32 compatibility, since there will be a magic flags that doesn't exist in win32.

My winelib application can't create a window of class RichEdit20A. It looks like the class is not being registered, because the DLL is not being loaded. Any clues why this might be?

RICHEDIT_CLASS20A is provided by riched20.dll, not riched32.dll. The windows 2000 implementation uses riched20.dll to implement riched32.dll (ie. moves the richedit code to riched20.dll and implements the RICHEDIT_CLASS10A class using the new RICHEDIT_CLASS20A class.

In short, the existing code registers the correct class name, but we need to make a new dll dlls/riched20, and do alot of work on the richedit control :)

Actually, since we need to work on riched20, we might as will implement that properly, rather than trying to reuse the edit control.

This isn't quite clear to me. It sounds like you are saying there is a riched20.dll in wine, but I can't find one. Are these statements correct?

• Wine's only implements RichEdit10A via riched32.dll. Wine does not implement RichEdit20A.
• Win2K (and later) provides riched20.dll which implements RICHEDIT_CLASS20A. This one I can verify as true on my winxp system.
• There are significant differences between the two window classes

My winelib app is hosting a 3rd party DLL which is specifically requesting "RichEdit20A". That suggests that the DLL was built with win2K or later, from what you were saying. Sounds like my options are to:

• hack their request to use the window class registered by wine's riched32.dll. How well could I expect this to work? How big are the diffs between 10A and 20A?
• get them to provide win2k's riched20.dll
• have them change their DLL to ask for RichEdit10A instead
• implement riched20.dll in Wine. Sounds like a big job!

We have an empty implementation of riched20 that simply forwards requests to edit. That is the wrong, and the existing riched32 code is broken too (since it uses the edit control).

CrossOver's riched20 code is attached... it's a quick and easy way to fix the problem, but the correct, longterm solution for Wine is to implement riched20.dll properly and then forward requests from riched32 to riched20 rather than the edit control, which is why this code is not in WineHQ.

The current implementation of riched32 is Wrong(tm).

We'll implement riched20 from scratch and do it the right way. When it's good enough, we can then rewrite riched32 to use the riched20 control instead of the edit control.

All that can be done in small incremental improvements, which is the key to making cooperative development work.

this patch (http://cvs.winehq.org/patch.py?id=14575) broke wine on a current Debian Sarge. Tested this on two machines. GUI applications do not respond any longer to mouse events. Tested regedit and winemine. Removing the patch solves the problem.