Wine Traffic #245 For 22�Oct�2004

WHAT'S NEW with Wine-20041019: (see ChangeLog for details)

• Beginnings of an architecture for Direct3D 9 support.
• Many debugger regressions fixed.
• Improvements to the IDL compiler and the COM headers.
• New MSCMS dll (color management support).
• Lots of bug fixes.

I have a win32 program written in C++. I am trying to port it to linux using winelib. The software contanis one exe file and several dll files. I would like to keep this structure when moving them to linux. That is, I still would like to generate several corresponding shared libraries (instead of statically linking all files together) in llinux and they will be loaded at the runtime.

However using winelib tools (winemaker, configure, make) for shared library generation, I get unresolved symbols error.

It seems that I have to write a spec file for each dll to import(and export) functions provide by other dlls. However, since they all written in c++ and what need to be imported can either be class or class method, and parameters can be class either. SO I don't know how to handle this in spec file. Beside each dll file exports many functions or classes. It would be tedious to write a lengthy spec file manually.

My question is that if it is possible that linking is not invoked during shared library generation under wine (but during the runtime)? How should I do it?

I don't have an answer, just tagging along here. I'm curious what other people have come up with, both for the long-term solutions and the best way to get it working today.

I'm working on a large C++ application that is divided into many shared objects. We are currently using a version of wine that is about a year old. With that version, we were able to get C++ shared libraries to link together by bypassing wine's DLL mechanism for all of the C++ symbols -- we just let the native linux loader resolve the C++ symbols. We just created empty .spec files for all of the C++ DLLs we wrote. When we called winebuild to generate the .spec.c file for each DLL, we would only tell winebuild that we were linking against DLLs that were part of wine (kernel32, user32, etc). Then when we did the actual linking for the DLL we were building, we would pass in the names of the other DLLs we depend on the typical way (-lour_library.dll), and the native linker would resolve those symbols for us. This worked very well for us. I suspect there are a few things that this would have broken, like calling GetProcAddress on one of our C++ symbols, but that isn't a feature we ever used.

We got a rude suprise when we tried to update to a current version of wine a month or two ago. The DLL loading mechanism had changed during that span, and hack we were using would no longer work. It looks like the current version of wine won't allow you to link to a wine DLL with the native linker. It expects all linking between wine DLLs to be done through winebuild, and it gets confused when a DLL gets loaded (by the shared loader) that wine didn't explicitly load. We haven't yet figured out what to do about this.

We could try to modify wine's loading mechanism to allow the kind of hackery we were doing before, but I don't know much about this part of wine and I'm not sure what the consequences of loding DLLs out-of-order are. I guess the other option is to generate .spec files for all of our DLLs, which is an option that doesn't really appeal to me either. It'd be pretty easy to write a script that will massage the output of nm(1) to generate a .spec file, but if you do that you're going to be exporting every symbol, most of which will probably be unused.

I seem to recall that there are experimental(?) modifications to gcc and binutils that understand declspec(__dllexport)-type annotations. I think this was part of mingw. IIRC, gcc would somehow mark the exported symbols in the object files, and a separate utility would scan the object files and create a .def file with all of the exported symbols. I'm not sure this stuff even compiles on linux. Does anyone have a solution they're happy with?

Do a .spec file for each C++ dll with one c function like:

void mydll_export( ) {
}
// you need real code + .spec declaration

In your app or in the dlls that use the above dll do a function like:

void using_ddls() {
mydll_export( ) ;
mydll2_export() ;
}
// this code forces the winebuild to pull-in listed dlls as well

link every thing the regular wine way this way DLLS load in the right order, get initialized, and their Windows "import tables", like calls to kernel32 etc, gets initialized by the loader. But (and here is where I'm out of dated) also specify the .so as a link option to the gcc linker. (ld)

In the old system, before winegcc. One would do -lmydll on the winebuild command line. And than in turn -lmydll on the ld command line - for resolve of C++ symbols. Now that we use winegcc I'm not sure what is the switch for additional libs like .so and static libs. Look maybe it is documented. (Dimi how do you add external libs to a winelib link stage under winegcc?)

But be careful with this approach. It is an order of a magnitude slower on load time than DLL linkage on windows. I came to a dead end with one of my projects, where I managed to compile and run every thing but I had to revert to PE compiled code because it took my app 4-6 minutes to load. (PE takes 40 seconds). It was a 1.2 M lines of C++ code divided in to 37 DLLs + MFC in a dll.

We have talked about the right solution with Alexander on Wineconf. What he suggested was:

1. make the __declspec( export ) macro expand to a gcc "section" declaration. So the compilers put all of them in a special section. No one knew how C++ classes behave with "sections" and we suspected each member function has to be put into the section by hand. On windows a __declspec( export ) on the class declaration automatically exports all members. Maybe MinGW could help out here.
2. Use a tool like nm or readelf to extract all symbols (including C++), in above section, and with a script convert them to a .spec file.
3. I think that current winebuild have problems with some variants of the C++ mangled symbols like the use of the "@" character. So maybe a new "C++" style function should be added to the .spec syntax, that will not try to interpret the symbols.

Thanks for the suggestion! Even if it isn't elegant, it sounds like it will work (and is very close to what we were doing with the older wine). I might even be able to coax our build system into generating all of the *_export() and using_dlls() functions automatically.

I'm working on a large project too, well over a million lines as reported by 'wc -l'. We've got about 50 shared libraries, but only 10 of those use wine; the rest are entirely native. We think our application takes too long to load, but not nearly 4-6 minutes. On a fast machine if you run it a few times in a row, it takes around 8 seconds to load. On a lesser machine that has been doing other things, it will often take 40 seconds or so. I'm not really sure where all of the time is spent. I've been assuming that the majority of it is spent by ld.so. Most of the DLLs have 200-300 symbols imported from the standard wine libraries, so the whole application is doing about 3000 fixups in our project. I can't see that taking too long (but I've never timed it). I think if I started using the method you suggested, it should add less than a hundred extra fixups for wine to perform -- we're really just changing what causes the libraries to load, not what they import.

Was all of the time your application took to load because of wine's DLL import mechanism, or because of the shared loader in linux? One of the reasons we are trying to upgrade to a more recent version of winelib is so that we can try some of the 'prelinking' tools to try to make our app load faster, so I'm curious where all of the time is spent.

Most definitely the Linux-shared loader. It took ages. The code is heavy C++ code full of templates with weak symbols, inline virtual functions, and plain horizontal code structure. almost any thing you can do to slow a linker down. The PE export-import tables are much better in this situation since there is no searching to do. (and no fixups) All searching if any is done in link-time. (Run time on the other hand is slower). Actually this app is very bad on Windows too. So we compile it: "all libraries static" when doing "Release" code. We use DLLs in debug because the static linking takes 15 minutes.

Speaking of which I was never able to statically link the winlib app on Linux. I put up a machine with 2-Gg of ram + setup 2Gg of swap space. The linker would work for 5-10 minutes than it would hit the swap space. Memory would go up and up until about 50 minutes where the kernel starts to kill every thing in sight including the Linker. MSVC++ does not do that, it uses temporary files. Lots of them, to finish the link. I did not even try none-debug builds because the all point of the winlib was that Developers could use full screen debugger (Kdevelop) to debug Linux problems. If we only have traces and relays than PE is Just good enough.

Thanks for letting me whine. ;-) Please do post your results on how to Link the .so on the winegcc command line. And also most important if you succeed with prelinking. I think that is the best way to go but I'm not a Linux guru and didn't manage to do it.

I took a look at the Startmenu. It does its job except for the following minor problems:

• Keyboard input is completely ignored
• Icon Backgrounds are white instead of grey
• All the Icons were drawn a few pixels below where they belong, which makes them a bit overdrawn by the Icon below.
• The Strings above the seperator were drawn next to the right border wich makes them partially overdrawn by the Buttons Icon. (Strings and Buttons beyond the seperator are drawn right)

If the mouse hovers over the button, the text redraws at the right place

Thanks. Could you try the attached patch which should fix the white background issue?

Also, could you try setting both OFFSET_X and OFFSET_Y to 0 in wine/dlls/comctl32/toolbar.c

The strings being in the wrong position is a bigger issue. If you are interested, the problem is that we recalculate and redraw our toolbar control too much so that the code to create the Start Menu expects that it won't be refreshed until it has finished setting the various properties of the toolbar. I have some unfinished work in the toolbar control at the moment, but I plan to use controlspy to see which messages should cause a redraw and which shouldn't.

You can download controlspy here: http://www.microsoft.com/msj/0798/controlspy.aspx

It is very useful for throwing messages at the common controls, but it cannot test custom draw functionality and it is becoming dated (it is missing new functionality from comctl32 v5.80 and new controls).

if((wndPtr->dwExStyle & WS_EX_LEFTSCROLLBAR) != 0)

if (wndPtr->dwExStyle & WS_EX_LEFTSCROLLBAR)

At any optimization level, gcc compiles this to identical code.

>From the C point of view these bit logics are identical. The compiler finds that out easy.

It's in misc/registry.c. And no, it's definitely not nice at all; unfortunately we can't use the value from the registry in the code that's supposed to load the registry ;-) (well, we could, but that will require some serious restructuring of that code).

Winedbg's watchpoints don't seem to work for me: when I try to watch a memory location, winedbg responds that a watchpoint has been set at a different, always constant location (I suspect this is actually in winedbg's memory space). Nothing happens when the location I was trying to watch changes.

Here is a cut-and-paste from my winedbg session. I try to watch memory location 0x0041a5ec, but I can set other breakpoints and see that the value has changed without the watchpoint taking effect.

Wine-dbg>run
In 32 bit mode.
0x4049a81f start_process+0xcf [/home/reuben/project/wine/wine/dlls/kernel/../../include/winternl.h:1249] in kernel32: jmp 0x4049a815 start_process+0xc5
[/home/reuben/project/wine/wine/dlls/kernel/process.c:1022] in kernel32
1249 static inline void WINAPI DbgBreakPoint(void) { __asm__ __volatile__("int3"); }
1249 static inline void WINAPI DbgBreakPoint(void) { __asm__ __volatile__("int3"); }
Wine-dbg>watch *0x0041a5ec
Watchpoint 1 at 0x405ae8c4
Wine-dbg>x 0x0041a5ec
00000000
Wine-dbg>break *0x004130ec
Breakpoint 2 at 0x004130ec
Wine-dbg>cont
fixme:ole:CoRegisterMessageFilter stub
Stopped on breakpoint 2 at 0x004130ec
Wine-dbg>x 0x0041a5ec
403e9d40
Wine-dbg>cond 2 $ecx==0x10
Wine-dbg>cont
Stopped on breakpoint 2 at 0x004130ec
Wine-dbg>x 0x0041a5ec
00000000
Wine-dbg>info break
Breakpoints:
2: y 0x004130ec (1)
stop when ($ecx == 16)
Watchpoints:
1: y 0x405ae8c4 on 4 bytes (W)

The fact that winedbg thinks the watchpoint is 0x405ae8c4 is especially puzzling, but it's entirely possible that the confusion lies with me rather than winedbg.

Thanks. That fixes the watchpoints, but introduces a couple of small problems:

1. in dbg.y, break_add_watch_from_lvalue should take only one argument (drop second argument)
2. in dbg.y, I have no minidump_write. Should this be replaced with dbg_printf("%s\n", $2);i ?

After fixing these two things, breakpoints work, although the instruction pointer displayed is the one just after the watched address gets written to. Perhaps this is the expected behavior, but it would be nice to have the instruction that makes the write instead.