Wine Traffic #230 For 9 Jul 2004

A couple of weeks ago, we decided to rewrite our System.Windows.Forms implementation. Today's implementation suffered from various problems:

• Debugging with Wine was extremely hard, and Wine was a moving target, internals change from release to release.
• Wine/GDI+ interactions were not efficient.
• Too many cooks contributed to the core, so it was hard to maintain.

So a couple of weeks ago we decided to take a step back and redesign it, as you remember one of the reasons for using Wine is that it would allow third-party components to P/Invoke and to hook up to WndProc, this is sadly required for most third-party commercial controls and for advanced applications like WebMatrix to run.

Our new implementation takes a different approach, it will be completely managed and will not use Wine, this should help the portability to new operating systems, and should help on the debugging side of things, and should help with the large set of Wine/GDI+ interactions that we have.

To support WndProc, this new version uses the most commony used WndProc events internally so applications that depend on this works, and we will have an optional plugin that would use WineLib to host advanced functionality.

This is a rewrite from scratch to support the new design, and we hope to have something in the next couple of weeks that will go into the Mono CVS repository.

The core is being worked on by Peter Bartok, and once the core is ready familiar faces: Jackson, Jordi and Ravindra will join the effort to implement the missing functionality, and we hope that interested developers will join this effort.

PDB might not be able to fuss, but I will. I recently asked about a way to figure out which version of Wine is in use at run-time because of the recent changes to wine_init()'s prototype. Alexandre appeared quite adamant that he was against introducing any such mechanism.

Wine has a bit of a problem with the Mono project and our much smaller FST project to support VST win32/x86 plugins on linux. It has previously been a self-contained project that did not provide libraries, and whose internal API was not accessed by anyone or anything outside of Wine itself. It now is being used by (at least) 2 projects as a *library*, and it can either accept that role - which implies providing (at least) runtime version info - or reject it, which would be a great shame.

To be frank, I think we could declare Wine interface stable *today*. Versioning APIs is not exactly rocket science, yes it implies compromises but it's quite clear that Wine 1.0 is years away. These people want to use it today. We have all kinds of goals for 0.9 and 1.0 mixed in together and we seem to have decided pretty much arbitrarily that they should all occur at once, but I don't see the logic behind that.

I really feel that yes we should support Wine as a library, and that means making the APIs we export stable ASAP. I especially think we should stop the nonsense where we use GNU symbol versions in libwine marked "1.0" while not actually keeping them stable.

OK, so having had that rant, I also think that not using Wine to implement System.Windows.Forms is crazy too (hmm, there's a theme here, guess I'm feeling contrary today). The only reason to implement SWF is to run .NET apps built for Windows, it's a sucky toolkit compared to GTK# and isn't going to be used for writing apps on Linux that's for sure.

Therefore perfect compatibility is crucial. SWF is a very leaky API, there are the obvious ones like WndProc and Control.Handle but it almost certainly leaks in other ways: for instance, the order in which events are fired, that sort of thing. This *matters*, I've seen many apps crash and burn in mysterious ways due to tiny differences in the order notifications were sent to it and other subtleties.

Now, I think some of Peters arguments are valid and some aren't. Let's take a look at them:

Wine is too hard to use as a library. It does complicated things with registers and threads, and requires wierd bootstrap code.

I'm not sure what to make of this one. The low level code isn't optional, it's required in order to provide the environment Win32 code requires. Mono likewise does some pretty complicated stuff to provide a .NET JITing runtime so you guys are clearly competent when it comes to low level code.

If you skip the hard stuff here then you'll just hit a brick wall further down the road when you start trying to run *real apps* which P/Invoke to native code, poke controls via their Handles and other icky things. .NET simply wasn't created for portability at all, regardless of what Microsoft might say, and production .NET apps developed for Windows will require a Windows environment.

Wine isn't portable enough.

I disagree here. I think this is bunk. It's been ported to a bunch of CPU archs and operating systems. It's probably runs on only slightly fewer platforms than Mono itself.

Wines APIs are unstable.

See above. I think Peter is right, our exported APIs are not that large, and making them stable is not that hard either. They don't even have to be hard-iron set-in-stone will never ever ever change stable, simply doing our best would be a big improvement over what we have now. The flip side of this is that nobody is forcing you to use the latest version of Wine, or even dynamically link to it in the first place, you could easily just import snapshots into Mono CVS.

People find it too hard to setup.

Well, I've been flicking through the June archives of mono-winforms-list and I'm not seeing this at all ... yes there are lots of people saying "I wrote this app and it doesn't work!" but then SWF isn't complete and neither is Wine, so no big surprises there. The only setup related problems I could see are related to hardcoded paths being compiled into binaries - this is trivial to fix inside Wine and Mono, in fact I wrote a patch months ago to do this for Wine (so it can run from any prefix). It'd be easy to get this applied. The other problems I've seen are just random crashes which could be in the SWF code or Wine, I don't know which.

Wine really is not hard to set up. You just install it, same as any regular library. Yes, you can then not do mv /opt/wine1 /opt/wine2 and have it run, but that's like a 10 minute fix as I already did 99% of the work.

GDI+/Wine interactions are too slow.

I suspect this isn't a problem for Microsoft because they implemented GDI+ in native code then bound it to .NET. You guys are trying to implement GDI+ in .NET and then make Wine use it, which is a suboptimal solution.

I think if you had done it the way Microsoft did by writing a GDI+ DLL for Wine and then mapped that to System.Drawing, there would not need to be this constant mapping into System.Drawing contexs and things would go a lot faster. Yes, that means implementing GDI+ in Win32/C, not much fun I know, but it's the way Microsoft did it and therefore if we want compatibility the way we must do it too (Wine will have to implement GDI+ itself at some point anyway).

All our problems will go away if we implement S.W.F in C#

I strongly disagree with this notion. Nobody sane implements a production widget toolkit unless they have to, they are HUGE, win32 especially so because it's so old. The listview control alone has a bazillion different modes. I think Dimi estimated it'd cost $145,000 to develop at current market rates. Even worse, you can't implement SWF in managed code and get correct compatibility without reimplementing most of Wine in managed code as well, which would be even sillier.

The plan of doing an emulated WndProc in C# with the most common case is just going to lead to an endless stream of people trying to run apps and reporting "blah blah message isn't implemented", or worse "blah app has native code and passes it Control.Handle". I'm very skeptical you can use a "winelib plugin" to fix that - how can this work when the HWND the native code is trying to use is fake? Oh, don't forget details like handles being valid in any process! Programs that use OLE embedding between themselves +depend on that fact.

It especially makes me sad to see this because eventually people are going to start coming to us with hybrid .NET/Win32 apps and want to run them via Wine or Mono. They won't run on Mono because they'll P/Invoke to native GUI code to do stuff that .NET/SWF can't do, and they won't run on Wine because we don't implement the .NET framework.

I really think we should make another attempt to work together on this one - last time Paul raised the issue of wine_init changing its prototype (which caused them pain), Alexandre actually reverted the change. Too late in some respects, the damage was already done, but I think it's wrong to say we aren't interested in working with you. The last time Mono even had a presence on this list, it was asking how to open Wine as a library and example code was written for you. None of the other concerns were ever raised here, as far as I remember.

Well, you seem to have made up your mind, but I just have this horrible sinking feeling that one day WineHQ is going to have to do our own implementation of S.W.F in order to be able to run apps of the complexity people demand. Half baked solutions along the lines of "sorry using Control.Handle/the registry/whatever isn't portable" won't cut it when this is the last app blocking a migration to free software but they can't/won't rewrite it.

Aric and I have been working on a replacement for MSI.DLL that would eliminate the need to download the Windows installer distributable to install .msi files.

We have made quite a bit of progress on this in the last couple of months, but the work is not yet complete. After I finish flushing out the work we've done, it should be possible to install some simple .msi files without any Windows DLLs at all.

We are still missing an implementation of msiexec, but it is a fairly trivial wrapper around msi.dll, so should not be too hard to implement.

The native msi dlls (ie. the MS redistributables) have worked in Wine for more than two years :) They contain a few annoying bugs which are hard to workaround...

Right now, Wine doesn't work at all on FreeBSD -STABLE:

wine: failed to initialize: /swtest/wine/dlls/ntdll.dll.so: mmap of entire address space failed: Cannot allocate memory

and before that I used to see deadlocks upon startup of non-trivial applications (such as Forte Agent, both 16bit and 32bit flavors).

I believe there are also signficant threadings issues on -CURRENT, so overall Wine is hardly, if at all, usable on any version of FreeBSD I have access to, even though I'm still working to keep it at least compilable on FreeBSD 4.9 and 5.2/5.3.

>From what I can see, there are two problems with the Wine build from CVS on FreeBSD -current that seem to be related to the use of BSD make. In the dlls 'setupapi' and 'version' directories there are source files called install.c. These confuse the build during a "make install". BSD make tries to compile and link a program called "install". Renaming the source files to something other than install.c (such as winstall.c) allows the build to succeed.

At run time, the error:

"wine: failed to initialize: /something/lib/wine/ntdll.dll.so: mmap of entire address space failed: Cannot allocate memory"

is caused by Wine attempting to mmap memory outside the user process address space. I see mmap addr around 0xd8100000 (mostly 0xd81eccd8) whereas the user address space limit on FreeBSD current is 0xbfc00000 (at least on my system).

This failure is well before any thread library issues are encountered.

I ran a test of mmap on FreeBSD current to see what address space I was able to mmap. Using objdump to identify the pages that the test program was linked to load in, the test program was able to mmap MAP_ANON, MAP_FIXED, MAP_NOCORE all memory from 0x0 to 0xbfc00000, except the pages at which the test program was mapped at.

I'll look further into how Wine is mmap'ing memory on FreeBSD.

>From what I can see, the FreeBSD mmap address allocation behaviour that I see isn't documented. From the documentation, I would have expected the Wine code to work - it doesn't seem unreasonable to me.

I think the FreeBSD kernel code needs to change. If this is to happen, it will only be in the FreeBSD5 tree. It is too late in the FreeBSD4 branch to make that sort of change since it's not really a bug fix.

To get Wine to work on FreeBSD4, there needs to be a way of making the reservation code optional. A simple mmap test in configure which snaffles memory above 0x80000000 and then tries to mmap some more memory without specifying a fixed address would detect if mmap is behaving in a way that would allow the Wine reservation code to function.

For FreeBSD5, which will become the stable branch sometime soon, I think the kernel code needs to change. I have a FreeBSD src commit bit, but I'm not a vm person, so I can only prototype a change and submit it for review. I'm not sure if the other developers will regard this change favourably, though. They may take the attitude that if Wine can be made to work with the FreeBSD kernel code as is, then Wine should be coded accordingly.

If the Wine code was restructured to make the reservation code optional, that would cover both FreeBSD4 and FreeBSD5. Then, if the FreeBSD mmap algorithm was to change in the future, the build could start using the reservation code at that time.

I believe we have a licensing issue with the crypto implementation in Wine and would like feedback relating to this. I know that some of the OpenSSL developers monitor this list so please provide feedback.

The first question is in regards to the following:

Clause 3 of OpenSSL license:

3. All advertising materials mentioning features or use of this software must display the following acknowledgment: "This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/)"

This stipulation seems to introduce a incompatibility with the GPL. For normal Wine this is not a big deal but if someone wants to make a Winelib application that is GPL or if I want to take Wines Wininet and Adavapi32 code in to ReactOS this presents a problem. Am I correct in reading it this way?

If I am correct and it is incompatible with with GPL then would any of the Wine developers object to changing out crypto implementation to use NSS from Mozilla? I have spoken with the developers of Network Security Services for Mozilla and it has recently been tri-licensed MPL/LGPL/GPL on CVS tip. http://www.mozilla.org/projects/security/pki/nss/

>From the what I understand this lib provides everything we need so all it really means is that our soft dependency on OpenSSL in now a soft dependency on Mozilla. If we want to use the webbrowser module in Wine then a Mozilla dependency will be there already so we might as well use the security implementation while we are at it.

Pango uses fribidi for it's bidi. As fribidi doesn't do shaping, I'm not sure where pango's shaping comes from. It may be an add-on.

In any case, I'm hoping that if/when fribidi starts to support UTF-16, pango will follow. Last time I looked at it, we were too far off from needing what pango had to offer. Getting a bidi edit control (sigh) is much higher on my todo (for which I have no time :-( )

The really sad thing here is that ICU really has everything we need. It's just that it's too simple for people to compile wine without it, that is the problem.

Just to sum up the IRC discussion with Mike:

1. I use ICU because it's the best we have at the moment. That doesn't revoke my right to hate it. Until I have an alternative (or someone else steps forward to do BiDi on Wine), that's what we are using.
2. Eat the strawberries.
3. Never tell amusing stories around geeks. They will either guess the punchline or find logic errors in the story.