Wine Traffic #236 For 20�Aug�2004

By Brian Vincent

Table Of Contents

Introduction

This is the 236th issue of the Wine Weekly News publication. Its main goal is to have surgery. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org (http://www.winehq.org)

Mailing List Stats For This Week

We looked at 180 posts in 578K.

There were 49 different contributors. 34 posted more than once. 22 posted last week too.

The top posters of the week were:

1. News: Wine-20040813, Website Updates

14�Aug�2004�-�20�Aug�2004 (2 posts) Archive Link: "News"

Topics: News

People: Alexandre Julliard,�WineHQ,�eWeek,�eweek,�CodeWeavers,�,�Newsforge,�Tom Wickline,�News,�Mike McCormack,�Dimi Paun

Just after last week's WWN came out, Alexandre released Wine-20040813:

WHAT'S NEW with Wine-20040813: (see ChangeLog (http://cvs.winehq.com/cvsweb/wine/ChangeLog?rev=1.86&content-type=text/x-cvsweb-markup) for details)

As noted on Wednesday, the WineHQ website has seen some recent updates:

Tom Wickline updated the status pages (http://www.winehq.com/site/status) , including the addition of builtin versions atl.dll, d3dxof.dll, dbghelp.dll, and mlang.dll. Our janitorial projects (http://www.winehq.com/site/janitorial) received a bit of their own janitorial clean up courtesy of Dimi Paun. They also saw the addition of -Wsign-compare as a useful gcc warning to turn on (suggested by Mike McCormack.) Finally, over on the downloads (http://www.winehq.com/site/download) page we've added Fedora Core 2 and White Box Enterprise Linux targets as part of the Red Hat packages maintained by Vincent B�ron.

Also on Wednesday, Newsforge put out a review (http://software.newsforge.com/software/04/08/18/068255.shtml) of CrossOver Office 3.0.1. The author, Jem Matzan, did an excellent job describing how Wine works and even going so far as to add a nice sidebar about the project. I thought he was a bit too critical of CodeWeavers' licensing, but otherwise it was a great review. You can find another blurb (http://www.eweek.com/article2/0,1759,1631278,00.asp) about CrossOver Office on eWeek from August 9th.

2. Generating Backtraces

17�Aug�2004�-�20�Aug�2004 (13 posts) Archive Link: "Backtrace Dumps"

Topics: Debugging

People: Robert Shearman,�Alexandre Julliard,�Dimitrie Paun,�Andreas Mohr,�Mike Hearn,�,�Rob Shearman,�Dimi Paun

Rob Shearman put together a new way to debug programs and submitted a patch with the following description:

I propose the addition of this new debug tool that will allow developers to output backtraces to the console, without having to interact with a debugger. It can give a useful indication of how functions are being called in situations where a break point is not appropriate. To use it, simply include "wine/backtrace.h" somewhere and then call dump_backtrace(), which will generate an output similar to the following:

Backtrace:

Alexandre didn't think that was the best approach, " I think it's better to let the debugger take care of that. If you don't want a real breakpoint you could define a custom exception to tell winedbg to just dump the backtrace and continue. "

Several people, however, liked the idea of just getting a simple backtrace. Dimi Paun explained:

I am not 100% how the patch that Robert's proposing would work in practice, but I can tell you (from working with Java for a long time now) that having readily available backtraces is invaluable.

I for one love backtraces, but on the other hand I don't much care for debuggers. Having access to them without being forced to go through the debugger would be much appreciated.

Andi Mohr echoed those sentiments:

Ditto.

Wine has always had massive problems with getting easily accessible debugging/diag support (doing debugging stuff IS hard, no question here...). Thus I think we generally want not less, but more support, as long as it helps (and as long as it doesn't cause a maintenance headache or ill-maintained subsystems due to too much "choice" in debugging options...). Since I'd think that that backtrace code should be fairly small and self-maintaining, I'd vote for including it.

Mike Hearn expressed concern about using the debugger:

In Java you can say this:

or some equivalent to get a printout of where you are, and it's very convenient.

I can see the reasoning behind keeping the code in the debugger then triggering it using a custom exception though, as that way we only have one piece of backtracing code.

My problem with this approach is that it relies on the exception actually getting through to the debugger instead of being trapped by the program code and swallowed. I guess we could install a vectored handler to boot the debugger and such but now the code is a lot more complex and confusing for newbies than just having some inline functions in the headers. As if the SEH code wasn't already confusing enough!

Andi agreed:

I'm sure every semi-involved Wine developer can imagine dozens of "reasons of the day" why winedbg doesn't launch properly on error again... Failure in wine exception handling code, failure to look up winedbg (both registry and disk), failure to pass winedbg cmdline parameters properly, failure to get winedbg started up properly, failure to get winedbg to parse the current modules and stack frame properly, ...

That's why a "no frills" debugging mechanism is a good idea IMHO.

P.S.: no offense to Eric. He's done TONS of very useful things to winedbg, and when considering how many fatal architecture changes winedbg had to go through, it's amazing that it still works pretty well. :-)

Alexandre still wasn't convinced and felt the right approach was to go with the full debugger on the backend, " Well, I can also imagine a lot of cases where the in-process backtrace won't work right, the main one being that since the code will never be used unless you want a backtrace, when you try to use it you'll realize that it no longer works, or doesn't even compile anymore. If we call the debugger instead, it will use the standard backtrace code which is always compiled in, and regularly used by a lot of people."

Rob offered to write a unit test to make sure the code stays up to date, but Alexandre reiterated that the debugger was the way to go. He thought a custom exception could be generated to get the debugger to generate a backtrace. Mike pointed out a potential problem, " This isn't reliable though, what about apps that swallow all exceptions with a try/except block? Ideally you want this sort of thing to just work and not have to figure out why your backtrace is never happening. "

Alexandre wasn't swayed, " In that case you have to start the debugger first. Of course there will be cases where it won't work as well as you'd like, you'll have the same with the in-process solution, there are many cases where it won't work. And actually the debugger solution has more chances to work reliably since being in a separate process it interferes much less with the running application."

3. Fun Project: Port Allegiance

18�Aug�2004 (1 post) Archive Link: "New Fun Project - Winelib port of Allegiance"

Topics: Winelib

People: Steven Edwards,�,�Microsoft

Steven Edwards came up with a project for anyone wanting to tackle porting a large codebase:

Microsoft Research has released a Shared Source version of the game Allegiance. This is a hefty 512 meg download and of course the license that it is under would prevent anyone from sharing the changes but this would really allow us to improve Winelib by porting such a massive game.

Grab your copy quick before they pull it down.

This could be an interesting project for a CS student if you can figure out a way to get credit for it.. independent study, perhaps?

4. poll vs. epoll

19�Aug�2004�-�20�Aug�2004 (5 posts) Archive Link: "poll vs. epoll"

Topics: IO, Architecture

People: Linux Weekly News,�Shachar Shemesh,�Mike Hearn,�Dan Kegel,�,�News

Sitting around waiting for a program to do something shouldn't be a very intensive task for an operating system. Yet, some programs are designed in such a way that it is a problem. Linux Weekly News described an example a few years ago:

The classic Unix way to wait for I/O events on multiple file descriptors is with the select() and poll() system calls. When a process invokes one of those calls, the kernel goes through the list of interesting file descriptors, checks to see if non-blocking I/O is available on any of them, and adds the calling process to a wait queue for each file descriptor that would block. This implementation works reasonably well when the number of file descriptors is small. But if a process is managing thousands of file descriptors, the select() and poll() calls must check every single one of them, and add the calling process to thousands of wait queues. For every single call. Needless to say, this approach does not scale very well.

Their solution (http://lwn.net/Articles/14168/) involved switching over to a new set of system calls, epoll(). Shachar Shemesh wanted to get thoughts on improving Wine's performance in this area:

I have a program (a server) that has a very large number of synchronization constructs. Even when everything is idle there, wineserver is taking 60% of the CPU, raising load average to 2.4. When load is applied, response time occasionally jumps from tenths of a second to two minutes.

I noticed that in server/fd.c, the wineserver is using "poll" to select between file descriptors. The application is going through this code over 2000 times a second, with over 380 file descriptors each time. I am wondering whether this can be the cause of the slowdown.

One of the possible solutions to this would be to use epoll instead. This, of course, would have to be backed by a configure check, as not all systems for which wine is intended support epoll. Another, arguably better, solution would be to use libevent for this purpose. Libevent has the distinct advantage that it automatically chooses the best tool for the job (epoll, poll, /dev/poll, or if all else fails, select). However, if there is another, independent use of poll, porting the semantics to libevent may prove non-trivial.

So what does the forum think?

One last question. What are the "users"? What constructs cause a new file descriptor to be allocated in the wineserver?

Mike Hearn replied first with answers to Shachar's last few questions, " Every thread in every client has 3 fds: command, reply, wait. Other fds are allocated as well I think for things like open files, but I'm not sure of the details. I know that when I hit these sorts of problems it was due to the large number of threads blowing the fd limit (there were lots of sockets as well)."

Dan Kegel like Shachar's idea and offered some advice:

By all means, let's try epoll.

FWIW, I wrote a wrapper layer that illustrates how to detect whether epoll etc. are available. I'm convinced that runtime detection is the only way to go. Compile time detection is insufficient. My code is at http://kegel.com/rn If you actually try to use it, let me know. I expect it might be useful mostly as a guide. It's edge-trigger oriented, but you can trivially add the flag (or remove the flag) needed to get level-triggered behavior again if you really need it. I highly recommend using edge-triggered behavior, though.

Shachar wondered how well that would work, " I'm sure that edge trigger yields better performance. The problem, however, what with all the potential races edge trigger introduces, and the fact I'm not sufficiently familiar with the wineserver semantics, that going edge trigger in a bug free way right now is beyond me."

Dan explained:

"potential race conditions" is the wrong way to think about it, really.

Switching to edge triggered epoll from select() or poll() is always an interesting intellectual challenge; you're fundamentally switching to an idiom where the app has to remember more than it did before. Once you've done it a few times, it gets easier.

If I have time, I'll have a look at the state machine inside wineserver and see how tricky it'd be.

5. Threading Issues?

17�Aug�2004�-�20�Aug�2004 (2 posts) Archive Link: "Threading issues? [ck-request@vds.kolivas.org: ck Digest, Vol 3, Issue 16]"

Topics: Architecture

People: Andreas Mohr,�Con Kolivas,�Wiktor Grebla,�Mike Hearn,�

Andi Mohr forwarded a thread from another mailing and wondered if Wine suffered from the same problem:

this is a nice little posting from the Con Kolivas Linux interactivity patches list. Note that it is about Cedega, not Wine, but it might easily affect Wine, too.

Are we doing some big no-nos in the threading area here? (or maybe it's just done the way it is because we cannot do it a different, better way?)

For the whole discussion, please see

I didn't dig into the details too much, but the problem seemed to arise when Wiktor Grebla couldn't get threads to synchronize properly. Con hypothesized, " See if it's a priority issue on the part of wine*. Be absolutely certain that wineserver, wine and the game are all run the _same_ priority and not a better priority than X. Send me the output of 'top -b -n 1' during the sound being choppy. Then try running _all_ the wine things at nice +19. This is a simple sanity check to see where the problem lies. "

Wiktor tried that and reported:

It's in fact as you say, all parts of wine have different priorities, and some of them are higher than X

After renicing all of them there is no problem with choppy sound in all of time wasting programs I've on my disk :>, and overall (subjective) performance is better of course.

Con then alluded to a problem on the part of Wine (or, perhaps just Cedega), " Thanks for details. You do not have different nice levels; that is PRI that is different, which is the "dynamic" priority that constantly changes by the scheduler. It is as I suspected, though - if you renice everything +19 and the problem goes away it is bad programming on the wine developers' part that use what's called sched_yield instead of proper locking between their threads. Don't concern yourself over the details, but at least you have a simple workaround... sigh "

Mike Hearn seems to have discussed this with Alexandre, and neither of them could diagnose the problem based on what was described, " Well, apparently we don't use sched_yield, so the problem must lie somewhere else. Maybe Con can help us out here? Alexandre says he doesn't know what the issue is either and somebody needs to investigate. I guess we do need to concern ourselves over the details :)"

Sharon And Joy

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.