Wine Traffic #259 For 28 Jan 2005 By Brian Vincent Table Of Contents * Standard Format * Text Format * XML Source * Introduction * Mailing List Stats For This Week * Threads Covered 1. 22 Jan 2005 - 28 Jan 2005 (1 post) News: IBM & Wine 2. 27 Jan 2005 (11 posts) Wine & Viruses 3. 19 Jan 2005 - 28 Jan 2005 (35 posts) Wiki's, AppDB, and Other Forums 4. 22 Jan 2005 - 23 Jan 2005 (2 posts) WineConf 2005: More Details 5. 22 Jan 2005 - 23 Jan 2005 (3 posts) More DCOM Notes 6. 23 Jan 2005 (3 posts) Web Based Presentations Introduction This is the 259th issue of the Wine Weekly News publication. Its main goal is to go to Vegas. It also serves to inform you of what's going on around Wine. Wine is an open source implementation of the Windows API on top of X and Unix. Think of it as a Windows compatibility layer. Wine does not require Microsoft Windows, as it is a completely alternative implementation consisting of 100% Microsoft-free code, but it can optionally use native system DLLs if they are available. You can find more info at www.winehq.org (http://www.winehq.org) Mailing List Stats For This Week We looked at 291 posts in 1083K. There were 88 different contributors. 46 posted more than once. 38 posted last week too. The top posters of the week were: * 19 posts in 68K by Mike Hearn * 15 posts in 38K by Alexandre Julliard * 14 posts in 57K by Francois Gouget * 13 posts in 63K by Tony Lambregts * 13 posts in 43K by Robert Shearman * Full Stats 1. News: IBM & Wine 22 Jan 2005 - 28 Jan 2005 (1 post) Archive Link: "News" Topics: News People: Techworld, , News A couple of stories came out this week focused on IBM's direction concerning Linux on the desktop. Specifically, Wine was mentioned in both and questioned IBM's internal use of it. Techworld wrote (http://www.techworld.com/opsys/ features/index.cfm?FeatureID=1151) : "Our chairman has challenged the IT organisation, and indeed all of IBM to move to a Linux based desktop by the end of 2005," Greenberg wrote in a November 2003 memo. "This means replacing productivity, Web access, and viewing tools with open standards based equivalents," he said. But doing it in an orderly fashion -- even assuming you could get buy-in from the individuals themselves, which in itself is a huge assumption -- is no overnight matter. And the fact IBM is still using Wine -- Windows emulation -- to run mission-critical applications such as Lotus Notes, its own product, is a demonstration of just how hard the process is. That article came on the heels of another one (http://www.techworld.com/opsys/ news/index.cfm?NewsID=3010) published on Techworld discussing the same topic. So IBM is having a hard time making a change? Didn't someone write a book about that once? (http://www.amazon.com/exec/obidos/tg/detail/-/0060523794/ 002-2141112-4428803?v=glance) 2. Wine & Viruses 27 Jan 2005 (11 posts) Archive Link: "PRESS: run windows viruses with wine ..." Topics: Security People: Marcus Meissner, Francois Gouget, David Hagood, Mike Hearn, , Hiji Marcus Meissner provided a link that also found it's way to Slashdot: Fun article ... author tests various viruses with WINE. http://os.newsforge.com/article.pl?sid=05/01/25/1430222&from=rss (http:// os.newsforge.com/article.pl?sid=05/01/25/1430222&from=rss) This started off a discussion about Wine and the dangers of being too much like Windows. As the article clearly shows, running viruses under Linux is pretty difficult - you definitely have to go out of your way to try to get one to run. I'll very briefly summarize some of the fears people had and why they don't necessarily apply to Wine. First, what happens if you have a real Windows partition and a virus searches throughout your system to embed itself somewhere? Francois Gouget pointed out: But under Linux if you mount a fat32 partition with all defaults then only root can write to it. If you want other users to have write to it, then you have to use the appropriate mount options to set the umask, user and group just right. Oh, and in case you missed all the warnings and documentation, don't run Wine as root. As long as you're not root, Wine only has access to the things you as a user have access to. Since a normal user can't erase a hard drive, a virus running under Wine isn't going to be able to either. However, Hiji pointed out this is still a real problem for people using Sun's JDS since currently mounts Windows partitions as read/write for regular users. Okay, so say a virus does manage to insert itself on your system. Say you've also gone a step further and use something like the Linux kernel's binfmt to automatically execute Windows programs. David Hagood pointed out that it's likely still not an issue: Yes, but then the kernel will only execute the file IF it has execute permissions - so when the worm drops BackOriface.exe on your drive and tries to run it, it won't as it won't have had the +x bit set. And a worm smart enough to realize it is running under Wine and able to make the syscall to set the +x bit probably will be smart enough to get a native executable for the infection. Regarding other potential problems, Mike Hearn warned: The biggest potential entrypoint for viruses in Wine is probably Internet Explorer. People who use that should know the risks involved, and it's massively unlikely anybody would use IE as their default browser on Linux. Fortunately, someone told me the other day we have a pretty good browser for Linux (http://www.mozilla.org) . 3. Wiki's, AppDB, and Other Forums 19 Jan 2005 - 28 Jan 2005 (35 posts) Archive Link: "Wine Knowledge Base" Topics: Project Management People: Mike Hearn, Dimitrie Paun, Chris Morgan, Tony Lambregts, , WineHQ, Dimi Paun Last week we mentioned a new wiki that had been set up (see WWN issue #258 ( ) ). Mike Hearn dropped an email to let everyone know of something similar he'd set up: I was going to wait a while (until it had a bit more content) before announcing this here, but as it seems people are setting up Wikis etc already here is my 2 cents: http://navi.cx/svn/misc/trunk/winekb/index.xml ** THIS IS NOT A WIKI ** Instead, it's a mini website that uses a simple dialect of XML suitable for embedding code and random text into. It's designed to be: * Easy to improve * Visually appealing * Controlled What do I mean by these things? Firstly, easy to improve: if you want to add some content to a page, save or wget the file to disk, edit it in your favourite text editor, and then send me the diff (or the full file and I'll diff it for you). You don't need any web design skill, as you can see if you do a "View Source" in the page. Secondly, visually appealing: I dislike the way Wikis look most of the time. XSL:T is a nice language for doing tree transformations so I used it. The page code looks like this:
* Installing NativeMSI and NativeDCOM * The ConfigFile * #winehq channel faq - read this when you have an issue
* AddingStuff to this site * InterestingPatches * WinelibAsBinaries
See? It's easy. Yet the resulting pages looks quite nice IMHO and are uniform in style. There's a new page template here: http://navi.cx/svn/misc/trunk/winekb/template.xml Thirdly, controlled: one of the objections Newman always had to a Wiki was that it'd fill up with crap or become another appdb, and then he'd have to maintain it. You can contribute to the knowledge base by sending patches to me or to wine-devel (if you think others would find the text useful), same as any other project. I won't be adding anything there that duplicates the appdb except maybe for extremely popular programs and libraries like MSI and DCOM. You can hack on it in other ways: the code for the site is managed using Subversion/SVK, so you can check out a copy of it like so: $ svn co http://navi.cx/svn/misc/trunk/winekb If you want to do a lot of work on it, use SVK which will let you branch changes and commit locally so you can keep your patches separated nicely. Right now there isn't much content. There's some stuff for users, like how to set up a basic config file, how to install native DCOM etc, and there is stuff for developers like a list of interesting-yet-bounced patches and some notes on the "Winelib programs as binaries" work me and Vincent did. It's designed for content that doesn't really fit the main WineHQ website. If I think something makes more sense in the developer guide I'll ask you to send the content there. In other words, the KB is designed to be half-way between the structured and professional WineHQ website and a Wiki. More discussion went on about this site and the wiki announced last week. To give a little history, Wine used to have something called the "FAQ-O-Matic" that allowed people to update it in a way very similar to a wiki. The problem is, no one ever updated it except for a couple of developers. A few years ago the FAQ-O-Matic was replaced with a static FAQ (http://www.winehq.com/site/docs /wine-faq/index) page. It definitely seems to be more useful than the old system. Similarly, Wine's AppDB (http://appdb.winehq.org) contains a ton of information about different applications. Unfortunately, one of the problems the AppDB developers are running into is the sheer amount of junk that's collected in there. (On a side note, the AppDB has seen some remarkable improvements over the past few months. If you haven't tried it out in a while, head over and sign up to be a maintainer for your favorite app.) So all that taken together makes everyone apprehensive about having yet another forum that has to be maintained. On the flipside, different people like different kinds of forums, so finding a one-size-fits-all method for doling out information probably isn't realistic. Dimi Paun took a look at Mike's stuff and replied to Mike's bullet points: Easy to improve It is easy, but not as easy a regular Wiki. By a long shot. Moreover a Wiki like MoinMoin is so common on the web that people know it, and they don't have to lear yet another markup format. And you can not compare the click-to-edit--edit--click-to-save ease of using a Wiki with the procedure to change this one (mentioning svk, diff, mail, etc, and you've lost 95% of people, right there). Visually appealing This is in the eye of the beholder. I am sorry, but MoinMoin doesn't look at all that bad that it's a reason to reinvent the wheel. Besides, a real Wiki has nice features (email notifications, history, etc) that we are all missing with a hand-rolled one. Controlled This has been a non-argument from the beginning. Jer has been put off by the silly FAQ-O-Matic, but that wasn't really a Wiki. There are so many Wikis on the web that work just fine and are useful (see the Arch Wiki, and so many others) that this has no support whatsoever. In fact, the lack of control (and the associated ease of editting) are the very reason to use Wikis, and we are giving these up for some irrational fears. Let's add a Wiki to the main site (MoinMoin in particular), and see what happens. If it doesn't work out, we'll take it down. I'm sure people will love it. Quite a bit of discussion went on to involve details of how the AppDB should function. Several people went back and forth on how apps should be rated and the value of even having a rating system. From there it changed into how testing could actually be done in a useful way. Tony Lambregts thought it would be nice if Alexandre could give a warning a few days before doing a release. The problem with that approach is there's no guarantee application maintainers will actually test an app or that the testing done will be comprehensive enough to catch regressions. A few details did seem to get sorted out for the AppDB hackers to work on. Thanks to Chris Morgan, Tony Lambregts, Jonathan Ernst, Paul van Schayck, and Stefan Pfl?ger for the work they've done. 4. WineConf 2005: More Details 22 Jan 2005 - 23 Jan 2005 (2 posts) Archive Link: "WineConf 2005: hotel info" Topics: WineConf 2005 People: Brian Vincent, Jeremy White, CodeWeavers, , WineHQ Plans for WineConf 2005 are coming together. I posted details on lodging this week: Earlier I added hotel info to the WineConf pages: http://www.winehq.com/site/ wineconf/travel (http://www.winehq.com/site/wineconf/travel) Here's what you need to know: * we have 50 rooms reserved for Fri and Sat night, we have some on Thursday and Monday night too. I'll drink bier with anyone arriving early or staying late. * 81 EUR for 2 nights * you have to send mail to the address on the web page to reserve a room, note the "keyword" * first come, first served, reservations must be made by March 29th * did I mention RSVP'ing on WineHQ (http://www.winehq.com/site/wineconf/rsvp) as well? Thanks goes out to Hans-Ulrich Schmid (WRS) and Ulrich Gemkow (University of Stuttgart) for getting together and figuring this out. Jeremy White followed up by encouraging everyone to please register: Okay, we've got 19 rsvp's for Wineconf. And that's missing me and a lot of other people that I know will be there, so shame on me and shame on you. So get over to www.winehq.org/wineconf and rsvp (including fellow folks from CodeWeavers; if you're reading this, you better plan on being there ). In all seriousness, we really need an accurate head count so we can plan properly; if there really only will be 25-30 people, that makes things a good bit different than 75 people. While you're there, join the wine-conf mailing list; I think that's the most appropriate place for us to start the discussions of the agenda and to start fine tuning the details. So if you don't subscribe to wine-conf, but you're planning to go, you're going to start missing out on everything but the nag messages If you're undecided, or if you're unsure the topics will interest you, I'd highly encourage you to show up. It should be a lot of fun. 5. More DCOM Notes 22 Jan 2005 - 23 Jan 2005 (3 posts) Archive Link: "More DCOM notes" Topics: RPC / COM / OLE People: Mike Hearn, , Microsoft, Unknown, Rob Shearman Mike Hearn put more notes together about the inner workings of DCOM. Rob Shearman replied with some small corrections, so I added those in italics This document assumes you are familiar with the basics of DCOM. If you aren't read this first: http://winehq.com/site/docs/wine-devel/dcom-1 (http://winehq.com/site/docs/ wine-devel/dcom-1) This is not suitable study material for beginners. Don't say I didn't warn you. Apartments Before a thread can use COM it must enter an apartment. Apartments are an abstraction of a COM objects thread safety level. There are many types of apartment but the only two we care about right now are single threaded apartments (STAs) and the multi-threaded apartment (MTA). Any given process may contain at most one MTA and potentially many STAs. This is because all objects in MTAs never care where they are invoked from and hence can all be treated the same. Since objects in STAs do care, they cannot be treated the same. You enter an apartment by calling CoInitializeEx and passing the desired thread model in as a parameter. The default if you use the deprecated CoInitialize is a STA, and this is the most common type of apartment used in COM. An object in the multi-threaded apartment may be accessed concurrently by multiple threads: eg, it's supposed to be entirely thread safe. It must also not care about thread-affinity, the object should react the same way no matter which thread is calling it. An object inside a STA does not have to be thread safe, and all calls upon it should come from the same thread - the thread that entered the apartment in the first place. The apartment system was originally designed to deal with the disparity between the Windows NT/C++ world in which threading was given a strong emphasis, and the Visual Basic world in which threading was barely supported and even if it had been fully supported most developers would not have used it. Visual Basic code is not truly multi-threaded, instead if you start a new thread you get an entirely new VM, with separate sets of global variables. Changes made in one thread do NOT reflect in another, which pretty much violates the expected semantics of multi- threading entirely but this is Visual Basic, so what did you expect? If you access a VB object concurrently from multiple threads, behind the scenes each VM runs in a STA and the calls are marshaled between the threads using DCOM. In the Windows 2000 release of COM, several new types of apartment were added, the most important of which are RTAs (the rental threaded apartment) in which concurrent access are serialised by COM using an apartment-wide lock but thread affinity is not guaranteed. Do we have any details about how you use RTAs? I have a feeling this was something that was due to be implemented for Windows 2000, but was dropped before it shipped. Structure of a marshaled interface pointer When an interface is marshaled using CoMarshalInterface, the result is a serialized OBJREF structure. An OBJREF actually contains a union, but we'll be assuming the variant that embeds a STDOBJREF here which is what's used by the system provided standard marshaling. A STDOBJREF (standard object reference) consists of the magic signature 'MEOW', then some flags, then the IID of the marshaled interface. Quite what MEOW stands for is a mystery, but it's definitely not "Microsoft Extended Object Wire". Next comes the STDOBJREF flags, identified by their SORF_ prefix. Most of these are reserved, and their purpose (if any) is unknown, but a few are defined. After the SORF flags comes a count of the references represented by this marshaled interface. Typically this will 1 in the case of a normal marshal, but may be 5 for table-strong marshals and 0 for table-weak marshals (the difference between these is explained below). This is wrong. It is 5 for normal marshals, and 0 for both table-weak and table-strong. The reasoning is this: In the general case, we want to know exactly when an object is unmarshaled and released, so we can accurately control the lifetime of the stub object. This is what happens when cPublicRefs is zero. However, in many cases, we only want to unmarshal an object once. Therefore, if we strengthen the rules to say when marshaling that we will only unmarshal once, then we no longer have to know when it is unmarshaled. Therefore, we can give out an arbitrary number of references when marshaling and basically say "don't call me, except when you die." The most interesting part of a STDOBJREF is the OXID, OID, IPID triple. This triple identifies any given marshaled interface pointer in the network. OXIDs are apartment identifiers, and are supposed to be unique network-wide. How this is guaranteed is currently unknown: the original algorithm Windows used was something like the current UNIX time and a local counter. OXIDs are generated and registered with the OXID resolver by performing local RPCs to the RPC subsystem (rpcss.exe). In a fully security-patched Windows system they appear to be randomly generated. This registration is done using the ILocalOxidResolver interface, however the exact structure of this interface is currently unknown. OIDs are object identifiers, and identify a stub manager. The stub manager manages interface stubs. For each exported COM object there are multiple interfaces and therefore multiple interface stubs (IRpcStubBuffer implementations). OIDs are apartment scoped. Each ifstub is identified by an IPID, which identifies a marshaled interface pointer. IPIDs are apartment scoped. Unmarshaling one of these streams therefore means setting up a connection to the object exporter (the apartment holding the marshaled interface pointer) and being able to send RPCs to the right ifstub. Each apartment has its own RPC endpoint and calls can be routed to the correct interface pointer by embedding the IPID into the call using RpcBindingSetObject. IRemUnknown, discussed below, uses a reserved IPID. Only in the implementation I've written and am currently testing now. The native version generates an IPID as per any other object and simply notifies the SCM of this IPID. Both standard and handler marshaled OBJREFs contains an OXID resolver endpoint which is an RPC string binding in a DUALSTRINGARRAY. This is necessary because an OXID alone is not enough to contact the host, as it doesn't contain any network address data. Instead, the combination of the remote OXID resolver RPC endpoint and the OXID itself are passed to the local OXID resolver. It then returns the apartment string binding. This step is an optimisation: technically the OBJREF itself could contain the string binding of the apartment endpoint and the OXID resolver could be bypassed, but by using this DCOM can optimise out a server round-trip by having the local OXID resolver cache the query results. The OXID resolver is a service in the RPC subsystem (rpcss.exe) which implements a raw (non object-oriented) RPC interface called IOXIDResolver. Despite the identical naming convention this is not a COM interface. Unmarshaling an interface pointer stream therefore consists of reading the OXID, OID and IPID from the STDOBJREF, then reading one or more RPC string bindings for the remote OXID resolver. Then RpcBindingFromStringBinding is used to convert this remote string binding into an RPC binding handle which can be passed to the local IOXIDResolver::ResolveOxid implementation along with the OXID. The local OXID resolver consults its list of same-machine OXIDs, then its cache of remote OXIDs, and if not found does an RPC to the remote OXID resolver using the binding handle passed in earlier. The result of the query is stored for future reference in the cache, and finally the unmarshaling application gets back the apartment string binding, the IPID of that apartments IRemUnknown implementation, and a security hint (let's ignore this for now). Once the remote apartments string binding has been located the unmarshalling process constructs an RPC Channel Buffer implementation with the connection handle and the IPID of the needed interface, loads and constructs the IRpcProxyBuffer implementation for that IID and connects it to the channel. Finally the proxy is passed back to the application. Handling IUnknown There are some subtleties here with respect to IUnknown. IUnknown itself is never marshaled directly: instead a version of it optimised for network usage is used. IRemUnknown is similar in concept to IUnknown except that it allows you to add and release arbitrary numbers of references at once, and it also allows you to query for multiple interfaces at once. IRemUnknown is used for lifecycle management, and for marshaling new interfaces on an object back to the client. Its definition can be seen in dcom.idl - basically the IRemUnknown::RemQueryInterface method takes an IPID and a list of IIDs, then returns STDOBJREFs of each new marshaled interface pointer. There is one IRemUnknown implementation per apartment, not per stub manager as you might expect. This is OK because IPIDs are apartment not object scoped. In fact, according to the DCOM draft spec, they are machine-scoped, but this implies apartment-scoped. Table marshaling Normally once you have unmarshaled a marshaled interface pointer that stream is dead, you can't unmarshal it again. Sometimes this isn't what you want. In this case, table marshaling can be used. There are two types: strong and weak. In table-strong marshaling, selected by a specific flag to CoMarshalInterface, a stream can be unmarshaled as many times as you like. Even if all the proxies are released, the marshaled object reference is still valid. Effectively the stream itself holds a ref on the object. To release the object entirely so its server can shut down, you must use CoReleaseMarshalData on the stream. In table-weak marshaling the stream can be unmarshaled many times, however the stream does not hold a ref. If you unmarshal the stream twice, once those two proxies have been released remote object will also be released. Attempting to unmarshal the stream at this point will yield CO_E_DISCONNECTED. RPC dispatch Exactly how RPC dispatch occurs depends on whether the exported object is in a STA or the MTA. If it's in the MTA then all is simple: the RPC dispatch thread can temporarily enter the MTA, perform the remote call, and then leave it again. If it's in a STA things get more complex, because of the requirement that only one thread can ever access the object. Instead, when entering a STA a hidden window is created implicitly by COM, and the user must manually pump the message loop in order to service incoming RPCs. The RPC dispatch thread performs the context switch into the STA by sending a message to the apartments window, which then proceeds to invoke the remote call in the right thread. RPC dispatch threads are pooled by the RPC runtime. When an incoming RPC needs to be serviced, a thread is pulled from the pool and invokes the call. The main RPC thread then goes back to listening for new calls. It's quite likely for objects in the MTA to therefore be servicing more than one call at once. Message filtering and re-entrancy When an outgoing call is made from a STA, it's possible that the remote server will re-enter the client, for instance to perform a callback. Because of this potential re-entrancy, when waiting for the reply to an RPC made inside a STA, COM will pump the message loop. That's because while this thread is blocked, the incoming callback will be dispatched by a thread from the RPC dispatch pool, so it must be processing messages. While COM is pumping the message loop, all incoming messages from the operating system are filtered through one or more message filters. These filters are themselves COM objects which can choose to discard, hold or forward window messages. The default message filter drops all input messages and forwards the rest. This is so that if the user chooses a menu option which triggers an RPC, they then cannot choose that menu option *again* and restart the function from the beginning. That type of unexpected re-entrancy is extremely difficult to debug, so it's disallowed. Unfortunately other window messages are allowed through, meaning that it's possible your UI will be required to repaint itself during an outgoing RPC. This makes programming with STAs more complex than it may appear, as you must be prepared to run all kinds of code any time an outgoing call is made. In turn this breaks the idea that COM should abstract object location from the programmer, because an object that was originally free-threaded and is then run from a STA could trigger new and untested codepaths in a program. Oh well, it was nice in theory. 6. Web Based Presentations 23 Jan 2005 (3 posts) Archive Link: "Simple web based presentation system useful for WineConf" Topics: WineConf 2005 People: Mike Hearn, Mike Hearn came up with a really simple framework for doing web-based presentations. It's something that isn't Wine specific and I bet a lot of you could use something like this. It uses a combination of XSL and XML and looks great in Mozilla: With the spring comes WineConf 2005, and perhaps some interesting presentations by people. Last time I went to WineConf I was asked to write a mini presentation to show newbies what Wine hacking was all about. In the end, there weren't many non-Wine developers there (which is what you'd expect, really) so we just had a discussion about how to reduce the learning curve instead. In retrospect that was far more interesting and useful than any presentation I could have done. Nonetheless, that presentation was there and when I wrote it, I also wrote a little XSL transform for slide shows. That's because I don't like PowerPoint that much, especially I don't like how sucky its web export is and slideshows are good to put on the web. So here is the presentation itself: http://bylands.dur.ac.uk/~mh/wineconf2004/ presentation.xml (http://bylands.dur.ac.uk/~mh/wineconf2004/presentation.xml) Just load it into a web browser like Mozilla or even IE6 (konq doesn't support xslt, sorry). As you can see it's nicely Wine themed :) Here is the transform if you want to use the same presentation system yourself: http://navi.cx/~mike/xslt-slides/gen.xsl (http://navi.cx/~mike/xslt-slides/ gen.xsl) Just look at the source code of presentation.xml to see how to use it. I'm hoping we'll have a similar mix for this years Wineconf as last year - Lionels OpenGL/games presentation in particular was interesting, but I think the mix of discussion and presentations was good. One problem with presentations is that they tend to be in quite specialist areas, eg a DCOM presentation wouldn't be all that useful because there aren't many people working on it. So I think a bias more towards discussion of process and project organisation, things like that, would be more useful. That's just my humble opinion of course ;) Sharon And Joy Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.