Wine Traffic #217 For 2 Apr 2004

Onebase provides fine integration of WINE software for easier and trouble-free installation and execution of Win32 applications through the "Onebase WINe support".

This support consists of the olm-wine master program and individual preconfiguration scripts for error-prone Win32 apps called .owm. These scripts with the olm-wine automate all the tasks and also dynamically notifies you of missing .dlls

Also an exclusive gallery of well-known tested Win applications has been provided to give you postive outcomes. It comes with the following features

• Easy setup for integration of WINE
• Auto regeneration of configuration file
• The required missing DLLs for a program will be notified to you
• And not to mention its usage of .dlls through DLL cache specification
• Preconfigure scripts called .owms avoids manual & trouble-prone configuration

Changes:

• Updated Wine config
• Updated system.reg
• Fake Windows drive is now in $HOME/.wine/fake_windows
• Added option to install some applications that work well with Wine
• Fixed some bugs.

I'm pleased to announce the public availability of a gateway from WineHQ CVS to the GNU arch version control system.

It marks the beginning of an experiment not just in new forms of source control, but in new ways for the Wine project to be organized and function. Nobody knows if it will catch on, but a few of us (me, dimi and alexandre) have been discussing trying out arch for some time. Carpe dium!

For those who already know arch, here are the details:

archive coordinates:

http://navi.cx/~mike/wine/archive
mike@winehq.org--wine/wine--mainline--0.9

It is synchronised with CVS on the hour, every hour and a cacherev is generated once a week. There are a couple of useful scripts in the root directory, branch and merge which are described in the README.ARCH file.

It is mirrored here:

http://mirrors.sourcecontrol.net/mike@winehq.org--wine/

For your convenience, and for people who have never used arch before, here is an adapted form of the README file. It's available in pseudo-HTML here:

http://navi.cx/~mike/wine/

README.ARCH
Mike Hearn <mike_at_navi.cx>

GNU arch is a distributed source control system. Unlike CVS, arch trees can be branched and branched again with no write access to the original archive. It has a variety of other interesting features - for instance, it uses atomic commits and can operate using a variety of network protocols. For more information see

http://wiki.gnuarch.org

The rest of this readme is going to take the form of a tutorial on how to use arch with Wine. We'll start with the really basic tasks that even non-developers may wish to use, and work up to advanced usage (branching and merging). It will not answer every question you have, for that use the wine-devel mailing list for discussion :)

Tracking Wine development

You might not be reading this file from an actual arch project tree, so let's start with how to get a copy of the gateway tree. First of all you need to install "tla", the reference implementation of arch. You can get it from the gnuarch.org website, but a simpler way is to use the prebuilt binary available here:

http://www.navi.cx/~mike/wine/tla

Download, chmod +x, copy to /usr/local/bin and you're set. The binary should work on any Linux distro. Yes, it's large, the arch build system contains local copies of various libraries.

Now it's installed, run the following command:

tla register-archive http://www.navi.cx/~mike/wine/archive/

Once it's registered succesfully (you get a message when that happens), you can run the following command to check out the tree:

tla get mike@winehq.org--wine/wine--mainline--0.9 winehq

This will place a copy of the gateway tree into the winehq/ directory. "winehq" is a good name to use, because you may later wish to download other branches or even make your own, but there will only ever be one WineHQ tree - the one controlled by Alexandre Julliard.

Yes, I know mike@winehq.org isn't a valid email address, it doesn't matter, the email is only for namespacing and identification reasons :)

The checkout will take a long time even on a fast link: the Wine sources are 10mb.

Once downloaded, updating the tree is easy:

tla update

So, now you have an arch project tree, what can you do? Arch has a lot of interesting commands, try "tla changelog | less" and "tla missing -s".

Have fun!

Simple development

The easiest way to use arch for development is the same way we use CVS: by making changes in a tree, generating a diff from those changes, then emailing in the patch. Let's see how that works.

First of all, we make a change to the tree. Then we use:

tla changes --diffs

to get a patch. In fact, this will generate a lot of extra stuff that you don't really need - feel free to remove it, but it's not required; patch can strip out garbage from patches automatically.

You can get diffs for individual files with "tla file-diffs".

Be warned! These commands work against the last revision you updated to, they do *not* work against the latest version in the remote archive like CVS does. So you probably want to update first.

However, unlike people just tracking the tree, you DO NOT want to use update! What update does, roughly speaking, is reverse changes made to the tree, apply the missing patches in turn, then reapply the changes. If there are any conflicts they appear in patch style .rej and .orig files.

There is a better way. Arch has a very interesting and powerful command called "star-merge". Star-merge allows you to merge your tree with another tree (subject to certain rules), in a Just Works(tm) fashion. It will deal with repeated merges, and it works if the person you are merging from previously merged with you.

To make this simpler, I've written a script that figures out what you branched from (if anything), and then merges from it. In effect, this is like "cvs update".

./merge

This operation can be quite slow (because it involves constructing multiple trees and comparing etc), but it is generic and will work both with the gateway tree and branches - and it gives 3-way merge inline conflict markers too.

If you want to do it faster, and you don't mind having patch style .rej files generated in the case of conflicts, try:

tla replay

which simply downloads and applies each patch in turn. "tla update" does work too, why not try them all to see which you prefer?

More advanced development

So, here comes the interesting part. Now we've covered the basics, it's time to find out how to make our own branches and commit to them.

You need to do a bit of setup. Firstly, you need to tell arch who you are:

tla my-id 'Mike Hearn <mike@navi.cx>'

You have to use an ID of this form, arch won't accept others. Now you need to create your own archive.

An archive is a rough equivalent of a CVS repository with one key, fundamental difference: an archive is a personal thing which can contain trees from many different projects. Typically a person has only one archive, though there's nothing to stop you having more. Archives are just directories and files, no special server software is required to run one. You just need some web space.

The basic workflow with arch goes something like this:

1. Write patch
2. Commit to branch
3. Upload to a mirror of your archive

Your actual archive is usually kept locally, which means you can work and commit offline. You then mirror the archive on a remote machine so others can access it. If people want to work on your tree, they branch it, set up their own archive and then you pull from that. In other words, only you have write access to your own archive.

So, you need to create a directory to hold your archive, like this:

tla make-archive mike@navi.cx--2004 ~/Code/Arch/2004
tla my-default-archive mike@navi.cx--2004

The named directory should not exist, but its parents should. An archive name must have the form of an email address followed by two dashes followed by a name. I've named it after the year of creation which is pretty common, but it can be called anything.

Now, to speed things up, you probably want to create a revision library. This is basically a cache (using hard links to be space efficient) of revisions. You probably want it. Make one like this:

mkdir ~/Code/Arch/RevLib
tla my-revision-library ~/Code/Arch/RevLib
tla library-config --sparse --greedy ~/Code/Arch/RevLib

OK. Now you just need some web space. Arch can write to web space using ftp, sftp (ssh), or WebDAV. You probably want sftp. If you don't have any web space you can use talk to me and I'll put you in touch with people who can give you some space specifically for hosting archive mirrors.

For now, don't bother with a remote mirror. Just play with a local copy, that way you can delete things if you make a mistake (you can delete a branch with rm -rf inside the archive, nothing will break).

Let's say you wish to branch the WineHQ tree, and start work on the winecfg program. The simplest way to do this is to create a directory somewhere that you will keep all your wine trees in. I use /source/wine/arch. Then do this:

tla get mike@winehq.org--wine/wine--mainline--0.9 winehq
cd winehq
./branch winecfg

The branch script is in the root of the gateway tree, and is written by me. It's designed to let you rapidly branch a given Wine tree. It works by "tagging" the new tree in arch, then duplicating the current tree using hard links before finally forcing the duplicate tree to the new branch.

WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! WARNING!

If you use the branch script you MUST use an editor that breaks hard links. emacs does by default, vim does if you :set bkc=no (though be warned, this will interfere with some programs that invoke vim like crontab -e). zile does not.

If you want to use an editor that does not break hard links you must remove the -l option from cp in the branch script so a real duplicate is made. This uses more disk space and is slower to copy, but is safer.

The result is a new directory called winecfg/ created as a sibling of the current directory. You can now commit to this tree however often you like. Running merge will update the branch from the tree you branched from (in this example, the gateway tree).

You can branch from that tree again, and so on. Other people can branch from your tree in the same way! Feeling giddy with power yet? No? Good. Remember that patches which aren't in the mainline tree aren't reaching 99.9% of the users - being able to branch is useful but don't use it as an excuse to not get patches upstreamed!

The new branch is put in the default archive and the name is derived from the current, for instance running branch in mike@winehq.org--wine/wine--mainline--0.9 with the parameter "winecfg" produces a branch with the name mike@navi.cx--2004/wine--winecfg--0.9

Now you need to publish a mirror:

tla make-archive --mirror mike@navi.cx--2004 sftp://navi.cx/home/mike/public_html/archive

and update it when you have done some commits

tla archive-mirror

which actually uploads the changes. If you are always connected, you can run the first make-archive command (back when you created your archive) with a remote location straight away, but doing it locally then mirroring is more common.

Finally, tell people the URL of your archive so they can register it and grab your tree. That's it!

Oh one other thing to mention is that hard linking trees together like the merge script does is sort of experimental with Wine. I've tried it and not seen any big issues as long as you use an editor that breaks hard links, but if you spot wierdness related to that let me know.

Have fun! If you have any questions ask them on the wine-devel mailing list.

Basically, we need to reserve address space. There are two ways to do that, the first is using my preloader (which is nearly turned into patch form to see us through the intermediate time), and the second is by using a linker script to define a new ELF section of the right size and location.

Because of a bug in the binfmt_elf module of the kernel, the linker script approach doesn't work. The kernel ignores the protection flags on the ELF section and doesn't make them restrictive enough - in other words, we will have the same problem that Alexandre's BSS hack had way back in the day.

As part of the ongoing efforts to work nicely with execshield/prelink, the following kernel patch was developed (not by me I hasten to add):

http://www.bitwagon.com/elfdiet/elfdiet.html

However it needs testing so the kernel guys can be sure it'll not break stuff. If you are the type who runs their own kernel, please apply this patch and let us know the results! Everything should be fine, it's just a bugfix for the edge cases, but the elf loader is security critical so it needs widespread usage before it can be upstreamed.

Is there such a thing as a source code visualization tool that makes a "map" of a program's source? Kind of like:

      main()
        |
        |
     function1()
        /\
       /  \
func2()   func3()
    |
    |
  func4()

I'm basically looking for something that will help me "see" how a program is laid out so that I can start working on it.

You are looking for "Source Navigator" It's here...

http://sourcenav.sourceforge.net/

It's pretty cool if you want to follow what goes where. All the pretty arrows too. (It's horazontal along the top of the screen, not vertical) You can follow functions, stucts, header files. I used it when I was a programmer...

I don't think an automatic tool would be very good at this with Wine. It's a ridiculously complicated codebase, not through choice mind you, that's just the way Windows is.

Really at some point we should provide better high level developer documentation like this - I remember finding it confusing too.

Basically, the most important code is all in the dlls/ directory. The other directories are either supporting code (for booting the emulator, unicode tables etc) or are holdovers from the old code layout (windows/) and is being slowly moved into the dlls/ directories. The code in windows/ is a part of USER, basically.

Knowing the windows architecture helps. Basically in NT based systems you get:

• NTDLL - low level OS services, threading, file io etc). the so called "native" API
• KERNEL - Win32 "layer", really mostly a set of forwarders to NTDLL
• USER - random stuff - core windowing mostly, as well as clipboard and a few widgets like the button control, edit box, menus etc
• COMCTL32 - Widget library. More advanced/modern controls like the toolbar, rebar, tree view etc. Pretty straightforward
• OLE* - Our COM/OLE implementation.

Is that what you meant? Best way to learn the source is by reading it, and then asking about the parts you don't understand basically.

I've been following Wine development for a while but now I think it's time to contribute something. I don't use Windows anymore (except for PowerPoint at work), but I decided it would be good to be able to play some games on Linux. So I grabbed Wine from CVS, compiled it up and tested some games, with the intention of finding where the problems were and hopefully fixing them. I have put my results on a web page at:

http://www.epcc.ed.ac.uk/~jamesp/wine-games.html

to avoid making this posting even longer. Note that I haven't looked in detail at most of these yet, I just tried them out once to see what would happen. I'm gradually working through the list, debugging them.

I've already submitted one patch (to solve a D3D blend mode problem that affected SonicR). I have traced a number of the other problems to specific aspects of Wine, but I'm not sure how to go about fixing them. Any advice on these points from experienced developers would be welcome