Wine Traffic #184 For 22�Aug�2003

In the process of separating ntdll from kernel32, one of the next hurdles is the implementation of NtCreateFile. Since it's rather a core API, some thoughts are needed. Moreover, there are a few shortcomings in current file handling we might want to address.

The goal of this small RFC is to:

1. gather all known shortcomings of current file implementation
2. propose some solutions and roadmaps for the implementation

feel free to add your own remarks to these lists.

shortcomings

1. no dynamicity in DOS drive management: user should be able, in his/her session(1), to either change the mapping of a DOS drive letter, or mount a SMB share somewhere...)
2. mounting should be done session wide and not process wide, and of course shared across processes
3. ntdll and kernel32 are not separated regarding file management
4. devices (vxd and DOS) management is still a hack
5. it may be necessary to introduce a notion of filesystem for some operations (open, read, write, seek, close, dir_enum, ioctl) in order to support several FS (unix, smb...)
6. poor management of ejectable devices
7. short/long file name conversion should be consistent (across several processes in the same session, across several sessions)

Note: (1) a session is linked to a running instance of the wineserver

proposal

1. "path gates" (target: #1, #2, #3/partly, #4/partly, #6/partly) all file names will be handled using the NT-style:
   • DOS paths (C:\foo\bar) into \??\C:\foo\bar
   • network paths (\\station\foo\bar) into \??\UNC\station\foo\bar
   • DOS device names (CON) into \??\CON
   • NT device names (\\.\PhysicalDrive0) into \??\PhysicalDrive0
   • a VXD ?? (normally \\.\VFD) into \??\VFD
   
   
   the device concept will disappear and be replaced by a "path gate":
   in NT-Win32 name space, point where to convert from a NT-path to a Unix-path
   
   for example, assuming we have created a path-gate from \\??\c: to /mnt/windows, the NT-path \\??\c:\foo\bar will be transformed into the unix-path /mnt/windows/foo/bar
   the path-gate will hold the current options that the device currently does (case sensivity, fixed/removable...)
   
   path-gates will be stored in the server (they are session wide)
   
   management of VxD and devices could benefit from it: they could be path gates pointing to them selves (or nothing ?)
2. FS (target: #4/partly, #5/partly, #6/partly)
   
   I'm still wondering if we really need a FS (even a very simplistic one) in Wine
3. short/long file names (target #7)
   
   since current short file name management seems satisfactory, there's no need IMO to move short/long names conversion in the server

It would be good to implement this "path gates" as near as possible like the way it is implemented in Windows NT/XP. This mapping is done using the NT object namespace. Object namespace is another virtual tree structure similar to a file system or the registry. It is accessed by functions like NtOpenDirectoryObject(), NtOpenSymbolicLinkObject(), NtQueryDirectoryObject(), NtQuerySymbolicLinkObject(), ... There has been a tool in the internet called "winobj.exe", with which you can examine this internal NT object namespace. However it is not compatible to XP, I don't know, if someone has yet updated it. But I do have a own program which can display it also for XP.

Drive mapping is stored as symbolic links for example at "\GLOBAL??\D:". This links point to the partitions, like e.g. "\Device\HarddiskVolume3". There are also the ArcNames, which you can find in the NT bootloader config file "boot.ini". At "\ArcName\multi(0)disk(0)rdisk(0)" you can find another symbolic link to "\Device\Harddisk0\Partition0".

This examples are from my XP system. System versions before XP used a bit of another mapping schema.

By the way: You can also find the whole registry as a subtree in the NT object namespace at paths like "\REGISTRY\HKEY_LOCAL_MACHINE\.... Would not be bad, if we could implement this also.

I do know how it's implemented in NT.

anyway, there are a few differences between ROS and Wine:

• in common, we do need to have the same interface as Windows do. This only applies to interface living in user land.
• kernel land is another matter. ROS needs to stick to NT's behavior, Wine needs to convert calls to what Unix does provide.
• ROS needs to mount physical drives, Wine makes (it's the current implementation, this is open to discussion) part of the Unix file hierarchy visible from the windows part

Regarding file support, the minimal things Wine need to do is:

• support the same file naming as NT does (mainly the \??\ part of the NT name space), the rest is not absolutely needed. Currently, Wine does not implement a unique name space for all the objects (as NT does, using sub dir for some differenciation) but rather different name spaces
• to allow the mapping from a NT path to a unix path, there are two simple ways to do it:
  1. use all NT name space and somehow inject Unix file name space (/ and all the subdirs) in it
  2. store the unix information (the path in the unix / hierarchy) as a specific attribute of the NT name space (at least the \??\ subtree)

I chose the second approach (you're more proposing the first one) because I felt it was more suited to wine behavior.

Concerning NTDLL's directory objects, I'm still wondering if we need (in Wine) to implement them. It's indeed needed in NTDLL's interface (object attribute for instance) and implements part of the "path gate" (especially some attributes like case sensivity in further lookups...). So, this is still part of the open questions I have (I may start without directory objects, and add them later on).

One of the strength of current NT name space is the orthogonality which is used across all objects in NTDLL (for example, in handle management, or sending requests - thru IRPs - at all the objects). The good side is to be able to convert an operation on a file into an IRP which is send to the FS which handles the file, which can in turn send it or transform it to another part... As wine implements a more monolithic approach, and since we don't have this orthoganility available today, I'm not yet convinced we need to stick more to the NT behavior than what I described above as the minimal requirements.

More to the point we need attributes to allow support of NT ACLs. I've been looking at the problem and progressing only very slowly, some of the problems are technical, others are nothing to do with Wine at all.

At the level we are dealing with here the essential problems (for those who have not examined the issues) are :-

• Not all supported O/S s offer ACLs.
• Those that do support POSIX ACLs which do not map exactly to NT ACLs
• Even in Systems which do support POSIX ACLs there is no guarantee that all disc filesystems at any one host will offer it, Of the Linux filesystem types only XFS supports ACLs out of the box. Work is in progress for ReiserFS and due to start for JFS. There is a patch for ext2/3. One map currently favoured is to use ext2/3/Reiser for the root filesystem and XFS for (some of) the rest).

All of the systems that do support ACLs use Extended Attributes to store them. I am still looking at limitations on number of EAs, but there is a possibility that an extreme case on NT may not be supported on Linux.

Since I've not yet got round to looking at TrustedBSD there's potentially another can of worms there.

My conclusion is that we do (regrettably) need a VFS layer to harmonise handling of all this lot and respond correctly when wine-ver is 9X.

I think Wine's goal is to provide this kind of feature of top of what the underlying OS provides (this is also the case for DOS HIDDEN and SYSTEM attributes => there are not available in standard Posix FS, so are managed by wine(*)). So IMO, file (protected by ACL) access should be provided by the underlying OS.

ACL manipulation (from windows to posix - even if 1003.1e hasn't been voted by POSIX) should be added (I'm not sure we have a 1:1 mapping anyway) however, I don't understand your remark about the winver 9x... IMO, if the Linux user mounts a NT partition with ACL (and has the proper privileges) it should run Wine on this partition rather transparently (except for ACL manipulation function) it's not my goal to support:

• real FS mounting in wine => this is the job of the OS, not wine's
• a real VFS (as Linux does)
• the (V)FS I'm talking about is more related to adaptation (as you mention), but I'm not sure I got you right

(*) except files starting with a . which get the HIDDEN attribute

Can't really answer your question, but on the kernel mailing list, There has been some discussion of whether these attributes should be exposed from mounted fat partitions: The fat fs maintainer has a patch, and is wondering what to do with it. (The patch exposes the fat attributes as extended attributes)

See the thread titled "[RFC] ioctl vs xattr for the filesystem specific attributes".

• All changes to the source code must be made available
• Anyone redistributing Wine must provide access to the source code under the original terms of the LGPL
• However, anyone who wishes to simply "link" their own Windows program to Wine can do so without having to make their source code available.

• Source code (including all changes from the original Wine sources) must be made available to people who receive a binary of Wine. This also applies if Wine is used as a library, in which case only the source of Wine (including all changes) must be made available.
• Simply linking with Wine does not mean you need to make the source code available for your program.

Apparently the changes from yesterday removed support of environment variables in registry keys (very useful for the "Path" keys in the "Drive" sections).

Can it be readded? Or is there a rational behind it?

Actually it's just the format of the Wine config has changed. Now all environment variables should be surrounded by the percent signes (%HOME%), and are handled by the normal registry code. Alexandre has changed the sample config as well.

If I simply remove -lpthread from

• dlls/glu32/Makefile
• dlls/d3d8/Makefile
• dlls/d3d9/Makefile
• dlls/d3dx8/Makefile
• dlls/opengl32/Makefile

Wine, builds without problems.

Top 10 reasons you want Fox ported to Wine:

10. Windows sux
9. Microsoft is evil
8. Test windows features without rebooting (as Doug mentioned)
7. Keep development on Linux
6. Run valgrind on the Windows port
5. Make sure the Win32 part is portable to multiple implementations
4. Help find missing features/problems/etc. in Wine
3. Some people may prefer the Wine widgets to the GTK+ ones
2. wxWindows is already ported to Wine, Fox needs that too! ;)

and the number one reason is:

1. It is just cool!!!

(what geek can resist this sort of virtualization? :))