Samba Traffic #19 For 13 Apr 2000

By Peter Samuelson

Table Of Contents

Introduction

For various reasons, not least among them that the week before last was unusually dry on the Samba news front, I did not publish a KC Samba last week. This past week should more than make up for any lack of news the week before, however. Luke Leighton is still churning out Samba-TNG alpha releases -- five in the past two weeks. Jeremy Allison is getting ever closer to releasing Samba 2.0.7, with pre-release 2.0.7pre3 now out and looking good. No showstopping bugs have been reported in the past week or so -- some longstanding glitches remain but nothing new seems to have jumped out -- so the long-awaited 2.0.7 may actually be on the horizon. This will be something of a milestone, since 2.0.6 has several known "issues" with Windows 2000. (Jeremy has noted that 2.0.6 predates Windows 2000, so they're not his issues!)

The real news is still on the TNG front. Luke Leighton popped in to see Tridge in Canberra in late March, and the two of them spent a few days merging features from the HEAD branch into the TNG branch. (That is why the CVS stats show Tridge committing to the TNG branch, which normally would be unusual.) The result of their toils is that now, not only does Samba-TNG have much-superior RPC (NT domain services) code in it, it now also has equivalent code for file, print and browsing services, so the well-worn advice "use TNG only as a logon server, and use another release on another machine for everything else" may no longer be necessary.

The other tidbit this week, for those of you watching the CVS stats, is that Elrond is now a Samba team member with direct write access to the repository. Our congratulations. We had been wondering why he wasn't already in the club, because for a long time he has been feeding patches -- a lot of patches -- through Luke. We're hoping they'll both work more efficiently now, which will be all to the good.

Mailing List Stats For This Week

We looked at 771 posts in 1594K.

There were 321 different contributors. 115 posted more than once. posted last week too.

The top posters of the week were:

 

1. UTMP, Cont.
24 Mar 2000 - 7 Apr 2000 (35 posts) Archive Link: "More utmp stuff"
People: David LeeJeremy AllisonElrondGiulio Orsero

The UTMP/WTMP saga continues BROKEN KCREF on samba-technical. To sum it up, David Lee's UTMP support patch has finally gone into Samba with the 2.0.7 pre-releases, and only now do we all realize what a complicated beast UTMP can be, especially portability-wise. David and the others are doing the best they can to ensure that the patch works for as many different Unix-like systems as possible. It's clearly an uphill battle, but the top of the hill may be in sight.

Freddie asked:

A question, why is there a utmp directory parameter? This from the GETUTENT(3) man page:

DESCRIPTION
utmpname() sets the name of the utmp-format file for the other utmp functions to access. If utmpname() is not used to set the filename before the other functions are used, they assume _PATH_UTMP, as defined in <paths.h>.

If there's already a valid OS default path (at least, there is on Linux/glibc 2.0.x/2.1.x), why force the user to specify it? Perhaps it should be an optional parameter, only needed if _PATH_UTMP isn't defined?

He also wondered why one user would show up multiple times in the utmp file while only being logged in once.

David answered both questions. For the first, he said, it's a work-in-progress, and eventually, ideally, Samba will auto-detect the locations of the files at compile-time and no parameter will be needed at all. For the second, he said: "But this "problem" is not a "problem". The aim of utmp is to show connections, not pids. in the Samba case, we have multiple connections between a PC and a UNIX host. So utmp, just like smbstatus, shows multiple connections." Elrond, once again (he made this point last week as well), noted that the UTMP portability wheel has been invented before, with GNU Screen, xterm, rxvt and other free software.

At this point David collated all the known UTMP issues into one post:

  • The utmp functionality seems to be proving popular. So we ought to continue to improve it and port it.
  • The sheer variety of utmp implementations means that porting is becoming increasingly non-trivial.
  • We need to pare down to the absolute minimum the work needed by the central Samba team to handle this can'o'worms. My guess is that they want to keep out of the gory details of implementation, but nevertheless keep a managerial eye over the process.
  • Samba 2.0.7pre2 (not pre1) is the base to work from.
  • I would strongly urge that this base be augmented by a patch I sent to Jeremy a few weeks ago (since re-sent) which makes a better attempt at getting the default pathnames.
  • We should strongly consider moving the code out of "smbd/connection.c" into a separate file, because the code will, of necessity, become increasingly littered with "#ifdef MY-LITTLE-OS" constructions (somewhat akin to "smbd/quotas.c").
  • The basic principle should be that for plug-and-play only "utmp = yes" is needed. That is, that compile-time and run-time should, by default arrange that the data appears in the system utmp-type files.
  • Thus "utmp dir" should, ideally, be only needed (1) for over-riding the (sensible) system-specfic defaults (2) to get someone started on a new port.
  • Despite the above downgrading of "utmp dir" to geek status, there should probably be a "wtmp dir" too.
  • Worthy consideration should be given to the idea of consolidating multiple connections from the same PC into one single utmp record, perhaps by a "utmp consolidate" parameter, and idea suggested and already implemented by "Freddie <freddie@ns1.nowait.net>" .

Jeremy Allison responded, agreeing with most of the above, and said that a lot of cleanup, including moving the UTMP support into its own source file, was scheduled for Samba 2.0.8. Several others posted opinions on "consolidating" UTMP entries, almost unanimously in favor of it.

So Freddie continued to hack on his "consolidation" patch, and started a new thread (http://samba.org/listproc/samba-technical/March2000/0320.html) , where he and David talked about it for awhile. The next issue turned out to be how to number the connections. The original UTMP code showed a user logged in on the TTY "smb/n" where n is the Samba connection number -- but the newfound problem was that, if some connections are skipped, n is now in a sparse range, which looks funny. While they were discussing the relative merits of sparse versus packed numbering, Jeremy broke in again: "If you want this in 2.0.7 official please get it to me asap. I have had no significant bug reports against 2.0.7 so I'm intending to fix up the last parts of packaging and documentation and then ship it....."

So, naturally, that thread ended and David started another (http://samba.org/listproc/samba-technical/March2000/0327.html) , announcing a new UTMP patch against Samba 2.0.7pre3. Giulio Orsero posted a few bug reports, which he and David hashed out for several posts.

Moral: if UTMP support doesn't work properly in Samba 2.0.7, don't be surprised. It's still very much an experimental feature. Support should shore up significantly by Samba 2.0.8.

 

2. Short-Filename Mangling: How Microsoft Does It
28 Mar 2000 - 30 Mar 2000 (8 posts) Archive Link: "NWFS mangled name algorithm."
People: Jeremy AllisonJerry CarterJohn MalmbergDave Collier-BrownElrond

Jeremy Allison posted a notice to samba-technical: "Timpanogas just released a netware filesystem module for Linux under the GPL. They use the following hash functions to generate mangled names :" He listed some code, then continued, "Andrew, any comments on changing to one of these in Samba instead of the (rather crappy:-) str_checksum() function ?" [The Timpanogas Research Group is headed by Jeff Merkey, a former kernel developer for NetWare, so he has probably been wrestling with short/long filename mangling for years.]

Nobody took the bait and talked about the actual new code involved. Jerry Carter commented, "Every time the filename mangling code changes to produce a different name, it is possible to break PIF file shortcuts under Windows 9x. I noticed this between 2.0.3 and 2.0.6. Only affects PIF files to DOS programs whose filenames are not 8.3 compliant obviously. Most often occurs when a batch file uses a LFN." To which John Malmberg replied,

It will not help much.

The only way not to break things is to do like M$soft does and store the mangled short filename as an attribute of the file.

This of course presents it's own problems, as the native operating system does not know about it to update it when the file is renamed.

That is assuming the underlying file system has a place such an attribute could be stored.

This problem exists even in a pure M$soft environment. Each time you change a long file name it recalculates the short name. Different revisions of Windose 9x and Windows NT use different mangling algorithms.

There is also no guarantee that a specific short name will be available when it is being calculated.

The only reason this problem appears less frequently in a pure Microsoft is that the short file name is an attribute of the file. In fact, it seems that the short file name is the true name of the file, and the long file name is the attribute.

One user that I know of, managed to create two directories with the name "Delete me test" on an NT share. The 8.3 filenames were unique, but the long file names were identical, including case! The only way to delete the directory was from a DOS cmd window using the 8.3 short name.

As to that last, Dave Collier-Brown was properly horrified. Elrond had a theory, though: "I highly guess, he was running nt on vfat and not ntfs. AFAIK ntfs is the other way round: You normaly have long names and the short names are just an "attribute" (the ntfs filesystem in Linux can present them as hardlinks. Oh: and ntfs also has hardlinks on its own, which nobody tells you loudly.). There's even a switch to turn of 8.3-filename-generation. If you only have nice 32bit apps, you can do so, cause they should support long names. If you have old apps, they wont even see the files with long names (AFAIK, haven't tested)" He was right, John confirmed.

 

3. The Flat NetBIOS Namespace Problem
28 Mar 2000 - 2 Apr 2000 (7 posts) Archive Link: "Acting as PDC"
People: Jerry CarterPanagiotis MalakoudisTim ColeRichard Sharpe

Panagiotis Malakoudis was having trouble getting his domain controller up and running. He posted the important bits from smb.conf, and all that. The symptom seemed to be that Windows98 complained about an "incorrect parameter".

Jerry Carter had seen it before. "Use different names for the server netbios name and workgroup name (possibly even need unique user names)." Panagiotis replied, "I'll be damned!!! It actualy worked. do you have any idea why this happens? Why can't you have the same netbios name as the workgroup name?" Jerry said he had never tracked down whether it was a bug or a spec limitation, but Tim Cole knew: "Because the Microsoft World has a flat namespace, and an even flatter in NetBIOS Land. In NetBIOS, users, servers, workgroups and more all exist in the same namespace." Richard Sharpe posted the failure scenario: whoever registers the NetBIOS name first, be it the computer or the workgroup, the other will be unable to do so. "One way or the other, you are SOL."

 

4. Beta-Tester of the Month
28 Mar 2000 - 6 Apr 2000 (17 posts) Archive Link: "Report on cvs of 28/3/00 1130 BST"
People: Tom CrummeyJason JensenLuke LeightonJean-François Micouleau

This was not a thread, just a minor phenomenon. Tom Crummey, who has expressed interest in deploying Samba domain controllers, has recently started testing a lot of Samba-TNG code and posting a lot of bug reports. Perhaps the title "Beta-Tester of the Month" is a bit exaggerated -- many people, after all, put in hours downloading, compiling and running Samba-TNG -- but Tom could certainly at least compete for the title. We count eleven full reports in about as many days.

Tom's reports are from a particularly useful vantage point, because Tom is running Solaris on a Sun Ultra. The UltraSPARC has two strikes against it: (1) it is big-endian, the opposite byte order from what most SMB clients use, including Windows and Windows NT; and (2) it is 64-bit, which is 32 bits more than what most SMB clients use, again including Windows and NT. (Even NT on Alpha is only 32-bit for most purposes.) Thus, in recent weeks, Luke Leighton has been struggling to get all his code to run properly on big-endian and 64-bit machines.

Excerpts:

 

5. TCP/IP, the One True Way
29 Mar 2000 - 31 Mar 2000 (5 posts) Archive Link: ""unbecome lmb" oddity"
People: Duncan CampbellHerb LewisDave Collier-BrownJerry Carter

We bring this thread in not because it is long or rich, but because an interesting point was well-made. Duncan Campbell posted the logs from an ongoing browser war (no, not Communicator versus Internet Explorer, but Samba NMBD versus Windows NT). He explained, "an NT box on the LAN appears to force an election about once a day. the samba server loses this election, as office politics dictate that it should, but then something breaks. the LAN becomes unbrowseable. samba's nmb log shows:" [insert log file trimmings here] "first, after announcing that it has stopped being master browser, why does it immediately attempt to become master browser again?"

Herb Lewis gave an good summary of what could be wrong: "If you have a microsoft machine on the same subnet that is a PDC for this domain this is exactly the behaviour I would wxpect to see. MS wants their machines to be everything. If it wins as domain master browser it will want to be local master as well. The other case that can cause this is a microsoft machine running multiple protocols (TCP/IP, netBEUI, IPX/SPX). Samba only listens on TCP/IP so it cannot participate in elections on other stacks. Again the microsoft philosophy of being everything comes into play again. It the machine wins on one protocol it will announce itself as the master on all protocols. Samba see this announcement and since it had previously won, it demotes itself and forces another election (per the protocol)." Dave Collier-Brown subscribed to that last theory, that NT was using a browser election on NetBEUI to declare itself a winner on TCP/IP, which would definitely confuse Samba, since Samba knows nothing of the NetBEUI election. He asked the world at large, "Any gurus have an idea why NT ends up on NetBEUI? Would it be the default protocol, perhaps? And if so, why did it ever provide browsing on TCP/IP? We might need to compare NT's initial election at boot time with this later one, to see what's different..." Jerry Carter said to check the protocol binding order.

Duncan couldn't check the protocol binding order or anything else, though: "sorry -- office politics came into play again. can't investigate, have been ordered to leave it in its current broken state. go fig. no, i have no logs showing samba giving up local browse master to the NT box in a happy way." He had his own theory about the real problem in his case: "this leads me to suspect my trouble is not now with samba but rather with the local netadmin. humans aren't well documented. :)"

The moral here, as Dave reiterated in another thread, referring back to this one: be not tempted by NetBEUI, IPX/SPX or any other protocols you don't actually need on your Windows boxes. Multiple protocols can cause subtle problems, when not every network host is using all the same ones.

 

6. Samba-TNG Alpha Releases (As Usual)
29 Mar 2000 - 4 Apr 2000 (5 posts) Archive Link: "samba-tng-alpha-1.4.tar.gz"
People: Michael HuletLuke LeightonMichael Breuer

Luke is keeping the pace up, with five more Samba-TNG alpha releases in two weeks. As Michael Hulet put it, "I was busy for a week and Samba went from alpha-1.3 to 1.8 (amazing!)." Release notes on alpha-1.4:

rpcclient and samedit etc. on sun ultras were failing because getopt cannot be reused. evidence of this is by doing a samedit "createuser username -p password" and the reported password on-screen is total garbage.

this was fixed by using the GNU getopt and getopt_long functions (hooray!) in the same way that rsync does.

i have access to a sun ultra 5, now, and have confirmed that TNG can have workstations joined to the domain, and that usrmgr works, and the passwords created etc on it all work.

Alpha-1.5:

stupid bug in 1.4, making it inoperable. sorry, folks.

starting a merge of cvs main, nmbd has been done, now the configure script. the configure script you may find that certain functionality doesn't work (e.g lonnie borntreger reported that -DWITH_PAM should now be -DHAVE_PAM).

Alpha-1.6:

mainly a maintenance update: all bzeros replaced, as was done in cvs main / 2.0.

readline detection added. if someone wants to add an autoconf test to detect -lcurses being needed by solaris readline, please create and send one.

usrmgr user-account changing is now accepted (on systems compiled with the default, --with-sam-pwdb=passdb, this means you can set the user's password and the account control bits: account disabled etc., all other changes are ignored), i missed out an info level.

there is one field of the SAM_USER_INFO_21 structure that is not well understood, which is giving grief across the board: AS/U and compatible systmems may now not work properly, sorry, i don't have access to it to do the appropriate tests.

elrond updated the libtool scripts so that they should set the LD_LIBRARY_PATH env. variable correctly, allowing the samba programs to be started without needing to do make install, you can run them from the source/bin directory (which i do all the time).

tested on both solaris 2.8 ultrasparc 5 (gcc 2.8.1) and mandrake linux on x86 (gcc 2.95.2):

profiles work, nt5 beta1, nt4 wks.

password changes work, nt4 wks.

join to domain works, nt5 beta1, nt4 wks.

printing not tested (by me), couple of reports of it not working.

Michael Breuer gave this one a big thumbs-up: "With 1.6 I can now join W2K systems to the domain... usrmgr works... overall this seems to be a great vintage." But it was Alpha-1.7 where things truly got interesting:

what you've all been waiting for: a merge of smbd from cvs main to SAMBA_TNG.

please help test this one lots, i may have missed something from the code i pulled over from cvs main (70,000 lines of code pretty much copy and diff style!)

i spent some time last night getting the security file/dir tab working, and lo and behold, you can view and change unix file perms (i am very impressed, jeremy!)

my next will be spoolssd from cvs main, i think, which jean-francois is developing: i will pull over his cvs main work to TNG.

When asked (in another thread) why he was merging the useful parts of the HEAD code into SAMBA_TNG, instead of vice versa as originally planned, Luke replied: "it turns out that it was far easier to go the other way round: andrew and i did nmbd in... a day, and i did smbd in two, maybe three. now, all of 2_0, main and TNG can be simultaneously updated. andrew and i are going to do an architecture review of TNG, deciding what bits are suitable and what bits need rewriting, my hope is to maintain the same aims / flexibility, just done in better ways."

Quite a few people complained that msdfs did not compile with alpha-1.7. It turns out that the tarball was missing the entire msdfs directory. "Oops!" So about thirteen hours later, Luke put out the Alpha-1.8 announcement:

nmbd merged. smbd merged. printing merged (couple of weird bugs, but it basically works. well done jean-francois!).

NOTE: if you understand NT printing and how it works, you will do OK with the new NT-style printing. i.e you have to install a printer on the server (you will need an NT cd for this) and you should create a share [print$] which is world-readable and admin-only writeable.

 

7. No ACL Support on Linux
1 Apr 2000 - 6 Apr 2000 (26 posts) Archive Link: "Samba on Linux with no ACL's is making things tough"
People: Michael MarschallPeter SamuelsonDave Collier-BrownJeremy AllisonSander StrikerLuke Leighton

Michael Marschall posted to the samba list asking about access control lists. Specifically: "I am presently in the process of moving my company's file server from Windows NT 4.0 over to Linux with SAMBA and the lack of ACL support in the ext2 filesystem is making things very difficult to design. To clarify I am NOT writing about Samba's support for NT ACL's on NTFS. I am writing to possibly get some tips for getting around the lack of ACL's in ext2." He went on to some length, describing the setup at his company and why he really needed ACL functionality.

I replied, "Samba does not directly support ACLs on any OS. That is to say, you can't manipulate them from the client. Sounds like this wouldn't be a problem for you, as you would set it all up directly on the server." Also, Michael had mentioned that Linux ReiserFS has ACL support, so I suggested he look at SuSE, since they ship and support ReiserFS with Linux. Dave Collier-Brown also replied. "Well, you just described the problems that ACLs on Multics were designed to solve, and one which Unix perms didn't attempt to address. Darn! Can you put an smb server on a machine with ACLs? Solaris, HP-UX and SGI at least have POSIX ACLs, which will suffice. The team is aware of the problem, but can't do much if the underlying OS doesn't have the functionality..."

Michael understood, but said that Linux was, for his company, the main reason to use Samba in the first place -- because of the cost. Dave, as a "Semi-serious suggestion from my employer" (Sun), suggested Solaris/x86 which can be obtained for the cost of the media. He continued, "In fact, you have an unusual case, one which Thompson and Ritchie consciously left out. Their replacement (groups) was insufficient. They really didn't want to get into "security aware" OSs, ones in which the users were expected to want to blow away each other. Unix can be adapted to this case, but it's hard. For years, the only "B2 Secure" OS was Multics, which ran on $1 Million hardware..."

Someone suggested the Linux "trustees" project (http://www.braysystems.com/linux/trustees.html) to Michael, and he posted back that he was trying it out, but that it seemed to be what he needed. Jeremy Allison said that the "trustees" approach would not (at least right away) work with the Samba ACL support scheduled for 2.0.8, but as a means of access control administered independently, it would work implicitly, because Samba uses the Unix kernel for actual file access checking. In another post, he expanded on what he has said in the past concerning Samba and ACL support: "For each ACL type a mapping will have to be written from the filesystem ACLs to NT ACLs. Currently this is planned for HPUX (of course :-), IRIX, Solaris and (maybe) AIX. A mapping may be done for one of the experimental Linux ACL implementations (the one at http://acl.bestbits.at/ is probably the one we'll use) but this code is not currently in any stable or developement kernel so it will definately be a configure option on Linux." Sander Striker asked, "Is it possible to interface the api to the trustee patch?" Jeremy said maybe, depending on the kernel API. The Samba ACL support API was designed for modularity, in consideration for the differing implementations out there.

Luke Leighton, in an aside, mentioned that he had thought about doing a side project of VMS/NT-style security in the Linux kernel. Jeremy warned him, "Maintaining a security subsystem is a tricky job though, not something to be taken lightly."

 

8. Last Call for 2.0.7
1 Apr 2000 - 8 Apr 2000 (25 posts) Archive Link: "Samba 2.0.7pre3 snapshot released."
People: Jeremy AllisonDave Collier-BrownGiulio OrseroGuilio OrseroPeter PolkinghorneUsing SambaDavid Lee

Jeremy Allison posted the pre-release announcement for 2.0.7pre3 to several Samba lists. He noted that the various people in charge of keeping package management scripts up to date should send him code to deal with two or three changes in what should be installed, then added:

The final things left to do before official 2.0.7 release are :

  1. Fix any reported bugs in this release
  2. Get "Using Samba" updates for 2.0.7.
  3. Update the packaging code for systems other than RPM based systems.

So official release is "close" - please download and test this code.

Several people complained at one time or another about how the --with-smbwrapper option doesn't work. In every case they turned out to be using Linux systems, where smbwrapper is currently unsupported due to issues with current versions of glibc.

Hiroshi Miura posted two patches for better code page support, particularly for multibyte character sets. Guilio Orsero posted his usual hodgepodge of unresolved bugs, mostly with smbclient but some with other subsystems. Jeremy promptly fixed one of the Samba bugs that smbclient showed up, something about enumerating the "." and ".." entries of a directory.

Later, Jeremy merged in patches from Dave Collier-Brown to update Using Samba, and most of the binary packaging modules have been updated recently, so that mostly takes care of his to-do list above. So Jeremy posted a "last call" announcement on 2.0.7: "2.0.7 looks ready to roll, code-wise. The remaining things are to check the utmp pre3 patch to see if it works (I still need feedback on that) and for the binary package maintainers to fix the packaging scripts." At this, Norbert Püschel posted a patch for Solaris quota handling, David Bonner posted one to fix an obvious typo in UTMP support, Giulio Orsero had a rather strange browsing bug which Jeremy found and fixed, David Lee had more UTMP fixes he wanted merged, and Peter Polkinghorne pointed out that the "wide links" parameter does not behave quite as advertized. On this last one, Jeremy chose to change the documentation, at least for now. It may sound like a flurry of last-minute patches, but they are mainly for spit-and-polish issues, except for the UTMP support, which is supposed to be experimental in this release anyway.

 

 

 

 

 

 

We Hope You Enjoy Samba Traffic
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License, version 2.0.