Samba Traffic #33 For 17 Nov 2000

Editor: Zack Brown

By John Quirk  and  Zack Brown

Samba Homepage (http://samba.org) | Samba List Archives (http://marc.theaimsgroup.com/#samba) | "Using Samba" (http://samba.he.net/using_samba/) | Samba Tips (http://www.redhat.com/support/docs/tips/Samba-Tips/Samba-Tips.html) | A Samba Doc Page (http://home.germany.net/101-69082/samba.html) | Samba Meta-FAQ (http://www.uwsg.iu.edu/software/source-docs/faq/Samba-meta-FAQ.html) | Samba For IRIX FAQ (http://www.sgi.com/software/samba/faq.html)

Table Of Contents

Introduction

Want to help write KC Samba? See the KC Authorship page (../author.html) , the KC Samba homepage (index.html) , and the Thread Summary FAQ (../summaryfaq.html) . Send any questions to the KCDevel mailing list. (mailto:kcdevel@zork.net)

Mailing List Stats For This Week

We looked at 525 posts in 2242K.

There were 185 different contributors. 73 posted more than once. 0 posted last week too.

The top posters of the week were:

1. Samba's VFS layer problems

2 Nov 2000 - 8 Nov 2000 (24 posts) Archive Link: "Samba login"

Summary By John Quirk

People: David Collier-BrownKenichi OkuyamaJeremy AllisonJohn E. MalmbergBrad SahrMichael Glauche

Nagy Tamás asked why he was unable to login when his network was lightly loaded and why he had lots of broken pipe errors in his log files.

David Collier-Brown noted that this question should have been asked on the main list and not on the technical and then he replied:

However, you're seeing the client disconnct for some reason:

the problem is finding out the reason so the team can fix the $#^!!!### thing.

I'm attaching a note on diagnosing it:

anyone want to add to or improve it, oh real experts?

Here is a link to David Note http://lists.samba.org/pipermail/samba-technical/2000-November/009801.html (http://lists.samba.org/pipermail/samba-technical/2000-November/009801.html)

Kenichi Okuyama also noted that he to had seen the problem he offered several suggestions:

Make smb.conf to set Send buffer size as small as possible, down to, well, my recommendation is '1460*2'. And if network goes down even then... make ENTIRE network speed faster, i.e if Server is connected to network with 100Mbps ethernet, connect client with 100Mbps ethernet as well. If Server with 1G, Client with 1G. Don't simply make Server 1G and Client 100M.

Linux's TCP layer tires to send as much IP packet as possible, upto Send buffer size or FIFO to NIC card size. If you still have someting to send, it sleeps and wait for someone to wake him up.

Problem is, on very heavy network, once context switch occur, and other process starts running, this process ( which is usually another smbd on server ) have high possibility of 'this process sending packet to client' as well. Once this 'second' process called "send()" system call, we have no chance of re-scheduling. 'second process' will start filling 'FIFO to NIC' even though there still exists something to send on 'first process'.

This means, 'From Macro point of view, TCP/IP layer does not keep FIFO' on Linux. This will cause one smbd process replying to client more frequently than other smbd process replying to client. If your Windows client was attached to "UNLUCKY" smbd process, turn around time will become longer and longer, until finally, connection cause timeout, and Client will cut the connection.

But if you look at this from Micro point of view, what you'll see is Server sending packet to single specific client, concentrated.

If Server is connected to network in fast NIC card, like 1Gbps, while Client is connected in 100Mbps, then sombody (usually HUB) have to change transfer speed using STORE AND FORWARD functionality.

This STORE AND FORWARD functionality requires buffer, which is stored on HUB. But since buffer size is not infinite, if packet to single client is concentrated too much, this buffer will overflow, which means packet drop will occur. Packet drop will cause re-transmittion, and if re-transmittion is being caused too freqently, 'sender(Server)'s TCP layer(which exists on individual socket ) will stop sending packet, waiting for network to settle, and this will also cause specific connection between specific client to be slow. Once one connection stop sending packet, other connection have higher possibility of succeed in sending and make packet reach to client.

So, 'from Micro point of view, priority to each network is not equal on heavy loaded network'.

...

1) Make ENTIRE network fast. Not partially, like making Server fast. Partial speed up will raise higher possibility of causing 'Micro' unfairness.

2) Make individual connection speed slow. This can be done by shrinking Send buffer size. Please remind that any setting that will cause 're-sending' will only cause network even more overloaded. So, please also look at :

http://www.dd.iij4u.or.jp/~okuyamak/Documents/tuning.english.html (http://www.dd.iij4u.or.jp/~okuyamak/Documents/tuning.english.html)

Kenichi Okuyama went onto discuss bottle necks within the samba code, focusing on the stat() system call versus the fstat() call. The role of Unix to Dos file name conversions. Kenichi felt it was time was to stop development on 2.2 and focus on a clean up of the current code base.

Jeremy Allison asked:

Have you taken a look at the re-design of the number of stat calls for the Miami conference ? This should help - the reason we have to do a stat call on an incoming path is that we have to map the case insensitive name given to smbd to a case sensitive name used by UNIX.

...

The change I made in Miami is to keep the results of this first stat around and use it instead of doing further stats (as we were doing). Can you take a look at the new code in 2.2.0 and give an assessment please, as you seem to be very good at this kind of work.

....

Well, the 2.2 code structure is actually *cleaner* than the 2.0.x codebase (IMHO - and Andrew's I think).

Most of the new functionality is being added in the RPC code which is completely orthoganal to the network layer and stat work you are describing here.

I'd like to be able to improve the network layer code, and reduce the stat calls whilst also adding in the RPC functionality that people need.

I'm very impressed with your work - let me know what else we can do to help !

Jeremy and Kenichi tarded comments about the samba source for a while then John E. Malmberg asked:

Is there some way to make this either configurable as a run time or a compile time option?

For OpenVMS, the case of the file specification is ignored when the file is looked up. All of this processing is just extra overhead.

Jeremy Allison replied:

Yes indeed - just set "case sensitive = no" - then all the statcache and much of this code is bypassed. I'd be very interested in a NetBench run with these two options on and off, on a VMS box as this would give us a relative comparison on how this extra processing affects us.

Brad Sahr also added:

I've been following this performance oriented thread with great interest. The reason I'm so interested is because I see so many repetitive calls (by Samba) into my VFS implementation for the same information, when it appears to me that Samba is processing a single SMB request.

I highly recommend that Samba provide the option for a VFS to supply an enhanced readdir that also returns stat information. Depending upon the VFS implementation, this could be implemented easily. It definitely would be easy for one VFS implementation that I'm aware of.

Michael Glauche did some quick tests with an old x86 and a 50% speed improvement with "case sensitive = Yes"

Jeremy and Kenichi continued dicussing what the Statcache section of the samba code was doing in the case insentive way when finally Jeremy Allison replied:

Ah - now I understand what you think we're doing. Take heart - we're *NOT* doing that. We do exactly what you suggest, which is to scan the directory and cache the relevent mappings to a case-insensitive version of the names within it.

The stat is done to check if the mapping is still valid.

The thread continued down the path of picking over the internals the Statcache and what it this code was trying achive several suggestions where made and Jeremy said he like the suggestions. The thread finaly continued on as "avoiding stat() races (Was: RE: Samba login)"

2. tdb diskspace leak in 2.2

15 Nov 2000 - 16 Nov 2000 (4 posts) Archive Link: "2.2.0 development: tdb diskspace leak?"

Summary By John Quirk

People: David LeeJeremy AllisonSimo Sorce

David Lee posted what to him looked like a bug in the TDB code.

On our Solaris 2.6 test system, with a CVS version of Samba 2.2.0 from around October 24th, the file "unexpected.tdb" seems to be growing and growing and ... (you get the picture). A few days ago it exceeded 9MB. I restarted Samba (and cleared the file during that), but it is now up to nearly 1MB and still growing. This server itself has been almost completely dormant during this period, although it is on a busy network.

But using "tdbtool" to examine the file usually shows only one or two active records in there (occasionally a few more). The strings "\MAILSLOT\BROWSE" and "\MAILSLOT\NET\NETLOGON" seem to occur in the records.

So is there a known diskspace leak somewhere in the tdb code? (A question primarily about code development.) Hopefully the answer is "known and fixed". But if not, any hints as to how to begin to tackle its investigation?

Also, I suppose, given that this file has the name "unexpected", I ought to ask a subsidiary question (from a user-site perspective) of "Is there something that should cause concern?".

Simo Sorce noticed after an item was deleted from the database the space was not freed and the files grew.

Jeremy Allison replied

No it's not actually a leak, but a problem we need to address soon. The problem is that if a tdb keeps getting full it will allocate more space for records, which it will never free. The space is not leaked - just allocated as "free" space - like a growing malloc pool. The only way to trim it is to copy all the entries into another db and then atomically rename. Currently this is done with the unexpected db when nmbd is restarted.

The unexpected db stores all UDP 137/138 packets that Samba receives that the original code wasn't expecting (ie. not a reply to anything sent, or a packet not sent specifically to this machine). We used to throw these packets onto the floor, but we store them now due to a bug in WinNT where it replies incorrectly to UDP 138 packets - by hardcoding the reply port to 138 instead of the source port of the request.

The unexpected packet db allows nmbd to be running (and bound to port 137/138) and for client tools that are not bound to these ports to send requests out and receive the replies correctly - nmbd gets the responses and puts them into unexpected.tdb for the client tools to pick them up.

What seems to be happening in your case is that there is a flood of packets on 137/138 that nmbd is storing in the unexpected db - now periodically it does go through and delete them when they've aged too much, but the allocated space remains.

The fix is to monitor the tdb size and get nmbd to initiate trim operations when it grows too large. I'll look into this code.

David Lee thanked Jeremy for his response and asked if there was a smb.conf option to this behaviour off. If there wasn't maybe one should be added.

There where no further posts to this thread.

3. Duplicated Usernames

8 Nov 2000 - 12 Nov 2000 (8 posts) Archive Link: "duplicated usernames, unknown account in NT "permissions" dialog (2.0.7)"

Summary By Zack Brown

People: David BannonSchlomo SchapiroSteve Gonzales

In the samba-ntdom list, someone reported using Samba 2.0.7 as a PDC, and seeing multiple instances of the same username in the NT permissions dialog. Trying to operate on any one of them would appear to work, but then on opening the dialog again, instead of the username there would appear, "XYZDOMAIN\Account unknown". David Bannon replied that Head 2.1pre-alpha was known to work in that situation, when it would work at all. He added that it wasn't the most stable release, and might not even still be available from the CVS tree. But he assumed that Samba-TNG (http://www.samba-tng.org) would also work in the same way, and added, "Or just wait around a bit and see if this functionality appears in 2.2 which will, most likely be a more stable product." Schlomo Schapiro replied, "this is quite interesting, can you with this head 2.1pre-alpha also give users admin rights over THEIR computers only (each on his own) ? I never met a samba that was able to do this, all only give this "Account Unknown" stuff :-(." [...] "This is actually the biggest problem since everybody wants to be boss on his own computer and still be able to log on to all others and till now this has prevented me from using Samba as PDC here :-(" David replied that it couldn't give blanket rights, but that, "really, get them out of this idea that should be admin of their own machine, much easier ..." and Steve Gonzales agreed, "Do NOT let everyone be a local Administrator. This is asking for much trouble. If you must give a little to the users, place the global Domain Users group into each local Power Users group. This will allow people to at least add printers, etc. BUT it will not let them administer the workstation." This made sense to Schlomo, but he added, "Problem is that if I don't give people admin rights they have to re-logon all the time for installations." Earlier, Schlomo had asked when 2.2 might be out, and David had replied, "nothing sure. Its something we would all really like though."

4. Status Of DocBook Conversion

8 Nov 2000 (5 posts) Archive Link: "Where does the DocBook converstion stand?"

Summary By Zack Brown

People: Gerald CarterMark KomarinskiDavid Bannon

In the samba-docs list, Mark Komarinski said he'd been given the time to help out on converting Samba docs into DocBook, and asked what he could do. Gerald Carter replied:

David Bannon is working on combining various FAQ's. I think ther first thing we need is an outline / HOWTO for Samba documentation. This would be very similar to the LDP Author HOWTO of course.

My purpose for this would be to remove as many questions as possible about getting started with Documentation. Thoughts? Comments? What formats should be included?

The next big thing after this is the man pages.

Mark Komarinski remarked:

I'm not sure if there's a DSSSL for creating man pages. Maybe some sort of DocBook->TeX->*roff chain?

HTML and .txt are going to be the easiest ones to generate. Using Jade and OpenJade, there are issues getting PS and PDF directly from the DocBook source. The LDP is using HTMLDOC, GPL software from the makers of CUPS, to go from HTML to PS/PDF.

5. Messing Up Windows Desktops

10 Nov 2000 - 9 Nov 2000 (3 posts) Archive Link: "Sharing profiles between NT4 and W2K on Samba 2.2"

Summary By Zack Brown

People: David BannonGreg Dickie

In the samba-ntdom list, David Bannon reported:

This one might be a bit scary. I just noticed that if you have a profile that was 'seen' by W2K, that is, you logged into a domain via W2K, there is a problem when next you logon to a NT4ws.

The network icon disappears from the control panel !! Nothing more (that I have found yet) but why on earth would that happen ?? Its not a permission thing, if you where clever enough to make a short cut to the network icon before playing with W2K you can run with it after.

I have been back and forwards a couple of times, there is no doubt that the problem is coming in via the profile, what does anyone think ??

Does this mean that profiles may not be 'shareable' between NT and W2K ? If it does this what other things will we find it doing....

Greg Dickie confirmed, "Uh ya, profiles do seem to get hosed once you login on a Win2K machine, mine now has these really fancy icons and a bunch of stuff no longer works the way it did." David added, "The key by the way is "HK_C_U\Control Panel\don't load", easy enough to find and easy enough to reverse but you will need to do it every time you go from W2K to NT. Better look at doing it in a policy..." End of thread.

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.