Debian Traffic #8 For 26 Oct 2000

Editor: Zack Brown

By Steve Robbins  and  Zack Brown

Debian Home Page (http://www.debian.org) | Weekly News (http://www.debian.org/News/weekly/) | Social Contract (http://www.debian.org/social_contract) | Constitution (http://www.debian.org/devel/constitution) | Policy Manual (http://www.debian.org/doc/debian-policy/)
Developer's Reference (http://www.debian.org/doc/packaging-manuals/developers-reference/) | Documentation Project (http://www.debian.org/doc/ddp) | debian-devel Archives (http://lists.debian.org/#debian-devel)

Table Of Contents

Introduction

Want to help write KC Debian? See the KC Authorship page (../author.html) the KC Debian homepage (index.html) , and the Thread Summary FAQ (../summaryfaq.html) . Send any questions to the KCDevel mailing list. (mailto:kcdevel@zork.net)

1. Kernel Sources In Binary Distribution

11 Oct 2000 - 16 Oct 2000 (41 posts) Archive Link: "What is a Kernel?"

Summary By Zack Brown

People: Ben PfaffMoshe ZadkaJacob KuntzThomas Bushnell

Moshe Zadka noticed that the kernel source was a 'required' package in Debian, which seemed insane to him. The official reason - that users compile new kernels - didn't make much sense, he thought, since users might compile any other package as well. A better idea, he thought, would be to have source .deb files for the kernel sources, so users could comile them into binary .debs and install them via normal package management. Ben Pfaff replied, "We have that already. It's called `kernel-package' and the command is `make-kpkg'." Moshe pointed out, "Yes, but for some strange reason, there is also a package called kernel-source." Jacob Kuntz clarified:

kernel-source contains the source code for a given kernel version.

kernel-package contains tools for creating custom debian packages for installing a specific kernel configuration on many machines.

for instance, if you have a cluster of webservers running debian, and they all need SMP, tulip, and ip aliasing, but not raid or nfs or ide support, you could install kernel-source, configure that kernel, then build a debian package with that kernel image using kernel-package.

Moshe said he understood this, but just felt that including the kernel sources was a strange special case, because other packages didn't have their sources included in the binary distribution, and he didn't like it. Jacob argued that all users needed the ability to compile their own kernel. He said:

it is not possible to make one kernel that will work for everyone. the resulting driver bloat would make a kernel to large to be bootable on many systems. and, we would be limited by the lowest common denominator system per architecture. would you want to run a kernel for 386 with <16 megs of ram on your quad xeon with 2 gigs of ram and all scsi devices? of course not!

taking away the ability to compile one's own kernel will take away the benefit of running a linux workalike. if you want an unusable, bloated system, i'll sell you my archive copy of windows 3.0.

By this point Moshe and Jacob were both getting a bit frustrated, and Jacob felt it was time to end the thread. Moshe replied,
I'm not getting through here, apparently. I realize that people need to compile their own kernel. My problem is that the packaging system is organized such that you have to cheat it to use a self-compiled kernel. You're telling it no kernel is installed, while you sneak off and install your kernel without it knowing about it. Then you lose: the kernel is outside the packaging system, and the kernel is a very important software package.
Some folks saw a misunderstanding in the discussion, and Thomas Bushnell explained,
You don't understand make-kpkg at all. It builds a kernel (from whatever source you like, maybe from the kernel-source package, and maybe just something you downloaded) and then it makes a .deb for that kernel which is a clean substitute for the kernel-image package.
At this point the conversation veered off into a question of how to avoid automatically upgrading from the user's custom (and presumably preferred) kernel to the latest version in the official Debian tree.

2. Star Office In Debian (And JDK Licensing Problems)

13 Oct 2000 - 22 Oct 2000 (18 posts) Archive Link: "StarOffice now GPL?!"

Summary By Zack Brown

People: Joseph CarterNicolás LichtmaierAdrian BunkChristian KurzStephen Zander

Daniel Reuter gave a pointer to Open Office (http://www.sun.com/software/staroffice/openoffice/) and asked if this meant the Star Office code might make it into Debian one day. Joseph Carter replied:

Indeed it does. I talked with people with Sun and the FSF about it yesterday. Sun is interested in helping it get packaged for Debian (and it seems I'm the one packaging it, though I probably won't be the only one working on it before it's functional and in Debian..) The people from the FSF have already been trying to get it to work and I'm hoping to have a patch from them to build it without non-free software shortly.

One problem at the moment appears to be Sun's JDK which is of course not GPL'd. I'm going to be very busy at work this week, but after that I will hopefully be able to start working on it. By that time, the major issues should be identified, though they may not yet be fixed (realize this thing takes 24 hours or more to build on a system like mine - a PIII 450 with 128 megs of RAM!)

Don't mean to sound rude, but the best way to see it packaged sooner is to help the efforts to get it building without Sun's JDK and don't bug me with silly questions like "When will it be uploaded?" Each one of those I have to read and answer will be that much less time I'll have to work on it. (Of course I fully expect that people will ignore my pleas to leave me be so I can work on it as I have time, but I've got to try anyway...)

Nicolás Lichtmaier suggested, "you should try to convince them that the fastest way to get StarOffice into Debian is releasing a Java runtime with a free license.. =)"

Adrian Bunk also replied to Daniel, saying Joseph planned to package it, but that if he didn't for some reason, Adrian would take it up. He mentioned, "The Debian package will perhaps might miss all the parts that require a JDK." Christian Kurz thought this would be a lot of work, "because a first test of Open Office showed me that configure keeps searching for a installed JDK 1.2.2. And it seems there's no way to stop configure from this only rewriting it." Several folks asked if there were not some free alternative to JDK, that would not force big rewrites on the Star Office code. Joseph said at one point, "We don't know if it can be made to work with Kaffe or perhaps Jikes or basically anything but Sun's JDK at the moment. Christopher, if I haven't gotten back to you in just over a week or so about that, please bug me about it. I've already spoken to others working on this and will get you in touch with them."

Elsewhere, Stephen Zander (JDK maintainer) remarked, "Hell will freeze over before jdk >= 1.2 gets into Debian unless Sun explicitly grants permission for third-parties to redistribute said jdk. Feel free to mount whatever letter/e-mail/in-person protests you wish to try and make this happen."

3. Permission To Package From Upstream Maintainer

13 Oct 2000 - 17 Oct 2000 (11 posts) Archive Link: "ITP: Star Office -- Open Source Office Suite"

Summary By Zack Brown

People: Clay CrouchAaron LehmannColin WatsonMarc HaberNicolás LichtmaierZed PobreJosip RodinBen ArmstrongJoseph CarterAdrian BunkAaron Lehmann

Clay Crouch announced his intention to package Star Office, and added that he'd withdraw the ITP if someone else had already claimed it. He added, "If no one disputes my ITP within the next few days, I will ask OpenOffice/Sun for permission to package, and should have a package ready by the time I recieve that permission. :^)" Adrian Bunk gave a link to one (http://lists.debian.org/debian-devel-0008/msg01789.html) or another (http://lists.debian.org/debian-devel-0010/msg00969.html) message from the archives, in which Joseph Carter or he (Adrian) would package it. Clay said this was fine, but added that if neither of them did it, he'd do it.

Aaron Lehmann pointed out elsewhere, "FYI, since it is GPL'd software, their license gives you permission to distribute it. You don't need any special permission." Colin Watson replied, "True, but I always find it nice to at least get myself onto the upstream author's radar before packaging something. It's perhaps more a case of politeness than legal requirement." Ben Armstrong thought that asking permission was the wrong way to do it. Simply notifying them of the intention to package it seemed polite enough. However, Marc Haber came in at this point, with:

The New Maintainer's Guide (http://www.debian.org/doc/maint-guide/ch-first.html#s-choose) specifically states "you should contact program's author(s) to check if they agree with packaging it."

For me, checking if the upstream author agrees is like asking for permission.

Nicolás Lichtmaier felt that the Guide was wrong, in that case.
It implies that the author has more rights than he really has (like when wget's author tried to force me to ship it without a manpage).
Zed Pobre qualified,
It's not a question of rights; it's a question of courtesy. I'd also point out that he didn't try to force you to ship it without a manpage; he got upset because you were shipping it with an obsolete manpage.
He and others added that there were valid reasons to ask the upstream maintainer for their opinion. As Josip Rodin put it,
if his program is in a highly alpha phase of development and he doesn't want a slew of bug reports all of the sudden.
Nicolás felt the Guide could be more clear on this point, and instead of asking permission, new maintainers could simply ask which version, etc., of a package would be best for Debian. Josip felt this would be overkill, and that the situation was fine as it stood.

4. Getting -dev and Build-depends Automatically

15 Oct 2000 - 16 Oct 2000 (5 posts) Archive Link: "new Debian user's thoughts on Debian installation"

Summary By Steve Robbins

People: E. Jay BerkenbiltRoland BauerschmidtJosip RodinJunichi Uekawa

E. Jay Berkenbilt, a self-described
software developer by vocation and avocation
with a long association with linux and unix, posted some thoughts about packaging and installation, after having installed Debian for the first time. Since he builds lots of stuff from source, his major grievance with the install was that
there seems to be no way to automatically get the -dev package corresponding to a runtime package
. He suggested that Debian's dependency logic should automatically
always get the packages required to BUILD anything I choose to install, and to always get the -dev packages corresponding a runtime package.

Jay was really making two separate suggestions, and to the second, Roland Bauerschmidt suggested, " Just install the -dev packages and you get what you want as they depend on the library packages. " This works fine for packages that you select yourself, if you remember to do so, but it won't help for packages pulled in by selecting something else. Also replying to Jay, Josip Rodin mentioned that, while not automatic, finding the library packages installed is not terribly hard, " they're mostly named libfoo-dev of libfooX-dev (where X is the major number from the SONAME). Try something like dpkg -l 'lib*' | grep ^.i | awk '{print $2}'. " This looks like the start of a small script to discover which library packages are missing their -dev counterpart...

Responding to Jay's first wish, an automatic method to install all packages required to build a given package, several people mentioned Build-Depends. Unfortunately, one still must extract this information and act on it by hand. The field is currently only used by the build daemons. Junichi Uekawa said, however, " Methinks it will be improved soon... For the moment, you can read debian-mentors mailing list archives to see how people are discussing the details of how to implement an "auto-builddependency-checker" and how difficult it is. "

5. Restarting Daemons After libc Upgrade

16 Oct 2000 - 19 Oct 2000 (59 posts) Archive Link: "All services that require a restart from libc6 upgrade... "

Summary By Steve Robbins

People: Ben CollinsRichard KettlewellAnthony TownsSteve RobbinsJason GunthorpeStephen ZanderHerbert XuErik Steffl

Ben Collins started off a long discussion with, " Ok, I'm tired of having to track all services that might need to be restarted after a libc6 upgrade. So here's what I am going to do. I want to require all packages that need this to declare a new reply in it's init script. " He outlined a mechanism that would allow the libc6 upgrade script to query each daemon in order to discover whether the daemon needed to be restarted. Ben's motivation was to avoid keeping a static list of services to restart " that is usually incomplete and wrong. " The policy amendment that he suggested clarifies why the restart is necessary, and the change required in init scripts:

Some daemons can be affected by libc upgrades. This usually happens if the daemon uses functions related to NSS (username, group, hostname and other name lookups). Because these functions rely on loading modules (libnss_*.so) that may not match the current libc mapped in memory with the program, it may need to be restarted when libc is upgraded. The libc package takes care of when this needs to occur, but your daemon will need to tell libc that it needs to be restarted in this case.

This is handled in the init.d script (see the section about init.d). In addition to the other options that the init.d script needs to handle, it also needs to check for "nss-check" and return "restart" in this case.

In the resulting discussion, a couple of people wondered whether this problem wasn't a bug to be fixed in libc, rather than foisting it onto all the other maintainers. Other folks argued about the mechanism details, many clearly uncomfortable about overloading the /etc/init.d script interface. Two alternatives were suggested, one being a separate script in /etc/nss-restart maintained by each package that needs it. The second alternative was to keep the weirdness out of view, with a mechanism for packages' install script to register with libc6 that they need a restart when the library changes.

At the end of the thread, Richard Kettlwell wondered:

...so when my Netscape stopped being able to find web pages (well, actually it couldn't even find my squid) after a libc upgrade, it wasn't actually a Netscape bug after all?

It's not only daemons that do name service lookups; user applications do to. I'd have thought that the solution more or less has to be on the Libc side of things, to avoid breaking user sessions.

Elsewhere, under the Subject: Another solution (was Re: All services that require a restart from libc6 upgrade... (http://lists.debian.org/debian-devel-0010/msg01202.html) , Anthony Towns speculated:

Surely there's some other way to work around this, ideally fixing the root cause not having a zillion other packages work around obscure incompatible changes in libc?

For example, insisting on libc's dynamic modules to link against *any* (past, present or future) version of libc or the other modules would fix this.

Since upstream libc seems to not care seven slightly about compatability, a possible workaround might be to distribute the old dynamic modules as well as the new ones, and (somehow) link whichever version will work. I'm not sure if that's possible, but at least adding cruft to a single package is better than adding cruft to n other packages for obscure internal reasons.

Ben Collins replied to say that Anthony didn't know what he was talking about: " When daemons are running, they carry with them the libc.so.6 that got loaded with it while the NSS modules that it uses remain on disk. Now upgrade libc.so.6, and the running daemon still has the old one loaded, but the NSS modules are new and linked against the new libc.so.6. BOOM. You can't "fix" this, it isn't broken. It is inherent in versioned libraries. " But this confused Steve Robbins, who wondered, " Isn't the point of versioning so that you can request the library with the proper ABI? If you leave the old libnss_* files on disk, why doesn't the running daemon --- with the old libc mapped --- cause the old NSS modules to be loaded? " Ben's cryptic reply --- " Because modules are loaded from disk, not memory. " --- suggests that Steve's question was not clearly phrased (we will pick up this thread later). Ben continued, kicking off the new thread with:

This brings up a good possible solution. Daemons can be required to link to the static nss libs, assuming I compile that into the -dev package. This means the modules would be static in the binary, and this breakage would not occur.

How about programs like this link with -lnss_compat? This would solve the problem by not requiring a restart at all, and keeping atleast standard name services working (maybe it needs -lnss_files -lnss_dns and -lnss_db, but that can be debated if this is initially acceptable).

Jason Gunthorpe thought that static linking was a bit ugly and wondered, " why not have a libc call like 'nss_load_modules' that dlopens every nss module that could possibly be needed at the start of the program and keeps them open. Add a call like that to the top of all the daemons and the problem goes away. " Ben rejected this suggestion, " Because then changes to nsswitch.conf will require restarting applications, or furthering this hack to check that too. Way too many problems. " Jason and Ben went into a dialogue about whether or not changes to nsswitch.conf propagate to running process, and whether Jason's proposal would solve the problem initial problem anyway. In the end, Jason demonstrated that changes to nsswitch.conf do not get propagated to running processes even with the current scheme, implying that Ben's objection is a red herring. No response from Ben.

Meanwhile, Erik Steffl, Steve Robbins, and Anthony Towns tried to clarify exactly why libc6 would not load the proper NSS libraries from disk. After some back-and-forth with Ben, it finally emerged that libc6-2.1.3, for example, will load a shared object named libnss_nis-2.so, which is a symbolic link to libnss_nis-2.1.3.so. This breaks during a libc6 upgrade because suddenly the symbolic link points to, say, libnss_nis-2.1.4.so. When asked why the libc could not simply load a fully-versioned filename, e.g. libnss_nis-2.1.3.so, Ben claimed that libc6 " can't use the *-2.1.3.so, because not all NSS modules are compiled with glibc, and even if they used that same convention, it would be next to impossible for them to stay in sync. " Anthony pointed out " Except that if libnss_strange.so.2 isn't as tightly synchronised with glibc as others (ie, you can mix and match the libnss and glibc versions), then there's not an issue. If it is tightly synchronised, then it needs to be properly versioned. " Ben made no further posts.

Responding to the posts suggesting that libc use fully-versioned NSS modules, Stephen Zander pointed out that if the daemons are not restarted at libc6 upgrade time, the old NSS files need to be left around, in case the daemon requests name service in the future. This, however, " would require fundamentally changing the packaging system as the removal of old files is intrinsic to its operation. " In response, Herbert Xu suggested that, " A trivial solution would be to open all the NSS modules at load time, " bringing us right back to Jason Gunthorpe's suggestion!

 

 

 

 

 

 

Sharon And Joy
 

Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.