Kernel Traffic #297 For 19 Feb 2005 By Zack Brown Table Of Contents * Standard Format * Text Format * XML Source * Czech Translation * Mailing List Stats For This Week * Threads Covered 1. 10 Jan 2005 - 28 Jan 2005 (70 User-Space OOM Killer posts) 2. 21 Jan 2005 - 27 Jan 2005 (26 Linux 2.6.11-rc2 Released posts) 3. 24 Jan 2005 - 27 Jan 2005 (132 Linux 2.6.11-rc2-mm1 Released posts) 4. 25 Jan 2005 - 28 Jan 2005 (3 Status Of 802.11x Support posts) 5. 27 Jan 2005 - 1 Feb 2005 (16 Removing eepro100, xircom_tulip_cb, posts) And iph5526 Drivers 6. 30 Jan 2005 usbutils Version 0.70 Released Mailing List Stats For This Week We looked at 1869 posts in 11MB. See the Full Statistics. There were 616 different contributors. 252 posted more than once. The average length of each message was 99 lines. +-----------------------------------------------------------------------------+ |The top posters of the week | The top subjects of the week were: | | were: | | |----------------------------+------------------------------------------------| |91 posts in 623KB by Adrian |70 posts in 332KB for "User space out of memory | |Bunk |approach" | |47 posts in 159KB by Greg KH|62 posts in 396KB for "2.6.11-rc2-mm1" | |40 posts in 159KB by Vojtech|56 posts in 316KB for "Memory leak in | |Pavlik |2.6.11-rc1?" | |38 posts in 199KB by Matt |44 posts in 217KB for "thoughts on kernel | |Mackall |security issues" | |36 posts in 221KB by Evgeniy|41 posts in 195KB for "2.6.11-rc2-mm1: SuperIO | |Polyakov |scx200 breakage" | +-----------------------------------------------------------------------------+ These stats generated by mboxstats version 2.2 1. User-Space OOM Killer 10 Jan 2005 - 28 Jan 2005 (70 posts) Archive Link: "User space out of memory approach" Topics: OOM Killer People: Mauricio Lin, Marcelo Tosatti, Edjard Souza Mota Mauricio Lin said, "We have done a comparison between the kernel version and user space version and apparently the behavior is similar. You can also get this patch and module to test it and compare with kernel OOM Killer. Here goes a patch and a module that moves the kernel space OOM Killer algorithm to user space. Let us know about your ideas." Marcelo Tosatti found the idea interesting, though he had no specific comments on the code itself. He did say, "The userspace OOM killer is dangerous though. You have to guarantee that allocations will NOT happen until the OOM killer is executed and the killed process is dead and has its pages freed - allocations under OOM can cause deadlocks. "OOM-killer-in-userspace" is unreliable, not sure if its worth the effort making it reliable (mlock it, flagged as PF_MEMALLOC, etc)." But he replied to himself, "Actually its only unreliable if its called from OOM time. The case here is you have a daemon which periodically writes to /proc/oom?" Mauricio replied, "Yes, let me explain the idea. When the memory consumption reaches a percentage of usage, as 98% or something like that, we call this as red zone. So when red zone is reached, the ranking algorithm is started to select which processes could be killed whe out of memory happens. If the memory comsumption is less than this percentage (not in red zone), the ranking algorithm is not executed. So we have a loop the check the memory comsumption all the time and if the red zone is reached, the ranking algorithm is started before the system get the out of memory state." Edjard Souza Mota also pointed out that this patch only moved the OOM killer's ranking algorithm to user-space, not the killer itself. He said, "In that sense the approach is different and might be worth testing, mainly for cases where we want to allow better policies of ranking. For example, an embedded device with few resources and important different running applications: whic one is the best? To my understanding the current ranking policy does not necessarily chooses the best one to be killed." This made more sense to Marcelo, and he felt the patch was more worth considering in this case. Folks continued discussing the technical details for awhile, with no real consensus on what the best solution would be. 2. Linux 2.6.11-rc2 Released 21 Jan 2005 - 27 Jan 2005 (26 posts) Archive Link: "Linux 2.6.11-rc2" Topics: Kernel Release Announcement People: Linus Torvalds, Sytse Wielinga, Christoph Hellwig, Udo A. Steinberg Linus Torvalds announced Linux 2.6.11-rc2, saying, "Ok, trying to calm things down again for a 2.6.11 release. Tons of small cleanups, annotations and fixes here. Driver updates, cpufreq, ppc, parisc, arm.. Pls check that I got it all." Udo A. Steinberg posted a netfilter compile-time bug, which Martin Josefsson diagnosed and fixed; Udo urged Linus to accept the patch, but Linus replied, "Please go through Davem, he's quite responsive, but prefers things like this to be sent to the netdev mailing list too if it hasn't been there already (netdev@oss.sgi.com)." Elsewhere, Sytse Wielinga said, "Linus, could you please put skb_copy_datagram back in place? It's not used anymore in the kernel, but the vmnet module (in vmware) still uses this interface to skb_copy_datagram_iovec." He posted a patch, but Christoph Hellwig replied, "Just fix vmware. Or upgrade to a fixed version" . 3. Linux 2.6.11-rc2-mm1 Released 24 Jan 2005 - 27 Jan 2005 (132 posts) Archive Link: "2.6.11-rc2-mm1" Topics: Kernel Release Announcement, Version Control People: Andrew Morton, Brice Goglin, Florian Bohrer, Dave Jones Andrew Morton announced Linux 2.6.11-rc2-mm1, saying: ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.11-rc1/ 2.6.11-rc2-mm1/ * Lots of updates and fixes all over the place. * On my test box there is no flashing cursor on the vga console. Known bug, please don't report it. Binary searching shows that the bug was introduced by cleanup-vc-array-access.patch but that patch is, unfortunately, huge. Brice Goglin reported: X does not work anymore when using DRI on my Compaq Evo N600c (Radeon Mobility M6 LY). My XFree 4.3 (from Debian testing) with DRI uses drm and radeon kernel modules. Instead of the usual gdm window, I get a black or noisy screen (remaining image parts of last working session). The mouse pointer works. Sysrq works. But Caps-lock doesn't work. The machine pings but I can't ssh. I don't know exactly what's happening. I don't see anything interesting in dmesg. Removing DRI from Xfree config (even if drm/radeon modules are loaded) makes X work again. Linus' 2.6.11-rc2 works fine. Florian Bohrer had the same problem, with the NVidia driver. He said it "seems that AGP is totally brocken." Close by, Dave Jones took responsibility for the problem, and asked folks to drop the agpgart-bk update for the moment. A couple of hours he posted a patch, but he added that there were still problems remaining. Brice gave it a try, and reported 100% success. He offered to help with any further tests Dave wanted; and Dave said, "It's quite remarkable that it works at all." 4. Status Of 802.11x Support 25 Jan 2005 - 28 Jan 2005 (3 posts) Archive Link: "Where Linux 802.11x support needs work" Topics: Bug Tracking, Ioctls People: Dan Williams, Dave Jones Dan Williams said: This list of stuff that should get fixed in Linux wireless grew out of my attempt to put a GUI on top of Linux wireless with NetworkManager (http:// people.redhat.com/dcbw/NetworkManager). This isn't, of course, a demand or anything, and I've been personally slowly fixing stuff up as I come to it (orinoco merge, fixing linux-wlan-ng, small kernel wireless driver patches), but I don't think anyone has posted a comprehensive list of where Linux wireless currently falls a bit short. I think the biggest issue here is that the Wireless Extensions API has stagnated a bit, and driver writers have gone off and done their own thing (for example, WPA support) because the WEAPI hasn't shown leadership in this area. That's fixable, and at this point doesn't seem to be a large amount of work since the main offender here is only WPA. Second, there are, for historical reasons most likely, areas where the WEAPI has multiple methods of encoding data to/from user space. For example, WE Quality values and WE Frequency/Channel values. Quality is either signed or unsigned 8-bit number, which (I believe) is either a raw dBm/rssi value or a percentage value, respectively. Frequency uses exponent & mantissa notation, OR a channel # stuffed into the exponent/mantissa structure. Things like that. Comments appreciated, and hopefully this may spark some wider effort to get a few things fixed. So without further ado, here's the list: * Quality values vary wildly or are absent 1. atmel doesn't return any quality data from scanned APs 2. ipw_2100 doesn't return _any_ quality data (as of v1.02) 3. Different quality methods for almost every driver o Prism54 does a quality as a percentage o airo mixes use of absolute and relative values in dBm o Average and max quality levels for almost all drivers are artificial and don't come from the the card in any way Work Item: normalize quality values. Wireless extensions supports two different types of quality data, either percentage or dBm. PICK ONE. I would recommend reporting only a Percentage value to user space with the SIOCGIWSTATS call, and having separate ioctl() calls for getting specific dBm/noise values if user-space applications _and_ the driver supports it. We cannot have user-space applications guessing which of 3 different quality algorithms the driver is reporting. * Frequency values vary wildly from iw_get_range 1. prism54 uses completely different exponent values than airo 2. airo, atmel, orinoco are the same Work Item: Normalize frequency values between wireless cards. Use actual frequencies in MHz rather than using Exponent & Mantissa format as now. Force user-space applications to convert channels->frequencies, based on what frequencies the driver says it supports. Or, fix drivers to report Frequency<->Channel pairs when they report their supported frequencies, but the point again is to PICK ONE and make all drivers do that. Remove the guessing-game from user-space and pick one API for drivers to use. * airo/prism54 seem to have problems with ip6 and cause panic 1. Some drivers don't NULL out their data after they are done with it, causing kernel panics later on down the line. See Red Hat bugzilla # 135432 for details, Dave Jones has a patch for the airo driver that seems to work better, which is in Red Hat 2.6.10 kernels. Work Item: Make sure all drivers dispose of and NULL out their data when they close, or fix kernel areas that might depend on that stale data. Or whatever the problem is. * Not all drivers have correct netlink support, if they even have it 1. orinoco is too twitchy, sends too many events (shouldn't send them during a scan for example) 2. atmel, airo, and others don't seem to have any netlink support Work Item: fix all drivers to ensure that when the card successfully associates with an access point, that it signals the kernel that its network link is "up". * Not all drivers support wirless scanning 1. orinoco driver mainly, support is upstream and is being slowly merged into the kernel driver Work Item: Speed up merge of upstream Orinoco into kernel orinoco * Firmware issues 1. Cisco aironet firmware upload is quite inconsistent, fails with 5.21 for example. Firmware <= 5.02 seems to be required for using WEP with most access points. Latest Cisco-provided driver is quite different than latest in-kernel driver Work Item: Figure out licensing issues between Cisco-provided driver for 2.4 kernel (which is MPL) and in-kernel airo driver (which is GPL). Then, figure out what changes were made to the Cisco-provided driver to support firmware up to 5.30.17, and make those changes in the in-kernel airo driver. * Ethtool support for all drivers 1. viro has done a lot of them, not sure if this is complete. * Ad-Hoc mode support is quite flaky or absent from most drivers 1. prism54 "mgmt tx queue full" errors on otherwise-working cards 2. madwifi resets bitrate to 0 when switching to ad-hoc mode Work Item: Fix drivers to support Ad-Hoc mode, attempt to get specs on their hardware & registers from manufacturers if we don't have that information yet for all "modern" cards. * WPA support is lacking or just in-progress, needs much help 1. The point here is that Wireless Extensions API has severely lagged behind the capabilities of current chipsets. There should be support _in_ Wireless Extensions for WPA and its associated technologies, instead of what all the drivers do now, which is separate, non-standard, private ioctl() calls for WPA settings. Work Item: standardize on an interface for WPA and its associated technologies, and implement that interface in Wireless Extensions API. Fix all drivers to use that API rather than private ioctl() calls. Some drivers that support WPA: atmel, madwifi, prism54, ipw2200. It would also be beneficial in this effort to support the calls that 802.1x stacks need (like wpa_supplicant and Open 802.1x) so that they don't have to patch the drivers (Open 802.1x) or create special per-driver hook modules (wpa_supplicant) to be able to capture the necessary authentication packets or set up the card's WPA settings. * Drivers deal with hidden ESSIDs differently 1. ipw2x00 traps " " and runs of \0 and changes it to "" in the driver, while other drivers just pass the blank string through Work Item: Standardize all drivers to simply pass an empty string through to user-space when the base station does not broadcast its ESSID. Drivers should _not_ be clever about this. Levels of Importance (my opinion): 1. All drivers _MUST_ support wireless scanning (*cough* orinoco *cough*) 2. WPA support needs to be standardized in Wireless Extensions 3. Consistent (and present) quality data among drivers, both for currently connected AP and for scanned APs 4. rtnetlink link notification for all drivers when they associate with an AP 5. Ad-Hoc mode support 6. Ethtool support 7. Cisco firmware issues 5. Removing eepro100, xircom_tulip_cb, And iph5526 Drivers 27 Jan 2005 - 1 Feb 2005 (16 posts) Archive Link: "[ANN] removal of certain net drivers coming soon: eepro100, xircom_tulip_cb, iph5526" Topics: Networking People: Jeff Garzik, Christoph Hellwig, Greg KH Jeff Garzik said: Though this has already been mentioned, I thought I would send out a reminder. The following net drivers are slated for removal "soon", in the next kernel version or so: 1. iphase (iph5526 a.k.a. drivers/net/fc/*) Been broken since 2.3 or 2.4. Only janitors have kept it compiling. 2. xircom_tulip_cb Unmaintained, and does not work for all xircom 32bit cards. xircom_cb, on the other hand, works for ALL xircom 32bit cards. 3. eepro100 Unmaintained; users should use e100. When I last mentioned eepro100 was going away, I got a few private emails saying complaining about issues not yet taken care of in e100. eepro100 will not be removed until these issues are resolved. On iphase, Christoph Hellwig pointed out that not even janitors kept it compiling, and in fact that it had been unable compile for over two years. Greg KH suggested adding all of these projects to the list of deprecated drivers in the Documentation/ directory. 6. usbutils Version 0.70 Released 30 Jan 2005 Archive Link: "ANNOUNCE: usbutils-0.70" Topics: FS: devfs, Hot-Plugging, Modems, Networking, PCI, USB, Version Control People: David Brownell David Brownell said: WHAT The "usbutils" package is most useful for the "lsusb" utility, which can provide considerable detail about the USB devices connected to your Linux system. (It's like "pciutils" is for PCI.) When making bug reports, or otherwise troubleshooting, "lsusb -v" output is very useful; often more so than /proc/bus/usb/devices output. For folk using Linux 2.4 kernels, this also provides "usbmodules" which can help with the "coldplug" problem: setting up devices that are connected before the OS is capable of running all of the necessary "hotplug" programs. WHY The last official release was version 0.11 in August 2002. This version should incorporate almost all of the bug fixes and patches that have been floating around since then. If you're using patches that haven't been merged, please resolve that. This "lsusb" version is aware of USB 2.0 features; the 0.11 version only understood USB 1.1 functionality. This version understands many more types of descriptors; it previously understood primarily audio and HID descriptors. * Communications Device Class (CDC) descriptors, for USB Modems, some Ethernet style links (such as cable modems), and many PDAs. * Hub descriptor. Not all hubs are equal, and previously only kernel CONFIG_USB_DEBUG messages showed how they differ. Older versions of "lsusb" only showed descriptors for some hubs; this shows them for all hubs, and also current status of each port. * Device qualifier. If a device supports high speed USB, it has one of these; otherwise, it doesn't. * Chip Card and Smart Card interfacing devices. * USB On-The-Go (OTG) devices. Starting to appear; some run Linux. * USB Debug devices. Currently exotic. * Unrecognized descriptors are dumped in hex; some previous versions discarded them, which made troubleshooting painful. Note that if the kernel HID driver is bound to a device, lsusb can't show its descriptors. You can workaround this by removing the "usbhid" module before running lsusb against that device. This version uses the system's version of "libusb", rather than its own private copy. That involved some API changes. It also accounted for the only loss of functionality: lsusb doesn't currently list the language strings supported by the device. Many devices don't seem to bother with anything beyond English. WHERE Download from one of the SourceForge mirrors for the Linux-USB project: http://sourceforge.net/project/showfiles.php?group_id=3581&package_id=142529 Also, current source is in CVS for the Linux-USB project at SourceForge. If you have any patches (support for CJKV strings?), prepare them against CVS and post them to the linux-usb-devel list. HOW It uses GNU autoconf, so configure and build it: $ tar xf usbutils-0.70.tar.gz $ cd usbutils-0.70 $ ./configure <-- see below for options $ make $ ./update-usbids.sh <-- optional $ su -c "make install" $ Significant options to "configure" include: --prefix=/ To replace the normal /sbin version of lsusb, rather than /usr/ local. --enable-usbmodules If you're using a Linux 2.4 based system or otherwise not using "udev" for coldplug. Sharon And Joy Kernel Traffic is grateful to be developed on a computer donated by Professor Greg Benson and Professor Allan Cruse in the Department of Computer Science at the University of San Francisco. This is the same department that invented FlashMob Computing. Kernel Traffic is hosted by the generous folks at kernel.org. All pages on this site are copyright their original authors, and distributed under the terms of the GNU General Public License version 2.0.