Kernel Traffic #269 For 19 Jul 2004

what are the requisite for stable inclusion of the UML update inside 2.6-mm (or directly 2.6.8)? Currently (splitting out a little piece, which should not be included) we have almost all the stuff inside arch/um and include/asm-um, the addition of <linux/ghash.h> and of two filesystems for UML use only, and this little hunk (plus 2 uses of it inside mm/page_alloc.c).

+#ifndef HAVE_ARCH_FREE_PAGE
+static inline void arch_free_page(struct page *page, int order) { }
+#endif

Could it go in as-is? I'm especially worried about having it included soon in 2.6.8, since last time it entered -mm and stayed there just for one release.

The patch correctly applies to 2.6.7 and works; the current code, instead, does not even compile at all, so there is no reason for not applying it (unless you want to remove UML support / but since you never said this, we need this patch applied). However, if you don't want some parts of the code, just tell me; I'm waiting for this before preparing the UML patch to send you

Also, I have some patches managed with your patch-scripts, which I'll send you after you include the UML patch.

About the STATE of the code:

Of the two filesystems, one (hostfs) now should work perfectly with 2.6 (I've just fixed one porting bug to 2.6, related to the force_delete() -> .drop_inode change documented in Documentation/filesystems/vfs.txt); the other maybe has some problems, but I can remove it from the patch (it also will probably be replaced soon by a more generic one, i.e. externfs).

I have no problem plopping it into -mm, as long as it doesn't cause me too much pain. It did cause patch management pain last time, but probably whatever is was interacting with has now been merged up so it'll be OK.

But for a merge into mainline we do need to get down and do some work on it - reintroducing ghash.h would not be welcome (I though Jeff was going to eliminate that?) and last time we looked the patch had some blockdev drivers in it which were doing antiquated 2.4 things.

Generally, UML in 2.6 seems to have fallen behind fairly seriously and at some stage we need to go through the exercise of splitting the patch up, reviewing and fixing all the bits and feeding it in.

Yup. I've come to the conclusion that I've painted myself into a corner a bit with BK and my currently style of working. I'm looking at quilt, and I'm pondering taking all the changes since the last time Linus merged UML (2.5.69 or something), and breaking them up into sensible patches.

That'll be a lot of work, but I think it's something that needs doing.

Good tool.

It's a bit like a loaded gun with no safety. You will learn a few new ways to shoot your foot off, and become good at first aid. You will want someway to keep personal revision history of your patches, to aid in such repair work. CVS or RCS or local bitkeeper or (for ancient hackers like me) raw SCCS or some such. Quilt handles the patches, but in and of itself has nothing to do with preserving history.

All software is divided into two parts - the concrete and the fluid.

Once something is accepted into the main kernel, it's concrete. You can never go back - you can only layer fixes on top. Bitkeeper rules for this stuff.

But work in progress, for which oneself is still the primary source, is fluid. You can slice and dice and redo it, and indeed you want to, to get the best patch set. Quilt and friends rule for this stuff.

Conclusion - use Quilt (with your favorite personal version control) on top of Bitkeeper.

Question - what tools are available for convenient patch set submission? Composing multiple, related email sets in a GUI emailer is a bit tedious and error prone. It's an obvious candidate for scripting.

The concepts behind quilt are all stolen from patch-scripts, so it has the same usability problem that patch-scripts has: forgetting to add a file to a patch before modifying it is painful. the ``quilt edit'' command helps somewhat. I do not have a good idea how to fix this in a more satisfactory way.

Quilt is missing some of the features of patch-scripts: there are no equivalents to export_patch, which renames exported patches so that the filename sort order equals the order of the patches in the series file. Neither is there a way to strip such sequencing prefixes when importing patches. (I consider this obsolete.) There is nothing kernel specific, and nothing specific to version control systems. Also there are no equiovalents to patch-scripts's new-kernel, mv-patch, patch-bomb, pstatus, rename-patch, tag-series, unitdiff.py commands.

On the other hand there are lots of small improvements, no more patch control files (that list the files a patch touches in patch-scripts), improved diffing and status inquiry functionality, patch dependency analysis, support for RPM packages. And there is more documentation.

Things I'm currently considering include:

• handling of meta-information such as one-line summary, author, description,
• support for diffstat (re-add; we had this in older versions),
• a patch-bomb equivalent.

All of the above things will potentially conflict with the goal of keeping the whole thing as policy-free and generally useful as possible.

At http://www.hyades-itea.org you will find a port of Adeos for Linux 2.6 to the ia64/SMP architecture, currently running on Bull's Novascale systems.

The main objective of the EU-funded (ITEA) HYADES project is to adapt standard technologies for applications that require real-time response, associated with heavy, parallel computations.

HYADES is a partnership led by the Thales Group, composed of various organizations interested in deterministic intensive computing on the Linux/ia64 platform, among which are Bull, MandrakeSoft and Dolphin. The contribution of the HYADES project to the community will extend beyond the port of Adeos, by adapting the RTAI/fusion experimental technology to fit their needs on the ia64/SMP architecture.

The Adeos/ia64 patch and others can also be found at the usual place: http://download.gna.org/adeos/patches/

Andrew Cagney discovered this problem while working on GDB. I suspect this bug has always been there, but I've only actually tested current 2.6 kernels.

When you single-step into a trap instruction, you actually don't get a SIGTRAP until the instruction after the trap instruction has also executed. I have demonstrated this in three cases: `into' generating a SIGSEGV that is suppressed via ptrace; an `int $0x80' system call entry; and a `sysenter' system call entry via the vsyscall entry point.

In https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=126699 you can find a working test program and full details on reproducing the problem using gdb.

This is documented Intel behaviour. It also guarantees that there is forward progress in some strange circumstances, if I remember correctly.

And I refuse to make the fast-path slower just because of this. Not only has Linux always worked like this, as far as I know all other x86 OS's also tend to just do the Intel behaviour thing.

If you are not concerned about existing users of PTRACE_O_TRACESYSGOOD calling PTRACE_SINGLESTEP and then being confused, then I have no objection. I consider you to be the authority on any such users there might be.

In that case, I'm happy to endorse Davide's original patch. I will look into extending it to cover x86-64's ia32 support as well.

I still wonder if anyone has any insight into why this issue does not arise for native x86-64's syscall/sysret. From my reading of the AMD64 manual, I would expect it to happen there as well. That is, sysret is the instruction that sets TF, and the manual says that the instruction after the one that sets TF gets executed before the trap. It would be convenient if sysret were a special case for this rule, since it makes it do what is best for the system call case. But I haven't found a mention of that in the manual.

I, and quite a few other folks, have been trying to get the Linux Trace Toolkit in the kernel for the past 5 years and the code being added is almost identical to what the audit patch adds, yet we've always got reponses such "this is bloated" and Linus told us that he didn't see the use of this kind of stuff.

Have we simply not figured out the secret handshake?

I'd really like to have some advice here since I believe we have tried every trick in the book: posting the patches for review, asking kernel developers for input, porting the patches to multiple architectures, modulirizing the system, etc.

The security code adds hooks everywhere too, but those deliver end-user functionality rather than being purely a developer support tool.

Developer support tools are good, but are not as persuasive as end-user features. Because the audience is smaller, and developers know how to apply patches and rebuild stuff.

This is probably one of the biggest misconception about LTT amongst kernel developers. So let me present this once more: LTT is _NOT_ for kernel developers, it has never been developed with this crowd in mind. LTT is and has _ALWAYS_ been intended for the end user.

The fact of the matter is that the events recorded by LTT are far too little in detail to help in any sort of kernel debugging. Don't take my word for it: I met Marcelo at OLS once and he recounted attempting to use LTT to track things in the kernel and how he found it NOT to be good enough for what he was doing. Ditto with Andrea.

How is this tool useful for the end user? Here's an excerpt from an e-mail I sent to Andrea and a few other SuSE folks explaining this some time ago:

What LTT is really good at, however, is to provide non-kernel gurus with an understanding of kernel dynamics. It is not reasonable to expect that every sysadmin will understand exactly how the kernel behaves and then rely on ktrace to isolate a problem (as I expect most kernel developers to be able to do). On the other hand, it is quite reasonable to expect sysadmins to be able to fire-up a tool which gives them a good idea of what's going on in a system. This may not help them find kernel bugs, but it will most certainly help them track down transient performance problems, and all other kernel- behavior-related bugs which are simply invisible to /proc, ps, and their friends.

The same goes for developers tracking synchronization problems. gdb won't help, strace won't help, etc. because they rely on ptrace() which itself modifies application behavior ... same applies to printf() etc.etc.etc. There's an entire category of problems for which current user-space tools are not adapted for and kernel debugging tools (ktrace including) are simply overkill.

Generally speaking, there isn't a single tool out there that currently exists that enables any end-user to understand the complex dynamic behavior between the Linux kernel, his applications and the outside world. And as you personally noted in the forward to Robert Love's book, the kernel is only getting more complicated. Using the trace points added by the LTT patch, the user-space utilities can provide a wealth of information to the end-user that he cannot possibly collect in any other way.

As you can see, the granularity of the details is not refined enough for any sort of kernel debugging, yet it is clear that an end-user or an application developer can benefit immensly from such information. Given the ever increasing complexity of the kernel, the ever increasing number of applications run on servers and workstations, and the ever increasing use of Linux in time-sensitive applications such as embedded systems, it seems to me that this type of capability is no less necessary then ptrace().

I'll conceed that LTT may be of some benefit for some driver developers in some cases and that it may help consolidate the slew of tracing mechanisms already included in the kernel as part of various drivers and subsystems, but the fact of the matter is that it is of little use for kernel developers. If a kernel developer needs tracing, he should be using ktrace.

ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.7/2.6.7-mm5/

• bk-acpi.patch is back. If devices mysteriously fail to function please try reverting it with

  patch -p1 -R -i ~/bk-acpi.patch

• last call for reviewers of the perfctr patches.
• s390, ppc64, ppc32 and IDE updates.

Rule Set Based Access Control (RSBAC) v1.2.3 has been released! Full information and downloads are available from http://www.rsbac.org

We are also proud to announce the relaunch of our Website and a set of worldwide mirrors.

RSBAC Key Features:

• Free Open Source (GPL) Linux kernel security extension
• Independent of governments and big companies
• Several well-known and new security models, e.g. MAC, ACL and RC
• Control over individual user and program network accesses
• Any combination of models possible
• Easily extendable: write your own model for runtime registration
• Now includes on-access virus scanning with Dazuko interface
• Support for current kernels in 2.4 and 2.6 series
• Stable for production use since January 2000

Between the first upload and this announcement, the first important security bugfixes had to be released, too, which also apply to previous versions. You can always find the latest bugfixes at http://www.rsbac.org/download/bugfixes, they are already included in some of the pre-patched kernel sources (-bfX) at http://www.rsbac.org/download/kernels/v1.2.3/

New features in RSBAC v1.2.3:

General

• Port to 2.6 kernel series with many internal changes
• Full log separation between system and RSBAC log
• Improved hiding of unaccessible processes

AUTH

• Learning mode, global and per-process

RC

• System boot role, now separate from root's role
• Extra process type for kernel threads for explicit access control
• Types for user objects

DAZ

• New 100% compatible Dazuko (www.dazuko.org) module
• On-access scanning through user space antivirus daemons
• In-kernel scanning result cache, speeding it all up significantly

ACL

• Global learning mode

PAX

• New PaX support module

JAIL

• Several security related and other bugfixes (it is strongly recommended to update)
• Linux capability restrictions for jailed processes

MAC

• Trusted-for-user list instead of single value

Please forward this announcement to where you think it is applicable, e.g. local or national security lists, newspapers or magazines, or your favourite Internet forum.

the number of changes this time is pretty small.

It includes network update from davem, PPC fcc enet driver fix, and most importantly, the missing chown() security checks which allowed users to change the group affiliation of arbitrary files on the system.

We are pleased to announce releasing new version of Linux Kernel State Tracer.

The Linux Kernel State Tracer(a.k.a. LKST) version 2.1.0 has been released. This version can be applied to linux-2.6.6. And platforms of LKST 2.1.0 are both IA32 and IA64.

LKST is a tool that supports to analyze of fault and evaluate for kernel. Especially it is usuful for analyzing the unanticipated fault of kernal.

The latest version of the LKST has new features:

• LSM hooking addon module. If you use this module, you can trace more detailed information of system calls. And you can use this module with SELinux.
• event flags which show the attribute of an event. You can get the attribute of events from /proc/lkst_etypes For more details, see howto.txt.

For more changes, see Changelog-2.1.0.txt <http://prdownloads.sourceforge.net/lkst/Changelog-2.1.0.txt?do wnload>

Remarks

For supporting IA64, we hacked KernelHooks.

LKST binaries, source code and documents are available in the following site,
https://sourceforge.net/projects/lkst/
http://sourceforge.jp/projects/lkst/

We prepared a mailing list written below in order to let users know update of LKST.

lkst-users@lists.sourceforge.net
lkst-users@lists.sourceforge.jp

To subscribe, please refer following URL,

http://lists.sourceforge.net/lists/listinfo/lkst-users http://lists.sourceforge.jp/mailman/listinfo/lkst-users

And if you have any comments, please send to the above list, or to another mailing-list written below.

lkst-develop@lists.sourceforge.net
lkst-develop@lists.sourceforge.jp

dmraid 1.0.0-rc1 available at http://people.redhat.com:/~heinzm/sw/dmraid/ in source and i386 rpm.

dmraid (Device-Mapper Raid tool) discovers, [de]activates and displays properties of software RAID sets (ie. ATARAID) and contained MSDOS partitions using the device-mapper runtime of the 2.6 kernel.

The following ATARAID types are supported on Linux 2.6:

Highpoint HPT37X
Highpoint HPT45X
Promise FastTrack
Silicon Image Medley

These ATARAID types can be discovered only in this version:
Intel Software RAID
LSI Logic MegaRAID

Please provide insight to support those metadata formats completely.

Thanks.

See file README, which comes with the source tarball for prerequisites to run this software and further instructions on installing and using dmraid!

Call for testers:

I need testers with the above ATARAID types, to check that the mapping created by this tool is correct (see options "-t -ay") and access to the ATARAID data is proper.

You can activate your ATARAID sets without danger of overwriting your metadata, because dmraid accesses it read-only unless you use option -E with -r in order to erase ATARAID metadata (see 'man dmraid')!

This is a release candidate version so you want to have backups of your valuable data *and* you want to test accessing your data read-only first in order to make sure that the mapping is correct before you go for read-write access.

The author is reachable at <Mauelshagen@RedHat.com>.

For test results, mapping information, discussions, questions, patches, enhancement requests, free beer offers and the like, please subscribe and mail to <ataraid@redhat.com>.

LTP-20040707

• Added a new test for bind() written by Dan Jones.
• Jacky Malcles added support for ext3 and some cleanup code.
• Fixes to fix DMAPI defect
• Changes for eliminating dmapi.h
• Applied patch from Gary Williams to change malloc() to calloc() b/c some archs don't like the use of uninitialized memory.
• Fix typo and change i to a 1 in the bufcmp function in diotest_routines.c
• Applied patch from Gary Williams that added an optional forth arguement to semctl as a union, not a pointer to pointer, b/c pointer to pointer causes ppc to explode. Union will automagically interpret the union as a pointer as necessary....now works on multiple archs.
• Made sure that the shm segment is cleaned up if the shmat() fails.
• Applied patch from Wu Zhou to correctly cleanup in case of a failure.
• Added definition for SHM_HUGETLB for cases where this is not defined.
• Applied patch from Steve Hill and Gary Williams for MIPS.
• Applied a timing fix to allow the test to run on more architectures.
• Applied results cleanup patch from Gary Williams.
• Corrected the logic in the test to use -lepoll or not.
• Applied PASS message cleanup patch from Gary Williams
• Fix invalid syntax "if undefined" in modify_ldt tests
• Applied patch from Gary Williams for personality() tests to initialize PER_LINUX so we can clearly see if the desired changes occur.
• Updated to Posixtestsuite-1.4.3