Kernel Traffic #37 For 4 Oct 1999

ACPI does device enumeration, and although current systems do not require that for booting, doing the enumeration after booting (when the daemon is running) means shutting down all affected subsystems (IDE, USB, PCMCIA, ...), reinitializing the hardware using the AML method the BIOS provided for this (there is no way to circumvent this, as only this AML code knows about the ACPI specific registers in the chipset) and then restart the drivers. Also, even if we do this, we still have to go to a solid VM when hardware is available that relies on ACPI enum.

The second big drawback is that not only sleeping but also proper waking depends on a userspace program, which needs to take precautions not to get swapped out because it needs to work even when only the CPU and memory are powered. You can easily shoot yourself in the foot with that. :-)

The third thing that disturbs me is that a userspace program tells the kernel where to peek and poke. This program can easily be replaced by someone breaking in from the network, while this is harder to do for the kernel. Also, you cannot easily determine what accesses are legal because the driver needs to be able to access all hardware. By a skilled attacker, this can be used to circumvent the file system and low level harddisk drivers, or to read or write arbitrary memory. Short, a huge security hole.

This is why I favor the static solution, even if it bloats the kernel and is hard to debug.

1. device enumeration/initialization

This will not be an issue until hardware that depends on ACPI for initialization appears. That's going to be a while and, even then, boot devices will always be initialized by the system so the OS can be loaded.

Yes, with the thin driver model, device re-initialization is going to be necessary on platforms without legacy initialization. And yes, that's going to be complicated but I think it is definitely workable.

2. device sleep/wake

Right, we can't rely on user-space for turning storage devices on and off. What we can do is pre-run the AML in user-space and upload the I/O access sequence to the kernel for execution.

3. security

It's just as easy to install a new kernel or kernel module as it is to replace a system daemon. If someone has root access you are vulnerable with either model.

#1 and #2 are definitely important issues but I think there are solutions to both while keeping the bulk of ACPI code in user-space.

You are going to need a much more convincing argument to get that much code into the kernel initially. Why don't we go the thin way and as specific issues arise or new hardware appears, then push more of the code into the kernel?

It Works For Me (tm).

I have the same problem -- a C++ program that includes kernel header files -- and your patch does indeed fix my problem. If it gets into the kernel then I can take some nasty preprocessor kludges out of my source.

Here's a little background: I have a program like strace, which monitors the system calls that *other programs* make. I need to #include the entire userspace-to-kernel ABI so that I can monitor properly. Using glibc headers doesn't meet my needs at all, because I am not calling these services, I am using ptrace to watching other processes call these services.

I chose to write my program in C++ because I like OO programming.

I ran into a similar problem with the user-mode kernel, and my fix for it might be applicable. I need to #include both user and kernel headers, the kernel headers because I'm building a kernel, the user headers because the lowest levels are implemented in terms of system calls.

I found out that you are asking for a world of hurt if you include both user and kernel headers into the same file. What I did was split my sources into "user" and "kernel" files, which include only user and kernel headers, respectively. Communication between these two groups happens in terms of basic C types.

So, what might work for C++ is to have little C files which include the kernel headers, which pass their information back out to the C++ files using types that everyone agrees on.

I reviewed the time-related stuff in 2.2.12 and I found potential SMP races in the time-related syscalls. Basically the only problem is that the time-syscalls are not protecting against each other. They are only protecting themself against the timer irq. And this may lead to SMP races.

Actually I don't know if these SMP races may be the source of the RTC screwedup with ntpd enabled (also considering that ntpd seems single threaded...). Anyway here it is the fix for the races I spotted so far. It's against 2.2.12 but it applyes cleanly to all 2.2.x 2.3.x kernels out there as there aren't been NTP/time changes since 2.2.0. You may want to try to reproduce the RTC SMP race with this fix applyed (I can't reproduce in any way here, but I am not sure I configured xntp hard enough to trigger the race).

IMHO there's nothing to change into the kernel.

I don't care at all how my gcc is been compiled for, the gcc output _must_ be the same for all GCC of such same version careless about which optimizations are been used to compile gcc itself. The makefiles must ensure that the libgcc is been compiled for i386 and the gcc build process may also create a different libgcc.a for each different architecture supported by gcc and then link the right one statically into the binary in function of the flag specifyed to the gcc executable.

The output of gcc must be the same even if I'm cross compiling the kernel for i386 on an Alpha. If I choose -march=386 and some 686 assembler goes into the binary then gcc is definitly buggy and must be fixed.

So the buggy compiler must _not_ be used. If you don't fix the kernel then the debugger will be sure fixed.

You just know the workaround and you can implement it by hand in the meantime.

That's MHO at least.

  Bus  1, device   0, function  1:
    Multimedia audio controller: Neomagic Unknown device (rev 0).
      Vendor id=10c8. Device id=8006.
      Medium devsel.  Fast back-to-back capable.  IRQ 5.
      Prefetchable 32 bit memory at 0xf5800000 [0xf5800008].
      Non-prefetchable 32 bit memory at 0xfda00000 [0xfda00000].

This is my attempt at getting smbfs to work with 2.3.x.

This patch:

• adds a 'get_cached_page' with inspiration from 2.2 mm/filemap.c
• changes the locking for inode->readpage/writepage
• tries to use macros instead of setting PG_ values
• smb_write_one_page is now of type writepage_t
• debug messages don't printk non-null terminated strings (it oops'ed me ...)

This should apply cleanly to any 2.3.18 (+ac#).

I have tested this on a few machines (plain 2.3.18, ac5, ac7). Me and at least one more person have been unable to crash it so maybe it is fit for a development kernel. Oh, not only does it not crash, it shows files and allows reading & writing. :)

Ok this should fix the last few bugs that needed closing, and some small fixes for the token ring (afaik sktr still doesnt work but someone now seems to know why which is good)

I've got several other drivers and good patches that are queued. I'm going to sit on most of them until 2.2.14. The only stuff I want for 2.2.13 now is bug reports (and preferably fixes). Just bug fixes.

I'd like to get the reported network delay stuff nailed if possible and also the SMP hang reports. I'm not interested in reports that include the knfsd, nfs client or Ingo raid patches. Not because I think they are bound to be the cause but because I want less variables - ditto stuff built with gcc 2.95 or later on x86

If you see hangs on unadulterated 2.2.13pre11 I definitely want to know. If you can run the ikd patch and other tools to get dumps even better.

olibox:~ # java -version
java_ns version "1.1.7B"
olibox:~ # javac
Cannot open /proc/01129 for GC

I'm not going to make myself popular here, but the network code is *not* easy to read.

#ifdef RANT

Consider: ip_route_output() returns a route, but you wonder if you have to free it somehow? You look through ip_route_input, then ip_route_input_slow, and verify that it's using a reference count.

Looking for _release, _unref etc. gets you nowhere; eventually you find some code that uses ip_rt_put. Cool, so the networking jargon for release is xxx_put().

So skb_put releases skbs then? Not fucking likely. kfree_skb does that: of course, it doesn't actually kfree the skb unless the refcnt is 0. skb_push and skb_pull have nothing to do with putting skbs on a stack. skb_realloc_headroom is not really realloc: it returns a copy of the packet with more headroom, without freeing the old one. Of course, the definition for `struct sk_buff' is not in `sk_buff.h', but `skbuff.h'.

Consider tracing the code path of an IP packet from one NIC to another involves 3 chained strategy functions (unless it's fast routed!): packet_type's descriptive `func()', the routing code's output(), and then the queuing disc's function if there is one.

Figuring out the packet_type's func is a simple matter of finding how they're registered (dev_add_pack of course!) and looking for it in ipv4/, to see that it's going to be ip_rcv().

Figuring out how the packet gets from ip_finish_output2 to dev_queue_xmit is something I've NEVER managed to do; grepping for dev_queue_xmit or hh_output gives nothing I can see, neither does the desparate resort of grepping for `output'.

But if you manage to get through that, can you figure out that the queuing discipline on the device has a queue: ethertap.c: doesn't set it statically, or in init_module, nor register_netdev(), try ethertap_probe(), no, but it calls net_init.c's ether_setup(), which doesn't set it either. grep'ing the directory reveals that noone mentions `qdisc'.

By guessing, I look in sched/sch_generic.c: my, this dev_init_scheduler() looks like it might be something, and grepping again finds it.

After all this, there's NO WAY you're going to bet that you didn't miss something, or get it wrong: trying to figure what fields are going to be valid in the skb at the end (and which of those is coincidence which won't be true in the next kernel version) is not realistic.

#endif /*RANT*/

There are *reasons* why none of the external add-ins to the networking stack have never been reliable worth a damn (masquerading, CIPE, Free/SWAN, redirection etc) and there have been holes in our packet filtering (early 2.0 iirc). The network code simply isn't readable on any scale larger than a single function, and it's not documented.

I have a huge respect for the hacking skills of the IP maintainers, but don't be under any illusions that outsiders can make mods to the 500,000 lines of networking code. They can't.

Changing the names now would be a crazy idea, but a few more comments in the code wouldn't go amiss.

A lot of the networking code is conceptually simple. There's a lot of it, which makes it tedious, but with a little time and patience, the code can be followed. However, every so often, you hit a function pointer, such as dst_entry.input or dst_entry.output. The problem with function pointers is that once you dereference one, to understand the code, you need to find where and why the pointer was set. A few lines of comments pointing people in the right direction would be incredibly helpful. Without them, the code has the property that to understand a small part of the code, you need to understand _all_ of the code, which is a bad property, both for reliablility and for security. I don't agree with Alexey's argument that the complex code is just an exam; I certainly wouldn't push for changing the code to make it easier for newbie programmers, but the existing code is tough even for experienced programmers.

Even if you have read all the code and understand what the existing code does right now, it is impossible to deduce from the code what the expected semantics of an interface are. This makes it difficult to write code that will stay working in future versions of the kernel. Without documentation, you have to understand how different parts of the kernel interact with each other, and as the code gets bigger and acquires more subsystems, this becomes a much harder task - the complexity of the interactions goes up much faster than the size of the code. With documented interfaces, it would be possible to find bugs by checking that the code on both sides of an interface obey the documented semantics and don't rely on undocumented semantics. Note that I'm not advocating setting kernel internal interfaces in stone, but I do want interfaces to be more obvious so people think harder about the consequences for other people's code when they change the semantics.

I've been following this code / documentation thread for a while.

Here's a few points for consideration:

1. Incorrect documentation is _far_ worse than no documentation.
2. Code gets updated, docs don't
3. Documenting interfaces is important, documenting implementation is a great deal less important.
4. Terseness is good.

Given all that, you might consider documenting just the internal interfaces and the semantics of those interfaces. For instance, what are the interfaces to an inode? What are the semantics of those interfaces? What's the locking model used from without the interface and from within the interface (they are frequently different).

Diving into a ton of details about implementations specifics is going to do nothing but clutter up the code. On the other hand, a set of docs which defined the interfaces to the following set of kernel "objects" would be something that a lot of people would use:

• files
• file systems
• block, char devices
• network devices
• scsi devices (and layers)
• processes
• sockets
• VM - both the VM address space and the page cache

That's probably an incomplete list but it's a good start. I personally would love to see just the interfaces documented - I could care less about the implementation - my experience is that the implementation tends to change and the docs tend to get out of date (and hence much worse than nothing because they are misleading). But interfaces tend to live for a long time.

TESO Security Advisory 26/09/1999

Linux Kernel 2.2.x ISN Vulnerability

Summary

A weakness within the TCP stack in Linux 2.2.x kernels has been discovered. The vulnerability makes it possible to "blind-spoof" TCP connections. It's therefore possible for an attacker to initiate a TCP connection from an arbitrary non existing or unresponding IP source address, exploiting IP address based access control mechanisms.

Linux 2.0.x kernels were tested against this attack and found not to be vulnerable in any case.

Systems Affected

All systems running the kernel versions 2.2.x of the Linux operating system. Linux 2.3.x systems may be affected, too, we didn't tested this versions.

In our test situations we noticed that it doesn't seem to matter whether the TCP syncookie functionality was enabled or not (enabled within the kernel and activated through the proc filesystem options).

Tests

This is the beginning of a log of a successfully mounted blind TCP spoofing attack agains a Linux 2.2.12 system. (tcpdump output formatted for better readability)

    16:23:02.727540 attacker.522  > victim.ssh  : S  446679473: 446679473(0)
    16:23:02.728371 victim.ssh    > attacker.522: S 3929852318:3929852318(0)
    16:23:02.734448 11.11.11.11.522 > victim.ssh: S  446679473: 446679473(0)
    16:23:02.734599 victim.ssh > 11.11.11.11.522: S 3929859164:3929859164(0)
    16:23:03.014941 attacker.522  >   victim.ssh: R  446679474: 446679474(0)
    16:23:05.983368 victim.ssh > 11.11.11.11.522: S 3929859164:3929859164(0)
    16:23:06.473192 11.11.11.11.522 > victim.ssh: . ack 3929855318
    16:23:06.473427 victim.ssh > 11.11.11.11.522: R 3929855318:3929855318(0)
    16:23:06.554958 11.11.11.11.522 > victim.ssh: . ack 3929855319
    16:23:06.555119 victim.ssh > 11.11.11.11.522: R 3929855319:3929855319(0)
    16:23:06.637731 11.11.11.11.522 > victim.ssh: . ack 3929855320
    16:23:06.637909 victim.ssh > 11.11.11.11.522: R 3929855320:3929855320(0)
    ...

The first ISN of the victim's host is 3929852318, which is within a SYNACK packet to the attackers host. This is unspoofed and can be easily snagged by the attacker. At the same time the attacker sent out the first unspoofed SYN packet he sent a spoofed SYN packet from 11.11.11.11 too. This packet is answered by the victims host too with the ISN of 3929859164. The difference between the first visible ISN and the second ISN is only (3929859164-3929852318) = 6846.

Please notice that all TCP and IP parameters of the spoofed packet, except for the IP source address are the same as of the unspoofed packet. This is important (see below).

This small differences within the initial TCP sequence number (ISN) is exploitable. In other tests, where both hosts were unlagged we even had differences below 500 sometimes.

We've managed to successfully blind spoof TCP connections on different Linux 2.2.x systems, that is reaching the TCP "ESTABLISHED" state without being able to sniff the victim host.

Impact

By sending packets from a trusted source address, attackers could possibly bypass address based authentication and security mechanisms.

There have been similiar exploiting technics, aimed especially at r* and NFS services, in the past that demonstrated the security impact of weak ISNs very well. We have written a working exploit to demonstrate the weakness.

Explanation

The problem relies on a implementation flaw within the random ISN algorithm in the Linux kernel.

The problem is within drivers/char/random.c, line 1684:

    __u32 secure_tcp_sequence_number(__u32 saddr, __u32 daddr,
                                     __u16 sport, __u16 dport)
    {
            ...
            static __u32    secret[12];
            ...
            secret[0]=saddr;
            secret[1]=daddr;
            secret[2]=(sport << 16) + dport;

            seq = (halfMD4Transform(secret+8, secret) &
                   ((1<<HASH_BITS)-1)) + count;
            ...
    }

As already said, in our spoofed TCP SYN packet only the IP source address differs, that is only secret[0], so of 12*4 random bytes used to create the sequence number from, only 4 bytes differ. Obviously the hash created by halfMD4Transform has similarities if the source and destination ports and the destination address are the same.

It seems that the src-adress is least-significant to the above MD4 algorithm. Changing the source-ports too, makes the 2 ISNs more differ. Due to the short gap of time, the last

            seq += tv.tv_usec + tv.tv_sec*1000000;

is useless. This may be the reason why this bug may have survived long: In any real network situation it is uncommon that the source and destination ports are equal in two different connections on one host.

Further analyzation of the hash algorithm in this routine may result in a better ISN prediction than the one we use (range prediction).

Solution

First: It's always unwise to rely on address based authentication, because in a sniffable enviroment, such as the Internet, there are always means of bypassing address based authentication.

Second: The press shouldn't hype this as _THE_ Linux bug.. everyone having looked at the ISNs/DNS Sequence numbers of any of Microsoft's operating systems knows that their 'random numbers' are _much_ easier targets to use for IP and DNS spoofing attacks. For a a description how the ISN numbers of the Microsoft Windows NT TCP stack have even weakened with the latest Service Packs, you may want to browse the latest postings to the Bugtraq security mailing list [1] or read [2]. Well.. not that it matters.. but who uses Microsoft software anyway ?

The Linux kernel developers have been notified at the same time as the public Linux community, so a safe patch should be available real soon.

Acknowledgments

The bugdiscovery and the exploit is due to:

Stealth http://www.kalug.lug.net/stealth

S. Krahmer http://www.cs.uni-potsdam.de/homepages/students/linuxer

This advisory has been written by typo and scut.
The tests and further analyzation were done by stealth and scut.
The demonstration exploit has been written by S. Krahmer.

Contact Information

The teso crew can be reached by mailing to teso@shellcode.org. Our webpage is at http://teso.scene.at/

References

[1] Mail to the Bugtraq mailing list From: Roy Hills <bugtraq-l@NTA-MONITOR.COM> Subject: NT Predictable Initial TCP Sequence numbers - changes observed with SP4

[2] Microsoft Knowledge Database Article ID: Q192292 "Unpredictable TCP Sequence Numbers in SP4".

[3] libUSI++, a spoofing library http://www.cs.uni-potsdam.de/homepages/students/linuxer/

[4] TESO http://teso.scene.at/

[5] S. Krahmer http://www.cs.uni-potsdam.de/homepages/students/linuxer

Disclaimer

This advisory does not claim to be complete or to be usable for any purpose. Especially information on the vulnerable systems may be inaccurate or wrong. The supplied exploit is not to be used for malicious purposes, but for educational purposes only.

This advisory is free for open distribution in unmodified form. Articles that are based on information from this advisory should include link [4] and [5].

Exploit

We've created a working exploit to demonstrate the vulnerability. The exploit needs libUSI++ installed, which can be obtained through [3].

The exploit is available from either

http://teso.scene.at/

or

http://www.cs.uni-potsdam.de/homepages/students/linuxer/

I think that the implementation of the secure TCP Sequence number algorithm within the Linux kernel is flawed from ground on.

I agree, using known data (port numbers, addresses) as "secret" data to hash is false. Instead a good PRNG should be used everytime a sequence number is needed, independent to any TCP data. Operating systems such as OpenBSD do have cryptographical safe Sequence number generators, why should Linux go the unsafe way ?

But again, this advisory shows the general lack of security within the old protocol suites. TCP Sequence numbers were never meant to be secure, a 32 bit number just cannot guarantee security at all. Later when the first spoofing vulnerabilities were discovered the reasons of the sequence number shifted from a "to order data" meaning to a "guarantee security" meaning.

Over the weekend we have seen several weird email problems in MTA logs and reports.

• Some people use Web-browsers for their email, and have had problems unsubscribing (or subscribing, even). Commonly the problem seems to be that they send email with VCARD attachments, and/or have TEXT and HTML forms of email text in the same message -- Majordomo understands only attachementless TEXT/PLAIN messages, NOTHING ELSE.

• People using services of MAIL.COM/INAME.NET "free email" are in danger of loosing their Linux related feeds, unless they will get their service provider to remove false static blacklisting entry for FUNET's new server address I am using as their fanout:

  <smtp email.com sbragion@email.com 60001>: ...\
    <<- RCPT To:<sbragion@email.com> ORCPT=rfc822;sbragion@email.com
    ->> 523 <sbragion@email.com>... The IP address 128.214.248.46 is blacklisted.
  Contact your ISP administrator for details.

  We are now doing extra tricks to redirect email to those people via other servers than the one they have blocked, but that situation won't last forever, and then those subscribers will likely be summarily DELETED. (That provider doesn't seem to respond to my email about the issue..)

• Some MTAs are doing email transport envelope address verifications, but are having some local problems and falsely rejecting VGER's emails:

  <smtp matrix.it mmeregalli@matrix.it 60001>: ...\
    <<- MAIL From:<owner-linux-kernel-outgoing@vger.rutgers.edu> BODY=8BITMIME SIZE=2551
    ->> 501 <owner-linux-kernel-outgoing@vger.rutgers.edu>...
  Sender domain must exist

  ANY five-hundred (5xx) series response leads very quickly to deletion of the recipient for which it happens..

  Any four-hundred (4xx) series response which are found in "delivery timed out" type of reports is also likely to cause removal of the recipients. (Right now lots of Taiwanese addresses have been removed after recent earth-quakes in there have downed their machines...)

• Some new software is around, possibly a SMTP "security" proxy/firewall:

  <smtp ncd.com keithp@ncd.com 60001>: ...\
     <<- HELO nic2.funet.fi
     ->> 500 Command unrecognized

  I would like to know what that is about ? No, not that one anymore - currently SMTP/smap, which works.

  <smtp mail.zhongxing.com Orient@mail.zhongxing.com 60001>: ...\
     <<- HELO nic2.funet.fi
     ->> 500 Session already established.
     The domain name [nic2.funet.fi] passed in with HELO will be ignored.
  The current domain name of sending SMTP is [web.china-insurance.com].

  but what is this ? (Telnet to their SMTP server - you see two spontaneous reply codes, although they do take a bit to emerge out...)

Thanks for help on any (sub-)issue you can provide.