Files
nexus/sreweekly/articles/192/03-fun-with-null-pointers.html
2026-09-12 17:23:01 +08:00

2499 lines
105 KiB
HTML
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html lang="en">
<head><title>Fun with NULL pointers, part 1 [LWN.net]</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta property="og:title" content="Fun with NULL pointers, part 1">
<meta property="og:site_name" content="LWN.net">
<meta property="og:type" content="article">
<meta property="og:description" content="By now, most readers will be familiar with the local kernel exploit recently posted by Brad Spe [...]">
<meta HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">
<META NAME="robots" CONTENT="noai, noimageai">
<link rel="icon" href="https://static.lwn.net/images/favicon.png"
type="image/png">
<link rel="alternate" type="application/rss+xml" title="LWN.net headlines" href="https://lwn.net/headlines/rss">
<link rel="stylesheet" href="/CSS/lwn-v1">
<style>div.MakeALink { display: none }</style>
</head>
<body>
<a name="t"></a>
<div id="menu"><a href="/" aria-label="Return to the front page">
<img src="https://static.lwn.net/images/logo/barepenguin-70.webp" class="logo"
border="0" alt="LWN.net Logo">
<span class="logo">LWN<br>.net</span>
<span class="logobl">News from the source</span></a>
<a href="/" aria-label="Return to the front page">
<img src="https://static.lwn.net/images/lcorner-ss.png" class="sslogo"
border="0" alt="LWN"></a><div class="navmenu-container">
<ul class="navmenu">
<li><a class="navmenu" href="#t"><b>Content</b></a><ul><li><a href="/current/">Weekly Edition</a></li><li><a href="/Archives/">Archives</a></li><li><a href="/Search/">Search</a></li><li><a href="/Kernel/">Kernel</a></li><li><a href="/Security/">Security</a></li><li><a href="/Calendar/">Events calendar</a></li><li><a href="/Comments/unread">Unread comments</a></li><li><hr></li><li><a href="/op/FAQ.lwn">LWN FAQ</a></li><li><a href="/op/AuthorGuide.lwn">Write for us</a></li></ul></li>
<li><a class="navmenu" href="#t"><b>Edition</b></a><ul><li><a href="/Articles/341620/">Return to the Security page</a></li></ul></li>
</ul></div>
</div> <!-- menu -->
<div class="topnav-spacer"></div>
<div class="top-banner not-print"><!-- no top ad served --></div>
<div class="topnav-container">
<div class="not-handset"><form action="https://lwn.net/Login/" method="post" name="loginform"
class="loginform">
<label><b>User:</b> <input type="text" name="uname" value="" size="8" id="uc" /></label>
<label><b>Password:</b> <input type="password" name="pword" size="8" id="pc" /></label> <input type="hidden" name="target" value="/Articles/342330/" /> <input type="submit" name="submit" value="Log in" /></form> |
<form action="https://lwn.net/subscribe/" method="post" class="loginform">
<input type="submit" name="submit" value="Subscribe" />
</form> |
<form action="https://lwn.net/Login/newaccount" method="post" class="loginform">
<input type="submit" name="submit" value="Register" />
</form>
</div>
<div class="handset-only">
<a href="/Login/"><b>Log in</b></a> /
<a href="/subscribe/"><b>Subscribe</b></a> /
<a href="/Login/newaccount"><b>Register</b></a>
</div>
</div><div class="maincolumn flexcol">
<div class="middlecolumn">
<div class="PageHeadline">
<h1>Fun with NULL pointers, part 1</h1>
</div>
<div class="ArticleText"><main>
<div class="FeatureByline">
By <b>Jonathan Corbet</b><br>July 20, 2009</br>
</div>
By now, most readers will be familiar with <a
href="http://lwn.net/Articles/341773/">the local kernel exploit</a> recently
posted by Brad Spengler. This vulnerability, which affects the 2.6.30
kernel (and a test version of the RHEL5 "2.6.18" kernel), is interesting in
a number of ways. This article will look in detail at how the exploit
works and the surprising chain of failures which made it possible.
<p>
The TUN/TAP driver provides a virtual network device which performs packet
tunneling; it's useful in a number of situations, including virtualization,
virtual private networks, and more. In normal usage of the TUN driver, a
program will open <tt>/dev/net/tun</tt>, then make an <tt>ioctl()</tt> call
to set up the network endpoints. Herbert Xu recently noticed a problem
where a lack of packet accounting could let a hostile application pin down
large amounts of kernel memory and generally degrade system performance.
His solution was <a
href="http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=33dccbb050bbe35b88ca8cf1228dcf3e4d4b3554">a
patch</a> which adds a "pseudo-socket" to the device which can be used by
the kernel's accounting mechanisms. Problem solved, but, as it turns out,
at the cost of adding a more severe problem.
<p>
The TUN device supports the <tt>poll()</tt> system call. The beginning of
the function implementing this functionality (in 2.6.30) looks like this:
<p>
<pre>
static unsigned int tun_chr_poll(struct file *file, poll_table * wait)
{
struct tun_file *tfile = file-&gt;private_data;
struct tun_struct *tun = __tun_get(tfile);
<u>struct sock *sk = tun-&gt;sk;</u>
unsigned int mask = 0;
if (!tun)
return POLLERR;
</pre>
<p><blockquote class="ad">
<b><tt>$ sudo subscribe today</tt></b>
<p>
Subscribe today and elevate your LWN privileges. You’ll have
access to all of LWN’s high-quality articles as soon as they’re
published, and help support LWN in the process. <a href="https://lwn.net/Promo/nst-sudo/claim">Act now</a> and you can start with a free trial subscription.
</blockquote>
<p>
The line of code which has been underlined above was added by Herbert's
patch; that is where things begin to go wrong. Well-written kernel code
takes care to avoid dereferencing pointers which might be NULL; in fact,
this code checks the <tt>tun</tt> pointer for just that condition. And
that's a good thing; it turns out that, if the configuring <tt>ioctl()</tt>
call has been made, <tt>tun</tt> will indeed be NULL. If all goes
according to plan, <tt>tun_chr_poll()</tt> will return an error status in
this case.
<p>
But Herbert's patch added a line which dereferences the pointer prior to
the check. That, of course, is a bug. In the normal course of operations,
the implications of this bug would be somewhat limited: it should cause a
kernel oops if <tt>tun</tt> is NULL. That oops will kill the process which
made the bad system call in the first place and put a scary traceback into
the system log, but not much more than that should happen. It should be,
at worst, a denial of service problem.
<p>
There is one little problem with that reasoning, though: NULL (zero) can
actually be a valid pointer address. By default, the very bottom of the
virtual address space
(the "zero page," along with a few pages above it) is set to disallow all
access as a way of catching null-pointer bugs (like the one described
above) in both user and kernel space.
But it is possible, using
the <tt>mmap()</tt> system call, to put
real memory at the bottom of the virtual address space. There are some
valid use cases for this functionality, including running legacy binaries.
Even so, most contemporary systems disable page-zero mappings through the
use of the <tt>mmap_min_addr</tt> sysctl knob.
<p>
<span class="PullQuote">
<span class="invisible">[PULL QUOTE: </span>
Security module checks
are supposed to be additive to the checks which are already made by the
kernel, but it didn't work that way this time.
<span class="invisible"> END QUOTE]</span>
</span>
This knob should prevent a user-space program from mapping the zero page,
and, thus, should ensure that null pointer dereferences cause a kernel
oops. But, for unknown reasons, the <tt>mmap()</tt> code in the 2.6.30 kernel
explicitly declines to enforce <tt>mmap_min_addr</tt> if the security
module mechanism has been configured into the kernel. That job, instead,
is left to the specific security module being used. Security module checks
are supposed to be additive to the checks which are already made by the
kernel, but it didn't work that way this time; with regard to page zero,
security modules can grant access which would otherwise be denied. To
complete the failure,
Red Hat's default SELinux policy allows mapping the zero page. So, in this
case, running SELinux actually decreased the security of the system.
<p>
Not that life is a whole lot better without SELinux.
In the absence of SELinux, the exploit will run up against the
<tt>mmap_min_addr</tt> limit, which would seem like enough to bring things
to a halt. That particular difficulty can be circumvented, though, through
the use of the <tt>personality()</tt> system call. Enabling the SVR4
personality causes a read-only page to be mapped at address zero when a
program is invoked with <tt>exec()</tt>, but
only if the process in question has the CAP_SYS_RAWIO capability.
So one more trick is required: the top-level exploit code will set the SVR4
personality, then use <tt>exec()</tt> to run the pulseaudio server with a
special plugin module. Pulseaudio is installed setuid root, so it will get
the zero page mapped at invocation time. By the time
the plugin code is called, pulseaudio has dropped its privileges, but, by
then, the zero page will be available to the exploit code, which can make
the page writeable and place its own data there.
<p>
As a result of all this, it is possible for a user-space process to map the
zero page and prevent <tt>tun_chr_poll()</tt> from causing a kernel oops.
But, one would think, that would not get an attacker very far, since that
function checks <tt>tun</tt> against NULL as the very next thing it does.
This is where the next interesting step in the chain of failures happens:
the GCC compiler will, by default, optimize the NULL test out. The
reasoning is that, since the pointer has already been dereferenced (and has
not been changed), it cannot be NULL. So there is no point in checking it.
Once again, this logic makes sense most of the time, but not in situations
where NULL might actually be a valid pointer.
<p>
So, an attacker is able to get into the body of <tt>tun_chr_poll()</tt>
with a NULL <tt>tun</tt> pointer. One then needs to figure out how to get
control of the kernel using this situation. The next step takes advantage
of this code from a little further into <tt>tun_chr_poll()</tt>:
<p>
<pre>
if (sock_writeable(sk) ||
(!test_and_set_bit(SOCK_ASYNC_NOSPACE, &amp;sk->sk_socket->flags) &amp;&amp;
sock_writeable(sk)))
mask |= POLLOUT | POLLWRNORM;
</pre>
<p>
The value of <tt>sk</tt>, remember, came from the dereferencing of
<tt>tun</tt>, so it's under the attacker's control.
<tt>SOCK_ASYNC_NOSPACE</tt> is zero, so the <tt>test_and_set_bit()</tt>
call can be used to unconditionally set the least significant bit of any word in
memory. As kernel memory corruptions go, this is a small one, but
it turns out to be enough. In Brad's exploit,
<tt>sk-&gt;sk_socket-&gt;flags</tt> points into the TUN driver's
<tt>file_operations</tt> structure; in particular, it points to the
<tt>mmap()</tt> function. The TUN driver does not support <tt>mmap()</tt>,
so that pointer is normally NULL; after the <tt>poll()</tt> call, that
pointer is now one instead.
<p>
The final step in the exploit is to call <tt>mmap()</tt> on a file
descriptor for the open TUN device. Since the internal <tt>mmap()</tt>
operation is no longer NULL (it has been set to one), the kernel will jump
to it. That address also lives within the zero page mapped by the exploit,
so it is under the attacker's control. The exploit will have populated
that address with another jump to its own code. So, when the kernel calls
(what it thinks is) the TUN driver's <tt>mmap()</tt> function, the result
is arbitrary code being run in kernel mode; at that point the exploit has
total control.
<p>
In well-designed systems, catastrophic failures are rarely the result of a
single failure. That is certainly the case here. Several things went
wrong to make this exploit possible: security modules were able to grant
access to low memory mappings contrary to system policy, the SELinux policy
allowed those mappings, pulseaudio can be exploited to make a specific
privileged operation available to exploit code, a NULL pointer was
dereferenced before being checked, the check was optimized out by the
compiler, and the code used the NULL pointer in a way which allowed the
attacker to take over the system. It is a long chain of failures, each of
which was necessary to make this exploit possible.
<p>
This particular vulnerability has been
closed, but there will almost certainly be others like it. See <a
href="http://lwn.net/Articles/342420/">the second
article</a> in this series for a look at how the kernel developers are
responding to this exploit.<br clear="all"><table class="IndexEntries">
<tr><th colspan=2>Index entries for this article</th></tr>
<tr><td><a href="/Kernel/Index">Kernel</a></td><td><a href="/Kernel/Index#Security-Vulnerabilities">Security/Vulnerabilities</a></td></tr>
<tr><td><a href="/Security/Index/">Security</a></td><td><a href="/Security/Index/#Linux_kernel">Linux kernel</a></td></tr>
<tr><td><a href="/Security/Index/">Security</a></td><td><a href="/Security/Index/#Vulnerabilities-Privilege_escalation">Vulnerabilities/Privilege escalation</a></td></tr>
</table><br clear="all">
<hr width="60%%" align="left">
<form action="/Login/" method="post">
<input type="hidden" name="target" value="/Articles/342330/" />
<input type="submit" name="login" value="Log in" /> to post comments
<p>
<p><a name="Comments"></a>
<a name="CommAnchor342407"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 20:14 UTC (Mon)
by <b>clugstj</b> (subscriber, #4020)
[<a href="/Articles/342407/">Link</a>] (4 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
It's even a little more convoluted than mentioned in the next-to-the-last paragraph. Since a pointer was set to 1, it also requires that the processor fetch opcodes at byte boundaries. No RISC processor would do that (AFAIK).<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342407/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342412"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 20:18 UTC (Mon)
by <b>jengelh</b> (subscriber, #33263)
[<a href="/Articles/342412/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Yeah but the fact that x86en supporting unaligned access dominate makes up for it :-/<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342412/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor342443"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:38 UTC (Mon)
by <b>spender</b> (guest, #23067)
[<a href="/Articles/342443/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
I could have written to the 2nd byte instead (choose your endianness) and the resulting address would be aligned.<br>
The kernel actually being able to use that address directly would depend on the architecture.<br>
<p>
I only chose the first byte because I already had my mapping at NULL, so it was easy to reuse it. The exploit primitive there though allows an arbitrary OR of 0x1 to any byte in memory.<br>
<p>
-Brad<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342443/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342464"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 22:27 UTC (Mon)
by <b>spender</b> (guest, #23067)
[<a href="/Articles/342464/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Clarification/fix: Since the OR is performed on an unsigned long instead of a single byte, then the address of the target may be subject to whatever alignment on architectures that care.<br>
<p>
-Brad<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342464/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342887"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">RISC can do that</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 18:13 UTC (Wed)
by <b>klossner</b> (subscriber, #30046)
[<a href="/Articles/342887/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
PowerPC silently drops the two low bits when loading an address into the PC, so a branch to 1 becomes a branch to 0. The misaligned-address exception occurs only for load/store instructions.<br>
<p>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342887/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342405"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 20:14 UTC (Mon)
by <b>jengelh</b> (subscriber, #33263)
[<a href="/Articles/342405/">Link</a>] (23 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
Once upon a dereference…<br>
<p>
<font class="QuotedText">&gt;pulseaudio is installed setuid root</font><br>
<p>
*cough*<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342405/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342445"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:39 UTC (Mon)
by <b>proski</b> (guest, #104)
[<a href="/Articles/342445/">Link</a>] (18 responses)
</p>
</div>
</summary>
We should ensure that setuid programs exit immediately if they are using non-default personalities.
<div class="CommentReplyButton">
<form action="/Articles/342445/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342454"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:53 UTC (Mon)
by <b>drag</b> (guest, #31333)
[<a href="/Articles/342454/">Link</a>] (17 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
It would be a lot better just to do away with setuid binaries altogether.<br>
<p>
That's what things like policykit and dbus are for...<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342454/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342483"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 23:33 UTC (Mon)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/342483/">Link</a>] (15 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
Oh yeah great. So instead of locating setuid binaries and knowing that <br>
*that* is my vulnerability surface, I have to parse a huge pile of XML and <br>
hope that there are no bugs in policykit and dbus that might cause <br>
unintended things to be run (and we know there have been *none* like that <br>
before). The setuid implementation in the kernel is tiny and trivially <br>
auditable by comparison, sharing virtually all its code with the <br>
tested-to-death-and-hopefully-audited ELF execve() implementation.<br>
<p>
PolicyKit has been done right once before. It was called 'userv'. <br>
PolicyKit itself is a huge step backwards if you actually want security.<br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342483/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342526"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 7:21 UTC (Tue)
by <b>gmaxwell</b> (guest, #30048)
[<a href="/Articles/342526/">Link</a>] (7 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
<p>
Yea… policy kit. Great stuff. <br>
<p>
By default fedora allows the desktop users to change the system time. All they must do is ender the *user's* password (not root!) and even that they only have to do it once.<br>
<p>
Great stuff great stuff. <br>
<p>
Although many people have pointed out the terrible security implications nothing has been done. Sometimes it really does take some high profile compromises to get things fixed.<br>
<p>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342526/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342543"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 9:45 UTC (Tue)
by <b>cortana</b> (subscriber, #24596)
[<a href="/Articles/342543/">Link</a>] (6 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
But that's not the fault of PolicyKit itself. Rather it's the fault of the distributor who shipped it with a policy that a) allows an unprivileged user to change the system time and b) does not force them to re-authenticate whenever they wish to do so.<br>
<p>
Concerns about the increased vulnerability surface caused by the complexity of PolicyKit are still justified, but Fedora's default policy being stupid is not relevant to that discussion. If we wanted to blame the system for allowing the user to do stupid things then we may as well all give up and move back to Windows. :)<br>
<p>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342543/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342548"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 9:56 UTC (Tue)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/342548/">Link</a>] (5 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
The concern isn't just that the vulnerability surface has increased: it's that we can't even easily tell what it is anymore.<br>
<p>
Determining the set of privileged code that could carry out operations on behalf of unprivileged users was fairly simple in the days before PolicyKit: find setuid/setgid binaries, chase their shared library dependencies and (if you're paranoid) see what they can dlopen(). Just a grep away, in any case.<br>
<p>
Now, we have to analyze the dbus and PolicyKit policies as well, and XML is... not terribly amenable to analysis with Unix-style shell tools. (Some Perl packages come with XML-style XPath-based grep tools, but they are a) rarely installed and b) seriously cumbersome. We really need an awk for XML.)<br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342548/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342567"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 12:52 UTC (Tue)
by <b>nim-nim</b> (subscriber, #34454)
[<a href="/Articles/342567/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
<font class="QuotedText">&gt; We really need an awk for XML.</font><br>
<p>
Just use xsltproc directly (though not having to use a detached xslt file would be nice)<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342567/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342926"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 22:00 UTC (Wed)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/342926/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Ew, no. Utterly un-awklike and doing awk-like transformations with XSLT is <br>
really quite painful. (And yes, you can do awklike languages for things <br>
other than text streams: see gvpr(1) for example.)<br>
<p>
(One of many problems is XSLT's heavy use of &lt;&gt;, which makes it very <br>
annoying to use from the shell prompt. Another is its astonishing <br>
verbosity. Another is its total lack of good taste in design... also the <br>
functional nature of it, while one of its nicer aspects, fits very badly <br>
with the shell in my experience.)<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342926/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342580"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 14:01 UTC (Tue)
by <b>gmaxwell</b> (guest, #30048)
[<a href="/Articles/342580/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Poliykit plays a role: If you go look at the discussions on the fedora list you'll see that there was some degree of argument what the actual behaviour was— Was it asking for a password at all (some people thought it wasn't because it only did so once) and was it asking for the root password? A lot of people had used and never realized that it was asking for their user password rather than root.<br>
<p>
SUID is more unambiguous. <br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342580/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor342767"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">xml awk</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 7:08 UTC (Wed)
by <b>Frej</b> (guest, #4165)
[<a href="/Articles/342767/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
xml cli tool <br>
<a href="http://xmlstar.sourceforge.net/">http://xmlstar.sourceforge.net/</a><br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342767/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342796"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">xml awk</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 11:18 UTC (Wed)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/342796/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
That looks nice. Not very awkish though...<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342796/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
</details>
</details>
<a name="CommAnchor342546"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 10:18 UTC (Tue)
by <b>cortana</b> (subscriber, #24596)
[<a href="/Articles/342546/">Link</a>] (4 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
You don't have to start at the XML. You can ask the PolicyKit daemon itself what you are currently allowed to do by running polkit-auth; you can get a list of everything you could possibly do with additional authentication by running polkit-auth --show-obtainable; and you can see all the actions that it is possible to perform on a system, and find out which users may perform that action, by running polkit-gnome-authorization. Of course, this requires you to trust that the PolicyKit daemon has not already been compromised. ;)<br>
<p>
I also think the docs for PolicyKit are pretty good; the library reference manual contains a Design Overview at &lt;<a href="http://hal.freedesktop.org/docs/PolicyKit/ref-design.html">http://hal.freedesktop.org/docs/PolicyKit/ref-design.html</a>&gt; that explains how all the parts of the system fit together.<br>
<p>
Of course the system is much more complex than the setuid mechanism in the kernel, but--assuming it is audited well--it has the potential to increase system security a lot, because it would allow you to eliminate other setuid programs on the system.<br>
<p>
There is certainly no shortage of badly written programs that are installed setuid, but fail to give away all additional privileges but the exact ones it wishes to keep and deal with any of the infinite combinations of things an attacker may do to them before exec is called, such as messing around with the PATH or another similarly dangerous environment variable; creating symlinks that will cause a badly written 'create temporary file' routine to overwrite vital system files; mapping memory to address 0 to bypass security checks in the kernel (<a href="http://lwn.net/Articles/342330/">http://lwn.net/Articles/342330/</a> anyone?) :)<br>
<p>
And of course, setuid does not only consist of one system call, but several (setuid, seteuid, setreuid and setresuid), all with subtly different semantics--sometimes modifying the saved user id, sometimes the real user id, yet others the effective uid... oh, and don't forget setfsuid and the capability mechanism. And the fact that different versions of different operating systems all implement subtly different semantics for these system calls... check out Setuid Demystified &lt;<a href="http://www.eecs.berkeley.edu/~daw/papers/setuid-usenix02.pdf">http://www.eecs.berkeley.edu/~daw/papers/setuid-usenix02.pdf</a>&gt; and the Secure Programming for Linux and Unix HOWTO &lt;<a href="http://www.dwheeler.com/secure-programs/Secure-Programs-HOWTO/index.html">http://www.dwheeler.com/secure-programs/Secure-Programs-H...</a>&gt; for more details and other attack mechanisms.<br>
<p>
So yeah, PolicyKit is complicated--but is it more complicated than the intersection of the setuid mechanism and all the other stuff an attacker can do to a setuid process before it is executed? :)<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342546/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342797"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 11:19 UTC (Wed)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/342797/">Link</a>] (2 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
So the only way you can see what actions can be run on a system with privilege is... to run a GNOME-specific GUI application?<br>
<p>
*sigh*<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342797/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342810"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 12:49 UTC (Wed)
by <b>cortana</b> (subscriber, #24596)
[<a href="/Articles/342810/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
It's not GNOME-specific. It just uses GTK+. Yes, it's unfortunate that there is no command-line equivalent, but it shouldn't be too difficult for one to be written--you are just querying the PolicyKit object after all.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342810/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342925"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 21:59 UTC (Wed)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/342925/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
I think one is essential if PolicyKit can be considered safe to use: we <br>
have to be able to do automated audits and yell at unexpected quiet <br>
changes, lest we miss attacks or accidental fumbles opening up holes.<br>
<p>
You're quite right on the fugliness of the UID-setting syscalls in Unix, <br>
but this is not helped by introducing *more* :/ again, userv did all this <br>
better (IMNSHO) many years ago. Why more people don't use it I have no <br>
idea.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342925/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor345836"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Aug 7, 2009 12:21 UTC (Fri)
by <b>jschrod</b> (subscriber, #1646)
[<a href="/Articles/345836/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
I'll have to say that I don't find PolicyKit's documentation very good. I don't know how it is for a developer, but for me as a long-time Unix admin it was frustrating when I had to delve into it a few weeks ago for the first time.<br>
<p>
The document linked by you was one of the 1st that I read, btw. It's not that I didn't understand what PolicyKit was supposed to be doing, that was quite clear from the start. And not even that I couldn't read the config files -- while being rather chatty, they were understandable.<br>
<p>
Where I didn't succeed, is finding concise information about the overall architecture: how PolicyKit / ConsoleKit / HAL / D-Bus / pam / login processes, both X and console, are *supposed* to work together. That would have delivered pointers to information sources that could have answered questions like Where are the names in the PolicyKit XML files from? Which possible names exist on a given system? Which daemons are started and by whom? What are common PolicyKit authorizations, since it is all about policy and nothing about content? The output of polkit-auth --show-obtainable is not sufficient, IMHO. E.g., I'd need to know what org.freedesktop.hal.lock is, and googling it on site:freedesktop.org does _not_ provide an adequate answer, at least not for me. (And yes, I've read <a href="http://people.freedesktop.org/~david/hal-spec/hal-spec.html">http://people.freedesktop.org/~david/hal-spec/hal-spec.html</a>.)<br>
<p>
It was not easy to find the answers to such questions in reasonable time frames because the overall structure had to be laborously reverse engineered by yours humbly. Well, maybe my Google foo may simply be not good enough.<br>
<p>
Just my 0.02 EUR. :-)<br>
<p>
Joachim<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/345836/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342736"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 2:14 UTC (Wed)
by <b>Baylink</b> (guest, #755)
[<a href="/Articles/342736/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
"vulnerability surface".<br>
<p>
Wow. That's an altogether neat concept. Thanks.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342736/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor343374"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 25, 2009 2:13 UTC (Sat)
by <b>mikov</b> (guest, #33179)
[<a href="/Articles/343374/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
I am with you. <br>
In my mind the key to reasonable security is simplicity. SUID is extremely simple. Compare changing a user's password using passwd to what has to happen in Windows.<br>
<p>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/343374/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342536"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 8:37 UTC (Tue)
by <b>mmahut</b> (guest, #45550)
[<a href="/Articles/342536/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
It will eventually go away if we start using file system compatibilities. As far as I know, they've been already implemented in Fedora 10, but RPM will support it just from Fedora 12.<br>
<p>
<a href="http://udrepper.livejournal.com/20709.html">http://udrepper.livejournal.com/20709.html</a><br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342536/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor342494"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 0:26 UTC (Tue)
by <b>kpfleming</b> (subscriber, #23250)
[<a href="/Articles/342494/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
What? pulseaudio is setuid-root *and* allows non-privileged users to load modules into its process space? What kind of insanity is that?<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342494/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342498"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 2:13 UTC (Tue)
by <b>smurf</b> (subscriber, #17840)
[<a href="/Articles/342498/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
A setuid program loading unprivileged modules usually isn't a problem if that program drops its privs before doing so.<br>
<p>
Usually.<br>
<p>
This whole mess goes to show that security is a hard-to-solve problem.<br>
<p>
Taking privs off pulseaudio (Linux has been capable of doing better than all-out-setuid for _how_ long, exactly ?!?) may or may not be the solution in this case, I haven't checked.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342498/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342822"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 14:24 UTC (Wed)
by <b>mezcalero</b> (subscriber, #45103)
[<a href="/Articles/342822/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
pa 0.9.16 is not setuid anymore, because the introduction of rtkit makes that unnecessary.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342822/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor347254"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Aug 15, 2009 9:01 UTC (Sat)
by <b>gvy</b> (guest, #11981)
[<a href="/Articles/347254/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Yeah, previously one had to introduce a rootkit. Now it's half-done by prominent distributors. :(<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/347254/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor342424"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:07 UTC (Mon)
by <b>nhippi</b> (subscriber, #34640)
[<a href="/Articles/342424/">Link</a>] (2 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
1) If gcc can detect the unnecessary NULL check, cant it warn about it? or coverity give a DEAD_CODE notification?<br>
<p>
2) Does anyone use personality(SVR4) these days? The older personalities could be moved behind a switch (/proc/sys/vm/legacy_personality = 0), where the default is off. While it is only small part of this exploit, this would again make future exploits slighly harder (until someone notices another piece of ignored legacy code).<br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342424/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342480"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 23:39 UTC (Mon)
by <b>ncm</b> (guest, #165)
[<a href="/Articles/342480/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
(1) Coverity has checks for this. The error is so common that it makes up a large fraction of warnings in typical Coverity reports.<br>
<p>
(2) Gcc certainly should have a warning for this. However, putting such a warning in "-W" would make people upset (see (1)). That's not meant to be an argument against. It might be easier to get it into -Wall, but it is generally very hard to move a warning from -W to -Wall or the reverse, so if it gets into -Wall it is probably stuck there forever. The warning would likely be issued only when compiling with strong optimization; without optimization the compiler would be unlikely to perform the analysis that would lead to it noticing the problem.<br>
<p>
(3) Depending on the offset within the struct type pointed to, the pointer value dereferenced might not refer to zero page, and thus might not trigger a SEGV signal in normal user-level code, even without memory-mapping games.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342480/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342737"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 2:18 UTC (Wed)
by <b>Baylink</b> (guest, #755)
[<a href="/Articles/342737/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Am I the only person who read "has checks for this" and thought "a wart is a crocky feature"?<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342737/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor342430"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:11 UTC (Mon)
by <b>taviso</b> (subscriber, #34037)
[<a href="/Articles/342430/">Link</a>] (2 responses)
</p>
</div>
</summary>
Note that the mmap_min_addr bypass using personality() was discovered by me and Julien Tinnes, we described it <a href="http://blog.cr0.org/2009/06/bypassing-linux-null-pointer.html">here</a>. We also sent a patch that corrected it to lklm here http://patchwork.kernel.org/patch/32598/
<div class="CommentReplyButton">
<form action="/Articles/342430/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342966"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 23, 2009 5:01 UTC (Thu)
by <b>johnflux</b> (guest, #58833)
[<a href="/Articles/342966/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
You, sir, are a coding god.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342966/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor343281"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 24, 2009 16:06 UTC (Fri)
by <b>rogue@autistici.org</b> (guest, #59770)
[<a href="/Articles/343281/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
And, may you be glorified for your find as much as spender is for his exploit.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/343281/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342423"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:15 UTC (Mon)
by <b>spender</b> (guest, #23067)
[<a href="/Articles/342423/">Link</a>] (2 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
My last name is Spengler (no idea why people assume my alias is my last name). The analysis of the exploit appears correct however.<br>
<p>
The choice of the tun file_operations struct was arbitrary: a different one could have been chosen if the attacker wanted the exploit to work against a custom kernel with CONFIG_DEBUG_RODATA enabled. As I've found, since 2007 when most of those structs were made const, people haven't kept up with the standard, so there are a ton of other reliable function pointers to choose from.<br>
<p>
The nature of the NULL tun pointer being confined to the tun_chr_poll() function (instead of getting leaked out through some means to other complex functions in the kernel) is what makes the vulnerability 100% reliably exploitable.<br>
<p>
-Brad<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342423/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342436"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fixed</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:18 UTC (Mon)
by <b>corbet</b> (editor, #1)
[<a href="/Articles/342436/">Link</a>] (1 responses)
</p>
</div>
</summary>
<blockquote><i>My last name is Spengler (no idea why people assume my alias is my last name).</i></blockquote>
<p>
Because they look somewhat the same and we make silly mistakes? I'm sorry about this one; it's been fixed.
<div class="CommentReplyButton">
<form action="/Articles/342436/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342442"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fixed</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:34 UTC (Mon)
by <b>spender</b> (guest, #23067)
[<a href="/Articles/342442/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
No problem, it seems to be a common mistake ;)<br>
<p>
-Brad<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342442/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor342444"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">mmap_min_addr and security modules</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:47 UTC (Mon)
by <b>fjpop</b> (guest, #30115)
[<a href="/Articles/342444/">Link</a>] (2 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
<font class="QuotedText">&gt; But, for unknown reasons, the mmap() code in the 2.6.30 kernel</font><br>
<font class="QuotedText">&gt; explicitly declines to enforce mmap_min_addr if the security module</font><br>
<font class="QuotedText">&gt; mechanism has been configured into the kernel. That job, instead, is</font><br>
<font class="QuotedText">&gt; left to the specific security module being used.</font><br>
<p>
There are plenty of systems where CONFIG_SECURITY_SELINUX is set, but <br>
where selinux is either not installed or not activated.<br>
<p>
So if the quoted text is correct, then all those systems would be missing <br>
an apparently useful (basic?) security check. Or is the text imprecise <br>
and does the kernel check if a security module is active before ignoring <br>
mmap_min_addr?<br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342444/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342450"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">mmap_min_addr and security modules</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 21:53 UTC (Mon)
by <b>corbet</b> (editor, #1)
[<a href="/Articles/342450/">Link</a>] (1 responses)
</p>
</div>
</summary>
The code which performs the check was:
<p>
<pre>
static inline unsigned long round_hint_to_min(unsigned long hint)
{
#ifdef CONFIG_SECURITY
hint &amp;= PAGE_MASK;
if (((void *)hint != NULL) &amp;&amp;
(hint &lt; mmap_min_addr))
return PAGE_ALIGN(mmap_min_addr);
#endif
return hint;
}
</pre>
<p>
So it was taken out at <i>compile</i> time; the presence of an actual security module is not really relevant.
<div class="CommentReplyButton">
<form action="/Articles/342450/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342460"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">mmap_min_addr and security modules</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 22:15 UTC (Mon)
by <b>spender</b> (guest, #23067)
[<a href="/Articles/342460/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
That's not the right check. security_file_mmap (which is either set by the capabilities module or overriden by the SELinux module) is what implements the final check. The one you pasted doesn't even apply for MAP_FIXED but is just to ensure that the allocator doesn't choose an address below mmap_min_addr when only a hint is specified.<br>
<p>
If SELinux is compiled into the kernel, it needs to be disabled at boot via the kernel command-line, otherwise it registers its hooks with LSM and overrides that of the capabilities module for security_file_mmap which performs the mmap_min_addr check.<br>
<p>
-Brad<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342460/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor342463"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 22:25 UTC (Mon)
by <b>mjw</b> (subscriber, #16740)
[<a href="/Articles/342463/">Link</a>] (3 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
Nice overview. What trick was used to actually open the /dev/net/tun device?<br>
On my CentOS systems it is:<br>
crw------- 1 root root 10, 200 Jul 16 12:18 /dev/net/tun<br>
so doesn't seem to be accessible by non-root normally.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342463/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342473"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 23:13 UTC (Mon)
by <b>dwmw2</b> (subscriber, #2063)
[<a href="/Articles/342473/">Link</a>]
</p>
</div>
</summary>
On up to date systems it should have mode 0666. Only users with CAP_NET_ADMIN can create new tun devices, but then they can be made persistent and given to specific users/groups &mdash; who need to be able to open <TT>/dev/net/tun</TT> in order to attach to them.
<div class="CommentReplyButton">
<form action="/Articles/342473/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor342477"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 23:18 UTC (Mon)
by <b>eparis</b> (guest, #33060)
[<a href="/Articles/342477/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
apparently some udev script which makes it world rw, I'm told (but haven't verified) that some VPN program changes it so they can run as normal users...<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342477/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342606"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 16:10 UTC (Tue)
by <b>dwmw2</b> (subscriber, #2063)
[<a href="/Articles/342606/">Link</a>]
</p>
</div>
</summary>
<BLOCKQUOTE><I>"apparently some udev script which makes it world rw, I'm told (but haven't verified) that some VPN program changes it so they can run as normal users..."</I></BLOCKQUOTE>
Yes, the udev script creates it with mode 0666 because that's the recommended configuration.<P>
It's been possible to make tun devices that can be used by non-root since <A HREF="http://git.kernel.org/?p=linux/kernel/git/tglx/history.git;a=commitdiff;h=4fdbe71c852f4b44203913c93a5d3a8a12041167#patch96">February 2002</A>.
<P>
However, it was only in <A HREF="http://git.kernel.org/linus/ca6bb5d7ab22ac79f608fe6cbc6b12de6a5a19f0">June 2006</A> that we made it reasonable to have 0666 permissions on <TT>/dev/net/tun</TT>, by adding the CAP_NET_ADMIN checks before creating new devices.
<P>
The <A HREF="http://www.infradead.org/openconnect.html">OpenConnect</A> VPN client, when used in conjunction with its NetworkManager plugin, will use this facility to run as its own unprivileged user. After the stupid tmpfile races we saw in Cisco's own client which runs as root, it seemed like an appropriate design choice for limiting security exposure (even though I couldn't <em>possibly</em> be as incompetent as the Cisco engineers).
<div class="CommentReplyButton">
<form action="/Articles/342606/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor342486"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 20, 2009 23:53 UTC (Mon)
by <b>job</b> (guest, #670)
[<a href="/Articles/342486/">Link</a>] (2 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
The fact that those changes make it into the kernel in the first place is slightly worrying. Should an unchecked pointer deference not be caught with a static code checker? What about Sparse, isn't that run routinely? Perhaps automatic comments could be added to code already at the mailing list publication stage.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342486/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342487"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 0:00 UTC (Tue)
by <b>spender</b> (guest, #23067)
[<a href="/Articles/342487/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
A static checker is useless unless someone actually bothers to fix the things it reports.<br>
<p>
It's already been agreed upon that static checkers can find these bugs, yet clearly they're being produced faster than they're fixed.<br>
<p>
-Brad<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342487/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor342490"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 0:04 UTC (Tue)
by <b>proski</b> (guest, #104)
[<a href="/Articles/342490/">Link</a>]
</p>
</div>
</summary>
Sparse doesn't check for it yet. Perhaps it should. But please realize that it's not just an "unchecked code dereference", it's a dereference before check.
<div class="CommentReplyButton">
<form action="/Articles/342490/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342558"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 10:57 UTC (Tue)
by <b>epa</b> (subscriber, #39769)
[<a href="/Articles/342558/">Link</a>] (9 responses)
</p>
</div>
</summary>
Since the zero page can be mapped and therefore 0x0 is a valid address, the zero bit pattern should not be used for the null pointer. The null pointer is supposed to be an invalid pointer that never points to any addressable memory. From the C99 standard:
<blockquote>If a null pointer constant is converted to a
pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function.</blockquote>
The C runtime should use some other magic value to represent the null pointer - and the kernel should promise never to map any addressable memory there.
<div class="CommentReplyButton">
<form action="/Articles/342558/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342560"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 11:07 UTC (Tue)
by <b>tialaramex</b> (subscriber, #21167)
[<a href="/Articles/342560/">Link</a>] (8 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
Now why would you want people to spill coffee all over their keyboards?<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342560/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342780"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 9:02 UTC (Wed)
by <b>epa</b> (subscriber, #39769)
[<a href="/Articles/342780/">Link</a>] (7 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
It would be possible to keep 0x0 for the null pointer while respecting the C99 standard: the compiler would need to put in a couple of extra instructions for every pointer comparison making sure that 0x0 != P for any value of P. Optionally, it could also put in a check before every pointer dereference making sure the pointer is not 0x0 and causing a SIGSEGV if it is (since the memory layout can no longer be relied on to guarantee that).<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342780/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor343318"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 24, 2009 20:38 UTC (Fri)
by <b>giraffedata</b> (guest, #1954)
[<a href="/Articles/343318/">Link</a>] (5 responses)
</p>
</div>
</summary>
<blockquote>
It would be possible to keep 0x0 for the null pointer while respecting the C99 standard: the compiler would need to put in a couple of extra instructions for every pointer comparison making sure that 0x0 != P for any value of P.
</blockquote>
<p>
Do we know Gcc doesn't do this? Seems like it would have to, to be C99 compliant.
<blockquote>
Optionally, it could also put in a check before every pointer dereference making sure the pointer is not 0x0 and causing a SIGSEGV if it is (since the memory layout can no longer be relied on to guarantee that).
</blockquote>
<p>
But then how would you represent a pointer to a data structure that resides at address 0? A pointer should be able to do that.
<p>
It would of course be unrealistically expensive on typical machines to represent a pointer with anything but a simple address, but pointer comparisons are rare enough that a few extra instructions for them seems worthwhile to maintain the null pointer concept.
<p>
Regardless of how the compiler chooses to represent pointers (null or otherwise), the optimization in question is logically sound. C99 says a dereference of a null pointer causes undefined behavior, so either a) tun is non-null and !tun must be false or b) tun is null and !tun can be anything, including false.
<div class="CommentReplyButton">
<form action="/Articles/343318/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor343336"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 24, 2009 22:06 UTC (Fri)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/343336/">Link</a>] (4 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
GCC certainly doesn't insert code checking if a pointer is NULL before <br>
every pointer dereference. Considering how common pointer dereferencing is <br>
in C and related languages, this would be a substantial slowdown for no <br>
real gain, given that multiuser OSes invariably trap such things, and <br>
non-multiuser OSes are specialist environments in which attacks by hostile <br>
local users are not so common (yet).<br>
<p>
That kernel space has to work when the lower part of its address space is <br>
effectively under the control of a hostile attacker is a unique problem <br>
which it is really not worth changing the C standard for, nor imposing <br>
vast overheads on all userspace code. -fno-delete-null-pointer-checks does <br>
the job.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/343336/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor343376"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 25, 2009 2:51 UTC (Sat)
by <b>giraffedata</b> (guest, #1954)
[<a href="/Articles/343376/">Link</a>] (3 responses)
</p>
</div>
</summary>
<blockquote>
GCC certainly doesn't insert code checking if a pointer is NULL before
every pointer dereference.
</blockquote>
<p>
Sure, but that wouldn't improve standards compliance anyway. I asked if GCC generates extra code to comply with the C99 requirement that a null pointer not be equal to any non-null one (while still allowing the existence of pointers to a data structure that resides at address 0). Thinking about it now, though, I don't see how any such code is possible since a null pointer still has to compare equal to another null pointer.
<p>
<blockquote>
That kernel space has to work when the lower part of its address space is
effectively under the control of a hostile attacker is a unique problem
which it is really not worth changing the C standard for,
</blockquote>
<p>
There has been no proposal to deal with this by changing the standard,
which GCC apparently ignores anyhow. And objection to GCC's conflation of null pointers and zero-address pointers wasn't that it's a security problem but that it's a basic correctness problem. Even without a hostile page 0, unless you proclaim data structures at address 0 don't exist, this optimization breaks code.
<div class="CommentReplyButton">
<form action="/Articles/343376/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor343400"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 25, 2009 12:49 UTC (Sat)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/343400/">Link</a>] (2 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
GCC doesn't ignore the standard: you simply can't have data structures <br>
that reside at address zero. Shaving off 1/2^32 or less of the address <br>
space, and disabling an optimization in the one place that cares about <br>
this (the kernel) does not seem like a terrible cost to me.<br>
<p>
Data structures at address zero do not exist on any sane C platform.<br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/343400/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor343434"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 25, 2009 23:18 UTC (Sat)
by <b>PaXTeam</b> (guest, #24616)
[<a href="/Articles/343434/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
<font class="QuotedText">&gt; Data structures at address zero do not exist on any sane C platform.</font><br>
<p>
so platforms without an MMU are not sane?<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/343434/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor343462"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 26, 2009 18:27 UTC (Sun)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/343462/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Well, you can't access a structure at address zero on such a platform <br>
without either disabling all optimizations that involve knowing which <br>
pointers are null (as the kernel now is) and taking great care to ensure <br>
that you never need anything that can point to said structure to be NULL <br>
at any time, or defining the null pointer to be other than all-bits-zero <br>
(allowed, but weird, about as rare as platforms with strange word sizes).<br>
<p>
I'd say that trying to access structures at address zero, MMU or no MMU, <br>
is extremely unusual and not really sane to handle in a general-purpose <br>
compiler. (GCC goes further than I would expect in actually having a <br>
switch that makes it possible to use such a barmy thing.)<br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/343462/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
</details>
</details>
</details>
<a name="CommAnchor343395"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">A zero pointer is not a null pointer</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 25, 2009 9:37 UTC (Sat)
by <b>spitzak</b> (guest, #4593)
[<a href="/Articles/343395/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
That won't work, plenty of code assumes that comparing two NULL pointers for equality will return <br>
true.<br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/343395/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
</details>
<a name="CommAnchor342562"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 11:48 UTC (Tue)
by <b>dunlapg</b> (guest, #57764)
[<a href="/Articles/342562/">Link</a>] (1 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
Who on earth implemented the gcc "optimization"? Check-for-null-after-use is almost always a bug, isn't it? It should be throwing a warning out, not optimizing the null check away.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342562/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342563"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 12:01 UTC (Tue)
by <b>johill</b> (subscriber, #25196)
[<a href="/Articles/342563/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
I'd think very often it's not. Imagine the NULL pointer check was inserted by a macro or an inlined function, into a function that's known to be called with a non-NULL argument, say because that argument is on the stack.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342563/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
<a name="CommAnchor342593"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 15:59 UTC (Tue)
by <b>amnon</b> (guest, #43382)
[<a href="/Articles/342593/">Link</a>] (16 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
This bug made me think about other scenarios where optimizations can change the meaning of the code:<br>
<p>
The program was accessing the pointer without checking for NULL. Since the result of dereferencing a null pointer is undefined, the compiler "inferred" that the pointer is never null, and optimized away the check.<br>
<p>
That's valid logic, but there are rare cases where code intentionally uses undefined behavior. For example, OpenSSL tries to get entropy by reading an uninitialized buffer. If at any point in the future an optimization is introduced in the compiler which assumes that uninitialized variables are never accessed, bad things could happen.<br>
<p>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342593/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342639"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 18:05 UTC (Tue)
by <b>BrucePerens</b> (guest, #2510)
[<a href="/Articles/342639/">Link</a>] (15 responses)
</p>
</div>
</summary>
<i>there are rare cases where code intentionally uses undefined behavior.</i><p>
No correct cases.<p>
Undefined really means undefined. There was <i>never</i> any guarantee that the value used in the OpenSSL code would provide any entropy. Undefined doesn't mean "random", "unknown", or "non-deterministic", it means "implementation dependent".<p>
So, yes, it's difficult to get entropy without a specific service that provides entropy. But that's what you need, not just a garbage variable.
<div class="CommentReplyButton">
<form action="/Articles/342639/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342658"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 19:42 UTC (Tue)
by <b>martinfick</b> (subscriber, #4455)
[<a href="/Articles/342658/">Link</a>] (14 responses)
</p>
</div>
</summary>
<i>it means "implementation dependent".</i>
<p>
which as others have pointed out, is the purpose of undefined in the C spec, which does not mean that it has <i>"No correct cases."</i>, but rather, that you'd better be familiar with the implementation you are using. It makes no sense to say: "Here is a feature not available elsewhere, that we think it valuable, but you should never use it!" does it?
</p>
<div class="CommentReplyButton">
<form action="/Articles/342658/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342662"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 20:04 UTC (Tue)
by <b>BrucePerens</b> (guest, #2510)
[<a href="/Articles/342662/">Link</a>] (12 responses)
</p>
</div>
</summary>
"Implementation dependent" means that the folks who change the compiler, the C library, the kernel, and any stack-smashing detection/prevention code that you are using have the right to change the behavior at any time without documenting the particular side-effect that your code is depending upon.<p>
Especially in the context of the various stack-smashing prevention hacks going around, you have no reason to believe that an uninitialized variable would not actually be initialized to a fixed value.<p>
Undefined stuff is not a service that you can count on. Ever.
<div class="CommentReplyButton">
<form action="/Articles/342662/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342664"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 20:11 UTC (Tue)
by <b>martinfick</b> (subscriber, #4455)
[<a href="/Articles/342664/">Link</a>] (11 responses)
</p>
</div>
</summary>
<i>"Implementation dependent" means that the folks who change the compiler, the C library, the kernel, and any stack-smashing detection/prevention code that you are using have the right to change the behavior at any time without documenting the particular side-effect that your code is depending upon.
</i>
<p>
No, it does not always mean this. In this particular case it may, but it is perfectly reasonable for a specific C compiler to specify its behavior when the standard says that it is undefined, that is the point of "undefined" and "Implementation dependent".
</p>
"Implementation dependent" != `cat /dev/urandom`
<div class="CommentReplyButton">
<form action="/Articles/342664/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342668"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 20:55 UTC (Tue)
by <b>BrucePerens</b> (guest, #2510)
[<a href="/Articles/342668/">Link</a>] (9 responses)
</p>
</div>
</summary>
The recent ext3 fsync() snafu is a good example of implementation dependent behavior becoming taken as an implicit guarantee. And then the developer had to reduce the scope of the promise for performance reasons. He ended up regretting that he had ever made that feature visible.<p>
Anyway, there was no such guarantee in this case.
<div class="CommentReplyButton">
<form action="/Articles/342668/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342711"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 22:39 UTC (Tue)
by <b>mjg59</b> (subscriber, #23239)
[<a href="/Articles/342711/">Link</a>] (8 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
The role of software is to be useful to its consumers. Software that fails in this will tend to end up being ignored in the long run.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342711/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342732"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 1:28 UTC (Wed)
by <b>BrucePerens</b> (guest, #2510)
[<a href="/Articles/342732/">Link</a>] (7 responses)
</p>
</div>
</summary>
Matt, you aren't seriously proposing that we provide some sort of user contract regarding the contents of uninitialized variables being reliable sources of entropy.<p>Regarding ext3, this problem came up because fsync() was implemented as a performance pig, at least in ext3, and rather than fix it we trained application developers that they'd be safe without it. Had fsync() been repaired when the mozilla problem came up, nobody would be arguing about it today.
<div class="CommentReplyButton">
<form action="/Articles/342732/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342733"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 1:39 UTC (Wed)
by <b>mjg59</b> (subscriber, #23239)
[<a href="/Articles/342733/">Link</a>] (6 responses)
</p>
</div>
</summary>
<div class="FormattedComment">
If a system left truly random numbers in uninitialised variables and applications started making use of that functionality (even if undocumented), then I think you'd need a very good reason to change that behaviour. But since I can't imagine any sane system ever doing that, no, I'm not proposing that we need a user contract on that point.<br>
<p>
Contrast that to the ext3 behaviour. While, yes, the behaviour of fsync() on ext3 did result in people being even less likely to use it, the fact that ext3 also made it possible to overwrite a file without having to go through a cumbersome sequence of fsync()ing both the data and the directory made it attractive to application writers. That behaviour dates back far beyond Firefox 3, as demonstrated by people's long-term complaints about XFS leaving their files full of zeros after a crash. ext4 now provides the same ease of use because people made it clear that they weren't going to use it otherwise. Future Linux filesystems are effectively forced to provide the same semantics, which is a good thing.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342733/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342748"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 3:28 UTC (Wed)
by <b>BrucePerens</b> (guest, #2510)
[<a href="/Articles/342748/">Link</a>] (5 responses)
</p>
</div>
</summary>
Sigh. Good developers are still going to do create temp, write, fsync file, link to permanent name, unlink temp. Fsync on the directory, though, shouldn't be necessary.
<div class="CommentReplyButton">
<form action="/Articles/342748/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342753"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 4:16 UTC (Wed)
by <b>mjg59</b> (subscriber, #23239)
[<a href="/Articles/342753/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Good developers aren't going to have to - good operating systems will provide guarantees above and beyond POSIX. Operating systems that don't will end up poorly supported and gradually become irrelevant.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342753/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor342968"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 23, 2009 5:07 UTC (Thu)
by <b>johnflux</b> (guest, #58833)
[<a href="/Articles/342968/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
I hope not. Good developers will realize that fsync is a complete overkill there. They don't need wait for the changes to actually be made to disk before continuing - they only need to make sure that the changes happen in the right order.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342968/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor344428"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 30, 2009 13:38 UTC (Thu)
by <b>forthy</b> (guest, #1525)
[<a href="/Articles/344428/">Link</a>] (2 responses)
</p>
</div>
</summary>
<p>The problem with fsync() is that it's semantics resemble whery much the
one of "PLEASE" in INTERCAL, which means it is a joke. fsync() basically
has <b>no</b> semantics, except "make it so" (make it persistent now).
Now, all file system operations are persistent, anyway, just not made
persistent <b>now</b>. You can't properly test (that is: automated),
because to test if there's a missing fsync(), you have to force an
unexpected reboot, and then check if there's any missing data. What's
worse: A number of popular Unix programming languages don't even have
fsync(), starting with all kinds of shell scripts. fsync() is a dirty hack
introduced into Unix because of broken (but extremely fast) file system
implementations.</p>
<p>We know that <a
href="http://www.asandler.com/jokes/computer/c.shtml">Unix is a joke</a>
for quite some time, but parts of the API like fsync() show that this is
not so far away from the truth ;-). From the kernel development side it is
always "easier" to maintain a sloppy specification and blame the loser,
but that's the wrong thinking. You are providing a service. Same thing for
GCC: Compiler writers provide a service. Using a sloppy specification for
questionable "optimizations" is wrong, as well. If the compiler writer
can't know that the code really will break when accessing the NULL
pointer, then he can't take the test out after having accessed an object.
I remember GCC taking out tests like if(x+n>x), because overflows are said
to be unspecified in the C language, but compiling code to a machine where
overflows were very specifically handled as wraparounds in two's
complement representation. This is all wrong thinking.</p>
<div class="CommentReplyButton">
<form action="/Articles/344428/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor344437"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 30, 2009 15:04 UTC (Thu)
by <b>foom</b> (subscriber, #14868)
[<a href="/Articles/344437/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
<font class="QuotedText">&gt; I remember GCC taking out tests like if(x+n&gt;x)</font><br>
<p>
Still does. You can use -fwrapv if you want to tell it that you want signed number overflow to be <br>
defined as wrapping.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/344437/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
<a name="CommAnchor344584"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 30, 2009 21:31 UTC (Thu)
by <b>nix</b> (subscriber, #2304)
[<a href="/Articles/344584/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
fsync(), brought to you by the same people who thought up 'volatile', <br>
another equally impossible-to-define-except-by-reference-to-implementation <br>
feature.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/344584/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
</details>
</details>
</details>
</details>
<a name="CommAnchor344432"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 30, 2009 13:31 UTC (Thu)
by <b>lysse</b> (guest, #3190)
[<a href="/Articles/344432/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
<font class="QuotedText">&gt; "Implementation dependent" != `cat /dev/urandom` </font><br>
<p>
Wasn't that where Bruce came in...? ;)<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/344432/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
<a name="CommAnchor344720"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Optimizations and undefined behavior</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 31, 2009 14:28 UTC (Fri)
by <b>hozelda</b> (guest, #19341)
[<a href="/Articles/344720/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
I know the conversation is partly about what developers should depend on or not, but keep in mind that "undefined behavior" and "implementation-defined behavior" mean something precise and completely different from each other in C99. See Chapter 3 (and 4) and informative Appendix J <a rel="nofollow" href="http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1336.pdf">http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1336.pdf</a><br>
<p>
</div>
<div class="CommentReplyButton">
<form action="/Articles/344720/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
</details>
<a name="CommAnchor342652"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">Fun with NULL pointers, part 1</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 19:14 UTC (Tue)
by <b>zeekec</b> (subscriber, #2414)
[<a href="/Articles/342652/">Link</a>] (2 responses)
</p>
</div>
</summary>
<a href="http://catb.org/jargon/html/N/nasal-demons.html">Nasal demons!</a>
<div class="CommentReplyButton">
<form action="/Articles/342652/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342691"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">#pragma and GCC</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 21, 2009 21:48 UTC (Tue)
by <b>BrucePerens</b> (guest, #2510)
[<a href="/Articles/342691/">Link</a>] (1 responses)
</p>
</div>
</summary>
Some time in the 80's, when RMS was still coding GCC, the response to #pragma in C was explicitly stated to be undefined, so that your compiler could have whatever magic happen that it wished.<p>
So, RMS coded GCC to start the game "rogue" when it saw #pragma.
<div class="CommentReplyButton">
<form action="/Articles/342691/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
<a name="CommAnchor342844"></a>
<details class="CommentBox" open>
<summary><h3 class="CommentTitle">#pragma and GCC</h3>
<div class="AnnLine">
<p class="CommentPoster"> Posted Jul 22, 2009 15:46 UTC (Wed)
by <b>zeekec</b> (subscriber, #2414)
[<a href="/Articles/342844/">Link</a>]
</p>
</div>
</summary>
<div class="FormattedComment">
Wouldn't that be a great response to a NULL dereference in the kernel.<br>
</div>
<div class="CommentReplyButton">
<form action="/Articles/342844/comment" method="post">
<input type="submit" value="Reply to this comment">
</form>
</div>
<p>
</details>
</details>
</details>
</main></div> <!-- ArticleText -->
</div> <!-- middlecolumn -->
<div class="rightcol not-print">
<ins data-revive-zoneid="12085"
data-revive-id="fa8c6c9da7f33852f7097c4a94da1070"></ins>
<script async src="//servedby.aqua-adserver.com/asyncjs.php"></script>
</div>
</div> <!-- maincolumn -->
<br clear="all">
<center>
<P>
<span class="ReallySmall">
Copyright &copy; 2009, Eklektix, Inc.<BR>
This article may be redistributed under the terms of the
<a href="http://creativecommons.org/licenses/by-sa/4.0/">Creative
Commons CC BY-SA 4.0</a> license<br>
Comments and public postings are copyrighted by their creators.<br>
Linux is a registered trademark of Linus Torvalds<br>
</span>
</center>
</body></html>