Daemonic Dispatches

'Tis the season for giving

Whether you celebrate Christmas, Hanukkah, Yule, Pancha Ganapati, Hogmanay, Newtonmas, or simply the end of the Gregorian year, odds are that you're giving gifts some time around now. We give gifts to family; we give gifts to friends; we donate to charities; and many people also offer up tithes to religious institutions. Gifts to individuals are a social bonding ritual — the voluntary transfer of wealth signals a lower bound on the value we place on a relationship, and the giving of non-monetary gifts in particular can be a way to communicate our level of personal understanding — but these do not apply to charitable and religious donations. For those, I think an entirely different explanation is required: We pay voluntary taxes in order to help create the world we want to live in.

This also applies to companies. I run an online backup service, and for the past two years I've donated all of the profits made during the month of December to the FreeBSD Foundation; I'm going to be doing the same thing this year too. I'm not doing this just because I'm a FreeBSD developer, because I use FreeBSD personally, or because I would never have launched Tarsnap if I hadn't been able to build on the open source code in FreeBSD: I'm doing it because I think supporting FreeBSD development will make the world a better place for both Tarsnap and many other startup companies.

I'm not alone in believing in corporate support of open source software, either. NetApp and Hudson River Trading, both major FreeBSD users, have each made donations of $50,000 or more in each of the past 3 years, and many other companies regularly donate. Some open source software organizations have much longer lists of major donors. And last year Gabriel Weinberg launched a FOSS Tithing movement by pledging that DuckDuckGo would tithe in support of open source software every year.

Most internet startup companies today would never exist without open source software. As Paul Graham has noted, open source software is one of the big reasons why it's now possible to launch a startup with just $20k and a few months of coding; with high quality free operating systems, databases and datastores, application frameworks, web servers and caches, it's now easy to build companies which would have been nearly impossible a decade ago. It would be easy to say that startup companies should contribute back to open source projects out of simple gratitude, but I know it can be hard to justify making business decisions on that basis alone. Instead, I'd like to ask the startup community to look to the future: Think about how much open source has helped you, and help to build a better world — one where open source will be able to help you even more.

And remember that we live in a world where most startup founders end up launching several companies over their careers: If the past decade of open source software development has made your current startup company possible, just think how much the next decade of open source software development will help your next startup company.

POSIX close(2) is broken

In the world of POSIX, everything is a file. Well, sort of. There's sockets and pipes, which behave rather like files except that you can't seek on them and they have some extra metadata. And there's devices, where sometimes you can only read and write appropriately-sized blocks, not individual bytes. And then there's terminals, which are all sorts of weird. But in all these cases, you've got a file descriptor, and when you're finished you release the resource by calling the close(2) system call.

There's just one small problem: The way POSIX has defined close(2) is completely and utterly broken.

A few days ago, Taylor R Campbell — the same guy who reported Tarsnap's incredibly stupid crypto bug back in January 2011 — sent me an email pointing out some peculiar language in the standard:

If close() is interrupted by a signal that is to be caught, it shall return -1 with errno set to [EINTR]...

Now, we're used to dealing with EINTR: Almost every system call we make can be interrupted and return EINTR. No problem: We just reissue the system call — keep trying until we get through without a signal interrupting us. That would be fine, except for these last few words:

... and the state of fildes is unspecified.

If close(2) returns EINTR and you call it again, you might get an EBADF ("this is not an open file descriptor") error back. Even worse, if you are running in a threaded process, a different thread might have opened a file and been assigned the same descriptor value, at which point your second close(2) call can succeed... at closing the wrong file. Throw in another file being opened and you've now got silent data corruption. On the other hand, if close(2) returns EINTR and you don't call it again, you might be leaking an open socket. For a short-lived process this might not matter, but it's certainly not something you want to do in a long-lived server.

After several days, I see a couple imperfect solutions. The first one is a refinement of the standard EINTR loop:

int
threadunsafe_close(int fd)
{

	if (close(fd) == 0)
		return (0);
	if (errno != EINTR)
		return (-1);
	while (close(fd)) {
		if (errno == EBADF)
			return (0);
		if (errno != EINTR)
			return (-1);
	}
	return (0);
}

Obviously that first solution, while working fine on single-threaded processes, is not safe with multiple threads due to the potential race against open(2) or some other system call reallocating the descriptor. A second option avoids the problem by preventing EINTR:

int
blocksignals_close(int fd)
{
	sigset_t set;
	sigset_t oset;
	int rc;
	int errno_saved;

	if (sigfillset(&set))
		return (-1);
	if (pthread_sigmask(SIG_SETMASK, &set, &oset))
		return (-1);
	if ((rc = close(fd)) != 0)
		errno_saved = errno;
	if (pthread_sigmask(SIG_SETMASK, &oset, NULL))
		return (-1);
	if (rc)
		errno = errno_saved;
	return (rc);
}

Unfortunately this second solution is also less than ideal: It can fail in several different ways (at least theoretically — POSIX doesn't define any errors which sigfillset or pthread_sigmask could return here, but implementations are allowed to invent other reasons to fail), which makes deciphering errno harder if -1 is returned; but more importantly, by blocking signals while calling close(2) it stops us from hitting ^C to interrupt the process if close(2) blocks for a long time (e.g., if it is causing data to be flushed to an NFS-mounted filesystem over a failing network).

What makes this problem particularly annoying is that there is no need for POSIX to have this ambiguity. Many operating systems solve this problem by simply not allowing close(2) to return EINTR; this is always going to be possible by the simple tactic of having close(2) mark the descriptor as deceased and then garbage collecting asynchronously, if nothing else. But even if close(2) can fail with EINTR, there's no reason to leave the descriptor state ambiguous: Immediately before returning to userland, the kernel simply needs to look at the descriptor and ask itself "is this descriptor still open?" — and then return success or EINTR respectively.

I hope that a future revision of the standard fixes this. In the mean time, anyone wanting to safely close a file descriptor without assuming more than the standard specifies has a lot of work to do.