Recently I’ve been writing code to spawn child processes that had to deal with the POSIX functions for querying and manipulating environment variables. I’ve only just realised how truly awful this interface is in the context of modern multi-threaded applications, and this post is simply me sharing the pain.
Many Unix, and some Windows, users will be familiar with environment variables.
These are key/value strings such as
SHELL=/bin/bash, and they
form part of the global environment provided to a process by the OS. Windows
has a similar concept, although it has a few subtle differences and in this
post I’m only discussing the situation in POSIX.
getenv() function is pretty modest - you pass in the name of an
environment variable and it returns you a pointer to the value. Simple enough,
but immediately the spidey sense starts tingling. The function returns a
char* instead of a
const char* for one thing, but “the application shall
ensure that it does not modify the string pointed to by the
function”. Well, OK, perhaps they didn’t have
const in the days this function
was written. They also presumably hadn’t heard of thread-safety or re-entrancy,
because anything that returns a static pointer pretty clearly does neither.
setenv() function is also fairly simple - you pass in a new variable
name and value, and a flag indicating whether you’re happy for the assignment
to overwrite any previous value. But the man page talks about this function
modifying the contents of
environ - oh yes, let’s talk about that first…
You’ll notice neither of the functions so far has given a way to iterate through
all the current environment variables that are set. It turns out that the only
POSIX-supported way to do this is use the global
environ variable provided by
the library. This is similar to
argv that’s passed into
main() except that
instead of an
argc equivalent, the
environ array is null-terminated. Things
start to smell a little fishy when you realise that
environ isn’t actually
declared in any header files - the application itself has to include something
like this, as taken from the POSIX page on environment variables.
extern char** environ;
OK, so just like
argv there’s some OS-supplied storage that contains the
environment. It’s not
const, but hey ho, neither is
argv and we seem
to cope fine with just not modifying that directly. Except that the key
point here is that
setenv() does modify
environ - the man page even
explicitly states that’s how it works. Unlike
argv, therefore, you can’t just
treat it as some effectively some read-only constant1 array
and quietly ignore the fact that the compiler won’t stop you modifying it.
It gets even more crazy when you realise that, according to
the man page for the exec family, it’s quite valid to replace
your entire environment by assigning a whole new value to
read that correctly - not updating the pointers within
repointing the whole thing at your own allocated memory.
So then, when
setenv() comes along and wants to modify this, how on earth
can it do so? It has no idea how big an array you’ve allocated in your own
code - it either has to copy the whole lot to somewhere provided by the
system, or cross its fingers and hope there’s enough space.
And don’t even get me started on the memory management Pandora’s Box that
In summary, therefore, I’ve decided that the only sensible course of action
is to use environment variables as little as possible. If you must use them
as opposed to command-line arguments, you should parse them away right at
the beginning of
main() and put them into other storage within your code,
never worrying about the environment again. If you’re writing a library…
Well, good luck with that - let’s hope your application doesn’t mess around
with the environment too badly before you want to query it. Whatever you
do, don’t update it!
It’s quite possible to work around all this brokenness, of course, as long as you can make some basic assumptions of sanity about your libraries. But it’s all just such a dirty little mess in the otherwise mostly sensible tidiness that POSIX has imposed on the various APIs that exist.
Surely there’s got to be a more sensible way to control the behaviour of applications and libraries? For example, we could have some sort of system-wide database of key/value pairs - unlike the environment it could be lovely and clean and type-safe, and all properly namespaced too. For performance reasons we could stick it in some almost unparseable binary blob. There’s no way such a system could be abused by applications, right? It would remain clean and usable, I’m sure. Now all we need is a snappy name for it - something that indicates the way that values can be registered with it. Perhaps, The Register? No, people will confuse it with the online tech site. What about The Repository? Hm, confusing with source control. I dunno, I’ll think about it some more.
Yes, I’m aware there are some use-cases for modifying
argv too, but I class those as unusual cases, and they also tend to be quite system-specific (assuming you want to resize the strings in the process). ↩