Andrew Ayer

If someone asked you whether it was safer to write a setuid program or a setgid program, what would you say? What about a setgid-only program versus a program that was both setuid and setgid? Instinctively, I would say setgid-only: setgid grants fewer privileges, which by principle of least privilege ought to be a good thing.

Not so. Here is a story that convinced me that setgid-only programs are inherently less safe than setuid programs.

The TAs at Brown's Computer Science department keep track of student grades using a homegrown system written in Python with some shell thrown in. One common command is report STUDENTLOGIN, which lists the grades for a particular student. The grades are stored in a file readable and writable only by the course's TA group, of which the TA staff are members.

Someone decided that it would be nice for students to be able to check their own grades at any time. So a setgid wrapper program called "mygrades" was written in C. It was owned by the course TA group and exec'd the report command with the username of the real user ID (i.e. the student who invoked the program) as the first argument. The wrapper program was written very carefully and scrutinized closely. In particular, the environment was completely wiped and replaced by a safe environment. What could possibly go wrong?

As it turns out, a lot. Here is a simplified version of the report command, which is a KornShell script that determines how it was called and execs the Python code appropriately:

#!/bin/ksh # Assume that this script is installed one directory down from the course directory root (e.g. in tabin) export COURSEDIR=$(readlink -f "$(dirname "$(whence "$0")")/..") export EVALPIG_ROOT=$(readlink -f "$(dirname "$(readlink -f "$(whence "$0")")")") export PYTHONPATH=$COURSEDIR/config:$EVALPIG_ROOT exec python -O -c "__import__('$(basename "$0")').main()" "$@"

Can you spot the vulnerability? Keep in mind, the environment is completely safe. So are the arguments (including $0).

The problem is that KornShell implements command substitution (i.e. $(...)) by creating a temporary file (in /tmp), redirecting the output of the command to the temp file, and then reading it into the variable. Since mygrades is a setgid wrapper, the user ID does not change. Consequentially, the temporary files are owned by the user who invoked mygrades. The user can write to one of the temporary files and as a result, inject arbitrary data into one of the script's variables. This is bad for any script, and really bad for the script above. An attacker could effectively set an arbitrary PYTHONPATH and thus execute arbitrary Python code with the permissions of the TA group.

Of course, the temporary file exists only for an instant (it's unlinked immediately after being opened), but I easily wrote a program that monitored /tmp with inotify and opened the file as soon as it was created. It didn't work every time, but if I ran mygrades in an infinite loop my attack would succeed within a minute.

What is counter-intuitive is that if the mygrades program had been setuid, this wouldn't have been a problem. The temp files would have been owned by the effective UID of mygrades and would have been tamperproof. Linux (and any decent Unix implementation) otherwise does a very good job protecting setgid programs from tampering by the process owner: the owner of a setgid process isn't allowed to ptrace it, debug it, etc. But the kernel can do nothing to protect the files that a setgid program creates on the filesystem. For this reason, unless you are sure that a program will not create temporary files at any point during its execution, or you are sure that a tampered-with temp file will not lead to a security compromise (which is probably a bad assumption), it is safer to make a program setuid than setgid-only.

Update: See my followup article, Easily Running FUSE in an Isolated Mount Namespace, for a solution to this problem.

FUSE is cool, but by its nature has to introduce some non-standard semantics that you wouldn't see with a "real" filesystem. Sometimes these non-standard semantics can cause problems, as demonstrated by a recent experience I had with rsync-based backups on a multi-user Linux server that I administer.

The server performs regular snapshot backups of the filesystem using rsync and its --link-dest option to provide hard link-based dedupe. One day, an enterprising user mounted a sshfs filesystem in his home directory. That night, the snapshot backup failed with this error message:

rsync: readlink_stat("/home/aganlxl/cit") failed: Permission denied (13) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1060) [sender=3.0.7]

The problem is that, by default, only the user who owns the FUSE mount is allowed to access it. Not even root can. This is a security measure, since a malicious user could wreck havoc with a malicious FUSE mount (imagine an infinite filesystem, for example).

This behavior can be changed (with the 'allow_root' mount option, which has to be explicitly enabled by the super user in fuse.conf), but that's not the answer. Besides the security implications, that would cause rsync to descend into the sshfs mount and start backing up the remote system!

The problem is that rsync needs to be able to access everything it's backing up. Running as root, this is usually not a problem. Root not being able to access something on the filesystem seems weird, but is actually nothing new - root-squashed NFS mounts can also cause this. But FUSE mounts are worse. To begin with, unlike root-squashed NFS mounts, users are allowed to plop down FUSE mounts anywhere they like.

But worse, root isn't even allowed to stat FUSE mounts. This means that rsync's -x option (to not cross filesystem boundaries) can't even be used to exclude FUSE mounts, since rsync needs to stat the directory to determine if it's a mount point! This behavior is outside of POSIX, which says that stat shall return EACCES if "search permission is denied for a component of the path prefix."

In my opinion, a worthwhile addition to fuse.conf would be an option to restrict FUSE mounts to specific directories. With such an option, the admin could restrict FUSE mounts to locations that aren't backed up.

Until then, FUSE is disabled on this particular server.

One problem with executing commands via ssh (that is, on ssh's command line, not via an interactive login shell) is that the command isn't terminated when the ssh connection dies. You can see this by running:

ssh otherhost /bin/sleep 600

and interrupting ssh with Ctrl+C. On otherhost, sleep will still be running. Its parent, the sshd process forked to handle the connection, will be gone, and sleep will have been reparented to init (PID 1).

You don't get this problem when you use ssh interactively. All the processes that you start have a controlling terminal, and when the connection dies and the controlling terminal goes away, the processes you started are killed with SIGHUP (unless they detached from the controlling terminal, such as with setsid).

One solution is to always allocate a terminal, by specifying ssh's -t option:

ssh -t otherhost /bin/sleep 600

But this isn't always feasible, especially if you're running ssh from a script which doesn't have a terminal.

We need a way to kill the remote command when its parent sshd process dies. Fortunately, on Linux, the prctl syscall provides a solution:

PR_SET_PDEATHSIG (since Linux 2.1.57)

Set the parent process death signal of the calling process to arg2 (either a signal value in the range 1..maxsig, or 0 to clear). This is the signal that the calling process will get when its parent dies. This value is cleared for the child of a fork(2).

We can write a simple C wrapper, called diewithparent, which calls prctl and then execs the command:

int main (int argc, char** argv) { prctl(PR_SET_PDEATHSIG, SIGTERM); execvp(argv[1], argv + 1); return 127; }

And use it like this:

ssh otherhost diewithparent /bin/sleep 600

Download the complete C source, which features error checking and options parsing (so you can specify the signal number). (Compile with cc -o diewithparent diewithparent.c.)

Naturally, this is a Linux-only solution. On other systems, the best solution (as far as I can tell) would be to fork and exec the command. In the parent, continuously poll the parent PID (with getppid()). When it changes to 1, you know the parent has died so you kill the command.