This post is about hardening /tmp and other temporary directories so that programs cannot be executed from them or privileges escalated. Why would you want to do this? Simply to give potential attackers less of a surface to work with. Many exploits involve the attacker gaining access to the local file system. Once they have access to the local filesystem they can download a payload and run it. It’s well understood that /tmp and other temporary directories can be written to and read from by all. It is less known that files in those directories can also be executed, making them extremely popular with the authors of malware.

Gratuitous picture of a Steyr TMP

I devised the steps outlined here while trying to clear out an infestation of cryptominers. I could see they were using /tmp to run the mining programs, but I couldn’t see the source of the infection. I reasoned that if I could prevent them using temporary directories I could put a lid on the infection while I found the original source. The generally accepted way of doing what I am about to propose is to create genuine disk partitions and mount them in the manner explained here (LVM makes this very easy). However as mine was a managed server with a single large ext4 filesystem, this wasn’t an option for me without a total server rebuild.

So the technique here explains how to create a file, format it as a file system then mount it as /tmp with additional flags to restrict use. This approach is probably not recommended for high-volume servers as it may introduce performance bottlenecks and risk corruption of /tmp.

First up, we need to create 2 files and initialise them as ext4 filesystems (one file for /tmp and another for /usr/tmp). I have chosen to keep the files in a directory called /usr/tmp-noexec.

# Create the directory to hold the files
sudo -i
cd /usr
mkdir tmp-noexec
cd tmp-noexec

# Create the files to hold the temporary file systems and fill them with zeroes
# /usr/tmp seems to have lots of stuff in it so make it 2Gb
dd if=/dev/zero of=tmp.img bs=1M count=100
# /usr/tmp seems to have lots of stuff in it so make it 2Gb
dd if=/dev/zero of=usr-tmp.img bs=1M count=2000

# Format the newly created files as ext4 filesystem
mkfs.ext4 tmp.img 
mkfs.ext4 usr-tmp.img

At this point, we will want to add the entries to /etc/fstab even though we aren’t going to use them just yet. The reason is that, once we’ve copied the data from the existing /tmp, the quicker we can get them mounted the less likely there is to be a loss of temporary data.

/usr/tmp-noexec/tmp.img	/tmp	ext4	defaults,nodev,nosuid,noexec	0	0
/usr/tmp-noexec/usr-tmp.img	/usr/tmp	ext4	defaults,nodev,nosuid,noexec	0	0
# I've also added nodev,nosuid,noexec to /dev/shm - as this was already mounted
# as a separate filesystem I did not need to create a filesystem file for it
tmpfs		/dev/shm	tmpfs	defaults,nodev,nosuid,noexec	0	0

With this done we need to copy the contents of the existing /tmp and /usr/tmp directories in. Strticly speaking we could just set them to mount automatically in /etc/fstab and reboot but:

  • I didn’t want a server reboot
  • Mouting them elsewhere so we can copy the files provides a useful test that the files have been correctly created
# Create some mount directories - can be anything
mkdir -p test/tmp
mkdir -p test/usr/tmp
# Mount the file images as file systems
mount tmp.img test/tmp
mount usr-tmp.img test/usr/tmp
# Copy in the contents of the existing directories
rsync -av /tmp test/tmp/
rsync -av /usr/tmp test/usr/tmp/
# Unmount the file images
umount test/tmp
umount test/usr/tmp
# Tidy up
rmdir test/tmp
rmdir test/usr/tmp
rmdir test

All that remains is to mount the new files as filesystems in place of the old directories

mount /tmp
mount /usr/tmp
umount -f /dev/shm
mount /dev/shm

It’s worth noting that:

  • it was slightly problematic unmounting /dev/shm as it was in use by the attackers. I had to delete the contents then wait about 3 seconds then unmount before their malware redeployed
  • this approach will leave the original /tmp and /usr/tmp directories intact on the original file system. If you unmount /tmp and /usr/tmp you’ll have things back as they were. This might be a good thing or a bad thing depending on your viewpoint

Once I’d completed this operation, I could immediately see the benefit in the Apache error log (/var/log/error_log)

bash: line 52: ./6gC7pKB: No such file or directory
./6gC7pKB: Permission denied
chmod: cannot access './6gC7pKB': No such file or directory
nohup: failed to run command './6gC7pKB': No such file or directory
bash: line 52: ./6gC7pKB: No such file or directory
./6gC7pKB: Permission denied
chmod: cannot access './6gC7pKB': No such file or directory
nohup: failed to run command './6gC7pKB': No such file or directory
bash: line 52: ./6gC7pKB: No such file or directory
./6gC7pKB: Permission denied

I had prevented the attackers running their mining programs and could then isolate the source of the intrusion at my leisure.