Setting the TZ environment variable saves thousands of system calls (2017)

TL; DR

This blog post explains how setting an environment variable can save thousands (or in some cases tens of thousands) of unnecessary system calls that can be generated by glibc in short periods of time.

This has been tested on Ubuntu Precise (12.04) and Ubuntu Xenial (16.04). This probably applies to other versions of Linux as well. It is very easy to test if this applies to you and correct it, if so. Keep reading for more details!

Quick summary

To avoid extra system calls on server processes where you won't update the timezone (or can restart the processes when you do), just set the TZ environment variable to {/etc/localtime( or another timezone file of your choice) for a process. This will prevent glibc from making extra (and unnecessary) system calls.

To understand why and how to test if your processes can benefit, read on!

The problem

In our previous blog post on Linux system calls, we explained the different system call methods and highlighted one very interesting method in particular: the vDSO system call method.

The goal of the vDSO system call method was to create a way for some very frequently used system calls (like gettimeofday, time, clock_gettime, etc.) to avoid having to enter the kernel and cause a context switch from user to kernel ground. The result of this method is that certain system calls, like those listed, can be used by programs at much lower cost.

What if every time you called one of those fast vDSO syscalls (like time), you also called a normal syscall like, say, stat, which doesn't go through the vDSO?

If you did that, you would essentially undo some of the performance boost you were supposed to gain from vDSO optimization in the first place; you would very often make a slow system call.

It turns out this happens quite often with a pair of functions commonly used together:

time: A vDSO-enabled system call used to get the number of seconds since the epoch, and localtime: A glibc-provided function that converts time output to a local time in the user's time zone. This is not a system call, but internally localtime can make a system call in some cases.

The time vDSO system call and the glibc localtime function are often used together in applications, either directly by the programmer or at a lower level without the knowledge of the programmer to format dates and times for everything , from log messages to SQL queries. . This pair is commonly used in Ruby on Rails.

It turns out that the glibc localtime function will check if the TZ environment variable is set. If it's not set (the two Ubuntus I tested don't set it), then glibc will use the stat system call whenever local time is called.< /p>

In other words: your system supports the time system call through the Linux kernel vDSO to avoid the cost of switching to the kernel. But, as soon as your program calls time, it calls localtime immediately afterwards, which still invokes a system call. So you eliminated a system call with the vDSO, but replaced it with another one.

Let's see an example program that demonstrates this behavior, how to use strace to detect it, and finally how to prevent it by setting the TZ environment variable.

Sample...