Linux Context Switching

The CPU has various C-States (Core States) that it can be changed to when idle, giving different levels of sleep, which enable better power saving. The relevant states range from fully powered up C0, down to deeper sleep C4 with very a low voltage supply.

For Intel chips, modern Linux kernels handle the C-States via the intel_idle driver. The Intel driver is used to drop the processor into lower states when in brief idle periods until it is needed; when needed the CPU will wind back up and deal with the interrupt. The CPU wind up can take time, which has an influence on context switching between threads.

Check what driver is being used:

cat /sys/devices/system/cpu/cpuidle/current_driver
intel_idle

Here the intel_idle driver is in use.

Speedy Context Switches

To squeeze as much speed out of the system when processing in very low latency, we can stop the CPU dropping into these deeper sleep states. To do this we configure the kernel to override the driver by using parameters on boot-up.

C-State Managed by BIOS

To switch off the driver and hand over c-state management to the BIOS firmware, you use the kernel parameter:

intel_idle.max_cstate=0

This will likely result in the core dropping from the C0 running state down to the C1 idle state when inactive. The wake-up latency isn't too bad in this state.

edit -- RedHat recommend:

processor.max_cstate=1

Keep the CPU in C0

To keep the processor active, you can switch it into a polling idle state.

intel_idle.max_cstate=0 idle=poll

This will keep the processor cycling in C0, constantly polling to see if it is needed; the result is the very lowest latency for wake-up and context switching between threads at the expense of higher power usage. With this setting, threads will switch in a small number of microseconds.