20-year-old Linux workaround still slows down AMD systems

Enlarge / A second-generation Epyc server chip from AMD, which may have been running 2002-era Linux code, slowing it down.

Getty Images

AMD has come a long way since 2002, but the Linux kernel still treats modern Threadrippers like Athlon-era systems, at least in one potentially lag-inducing way.

AMD engineer Prateek Nayak recently submitted a fix to Linux‘s idle CPU drivers that “would skip the bogus expectation for CPUs based on the Zen microarchitecture.” When ACPI support was added to the Linux kernel in 2002 – written by Andy Grover, committed by Linus Torvalds – it included a “dummy wait operation”. The system essentially reads the data with no other purpose than to delay the next instruction until the CPU can completely halt with the STPCLK# command. This provided some power saving and compatibility during the early days of ACPI implementation when some chipsets did not enter an idle state when expected.

But today’s Zen-based AMD chips don’t need this workaround, and as Nayak writes, it hurts them, at least in specific Linux workloads. Tests with instruction-based sampling (IBS) workloads show that “a significant amount of time is spent in the dummy operation, which is incorrectly counted as a C-State residence”. The processor, seeing all this low-effort dummy work, may push into a deeper, slower C-state, which then makes the processor take longer to “wake up”, especially on work that requires many switches between busy and idle states.

Nayak ran tests in tbench on a dual-socket Zen3 system against the stock Linux kernel, a kernel with the C2 state fully disabled, and a kernel with the dummy wait operation corrected. Its patched version saw a 1390% increase in minimum MB/s throughput and a 51% increase in average MB/s over the stock kernel, often just a little behind disabling C2 completely.

Intel systems have avoided AMD’s legacy curse because they’ve been using an MWAIT-based system for at least a decade, according to the Phoronix blog. This led to an urgent patch submitted by Intel’s Dave Hansen. His solution was to limit the “mock wait” to Intel systems, where it wouldn’t affect “remote modern Intel systems”, and to add comments to inactive kernel drivers that explain what’s going on – and encourage those which reads to “consider moving your system to a more modern idle mechanism.”

If an urgent patch removing or limiting “dummy wait” is submitted this week, it could likely create the Linux 6.0 kernel, which Torvalds expects to ship next week.


Comments are closed.