Side channel attacks such as Spectre or Meltdown allow data leakage from an
unwilling process. Until now, transient execution side channel attacks
primarily leveraged cache-based side channels to leak information. The very
purpose of a cache, that of providing faster access to a subset of data, enables
information leakage. While the world focused on a string of exploits leveraging
caches (and the memory hierarchy pyramid) and defenders tried to block data
leakage through it, we look at the core tenet enabling the channel: contention.
Contention in a CPU is not limited to cache capacity; it manifests itself in a
variety of forms when resources are shared. Freely sharing resources among
untrusted entities allows an attacker process to infer when another (victim)
process is contending for the resource, thereby slowing down the attacker.
A less obvious form of contention arises in Simultaneously Multi-Threaded (SMT)
cores. The latter lay the foundation for nearly all modern x86 CPUs, IBM POWER8,
Oracle T5 and Cavium ThunderX2/X3. In SMT, scheduling units called ports are
shared among threads of execution which can be exploited to leak information.
Port contention as a phenomenon has been previously discussed by Anders Fogh in
his 2016 blog post.
We precisely characterize the port-induced side channel (that we call SMoTher)
and demonstrate that it is possible to detect a sequence as small as a single
(schedulable) instruction tied at design time to a specific subset of ports by
leveraging contention. Leveraging SMoTher (instead of a cache-based side
channel), we present a powerful, practical transient execution attack to leak
secrets that may be held in registers or the closely-coupled L1 cache, called
SMoTherSpectre. The full paper is on arXiv,
the work is a collaboration between the EPFL HexHive
and PARSA labs, and IBM Research
Zurich and joint work between Atri
Bhattacharyya, Alexandra Sandulescu, Matthias Neugschwandtner, Alessandro
Payer, and Anil Kurmus.
Simultaneous multi-threading and scheduling
A Simultaneously Multi-threaded (SMT) CPU fetches and executes instructions for
more than one thread on the same core. To the operating system/user, it appears
as a greater number of logical cores than physical cores. The former is used
to denote the capability to execute a thread, while the latter is the physical
implementation of a unit (execution pipeline, registers, caches) called a core.
These (colocated) threads have a few dedicated components per thread (fetch unit
and architectural registers), while sharing the rest of the pipeline (branch
predictors, reservation station, ports, execution units). Implementations differ
in which components are shared or dedicated and the number of threads per
A typical modern, out-of-order processor schedules micro-ops from an unified
reservation station to specialized execution units. See this
for an overview of scheduling on recent Intel microarchitectures. Core-series
processors contain 5-8 ports to perform this scheduling. Each port is
responsible for a fixed subset of execution units. Intel Skylake processors,
specifically, contain eight ports. Four of them (0,1,5 and 6) are used to
schedule operations to integer, floating-point, vector execution units among
others. The other four ports handle loads, stores, and address generation
operations. The execution units for the most commonly executed micro-ops are
replicated and associated with multiple ports. With SMT, micro-ops from both
co-located threads may reside in the same reservation station(s). In each cycle,
a single micro-op from either thread may be scheduled by each port. See
scheduling Ports on Intel Skylake
for details of Intel Skylake scheduling.
When SMT threads have ready micro-ops which can use the same port, they must
contend for it each cycle. Each thread would need to wait for a few cycles when
the port under contention chooses to schedule a micro-op from another thread,
causing a slowdown. This slowdown is detectable (by taking timestamps using
rdtsc on Intel's CPUs), and allows a specially crafted thread to measure the
co-located thread's utilization of a port.
Suppose threads A (Attacker) and V (Victim) run on the same physical Skylake
crc32 is scheduled by port 1 and
fadd is scheduled by port 5. If
a thread V only uses other ports (example, running
fadds), thread A running 20
crc32 instructions should require 20 cycles. However, if the
reservation station also contains a single ready
crc32 instruction from
thread V, there should be one cycle where port 1 chooses it over the micro-ops
from A. Overall, thread A now runs the same instruction in 21 cycles, which is a
5% slowdown. Longer sequences with contention have lead to attacker slowdown
up to 35% in our experiments.
Each instruction in a sequence of code can be scheduled on specific ports. This
allows us to create a port-fingerprint for every sequence, consisting of the
expected utilization of each of the ports while scheduling that sequence. For a
pair of victim sequences, V_a and V_b, with different port-fingerprints, a
carefully crafted attacker thread can identify which victim sequence is
concurrently run. Essentially, the attacker chooses one or more ports for which
the victim sequences differ in the signature. By timing instructions
specifically scheduled on these ports, the attacker can measure contention.
Higher contention means that a concurrent victim is using the same ports (and
vice versa), identifying the sequence. We call such pairs of instruction
To leak information, we look at data-dependent control flow (conditional
branches) leading to SMoTher-differentiable sequences (as branch target and
fallthrough). We call this a SMoTher-gadget. The attacker can identify the
sequence following the branch, and thereby infer the outcome of the condition.
The information leaked depends on the branch condition. Common examples include
specific bits in registers or memory (for example,
TEST 0x1, al; jz TGT, or
CMPB 0x0, (rdx); jl TGT).
SMoTherSpectre is a speculative code-reuse attack. Speculative execution at
particular points of a victim's execution is influenced to execute
SMoTher-gadgets, leaking information.
As an example, an attacker can use branch target injection (BTI) to redirect the
speculative execution following an indirect jump/call (until the target is
calculated) on a co-located victim process (shared branch predictor). This code
sequence, including the indirect branch, is named the BTI gadget. At this
point, we require a register or a memory location (with a pointer to it) to hold
the secret to be leaked. The code below is an example BTI gadget, where a secret
is loaded into
rdi and the indirect jump will eventually branch to the
pointer target loaded into
load rdi, (secret)
load rax, (pointer)
The poisoned target leads the victim to execute a different data-dependent
control-flow sequence--i.e., a conditional branch--somewhere in the victim code.
The condition must use the secret, so that the branch leaks the outcome of the
condition if the attacker can figure out which instructions the victim executed
just afterwards (i.e., either the branch target or the fall-through). For
SMoTher-gadgets, conditional branches where the two subsequent paths are
SMoTher-differentiable, the attacker can figure out if the branch was taken or
not taken by introducing contention on one or more ports which the target and
fallthrough use for different number of cycles. In the SMoTher-gadget shown
crc32 is scheduled by port 1 on Skylake, while
ror is scheduled on
ports 0 and 6.
cmp rdi, 0
An attacker running a sequence of
crc32 instructions will contend on port 1
if the victim branch falls though and runs
crc32 instructions too. The
slowdown due to contention can be detected by the attacker, using
timestamps to count the number of cycles taken to run its sequence, and allows
it to infer that the victim's secret is not less than 0.
SMoTherSpectre requires two gadgets in the victim code base:
- A BTI gadget (to trigger speculation):
C/C++ compilers typically use indirect call instructions to implement calls
using function pointers/virtual-function calls in code which does not deploy
repoline defences. OpenSSL's EVP library uses such a pointer to, e.g.,
encrypt/decrypt using the selected cipher.
i = ctx->cipher->do_cipher(ctx, out, in, inl);
Further, a secret argument may be held in registers. In the example above,
rdx holds a pointer to the secret plaintext being encrypted.
- A SMoTher gadget (to leak the secret):
Every conditional branch in a victim's binary is a potential SMoTher gadget.
In fact, even unintended sequences which may be interpreted as a conditional
jump can be used (similar to ROP). However, an attacker needs to be able to
differentiate between the target and fallthrough sequences using port contention.
It turns out that hundreds to thousands of such sequences exist in
different degrees of SMoTher-differentiability). The paper describes in more
detail our methodology for finding and ranking SMoTher-gadgets.
The vast availability of SMoTher gadgets makes SMoTherSpectre such a powerful
attack. While the first stage is similar to other speculative execution attacks,
the side channel to leak information is different and more readily available
than cache-based side channels. While each step leaks only one bit of
information (the conditional branch that depends on the secret value was taken
or not taken), SMoTher-gadgets are more readily available and can be combined
the leak information.
We release the proof of concept
code to enable other researchers to
reproduce, evaluate, and assess this side channel. It uses the gadgets described
above. Separate processes are used for the attacker and victim. Per iteration of
the experiment, a randomly-generated bit (representing the victim's secret) is
written by the victim process to a file, while the attacker writes its guess to
another file. For each secret, we repeat the attack multiple times to allow the
attacker to get multiple samples. Multiple samples allows the attacker to
eliminate some of the noise that invariably creeps into a timing experiment at
such fine granularity. We run 1,000 iterations and compare the files in
post-processing to calculate the attacker's accuracy in guessing the secret.
The histogram of the attacker's timing for the
crc32 sequence in the SMoTher
phase shows separate plots for when the actual secret is zero or one.
Specifically, the attacker's timing is the average over 9 runs with the same
secret. This figure clearly shows that an attacker can use a threshold of 94
cycles to make a guess of the secret with high confidence.
Overall, our attacker was able to guess the secret with an accuracy from 60%
(with one sample) to 98% (with 9 samples).
We also created a concept exploit for OpenSSL's (commit
27-Nov-2018) high level EnVeloP (EVP) API. We modelled a victim program using
OpenSSL to encrypt data. Specifically, the program calls
encrypt chunks of data. An indirect call in the function serves as our BTI
At the BTI gadget, the register
rdx holds a pointer to the plaintext being
encrypted (victim secret). A SMoTher-gadget from
glibc is used, comparing the
first byte in memory referenced by
rdx against zero. In effect, this is
leaking information about the first byte of the plaintext.
The distribution of the attacker's timing with different secret values are
distinguishable by statistical tests such as Student's t-test, implying that an
attacker which is able to run multiple encryption runs of the same plaintext can
identify the secret.
Mitigating SMoTherSpectre is possible by mitigating the port contention side
channel or the transient execution side channel (BTI in our PoC).
There are a range of mitigations for BTI (commonly known as Spectre v2
mitigations), including enabling the Single Thread Indirect Branch Predictors
(STIBP) feature on Intel processors. We saw that microcode updates
2017-07-07 and after as published by Intel prevented BTI on our released PoC
code. All security-critical userspace programs should be compiled with
However, BTI is one of multiple avenues for influencing indirect branches (or
returns) on victim processes. Newer Spectre variants continue to propose
alternate methods for influencing branch speculation. Therefore, defense against
SMoTher is required to fully mitigate this transient execution attack.
The general idea of preventing SMoTher leaking information is to ensure that two
threads with different privileges (in the general sense) do not compete for the
same execution port. An obvious scenario is threads from separate users sharing
a physical core. However, in certain cases, threads from the same Linux
can represent different mutually-untrusting entities. Therefore, the strongest
defence is disabling simultaneous multi-threading.
We discovered the SMoTher side channel in June 2018 and developed the
SMoTherSpectre speculative side channel proof of concept in November 2018. The
co-authors at IBM Research disclosed the findings internally to IBM. We
disclosed the vulnerability to Intel (on December 05, 2018) and to OpenSSL (on
December 05, 2018). AMD was also notified as part of IBM's internal disclosure
process. After the acknowledgement of receiving our PoC, we did not receive any
feedback from Intel or OpenSSL. The IBM internal disclosure process completed on
February 28, 2019 and we are releasing the details of the vulnerability on March
The full paper is on arXiv. The PoC
code enables reproduction. Contact:
Mathias Payer or @gannimo on