This blog post is a summarized and introductory write up of our paper recently
accepted at ESORICS 2026
to be presented in Rome, Italy between September 14-16, 2026: “A Systematic Look
at Quality-of-Service Feature Effects on Side Channels in AMD SEV-SNP”. Not all
the scientific details are mentioned in this blog post. If you’re interested,
there’s a full paper with more rigorous, peer-reviewed-and-accepted prose.
During our research, we referred to our work as Business Cat, or a cat with an
MBA (it will make sense later). Here’s an illustration of a Business Cat drawn
by my sister:
Introduction: Virtualization and Confidential Computing
Hardware-assisted virtualization (Intel VT-x, AMD SVM or AMD-V) have existed
since the mid-2000s. The
hypervisor (or VMM) is responsible for telling the CPU which virtual machine is
to be run, and the CPU hands control back to the hypervisor when the virtual
machine requires some support, e.g., emulation. This control that the
hypervisor has over a VM has not been so great from a security standpoint, since
the hypervisor can determine what’s happening inside the VM, or make it
incorrectly execute its code (largely oversimplified). Running such traditional
VMs on computers you don’t own in a data center whose location may be unknown
can be quite… unnerving — what if the cloud provider sees all your data?
The idea of confidential computing has been
around for quite some time, but the hardware support really came within the last
decade or so. In the confidential computing paradigm, the confidential VM’s
(CVM’s) execution isn’t that dependent on the hypervisor anymore, but rather
the CVM depends more on the CPU. While the CVM’s kernel should be aware that
it’s being run in this confidential mode, user applications can mostly be
“lift-and-shifted” into the CVM1.
AMD’s confidential computing suite is called Secure Encrypted
Virtualization, and their third and
latest generation is called SEV Secure Nested Paging (SEV-SNP). Intel also has
their confidential extensions called Trusted Domain
Extensions
If you’re interested in more technical details and a historical perspective of
virtualization and confidential computing (on AMD CPUs specifically), I wrote
about it in my master’s thesis (Chapter 2).
Threat Model
A quite powerful adversary that arises with confidential computing and trusted
execution environments is the malicious / compromised hypervisor. The CPU
guarantees protection of “data in use”, i.e., data being computed upon. The
hypervisor can, therefore, do whatever it wants to the system in order to
exfiltrate data about the CVM. Meanwhile, the CPU is supposed to protect the
CVM. The hypervisor is only limited by what the CPU prevents; apart from that,
it’s all game.
AMD CPUs offer the Platform Quality of Service (PQOS) features. These are split
into two categories: allocation features and monitoring features2. The
allocation features allow system software to split and provision resources
like cache and memory bandwidth, so that one task (process / VM) doesn’t hog
that entire resource. The monitoring features enable system software to
monitor the aforementioned resources, so that system software can make decisions
based on the resources’ usage (Should it throttle a process’ bandwidth? Should
it restrict its L3 cache usage?). Check out Resource
Provisioning in the cloud
if this sounds interesting.
Intel also offers similar features with their Resource Director
Technology
(RDT). Note that both Intel and AMD’s technologies share similar feature names,
but differ slightly. We’ll be using the Linux kernel’s
convention, which mainly goes
with Intel’s naming convention (probably because the documentation page was
written by Intel engineers :D).
The Quality of Service features are split between allocating and monitoring technologies.
Resources are partitioned amongst Classes of Service, which currently is
assigned at per-logical core level (SMT thread). Multiple logical cores can be
assigned to the same Class of Service, i.e., that they can all be given the
same share of the resource. On AMD CPUs, these classes of service are further
split between core complexes, i.e., system software can split and monitor
resources per-core-complex.
There are five notable QoS features that we investigated: CAT, CDP, MBA, MBM,
and CMT, which we discuss below.
> Cache Allocation Technology (CAT)
CAT partitions the L3 cache ways. On our AMD systems, there are 16 L3 cache
ways, i.e, the L3 cache can be split into 16 independent chunks for all the
logical cores of a core complex.
L3 cache split with core's 0 and 2 being assigned to the same four cache ways, i.e. 4/16th of L3 cache, core 1 assigned to two cache ways, and core 3 assigned to one cache way
> Code & Data Prioritization (CDP)
CDP allows system software to partition the L3 cache between code and data. One
advantageous scenario of this could be when processing and streaming data, the
cache lines corresponding to code will remain in cache and not be evicted by the
data streaming.
L3 cache split with core's 0 and 2 being assigned to the same two cache ways for code and two other cache ways for data, core 1 assigned to one cache way for code and data each, and core 3 assigned to one cache way for code and data each
> Memory Bandwidth Allocation (MBA)
MBA, called L3 External Bandwidth Enforcement (L3BE) by AMD, splits the memory
bandwidth, i.e., the rate of data being transferred between CPU cache and
memory. System software can throttle a Class of Service’s memory bandwidth in
multiples of 1/8 GB/s. We find that AMD’s implementation has a common rate limit
for a Class of Service. When SMT-thread 1, assigned to the same Class of Service
as SMT-thread 2, applies pressure on cache-to-memory bandwidth, SMT-thread 2
also slows down since they’re in the same Class of Service.
At less restrictive limits (like the default unrestricted limit we use on our
AMD laptops everyday), this effect is barely visible. At more restrictive
limits, fewer transfers are required to hit this threshold for rate limiting.
Furthermore, the slow down is also different for the MBA limit applied:
Here’s a very simplified representation of what we think goes on under the
hood:
Core 0 and 2, part of the same Class of Service, are restricted the most; Cores 1 and 3 are unrestricted: their data flows between cache and RAM without any slowdowns or rate limiting. Note that this limit is applied in both directions: between cache and RAM.
Intel’s MBA behaves differently than AMD’s MBA implementation; furthermore,
Intel has more than one generation of MBA. See Memory
Band-Aid.
> Memory Bandwidth Monitoring (MBM)
MBM reports the amount of data transferred between CPU cache and memory3. On
AMD systems, our investigation shows that the CPU reports this information with
64 byte resolution. Since CPU cache lines are 64 bytes large, MBA on AMD
systems reports data transfer with a cache line size resolution. This is
incredibly precise.
> Cache Monitoring Technology (CMT)
CMT reports the amount L3 cache occupied by a specified group of SMT
threads3. On AMD systems, the CPU reports this information in multiples of 16
KiB.
The Side Channel
So we have two features which report data: Memory Bandwidth Monitoring (MBM) and
Cache Monitoring Technology (CMT). Well, what if a malicious hypervisor utilizes
these numbers to infer what a Confidential VM (CVM) is doing?
Furthermore, what if the hypervisor employs the allocation features (CAT, CDP,
and MBA) to amplify the inference?
We had a hypothesis. When data goes from memory to cache, the CPU caches it
while that data is used and reused, until that piece of data is evicted. That
would show up as one blip of data transfer using MBM. Well, what if we used CAT
to shrink the cache, so that data might end up being evicted to memory early.
If the CPU needs that piece of data again, well, it’s going to be transferred
back to cache, showing up as an extra blip of data transfer4.
We tested this theory out, and well what da ya know. We have some side-channel
attacks.
Our work was performed on our local AMD EPYC 8024P (Zen 4c) machine. Both the
hypervisor (Linux 6.11) and spawned CVMs (Linux 6.8) ran Ubuntu 22.04.
> Bleichenbacher-style attack on RSA
Bleichebacher’s
attack
on RSA back in 1998 targeted PKCS#1 v1.5. Without going too much into the math,
an attacker repeatedly sends a server thousands (or even millions) of carefully
modified ciphertexts and observes whether the server accepts or rejects them as
having valid padding. This “padding oracle” leaks just enough information that,
over many queries, the attacker can slowly recover the plaintext of a target
ciphertext even without knowing the RSA private key.
Hubert Kairo, in 2023, found that “Bleichenbacher-style attacks on RSA
decryption are not only still possible, but also that vulnerable implementations
are common”. You can check their paper out here: Everlasting ROBOT: the Marvin
Attack.
We used a target that Kairo found, M2Crypto, a Python wrapper for OpenSSL. When
you give OpenSSL a ciphertext with bad padding, it throws an error. With
M2Crypto, that error gets turned into its own special “RSAError” exception. In
our paper, we used the older, vulnerable versions of both packages (M2Crypto
0.38.0 and OpenSSL 3.0.2). Both have been mitigated since 2023. A developer
might think that since OpenSSL handles the actual cryptography, any security
problems would be OpenSSL’s responsibility and not the wrapper’s. There are many
more such interactions that Kairo found, so feel free to check the paper out. We
just used the first one that worked.
Kairo targeted the timing difference with M2Crypto handling the RSAError
exception. We target the amount of python interpreter + wrapper code flowing
between memory and cache.
When the ciphertext is valid, we expect some normal amount of data transfer
between memory and cache. When it’s invalid, we expect more data transfer for
the interpreter + library code to handle the exception, and we expect to see
that over time. And indeed, that’s what we observe when we measure the memory
bandwidth management (MBM). Here, we show the average of 1000 recordings when a
hypervisor is reading MBM values while the CVM performs a decryption (left):
Y-axis is change in MBM values from one reading to the next.
The first peak between a well-formed ciphertext (red-dotted line) is roughly
0.04 seconds earlier than a malformed ciphertext (blue-solid line). The y-axis
here is the change in consecutive MBM readings. We can try to extend this
window by slowing down the CVM’s operation with maximally applying CAT, i.e.,
restricting L3 cache ways. In this scenario, we extend the window by 0.012
seconds.
A malicious hypervisor would find this attack better than only relying on
network timing (as the original paper showed). At some point, Google Cloud had a
docs page that said “AMD SEV-SNP VMs are known to be subject to high network
latency”, or something along those lines… but these big companies keep
FREAKING link-rotting their docs pages5. So unfortunately, you’ll have to
take my word and some of our hand-wavy observations that SEV-SNP CVMs sometimes
have weird network fluctuations.
> Detecting Key Presses
Imagine a scenario where an employee remotely connects to the CVM with a thin
client, say using the VNC protocol. They’re interacting with the CVM like a
desktop by opening apps and carrying out work-related tasks, with the belief
that the CVM offers security guarantees and even a compromised hypervisor cannot
figure out what’s going on inside the CVM.
They go about their work, maybe typing code, visiting websites, entering
passwords… well, using QoS features, we can detect when they press keys. If we
know the time between key presses, well, that’s a side channel. The
inter-keystroke timing attack has been known for more than two decades: the time
between key presses can reveal what the original set of keys might have been
(Song et
al.,
Zhang et
al.,
Qiu and Chuang et al., Google
Scholar).
When typing “windrunner”, the two consecutive n’s will be typed a bit faster
than the other characters. Detecting key presses is the first step to an
inter-keystroke timing attack.
Our work shows that key-press detection is possible, assuming the scenario
outlined above. We recorded the change in consecutive MBM values when typing
into VS Code (top), typing a password into Firefox (middle), and in Gedit
(bottom). The arrows here are the ground-truth key presses, while for VS Code
(top), the (many, many!) key presses were between the dashed lines.
Y-axis is change in MBM values from one reading to the next.
For the VS Code and Firefox images, the typist was sitting 7000 KM away and at
least 16 network hops from the server. We performed our evaluation to reliably
detect key presses in five settings: maximal CAT, maximal MBA, maximal CAT and
MBA, maximal CDP, and no restrictions. For this experiment, we had someone who
typed 400 characters at 7.2 keys/s, and the hypervisor recorded MBM values at
100 Hz.
QoS Setting
F1 Score
No Restriction
89.11%
Maximal CDP
79.64%
Maximal MBA
76.84%
Maximal MBA & CAT
76.58%
Maximal CAT
96.05%
It’s pretty evident that no restrictions already landed a good F1 score of
89.11% at detecting key presses, but it’s even more evident that MBA really
worsens the F1 score. In fact, that’s a theme we see throughout our attacks: MBA
does not “amplify” the attacks, but rather slows them down (as is designed).
CAT, on the other hand, really amplifies attacks: reducing L3 cache ways from 16
to 1 (i.e., going from 32 MiB cache to 2 MiB cache), increases the F1 score to
96.05%.
> Website Fingerprinting
In the same scenario as the key-press detection (an employee remotely connects
to the CVM with a thin client, say using the VNC protocol), we perform a website
fingerprinting attack: the hypervisor attempts to figure out which website the
CVM is visiting using side channels. This has been a huge field of research in
computer
networks,
especially the Tor network, since if you can figure out which website a Tor user
is visiting, you’re breaking the privacy expectations.
Similarly, if a malicious hypervisor could use microarchitectural side channels
(like Quality of Service features) to determine which website a CVM is visiting,
the malicious hypervisor could break the privacy and confidentiality
expectations that the CVM had.
We test our hypothesis out with the top-20 websites in an open-world
scenario6. We evaluate the attack with 6 CAT restrictions and 11 MBA limits
and collect both CMT and MBA values. We find a similar result as before: MBA
restrictions slow down the attack and generally make it worse, while CAT
restrictions amplify the attack. The only exception is at the maximal MBA
restriction, where the F1 score is the highest, at 72.10%.
CAT Ways (× 2 MiB)
F1 Score
MBA Bandwidth Limit
F1 Score
1
70.85
128 MiB/s
72.10
2
67.85
256 MiB/s
43.48
3
72.23
512 MiB/s
44.61
4
68.80
1 GiB/s
42.31
8
70.79
2 GiB/s
51.13
12
66.40
4 GiB/s
44.27
16
44.72
8 GiB/s
41.67
16 GiB/s
45.74
32 GiB/s
46.65
64 GiB/s
45.45
128 GiB/s
49.26
Unlimited
44.72
However, we need to talk about how the CVM performs with these QoS restrictions
in place.
Performance Impact
I’m copying this straight from the paper, since we put it very succinctly there:
At stricter MBA limits, we find that browsing experience within the CVM is
negatively affected. According to Speedometer3.0, the browser performs 10×
faster with unlimited MBA than with the most-restrictive limit. CAT restrictions
have less impact than MBA limits; the benchmark reports only 2× faster with all
cache ways versus one cache way.
While the most restrictive MBA limit might result in a higher F1 score, the
browsing experience in the CVM is terrible. It’s noticeably slower, and I’d
pull my CVM from a cloud if I experienced that slowness. However, with the most
restrictive CAT limit, the browsing experience isn’t noticeably bad. It’s a
bit worse, yeah, but not awful. I’ve faced worse response times from my family’s
Windows’ laptops.
> Covert Channels
We demonstrated that
Prime+Probe side
channel gets amplified by building a covert channel. When you have less cache,
the covert channel speed increases. We refer to the paper for more details.
We also demonstrated a new side channel: the MBA rate-limit. Since hardware
applies a rate limit when there’s some threshold of memory-to-cache transfers
for a particular MBA limit, we find that we can use this contention-like side
channel to build a covert channel reaching speeds up to 205.8 Bytes/s at an MBA
limit of 1 GiB/s. The paper has more details and a pretty heatmap (Figure 6).
Responsible Disclosure
We responsibly disclosed our findings to AMD’s PSIRT team in August 2025. They
acknowledged our findings, but stated that side-channel attacks are out of
scope. They have, however, put up a security bulletin informing clients that
such attacks are possible:
https://www.amd.com/en/resources/product-security/bulletin/amd-sb-3035.html.
Wrapping Up
Our paper shows that Quality of Service features can be used to extract some
information about how a AMD SEV-SNP confidential VM runs. Unlike other research,
which requires powerful but active-tampering techniques like single
stepping7, our
research does not slow down the CVM’s execution as much. QoS features are good
features - they can help regulate unruly applications/VMs and can let
smaller-footprint programs breathe for execution time. However, these very
good-features can also open up the door for side-channel attacks, provided the
attacker is strong enough to read them, just like a malicious hypervisor.
Of all two monitoring features, MBM — which reports data flow between cache
and memory — happens to be the most precise (64 Byte precision) to learn
what’s happening inside the CVM. Of the three allocation features, CAT —
restricts L3 cache ways —, seems to be the best for amplifying attacks.
I now hope you understand the title, “A CAT with an MBA”:
@inproceedings{Neela2026Quality,
author = {Neela, Sudheendra Raghav and Fiedler, Carina and Zhang, Ruiyi and Schröder, Robin Leander and Schwarz, Michael and Gruss, Daniel},
booktitle = {ESORICS},
title = {{A Systematic Look at Quality-of-Service Feature Effects on Side Channels in AMD SEV-SNP}},
year = {2026}
}
The features I write about here may change in the future. For brevity, I’m
describing the “big” five features that have existed for quite some time
now. ↩︎
The Resource Management Identifier (RMID) is used to specify the SMT
threads to monitor for both MBM and CMT. ↩︎↩︎
Okay well, um actually, it
would be two additional blips of data transfer. Once when evicted (because
MBM counts that too), and the other when going back into cache. ↩︎
I even made sure to archive it in 2024 for this exact reason, but I guess
Google systematically asks wayback to take down docs pages with older
content?…: Wayback.
Companies that do this piss me off: link-rotting a big chunk of the internet
and/or constantly restructuring your docs pages (I’m looking at you Intel,
AMD, Microsoft, and Google — you don’t need to change your system every
two business days!). Thank the penguin gods for the stability of Linux’s man
pages.
Typically we do top-100 in website fingerprinting attacks using
system-level side channels. However, for this work, we performed our website
fingerprinting in a LOT of scenarios and had to repeat the entire experiment
18 times on a shared system. This is why we limited ourselves to top-20
websites only. The recording alone took 60-65 hours and produced gigabytes
of (uncompressed) data. ↩︎
Single stepping is an advanced technique where a hypervisor steps the VM
one instruction at a time. The idea is to first “measure” some state of the
CPU, let the VM execute a single instruction, “measure” the state again.
The difference in these states can give insight into what the VM just did.
Of course, I’m vastly oversimplifying… but a lot of single stepping
attacks have done this. ↩︎