Skip to contentSkip to conversation

l1tf: Kernel suggests I throw away third of my memory. I'd rather not

DISCUSSION Pavel Machek 5 participants 8 messages Oct 17, 2018 – Oct 18, 2018

lore
Share & tools
lore
#0 lore raw

Hi!

6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not going to be pleasant.

l1tf.html says:

# The Linux kernel contains a mitigation for this attack vector, PTE # inversion, which is permanently enabled and has no performance

# impact.

I don't believe it has "no" performance impact, but I guess it is lost in the noise.

# The kernel ensures that the address bits of PTEs, which are

# not marked present, never point to cacheable physical memory space.

# A system with an up to date kernel is protected against attacks from

# malicious user space applications.

These are not true.

cat /sys/devices/system/cpu/vulnerabilities/l1tf Vulnerable
uname -a
Linux amd 4.19.0-rc8-next-20181017autobisect1539371050 #189 SMP Wed

Oct 17 12:04:23 CEST 2018 i686 GNU/Linux

Now question is... can we do better? Kernel stores information about swapped-out pages there, right? That sounds like a cool hack, but maybe it is time to get rid of that hack?

As a workaround, can I simply do swapoff -a to be safe for now?

Thanks,
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#1 lore raw
Quoted 5 lines · Pavel Machek

Hi!

6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not going to be pleasant.

Quoted 8 lines

l1tf.html says:

# The Linux kernel contains a mitigation for this attack vector, PTE # inversion, which is permanently enabled and has no performance

# impact.

I don't believe it has "no" performance impact, but I guess it is lost in the noise.

Please prove otherwise. I would be more than surprised if inverting pfn part of the pte is noticeable. But I can be wrong of course.

Quoted 6 lines

# The kernel ensures that the address bits of PTEs, which are

# not marked present, never point to cacheable physical memory space.

# A system with an up to date kernel is protected against attacks from

# malicious user space applications.
Quoted 7 lines

These are not true.

cat /sys/devices/system/cpu/vulnerabilities/l1tf Vulnerable
uname -a
Linux amd 4.19.0-rc8-next-20181017autobisect1539371050 #189 SMP Wed

Oct 17 12:04:23 CEST 2018 i686 GNU/Linux

This is a result of you having memory above MAX_PFN/2 right?

Quoted 3 lines

Now question is... can we do better? Kernel stores information about swapped-out pages there, right? That sounds like a cool hack, but maybe it is time to get rid of that hack?

Patches are welcome.

Quoted 1 line

As a workaround, can I simply do swapoff -a to be safe for now?

Well, that depends. Do you care about PROT_NONE attacks as well? If not then no-swap would help you. But even then no-swap is rather theoretical attack on a physical host unless you allow an arbitrary swapout to a malicious user (e.g. allow a user controlled memcg hard limit that would cause excessive local swapouts).

Signature · 2 lines
Michal Hocko
SUSE Labs
#2 lore raw

Hi!

Quoted 9 lines
Quoted 9 lines

6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not going to be pleasant.

l1tf.html says:

# The Linux kernel contains a mitigation for this attack vector, PTE # inversion, which is permanently enabled and has no performance

# impact.
Quoted 5 lines
Quoted 2 lines

I don't believe it has "no" performance impact, but I guess it is lost in the noise.

Please prove otherwise. I would be more than surprised if inverting pfn part of the pte is noticeable. But I can be wrong of course.

I'm not saying its noticeable. I'm saying that inversion takes few clock cycles (including a branch?) and that neither caches nor RAM is free.

"no noticeable performance impact" I'd agree with :).

Quoted 8 lines
Quoted 8 lines

# The kernel ensures that the address bits of PTEs, which are

# not marked present, never point to cacheable physical memory space.

# A system with an up to date kernel is protected against attacks from

# malicious user space applications.

These are not true.

Quoted 7 lines
Quoted 5 lines

cat /sys/devices/system/cpu/vulnerabilities/l1tf Vulnerable
uname -a
Linux amd 4.19.0-rc8-next-20181017autobisect1539371050 #189 SMP Wed

Oct 17 12:04:23 CEST 2018 i686 GNU/Linux

This is a result of you having memory above MAX_PFN/2 right?

Yes.

Quoted 5 lines
Quoted 3 lines

Now question is... can we do better? Kernel stores information about swapped-out pages there, right? That sounds like a cool hack, but maybe it is time to get rid of that hack?

Patches are welcome.

Cooperation will be needed if you want to see patches. As in... answering the questions above.

Quoted 7 lines
Quoted 1 line

As a workaround, can I simply do swapoff -a to be safe for now?

Well, that depends. Do you care about PROT_NONE attacks as well? If not then no-swap would help you. But even then no-swap is rather theoretical attack on a physical host unless you allow an arbitrary swapout to a malicious user (e.g. allow a user controlled memcg hard limit that would cause excessive local swapouts).

PROT_NONE attack.. aha, so kernel stores not only information about swapped-out pages but also about file-backed pages that are currently not present? Hmm. That makes it more complex :-(.

Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#3 lore raw

On Wed 17-10-18 13:32:26, Pavel Machek wrote:
[...]

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Now question is... can we do better? Kernel stores information about swapped-out pages there, right? That sounds like a cool hack, but maybe it is time to get rid of that hack?

Patches are welcome.

Cooperation will be needed if you want to see patches. As in... answering the questions above.

The question is whether that is really worth it. L1TF is mostly about virtual environments. If you are running in a native HW then I wouldn't lose much sleep over it. a) pfns stored in PROT_NONE entries are controlled by the OS b) swap based attacks with something interesting in L1$ colliding with the swap entry is theoretical at best.

Signature · 2 lines
Michal Hocko
SUSE Labs
#6 lore raw
In reply to Pavel Machek
Quoted 4 lines · Pavel Machek

Hi!

6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not going to be pleasant.

Just rebuild your kernel with PAE? I assume your CPU supports it.

This will also give you NX, which if you're really worried about security is far more important than L1TF.

If you don't worry about security just ignore.

-Andi

#7 lore raw
Quoted 7 lines · Andi Kleen
Quoted 4 lines · Pavel Machek

Hi!

6a012288 suggests I throw away 1GB on RAM. On 3GB system.. that is not going to be pleasant.

Just rebuild your kernel with PAE? I assume your CPU supports it.

I think it is built with PAE or this would kick in:

#if CONFIG_PGTABLE_LEVELS == 2
        pr_warn("Kernel not compiled for PAE. No mitigation for L1TF\n");
        return;

#endif

I.e. no MAX_PA/2 messages.

Quoted 7 lines

This will also give you NX, which if you're really worried about security is far more important than L1TF.

If you don't worry about security just ignore.

-Andi

#4 lore raw
In reply to Pavel Machek
Quoted 8 lines · Pavel Machek
Quoted 5 lines

Well, that depends. Do you care about PROT_NONE attacks as well? If not then no-swap would help you. But even then no-swap is rather theoretical attack on a physical host unless you allow an arbitrary swapout to a malicious user (e.g. allow a user controlled memcg hard limit that would cause excessive local swapouts).

PROT_NONE attack.. aha, so kernel stores not only information about swapped-out pages but also about file-backed pages that are currently not present? Hmm. That makes it more complex :-(.

There are also migration PTE entries that are "swap-like". They can exist even if you swapoff -a.

Can we do better? Sure. I think we'd all be happy to review patches that improve the situation if folks have simple ideas for improvement.

#5 lore raw
Quoted 10 lines · Dave Hansen
Quoted 8 lines · Pavel Machek
Quoted 5 lines

Well, that depends. Do you care about PROT_NONE attacks as well? If not then no-swap would help you. But even then no-swap is rather theoretical attack on a physical host unless you allow an arbitrary swapout to a malicious user (e.g. allow a user controlled memcg hard limit that would cause excessive local swapouts).

PROT_NONE attack.. aha, so kernel stores not only information about swapped-out pages but also about file-backed pages that are currently not present? Hmm. That makes it more complex :-(.

Quoted 5 lines

There are also migration PTE entries that are "swap-like". They can exist even if you swapoff -a.

Can we do better? Sure. I think we'd all be happy to review patches that improve the situation if folks have simple ideas for improvement.

I think if would be rather unfortunate changing this fundamentally due to a HW bug that should hopefully eventually go away with new CPUs, and with MAX_PA/2 being limited only on very old CPUs. I don't think using page table entries like this is a hack. The pte must IMHO always carry some information that identifies is as swap entry or prot_none or whatever. If we had to look up the swap offset (or pfn associated with PROT_NONE pte elsewhere), there would be additional overhead both for storing this mapping and for looking it up.
PTEs seems just like a natural place for this IMHO, and it's defined that the bits can be used by the kernel however it likes for !present PTEs. Why rewrite it all just because some CPUs implementation is not as defined...