Skip to content

x86/kpti: Kernel Page Table Isolation (was KAISER)

Thomas Gleixner linux-mm lkml xen-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest60 patches 58 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/60x86/entry/64/paravirt: Use paravirt-safe macro to access eflagsarch/x86/entry/entry_64.S arch/x86/include/asm/irqflags.h arch/x86/include/asm/paravirt.h +1 more+19 -32 replies
2/60x86/unwinder/orc: Dont bail on stack overflowarch/x86/kernel/unwind_orc.c+12 -22 replies
3/60x86/unwinder: Handle stack overflows more gracefullyarch/x86/include/asm/kdebug.h arch/x86/include/asm/unwind.h arch/x86/kernel/dumpstack.c +2 more+66 -610 replies
4/60x86/irq: Remove an old outdated comment about context tracking racesarch/x86/kernel/irq.c+0 -120 replies
5/60x86/irq/64: Print the offending IP in the stack overflow warningarch/x86/kernel/irq_64.c+2 -20 replies
6/60x86/entry/64: Allocate and enable the SYSENTER stackarch/x86/entry/entry_64_compat.S arch/x86/include/asm/processor.h arch/x86/kernel/asm-offsets.c +4 more+10 -140 replies
7/60x86/dumpstack: Add get_stack_info() support for the SYSENTER stackarch/x86/include/asm/stacktrace.h arch/x86/kernel/dumpstack.c arch/x86/kernel/dumpstack_32.c +1 more+34 -00 replies
8/60x86/entry/gdt: Put per-CPU GDT remaps in ascending orderarch/x86/include/asm/desc.h+1 -10 replies
9/60x86/mm/fixmap: Generalize the GDT fixmap mechanism, introduce struct cpu_entry_areaarch/x86/include/asm/desc.h arch/x86/include/asm/fixmap.h arch/x86/kernel/cpu/common.c +1 more+44 -180 replies
10/60x86/kasan/64: Teach KASAN about the cpu_entry_areaarch/x86/mm/kasan_init_64.c+17 -10 replies
11/60x86/entry: Fix assumptions that the HW TSS is at the beginning of cpu_tssarch/x86/include/asm/desc.h arch/x86/include/asm/processor.h arch/x86/kernel/cpu/common.c +3 more+35 -310 replies
12/60x86/dumpstack: Handle stack overflow on all stacksarch/x86/kernel/dumpstack.c+14 -100 replies
13/60x86/entry: Move SYSENTER_stack to the beginning of struct tss_structarch/x86/include/asm/processor.h arch/x86/kernel/cpu/common.c+33 -90 replies
14/60x86/entry: Remap the TSS into the CPU entry areaarch/x86/entry/entry_32.S arch/x86/include/asm/fixmap.h arch/x86/kernel/asm-offsets.c +4 more+58 -151 reply
15/60x86/entry/64: Separate cpu_current_top_of_stack from TSS.sp0arch/x86/include/asm/processor.h arch/x86/include/asm/thread_info.h arch/x86/kernel/asm-offsets_64.c +2 more+26 -60 replies
16/60x86/espfix/64: Stop assuming that pt_regs is on the entry stackarch/x86/kernel/traps.c+28 -90 replies
17/60x86/entry/64: Use a per-CPU trampoline stack for IDT entriesarch/x86/entry/entry_64.S arch/x86/entry/entry_64_compat.S arch/x86/include/asm/switch_to.h +3 more+65 -290 replies
18/60x86/entry/64: Return to userspace from the trampoline stackarch/x86/entry/entry_64.S+51 -40 replies
19/60x86/entry/64: Create a per-CPU SYSCALL entry trampolinearch/x86/entry/entry_64.S arch/x86/include/asm/fixmap.h arch/x86/kernel/asm-offsets.c +2 more+84 -11 reply
20/60x86/entry/64: Move the IST stacks into struct cpu_entry_areaarch/x86/include/asm/fixmap.h arch/x86/kernel/cpu/common.c arch/x86/kernel/traps.c+57 -320 replies
21/60x86/entry/64: Remove the SYSENTER stack canaryarch/x86/include/asm/processor.h arch/x86/kernel/dumpstack.c arch/x86/kernel/process.c +1 more+1 -110 replies
22/60x86/entry: Clean up the SYSENTER_stack codearch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/fixmap.h +4 more+21 -231 reply
23/60x86/entry/64: Make cpu_entry_area.tss read-onlyarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/fixmap.h +13 more+60 -481 reply
24/60x86/paravirt: Dont patch flush_tlb_singlearch/x86/kernel/paravirt_patch_64.c+0 -21 reply
25/60x86/paravirt: Provide a way to check for hypervisorsarch/x86/include/asm/hypervisor.h+15 -101 reply
26/60x86/cpufeature: Make cpu bugs stickyarch/x86/include/asm/cpufeature.h arch/x86/include/asm/processor.h arch/x86/kernel/cpu/common.c+7 -51 reply
27/60x86/cpufeatures: Add X86_BUG_CPU_INSECUREarch/x86/include/asm/cpufeatures.h arch/x86/kernel/cpu/common.c+6 -01 reply
28/60x86/mm/kpti: Disable global pages if KERNEL_PAGE_TABLE_ISOLATION=yarch/x86/mm/init.c+9 -31 reply
29/60x86/mm/kpti: Prepare the x86/entry assembly code for entry/exit CR3 switchingarch/x86/entry/calling.h arch/x86/entry/entry_64.S arch/x86/entry/entry_64_compat.S+128 -70 replies
30/60x86/mm/kpti: Add infrastructure for page table isolationDocumentation/admin-guide/kernel-parameters.txt arch/x86/boot/compressed/pagetable.c arch/x86/entry/calling.h +6 more+121 -31 reply
31/60x86/mm/kpti: Add mapping helper functionsarch/x86/include/asm/pgtable_64.h+139 -03 replies
32/60x86/mm/kpti: Allow NX poison to be set in p4d/pgdarch/x86/include/asm/pgtable.h+12 -21 reply
33/60x86/mm/kpti: Allocate a separate user PGDarch/x86/kernel/head_64.S arch/x86/mm/pgtable.c+41 -52 replies
34/60x86/mm/kpti: Populate user PGDarch/x86/include/asm/pgtable.h arch/x86/mm/kpti.c+46 -01 reply
35/60x86/espfix: Ensure that ESPFIX is visible in user PGDarch/x86/kernel/espfix_64.c+16 -01 reply
36/60x86/mm/kpti: Add functions to clone kernel PMDsarch/x86/mm/kpti.c+102 -01 reply
37/60x86mm//kpti: Force entry through trampoline when KPTI activearch/x86/kernel/cpu/common.c+4 -11 reply
38/60x86/fixmap: Move cpu entry area into a separate PMDarch/x86/include/asm/fixmap.h+10 -41 reply
39/60x86/mm/kpti: Share cpu_entry_area PMDsarch/x86/mm/kpti.c+18 -01 reply
40/60x86: PMD align entry textarch/x86/kernel/vmlinux.lds.S+8 -01 reply
41/60x86/mm/kpti: Share entry text PMDarch/x86/mm/kpti.c+10 -01 reply
42/60x86/fixmap: Move IDT fixmap into the cpu_entry_area rangearch/x86/include/asm/fixmap.h+1 -10 replies
43/60x86/fixmap: Add debugstore entries to cpu_entry_areaarch/x86/events/intel/ds.c arch/x86/events/perf_event.h arch/x86/include/asm/fixmap.h +2 more+75 -211 reply
44/60x86/events/intel/ds: Map debug buffers in fixmaparch/x86/events/intel/ds.c arch/x86/events/perf_event.h+73 -430 replies
45/60x86/fixmap: Add ldt entries to user shared fixmaparch/x86/include/asm/fixmap.h arch/x86/kernel/cpu/common.c+5 -00 replies
46/60x86/ldt: Rename ldt_struct->entries memberarch/x86/events/core.c arch/x86/include/asm/mmu_context.h arch/x86/kernel/ldt.c +4 more+22 -200 replies
47/60x86/ldt: Map LDT entries into fixmaparch/x86/include/asm/mmu_context.h arch/x86/kernel/ldt.c+84 -472 replies
48/60x86/mm: Move the CR3 construction functions to tlbflush.harch/x86/include/asm/mmu_context.h arch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c+31 -320 replies
49/60x86/mm: Remove hard-coded ASID limit checksarch/x86/include/asm/tlbflush.h+18 -20 replies
50/60x86/mm: Put MMU to hardware ASID translation in one placearch/x86/include/asm/tlbflush.h+18 -110 replies
51/60x86/mm: Allow flushing for future ASID switchesarch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c+69 -106 replies
52/60x86/mm: Abstract switching CR3arch/x86/mm/tlb.c+20 -20 replies
53/60x86/mm: Use/Fix PCID to optimize user/kernel switchesarch/x86/entry/calling.h arch/x86/entry/entry_64.S arch/x86/entry/entry_64_compat.S +6 more+160 -333 replies
54/60x86/mm: Optimize RESTORE_CR3arch/x86/entry/calling.h arch/x86/entry/entry_64.S+30 -40 replies
55/60x86/mm: Use INVPCID for __native_flush_tlb_single()arch/x86/include/asm/cpufeatures.h arch/x86/include/asm/tlbflush.h arch/x86/mm/init.c+60 -284 replies
56/60x86/mm/kpti: Disable native VSYSCALLarch/x86/Kconfig+8 -01 reply
57/60x86/mm/kpti: Add Kconfigsecurity/Kconfig+10 -03 replies
58/60x86/mm/debug_pagetables: Add page table directoryarch/x86/mm/debug_pagetables.c+10 -50 replies
59/60x86/mm/dump_pagetables: Check user space page table for WX pagesarch/x86/include/asm/pgtable.h arch/x86/mm/debug_pagetables.c arch/x86/mm/dump_pagetables.c+27 -60 replies
60/60x86/mm/debug_pagetables: Allow dumping current pagetablesarch/x86/include/asm/pgtable.h arch/x86/mm/debug_pagetables.c arch/x86/mm/dump_pagetables.c+73 -60 replies

Reviews trailers per patch

34 of 60 patches have a Reviewed-by or Acked-by.

PatchThomas GleixnerBorislav PetkovDave HansenPeter ZijlstraJosh PoimboeufJuergen Gross
1/60R-b R-b
2/60
3/60R-b
4/60R-b R-b
5/60R-b R-b
6/60R-b R-b
7/60R-b
8/60R-b R-b
9/60R-b R-b
10/60
11/60R-b R-b R-b
12/60R-b
13/60R-b R-b
14/60R-b R-b
15/60R-b R-b
16/60R-b R-b
17/60R-b R-b
18/60R-b R-b
19/60R-b R-b
20/60R-b R-b
21/60R-b R-b
22/60R-b R-b
23/60R-b
24/60A-b R-b R-b
25/60R-b
26/60R-b
27/60
28/60R-b
29/60R-b R-b
30/60R-b
31/60
32/60R-b
33/60R-b
34/60R-b
35/60
36/60Originally-by
37/60R-b
38/60
39/60R-b
40/60R-b
41/60R-b
42/60
43/60
44/60
45/60
46/60
47/60
48/60
49/60
50/60
51/60
52/60
53/60
54/60
55/60
56/60
57/60
58/60
59/60
60/60

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.

Read the v1 thread

Cover letter [patch 00/60] x86/kpti: Kernel Page Table Isolation (was KAISER)

This series is a major overhaul of the KAISER patches:

1) Entry code

Mostly the same, except for a handful of fixlets and delta improvements folded into the corresponding patches

New: Map TSS read only into the user space visible mapping

This is 64bit only, as 32bit needs the TSS mapped RW

AMD confirmed that there is no issue with that. It would be nice to get confirmation from Intel as well.

2) Namespace

Several people including Linus requested to change the KAISER name.

We came up with a list of technically correct acronyms:

User Address Space Separation, prefix uass_

Forcefully Unmap Complete Kernel With Interrupt Trampolines, prefix fuckwit_

but we are politically correct people so we settled for

Kernel Page Table Isolation, prefix kpti_

Linus, your call :)

3) The actual isolation patches

- Replaced the magic kaiser_add/remove_mapping() code by mapping everything which needs to be shared with user space into the fixmap

- PMD aligned the shared fixmap so the PTE page can be shared between user and kernel space page tables

- Integrated all fixes and Peters rewrite of the PCID/TLB flush code.

- Restructured the patch set in a way that it is simpler to review

- Got rid of the strange wording of shadow page tables, because they are not shadowish at all. KASAN, virt etc. use shadows, but these tables are actively in use and integral part of the functionality

- Moved the debugfs files into a new directory so they don't clutter the debugfs root directory.

LIMITATIONS:

- allmod/yes config builds fail right now because the fixmap grows too large and breaks the EFI assumptions. This is still investigated.

A possible solution is just to use one of the address space holes and grab a separate pgdir to map the cpu entry area. Not hard to do and it wont change much of the principle of these patches.

TODOs:

- This needs a thorough review again. Sorry.

- Please verify that all fixlets have been integrated. The mail threads are horribly scattered so I might have missed something.

- Rewrite documentation. I dropped the documentation patch as it not longer applies.

- Handle native vsyscalls. Right now the patch set supports only emulation, but it should be possible to support native as well. Nothing urgent, I'd rather prefer to kill them completely.

- Populate a branch with minimal prerequisite patches to apply.

Thanks to Andy Lutomirsky, Peter Zijlstra, Ingo Molnar, Borislav Petkov and Dave Hansen for all the help with this.

The patches apply on top of

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/urgent

and are available from git in

git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.x86/kpti

and as tarball from

    https://tglx.de/~tglx/patches-kpti-119.tar.bz2

    Signature file for the uncompressed tarball

    https://tglx.de/~tglx/patches-kpti-119.tar.sig

Thanks,

tglx