x86/iopl: Prevent user space from using CLI/STI with iopl(3)
Subsystems (5)
cover x86/iopl: Prevent user space from using CLI/STI with iopl(3)
This is the result of the previous discussion about assumptions that user space always runs with interrupts enabled:
https://lore.kernel.org/r/[email protected]
The infinite wisdom of hardware designers coupled the I/O permission level of accessing all 65536 I/O ports with the ability to use CLI/STI.
iopl(3), if granted gives this ability to user space. That's broken in several ways:
1) User space can lock up the machine when an interrupt disabled region runs into an infinite loop.
2) Disabling interrupts in user space has no semantics, at least no well defined, consistent and understandable semantics.
syscalls and exceptions ignore that state and can block, preempt etc.
#1 could be arguably achieved by fiddling with the wrong I/O ports as well.
#2 is the real issue:
It causes a problem in the user/kernel interface and in exception handlers as it is a common assumption that user space executes with interrupts enabled. But with IOPL(3) this assumption is not correct. Neither for syscalls nor for exceptions.
There is code in the low level entry and exception handlers which makes this assumption. Even experienced kernel developers trip over that as shown in the discussion referenced above.
Ideally we should delete iopl(), but there are existing users including DPDK. None of those I checked rely on the CLI/STI ability. They all use it for conveniance to access I/O ports.
The only thing I found using CLI/STI was some really ancient X implementation. So dragons might be lurking, but that X stuff really won't work on a current kernel anymore :)
After quite some discussion I came up with a solution to emulate IOPL via the I/O bitmap mechanism without copying 8k of zeroed bitmap on every context switch which is the main concern of people who prefer iopl() over ioperm().
The trick is to use the io-bitmap offset in the TSS to point the CPU to a bitmap with all bits cleared. This is slightly slower than just relying on the IOPL magic in (E)FLAGS, but it's almost not noticeable.
The same trick can be used when switching away from a task which uses an I/O bitmap to a task which does not. Instead of cleaning up the bitmap storage, just point the I/O bitmap offset to a location which is outside of the TSS limit. That puts the copy overhead solely on tasks which have actually an I/O bitmap installed. The copy mechanism is quite stupid as well as it starts always from 0 even if the first cleared bit is right at the end of the bitmap.
The following series addresses this.
The first few patches are preparatory and consolidate needlessly duplicated code to avoid duplicating all the changes for the IOPL emulation.
At the end it removes the legacy support completely which cleans up quite some code all over the place including paravirt.
The improvement for switching away from an I/O bitmap using task to a sane task w/o I/O bitmap is quite measurable in a microbench mark.
Also avoiding to copy several kilobytes just to update a tiny region has a measurable impact.
Removing CLI/STI from iopl() allows us to consolidate and simplify the entry and exception code instead of wasting time and racking nerves by analysing the world and some more whether there is an implicit assumption of user space having interrupts always enabled.
The series is also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.x86/iopl
Thanks,
tglx
8<---------------
arch/x86/Kconfig | 26 ++++ arch/x86/include/asm/paravirt.h | 4 arch/x86/include/asm/paravirt_types.h | 2 arch/x86/include/asm/processor.h | 92 ++++++++------- arch/x86/include/asm/ptrace.h | 6 + arch/x86/include/asm/switch_to.h | 10 + arch/x86/include/asm/xen/hypervisor.h | 2 arch/x86/kernel/cpu/common.c | 176 +++++++++++------------------ arch/x86/kernel/doublefault.c | 2 arch/x86/kernel/ioport.c | 203 ++++++++++++++++++++++++---------- arch/x86/kernel/paravirt.c | 2 arch/x86/kernel/process.c | 177 ++++++++++++++++++++++++----- arch/x86/kernel/process_32.c | 77 ------------ arch/x86/kernel/process_64.c | 86 -------------- arch/x86/kernel/ptrace.c | 2 arch/x86/xen/enlighten_pv.c | 10 - tools/testing/selftests/x86/iopl.c | 104 +++++++++++++++-- 17 files changed, 556 insertions(+), 425 deletions(-)
PATCH 1/9 x86/ptrace: Prevent truncation of bitmap size
The active() callback of the IO bitmap regset divides the IO bitmap size by the word size (32/64 bit). As the I/O bitmap size is in bytes the active check fails for bitmap sizes of 1-3 bytes on 32bit and 1-7 bytes on 64bit.
Use DIV_ROUND_UP() instead.
--- arch/x86/kernel/ptrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/ptrace.c +1 -1
@@ -697,7 +697,7 @@ static int ptrace_set_debugreg(struct tastatic int ioperm_active (struct task_struct *target,const struct user_regset *regset) { -return target->thread.io_bitmap_max / regset->size; +return DIV_ROUND_UP (target->thread.io_bitmap_max, regset->size); } static int ioperm_get(struct task_struct *target,
Re: [patch 1/9] x86/ptrace: Prevent truncation of bitmap size
Quoted 14 lines · Thomas Gleixner
The
active()callback of the IO bitmap regset divides the IO bitmap size by the word size (32/64 bit). As the I/O bitmap size is in bytes the active check fails for bitmap sizes of 1-3 bytes on 32bit and 1-7 bytes on 64bit.Use
DIV_ROUND_UP()instead.Signed-off-by Thomas Gleixner <[email protected]>--- arch/x86/kernel/ptrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/ptrace.c static int ptrace_set_debugreg(struct ta · line 703
Quoted 8 lines
arch/x86/kernel/ptrace.c +1
-1@@ -697,7 +697,7 @@ static int ptrace_set_debugreg(struct tastatic int ioperm_active (struct task_struct *target,const struct user_regset *regset) { -return target->thread.io_bitmap_max / regset->size; +return DIV_ROUND_UP (target->thread.io_bitmap_max, regset->size); } static int ioperm_get(struct task_struct *target,
But the ioperm core dumping logic in ptrace.c looks a bit weird. For example why do we alias REGSET_IOPERM64 to REGSET_XFP:
enum x86_regset {
REGSET_GENERAL,
REGSET_FP,
REGSET_XFP,
REGSET_IOPERM64 = REGSET_XFP,
REGSET_XSTATE,
REGSET_TLS,
REGSET_IOPERM32,
};
This has been so since the original regset commit (325af5fb1418).
Unless I'm misreading the code this makes either REGSET_XFP or REGSET_IOPERM64 misbehave? What am I missing?
Thanks,
Ingo
PATCH 2/9 x86/process: Unify copy_thread_tls()
In reply to Thomas GleixnerWhile looking at the TSS io bitmap it turned out that any change in that area would require identical changes to copy_thread_tls(). The 32 and 64 bit variants share sufficient code to consolidate them into a common function to avoid duplication of upcoming modifications.
--- arch/x86/include/asm/ptrace.h | 6 ++ arch/x86/include/asm/switch_to.h | 10 ++++ arch/x86/kernel/process.c | 94 +++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/process_32.c | 68 ---------------------------- arch/x86/kernel/process_64.c | 75 ------------------------------- 5 files changed, 110 insertions(+), 143 deletions(-)
arch/x86/include/asm/ptrace.h +6 -0
@@ -361,5 +361,11 @@ extern int do_get_thread_area(struct tasextern int do_set_thread_area (struct task_struct *p,int idx,struct user_desc __user *info,int can_allocate); +#ifdef CONFIG_X86_64 +# define do_set_thread_area_64 (p, s, t) do_arch_prctl_64(p, s, t) +#else +# define do_set_thread_area_64 (p, s, t) (0) +#endif +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_PTRACE_H */
arch/x86/include/asm/switch_to.h +10 -0
@@ -103,7 +103,17 @@ static inline void update_task_stack(strif (static_cpu_has (X86_FEATURE_XENPV ))load_sp0 (task_top_of_stack (task)); #endif +} +static inline void kthread_frame_init (struct inactive_task_frame *frame, +unsigned long fun,unsigned long arg) +{ + frame->bx = fun; +#ifdef CONFIG_X86_32 + frame->di = arg; +#else + frame->r12 = arg; +#endif }#endif /* _ASM_X86_SWITCH_TO_H */
arch/x86/kernel/process.c +94 -0
100 lines not on this page · raw message
arch/x86/kernel/process_32.c +0 -68
74 lines not on this page · raw message
arch/x86/kernel/process_64.c +0 -75
81 lines not on this page · raw message
Re: [patch 2/9] x86/process: Unify copy_thread_tls()
Quoted 61 lines · Thomas Gleixner
While looking at the TSS io bitmap it turned out that any change in that area would require identical changes to
copy_thread_tls(). The 32 and 64 bit variants share sufficient code to consolidate them into a common function to avoid duplication of upcoming modifications.Signed-off-by Thomas Gleixner <[email protected]>--- arch/x86/include/asm/ptrace.h | 6 ++ arch/x86/include/asm/switch_to.h | 10 ++++ arch/x86/kernel/process.c | 94 +++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/process_32.c | 68 ---------------------------- arch/x86/kernel/process_64.c | 75 ------------------------------- 5 files changed, 110 insertions(+), 143 deletions(-)arch/x86/include/asm/ptrace.h +6
-0@@ -361,5 +361,11 @@ extern int do_get_thread_area(struct tasextern int do_set_thread_area (struct task_struct *p,int idx,struct user_desc __user *info,int can_allocate); +#ifdef CONFIG_X86_64 +# define do_set_thread_area_64 (p, s, t) do_arch_prctl_64(p, s, t) +#else +# define do_set_thread_area_64 (p, s, t) (0) +#endif +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_PTRACE_H */ arch/x86/include/asm/switch_to.h +10
-0@@ -103,7 +103,17 @@ static inline void update_task_stack(strif (static_cpu_has (X86_FEATURE_XENPV ))load_sp0 (task_top_of_stack (task)); #endif +} +static inline void kthread_frame_init (struct inactive_task_frame *frame, +unsigned long fun,unsigned long arg) +{ + frame->bx = fun; +#ifdef CONFIG_X86_32 + frame->di = arg; +#else + frame->r12 = arg; +#endif }#endif /* _ASM_X86_SWITCH_TO_H */ arch/x86/kernel/process.c +7
-0@@ -132,6 +132,100 @@ void exit_thread(struct task_struct *tskfpu__drop (fpu); } +static int set_new_tls (struct task_struct *p,unsigned long tls) +{ +struct user_desc __user *utls = (struct user_desc __user *)tls; + +if (in_ia32_syscall ()) +return do_set_thread_area (p,-1 , utls,0 ); +else
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 151
Quoted 10 lines
arch/x86/kernel/process.c +10
-0+return do_set_thread_area_64 (p,ARCH_SET_FS , tls); +} + +static inline int copy_io_bitmap (struct task_struct *tsk) +{ +if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP ))) +return 0 ; + + tsk->thread.io_bitmap_ptr =kmemdup (current->thread.io_bitmap_ptr, +IO_BITMAP_BYTES ,GFP_KERNEL );
tsk->thread.io_bitmap_max = current->thread.io_bitmap_max?
I realize you inherited this from the code you're refactoring, but it does seem to be missing.
Quoted 36 lines
arch/x86/kernel/process.c +36
-0+if (!tsk->thread.io_bitmap_ptr) { + tsk->thread.io_bitmap_max =0 ; +return -ENOMEM ; + } +set_tsk_thread_flag (tsk,TIF_IO_BITMAP ); +return 0 ; +} + +static inline void free_io_bitmap (struct task_struct *tsk) +{ +if (tsk->thread.io_bitmap_ptr) { +kfree (tsk->thread.io_bitmap_ptr); + tsk->thread.io_bitmap_ptr =NULL ; + tsk->thread.io_bitmap_max =0 ; + } +} + +int copy_thread_tls (unsigned long clone_flags,unsigned long sp, +unsigned long arg,struct task_struct *p,unsigned long tls) +{ +struct inactive_task_frame *frame; +struct fork_frame *fork_frame; +struct pt_regs *childregs; +int ret; + + childregs =task_pt_regs (p); + fork_frame = container_of(childregs, struct fork_frame, regs); + frame = &fork_frame->frame; + + frame->bp =0 ; + frame->ret_addr = (unsigned long ) ret_from_fork; + p->thread.sp = (unsigned long ) fork_frame; + p->thread.io_bitmap_ptr =NULL ; +memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps)); + +#ifdef CONFIG_X86_64
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 197
Quoted 10 lines
arch/x86/kernel/process.c +10
-0+savesegment (gs, p->thread.gsindex); + p->thread.gsbase = p->thread.gsindex ?0 : current->thread.gsbase; +savesegment (fs, p->thread.fsindex); + p->thread.fsbase = p->thread.fsindex ?0 : current->thread.fsbase; +savesegment (es, p->thread.es); +savesegment (ds, p->thread.ds); +#else +/* Clear all status flags including IF and set fixed bit. */ + frame->flags =X86_EFLAGS_FIXED ; +#endif
Want to do another commit to make the eflags fixup unconditional? I'm wondering if we have a bug.
Other than these questions:
Re: [patch 2/9] x86/process: Unify copy_thread_tls()
Quoted 11 lines · Andy Lutomirski
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 151
Quoted 7 lines · Thomas Gleixner
arch/x86/kernel/process.c +7
-0+static inline int copy_io_bitmap (struct task_struct *tsk) +{ +if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP ))) +return 0 ; + + tsk->thread.io_bitmap_ptr =kmemdup (current->thread.io_bitmap_ptr, +IO_BITMAP_BYTES ,GFP_KERNEL );
tsk->thread.io_bitmap_max=current->thread.io_bitmap_max?
Quoted 2 lines
I realize you inherited this from the code you're refactoring, but it does seem to be missing.
It already got copied with the task struct :)
Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/process.c +6
-0+#ifdef CONFIG_X86_64 +savesegment (gs, p->thread.gsindex); + p->thread.gsbase = p->thread.gsindex ?0 : current->thread.gsbase; +savesegment (fs, p->thread.fsindex); + p->thread.fsbase = p->thread.fsindex ?0 : current->thread.fsbase; +savesegment (es, p->thread.es);
Quoted 8 lines
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 197
Quoted 5 lines
arch/x86/kernel/process.c +5
-0+savesegment (ds, p->thread.ds); +#else +/* Clear all status flags including IF and set fixed bit. */ + frame->flags =X86_EFLAGS_FIXED ; +#endif Want to do another commit to make the eflags fixup unconditional? I'm wondering if we have a bug.
Let me look at that.
Thanks,
tglx
Re: [patch 2/9] x86/process: Unify copy_thread_tls()
Quoted 32 lines · Thomas Gleixner
Quoted 11 lines · Andy Lutomirski
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 151
Quoted 7 lines · Thomas Gleixner
arch/x86/kernel/process.c +7
-0+static inline int copy_io_bitmap (struct task_struct *tsk) +{ +if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP ))) +return 0 ; + + tsk->thread.io_bitmap_ptr =kmemdup (current->thread.io_bitmap_ptr, +IO_BITMAP_BYTES ,GFP_KERNEL );
tsk->thread.io_bitmap_max=current->thread.io_bitmap_max?Quoted 2 lines
I realize you inherited this from the code you're refactoring, but it does seem to be missing.
It already got copied with the task struct :)
Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/process.c +6
-0+#ifdef CONFIG_X86_64 +savesegment (gs, p->thread.gsindex); + p->thread.gsbase = p->thread.gsindex ?0 : current->thread.gsbase; +savesegment (fs, p->thread.fsindex); + p->thread.fsbase = p->thread.fsindex ?0 : current->thread.fsbase; +savesegment (es, p->thread.es);Quoted 8 lines
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 197
Quoted 5 lines
arch/x86/kernel/process.c +5
-0+savesegment (ds, p->thread.ds); +#else +/* Clear all status flags including IF and set fixed bit. */ + frame->flags =X86_EFLAGS_FIXED ; +#endif Want to do another commit to make the eflags fixup unconditional? I'm wondering if we have a bug.
Quoted 1 line
Let me look at that.
64bit does not have flags in the inactive_task_frame ...
Thanks,
tglx
Re: [patch 2/9] x86/process: Unify copy_thread_tls()
Quoted 34 lines · Thomas Gleixner
Quoted 32 lines · Thomas Gleixner
Quoted 11 lines · Andy Lutomirski
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 151
Quoted 7 lines · Thomas Gleixner
arch/x86/kernel/process.c +7
-0+static inline int copy_io_bitmap (struct task_struct *tsk) +{ +if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP ))) +return 0 ; + + tsk->thread.io_bitmap_ptr =kmemdup (current->thread.io_bitmap_ptr, +IO_BITMAP_BYTES ,GFP_KERNEL );
tsk->thread.io_bitmap_max=current->thread.io_bitmap_max?Quoted 2 lines
I realize you inherited this from the code you're refactoring, but it does seem to be missing.
It already got copied with the task struct :)
Quoted 14 lines
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 197
Quoted 11 lines
arch/x86/kernel/process.c +11
-0+#ifdef CONFIG_X86_64 +savesegment (gs, p->thread.gsindex); + p->thread.gsbase = p->thread.gsindex ?0 : current->thread.gsbase; +savesegment (fs, p->thread.fsindex); + p->thread.fsbase = p->thread.fsindex ?0 : current->thread.fsbase; +savesegment (es, p->thread.es); +savesegment (ds, p->thread.ds); +#else +/* Clear all status flags including IF and set fixed bit. */ + frame->flags =X86_EFLAGS_FIXED ; +#endif Want to do another commit to make the eflags fixup unconditional? I'm wondering if we have a bug.
Quoted 4 lines
Quoted 1 line
Let me look at that.
64bit does not have flags in the
inactive_task_frame...
Hmm. One more thing to unify, I guess.
--Andy
Re: [patch 2/9] x86/process: Unify copy_thread_tls()
Quoted 6 lines · Andy Lutomirski
Quoted 2 lines · Thomas Gleixner
64bit does not have flags in the
inactive_task_frame...Hmm. One more thing to unify, I guess.
All that takes is porting objtool to 32bit.
See commit: 64604d54d311 ("sched/x86_64: Don't save flags on context switch")
PATCH 3/9 x86/cpu: Unify cpu_init()
In reply to Thomas GleixnerSimilar to copy_thread_tls() the 32bit and 64bit implementations of cpu_init() are very similar and unification avoids duplicate changes in the future.
--- arch/x86/kernel/cpu/common.c | 172 ++++++++++++++++--------------------------- 1 file changed, 66 insertions(+), 106 deletions(-)
arch/x86/kernel/cpu/common.c +66 -106
@@ -1749,7 +1749,7 @@ static void wait_for_master_cpu(int cpu) }#ifdef CONFIG_X86_64 -static void setup_getcpu (int cpu) +static inline void setup_getcpu (int cpu) {unsigned long cpudata =vdso_encode_cpunode (cpu,early_cpu_to_node (cpu));struct desc_struct d = { };
@@ -1769,7 +1769,43 @@ static void setup_getcpu(int cpu)write_gdt_entry (get_cpu_gdt_rw (cpu),GDT_ENTRY_CPUNODE , &d,DESCTYPE_S ); } + +static inline void ucode_cpu_init (int cpu) +{ +if (cpu) +load_ucode_ap (); +} + +static inline void tss_setup_ist (struct tss_struct *tss) +{ +/* Set up the per-CPU TSS IST stacks */ + tss->x86_tss.ist[IST_INDEX_DF ] =__this_cpu_ist_top_va (DF ); + tss->x86_tss.ist[IST_INDEX_NMI ] =__this_cpu_ist_top_va (NMI ); + tss->x86_tss.ist[IST_INDEX_DB ] =__this_cpu_ist_top_va (DB ); + tss->x86_tss.ist[IST_INDEX_MCE ] =__this_cpu_ist_top_va (MCE ); +} + +static inline void gdt_setup_doublefault_tss (int cpu) { } + +#else /* CONFIG_X86_64 */ + +static inline void setup_getcpu (int cpu) { } + +static inline void ucode_cpu_init (int cpu) +{ +show_ucode_info_early (); +} + +static inline void tss_setup_ist (struct tss_struct *tss) { } + +static inline void gdt_setup_doublefault_tss (int cpu) +{ +#ifdef CONFIG_DOUBLEFAULT +/* Set up the doublefault TSS pointer in the GDT */ +__set_tss_desc (cpu,GDT_ENTRY_DOUBLEFAULT_TSS , &doublefault_tss);#endif +} +#endif /* !CONFIG_X86_64 */ /* * cpu_init() initializes state that is per-CPU. Some data is already
@@ -1777,21 +1813,15 @@ static void setup_getcpu(int cpu)* and IDT. We reload them nevertheless, this function acts as a * 'CPU state barrier', nothing should get across. */ -#ifdef CONFIG_X86_64 -void cpu_init (void ) { +struct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw); +struct task_struct *cur = current;int cpu =raw_smp_processor_id (); -struct task_struct *me; -struct tss_struct *t; -int i;wait_for_master_cpu (cpu); -if (cpu) -load_ucode_ap (); - - t = &per_cpu (cpu_tss_rw, cpu); +ucode_cpu_init (cpu);#ifdef CONFIG_NUMA if (this_cpu_read(numa_node) == 0 &&
@@ -1800,63 +1830,48 @@ void cpu_init(void)#endif setup_getcpu (cpu); - me = current; -pr_debug ("Initializing CPU#%d\n" , cpu); -cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE ); +if (IS_ENABLED (CONFIG_X86_64 ) ||cpu_feature_enabled (X86_FEATURE_VME ) || +boot_cpu_has (X86_FEATURE_TSC ) ||boot_cpu_has (X86_FEATURE_DE )) +cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE );/* * Initialize the per-CPU GDT with the boot GDT, * and set up the GDT descriptor: */ -switch_to_new_gdt (cpu); -loadsegment (fs,0 ); -load_current_idt (); -memset (me->thread.tls_array,0 ,GDT_ENTRY_TLS_ENTRIES *8 ); -syscall_init (); - -wrmsrl (MSR_FS_BASE ,0 ); -wrmsrl (MSR_KERNEL_GS_BASE ,0 ); -barrier (); +if (IS_ENABLED (CONFIG_X86_64 )) { +loadsegment (fs,0 ); +memset (cur->thread.tls_array,0 ,GDT_ENTRY_TLS_ENTRIES *8 ); +syscall_init (); + +wrmsrl (MSR_FS_BASE ,0 ); +wrmsrl (MSR_KERNEL_GS_BASE ,0 ); +barrier (); -x86_configure_nx (); -x2apic_setup (); - -/* -* set up and load the per-CPU TSS -*/ -if (!t->x86_tss.ist[0 ]) { - t->x86_tss.ist[IST_INDEX_DF ] =__this_cpu_ist_top_va (DF ); - t->x86_tss.ist[IST_INDEX_NMI ] =__this_cpu_ist_top_va (NMI ); - t->x86_tss.ist[IST_INDEX_DB ] =__this_cpu_ist_top_va (DB ); - t->x86_tss.ist[IST_INDEX_MCE ] =__this_cpu_ist_top_va (MCE ); +x2apic_setup (); } - t->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; - -/* -* <= is required because the CPU will access up to -* 8 bits beyond the end of the IO permission bitmap. -*/ -for (i =0 ; i <=IO_BITMAP_LONGS ; i++) - t->io_bitmap[i] = ~0UL ; -mmgrab (&init_mm); - me->active_mm = &init_mm; -BUG_ON (me->mm); + cur->active_mm = &init_mm; +BUG_ON (cur->mm);initialize_tlbstate_and_flush (); -enter_lazy_tlb (&init_mm, me); +enter_lazy_tlb (&init_mm, cur); -/* -* Initialize the TSS. sp0 points to the entry trampoline stack -* regardless of what task is running. -*/ +/* Initialize the TSS. */ +tss_setup_ist (tss); + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; +memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss); +load_TR_desc (); +/* +* sp0 points to the entry trampoline stack regardless of what task +* is running. +*/ load_sp0 ((unsigned long )(cpu_entry_stack (cpu) +1 ));load_mm_ldt (&init_mm);
@@ -1864,6 +1879,8 @@ void cpu_init(void)clear_all_debug_regs ();dbg_restore_debug_regs (); +gdt_setup_doublefault_tss (cpu); +fpu__init_cpu ();if (is_uv_system ())
@@ -1872,63 +1889,6 @@ void cpu_init(void)load_fixmap_gdt (cpu); } -#else - -void cpu_init (void ) -{ -int cpu =smp_processor_id (); -struct task_struct *curr = current; -struct tss_struct *t = &per_cpu (cpu_tss_rw, cpu); - -wait_for_master_cpu (cpu); - -show_ucode_info_early (); - -pr_info ("Initializing CPU#%d\n" , cpu); - -if (cpu_feature_enabled (X86_FEATURE_VME ) || -boot_cpu_has (X86_FEATURE_TSC ) || -boot_cpu_has (X86_FEATURE_DE )) -cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE ); - -load_current_idt (); -switch_to_new_gdt (cpu); - -/* -* Set up and load the per-CPU TSS and LDT -*/ -mmgrab (&init_mm); - curr->active_mm = &init_mm; -BUG_ON (curr->mm); -initialize_tlbstate_and_flush (); -enter_lazy_tlb (&init_mm, curr); - -/* -* Initialize the TSS. sp0 points to the entry trampoline stack -* regardless of what task is running. -*/ -set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss); -load_TR_desc (); -load_sp0 ((unsigned long )(cpu_entry_stack (cpu) +1 )); - -load_mm_ldt (&init_mm); - - t->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; - -#ifdef CONFIG_DOUBLEFAULT -/* Set up doublefault TSS pointer in the GDT */ -__set_tss_desc (cpu,GDT_ENTRY_DOUBLEFAULT_TSS , &doublefault_tss); -#endif - -clear_all_debug_regs (); -dbg_restore_debug_regs (); - -fpu__init_cpu (); - -load_fixmap_gdt (cpu); -} -#endif -/* * The microcode loader calls this upon late microcode load to recheck features, * only when microcode has been updated. Caller holds microcode_mutex and CPU
Re: [patch 3/9] x86/cpu: Unify cpu_init()
Quoted 4 lines · Thomas Gleixner
Similar to
copy_thread_tls()the 32bit and 64bit implementations ofcpu_init()are very similar and unification avoids duplicate changes in the future.
I believe you :)
kbuild test robot
build botRe: [patch 3/9] x86/cpu: Unify cpu_init()
In reply to Thomas GleixnerI love your patch! Yet something to improve:
Hi Thomas,
I love your patch! Yet something to improve:
[auto build test ERROR on xen-tip/linux-next]
[also build test ERROR on v5.4-rc6 next-20191108]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Thomas-Gleixner/x86-iopl-Prevent-user-space-from-using-CLI-STI-with-iopl-3/20191109-051154 base: https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git linux-next config: i386-tinyconfig (attached as .config) compiler: gcc-7 (Debian 7.4.0-14) 7.4.0 reproduce: # save the attached .config to linux build tree make ARCH=i386
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
arch/x86/kernel/cpu/common.c: In function 'cpu_init':
>> arch/x86/kernel/cpu/common.c:1886:6: error: implicit declaration of function 'is_uv_system'; did you mean 'put_filesystem'? [-Werror=implicit-function-declaration]
if (is_uv_system())
^~~~~~~~~~~~
put_filesystem
>> arch/x86/kernel/cpu/common.c:1887:3: error: implicit declaration of function 'uv_cpu_init'; did you mean 'cpu_init'? [-Werror=implicit-function-declaration]
uv_cpu_init();
^~~~~~~~~~~
cpu_initcc1: some warnings being treated as errors
vim +1886 arch/x86/kernel/cpu/common.c
b2e2ba578e016a arch/x86/kernel/cpu/common.c Chang S. Bae 2018-09-18 1809
d2cbcc49e2bfd6 arch/i386/kernel/cpu/common.c Rusty Russell 2007-05-02 1810 /*
d2cbcc49e2bfd6 arch/i386/kernel/cpu/common.c Rusty Russell 2007-05-02 1811 * cpu_init() initializes state that is per-CPU. Some data is already
d2cbcc49e2bfd6 arch/i386/kernel/cpu/common.c Rusty Russell 2007-05-02 1812 * initialized (naturally) in the bootstrap process, such as the GDT
d2cbcc49e2bfd6 arch/i386/kernel/cpu/common.c Rusty Russell 2007-05-02 1813 * and IDT. We reload them nevertheless, this function acts as a
d2cbcc49e2bfd6 arch/i386/kernel/cpu/common.c Rusty Russell 2007-05-02 1814 * 'CPU state barrier', nothing should get across.
d2cbcc49e2bfd6 arch/i386/kernel/cpu/common.c Rusty Russell 2007-05-02 1815 */
148f9bb87745ed arch/x86/kernel/cpu/common.c Paul Gortmaker 2013-06-18 1816 void cpu_init(void)
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1817 {
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1818 struct tss_struct *tss = this_cpu_ptr(&cpu_tss_rw);
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1819 struct task_struct *cur = current;
f6ef73224a0f04 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-04-14 1820 int cpu = raw_smp_processor_id();
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1821
ce4b1b16502b18 arch/x86/kernel/cpu/common.c Igor Mammedov 2014-06-20 1822 wait_for_master_cpu(cpu);
ce4b1b16502b18 arch/x86/kernel/cpu/common.c Igor Mammedov 2014-06-20 1823
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1824 ucode_cpu_init(cpu);
0f3fa48a7eaf5d arch/x86/kernel/cpu/common.c Ingo Molnar 2009-03-14 1825
e7a22c1ebcc1ca arch/x86/kernel/cpu/common.c Brian Gerst 2009-01-19 1826 #ifdef CONFIG_NUMA
27fd185f3d9e83 arch/x86/kernel/cpu/common.c Fenghua Yu 2012-11-13 1827 if (this_cpu_read(numa_node) == 0 &&
e534c7c5f8d6e9 arch/x86/kernel/cpu/common.c Lee Schermerhorn 2010-05-26 1828 early_cpu_to_node(cpu) != NUMA_NO_NODE)
e534c7c5f8d6e9 arch/x86/kernel/cpu/common.c Lee Schermerhorn 2010-05-26 1829 set_numa_node(early_cpu_to_node(cpu));
e7a22c1ebcc1ca arch/x86/kernel/cpu/common.c Brian Gerst 2009-01-19 1830 #endif
b2e2ba578e016a arch/x86/kernel/cpu/common.c Chang S. Bae 2018-09-18 1831 setup_getcpu(cpu);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1832
2eaad1fddd7450 arch/x86/kernel/cpu/common.c Mike Travis 2009-12-10 1833 pr_debug("Initializing CPU#%d\n", cpu);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1834
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1835 if (IS_ENABLED(CONFIG_X86_64) || cpu_feature_enabled(X86_FEATURE_VME) ||
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1836 boot_cpu_has(X86_FEATURE_TSC) || boot_cpu_has(X86_FEATURE_DE))
375074cc736ab1 arch/x86/kernel/cpu/common.c Andy Lutomirski 2014-10-24 1837 cr4_clear_bits(X86_CR4_VME|X86_CR4_PVI|X86_CR4_TSD|X86_CR4_DE);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1838
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1839 /*
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1840 * Initialize the per-CPU GDT with the boot GDT,
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1841 * and set up the GDT descriptor:
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1842 */
552be871e67ff5 arch/x86/kernel/cpu/common.c Brian Gerst 2009-01-30 1843 switch_to_new_gdt(cpu);
cf910e83ae2369 arch/x86/kernel/cpu/common.c Seiji Aguchi 2013-06-20 1844 load_current_idt();
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1845
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1846 if (IS_ENABLED(CONFIG_X86_64)) {
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1847 loadsegment(fs, 0);
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1848 memset(cur->thread.tls_array, 0, GDT_ENTRY_TLS_ENTRIES * 8);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1849 syscall_init();
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1850
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1851 wrmsrl(MSR_FS_BASE, 0);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1852 wrmsrl(MSR_KERNEL_GS_BASE, 0);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1853 barrier();
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1854
659006bf3ae37a arch/x86/kernel/cpu/common.c Thomas Gleixner 2015-01-15 1855 x2apic_setup();
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1856 }
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1857
f1f1007644ffc8 arch/x86/kernel/cpu/common.c Vegard Nossum 2017-02-27 1858 mmgrab(&init_mm);
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1859 cur->active_mm = &init_mm;
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1860 BUG_ON(cur->mm);
72c0098d92cedb arch/x86/kernel/cpu/common.c Andy Lutomirski 2017-09-06 1861 initialize_tlbstate_and_flush();
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1862 enter_lazy_tlb(&init_mm, cur);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1863
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1864 /* Initialize the TSS. */
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1865 tss_setup_ist(tss);
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1866 tss->x86_tss.io_bitmap_base = IO_BITMAP_OFFSET;
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1867 memset(tss->io_bitmap, 0xff, sizeof(tss->io_bitmap));
72f5e08dbba2d0 arch/x86/kernel/cpu/common.c Andy Lutomirski 2017-12-04 1868 set_tss_desc(cpu, &get_cpu_entry_area(cpu)->tss.x86_tss);
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1869
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1870 load_TR_desc();
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1871 /*
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1872 * sp0 points to the entry trampoline stack regardless of what task
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1873 * is running.
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1874 */
4fe2d8b11a370a arch/x86/kernel/cpu/common.c Dave Hansen 2017-12-04 1875 load_sp0((unsigned long)(cpu_entry_stack(cpu) + 1));
20bb83443ea790 arch/x86/kernel/cpu/common.c Andy Lutomirski 2017-11-02 1876
37868fe113ff2b arch/x86/kernel/cpu/common.c Andy Lutomirski 2015-07-30 1877 load_mm_ldt(&init_mm);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1878
9766cdbcb26038 arch/x86/kernel/cpu/common.c Jaswinder Singh Rajput 2009-03-14 1879 clear_all_debug_regs();
0bb9fef9134cf4 arch/x86/kernel/cpu/common.c Jason Wessel 2010-05-20 1880 dbg_restore_debug_regs();
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1881
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1882 gdt_setup_doublefault_tss(cpu);
f4accbf9aad814 arch/x86/kernel/cpu/common.c Thomas Gleixner 2019-11-06 1883
21c4cd108a1b14 arch/x86/kernel/cpu/common.c Ingo Molnar 2015-04-26 1884 fpu__init_cpu();
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1885
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 @1886 if (is_uv_system())
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 @1887 uv_cpu_init();
69218e47994da6 arch/x86/kernel/cpu/common.c Thomas Garnier 2017-03-14 1888
69218e47994da6 arch/x86/kernel/cpu/common.c Thomas Garnier 2017-03-14 1889 load_fixmap_gdt(cpu);
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1890 }
1ba76586f778be arch/x86/kernel/cpu/common.c Yinghai Lu 2008-09-04 1891 :::::: The code at line 1886 was first introduced by commit
:::::: 1ba76586f778be327e452180d8378e40ee70f066 x86: cpu/common*.c have same cpu_init(), with copying and #ifdef
:::::: TO: Yinghai Lu <[email protected]>
:::::: CC: Ingo Molnar <[email protected]>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org Intel Corporation
PATCH 4/9 x86/io: Speedup schedule out of I/O bitmap user
In reply to Thomas GleixnerThere is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the io_bitmap_base member of the tss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.
If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
--- arch/x86/include/asm/processor.h | 38 +++++++++++++++++-------- arch/x86/kernel/cpu/common.c | 3 + arch/x86/kernel/doublefault.c | 2 - arch/x86/kernel/process.c | 59 +++++++++++++++++++++------------------ 4 files changed, 61 insertions(+), 41 deletions(-)
arch/x86/include/asm/processor.h +26 -12
@@ -330,8 +330,23 @@ struct x86_hw_tss {#define IO_BITMAP_BITS 65536#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) -#define IO_BITMAP_OFFSET (offsetof(struct tss_struct, io_bitmap) - offsetof(struct tss_struct, x86_tss)) -#define INVALID_IO_BITMAP_OFFSET 0x8000 + +#define IO_BITMAP_OFFSET_VALID \ + (offsetof(struct tss_struct, io_bitmap) - \ + offsetof(struct tss_struct, x86_tss)) + +/* +* sizeof(unsigned long) coming from an extra "long" at the end +* of the iobitmap. +* +* -1? seg base+limit should be pointing to the address of the +* last valid byte +*/ +#define __KERNEL_TSS_LIMIT \ + (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + +/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ +#define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)struct entry_stack {unsigned long words[64 ];
@@ -350,6 +365,15 @@ struct tss_struct {struct x86_hw_tss x86_tss;/* +* Store the dirty size of the last io bitmap offender. The next +* one will have to do the cleanup as the switch out to a non +* io bitmap user will just set x86_tss.io_bitmap_base to a value +* outside of the TSS limit. So for sane tasks there is no need +* to actually touch the io_bitmap at all. +*/ +unsigned int io_bitmap_prev_max; + +/* * The extra 1 is there because the CPU will access an * additional byte beyond the end of the IO permission * bitmap. The extra byte must be all 1 bits, and must
@@ -360,16 +384,6 @@ struct tss_struct { DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw); -/* -* sizeof(unsigned long) coming from an extra "long" at the end -* of the iobitmap. -* -* -1? seg base+limit should be pointing to the address of the -* last valid byte -*/ -#define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) -/* Per CPU interrupt stacks */ struct irq_stack {char stack[IRQ_STACK_SIZE ];
arch/x86/kernel/cpu/common.c +2 -1
@@ -1863,7 +1863,8 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + tss->io_bitmap_prev_max =0 ;memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);
arch/x86/kernel/doublefault.c +1 -1
@@ -54,7 +54,7 @@ struct x86_hw_tss doublefault_tss __cach .sp0 = STACK_START, .ss0 = __KERNEL_DS, .ldt = 0, - .io_bitmap_base = INVALID_IO_BITMAP_OFFSET, + .io_bitmap_base = IO_BITMAP_OFFSET_INVALID, .ip = (unsigned long) doublefault_fn,/* 0x2 bit is always set */
arch/x86/kernel/process.c +32 -27
104 lines not on this page · raw message
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
Quoted 8 lines · Thomas Gleixner
There is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the
io_bitmap_basemember of thetss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.
Quoted 6 lines
If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
This also nicely aligns with that the context switch time is accounted to the next task. So by doing the expensive part on switch-in gets it all accounted to the task that caused it.
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
Quoted 12 lines · Peter Zijlstra
Quoted 11 lines · Thomas Gleixner
There is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the
io_bitmap_basemember of thetss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
Quoted 7 lines
Quoted 3 lines
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
This also nicely aligns with that the context switch time is accounted to the next task. So by doing the expensive part on switch-in gets it all accounted to the task that caused it.
Just that I can't add the storage to tss_struct due to the VMX insanity of setting TSS limit hard to 0x67 on vmexit instead of restoring the host value.
Did I say that already that virt creates more problems than it solves?
Thanks,
tglx
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
Quoted 17 lines · Thomas Gleixner
Quoted 16 lines · Peter Zijlstra
Quoted 14 lines · Thomas Gleixner
There is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the
io_bitmap_basemember of thetss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
Quoted 7 lines
Quoted 3 lines
This also nicely aligns with that the context switch time is accounted to the next task. So by doing the expensive part on switch-in gets it all accounted to the task that caused it.
Just that I can't add the storage to
tss_structdue to the VMX insanity of setting TSS limit hard to0x67on vmexit instead of restoring the host value.
Well, I can. The build bugon in vmx.c is just bogus.
Quoted 1 line
Did I say that already that virt creates more problems than it solves?
Still stands.
tglx
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
Quoted 22 lines · Thomas Gleixner
Quoted 21 lines · Thomas Gleixner
Quoted 19 lines · Peter Zijlstra
Quoted 8 lines · Thomas Gleixner
There is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the
io_bitmap_basemember of thetss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.Quoted 6 lines
If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
This also nicely aligns with that the context switch time is accounted to the next task. So by doing the expensive part on switch-in gets it all accounted to the task that caused it.
Quoted 5 lines
Quoted 3 lines
Just that I can't add the storage to
tss_structdue to the VMX insanity of setting TSS limit hard to0x67on vmexit instead of restoring the host value.Well, I can. The build bugon in
vmx.cis just bogus.
SDM vol 3 27.5.2 says the BUILD_BUG_ON is right. Or am I misunderstanding you?
I'm reasonably confident that the TSS limit is indeed 0x67 after VM exit, and I wrote the existing code that tries to optimize this to avoid LTR when not needed.
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
Quoted 6 lines · Andy Lutomirski
Quoted 5 lines · Thomas Gleixner
Quoted 3 lines · Thomas Gleixner
Just that I can't add the storage to
tss_structdue to the VMX insanity of setting TSS limit hard to0x67on vmexit instead of restoring the host value.
Quoted 8 lines
Quoted 1 line
Well, I can. The build bugon in
vmx.cis just bogus.SDM vol 3 27.5.2 says the
BUILD_BUG_ONis right. Or am I misunderstanding you?I'm reasonably confident that the TSS limit is indeed
0x67after VM exit, and I wrote the existing code that tries to optimize this to avoid LTR when not needed.
The BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 == 0x67) in the VMX code is bogus in two aspects:
1) This wants to be in generic x86 code
2) The IO_BITMAP_OFFSET is not the right thing to check because it makes asssumptions about the layout of tss_struct. Nothing requires that the I/O bitmap is placed right after x86_tss, which is the hardware mandated tss structure. It pointlessly makes restrictions on the struct tss_struct layout.
The proper thing to check is:
- Offset of x86_tss in tss_struct is 0
- Size of x86_tss == 0x68
We already have the page alignment sanity check off TSS in cpu_entry_area.c. That's where this should have gone into in the first place.
Thanks,
tglx
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
Quoted 16 lines · Thomas Gleixner
Quoted 11 lines · Andy Lutomirski
Quoted 7 lines · Thomas Gleixner
Quoted 3 lines · Thomas Gleixner
Just that I can't add the storage to
tss_structdue to the VMX insanity of setting TSS limit hard to0x67on vmexit instead of restoring the host value.Well, I can. The build bugon in
vmx.cis just bogus.SDM vol 3 27.5.2 says the
BUILD_BUG_ONis right. Or am I misunderstanding you?
Quoted 8 lines
Quoted 3 lines
I'm reasonably confident that the TSS limit is indeed
0x67after VM exit, and I wrote the existing code that tries to optimize this to avoid LTR when not needed.The
BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 == 0x67)in the VMX code is bogus in two aspects:1) This wants to be in generic x86 code
I think disagree. The only thing special about 0x67 is that VMX hard codes it. It’s specifically a VMX-ism. So I think the VMX code should indeed assert that 0x67 is a safe value.
Quoted 6 lines
2) The
IO_BITMAP_OFFSETis not the right thing to check because it makes asssumptions about the layout oftss_struct. Nothing requires that the I/O bitmap is placed right afterx86_tss, which is the hardware mandated tss structure. It pointlessly makes restrictions on the structtss_structlayout.
I agree with this.
Quoted 14 lines
The proper thing to check is:
- Offset of
x86_tssintss_structis 0
- Size ofx86_tss==0x68We already have the page alignment sanity check off TSS in
cpu_entry_area.c. That's where this should have gone into in the first place.Thanks,
tglx
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
Quoted 9 lines · Andy Lutomirski
Quoted 9 lines · Thomas Gleixner
Quoted 6 lines · Andy Lutomirski
SDM vol 3 27.5.2 says the
BUILD_BUG_ONis right. Or am I misunderstanding you?I'm reasonably confident that the TSS limit is indeed
0x67after VM exit, and I wrote the existing code that tries to optimize this to avoid LTR when not needed.
Quoted 8 lines
Quoted 4 lines
The
BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 == 0x67)in the VMX code is bogus in two aspects:1) This wants to be in generic x86 code
I think disagree. The only thing special about
0x67is that VMX hard codes it. It’s specifically a VMX-ism. So I think the VMX code should indeed assert that0x67is a safe value.
Yes, it is a VMX specific issue, but I really prefer the build to fail in the common code without having to enable VMX if something changes the TSS layout in incompatible ways.
Thanks,
tglx
Re: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
In reply to Thomas GleixnerQuoted 32 lines · Thomas Gleixner
There is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the
io_bitmap_basemember of thetss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
Signed-off-by Thomas Gleixner <[email protected]>--- arch/x86/include/asm/processor.h | 38 +++++++++++++++++-------- arch/x86/kernel/cpu/common.c | 3 + arch/x86/kernel/doublefault.c | 2 - arch/x86/kernel/process.c | 59 +++++++++++++++++++++------------------ 4 files changed, 61 insertions(+), 41 deletions(-)arch/x86/include/asm/processor.h +1
-2@@ -330,8 +330,23 @@ struct x86_hw_tss {#define IO_BITMAP_BITS 65536#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) -#define IO_BITMAP_OFFSET (offsetof(struct tss_struct, io_bitmap) - offsetof(struct tss_struct, x86_tss)) -#define INVALID_IO_BITMAP_OFFSET 0x8000 +
arch/x86/include/asm/processor.h struct x86_hw_tss { · line 343
Quoted 10 lines
arch/x86/include/asm/processor.h +10
-0+#define IO_BITMAP_OFFSET_VALID \ + (offsetof(struct tss_struct, io_bitmap) - \ + offsetof(struct tss_struct, x86_tss)) + +/* +* sizeof(unsigned long) coming from an extra "long" at the end +* of the iobitmap. +* +* -1? seg base+limit should be pointing to the address of the +* last valid byte
What's with the '?'
Quoted 10 lines
arch/x86/include/asm/processor.h +6
-0+*/ +#define __KERNEL_TSS_LIMIT \ + (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + +/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ +#define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)struct entry_stack {unsigned long words[64 ];
arch/x86/include/asm/processor.h struct tss_struct { · line 374
Quoted 10 lines
arch/x86/include/asm/processor.h +7
-0@@ -350,6 +365,15 @@ struct tss_struct {struct x86_hw_tss x86_tss;/* +* Store the dirty size of the last io bitmap offender. The next +* one will have to do the cleanup as the switch out to a non +* io bitmap user will just set x86_tss.io_bitmap_base to a value +* outside of the TSS limit. So for sane tasks there is no need +* to actually touch the io_bitmap at all. +*/ +unsigned int io_bitmap_prev_max;
Hmm. I'm wondering if a clearer way to say this would be:
io_bitmap_max_allowed_byte: offset from the start of the io bitmap to the first byte that might contain a zero bit. If we switch from a task that uses ioperm() to one that does not, we will invalidate the io bitmap by changing the offset. The next task that starts using the io bitmap again will need to make sure it updates all the bytes through io_bitmap_max_allowed_byte.
But your description is okay, too.
It's worth noting that, due to Meltdown, this patch leaks the io bitmap of io bitmap-using tasks. I'm not sure I care.
Quoted 79 lines
arch/x86/include/asm/processor.h +2
-10+ +/* * The extra 1 is there because the CPU will access an * additional byte beyond the end of the IO permission * bitmap. The extra byte must be all 1 bits, and must @@ -360,16 +384,6 @@ struct tss_struct { DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw); -/* -* sizeof(unsigned long) coming from an extra "long" at the end -* of the iobitmap. -* -* -1? seg base+limit should be pointing to the address of the -* last valid byte -*/ -#define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) -/* Per CPU interrupt stacks */ struct irq_stack {char stack[IRQ_STACK_SIZE ];arch/x86/kernel/cpu/common.c +2
-1@@ -1863,7 +1863,8 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + tss->io_bitmap_prev_max =0 ;memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);arch/x86/kernel/doublefault.c +1
-1@@ -54,7 +54,7 @@ struct x86_hw_tss doublefault_tss __cach .sp0 = STACK_START, .ss0 = __KERNEL_DS, .ldt = 0, - .io_bitmap_base = INVALID_IO_BITMAP_OFFSET, + .io_bitmap_base = IO_BITMAP_OFFSET_INVALID, .ip = (unsigned long) doublefault_fn,/* 0x2 bit is always set */ arch/x86/kernel/process.c +4
-11@@ -72,18 +72,9 @@#ifdef CONFIG_X86_32 .ss0 = __KERNEL_DS, .ss1 = __KERNEL_CS, - .io_bitmap_base = INVALID_IO_BITMAP_OFFSET, #endif + .io_bitmap_base =IO_BITMAP_OFFSET_INVALID , }, -#ifdef CONFIG_X86_32 -/* -* Note that the .io_bitmap member must be extra-big. This is because -* the CPU will access an additional byte beyond the end of the IO -* permission bitmap. The extra byte must be all 1 bits, and must -* be within the limit. -*/ - .io_bitmap = { [0 ... IO_BITMAP_LONGS] = ~0 }, -#endif };EXPORT_PER_CPU_SYMBOL (cpu_tss_rw);@@ -112,18 +103,18 @@ void exit_thread(struct task_struct *tskstruct thread_struct *t = &tsk->thread;unsigned long *bp = t->io_bitmap_ptr;struct fpu *fpu = &t->fpu; +struct tss_struct *tss;if (bp) { -struct tss_struct *tss = &per_cpu (cpu_tss_rw,get_cpu ()); +preempt_disable (); + tss =this_cpu_ptr (&cpu_tss_rw); t->io_bitmap_ptr =NULL ;
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 116
Quoted 10 lines
arch/x86/kernel/process.c +3
-6-clear_thread_flag (TIF_IO_BITMAP ); -/* -* Careful, clear this in the TSS too: -*/ -memset (tss->io_bitmap,0xff , t->io_bitmap_max); t->io_bitmap_max =0 ; -put_cpu (); +clear_thread_flag (TIF_IO_BITMAP ); +/* Invalidate the io bitmap base in the TSS */ + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ;
The first time I read this, I thought this code was all unnecessary. But now I think I understand. How about a comment:
/*
* We're about to free the IO bitmap. We need to disable it in the TSS
* too so that switch_to() doesn't see an inconsistent state. */
In any case:
kbuild test robot
build botRe: [patch 4/9] x86/io: Speedup schedule out of I/O bitmap user
In reply to Thomas GleixnerI love your patch! Yet something to improve:
Hi Thomas,
I love your patch! Yet something to improve:
[auto build test ERROR on xen-tip/linux-next]
[cannot apply to v5.4-rc6 next-20191108]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Thomas-Gleixner/x86-iopl-Prevent-user-space-from-using-CLI-STI-with-iopl-3/20191109-051154 base: https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git linux-next config: x86_64-lkp (attached as .config) compiler: gcc-7 (Debian 7.4.0-14) 7.4.0 reproduce: # save the attached .config to linux build tree make ARCH=x86_64
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
In file included from include/linux/export.h:42:0,
from include/linux/linkage.h:7,
from include/linux/fs.h:5,
from include/linux/highmem.h:5,
from arch/x86/kvm/vmx/vmx.c:17:
arch/x86/kvm/vmx/vmx.c: In function 'vmx_vcpu_load_vmcs':
>> arch/x86/kvm/vmx/vmx.c:1347:16: error: 'IO_BITMAP_OFFSET' undeclared (first use in this function); did you mean 'IO_BITMAP_BYTES'?
BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 != 0x67);
^
include/linux/compiler.h:330:9: note: in definition of macro '__compiletime_assert'
if (!(condition)) \
^~~~~~~~~
include/linux/compiler.h:350:2: note: in expansion of macro '_compiletime_assert'
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
#define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:50:2: note: in expansion of macro 'BUILD_BUG_ON_MSG'
BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
^~~~~~~~~~~~~~~~
arch/x86/kvm/vmx/vmx.c:1347:3: note: in expansion of macro 'BUILD_BUG_ON'
BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 != 0x67);
^~~~~~~~~~~~
arch/x86/kvm/vmx/vmx.c:1347:16: note: each undeclared identifier is reported only once for each function it appears in
BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 != 0x67);
^
include/linux/compiler.h:330:9: note: in definition of macro '__compiletime_assert'
if (!(condition)) \
^~~~~~~~~
include/linux/compiler.h:350:2: note: in expansion of macro '_compiletime_assert'
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
#define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:50:2: note: in expansion of macro 'BUILD_BUG_ON_MSG'
BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
^~~~~~~~~~~~~~~~
arch/x86/kvm/vmx/vmx.c:1347:3: note: in expansion of macro 'BUILD_BUG_ON'
BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 != 0x67);
^~~~~~~~~~~~vim +1347 arch/x86/kvm/vmx/vmx.c
1be0e61c1f255f arch/x86/kvm/vmx.c Xiao Guangrong 2016-03-22 1297
8ef863e67a89c7 arch/x86/kvm/vmx/vmx.c Sean Christopherson 2019-05-07 1298 void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu)
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1299 {
a2fa3e9f52d875 drivers/kvm/vmx.c Gregory Haskins 2007-07-27 1300 struct vcpu_vmx *vmx = to_vmx(vcpu);
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1301 bool already_loaded = vmx->loaded_vmcs->cpu == cpu;
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1302
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1303 if (!already_loaded) {
fe0e80befd4d3a arch/x86/kvm/vmx.c David Hildenbrand 2017-03-10 1304 loaded_vmcs_clear(vmx->loaded_vmcs);
92fe13be74303a arch/x86/kvm/vmx.c Dongxiao Xu 2010-05-11 1305 local_irq_disable();
8f536b7697a0d4 arch/x86/kvm/vmx.c Zhang Yanfei 2012-12-06 1306 crash_disable_local_vmclear(cpu);
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1307
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1308 /*
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1309 * Read loaded_vmcs->cpu should be before fetching
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1310 * loaded_vmcs->loaded_vmcss_on_cpu_link.
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1311 * See the comments in __loaded_vmcs_clear().
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1312 */
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1313 smp_rmb();
5a560f8b5ed361 arch/x86/kvm/vmx.c Xiao Guangrong 2012-11-28 1314
d462b8192368f1 arch/x86/kvm/vmx.c Nadav Har'El 2011-05-24 1315 list_add(&vmx->loaded_vmcs->loaded_vmcss_on_cpu_link,
d462b8192368f1 arch/x86/kvm/vmx.c Nadav Har'El 2011-05-24 1316 &per_cpu(loaded_vmcss_on_cpu, cpu));
8f536b7697a0d4 arch/x86/kvm/vmx.c Zhang Yanfei 2012-12-06 1317 crash_enable_local_vmclear(cpu);
92fe13be74303a arch/x86/kvm/vmx.c Dongxiao Xu 2010-05-11 1318 local_irq_enable();
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1319 }
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1320
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1321 if (per_cpu(current_vmcs, cpu) != vmx->loaded_vmcs->vmcs) {
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1322 per_cpu(current_vmcs, cpu) = vmx->loaded_vmcs->vmcs;
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1323 vmcs_load(vmx->loaded_vmcs->vmcs);
15d45071523d89 arch/x86/kvm/vmx.c Ashok Raj 2018-02-01 1324 indirect_branch_prediction_barrier();
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1325 }
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1326
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1327 if (!already_loaded) {
59c58ceb29d0f0 arch/x86/kvm/vmx.c Andy Lutomirski 2017-03-22 1328 void *gdt = get_current_gdt_ro();
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1329 unsigned long sysenter_esp;
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1330
b80c76ec982c00 arch/x86/kvm/vmx.c Jim Mattson 2016-07-29 1331 kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
92fe13be74303a arch/x86/kvm/vmx.c Dongxiao Xu 2010-05-11 1332
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1333 /*
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1334 * Linux uses per-cpu TSS and GDT, so set these when switching
e0c230634af999 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1335 * processors. See 22.2.4.
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1336 */
e0c230634af999 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1337 vmcs_writel(HOST_TR_BASE,
72f5e08dbba2d0 arch/x86/kvm/vmx.c Andy Lutomirski 2017-12-04 1338 (unsigned long)&get_cpu_entry_area(cpu)->tss.x86_tss);
59c58ceb29d0f0 arch/x86/kvm/vmx.c Andy Lutomirski 2017-03-22 1339 vmcs_writel(HOST_GDTR_BASE, (unsigned long)gdt); /* 22.2.4 */
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1340
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1341 /*
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1342 * VM exits change the host TR limit to 0x67 after a VM
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1343 * exit. This is okay, since 0x67 covers everything except
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1344 * the IO bitmap and have have code to handle the IO bitmap
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1345 * being lost after a VM exit.
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1346 */
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 @1347 BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 != 0x67);
b7ffc44d5b2ea1 arch/x86/kvm/vmx.c Andy Lutomirski 2017-02-20 1348
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1349 rdmsrl(MSR_IA32_SYSENTER_ESP, sysenter_esp);
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1350 vmcs_writel(HOST_IA32_SYSENTER_ESP, sysenter_esp); /* 22.2.3 */
ff2c3a1803775c arch/x86/kvm/vmx.c Haozhong Zhang 2015-10-20 1351
d462b8192368f1 arch/x86/kvm/vmx.c Nadav Har'El 2011-05-24 1352 vmx->loaded_vmcs->cpu = cpu;
6aa8b732ca01c3 drivers/kvm/vmx.c Avi Kivity 2006-12-10 1353 }
28b835d60fcc24 arch/x86/kvm/vmx.c Feng Wu 2015-09-18 1354
2680d6da455b63 arch/x86/kvm/vmx.c Owen Hofmann 2016-03-01 1355 /* Setup TSC multiplier */
2680d6da455b63 arch/x86/kvm/vmx.c Owen Hofmann 2016-03-01 1356 if (kvm_has_tsc_control &&
c95ba92afb238a arch/x86/kvm/vmx.c Peter Feiner 2016-08-17 1357 vmx->current_tsc_ratio != vcpu->arch.tsc_scaling_ratio)
c95ba92afb238a arch/x86/kvm/vmx.c Peter Feiner 2016-08-17 1358 decache_tsc_multiplier(vmx);8ef863e67a89c7 arch/x86/kvm/vmx/vmx.c Sean Christopherson 2019-05-07 1359 } 8ef863e67a89c7 arch/x86/kvm/vmx/vmx.c Sean Christopherson 2019-05-07 1360
:::::: The code at line 1347 was first introduced by commit
:::::: b7ffc44d5b2ea163899d09289ca7743d5c32e926 x86/kvm/vmx: Defer TR reload after VM exit
:::::: TO: Andy Lutomirski <[email protected]>
:::::: CC: Paolo Bonzini <[email protected]>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org Intel Corporation
PATCH 5/9 x86/ioport: Reduce ioperm impact for sane usage further
In reply to Thomas GleixnerSane ioperm() users only set the few bits in the I/O space which they need to access. But the update logic of the TSS I/O bitmap copies always from byte 0 to the last byte in the tasks bitmap which contains a zero permission bit.
That means that for access only to port 65335 the full 8K bitmap has to be copied even if all the bytes in the TSS I/O bitmap are already filled with 0xff.
Calculate both the position of the first zero bit and the last zero bit to limit the range which needs to be copied. This does not solve the problem when the previous tasked had only byte 0 cleared and the next one has only byte 65535 cleared, but trying to solve that would be too complex and heavyweight for the context switch path. As the ioperm() usage is very rare the case which is optimized is the single task/process which uses ioperm().
--- arch/x86/include/asm/processor.h | 28 ++++--- arch/x86/kernel/cpu/common.c | 3 arch/x86/kernel/ioport.c | 141 +++++++++++++++++++++++++++------------ arch/x86/kernel/process.c | 51 +++++++++----- arch/x86/kernel/ptrace.c | 2 5 files changed, 152 insertions(+), 73 deletions(-)
arch/x86/include/asm/processor.h +16 -12
@@ -365,19 +365,19 @@ struct tss_struct {struct x86_hw_tss x86_tss;/* -* Store the dirty size of the last io bitmap offender. The next -* one will have to do the cleanup as the switch out to a non -* io bitmap user will just set x86_tss.io_bitmap_base to a value -* outside of the TSS limit. So for sane tasks there is no need -* to actually touch the io_bitmap at all. +* Store the dirty byte range of the last io bitmap offender. The +* next one will have to do the cleanup because the switch out to a +* non I/O bitmap user will just set x86_tss.io_bitmap_base to a +* value outside of the TSS limit to not penalize tasks which do +* not use the I/O bitmap at all. */ -unsigned int io_bitmap_prev_max; +unsigned int io_zerobits_start; +unsigned int io_zerobits_end;/* -* The extra 1 is there because the CPU will access an -* additional byte beyond the end of the IO permission -* bitmap. The extra byte must be all 1 bits, and must -* be within the limit. +* The extra 1 is there because the CPU will access an additional +* byte beyond the end of the I/O permission bitmap. The extra byte +* must have all bits set and must be within the TSS limit. */ unsigned long io_bitmap[IO_BITMAP_LONGS +1 ]; }__aligned(PAGE_SIZE) ;
@@ -496,8 +496,12 @@ struct thread_struct {/* IO permissions: */ unsigned long *io_bitmap_ptr;unsigned long iopl; -/* Max allowed port in the bitmap, in bytes: */ -unsigned io_bitmap_max; +/* +* The byte range in the I/O permission bitmap which contains zero +* bits. +*/ +unsigned int io_zerobits_start; +unsigned int io_zerobits_end;mm_segment_t addr_limit;
arch/x86/kernel/cpu/common.c +2 -1
@@ -1864,7 +1864,8 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; - tss->io_bitmap_prev_max =0 ; + tss->io_zerobits_start =IO_BITMAP_BYTES ; + tss->io_zerobits_end =0 ;memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);
arch/x86/kernel/ioport.c +100 -41
172 lines not on this page · raw message
arch/x86/kernel/process.c +33 -18
81 lines not on this page · raw message
arch/x86/kernel/ptrace.c +1 -1
@@ -697,7 +697,7 @@ static int ptrace_set_debugreg(struct tastatic int ioperm_active (struct task_struct *target,const struct user_regset *regset) { -return DIV_ROUND_UP (target->thread.io_bitmap_max, regset->size); +return DIV_ROUND_UP (target->thread.io_zerobits_end, regset->size); } static int ioperm_get(struct task_struct *target,
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 7 lines · Thomas Gleixner
Calculate both the position of the first zero bit and the last zero bit to limit the range which needs to be copied. This does not solve the problem when the previous tasked had only byte 0 cleared and the next one has only byte 65535 cleared, but trying to solve that would be too complex and heavyweight for the context switch path. As the
ioperm()usage is very rare the case which is optimized is the single task/process which usesioperm().
Hmm.
I may read this patch wrong, but from what I can tell, if we really have just one process with an io bitmap, we're doing unnecessary copies.
If we really have just one process that has an iobitmap, I think we could just keep the bitmap of that process entirely unchanged. Then, when we switch away from it, we set the io_bitmap_base to an invalid base outside the TSS segment, and when we switch back, we set it back to the valid one. No actual bitmap copies at all.
So I think that rather than the "begin/end offset" games, we should perhaps have a "what was the last process that used the IO bitmap for this TSS" pointer (and, I think, some sequence counter, so that when the process updates its bitmap, it invalidates that case)?
Of course, you can do *nboth*, but if we really think that the common
case is "one special process", then I think the begin/end offset is useless, but a "last bitmap process" would be very useful.
Am I missing something?
Linus
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 15 lines · Linus Torvalds
I may read this patch wrong, but from what I can tell, if we really have just one process with an io bitmap, we're doing unnecessary copies.
If we really have just one process that has an iobitmap, I think we could just keep the bitmap of that process entirely unchanged. Then, when we switch away from it, we set the
io_bitmap_baseto an invalid base outside the TSS segment, and when we switch back, we set it back to the valid one. No actual bitmap copies at all.So I think that rather than the "begin/end offset" games, we should perhaps have a "what was the last process that used the IO bitmap for this TSS" pointer (and, I think, some sequence counter, so that when the process updates its bitmap, it invalidates that case)?
Quoted 5 lines
Of course, you can do *nboth*, but if we really think that the common
case is "one special process", then I think the begin/end offset is useless, but a "last bitmap process" would be very useful.Am I missing something?
No. You are right. I'll have a look at that.
Thanks,
tglx
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Linus TorvaldsQuoted 26 lines · Linus Torvalds
Quoted 7 lines · Thomas Gleixner
Calculate both the position of the first zero bit and the last zero bit to limit the range which needs to be copied. This does not solve the problem when the previous tasked had only byte 0 cleared and the next one has only byte 65535 cleared, but trying to solve that would be too complex and heavyweight for the context switch path. As the
ioperm()usage is very rare the case which is optimized is the single task/process which usesioperm().Hmm.
I may read this patch wrong, but from what I can tell, if we really have just one process with an io bitmap, we're doing unnecessary copies.
If we really have just one process that has an iobitmap, I think we could just keep the bitmap of that process entirely unchanged. Then, when we switch away from it, we set the
io_bitmap_baseto an invalid base outside the TSS segment, and when we switch back, we set it back to the valid one. No actual bitmap copies at all.So I think that rather than the "begin/end offset" games, we should perhaps have a "what was the last process that used the IO bitmap for this TSS" pointer (and, I think, some sequence counter, so that when the process updates its bitmap, it invalidates that case)?
Quoted 5 lines
Of course, you can do *nboth*, but if we really think that the common
case is "one special process", then I think the begin/end offset is useless, but a "last bitmap process" would be very useful.Am I missing something?
In fact on SMP systems this would result in a very nice optimization:
pretty quickly *all* TSS's would be populated with that single task's bitmap, and it would persist even across migrations from CPU to CPU.
I'd love to get rid of the offset caching and bit scanning games as well
- it doesn't really help in a number of common scenarios and it complicates this non-trivial piece of code a *LOT* - and we probably don't really have the natural testing density of this code anymore to find any regressions quickly.
So intuitively I'd suggest we gravitate towards the simplest implementation, with a good caching optimization for the single-task case.
I.e. the model I'm suggesting is that if a task uses ioperm() or iopl() then it should have a bitmap from that point on until exit(), even if it's all zeroes or all ones. Most applications that are using those primitives really need it all the time and are using just a few ioports, so all the tracking doesn't help much anyway.
On a related note, another simplification would be that in principle we could also use just a single bitmap and emulate iopl() as an ioperm(all) or ioperm(none) calls. Yeah, it's not fully ABI compatible for mixed ioperm()/iopl() uses, but is that ABI actually being relied on in practice?
Thanks,
Ingo
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 5 lines · Ingo Molnar
I.e. the model I'm suggesting is that if a task uses
ioperm()oriopl()then it should have a bitmap from that point on untilexit(), even if it's all zeroes or all ones. Most applications that are using those primitives really need it all the time and are using just a few ioports, so all the tracking doesn't help much anyway.
I'd go even further, considering that any task having called ioperm() or iopl() once is granted access to all 64k ports for life: if the task was granted access to any port, it will be able to request access for any other port anyway. And we cannot claim that finely filtering accesses brings any particular reliability in my opinion, considering that it's generally possible to make the system really sick by starting to play with most I/O ports. So for me that becomes a matter of trusted vs not trusted task. Then we can simply have two pages of 0xFF to describe their I/O access bitmap.
Quoted 5 lines
On a related note, another simplification would be that in principle we could also use just a single bitmap and emulate
iopl()as an ioperm(all) or ioperm(none) calls. Yeah, it's not fully ABI compatible for mixed ioperm()/iopl() uses, but is that ABI actually being relied on in practice?
You mean you'd have a unified map for all tasks ? In this case I think it's simpler and equivalent to simply ignore the values in the calls and grant full perms to the 64k ports range after the calls were validated. I could be totally wrong and missing something obvious though.
Regards,
Willy
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 23 lines · Willy Tarreau
Quoted 5 lines · Ingo Molnar
I.e. the model I'm suggesting is that if a task uses
ioperm()oriopl()then it should have a bitmap from that point on untilexit(), even if it's all zeroes or all ones. Most applications that are using those primitives really need it all the time and are using just a few ioports, so all the tracking doesn't help much anyway.I'd go even further, considering that any task having called
ioperm()oriopl()once is granted access to all 64k ports for life: if the task was granted access to any port, it will be able to request access for any other port anyway. And we cannot claim that finely filtering accesses brings any particular reliability in my opinion, considering that it's generally possible to make the system really sick by starting to play with most I/O ports. So for me that becomes a matter of trusted vs not trusted task. Then we can simply have two pages of0xFFto describe their I/O access bitmap.Quoted 5 lines
On a related note, another simplification would be that in principle we could also use just a single bitmap and emulate
iopl()as an ioperm(all) or ioperm(none) calls. Yeah, it's not fully ABI compatible for mixed ioperm()/iopl() uses, but is that ABI actually being relied on in practice?
Quoted 5 lines
You mean you'd have a unified map for all tasks ? In this case I think it's simpler and equivalent to simply ignore the values in the calls and grant full perms to the 64k ports range after the calls were validated. I could be totally wrong and missing something obvious though.
Changing ioperm(single port, port range) to be ioperm(all) is going to break a bunch of test cases which actually check whether the permission is restricted to a single I/O port or the requested port range.
Thanks,
tglx
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 3 lines · Thomas Gleixner
Changing
ioperm(single port, port range)to be ioperm(all) is going to break a bunch of test cases which actually check whether the permission is restricted to a single I/O port or the requested port range.
But out of curiosity, are these solely test cases or things that real applications do ? We could imagine having a sysctl entry to indicate whether or not we want strict compatibility with older code in which case we'd take the slow path, or a modernized behavior using only the fast path. If we managed to deal with mmap_min_addr over time, I think it should be manageable to deal with the rare applications using ioperm().
Willy
[email protected]
reviewerRe: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Thomas GleixnerQuoted 42 lines · Thomas Gleixner
Quoted 2 lines · Willy Tarreau
Quoted 1 line · Ingo Molnar
I.e. the model I'm suggesting is that if a task uses
ioperm()or
iopl()Quoted 1 line
Quoted 1 line
then it should have a bitmap from that point on until
exit(), evenif
Quoted 3 lines
Quoted 3 lines
it's all zeroes or all ones. Most applications that are using those
primitives really need it all the time and are using just a few
ioports,
Quoted 4 lines
Quoted 1 line
so all the tracking doesn't help much anyway.
I'd go even further, considering that any task having called
ioperm()oriopl()once is granted access to all 64k ports for life: if thetask
Quoted 1 line
was granted access to any port, it will be able to request access for
any
Quoted 2 lines
other port anyway. And we cannot claim that finely filtering accesses brings any particular reliability in my opinion, considering that
it's
Quoted 2 lines
generally possible to make the system really sick by starting to play with most I/O ports. So for me that becomes a matter of trusted vs
not
Quoted 4 lines
trusted task. Then we can simply have two pages of
0xFFto describe their I/O access bitmap.Quoted 1 line
On a related note, another simplification would be that in
principle we
Quoted 1 line
Quoted 1 line
could also use just a single bitmap and emulate
iopl()as anioperm(all)
Quoted 1 line
Quoted 1 line
or ioperm(none) calls. Yeah, it's not fully ABI compatible for
mixed
Quoted 4 lines
Quoted 2 lines
ioperm()/iopl() uses, but is that ABI actually being relied on in practice?
You mean you'd have a unified map for all tasks ? In this case I
think
Quoted 4 lines
it's simpler and equivalent to simply ignore the values in the calls and grant full perms to the 64k ports range after the calls were validated. I could be totally wrong and missing something obvious though.
Quoted 8 lines
Changing
ioperm(single port, port range)to be ioperm(all) is going to break a bunch of test cases which actually check whether the permission is
restricted to a single I/O port or the requested port range.Thanks,
tglx
This seems very undesirable... as much as we might wish otherwise, the port bitmap is the equivalent to the MMU, and there are definitely users doing direct device I/O out there.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 11 lines · On Thu, Nov 07, 2019 at 02:19:19AM -0800, [email protected] wrote:
Quoted 7 lines
Changing
ioperm(single port, port range)to be ioperm(all) is going to break a bunch of test cases which actually check whether the permission is restricted to a single I/O port or the requested port range.Thanks,
tglx
This seems very undesirable... as much as we might wish otherwise, the port bitmap is the equivalent to the MMU, and there are definitely users doing direct device I/O out there.
Doing these, sure, but doing these while ranges are really checked ? I mean, the MMU grants you access to the pages you were assigned. Here with the I/O bitmap you just have to ask for access to port X and you get it. I could understand the benefit if we had EBUSY in return but that's not the case, you can actually request access to a port range another device driver or process is currently using, and mess up with what it does even by accident. I remember streaming 1-bit music in userland from the LED of my floppy drive in the late-90s, it used to cause some trouble to the floppy driver when using mtools in parallel :-)
Willy
[email protected]
reviewerRe: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 28 lines · Willy Tarreau
Quoted 1 line · On Thu, Nov 07, 2019 at 02:19:19AM -0800, [email protected] wrote:
Quoted 1 line
Changing
ioperm(single port, port range)to be ioperm(all) is goingto
Quoted 1 line
Quoted 1 line
break a bunch of test cases which actually check whether the
permission
Quoted 7 lines
Quoted 5 lines
is restricted to a single I/O port or the requested port range.
Thanks,
tglx
This seems very undesirable... as much as we might wish otherwise,
the port
Quoted 1 line
bitmap is the equivalent to the MMU, and there are definitely users
doing
Quoted 1 line
direct device I/O out there.
Doing these, sure, but doing these while ranges are really checked ? I mean, the MMU grants you access to the pages you were assigned. Here with the I/O bitmap you just have to ask for access to port X and you get it. I could understand the benefit if we had EBUSY in return but that's not the case, you can actually request access to a port range another device driver or process is currently using, and mess up with what it does even by accident. I remember streaming 1-bit music in userland from the LED of my floppy drive in the late-90s, it used to cause some trouble to the floppy driver when using mtools in parallel :-)
Quoted 1 line
Willy
You get access to the ports you are assigned, just like pages you are assigned... the rest is kernel policy, or, for that matter, privileged userspace (get permissions to the necessary ports, then drop privilege... the usual stuff.)
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 4 lines · On Thu, Nov 07, 2019 at 02:50:20AM -0800, [email protected] wrote:
You get access to the ports you are assigned, just like pages you are assigned... the rest is kernel policy, or, for that matter, privileged userspace (get permissions to the necessary ports, then drop privilege... the usual stuff.)
I agree, my point is that there's already no policy checking at the moment ports are assigned, hence a process having the permissions to request just port 0x70-0x71 to read the hwclock will also have permission to request access to the sensor chip a 0x2E and trigger a watchdog reset or stop the CPU fan. Thus any policy enforcement is solely done by the requesting process itself, assuming it doesn't simply use iopl() already, which grants everything.
This is why I'm really wondering if the real use cases that need all this stuff still exist at all in practice.
Willy
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 14 lines · Willy Tarreau
Quoted 4 lines · On Thu, Nov 07, 2019 at 02:50:20AM -0800, [email protected] wrote:
You get access to the ports you are assigned, just like pages you are assigned... the rest is kernel policy, or, for that matter, privileged userspace (get permissions to the necessary ports, then drop privilege... the usual stuff.)
I agree, my point is that there's already no policy checking at the moment ports are assigned, hence a process having the permissions to request just port 0x70-0x71 to read the hwclock will also have permission to request access to the sensor chip a
0x2Eand trigger a watchdog reset or stop the CPU fan. Thus any policy enforcement is solely done by the requesting process itself, assuming it doesn't simply useiopl()already, which grants everything.
Quoted 2 lines
This is why I'm really wondering if the real use cases that need all this stuff still exist at all in practice.
My memory is that the applications that didn't need fine grain access to ports would just use iopl.
Further a quick look shows that dosemu uses ioperm in a fine grained manner. From memory it would allow a handful of ports to allow directly accessing a device and depended on the rest of the port accesses to be disallowed so it could trap and emulate them.
So yes I do believe making ioperm ioperm(all) will break userspace.
Eric
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 5 lines · Eric W. Biederman
Further a quick look shows that dosemu uses ioperm in a fine grained manner. From memory it would allow a handful of ports to allow directly accessing a device and depended on the rest of the port accesses to be disallowed so it could trap and emulate them.
Yes. Making ioperm() some all-or-nothing thing would be horribly bad, and has no real advantages that I can see.
Linus
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Eric W. BiedermanQuoted 6 lines · Eric W. Biederman
Further a quick look shows that dosemu uses ioperm in a fine grained manner. From memory it would allow a handful of ports to allow directly accessing a device and depended on the rest of the port accesses to be disallowed so it could trap and emulate them.
So yes I do believe making ioperm ioperm(all) will break userspace.
OK, and I must confess I almost forgot about dosemu, that's a good point!
Thanks,
Willy
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Ingo MolnarQuoted 39 lines · Ingo Molnar
Quoted 26 lines · Linus Torvalds
Quoted 7 lines · Thomas Gleixner
Calculate both the position of the first zero bit and the last zero bit to limit the range which needs to be copied. This does not solve the problem when the previous tasked had only byte 0 cleared and the next one has only byte 65535 cleared, but trying to solve that would be too complex and heavyweight for the context switch path. As the
ioperm()usage is very rare the case which is optimized is the single task/process which usesioperm().Hmm.
I may read this patch wrong, but from what I can tell, if we really have just one process with an io bitmap, we're doing unnecessary copies.
If we really have just one process that has an iobitmap, I think we could just keep the bitmap of that process entirely unchanged. Then, when we switch away from it, we set the
io_bitmap_baseto an invalid base outside the TSS segment, and when we switch back, we set it back to the valid one. No actual bitmap copies at all.So I think that rather than the "begin/end offset" games, we should perhaps have a "what was the last process that used the IO bitmap for this TSS" pointer (and, I think, some sequence counter, so that when the process updates its bitmap, it invalidates that case)?
Quoted 5 lines
Of course, you can do *nboth*, but if we really think that the common
case is "one special process", then I think the begin/end offset is useless, but a "last bitmap process" would be very useful.Am I missing something?
In fact on SMP systems this would result in a very nice optimization:
pretty quickly *all* TSS's would be populated with that single task's bitmap, and it would persist even across migrations from CPU to CPU.
Quoted 5 lines
I'd love to get rid of the offset caching and bit scanning games as well
- it doesn't really help in a number of common scenarios and it complicates this non-trivial piece of code a *LOT* - and we probably don't really have the natural testing density of this code anymore to find any regressions quickly.
I think we should not over-optimize this. I am all for penalizing ioperm() and iopl() users as much as is convenient for us. There is simply no legitimate use case. Sorry, DPDK, but "virtio-legacy sucks, let's optimize the crap out of something that is slow anyway and use iopl()" is not a good excuse. Just use the %*!7 syscall to write to /sys/.../resource0 and suck up the probably negligible performance hit. And tell your customers to upgrade their hypervisors. And quite kvetching about performance of the control place on an old software-emulated NIC while you're at it.
For the TLB case, it's worth tracking who last used which ASID and whether it's still up to date, since *everyone* uses the MMU. For ioperm, I don't really believe this is worth it.
Quoted 4 lines
So intuitively I'd suggest we gravitate towards the simplest implementation, with a good caching optimization for the single-task case.
I agree with the first bit, but caching on an SMP system is necessarily subtle. Some kind of invalidation is needed.
Quoted 7 lines
I.e. the model I'm suggesting is that if a task uses
ioperm()oriopl()then it should have a bitmap from that point on untilexit(), even if it's all zeroes or all ones. Most applications that are using those primitives really need it all the time and are using just a few ioports, so all the tracking doesn't help much anyway.
Quoted 6 lines
On a related note, another simplification would be that in principle we could also use just a single bitmap and emulate
iopl()as an ioperm(all) or ioperm(none) calls. Yeah, it's not fully ABI compatible for mixed ioperm()/iopl() uses, but is that ABI actually being relied on in practice?
Let's please keep the ABI. Or rather, let's attempt to eventually remove the ABI, but let's not change it in the mean time please.
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Thomas GleixnerQuoted 20 lines · Thomas Gleixner
arch/x86/include/asm/processor.h +7
-6@@ -365,19 +365,19 @@ struct tss_struct {struct x86_hw_tss x86_tss;/* -* Store the dirty size of the last io bitmap offender. The next -* one will have to do the cleanup as the switch out to a non -* io bitmap user will just set x86_tss.io_bitmap_base to a value -* outside of the TSS limit. So for sane tasks there is no need -* to actually touch the io_bitmap at all. +* Store the dirty byte range of the last io bitmap offender. The +* next one will have to do the cleanup because the switch out to a +* non I/O bitmap user will just set x86_tss.io_bitmap_base to a +* value outside of the TSS limit to not penalize tasks which do +* not use the I/O bitmap at all. */ -unsigned int io_bitmap_prev_max; +unsigned int io_zerobits_start; +unsigned int io_zerobits_end;/*
arch/x86/include/asm/processor.h struct tss_struct { · line 383
Quoted 10 lines
arch/x86/include/asm/processor.h +3
-4-* The extra 1 is there because the CPU will access an -* additional byte beyond the end of the IO permission -* bitmap. The extra byte must be all 1 bits, and must -* be within the limit. +* The extra 1 is there because the CPU will access an additional +* byte beyond the end of the I/O permission bitmap. The extra byte +* must have all bits set and must be within the TSS limit. */ unsigned long io_bitmap[IO_BITMAP_LONGS +1 ]; }__aligned(PAGE_SIZE) ;
Note that on 32-bit kernels this blows up our CPU area calculations:
./include/linux/compiler.h:350:38: error: call to ‘__compiletime_assert_181’ declared with attribute error: BUILD_BUG_ON failed: CPU_ENTRY_AREA_PAGES * PAGE_SIZE < CPU_ENTRY_AREA_MAP_SIZE ./include/linux/compiler.h:331:4: note: in definition of macro ‘__compiletime_assert’./include/linux/compiler.h:350:2: note: in expansion of macro ‘_compiletime_assert’./include/linux/build_bug.h:39:37: note: in expansion of macro ‘compiletime_assert’./include/linux/build_bug.h:50:2: note: in expansion of macro ‘BUILD_BUG_ON_MSG’arch/x86/mm/cpu_entry_area.c:181:2: note: in expansion of macro ‘BUILD_BUG_ON’
make[2]: *** [scripts/Makefile.build:265: arch/x86/mm/cpu_entry_area.o] Error 1
Thanks,
Ingo
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 15 lines · Ingo Molnar
arch/x86/include/asm/processor.h struct tss_struct { · line 383
Quoted 11 lines · Thomas Gleixner
arch/x86/include/asm/processor.h +3
-4/* -* The extra 1 is there because the CPU will access an -* additional byte beyond the end of the IO permission -* bitmap. The extra byte must be all 1 bits, and must -* be within the limit. +* The extra 1 is there because the CPU will access an additional +* byte beyond the end of the I/O permission bitmap. The extra byte +* must have all bits set and must be within the TSS limit. */ unsigned long io_bitmap[IO_BITMAP_LONGS +1 ]; }__aligned(PAGE_SIZE) ;Note that on 32-bit kernels this blows up our CPU area calculations:
Quoted 7 lines
./include/linux/compiler.h:350:38: error: call to ‘__compiletime_assert_181’ declared with attribute error:BUILD_BUG_ONfailed:CPU_ENTRY_AREA_PAGES*PAGE_SIZE<CPU_ENTRY_AREA_MAP_SIZE./include/linux/compiler.h:331:4: note: in definition of macro ‘__compiletime_assert’./include/linux/compiler.h:350:2: note: in expansion of macro ‘_compiletime_assert’./include/linux/build_bug.h:39:37: note: in expansion of macro ‘compiletime_assert’./include/linux/build_bug.h:50:2: note: in expansion of macro ‘BUILD_BUG_ON_MSG’arch/x86/mm/cpu_entry_area.c:181:2: note: in expansion of macro ‘BUILD_BUG_ON’
make[2]: *** [scripts/Makefile.build:265:arch/x86/mm/cpu_entry_area.o] Error 1
Duh. /me goes investigate.
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Thomas GleixnerQuoted 13 lines · Thomas Gleixner
arch/x86/kernel/ioport.c +13
-0+/* Update the bitmap */ +if (turn_on) { +bitmap_clear (bitmap, from, num); + }else { +bitmap_set (bitmap, from, num); + } + +/* Get the new range */ + first =find_first_zero_bit (bitmap,IO_BITMAP_BITS ); + +for (last = next = first; next <IO_BITMAP_BITS ; last = next) { +/* Find the next set bit and update last */ + next =find_next_bit (bitmap,IO_BITMAP_BITS , last);
arch/x86/kernel/ioport.c long ksys_ioperm(unsigned long from, uns · line 83
Quoted 10 lines
arch/x86/kernel/ioport.c +10
-0+ last = next -1 ; +if (next ==IO_BITMAP_BITS ) +break ; +/* Find the next zero bit and continue searching */ + next =find_next_zero_bit (bitmap,IO_BITMAP_BITS , next); + } + +/* Calculate the byte boundaries for the updated region */ + copy_start = from /8 ; + copy_len = (round_up (from + num,8 ) /8 ) - copy_start;
This might seem like a small detail, but since we do the range tracking and copying at byte granularity anyway, why not do the zero range search at byte granularity as well?
I bet it's faster and simpler as well than the bit-searching.
We could also change over the bitmap to a char or u8 based array and lose all the sizeof(long) indexing headaches, resulting type casts, for anything but the actual bitmap_set/clear() calls, etc.?
I.e. now that most of the logic is byte granular, the basic data structure might as well reflect that?
Quoted 7 lines
arch/x86/kernel/ioport.c +7
-0+/* +* Update the per thread storage and the TSS bitmap. This must be +* done with preemption disabled to prevent racing against a +* context switch. +*/ +preempt_disable (); + tss =this_cpu_ptr (&cpu_tss_rw);
arch/x86/kernel/ioport.c long ksys_ioperm(unsigned long from, uns · line 101
Quoted 10 lines
arch/x86/kernel/ioport.c +9
-0+if (!t->io_bitmap_ptr) { +unsigned int tss_start = tss->io_zerobits_start; +/* +* If the task did not use the I/O bitmap yet then the +* perhaps stale content in the TSS needs to be taken into +* account. If tss start is out of bounds the TSS storage +* does not contain a zero bit and it's sufficient just to +* copy the new range over. +*/
s/tss/TSS
Quoted 14 lines
arch/x86/kernel/ioport.c +14
-0+if (tss_start <IO_BITMAP_BYTES ) { +unsigned int tss_end = tss->io_zerobits_end; +unsigned int copy_end = copy_start + copy_len; + + copy_start =min (tss_start, copy_start); + copy_len =max (tss_end, copy_end) - copy_start; + } + } + +/* Copy the changed range over to the TSS bitmap */ + dst = (char *)tss->io_bitmap; + src = (char *)bitmap; +memcpy (dst + copy_start, src + copy_start, copy_len); +
arch/x86/kernel/ioport.c long ksys_ioperm(unsigned long from, uns · line 125
Quoted 10 lines
arch/x86/kernel/ioport.c +10
-0+if (first >=IO_BITMAP_BITS ) { +/* +* If the resulting bitmap has all permissions dropped, clear +* TIF_IO_BITMAP and set the IO bitmap offset in the TSS to +* invalid. Deallocate both the new and the thread's bitmap. +*/ +clear_thread_flag (TIF_IO_BITMAP ); + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + tofree = bitmap; + bitmap =NULL ;
BTW., wouldn't it be simpler to just say that if a thread uses IO ops even once, it gets a bitmap and that's it? I.e. we could further simplify this seldom used piece of code.
arch/x86/kernel/ioport.c long ksys_ioperm(unsigned long from, uns · line 135
Quoted 10 lines
arch/x86/kernel/ioport.c +6
-0+ }else {/* +* I/O bitmap contains zero bits. Set TIF_IO_BITMAP, make +* the bitmap offset valid and make sure that the TSS limit +* is correct. It might have been wreckaged by a VMEXiT. */ +set_thread_flag (TIF_IO_BITMAP ); + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;refresh_tss_limit (); }
I'm wondering, shouldn't we call refresh_tss_limit() in both branches, or is a VMEXIT-wreckaged TSS limit harmless if we otherwise have io_bitmap_base set to IO_BITMAP_OFFSET_INVALID?
Quoted 11 lines
arch/x86/kernel/ioport.c +7
-0/* +* Update the range in the thread and the TSS * +* Get the byte position of the first zero bit and calculate +* the length of the range in which zero bits exist. */ + start = first /8 ; + end = first <IO_BITMAP_BITS ?round_up (last,8 ) /8 :0 ; + t->io_zerobits_start = tss->io_zerobits_start = start; + t->io_zerobits_end = tss->io_zerobits_end = end;
arch/x86/kernel/ioport.c long ksys_ioperm(unsigned long from, uns · line 157
Quoted 10 lines
arch/x86/kernel/ioport.c +5
-0/* +* Finally exchange the bitmap pointer in the thread. */ + bitmap =xchg (&t->io_bitmap_ptr, bitmap); +preempt_enable (); +kfree (bitmap); +kfree (tofree);return 0 ;
Thanks,
Ingo
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 6 lines · Ingo Molnar
* Thomas Gleixner <[email protected]> wrote:
This might seem like a small detail, but since we do the range tracking and copying at byte granularity anyway, why not do the zero range search at byte granularity as well?I bet it's faster and simpler as well than the bit-searching.
Not necessarily. The bitmap search uses unsigned long internally at least to the point where it finds a zero bit.
But probably we should just ditch that optimization and rather have the sequence number to figure out whether something needs to be copied at all.
Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/ioport.c +6
-0+if (first >=IO_BITMAP_BITS ) { +/* +* If the resulting bitmap has all permissions dropped, clear +* TIF_IO_BITMAP and set the IO bitmap offset in the TSS to +* invalid. Deallocate both the new and the thread's bitmap. +*/
Quoted 8 lines
arch/x86/kernel/ioport.c long ksys_ioperm(unsigned long from, uns · line 125
Quoted 4 lines
arch/x86/kernel/ioport.c +4
-0+clear_thread_flag (TIF_IO_BITMAP ); + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + tofree = bitmap; + bitmap =NULL ;BTW., wouldn't it be simpler to just say that if a thread uses IO ops even once, it gets a bitmap and that's it? I.e. we could further simplify this seldom used piece of code.
Maybe.
Quoted 6 lines
Quoted 6 lines
arch/x86/kernel/ioport.c +4
-0+ }else {/* +* I/O bitmap contains zero bits. Set TIF_IO_BITMAP, make +* the bitmap offset valid and make sure that the TSS limit +* is correct. It might have been wreckaged by a VMEXiT. */
Quoted 8 lines
arch/x86/kernel/ioport.c long ksys_ioperm(unsigned long from, uns · line 135
Quoted 4 lines
arch/x86/kernel/ioport.c +2
-0+set_thread_flag (TIF_IO_BITMAP ); + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;refresh_tss_limit (); }I'm wondering, shouldn't we call
refresh_tss_limit()in both branches, or is a VMEXIT-wreckaged TSS limit harmless if we otherwise haveio_bitmap_baseset toIO_BITMAP_OFFSET_INVALID?
Yes. because the VMEXIT wreckage restricts TSS limit to 0x67 which is the end of the hardware tss struct. As the invalid offset points in any case outside the TSS limit it does not matter. It always #GP's when an I/O access happens in user space.
Thanks,
tglx
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Thomas GleixnerQuoted 9 lines · Thomas Gleixner
Sane
ioperm()users only set the few bits in the I/O space which they need to access. But the update logic of the TSS I/O bitmap copies always from byte 0 to the last byte in the tasks bitmap which contains a zero permission bit.That means that for access only to port 65335 the full 8K bitmap has to be copied even if all the bytes in the TSS I/O bitmap are already filled with
0xff.
Quoted 6 lines
Calculate both the position of the first zero bit and the last zero bit to limit the range which needs to be copied. This does not solve the problem when the previous tasked had only byte 0 cleared and the next one has only byte 65535 cleared, but trying to solve that would be too complex and heavyweight for the context switch path. As the
ioperm()usage is very rare the case which is optimized is the single task/process which usesioperm().
Here is a different idea: We already map the TSS virtually in cpu_entry_area. Why not page-align the IO bitmap and remap it to the task's bitmap on task switch? That would avoid all copying on task switch.
--
Brian Gerst
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 5 lines · Brian Gerst
Here is a different idea: We already map the TSS virtually in
cpu_entry_area. Why not page-align the IO bitmap and remap it to the task's bitmap on task switch? That would avoid all copying on task switch.
We map the tss _once_, statically, percpu, without ever changing it, and then we just (potentially) change a couple of fields in it on process switch.
Your idea isn't horrible, but it would involve a TLB flush for the page when the io bitmap changes. Which is almost certainly more expensive than just copying the bitmap intelligently.
Particularly since I do think that the copy can basically be done effectively never, assuming there really aren't multiple concurrent users of ioperm() (and iopl).
Linus
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 12 lines · Linus Torvalds
Quoted 5 lines · Brian Gerst
Here is a different idea: We already map the TSS virtually in
cpu_entry_area. Why not page-align the IO bitmap and remap it to the task's bitmap on task switch? That would avoid all copying on task switch.We map the tss _once_, statically, percpu, without ever changing it, and then we just (potentially) change a couple of fields in it on process switch.
Quoted 7 lines
Your idea isn't horrible, but it would involve a TLB flush for the page when the io bitmap changes. Which is almost certainly more expensive than just copying the bitmap intelligently.
Particularly since I do think that the copy can basically be done effectively never, assuming there really aren't multiple concurrent users of
ioperm()(and iopl).
There wouldn't have to be a flush on every task switch. If we make it so that tasks that don't use a bitmap just unmap the pages in the cpu_entry_area and set tss.io_bitmap_base to outside the segment limit, we would only have to flush when switching from a task using the bitmap (because the next task uses a different bitmap or we are unmapping it). If the previous task doesn't have a bitmap the pages in cpu_entry_area were unmapped and can't be in the TLB, so no flush is needed.
Going a step further, we could track which task is mapped to the current cpu like proposed above, and only flush when a different task needs the IO bitmap, or when the bitmap is being freed on task exit.
--
Brian Gerst
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 22 lines · Brian Gerst
Quoted 12 lines · Linus Torvalds
Quoted 5 lines · Brian Gerst
Here is a different idea: We already map the TSS virtually in
cpu_entry_area. Why not page-align the IO bitmap and remap it to the task's bitmap on task switch? That would avoid all copying on task switch.We map the tss _once_, statically, percpu, without ever changing it, and then we just (potentially) change a couple of fields in it on process switch.
Quoted 7 lines
Your idea isn't horrible, but it would involve a TLB flush for the page when the io bitmap changes. Which is almost certainly more expensive than just copying the bitmap intelligently.
Particularly since I do think that the copy can basically be done effectively never, assuming there really aren't multiple concurrent users of
ioperm()(and iopl).
Quoted 8 lines
There wouldn't have to be a flush on every task switch. If we make it so that tasks that don't use a bitmap just unmap the pages in the
cpu_entry_areaand set tss.io_bitmap_base to outside the segment limit, we would only have to flush when switching from a task using the bitmap (because the next task uses a different bitmap or we are unmapping it). If the previous task doesn't have a bitmap the pages incpu_entry_areawere unmapped and can't be in the TLB, so no flush is needed.
Funny. I was just debating exactly this with Peter Ziljstra over IRC :)
Quoted 3 lines
Going a step further, we could track which task is mapped to the current cpu like proposed above, and only flush when a different task needs the IO bitmap, or when the bitmap is being freed on task exit.
Yes.
But, we really should check what aside of DoSemu is using this still. None of my machines I checked have a single instance of ioperm()/iopl() usage.
So the real question is whether it's worth the trouble or if we are just better off to copy if there is an actual user and the sequence count of the bitmap is different than the one which was active last time.
Thanks,
tglx
[email protected]
reviewerRe: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 56 lines · Thomas Gleixner
Quoted 4 lines · Brian Gerst
Quoted 2 lines · Linus Torvalds
On Thu, Nov 7, 2019 at 11:24 AM Brian Gerst <[email protected]>
Quoted 3 lines · wrote:
Quoted 3 lines
Quoted 3 lines
Here is a different idea: We already map the TSS virtually in
cpu_entry_area. Why not page-align the IO bitmap and remap it tothe
Quoted 1 line
Quoted 1 line
Quoted 1 line
task's bitmap on task switch? That would avoid all copying on
task
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
switch.
We map the tss _once_, statically, percpu, without ever changing
it,
Quoted 7 lines
Quoted 7 lines
and then we just (potentially) change a couple of fields in it on process switch.
Your idea isn't horrible, but it would involve a TLB flush for the page when the io bitmap changes. Which is almost certainly more expensive than just copying the bitmap intelligently.
Quoted 5 lines
Quoted 3 lines
Particularly since I do think that the copy can basically be done effectively never, assuming there really aren't multiple concurrent users of
ioperm()(and iopl).There wouldn't have to be a flush on every task switch. If we make
it
Quoted 7 lines
so that tasks that don't use a bitmap just unmap the pages in the
cpu_entry_areaand set tss.io_bitmap_base to outside the segment limit, we would only have to flush when switching from a task using the bitmap (because the next task uses a different bitmap or we are unmapping it). If the previous task doesn't have a bitmap the pages incpu_entry_areawere unmapped and can't be in the TLB, so no flush is needed.Funny. I was just debating exactly this with Peter Ziljstra over IRC :)
Quoted 3 lines
Going a step further, we could track which task is mapped to the current cpu like proposed above, and only flush when a different task needs the IO bitmap, or when the bitmap is being freed on task exit.
Yes.
But, we really should check what aside of DoSemu is using this still. None
of my machines I checked have a single instance of ioperm()/iopl() usage.So the real question is whether it's worth the trouble or if we are just
better off to copy if there is an actual user and the sequence count of the
bitmap is different than the one which was active last time.
Quoted 3 lines
Thanks,
tglx
I have written suffer using this, because of far better real time performance. I just want to punch a hole (just like mmapping an MMIO device.)
I do agree that let's not optimize for the rare case.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Brian GerstQuoted 2 lines · Brian Gerst
There wouldn't have to be a flush on every task switch.
No. But we'd have to flush on any switch that currently does that memcpy.
And my point is that a tlb flush (even the single-page case) is likely more expensive than the memcpy.
Quoted 3 lines
Going a step further, we could track which task is mapped to the current cpu like proposed above, and only flush when a different task needs the IO bitmap, or when the bitmap is being freed on task exit.
Well, that's exactly my "track the last task" optimization for copying the thing.
IOW, it's the same optimization as avoiding the memcpy.
Which I think is likely very effective, but also makes it fairly pointless to then try to be clever..
So the basic issue remains that playing VM games has almost universally been slower and more complex than simply not playing VM games. TLB flushes - even invlpg - tends to be pretty slow.
Of course, we probably end up invalidating the TLB's anyway, so maybe in this case we don't care. The ioperm bitmap is _technically_ per-thread, though, so it should be flushed even if the VM isn't flushed...
Linus
H. Peter Anvin
reviewerRe: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 21 lines · Linus Torvalds
Quoted 2 lines · Brian Gerst
There wouldn't have to be a flush on every task switch.
No. But we'd have to flush on any switch that currently does that memcpy.
And my point is that a tlb flush (even the single-page case) is likely more expensive than the memcpy.
Quoted 3 lines
Going a step further, we could track which task is mapped to the current cpu like proposed above, and only flush when a different task needs the IO bitmap, or when the bitmap is being freed on task exit.
Well, that's exactly my "track the last task" optimization for copying the thing.
IOW, it's the same optimization as avoiding the memcpy.
Which I think is likely very effective, but also makes it fairly pointless to then try to be clever..
Quoted 9 lines
So the basic issue remains that playing VM games has almost universally been slower and more complex than simply not playing VM games. TLB flushes - even invlpg - tends to be pretty slow.
Of course, we probably end up invalidating the TLB's anyway, so maybe in this case we don't care. The ioperm bitmap is _technically_ per-thread, though, so it should be flushed even if the VM isn't flushed...
One option, probably a lot saner (if we care at all, after all, copying 8K really isn't that much, but it might have some impact on real-time processes, which is one of the rather few use cases for direct I/O) would be to keep the bitmask in a pre-formatted TSS (ioperm being per thread, so no concerns about the TSS being in use on another processor), and copy the TSS fields (88 bytes) over if and only if the thread has been migrated to a different CPU, then switch the TSS rather than switching For the common case (no ioperms) we use the standard per-cpu TSS.
That being said, I don't actually know that copying 88 bytes + LTR is any cheaper than copying 8K.
-hpa
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
Quoted 42 lines · H. Peter Anvin
Quoted 21 lines · Linus Torvalds
Quoted 2 lines · Brian Gerst
There wouldn't have to be a flush on every task switch.
No. But we'd have to flush on any switch that currently does that memcpy.
And my point is that a tlb flush (even the single-page case) is likely more expensive than the memcpy.
Quoted 3 lines
Going a step further, we could track which task is mapped to the current cpu like proposed above, and only flush when a different task needs the IO bitmap, or when the bitmap is being freed on task exit.
Well, that's exactly my "track the last task" optimization for copying the thing.
IOW, it's the same optimization as avoiding the memcpy.
Which I think is likely very effective, but also makes it fairly pointless to then try to be clever..
Quoted 9 lines
So the basic issue remains that playing VM games has almost universally been slower and more complex than simply not playing VM games. TLB flushes - even invlpg - tends to be pretty slow.
Of course, we probably end up invalidating the TLB's anyway, so maybe in this case we don't care. The ioperm bitmap is _technically_ per-thread, though, so it should be flushed even if the VM isn't flushed...
One option, probably a lot saner (if we care at all, after all, copying 8K really isn't that much, but it might have some impact on real-time processes, which is one of the rather few use cases for direct I/O) would be to keep the bitmask in a pre-formatted TSS (ioperm being per thread, so no concerns about the TSS being in use on another processor), and copy the TSS fields (88 bytes) over if and only if the thread has been migrated to a different CPU, then switch the TSS rather than switching For the common case (no ioperms) we use the standard per-cpu TSS.
Quoted 2 lines
That being said, I don't actually know that copying 88 bytes + LTR is any cheaper than copying 8K.
I don't think that can work. The TSS has to be at a fixed address in the cpu_entry_area so that it is visible when running in usermode (thanks to Meltdown).
--
Brian Gerst
Re: [patch 5/9] x86/ioport: Reduce ioperm impact for sane usage further
In reply to Linus TorvaldsQuoted 16 lines · Linus Torvalds
Quoted 2 lines · Brian Gerst
There wouldn't have to be a flush on every task switch.
No. But we'd have to flush on any switch that currently does that memcpy.
And my point is that a tlb flush (even the single-page case) is likely more expensive than the memcpy.
Quoted 3 lines
Going a step further, we could track which task is mapped to the current cpu like proposed above, and only flush when a different task needs the IO bitmap, or when the bitmap is being freed on task exit.
Well, that's exactly my "track the last task" optimization for copying the thing.
Quoted 9 lines
IOW, it's the same optimization as avoiding the memcpy.
Which I think is likely very effective, but also makes it fairly pointless to then try to be clever..
So the basic issue remains that playing VM games has almost universally been slower and more complex than simply not playing VM games. TLB flushes - even invlpg - tends to be pretty slow.
With my TLB-handling-writing-and-reviewing code on, NAK to any VM games here.
Honestly, I almost think we should unconditionally copy the whole 8K for the case where the ioperm() syscall has been used (i.e. not emulating iopl()). The benefit simply does not justify the risk of getting it wrong. I'm okay, but barely, with optimizing the end of the copied range. Optimizing the start of the copied range is pushing it. Playing MMU tricks and getting all the per-task-ioperm and invalidation right is way beyond reasonable.
Even the time spent discussing how to optimize a case that has literally one known user that none of us can bring ourselves to care about much seems wasteful.
PATCH 6/9 x86/iopl: Fixup misleading comment
In reply to Thomas GleixnerThe comment for the sys_iopl() implementation is outdated and actively misleading in some parts. Fix it up.
--- arch/x86/kernel/ioport.c | 35 +++++++++++++++++++++++++++-------- 1 file changed, 27 insertions(+), 8 deletions(-)
arch/x86/kernel/ioport.c +27 -8
@@ -22,7 +22,7 @@#include <asm/desc.h> /* -* this changes the io permissions bitmap in the current task. +* This changes the io permissions bitmap in the current task. */ long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) {
@@ -163,14 +163,24 @@ SYSCALL_DEFINE3(ioperm, unsigned long, f }/* -* sys_iopl has to be used when you want to access the IO ports -* beyond the 0x3ff range: to get the full 65536 ports bitmapped -* you'd need 8kB of bitmaps/process, which is a bit excessive. +* The sys_iopl functionality depends on the level argument, which if +* granted for the task is used by the CPU to check I/O instruction and +* CLI/STI against the current priviledge level (CPL). If CPL is less than +* or equal the tasks IOPL level the instructions take effect. If not a #GP +* is raised. The default IOPL is 0, i.e. no permissions. * -* Here we just change the flags value on the stack: we allow -* only the super-user to do it. This depends on the stack-layout -* on system-call entry - see also fork() and the signal handling -* code. +* Setting IOPL to level 0-2 is disabling the userspace access. Only level +* 3 enables it. If set it allows the user space thread: +* +* - Unrestricted access to all 65535 I/O ports +* - The usage of CLI/STI instructions +* +* The advantage over ioperm is that the context switch does not require to +* update the I/O bitmap which is especially true when a large number of +* ports is accessed. But the allowance of CLI/STI in userspace is +* considered a major problem. +* +* IOPL is strictly per thread and inherited on fork. */ SYSCALL_DEFINE1(iopl, unsigned int, level) {
@@ -191,9 +201,18 @@ SYSCALL_DEFINE1(iopl, unsigned int, leve security_locked_down(LOCKDOWN_IOPORT))return -EPERM ; } +/* +* Change the flags value on the return stack, which has been set +* up on system-call entry. See also the fork and signal handling +* code how this is handled. +*/ regs->flags = (regs->flags & ~X86_EFLAGS_IOPL ) | (level <<X86_EFLAGS_IOPL_BIT ); +/* Store the new level in the thread struct */ t->iopl = level <<X86_EFLAGS_IOPL_BIT ; +/* +* X86_32 switches immediately and XEN handles it via emulation. +*/ set_iopl_mask (t->iopl);return 0 ;
PATCH 7/9 x86/iopl: Restrict iopl() permission scope
In reply to Thomas GleixnerThe access to the full I/O port range can be also provided by the TSS I/O bitmap, but that would require to copy 8k of data on scheduling in the task. As shown with the sched out optimization TSS.io_bitmap_base can be used to switch the incoming task to a preallocated I/O bitmap which has all bits zero, i.e. allows access to all I/O ports.
Implementing this allows to provide an iopl() emulation mode which restricts the IOPL level 3 permissions to I/O port access but removes the STI/CLI permission which is coming with the hardware IOPL mechansim.
Provide a config option to switch IOPL to emulation mode, make it the default and while at it also provide an option to disable IOPL completely.
--- arch/x86/Kconfig | 32 ++++++++++++++++ arch/x86/include/asm/processor.h | 24 +++++++++--- arch/x86/kernel/cpu/common.c | 4 +- arch/x86/kernel/ioport.c | 75 ++++++++++++++++++++++++++++++--------- arch/x86/kernel/process.c | 15 +++++-- 5 files changed, 123 insertions(+), 27 deletions(-)
arch/x86/Kconfig +32 -0
@@ -1254,6 +1254,38 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. +choice + prompt "IOPL" + default X86_IOPL_EMULATION + +config X86_IOPL_EMULATION + bool "IOPL Emulation" + ---help--- + Legacy IOPL support is an overbroad mechanism which allows user + space aside of accessing all 65536 I/O ports also to disable + interrupts. To gain this access the caller needs CAP_SYS_RAWIO + capabilities and permission from eventually active security + modules. + + The emulation restricts the functionality of the syscall to + only allowing the full range I/O port access, but prevents the + ability to disable interrupts from user space. + +config X86_IOPL_LEGACY + bool "IOPL Legacy" + ---help--- + Allow the full IOPL permissions, i.e. user space access to all + 65536 I/O ports and also the ability to disable interrupts, which + is overbroad and can result in system lockups. + +config X86_IOPL_NONE + bool "IOPL None" + ---help--- + Disable the IOPL permission syscall. That's the safest option as + no sane application should depend on this functionality. + +endchoice + config TOSHIBA tristate "Toshiba Laptop support" depends on X86_32
arch/x86/include/asm/processor.h +19 -5
@@ -331,8 +331,12 @@ struct x86_hw_tss {#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) -#define IO_BITMAP_OFFSET_VALID \ - (offsetof(struct tss_struct, io_bitmap) - \ +#define IO_BITMAP_OFFSET_VALID_MAP \ + (offsetof(struct tss_struct, io_bitmap_map) - \ + offsetof(struct tss_struct, x86_tss)) + +#define IO_BITMAP_OFFSET_VALID_ALL \ + (offsetof(struct tss_struct, io_bitmap_all) - \ offsetof(struct tss_struct, x86_tss))/*
@@ -343,7 +347,7 @@ struct x86_hw_tss {* last valid byte */ #define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + (IO_BITMAP_OFFSET_VALID_ALL + IO_BITMAP_BYTES + sizeof(unsigned long) - 1)/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ #define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)
@@ -379,7 +383,8 @@ struct tss_struct {* byte beyond the end of the I/O permission bitmap. The extra byte * must have all bits set and must be within the TSS limit. */ -unsigned long io_bitmap[IO_BITMAP_LONGS +1 ]; +unsigned long io_bitmap_map[IO_BITMAP_LONGS +1 ]; +unsigned long io_bitmap_all[IO_BITMAP_LONGS +1 ]; }__aligned(PAGE_SIZE) ; DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw);
@@ -495,7 +500,6 @@ struct thread_struct {#endif /* IO permissions: */ unsigned long *io_bitmap_ptr; -unsigned long iopl;/* * The byte range in the I/O permission bitmap which contains zero * bits.
@@ -503,6 +507,13 @@ struct thread_struct {unsigned int io_zerobits_start;unsigned int io_zerobits_end; +/* +* IOPL. Priviledge level dependent I/O permission which includes +* user space CLI/STI when granted. +*/ +unsigned long iopl; +unsigned long iopl_emul; +mm_segment_t addr_limit; unsigned int sig_on_uaccess_err:1;
@@ -524,6 +535,9 @@ static inline void arch_thread_struct_wh *size = fpu_kernel_xstate_size; } +extern void tss_update_io_bitmap (struct tss_struct *tss, +struct thread_struct *thread); +/* * Thread-synchronous status. *
arch/x86/kernel/cpu/common.c +3 -1
@@ -1866,7 +1866,9 @@ void cpu_init(void) tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; tss->io_zerobits_start =IO_BITMAP_BYTES ; tss->io_zerobits_end =0 ; -memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap)); +memset (tss->io_bitmap_map,0xff ,sizeof (tss->io_bitmap_map)); +/* Invalidate the extra tail entry of the allow all I/O bitmap */ + tss->io_bitmap_all[IO_BITMAP_LONGS ] = ~0UL ;set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();
arch/x86/kernel/ioport.c +58 -17
108 lines not on this page · raw message
arch/x86/kernel/process.c +11 -4
@@ -357,8 +357,7 @@ void arch_setup_new_exec(void) } } -static void tss_update_io_bitmap (struct tss_struct *tss, -struct thread_struct *thread) +void tss_update_io_bitmap (struct tss_struct *tss,struct thread_struct *thread) {unsigned int start, len;char *src, *dst;
@@ -387,9 +386,17 @@ static inline void switch_to_bitmap(stru unsigned long tifp, unsigned long tifn) {struct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw); +u16 *base = &tss->x86_tss.io_bitmap_base;if (tifn & _TIF_IO_BITMAP) { -tss_update_io_bitmap (tss, next); +/* +* IF IOPL emulation is enabled and the emulated I/O +* priviledge level is 3, switch to the 'grant all' bitmap. +*/ +if (IS_ENABLED (CONFIG_IOPL_EMULATION ) && next->iopl_emul ==3 ) + *base =IO_BITMAP_OFFSET_VALID_ALL ; +else +tss_update_io_bitmap (tss, next);/* * Make sure that the TSS limit is covering the io bitmap. * It might have been cut down by a VMEXIT to 0x67 which
@@ -405,7 +412,7 @@ static inline void switch_to_bitmap(stru* by moving it outside the TSS limit so any subsequent I/O * access from user space will trigger a #GP. */ - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + *base =IO_BITMAP_OFFSET_INVALID ; } }
Re: [patch 7/9] x86/iopl: Restrict iopl() permission scope
On Wed, Nov 06, 2019 at 08:35:06PM +0100, Thomas Gleixner wrote:
Something like:
arch/x86/include/asm/processor.h struct tss_struct { · line 382
Quoted 5 lines
arch/x86/include/asm/processor.h +0
-1@@ -379,7 +383,8 @@ struct tss_struct {* byte beyond the end of the I/O permission bitmap. The extra byte * must have all bits set and must be within the TSS limit. */ -unsigned long io_bitmap[IO_BITMAP_LONGS +1 ];
#ifndef X86_IOPL_NONE
arch/x86/include/asm/processor.h struct tss_struct { · line 386
Quoted 1 line
arch/x86/include/asm/processor.h +1
-0+unsigned long io_bitmap_map[IO_BITMAP_LONGS +1 ];
#ifdef X86_IOPL_EMLATION
arch/x86/include/asm/processor.h struct tss_struct { · line 387
Quoted 1 line
arch/x86/include/asm/processor.h +1
-0+unsigned long io_bitmap_all[IO_BITMAP_LONGS +1 ];
#endif /* X86_IOPL_EMLATION */
#endif /* !X86_IOPL_NONE */
arch/x86/include/asm/processor.h struct tss_struct { · line 388
Quoted 1 line
arch/x86/include/asm/processor.h +0
-0}__aligned(PAGE_SIZE) ;
Would allow us to reclaim those 8/16K bitmaps for LEGACY/NONE kernels.
Re: [patch 7/9] x86/iopl: Restrict iopl() permission scope
In reply to Thomas GleixnerQuoted 214 lines · Thomas Gleixner
The access to the full I/O port range can be also provided by the TSS I/O bitmap, but that would require to copy 8k of data on scheduling in the task. As shown with the sched out optimization TSS.io_bitmap_base can be used to switch the incoming task to a preallocated I/O bitmap which has all bits zero, i.e. allows access to all I/O ports.
Implementing this allows to provide an
iopl()emulation mode which restricts the IOPL level 3 permissions to I/O port access but removes the STI/CLI permission which is coming with the hardware IOPL mechansim.Provide a config option to switch IOPL to emulation mode, make it the default and while at it also provide an option to disable IOPL completely.
Signed-off-by Thomas Gleixner <[email protected]>--- arch/x86/Kconfig | 32 ++++++++++++++++ arch/x86/include/asm/processor.h | 24 +++++++++--- arch/x86/kernel/cpu/common.c | 4 +- arch/x86/kernel/ioport.c | 75 ++++++++++++++++++++++++++++++--------- arch/x86/kernel/process.c | 15 +++++-- 5 files changed, 123 insertions(+), 27 deletions(-)arch/x86/Kconfig +32
-0@@ -1254,6 +1254,38 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. +choice + prompt "IOPL" + default X86_IOPL_EMULATION + +config X86_IOPL_EMULATION + bool "IOPL Emulation" + ---help--- + Legacy IOPL support is an overbroad mechanism which allows user + space aside of accessing all 65536 I/O ports also to disable + interrupts. To gain this access the caller needs CAP_SYS_RAWIO + capabilities and permission from eventually active security + modules. + + The emulation restricts the functionality of the syscall to + only allowing the full range I/O port access, but prevents the + ability to disable interrupts from user space. + +config X86_IOPL_LEGACY + bool "IOPL Legacy" + ---help--- + Allow the full IOPL permissions, i.e. user space access to all + 65536 I/O ports and also the ability to disable interrupts, which + is overbroad and can result in system lockups. + +config X86_IOPL_NONE + bool "IOPL None" + ---help--- + Disable the IOPL permission syscall. That's the safest option as + no sane application should depend on this functionality. + +endchoice + config TOSHIBA tristate "Toshiba Laptop support" depends on X86_32arch/x86/include/asm/processor.h +19
-5@@ -331,8 +331,12 @@ struct x86_hw_tss {#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) -#define IO_BITMAP_OFFSET_VALID \ - (offsetof(struct tss_struct, io_bitmap) - \ +#define IO_BITMAP_OFFSET_VALID_MAP \ + (offsetof(struct tss_struct, io_bitmap_map) - \ + offsetof(struct tss_struct, x86_tss)) + +#define IO_BITMAP_OFFSET_VALID_ALL \ + (offsetof(struct tss_struct, io_bitmap_all) - \ offsetof(struct tss_struct, x86_tss))/* @@ -343,7 +347,7 @@ struct x86_hw_tss {* last valid byte */ #define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + (IO_BITMAP_OFFSET_VALID_ALL + IO_BITMAP_BYTES + sizeof(unsigned long) - 1)/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ #define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)@@ -379,7 +383,8 @@ struct tss_struct {* byte beyond the end of the I/O permission bitmap. The extra byte * must have all bits set and must be within the TSS limit. */ -unsigned long io_bitmap[IO_BITMAP_LONGS +1 ]; +unsigned long io_bitmap_map[IO_BITMAP_LONGS +1 ]; +unsigned long io_bitmap_all[IO_BITMAP_LONGS +1 ]; }__aligned(PAGE_SIZE) ; DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw);@@ -495,7 +500,6 @@ struct thread_struct {#endif /* IO permissions: */ unsigned long *io_bitmap_ptr; -unsigned long iopl;/* * The byte range in the I/O permission bitmap which contains zero * bits. @@ -503,6 +507,13 @@ struct thread_struct {unsigned int io_zerobits_start;unsigned int io_zerobits_end; +/* +* IOPL. Priviledge level dependent I/O permission which includes +* user space CLI/STI when granted. +*/ +unsigned long iopl; +unsigned long iopl_emul; +mm_segment_t addr_limit; unsigned int sig_on_uaccess_err:1;@@ -524,6 +535,9 @@ static inline void arch_thread_struct_wh *size = fpu_kernel_xstate_size; } +extern void tss_update_io_bitmap (struct tss_struct *tss, +struct thread_struct *thread); +/* * Thread-synchronous status. * arch/x86/kernel/cpu/common.c +3
-1@@ -1866,7 +1866,9 @@ void cpu_init(void) tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; tss->io_zerobits_start =IO_BITMAP_BYTES ; tss->io_zerobits_end =0 ; -memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap)); +memset (tss->io_bitmap_map,0xff ,sizeof (tss->io_bitmap_map)); +/* Invalidate the extra tail entry of the allow all I/O bitmap */ + tss->io_bitmap_all[IO_BITMAP_LONGS ] = ~0UL ;set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();arch/x86/kernel/ioport.c +18
-17@@ -109,7 +109,7 @@ long ksys_ioperm(unsigned long from, uns }/* Copy the changed range over to the TSS bitmap */ - dst = (char *)tss->io_bitmap; + dst = (char *)tss->io_bitmap_map; src = (char *)bitmap;memcpy (dst + copy_start, src + copy_start, copy_len);@@ -130,7 +130,7 @@ long ksys_ioperm(unsigned long from, uns* is correct. It might have been wreckaged by a VMEXiT. */ set_thread_flag (TIF_IO_BITMAP ); - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ; + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID_MAP ;refresh_tss_limit (); }@@ -184,36 +184,77 @@ SYSCALL_DEFINE3(ioperm, unsigned long, f*/ SYSCALL_DEFINE1(iopl, unsigned int, level) { -struct pt_regs *regs =current_pt_regs ();struct thread_struct *t = ¤t->thread; +struct pt_regs *regs =current_pt_regs (); +unsigned int old;/* * Careful: the IOPL bits in regs->flags are undefined under Xen PV * and changing them has no effect. */ -unsigned int old = t->iopl >>X86_EFLAGS_IOPL_BIT ; +if (IS_ENABLED (CONFIG_X86_IOPL_NONE )) +return -ENOSYS ;if (level >3 )return -EINVAL ; + +if (IS_ENABLED (CONFIG_X86_IOPL_EMULATION )) + old = t->iopl_emul; +else + old = t->iopl >>X86_EFLAGS_IOPL_BIT ; + +/* No point in going further if nothing changes */ +if (level == old) +return 0 ; +/* Trying to gain more privileges? */ if (level > old) {if (!capable (CAP_SYS_RAWIO ) ||security_locked_down (LOCKDOWN_IOPORT ))return -EPERM ; } -/* -* Change the flags value on the return stack, which has been set -* up on system-call entry. See also the fork and signal handling -* code how this is handled. -*/ - regs->flags = (regs->flags & ~X86_EFLAGS_IOPL ) | - (level <<X86_EFLAGS_IOPL_BIT ); -/* Store the new level in the thread struct */ - t->iopl = level <<X86_EFLAGS_IOPL_BIT ; -/* -* X86_32 switches immediately and XEN handles it via emulation. -*/ -set_iopl_mask (t->iopl); + +if (IS_ENABLED (CONFIG_X86_IOPL_EMULATION )) {
arch/x86/kernel/ioport.c SYSCALL_DEFINE3(ioperm, unsigned long, f · line 227
Quoted 10 lines
arch/x86/kernel/ioport.c +10
-0+struct tss_struct *tss; +unsigned int tss_base; + +/* Prevent racing against a task switch */ +preempt_disable (); + tss =this_cpu_ptr (&cpu_tss_rw); +if (level ==3 ) { +/* Grant access to all I/O ports */ +set_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_VALID_ALL ;
Where is the actual TSS updated?
arch/x86/kernel/ioport.c SYSCALL_DEFINE3(ioperm, unsigned long, f · line 236
Quoted 9 lines
arch/x86/kernel/ioport.c +9
-0+ }else if (t->io_bitmap_ptr) { +/* Thread has a I/O bitmap */ +tss_update_io_bitmap (tss, t); +set_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_VALID_MAP ; + }else { +/* Take it out of the context switch work burden */ +clear_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_INVALID ;
Ditto.
I think what you need to do is have a single function, called by exit_thread(), switch_to(), and here, that updates the TSS to match a given task's IO bitmap state. This is probably switch_to_bitmap() or similar.
(Maybe it already is, but I swear I checked all the patches in the series and I can't find the body of tss_update_io_bitmap(). But you should call it in all branches of this if-else thing.)
Re: [patch 7/9] x86/iopl: Restrict iopl() permission scope
Quoted 15 lines · Andy Lutomirski
arch/x86/kernel/ioport.c SYSCALL_DEFINE3(ioperm, unsigned long, f · line 227
Quoted 12 lines · Thomas Gleixner
arch/x86/kernel/ioport.c +12
-0+ +if (IS_ENABLED (CONFIG_X86_IOPL_EMULATION )) { +struct tss_struct *tss; +unsigned int tss_base; + +/* Prevent racing against a task switch */ +preempt_disable (); + tss =this_cpu_ptr (&cpu_tss_rw); +if (level ==3 ) { +/* Grant access to all I/O ports */ +set_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_VALID_ALL ;Where is the actual TSS updated?
Here. It sets the offset to the all zero bitmap. That's all it needs.
Quoted 12 lines
arch/x86/kernel/ioport.c SYSCALL_DEFINE3(ioperm, unsigned long, f · line 236
Quoted 9 lines
arch/x86/kernel/ioport.c +9
-0+ }else if (t->io_bitmap_ptr) { +/* Thread has a I/O bitmap */ +tss_update_io_bitmap (tss, t); +set_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_VALID_MAP ; + }else { +/* Take it out of the context switch work burden */ +clear_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_INVALID ;Ditto.
Quoted 4 lines
I think what you need to do is have a single function, called by
exit_thread(),switch_to(), and here, that updates the TSS to match a given task's IO bitmap state. This is probablyswitch_to_bitmap()or similar.
Well, no. exit_thread() and this here actually fiddle with the TIF bit which is not what the switch to case does. There is some stuff which can be shared.
Quoted 3 lines
(Maybe it already is, but I swear I checked all the patches in the series and I can't find the body of
tss_update_io_bitmap(). But you should call it in all branches of this if-else thing.)
It's in that very same patch:
arch/x86/kernel/process.c void arch_setup_new_exec(void) · line 361
Quoted 4 lines
arch/x86/kernel/process.c +1
-2-static void tss_update_io_bitmap (struct tss_struct *tss, -struct thread_struct *thread) +void tss_update_io_bitmap (struct tss_struct *tss,struct thread_struct *thread) {
Let me try to get a bit more reuse. Which still leaves the TIF bit fiddling in this code path.
Thanks,
tglx
Re: [patch 7/9] x86/iopl: Restrict iopl() permission scope
Quoted 18 lines · Thomas Gleixner
Quoted 15 lines · Andy Lutomirski
arch/x86/kernel/ioport.c SYSCALL_DEFINE3(ioperm, unsigned long, f · line 227
Quoted 12 lines · Thomas Gleixner
arch/x86/kernel/ioport.c +12
-0+ +if (IS_ENABLED (CONFIG_X86_IOPL_EMULATION )) { +struct tss_struct *tss; +unsigned int tss_base; + +/* Prevent racing against a task switch */ +preempt_disable (); + tss =this_cpu_ptr (&cpu_tss_rw); +if (level ==3 ) { +/* Grant access to all I/O ports */ +set_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_VALID_ALL ;Where is the actual TSS updated?
Quoted 1 line
Here. It sets the offset to the all zero bitmap. That's all it needs.
Ah, I see.
Quoted 8 lines
Quoted 4 lines
I think what you need to do is have a single function, called by
exit_thread(),switch_to(), and here, that updates the TSS to match a given task's IO bitmap state. This is probablyswitch_to_bitmap()or similar.Well, no.
exit_thread()and this here actually fiddle with the TIF bit which is not what the switch to case does. There is some stuff which can be shared.
I was thinking that the code that read iopl_emul and t->io_bitmap_ptr and updated x86_tss.io_bitmap_base could be factored out of switch_to(). Suppose you call that tss_update_io_bitmap(). And you could have another tiny helper:
static void update_tif_io_bitmap(void)
{
if (...->iopl_emul || ...->io_bitmap_ptr)
set_thread_flag(TIF_IO_BITMAP);
else
clear_thread_flag(TIF_IO_BITMAP);
}Then the whole iopl emulation path becomes:
preempt_disable();
...->iopl_emul = iopl;update_tif_io_bitmap();tss_update_io_bitmap();
and switch_to() does
if (new or prev has TIF_IO_BITMAP)
tss_update_io_bitmap();
and exit_thread() does:
...->iopl_emul = 0;
kfree(...);
...->io_bitmap_ptr = 0; clear_thread_flag(TIF_IO_BITMAP) or update_tif_io_bitmap(); tss_update_io_bitmap();
and ioperm() does:
kmalloc (if needed);
update the thread_struct copy of the bitmap. set_thread_flag(TIF_IO_BITMAP) or update_tif_io_bitmap(); tss_update_io_bitmap();
Does that make sense?
Quoted 9 lines
Quoted 4 lines
(Maybe it already is, but I swear I checked all the patches in the series and I can't find the body of
tss_update_io_bitmap(). But you should call it in all branches of this if-else thing.)It's in that very same patch:
Quoted 2 lines
arch/x86/kernel/process.c +0
-2-static void tss_update_io_bitmap (struct tss_struct *tss, -struct thread_struct *thread)
But where did the line you just deleted come from? I'm obviously just bad at reading emails somewhere.
Quoted 9 lines
Quoted 2 lines
+void
tss_update_io_bitmap(struct tss_struct *tss, struct thread_struct *thread)
{Let me try to get a bit more reuse. Which still leaves the TIF bit fiddling in this code path.
Thanks,
tglx
Re: [patch 7/9] x86/iopl: Restrict iopl() permission scope
Quoted 40 lines · Andy Lutomirski
Quoted 18 lines · Thomas Gleixner
Quoted 15 lines · Andy Lutomirski
arch/x86/kernel/ioport.c SYSCALL_DEFINE3(ioperm, unsigned long, f · line 227
Quoted 12 lines · Thomas Gleixner
arch/x86/kernel/ioport.c +12
-0+ +if (IS_ENABLED (CONFIG_X86_IOPL_EMULATION )) { +struct tss_struct *tss; +unsigned int tss_base; + +/* Prevent racing against a task switch */ +preempt_disable (); + tss =this_cpu_ptr (&cpu_tss_rw); +if (level ==3 ) { +/* Grant access to all I/O ports */ +set_thread_flag (TIF_IO_BITMAP ); + tss_base =IO_BITMAP_OFFSET_VALID_ALL ;Where is the actual TSS updated?
Quoted 1 line
Here. It sets the offset to the all zero bitmap. That's all it needs.
Ah, I see.
Quoted 8 lines
Quoted 4 lines
I think what you need to do is have a single function, called by
exit_thread(),switch_to(), and here, that updates the TSS to match a given task's IO bitmap state. This is probablyswitch_to_bitmap()or similar.Well, no.
exit_thread()and this here actually fiddle with the TIF bit which is not what the switch to case does. There is some stuff which can be shared.I was thinking that the code that read
iopl_emulandt->io_bitmap_ptrand updated x86_tss.io_bitmap_base could be factored out ofswitch_to(). Suppose you call thattss_update_io_bitmap(). And you could have another tiny helper:static void
update_tif_io_bitmap(void)
{
if (...->iopl_emul || ...->io_bitmap_ptr)
Quoted 6 lines
set_thread_flag(TIF_IO_BITMAP); else clear_thread_flag(TIF_IO_BITMAP); }Then the whole iopl emulation path becomes:
Yes. I'm almost done with that already :)
Quoted 7 lines
Quoted 4 lines
It's in that very same patch:
Quoted 2 lines
arch/x86/kernel/process.c +0
-2-static void tss_update_io_bitmap (struct tss_struct *tss, -struct thread_struct *thread)But where did the line you just deleted come from? I'm obviously just bad at reading emails somewhere.
patch 5/9 :)
Let me finish that stuff and send out another version which has that reuse of the switch to code in a separate patch.
There is another new detail in the series. I split out all the io bitmap data into a separate data struct, which also allows me to stick a refcount into it which removes the kmemdup() from the fork path.
Thanks,
tglx
kbuild test robot
build botRe: [patch 7/9] x86/iopl: Restrict iopl() permission scope
In reply to Thomas GleixnerI love your patch! Yet something to improve:
Hi Thomas,
I love your patch! Yet something to improve:
[auto build test ERROR on xen-tip/linux-next]
[also build test ERROR on v5.4-rc6 next-20191108]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Thomas-Gleixner/x86-iopl-Prevent-user-space-from-using-CLI-STI-with-iopl-3/20191109-051154 base: https://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git linux-next config: i386-tinyconfig (attached as .config) compiler: gcc-7 (Debian 7.4.0-14) 7.4.0 reproduce: # save the attached .config to linux build tree make ARCH=i386
If you fix the issue, kindly add following tag
All errors (new ones prefixed by >>):
In file included from include/linux/export.h:42:0,
from include/linux/linkage.h:7,
from include/linux/preempt.h:10,
from include/linux/spinlock.h:51,
from arch/x86//mm/cpu_entry_area.c:3:
In function 'setup_cpu_entry_area_ptes',
inlined from 'setup_cpu_entry_areas' at arch/x86//mm/cpu_entry_area.c:197:2:
>> include/linux/compiler.h:350:38: error: call to '__compiletime_assert_181' declared with attribute error: BUILD_BUG_ON failed: CPU_ENTRY_AREA_PAGES * PAGE_SIZE < CPU_ENTRY_AREA_MAP_SIZE
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
^
include/linux/compiler.h:331:4: note: in definition of macro '__compiletime_assert'
prefix ## suffix(); \
^~~~~~
include/linux/compiler.h:350:2: note: in expansion of macro '_compiletime_assert'
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
#define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:50:2: note: in expansion of macro 'BUILD_BUG_ON_MSG'
BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
^~~~~~~~~~~~~~~~
arch/x86//mm/cpu_entry_area.c:181:2: note: in expansion of macro 'BUILD_BUG_ON'
BUILD_BUG_ON(CPU_ENTRY_AREA_PAGES * PAGE_SIZE < CPU_ENTRY_AREA_MAP_SIZE);
^~~~~~~~~~~~
--
In file included from include/linux/export.h:42:0,
from include/linux/linkage.h:7,
from include/linux/preempt.h:10,
from include/linux/spinlock.h:51,
from arch/x86/mm/cpu_entry_area.c:3:
In function 'setup_cpu_entry_area_ptes',
inlined from 'setup_cpu_entry_areas'@arch/x86/mm/cpu_entry_area.c:197:2:
>> include/linux/compiler.h:350:38: error: call to '__compiletime_assert_181' declared with attribute error: BUILD_BUG_ON failed: CPU_ENTRY_AREA_PAGES * PAGE_SIZE < CPU_ENTRY_AREA_MAP_SIZE
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
^
include/linux/compiler.h:331:4: note: in definition of macro '__compiletime_assert'
prefix ## suffix(); \
^~~~~~
include/linux/compiler.h:350:2: note: in expansion of macro '_compiletime_assert'
_compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
^~~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:39:37: note: in expansion of macro 'compiletime_assert'
#define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
^~~~~~~~~~~~~~~~~~
include/linux/build_bug.h:50:2: note: in expansion of macro 'BUILD_BUG_ON_MSG'
BUILD_BUG_ON_MSG(condition, "BUILD_BUG_ON failed: " #condition)
^~~~~~~~~~~~~~~~
arch/x86/mm/cpu_entry_area.c:181:2: note: in expansion of macro 'BUILD_BUG_ON'
BUILD_BUG_ON(CPU_ENTRY_AREA_PAGES * PAGE_SIZE < CPU_ENTRY_AREA_MAP_SIZE);
^~~~~~~~~~~~vim +/__compiletime_assert_181 +350 include/linux/compiler.h
9a8ab1c39970a4 Daniel Santos 2013-02-21 336 9a8ab1c39970a4 Daniel Santos 2013-02-21 337 #define _compiletime_assert(condition, msg, prefix, suffix) \ 9a8ab1c39970a4 Daniel Santos 2013-02-21 338 __compiletime_assert(condition, msg, prefix, suffix) 9a8ab1c39970a4 Daniel Santos 2013-02-21 339 9a8ab1c39970a4 Daniel Santos 2013-02-21 340 /** 9a8ab1c39970a4 Daniel Santos 2013-02-21 341 * compiletime_assert - break build and emit msg if condition is false 9a8ab1c39970a4 Daniel Santos 2013-02-21 342 * @condition: a compile-time constant condition to check 9a8ab1c39970a4 Daniel Santos 2013-02-21 343 * @msg: a message to emit if condition is false 9a8ab1c39970a4 Daniel Santos 2013-02-21 344 * 9a8ab1c39970a4 Daniel Santos 2013-02-21 345 * In tradition of POSIX assert, this macro will break the build if the 9a8ab1c39970a4 Daniel Santos 2013-02-21 346 * supplied condition is *false*, emitting the supplied error message if the 9a8ab1c39970a4 Daniel Santos 2013-02-21 347 * compiler has support to do so. 9a8ab1c39970a4 Daniel Santos 2013-02-21 348 */ 9a8ab1c39970a4 Daniel Santos 2013-02-21 349 #define compiletime_assert(condition, msg) \
9a8ab1c39970a4 Daniel Santos 2013-02-21 @350 _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)9a8ab1c39970a4 Daniel Santos 2013-02-21 351
:::::: The code@line 350 was first introduced by commit
:::::: 9a8ab1c39970a4938a72d94e6fd13be88a797590 bug.h, compiler.h: introduce compiletime_assert & BUILD_BUG_ON_MSG
:::::: TO: Daniel Santos <[email protected]> :::::: CC: Linus Torvalds <[email protected]>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org Intel Corporation
PATCH 8/9 x86/iopl: Remove legacy IOPL option
In reply to Thomas GleixnerThe IOPL emulation via the I/O bitmap is sufficient. Remove the legacy cruft dealing with the (e)flags based IOPL mechanism.
--- arch/x86/Kconfig | 10 ---- arch/x86/include/asm/paravirt.h | 4 - arch/x86/include/asm/paravirt_types.h | 2 arch/x86/include/asm/processor.h | 26 +--------- arch/x86/include/asm/xen/hypervisor.h | 2 arch/x86/kernel/ioport.c | 84 +++++++++++----------------------- arch/x86/kernel/paravirt.c | 2 arch/x86/kernel/process_32.c | 9 --- arch/x86/kernel/process_64.c | 11 ---- arch/x86/xen/enlighten_pv.c | 10 ---- 10 files changed, 34 insertions(+), 126 deletions(-)
arch/x86/Kconfig +2 -8
@@ -1269,14 +1269,8 @@ config X86_IOPL_EMULATION The emulation restricts the functionality of the syscall to only allowing the full range I/O port access, but prevents the - ability to disable interrupts from user space. - -config X86_IOPL_LEGACY - bool "IOPL Legacy" - ---help--- - Allow the full IOPL permissions, i.e. user space access to all - 65536 I/O ports and also the ability to disable interrupts, which - is overbroad and can result in system lockups. + ability to disable interrupts from user space which would be + granted if the hardware IOPL mechanism would be used. config X86_IOPL_NONE bool "IOPL None"
arch/x86/include/asm/paravirt.h +0 -4
@@ -294,10 +294,6 @@ static inline void write_idt_entry(gate_ {PVOP_VCALL3 (cpu.write_idt_entry, dt, entry, g); } -static inline void set_iopl_mask (unsigned mask) -{ -PVOP_VCALL1 (cpu.set_iopl_mask, mask); -}static inline void paravirt_activate_mm (struct mm_struct *prev,struct mm_struct *next)
arch/x86/include/asm/paravirt_types.h +0 -2
@@ -140,8 +140,6 @@ struct pv_cpu_ops {void (*load_sp0)(unsigned long sp0); -void (*set_iopl_mask)(unsigned mask); -void (*wbinvd)(void );/* cpuid emulation, mostly so that caps bits can be disabled */
arch/x86/include/asm/processor.h +3 -23
@@ -508,10 +508,10 @@ struct thread_struct {unsigned int io_zerobits_end;/* -* IOPL. Priviledge level dependent I/O permission which includes -* user space CLI/STI when granted. +* IOPL. Priviledge level dependent I/O permission which is +* emulated via the I/O bitmap to prevent user space from disabling +* interrupts. */ -unsigned long iopl;unsigned long iopl_emul;mm_segment_t addr_limit;
@@ -547,25 +547,6 @@ extern void tss_update_io_bitmap(struct*/ #define TS_COMPAT 0x0002/* 32bit syscall active (64BIT)*/ -/* -* Set IOPL bits in EFLAGS from given mask -*/ -static inline void native_set_iopl_mask (unsigned mask) -{ -#ifdef CONFIG_X86_32 -unsigned int reg; - - asmvolatile ("pushfl;" -"popl %0;" -"andl %1, %0;" -"orl %2, %0;" -"pushl %0;" -"popfl" - :"=&r" (reg) - :"i" (~X86_EFLAGS_IOPL ),"r" (mask)); -#endif -} -static inline void native_load_sp0 (unsigned long sp0) {
@@ -605,7 +586,6 @@ static inline void load_sp0(unsigned lon native_load_sp0(sp0); } -#define set_iopl_mask native_set_iopl_mask#endif /* CONFIG_PARAVIRT_XXL */ /* Free all resources held by a thread. */
arch/x86/include/asm/xen/hypervisor.h +0 -2
@@ -62,6 +62,4 @@ void xen_arch_register_cpu(int num);void xen_arch_unregister_cpu (int num);#endif -extern void xen_set_iopl_mask (unsigned mask); -#endif /* _ASM_X86_XEN_HYPERVISOR_H */
arch/x86/kernel/ioport.c +29 -55
113 lines not on this page · raw message
arch/x86/kernel/paravirt.c +0 -2
@@ -341,8 +341,6 @@ struct paravirt_patch_template pv_ops = .cpu.iret = native_iret, .cpu.swapgs = native_swapgs, - .cpu.set_iopl_mask = native_set_iopl_mask, - .cpu.start_context_switch = paravirt_nop, .cpu.end_context_switch = paravirt_nop,
arch/x86/kernel/process_32.c +0 -9
@@ -187,15 +187,6 @@ EXPORT_SYMBOL_GPL(start_thread);*/ load_TLS (next, cpu); -/* -* Restore IOPL if needed. In normal use, the flags restore -* in the switch assembly will handle this. But if the kernel -* is running virtualized at a non-zero CPL, the popf will -* not restore flags, so it must be done in a separate step. -*/ -if (get_kernel_rpl () &&unlikely (prev->iopl != next->iopl)) -set_iopl_mask (next->iopl); -switch_to_extra (prev_p, next_p);/*
arch/x86/kernel/process_64.c +0 -11
@@ -497,17 +497,6 @@ void compat_start_thread(struct pt_regsswitch_to_extra (prev_p, next_p); -#ifdef CONFIG_XEN_PV -/* -* On Xen PV, IOPL bits in pt_regs->flags have no effect, and -* current_pt_regs()->flags may not match the current task's -* intended IOPL. We need to switch it manually. -*/ -if (unlikely (static_cpu_has (X86_FEATURE_XENPV ) && - prev->iopl != next->iopl)) -xen_set_iopl_mask (next->iopl); -#endif -if (static_cpu_has_bug (X86_BUG_SYSRET_SS_ATTRS )) {/* * AMD CPUs have a misfeature: SYSRET sets the SS selector but
arch/x86/xen/enlighten_pv.c +0 -10
@@ -829,15 +829,6 @@ static void xen_load_sp0(unsigned long sthis_cpu_write (cpu_tss_rw.x86_tss.sp0, sp0); } -void xen_set_iopl_mask (unsigned mask) -{ -struct physdev_set_iopl set_iopl; - -/* Force the change at ring 0. */ - set_iopl.iopl = (mask ==0 ) ?1 : (mask >>12 ) &3 ; -HYPERVISOR_physdev_op (PHYSDEVOP_set_iopl, &set_iopl); -} -static void xen_io_delay (void ) { }
@@ -1047,7 +1038,6 @@ static const struct pv_cpu_ops xen_cpu_o .write_idt_entry = xen_write_idt_entry, .load_sp0 = xen_load_sp0, - .set_iopl_mask = xen_set_iopl_mask, .io_delay = xen_io_delay,/* Xen takes care of %gs when switching to usermode for us */
Re: [patch 8/9] x86/iopl: Remove legacy IOPL option
Quoted 4 lines · Thomas Gleixner
The IOPL emulation via the I/O bitmap is sufficient. Remove the legacy cruft dealing with the (e)flags based IOPL mechanism.
Signed-off-by Thomas Gleixner <[email protected]>
Juergen
[email protected]
reviewerRe: [patch 8/9] x86/iopl: Remove legacy IOPL option
Quoted 11 lines · Jürgen Groß
Quoted 1 line · Thomas Gleixner
The IOPL emulation via the I/O bitmap is sufficient. Remove the
legacy
Quoted 3 lines
cruft dealing with the (e)flags based IOPL mechanism.
Signed-off-by Thomas Gleixner <[email protected]>Juergen
Good grief, good riddance.
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Stephen Hemminger
Acked-byRe: [patch 8/9] x86/iopl: Remove legacy IOPL option
In reply to Jürgen GroßQuoted 7 lines · On Thu, 7 Nov 2019 07:11:49 +0100 Jürgen Groß <[email protected]> wrote:
Quoted 4 lines · Thomas Gleixner
The IOPL emulation via the I/O bitmap is sufficient. Remove the legacy cruft dealing with the (e)flags based IOPL mechanism.
Signed-off-by Thomas Gleixner <[email protected]>
I am happy to see this go. The DPDK only did initially because it was only way virtio would work at the time.
Re: [patch 8/9] x86/iopl: Remove legacy IOPL option
In reply to Thomas GleixnerQuoted 2 lines · Thomas Gleixner
The IOPL emulation via the I/O bitmap is sufficient. Remove the legacy cruft dealing with the (e)flags based IOPL mechanism.
Much joy at seeing this code go!
PATCH 9/9 selftests/x86/iopl: Verify that CLI/STI result in #GP
In reply to Thomas GleixnerAdd tests that the now emulated iopl() functionality does not longer allow user space to disable interrupts.
--- tools/testing/selftests/x86/iopl.c | 104 +++++++++++++++++++++++++++++++++---- 1 file changed, 93 insertions(+), 11 deletions(-)
tools/testing/selftests/x86/iopl.c +93 -11
@@ -35,6 +35,16 @@ static void sethandler(int sig, void (*h } +static void clearhandler (int sig) +{ +struct sigaction sa; +memset (&sa,0 ,sizeof (sa)); + sa.sa_handler =SIG_DFL ; +sigemptyset (&sa.sa_mask); +if (sigaction (sig, &sa,0 )) +err (1 ,"sigaction" ); +} +static jmp_buf jmpbuf;static void sigsegv (int sig,siginfo_t *si,void *ctx_void)
@@ -42,22 +52,100 @@ static void sigsegv(int sig, siginfo_t *siglongjmp (jmpbuf,1 ); } +static bool try_outb (unsigned short port) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("outb %%al, %w[port]" + : : [port]"Nd" (port),"a" (0 )); +return true; + } +clearhandler (SIGSEGV ); +} + +static void expect_gp_outb (unsigned short port) +{ +if (try_outb (port)) { +printf ("[FAIL]\toutb to 0x%02hx worked\n" , port); + nerrs++; + } + +printf ("[OK]\toutb to 0x%02hx failed\n" , port); +} + +static bool try_cli (void ) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("cli" ); +return true; + } +clearhandler (SIGSEGV ); +} + +static bool try_sti (void ) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("sti" ); +return true; + } +clearhandler (SIGSEGV ); +} + +static void expect_gp_sti (void ) +{ +if (try_sti ()) { +printf ("[FAIL]\tSTI worked\n" ); + nerrs++; + }else { +printf ("[OK]\tSTI faulted\n" ); + } +} + +static void expect_gp_cli (void ) +{ +if (try_cli ()) { +printf ("[FAIL]\tCLI worked\n" ); + nerrs++; + }else { +printf ("[OK]\tCLI faulted\n" ); + } +} +int main (void ) {cpu_set_t cpuset; +CPU_ZERO (&cpuset);CPU_SET (0 , &cpuset);if (sched_setaffinity (0 ,sizeof (cpuset), &cpuset) !=0 )err (1 ,"sched_setaffinity to CPU 0" );/* Probe for iopl support. Note that iopl(0) works even as nonroot. */ -if (iopl (3 ) !=0 ) { +switch (iopl (3 )) { +case 0 : +break ; +case -ENOSYS : +printf ("[OK]\tiopl() nor supported\n" ); +return 0 ; +default :printf ("[OK]\tiopl(3) failed (%d) -- try running as root\n" , errno);return 0 ; } -/* Restore our original state prior to starting the test. */ +/* Make sure that CLI/STI are blocked even with IOPL level 3 */ +expect_gp_cli (); +expect_gp_sti (); + +/* Restore our original state prior to starting the fork test. */ if (iopl (0 ) !=0 )err (1 ,"iopl(0)" );
@@ -90,14 +178,9 @@ int main(void)printf ("[RUN]\tparent: write to 0x80 (should fail)\n" ); -sethandler (SIGSEGV , sigsegv,0 ); -if (sigsetjmp (jmpbuf,1 ) !=0 ) { -printf ("[OK]\twrite was denied\n" ); - }else { - asmvolatile ("outb %%al, $0x80" : :"a" (0 )); -printf ("[FAIL]\twrite was allowed\n" ); - nerrs++; - } +expect_gp_outb (0x80 ); +expect_gp_cli (); +expect_gp_sti ();/* Test the capability checks. */ printf ("\tiopl(3)\n" );
@@ -133,4 +216,3 @@ int main(void) done:return nerrs ?1 :0 ; } -
PATCH 1/1 x86/iopl: Remove unused local variable, update comments in ksys_ioperm()
In reply to Thomas GleixnerQuoted 3 lines · Thomas Gleixner
The series is also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitWIP.x86/iopl
Very nice series - I fully agree with this simplification of ioperm legacies.
On x86-64 defconfig new warning in ioport.c:
arch/x86/kernel/ioport.c:184:18: warning: unused variable ‘regs’ [-Wunused-variable]
This local variable can simply be removed, now that we don't rely on regs->flags anymore. See the patch below.
I also removed the now stale comment about the Xen PV quirk/incompatibility.
Thanks,
Ingo
--- arch/x86/kernel/ioport.c | 5 ----- 1 file changed, 5 deletions(-)
arch/x86/kernel/ioport.c +0 -5
@@ -181,15 +181,10 @@ SYSCALL_DEFINE3(ioperm, unsigned long, from, unsigned long, num, int, turn_on) SYSCALL_DEFINE1(iopl, unsigned int, level) {struct thread_struct *t = ¤t->thread; -struct pt_regs *regs =current_pt_regs ();struct tss_struct *tss;unsigned int tss_base;unsigned int old; -/* -* Careful: the IOPL bits in regs->flags are undefined under Xen PV -* and changing them has no effect. -*/ if (IS_ENABLED (CONFIG_X86_IOPL_NONE ))return -ENOSYS ;