x86/iopl: Prevent user space from using CLI/STI with iopl(3)
v3 cover x86/iopl: Prevent user space from using CLI/STI with iopl(3)
This is the third version of the attempt to confine the unwanted side effects of iopl(). The first version of this series can be found here:
https://lore.kernel.org/r/[email protected]
Second version is here:
https://lore.kernel.org/r/[email protected]
The V1 cover letter also contains a longer variant of the background. Summary:
iopl(level = 3) enables aside of access to all 65536 I/O ports also the usage of CLI/STI in user space.
Disabling interrupts in user space can lead to system lockups and breaks assumptions in the kernel that userspace always runs with interrupts enabled.
iopl() is often preferred over ioperm() as it avoids the overhead of copying the tasks I/O bitmap to the TSS bitmap on context switch. This overhead can be avoided by providing a all zeroes bitmap in the TSS and switching the TSS bitmap offset to this permit all IO bitmap. It's marginally slower than iopl() which is a one time setup, but prevents the usage of CLI/STI in user space.
The changes vs. V3:
- Split out the restructuring of the first/subsequent ioperm() invocation into a seperate patch to address the inconsisteny which Andy detected in the patch which introduces the concept of invalidating the I/O bitmap base to speed up context switching. This change is moved in front so the subsequent changes are functionally correct.
- Moved the non HW TSS data related to I/O bitmap(s) into a seperate data structure. Modified version of Ingos proposed patch.
- Made struct memeber names more consistent (Ingo)
- Dropped the bitmap union. It is not longer necessary because V2 already dropped the finer grained copying algorithm. The sequence count approach should avoid most of the copying overhead when the number of ioperm() using processes is very low which is the normal case.
- Dropped the pointer storage of the bitmap in the TSS data as it is not required (Peter, Andy)
- Fixed the missing refcount setting in the bitmap duplication code path. (Peter, Andy)
- Updated changelog and comment to explain the bitmap invalidation logic. (Andy)
- Removed TIF_IO_BITMAP from the TIF flags which are evaluated on the next task for entering the slow path.
- Folded the NULL pointer check fix
- Simplified the config option in the legacy removal patch (Andy)
- Extended the scope of the config option to disable ioperm() along with iopl() which also mokes all related storage and functions compile time conditional. (Andy)
The series is also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.x86/iopl
Thanks,
tglx
--- arch/x86/Kconfig | 18 ++ arch/x86/entry/common.c | 4 arch/x86/include/asm/io_bitmap.h | 29 ++++ arch/x86/include/asm/paravirt.h | 4 arch/x86/include/asm/paravirt_types.h | 2 arch/x86/include/asm/pgtable_32_types.h | 2 arch/x86/include/asm/processor.h | 113 ++++++++++------- arch/x86/include/asm/ptrace.h | 6 arch/x86/include/asm/switch_to.h | 10 + arch/x86/include/asm/thread_info.h | 14 +- arch/x86/include/asm/xen/hypervisor.h | 2 arch/x86/kernel/cpu/common.c | 188 ++++++++++++---------------- arch/x86/kernel/doublefault.c | 2 arch/x86/kernel/ioport.c | 209 +++++++++++++++++++++----------- arch/x86/kernel/paravirt.c | 2 arch/x86/kernel/process.c | 200 ++++++++++++++++++++++++------ arch/x86/kernel/process_32.c | 77 ----------- arch/x86/kernel/process_64.c | 86 ------------- arch/x86/kernel/ptrace.c | 12 + arch/x86/kvm/vmx/vmx.c | 8 - arch/x86/mm/cpu_entry_area.c | 8 + arch/x86/xen/enlighten_pv.c | 10 - tools/testing/selftests/x86/ioperm.c | 16 ++ tools/testing/selftests/x86/iopl.c | 129 ++++++++++++++++++- 24 files changed, 674 insertions(+), 477 deletions(-)
PATCH v3 1/20 x86/ptrace: Prevent truncation of bitmap size
From: Thomas Gleixner <[email protected]>
The active() callback of the IO bitmap regset divides the IO bitmap size by the word size (32/64 bit). As the I/O bitmap size is in bytes the active check fails for bitmap sizes of 1-3 bytes on 32bit and 1-7 bytes on 64bit.
Use DIV_ROUND_UP() instead.
--- arch/x86/kernel/ptrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/ptrace.c +1 -1
@@ -697,7 +697,7 @@ static int ptrace_set_debugreg(struct tastatic int ioperm_active (struct task_struct *target,const struct user_regset *regset) { -return target->thread.io_bitmap_max / regset->size; +return DIV_ROUND_UP (target->thread.io_bitmap_max, regset->size); } static int ioperm_get(struct task_struct *target,
[tip: x86/iopl] x86/ptrace: Prevent truncation of bitmap size
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: c5d623b17c2424272b3355a524b094ff02cfd9aa Gitweb: https://git.kernel.org/tip/c5d623b17c2424272b3355a524b094ff02cfd9aa Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:41 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:14:59 +01:00
x86/ptrace: Prevent truncation of bitmap size
The active() callback of the IO bitmap regset divides the IO bitmap size by the word size (32/64 bit). As the I/O bitmap size is in bytes the active check fails for bitmap sizes of 1-3 bytes on 32bit and 1-7 bytes on 64bit.
Use DIV_ROUND_UP() instead.
--- arch/x86/kernel/ptrace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/ptrace.c +1 -1
@@ -697,7 +697,7 @@ static int ptrace_set_debugreg(struct task_struct *tsk, int n,static int ioperm_active (struct task_struct *target,const struct user_regset *regset) { -return target->thread.io_bitmap_max / regset->size; +return DIV_ROUND_UP (target->thread.io_bitmap_max, regset->size); } static int ioperm_get(struct task_struct *target,
PATCH v3 2/20 x86/process: Unify copy_thread_tls()
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
While looking at the TSS io bitmap it turned out that any change in that area would require identical changes to copy_thread_tls(). The 32 and 64 bit variants share sufficient code to consolidate them into a common function to avoid duplication of upcoming modifications.
--- arch/x86/include/asm/ptrace.h | 6 ++ arch/x86/include/asm/switch_to.h | 10 ++++ arch/x86/kernel/process.c | 94 +++++++++++++++++++++++++++++++++++++++ arch/x86/kernel/process_32.c | 68 ---------------------------- arch/x86/kernel/process_64.c | 75 ------------------------------- 5 files changed, 110 insertions(+), 143 deletions(-)
arch/x86/include/asm/ptrace.h +6 -0
@@ -361,5 +361,11 @@ extern int do_get_thread_area(struct tasextern int do_set_thread_area (struct task_struct *p,int idx,struct user_desc __user *info,int can_allocate); +#ifdef CONFIG_X86_64 +# define do_set_thread_area_64 (p, s, t) do_arch_prctl_64(p, s, t) +#else +# define do_set_thread_area_64 (p, s, t) (0) +#endif +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_PTRACE_H */
arch/x86/include/asm/switch_to.h +10 -0
@@ -103,7 +103,17 @@ static inline void update_task_stack(strif (static_cpu_has (X86_FEATURE_XENPV ))load_sp0 (task_top_of_stack (task)); #endif +} +static inline void kthread_frame_init (struct inactive_task_frame *frame, +unsigned long fun,unsigned long arg) +{ + frame->bx = fun; +#ifdef CONFIG_X86_32 + frame->di = arg; +#else + frame->r12 = arg; +#endif }#endif /* _ASM_X86_SWITCH_TO_H */
arch/x86/kernel/process.c +94 -0
100 lines not on this page · raw message
arch/x86/kernel/process_32.c +0 -68
74 lines not on this page · raw message
arch/x86/kernel/process_64.c +0 -75
81 lines not on this page · raw message
Re: [patch V3 02/20] x86/process: Unify copy_thread_tls()
Quoted 3 lines · Thomas Gleixner
arch/x86/kernel/process.c +2
-0+int copy_thread_tls (unsigned long clone_flags,unsigned long sp, +unsigned long arg,struct task_struct *p,unsigned long tls)
...
Quoted 1 line
+#ifdef
CONFIG_X86_64
..
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 197
Quoted 4 lines
arch/x86/kernel/process.c +4
-0+#else +/* Clear all status flags including IF and set fixed bit. */ + frame->flags =X86_EFLAGS_FIXED ; +#endif
Hmm. The unification I like, but it also shows these differences that I don't remember the reason for.
Remind me why __switch_to_asm() on 32-bit safes eflags, but we don't do it on x86-64?
The comment just talks about callee-saved registers, but flags isn't callee-saved, so there's something else going on.
This patch clearly doesn't change anything, I'm not complaining about the patch at all. I'm just wondering about the odd difference that the patch exposes.
Linus
Re: [patch V3 02/20] x86/process: Unify copy_thread_tls()
Quoted 18 lines · Linus Torvalds
Quoted 3 lines · Thomas Gleixner
arch/x86/kernel/process.c +2
-0+int copy_thread_tls (unsigned long clone_flags,unsigned long sp, +unsigned long arg,struct task_struct *p,unsigned long tls)...
Quoted 1 line
+#ifdef
CONFIG_X86_64..
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 197
Quoted 4 lines
arch/x86/kernel/process.c +4
-0+#else +/* Clear all status flags including IF and set fixed bit. */ + frame->flags =X86_EFLAGS_FIXED ; +#endif Hmm. The unification I like, but it also shows these differences that I don't remember the reason for.
Remind me why
__switch_to_asm()on 32-bit safes eflags, but we don't do it on x86-64?
Quoted 6 lines
The comment just talks about callee-saved registers, but flags isn't callee-saved, so there's something else going on.
This patch clearly doesn't change anything, I'm not complaining about the patch at all. I'm just wondering about the odd difference that the patch exposes.
See commit: 64604d54d311 ("sched/x86_64: Don't save flags on context switch")
We need "only" make objtool support 32bit :)
Thanks,
tglx
Re: [patch V3 02/20] x86/process: Unify copy_thread_tls()
Quoted 4 lines · Thomas Gleixner
See commit:
64604d54d311("sched/x86_64: Don't save flags on context switch")We need "only" make objtool support 32bit :)
Duh, I knew that.
Maybe just a comment in the structure and/or the __switch_to_asm() so that next time I forget I won't look like such a tool.
The "Save callee-saved registers" comment we have now in the 32-bit __switch_to_asm() really is misleading and incorrect wrt the pushfl.
Linus
Re: [patch V3 02/20] x86/process: Unify copy_thread_tls()
Quoted 6 lines · Linus Torvalds
Quoted 4 lines · Thomas Gleixner
See commit:
64604d54d311("sched/x86_64: Don't save flags on context switch")We need "only" make objtool support 32bit :)
Quoted 7 lines
Duh, I knew that.
Maybe just a comment in the structure and/or the
__switch_to_asm()so that next time I forget I won't look like such a tool.The "Save callee-saved registers" comment we have now in the 32-bit
__switch_to_asm()really is misleading and incorrect wrt the pushfl.
Yeah. Let me fix that.
Thanks,
tglx
Re: [patch V3 02/20] x86/process: Unify copy_thread_tls()
In reply to Linus TorvaldsQuoted 19 lines · Linus Torvalds
Quoted 3 lines · Thomas Gleixner
arch/x86/kernel/process.c +2
-0+int copy_thread_tls (unsigned long clone_flags,unsigned long sp, +unsigned long arg,struct task_struct *p,unsigned long tls)...
Quoted 1 line
+#ifdef
CONFIG_X86_64..
arch/x86/kernel/process.c void exit_thread(struct task_struct *tsk · line 197
Quoted 4 lines
arch/x86/kernel/process.c +4
-0+#else +/* Clear all status flags including IF and set fixed bit. */ + frame->flags =X86_EFLAGS_FIXED ; +#endif Hmm. The unification I like, but it also shows these differences that I don't remember the reason for.
Remind me why
__switch_to_asm()on 32-bit safes eflags, but we don't do it on x86-64?
Quoted 6 lines
The comment just talks about callee-saved registers, but flags isn't callee-saved, so there's something else going on.
This patch clearly doesn't change anything, I'm not complaining about the patch at all. I'm just wondering about the odd difference that the patch exposes.
It's to prevent AC from leaking into the next task. objtool can verify that 64-bit code doesn't schedule when AC is set, but it doesn't work on 32-bit. We could probably just do a CLAC on switch and when it switches back to that task you would get an oops and it would get noticed. It's likely though that the 64-bit coverage has fixed most of the places where this happened.
--
Brian Gerst
[tip: x86/iopl] x86/process: Unify copy_thread_tls()
In reply to Thomas GleixnerThe following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 320044006807464e3e80c11b3afbc08a57767f29 Gitweb: https://git.kernel.org/tip/320044006807464e3e80c11b3afbc08a57767f29 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:42 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:14:59 +01:00
x86/process: Unify copy_thread_tls()
While looking at the TSS io bitmap it turned out that any change in that area would require identical changes to copy_thread_tls(). The 32 and 64 bit variants share sufficient code to consolidate them into a common function to avoid duplication of upcoming modifications.
--- arch/x86/include/asm/ptrace.h | 6 ++- arch/x86/include/asm/switch_to.h | 10 +++- arch/x86/kernel/process.c | 94 +++++++++++++++++++++++++++++++- arch/x86/kernel/process_32.c | 68 +---------------------- arch/x86/kernel/process_64.c | 75 +------------------------- 5 files changed, 110 insertions(+), 143 deletions(-)
arch/x86/include/asm/ptrace.h +6 -0
@@ -361,5 +361,11 @@ extern int do_get_thread_area(struct task_struct *p, int idx,extern int do_set_thread_area (struct task_struct *p,int idx,struct user_desc __user *info,int can_allocate); +#ifdef CONFIG_X86_64 +# define do_set_thread_area_64 (p, s, t) do_arch_prctl_64(p, s, t) +#else +# define do_set_thread_area_64 (p, s, t) (0) +#endif +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_PTRACE_H */
arch/x86/include/asm/switch_to.h +10 -0
@@ -103,7 +103,17 @@ static inline void update_task_stack(struct task_struct *task)if (static_cpu_has (X86_FEATURE_XENPV ))load_sp0 (task_top_of_stack (task)); #endif +} +static inline void kthread_frame_init (struct inactive_task_frame *frame, +unsigned long fun,unsigned long arg) +{ + frame->bx = fun; +#ifdef CONFIG_X86_32 + frame->di = arg; +#else + frame->r12 = arg; +#endif }#endif /* _ASM_X86_SWITCH_TO_H */
arch/x86/kernel/process.c +94 -0
100 lines not on this page · raw message
arch/x86/kernel/process_32.c +0 -68
74 lines not on this page · raw message
arch/x86/kernel/process_64.c +0 -75
81 lines not on this page · raw message
PATCH v3 3/20 x86/cpu: Unify cpu_init()
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
Similar to copy_thread_tls() the 32bit and 64bit implementations of cpu_init() are very similar and unification avoids duplicate changes in the future.
---
V2: Fix 32bit build by removing the pointless #ifdef around the uv header include.
--- arch/x86/kernel/cpu/common.c | 175 ++++++++++++++++--------------------------- 1 file changed, 66 insertions(+), 109 deletions(-)
arch/x86/kernel/cpu/common.c +66 -109
@@ -53,10 +53,7 @@#include <asm/microcode_intel.h> #include <asm/intel-family.h> #include <asm/cpu_device_id.h> - -#ifdef CONFIG_X86_LOCAL_APIC #include <asm/uv/uv.h> -#endif #include "cpu.h"
@@ -1749,7 +1746,7 @@ static void wait_for_master_cpu(int cpu) }#ifdef CONFIG_X86_64 -static void setup_getcpu (int cpu) +static inline void setup_getcpu (int cpu) {unsigned long cpudata =vdso_encode_cpunode (cpu,early_cpu_to_node (cpu));struct desc_struct d = { };
@@ -1769,7 +1766,43 @@ static void setup_getcpu(int cpu)write_gdt_entry (get_cpu_gdt_rw (cpu),GDT_ENTRY_CPUNODE , &d,DESCTYPE_S ); } + +static inline void ucode_cpu_init (int cpu) +{ +if (cpu) +load_ucode_ap (); +} + +static inline void tss_setup_ist (struct tss_struct *tss) +{ +/* Set up the per-CPU TSS IST stacks */ + tss->x86_tss.ist[IST_INDEX_DF ] =__this_cpu_ist_top_va (DF ); + tss->x86_tss.ist[IST_INDEX_NMI ] =__this_cpu_ist_top_va (NMI ); + tss->x86_tss.ist[IST_INDEX_DB ] =__this_cpu_ist_top_va (DB ); + tss->x86_tss.ist[IST_INDEX_MCE ] =__this_cpu_ist_top_va (MCE ); +} + +static inline void gdt_setup_doublefault_tss (int cpu) { } + +#else /* CONFIG_X86_64 */ + +static inline void setup_getcpu (int cpu) { } + +static inline void ucode_cpu_init (int cpu) +{ +show_ucode_info_early (); +} + +static inline void tss_setup_ist (struct tss_struct *tss) { } + +static inline void gdt_setup_doublefault_tss (int cpu) +{ +#ifdef CONFIG_DOUBLEFAULT +/* Set up the doublefault TSS pointer in the GDT */ +__set_tss_desc (cpu,GDT_ENTRY_DOUBLEFAULT_TSS , &doublefault_tss);#endif +} +#endif /* !CONFIG_X86_64 */ /* * cpu_init() initializes state that is per-CPU. Some data is already
@@ -1777,21 +1810,15 @@ static void setup_getcpu(int cpu)* and IDT. We reload them nevertheless, this function acts as a * 'CPU state barrier', nothing should get across. */ -#ifdef CONFIG_X86_64 -void cpu_init (void ) { +struct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw); +struct task_struct *cur = current;int cpu =raw_smp_processor_id (); -struct task_struct *me; -struct tss_struct *t; -int i;wait_for_master_cpu (cpu); -if (cpu) -load_ucode_ap (); - - t = &per_cpu (cpu_tss_rw, cpu); +ucode_cpu_init (cpu);#ifdef CONFIG_NUMA if (this_cpu_read(numa_node) == 0 &&
@@ -1800,63 +1827,48 @@ void cpu_init(void)#endif setup_getcpu (cpu); - me = current; -pr_debug ("Initializing CPU#%d\n" , cpu); -cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE ); +if (IS_ENABLED (CONFIG_X86_64 ) ||cpu_feature_enabled (X86_FEATURE_VME ) || +boot_cpu_has (X86_FEATURE_TSC ) ||boot_cpu_has (X86_FEATURE_DE )) +cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE );/* * Initialize the per-CPU GDT with the boot GDT, * and set up the GDT descriptor: */ -switch_to_new_gdt (cpu); -loadsegment (fs,0 ); -load_current_idt (); -memset (me->thread.tls_array,0 ,GDT_ENTRY_TLS_ENTRIES *8 ); -syscall_init (); +if (IS_ENABLED (CONFIG_X86_64 )) { +loadsegment (fs,0 ); +memset (cur->thread.tls_array,0 ,GDT_ENTRY_TLS_ENTRIES *8 ); +syscall_init (); + +wrmsrl (MSR_FS_BASE ,0 ); +wrmsrl (MSR_KERNEL_GS_BASE ,0 ); +barrier (); -wrmsrl (MSR_FS_BASE ,0 ); -wrmsrl (MSR_KERNEL_GS_BASE ,0 ); -barrier (); - -x86_configure_nx (); -x2apic_setup (); - -/* -* set up and load the per-CPU TSS -*/ -if (!t->x86_tss.ist[0 ]) { - t->x86_tss.ist[IST_INDEX_DF ] =__this_cpu_ist_top_va (DF ); - t->x86_tss.ist[IST_INDEX_NMI ] =__this_cpu_ist_top_va (NMI ); - t->x86_tss.ist[IST_INDEX_DB ] =__this_cpu_ist_top_va (DB ); - t->x86_tss.ist[IST_INDEX_MCE ] =__this_cpu_ist_top_va (MCE ); +x2apic_setup (); } - t->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; - -/* -* <= is required because the CPU will access up to -* 8 bits beyond the end of the IO permission bitmap. -*/ -for (i =0 ; i <=IO_BITMAP_LONGS ; i++) - t->io_bitmap[i] = ~0UL ; -mmgrab (&init_mm); - me->active_mm = &init_mm; -BUG_ON (me->mm); + cur->active_mm = &init_mm; +BUG_ON (cur->mm);initialize_tlbstate_and_flush (); -enter_lazy_tlb (&init_mm, me); +enter_lazy_tlb (&init_mm, cur); -/* -* Initialize the TSS. sp0 points to the entry trampoline stack -* regardless of what task is running. -*/ +/* Initialize the TSS. */ +tss_setup_ist (tss); + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; +memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss); +load_TR_desc (); +/* +* sp0 points to the entry trampoline stack regardless of what task +* is running. +*/ load_sp0 ((unsigned long )(cpu_entry_stack (cpu) +1 ));load_mm_ldt (&init_mm);
@@ -1864,6 +1876,8 @@ void cpu_init(void)clear_all_debug_regs ();dbg_restore_debug_regs (); +gdt_setup_doublefault_tss (cpu); +fpu__init_cpu ();if (is_uv_system ())
@@ -1872,63 +1886,6 @@ void cpu_init(void)load_fixmap_gdt (cpu); } -#else - -void cpu_init (void ) -{ -int cpu =smp_processor_id (); -struct task_struct *curr = current; -struct tss_struct *t = &per_cpu (cpu_tss_rw, cpu); - -wait_for_master_cpu (cpu); - -show_ucode_info_early (); - -pr_info ("Initializing CPU#%d\n" , cpu); - -if (cpu_feature_enabled (X86_FEATURE_VME ) || -boot_cpu_has (X86_FEATURE_TSC ) || -boot_cpu_has (X86_FEATURE_DE )) -cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE ); - -load_current_idt (); -switch_to_new_gdt (cpu); - -/* -* Set up and load the per-CPU TSS and LDT -*/ -mmgrab (&init_mm); - curr->active_mm = &init_mm; -BUG_ON (curr->mm); -initialize_tlbstate_and_flush (); -enter_lazy_tlb (&init_mm, curr); - -/* -* Initialize the TSS. sp0 points to the entry trampoline stack -* regardless of what task is running. -*/ -set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss); -load_TR_desc (); -load_sp0 ((unsigned long )(cpu_entry_stack (cpu) +1 )); - -load_mm_ldt (&init_mm); - - t->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; - -#ifdef CONFIG_DOUBLEFAULT -/* Set up doublefault TSS pointer in the GDT */ -__set_tss_desc (cpu,GDT_ENTRY_DOUBLEFAULT_TSS , &doublefault_tss); -#endif - -clear_all_debug_regs (); -dbg_restore_debug_regs (); - -fpu__init_cpu (); - -load_fixmap_gdt (cpu); -} -#endif -/* * The microcode loader calls this upon late microcode load to recheck features, * only when microcode has been updated. Caller holds microcode_mutex and CPU
[tip: x86/iopl] x86/cpu: Unify cpu_init()
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: fff628d736f261ab0bebb94f97678710f48df75e Gitweb: https://git.kernel.org/tip/fff628d736f261ab0bebb94f97678710f48df75e Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:43 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:00 +01:00
x86/cpu: Unify cpu_init()
Similar to copy_thread_tls() the 32bit and 64bit implementations of cpu_init() are very similar and unification avoids duplicate changes in the future.
--- arch/x86/kernel/cpu/common.c | 173 ++++++++++++---------------------- 1 file changed, 65 insertions(+), 108 deletions(-)
arch/x86/kernel/cpu/common.c +65 -108
@@ -53,10 +53,7 @@#include <asm/microcode_intel.h> #include <asm/intel-family.h> #include <asm/cpu_device_id.h> - -#ifdef CONFIG_X86_LOCAL_APIC #include <asm/uv/uv.h> -#endif #include "cpu.h"
@@ -1749,7 +1746,7 @@ static void wait_for_master_cpu(int cpu) }#ifdef CONFIG_X86_64 -static void setup_getcpu (int cpu) +static inline void setup_getcpu (int cpu) {unsigned long cpudata =vdso_encode_cpunode (cpu,early_cpu_to_node (cpu));struct desc_struct d = { };
@@ -1769,7 +1766,43 @@ static void setup_getcpu(int cpu)write_gdt_entry (get_cpu_gdt_rw (cpu),GDT_ENTRY_CPUNODE , &d,DESCTYPE_S ); } + +static inline void ucode_cpu_init (int cpu) +{ +if (cpu) +load_ucode_ap (); +} + +static inline void tss_setup_ist (struct tss_struct *tss) +{ +/* Set up the per-CPU TSS IST stacks */ + tss->x86_tss.ist[IST_INDEX_DF ] =__this_cpu_ist_top_va (DF ); + tss->x86_tss.ist[IST_INDEX_NMI ] =__this_cpu_ist_top_va (NMI ); + tss->x86_tss.ist[IST_INDEX_DB ] =__this_cpu_ist_top_va (DB ); + tss->x86_tss.ist[IST_INDEX_MCE ] =__this_cpu_ist_top_va (MCE ); +} + +static inline void gdt_setup_doublefault_tss (int cpu) { } + +#else /* CONFIG_X86_64 */ + +static inline void setup_getcpu (int cpu) { } + +static inline void ucode_cpu_init (int cpu) +{ +show_ucode_info_early (); +} + +static inline void tss_setup_ist (struct tss_struct *tss) { } + +static inline void gdt_setup_doublefault_tss (int cpu) +{ +#ifdef CONFIG_DOUBLEFAULT +/* Set up the doublefault TSS pointer in the GDT */ +__set_tss_desc (cpu,GDT_ENTRY_DOUBLEFAULT_TSS , &doublefault_tss);#endif +} +#endif /* !CONFIG_X86_64 */ /* * cpu_init() initializes state that is per-CPU. Some data is already
@@ -1777,21 +1810,15 @@ static void setup_getcpu(int cpu)* and IDT. We reload them nevertheless, this function acts as a * 'CPU state barrier', nothing should get across. */ -#ifdef CONFIG_X86_64 -void cpu_init (void ) { +struct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw); +struct task_struct *cur = current;int cpu =raw_smp_processor_id (); -struct task_struct *me; -struct tss_struct *t; -int i;wait_for_master_cpu (cpu); -if (cpu) -load_ucode_ap (); - - t = &per_cpu (cpu_tss_rw, cpu); +ucode_cpu_init (cpu);#ifdef CONFIG_NUMA if (this_cpu_read(numa_node) == 0 &&
@@ -1800,63 +1827,48 @@ void cpu_init(void)#endif setup_getcpu (cpu); - me = current; -pr_debug ("Initializing CPU#%d\n" , cpu); -cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE ); +if (IS_ENABLED (CONFIG_X86_64 ) ||cpu_feature_enabled (X86_FEATURE_VME ) || +boot_cpu_has (X86_FEATURE_TSC ) ||boot_cpu_has (X86_FEATURE_DE )) +cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE );/* * Initialize the per-CPU GDT with the boot GDT, * and set up the GDT descriptor: */ -switch_to_new_gdt (cpu); -loadsegment (fs,0 ); -load_current_idt (); -memset (me->thread.tls_array,0 ,GDT_ENTRY_TLS_ENTRIES *8 ); -syscall_init (); +if (IS_ENABLED (CONFIG_X86_64 )) { +loadsegment (fs,0 ); +memset (cur->thread.tls_array,0 ,GDT_ENTRY_TLS_ENTRIES *8 ); +syscall_init (); -wrmsrl (MSR_FS_BASE ,0 ); -wrmsrl (MSR_KERNEL_GS_BASE ,0 ); -barrier (); +wrmsrl (MSR_FS_BASE ,0 ); +wrmsrl (MSR_KERNEL_GS_BASE ,0 ); +barrier (); -x86_configure_nx (); -x2apic_setup (); - -/* -* set up and load the per-CPU TSS -*/ -if (!t->x86_tss.ist[0 ]) { - t->x86_tss.ist[IST_INDEX_DF ] =__this_cpu_ist_top_va (DF ); - t->x86_tss.ist[IST_INDEX_NMI ] =__this_cpu_ist_top_va (NMI ); - t->x86_tss.ist[IST_INDEX_DB ] =__this_cpu_ist_top_va (DB ); - t->x86_tss.ist[IST_INDEX_MCE ] =__this_cpu_ist_top_va (MCE ); +x2apic_setup (); } - t->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; - -/* -* <= is required because the CPU will access up to -* 8 bits beyond the end of the IO permission bitmap. -*/ -for (i =0 ; i <=IO_BITMAP_LONGS ; i++) - t->io_bitmap[i] = ~0UL ; -mmgrab (&init_mm); - me->active_mm = &init_mm; -BUG_ON (me->mm); + cur->active_mm = &init_mm; +BUG_ON (cur->mm);initialize_tlbstate_and_flush (); -enter_lazy_tlb (&init_mm, me); +enter_lazy_tlb (&init_mm, cur); -/* -* Initialize the TSS. sp0 points to the entry trampoline stack -* regardless of what task is running. -*/ +/* Initialize the TSS. */ +tss_setup_ist (tss); + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; +memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss); +load_TR_desc (); +/* +* sp0 points to the entry trampoline stack regardless of what task +* is running. +*/ load_sp0 ((unsigned long )(cpu_entry_stack (cpu) +1 ));load_mm_ldt (&init_mm);
@@ -1864,6 +1876,8 @@ void cpu_init(void)clear_all_debug_regs ();dbg_restore_debug_regs (); +gdt_setup_doublefault_tss (cpu); +fpu__init_cpu ();if (is_uv_system ())
@@ -1872,63 +1886,6 @@ void cpu_init(void)load_fixmap_gdt (cpu); } -#else - -void cpu_init (void ) -{ -int cpu =smp_processor_id (); -struct task_struct *curr = current; -struct tss_struct *t = &per_cpu (cpu_tss_rw, cpu); - -wait_for_master_cpu (cpu); - -show_ucode_info_early (); - -pr_info ("Initializing CPU#%d\n" , cpu); - -if (cpu_feature_enabled (X86_FEATURE_VME ) || -boot_cpu_has (X86_FEATURE_TSC ) || -boot_cpu_has (X86_FEATURE_DE )) -cr4_clear_bits (X86_CR4_VME |X86_CR4_PVI |X86_CR4_TSD |X86_CR4_DE ); - -load_current_idt (); -switch_to_new_gdt (cpu); - -/* -* Set up and load the per-CPU TSS and LDT -*/ -mmgrab (&init_mm); - curr->active_mm = &init_mm; -BUG_ON (curr->mm); -initialize_tlbstate_and_flush (); -enter_lazy_tlb (&init_mm, curr); - -/* -* Initialize the TSS. sp0 points to the entry trampoline stack -* regardless of what task is running. -*/ -set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss); -load_TR_desc (); -load_sp0 ((unsigned long )(cpu_entry_stack (cpu) +1 )); - -load_mm_ldt (&init_mm); - - t->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; - -#ifdef CONFIG_DOUBLEFAULT -/* Set up doublefault TSS pointer in the GDT */ -__set_tss_desc (cpu,GDT_ENTRY_DOUBLEFAULT_TSS , &doublefault_tss); -#endif - -clear_all_debug_regs (); -dbg_restore_debug_regs (); - -fpu__init_cpu (); - -load_fixmap_gdt (cpu); -} -#endif -/* * The microcode loader calls this upon late microcode load to recheck features, * only when microcode has been updated. Caller holds microcode_mutex and CPU
PATCH v3 4/20 x86/tss: Fix and move VMX BUILD_BUG_ON()
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
The BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 == 0x67) in the VMX code is bogus in two aspects:
1) This wants to be in generic x86 code simply to catch issues even when VMX is disabled in Kconfig.
2) The IO_BITMAP_OFFSET is not the right thing to check because it makes asssumptions about the layout of tss_struct. Nothing requires that the I/O bitmap is placed right after x86_tss, which is the hardware mandated tss structure. It pointlessly makes restrictions on the struct tss_struct layout.
The proper thing to check is:
- Offset of x86_tss in tss_struct is 0
- Size of x86_tss == 0x68
Move it to the other build time TSS checks and make it do the right thing.
---
V2: New patch
--- arch/x86/kvm/vmx/vmx.c | 8 -------- arch/x86/mm/cpu_entry_area.c | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/vmx.c +0 -8
@@ -1338,14 +1338,6 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu (unsigned long)&get_cpu_entry_area(cpu)->tss.x86_tss);vmcs_writel (HOST_GDTR_BASE , (unsigned long )gdt);/* 22.2.4 */ -/* -* VM exits change the host TR limit to 0x67 after a VM -* exit. This is okay, since 0x67 covers everything except -* the IO bitmap and have have code to handle the IO bitmap -* being lost after a VM exit. -*/ -BUILD_BUG_ON (IO_BITMAP_OFFSET -1 !=0x67 ); -rdmsrl (MSR_IA32_SYSENTER_ESP , sysenter_esp);vmcs_writel (HOST_IA32_SYSENTER_ESP , sysenter_esp);/* 22.2.3 */
arch/x86/mm/cpu_entry_area.c +8 -0
@@ -161,6 +161,14 @@ static void __init setup_cpu_entry_area(BUILD_BUG_ON ((offsetof(struct tss_struct , x86_tss) ^ offsetofend(struct tss_struct, x86_tss)) & PAGE_MASK);BUILD_BUG_ON (sizeof (struct tss_struct ) %PAGE_SIZE !=0 ); +/* +* VMX changes the host TR limit to 0x67 after a VM exit. This is +* okay, since 0x67 covers the size of struct x86_hw_tss. Make sure +* that this is correct. +*/ +BUILD_BUG_ON (offsetof(struct tss_struct , x86_tss) !=0 ); +BUILD_BUG_ON (sizeof (struct x86_hw_tss ) !=0x68 ); +cea_map_percpu_pages (&cea->tss, &per_cpu (cpu_tss_rw, cpu),sizeof (struct tss_struct ) /PAGE_SIZE , tss_prot);
[tip: x86/iopl] x86/tss: Fix and move VMX BUILD_BUG_ON()
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 4dee588dc12914966faf902932238cb09a493d1c Gitweb: https://git.kernel.org/tip/4dee588dc12914966faf902932238cb09a493d1c Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:44 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:00 +01:00
x86/tss: Fix and move VMX BUILD_BUG_ON()
The BUILD_BUG_ON(IO_BITMAP_OFFSET - 1 == 0x67) in the VMX code is bogus in two aspects:
1) This wants to be in generic x86 code simply to catch issues even when VMX is disabled in Kconfig.
2) The IO_BITMAP_OFFSET is not the right thing to check because it makes asssumptions about the layout of tss_struct. Nothing requires that the I/O bitmap is placed right after x86_tss, which is the hardware mandated tss structure. It pointlessly makes restrictions on the struct tss_struct layout.
The proper thing to check is:
- Offset of x86_tss in tss_struct is 0
- Size of x86_tss == 0x68
Move it to the other build time TSS checks and make it do the right thing.
--- arch/x86/kvm/vmx/vmx.c | 8 -------- arch/x86/mm/cpu_entry_area.c | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-)
arch/x86/kvm/vmx/vmx.c +0 -8
@@ -1338,14 +1338,6 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu) (unsigned long)&get_cpu_entry_area(cpu)->tss.x86_tss);vmcs_writel (HOST_GDTR_BASE , (unsigned long )gdt);/* 22.2.4 */ -/* -* VM exits change the host TR limit to 0x67 after a VM -* exit. This is okay, since 0x67 covers everything except -* the IO bitmap and have have code to handle the IO bitmap -* being lost after a VM exit. -*/ -BUILD_BUG_ON (IO_BITMAP_OFFSET -1 !=0x67 ); -rdmsrl (MSR_IA32_SYSENTER_ESP , sysenter_esp);vmcs_writel (HOST_IA32_SYSENTER_ESP , sysenter_esp);/* 22.2.3 */
arch/x86/mm/cpu_entry_area.c +8 -0
@@ -161,6 +161,14 @@ static void __init setup_cpu_entry_area(unsigned int cpu)BUILD_BUG_ON ((offsetof(struct tss_struct , x86_tss) ^ offsetofend(struct tss_struct, x86_tss)) & PAGE_MASK);BUILD_BUG_ON (sizeof (struct tss_struct ) %PAGE_SIZE !=0 ); +/* +* VMX changes the host TR limit to 0x67 after a VM exit. This is +* okay, since 0x67 covers the size of struct x86_hw_tss. Make sure +* that this is correct. +*/ +BUILD_BUG_ON (offsetof(struct tss_struct , x86_tss) !=0 ); +BUILD_BUG_ON (sizeof (struct x86_hw_tss ) !=0x68 ); +cea_map_percpu_pages (&cea->tss, &per_cpu (cpu_tss_rw, cpu),sizeof (struct tss_struct ) /PAGE_SIZE , tss_prot);
PATCH v3 5/20 x86/iopl: Cleanup include maze
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
Get rid of superfluous includes.
---
V2: New patch
--- arch/x86/kernel/ioport.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-)
arch/x86/kernel/ioport.c +4 -12
@@ -3,22 +3,14 @@* This contains the io-permission bitmap code - written by obz, with changes * by Linus. 32/64 bits code unification by Miguel Botón. */ - -#include <linux/sched.h> -#include <linux/sched/task_stack.h> -#include <linux/kernel.h> #include <linux/capability.h> -#include <linux/errno.h> -#include <linux/types.h> -#include <linux/ioport.h> #include <linux/security.h> -#include <linux/smp.h> -#include <linux/stddef.h> -#include <linux/slab.h> -#include <linux/thread_info.h> #include <linux/syscalls.h> #include <linux/bitmap.h> -#include <asm/syscalls.h> +#include <linux/ioport.h> +#include <linux/sched.h> +#include <linux/slab.h> +#include <asm/desc.h> /*
[tip: x86/iopl] x86/iopl: Cleanup include maze
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 2a03b5d8fe438f798aa4c764779d3d8e0e0710d5 Gitweb: https://git.kernel.org/tip/2a03b5d8fe438f798aa4c764779d3d8e0e0710d5 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:45 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:01 +01:00
x86/iopl: Cleanup include maze
Get rid of superfluous includes.
--- arch/x86/kernel/ioport.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-)
arch/x86/kernel/ioport.c +4 -12
@@ -3,22 +3,14 @@* This contains the io-permission bitmap code - written by obz, with changes * by Linus. 32/64 bits code unification by Miguel Botón. */ - -#include <linux/sched.h> -#include <linux/sched/task_stack.h> -#include <linux/kernel.h> #include <linux/capability.h> -#include <linux/errno.h> -#include <linux/types.h> -#include <linux/ioport.h> #include <linux/security.h> -#include <linux/smp.h> -#include <linux/stddef.h> -#include <linux/slab.h> -#include <linux/thread_info.h> #include <linux/syscalls.h> #include <linux/bitmap.h> -#include <asm/syscalls.h> +#include <linux/ioport.h> +#include <linux/sched.h> +#include <linux/slab.h> +#include <asm/desc.h> /*
PATCH v3 6/20 x86/ioperm: Simplify first ioperm() invocation logic
In reply to Thomas GleixnerOn the first allocation of a task the I/O bitmap needs to be allocated. After the allocation it is installed as an empty bitmap and immediately afterwards updated.
Avoid that and just do the initial updates (store bitmap pointer, set TIF flag and make TSS limit valid) in the update path unconditionally. If the bitmap was already active this is redundant but harmless.
Preparatory change for later optimizations in the context switch code.
---
V3: Split out from a later patch for simpler review and to make the first
context switch optimization correct.
--- arch/x86/kernel/ioport.c | 57 +++++++++++++++++++++++------------------------ 1 file changed, 28 insertions(+), 29 deletions(-)
arch/x86/kernel/ioport.c +28 -29
@@ -18,9 +18,10 @@*/ long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) { +unsigned int i, max_long, bytes, bytes_updated;struct thread_struct *t = ¤t->thread;struct tss_struct *tss; -unsigned int i, max_long, bytes, bytes_updated; +unsigned long *bitmap;if ((from + num <= from) || (from + num >IO_BITMAP_BITS ))return -EINVAL ;
@@ -33,59 +34,57 @@ long ksys_ioperm(unsigned long from, uns* IO bitmap up. ioperm() is much less timing critical than clone(), * this is why we delay this operation until now: */ -if (!t->io_bitmap_ptr) { -unsigned long *bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL ); - + bitmap = t->io_bitmap_ptr; +if (!bitmap) { + bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );if (!bitmap)return -ENOMEM ;memset (bitmap,0xff ,IO_BITMAP_BYTES ); - t->io_bitmap_ptr = bitmap; -set_thread_flag (TIF_IO_BITMAP ); - -/* -* Now that we have an IO bitmap, we need our TSS limit to be -* correct. It's fine if we are preempted after doing this: -* with TIF_IO_BITMAP set, context switches will keep our TSS -* limit correct. -*/ -preempt_disable (); -refresh_tss_limit (); -preempt_enable (); }/* -* do it in the per-thread copy and in the TSS ... -* -* Disable preemption via get_cpu() - we must not switch away -* because the ->io_bitmap_max value must match the bitmap -* contents: +* Update the bitmap and the TSS copy with preemption disabled to +* prevent a race against context switch. */ - tss = &per_cpu (cpu_tss_rw,get_cpu ()); - +preempt_disable ();if (turn_on) -bitmap_clear (t->io_bitmap_ptr, from, num); +bitmap_clear (bitmap, from, num);else -bitmap_set (t->io_bitmap_ptr, from, num); +bitmap_set (bitmap, from, num);/* * Search for a (possibly new) maximum. This is simple and stupid, * to keep it obviously correct: */ max_long =0 ; -for (i =0 ; i <IO_BITMAP_LONGS ; i++) -if (t->io_bitmap_ptr[i] != ~0UL ) +for (i =0 ; i <IO_BITMAP_LONGS ; i++) { +if (bitmap[i] != ~0UL ) max_long = i; + } bytes = (max_long +1 ) *sizeof (unsigned long ); bytes_updated =max (bytes, t->io_bitmap_max); +/* Update the thread data */ t->io_bitmap_max = bytes; +/* +* Store the bitmap pointer (might be the same if the task already +* head one). Set the TIF flag, just in case this is the first +* invocation. +*/ + t->io_bitmap_ptr = bitmap; +set_thread_flag (TIF_IO_BITMAP ); -/* Update the TSS: */ +/* Update the TSS */ + tss =this_cpu_ptr (&cpu_tss_rw);memcpy (tss->io_bitmap, t->io_bitmap_ptr, bytes_updated); +/* Store the new end of the zero bits */ + tss->io_bitmap_prev_max = bytes; +/* Make sure the TSS limit covers the I/O bitmap. */ +refresh_tss_limit (); -put_cpu (); +preempt_enable ();return 0 ; }
kbuild test robot
build botRe: [patch V3 06/20] x86/ioperm: Simplify first ioperm() invocation logic
I love your patch! Yet something to improve:
Hi Thomas,
I love your patch! Yet something to improve:
[auto build test ERROR on tip/x86/core]
[cannot apply to v5.4-rc7 next-20191111]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Thomas-Gleixner/x86-iopl-Prevent-user-space-from-using-CLI-STI-with-iopl-3/20191114-053418 base: https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git 004e8dce9c5595697951f7cd0e9f66b35c92265e config: i386-tinyconfig (attached as .config) compiler: gcc-7 (Debian 7.4.0-14) 7.4.0 reproduce: # save the attached .config to linux build tree make ARCH=i386
If you fix the issue, kindly add following tag
Note: the linux-review/Thomas-Gleixner/x86-iopl-Prevent-user-space-from-using-CLI-STI-with-iopl-3/20191114-053418 HEAD b14095464382fae4bf55028eb1cebe0e3cc1b9c8 builds fine.
It only hurts bisectibility.
All errors (new ones prefixed by >>):
arch/x86/kernel/ioport.c: In function 'ksys_ioperm':
>> arch/x86/kernel/ioport.c:83:7: error: 'struct tss_struct' has no member named 'io_bitmap_prev_max'; did you mean 'io_bitmap'?
tss->io_bitmap_prev_max = bytes;
^~~~~~~~~~~~~~~~~~
io_bitmapvim +83 arch/x86/kernel/ioport.c
15
16 /*
17 * this changes the io permissions bitmap in the current task.
18 */
19 long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)
20 {
21 unsigned int i, max_long, bytes, bytes_updated;
22 struct thread_struct *t = ¤t->thread;
23 struct tss_struct *tss;
24 unsigned long *bitmap;
25
26 if ((from + num <= from) || (from + num > IO_BITMAP_BITS))
27 return -EINVAL;
28 if (turn_on && (!capable(CAP_SYS_RAWIO) ||
29 security_locked_down(LOCKDOWN_IOPORT)))
30 return -EPERM;
31
32 /*
33 * If it's the first ioperm() call in this thread's lifetime, set the
34 * IO bitmap up. ioperm() is much less timing critical than clone(),
35 * this is why we delay this operation until now:
36 */
37 bitmap = t->io_bitmap_ptr;
38 if (!bitmap) {
39 bitmap = kmalloc(IO_BITMAP_BYTES, GFP_KERNEL);
40 if (!bitmap)
41 return -ENOMEM;
42
43 memset(bitmap, 0xff, IO_BITMAP_BYTES);
44 }
45
46 /*
47 * Update the bitmap and the TSS copy with preemption disabled to
48 * prevent a race against context switch.
49 */
50 preempt_disable();
51 if (turn_on)
52 bitmap_clear(bitmap, from, num);
53 else
54 bitmap_set(bitmap, from, num);
55
56 /*
57 * Search for a (possibly new) maximum. This is simple and stupid,
58 * to keep it obviously correct:
59 */
60 max_long = 0;
61 for (i = 0; i < IO_BITMAP_LONGS; i++) {
62 if (bitmap[i] != ~0UL)
63 max_long = i;
64 }
65
66 bytes = (max_long + 1) * sizeof(unsigned long);
67 bytes_updated = max(bytes, t->io_bitmap_max);
68
69 /* Update the thread data */
70 t->io_bitmap_max = bytes;
71 /*
72 * Store the bitmap pointer (might be the same if the task already
73 * head one). Set the TIF flag, just in case this is the first
74 * invocation.
75 */
76 t->io_bitmap_ptr = bitmap;
77 set_thread_flag(TIF_IO_BITMAP);
78
79 /* Update the TSS */
80 tss = this_cpu_ptr(&cpu_tss_rw);
81 memcpy(tss->io_bitmap, t->io_bitmap_ptr, bytes_updated);
82 /* Store the new end of the zero bits */
> 83 tss->io_bitmap_prev_max = bytes;
84 /* Make sure the TSS limit covers the I/O bitmap. */
85 refresh_tss_limit();
86
87 preempt_enable();
88
89 return 0;
90 }
91 ---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org Intel Corporation
[tip: x86/iopl] x86/ioperm: Simplify first ioperm() invocation logic
In reply to Thomas GleixnerThe following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 3817927e6f5686a5c7571ae33f7b1a8483821c5b Gitweb: https://git.kernel.org/tip/3817927e6f5686a5c7571ae33f7b1a8483821c5b Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:46 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:01 +01:00
x86/ioperm: Simplify first ioperm() invocation logic
On the first allocation of a task the I/O bitmap needs to be allocated. After the allocation it is installed as an empty bitmap and immediately afterwards updated.
Avoid that and just do the initial updates (store bitmap pointer, set TIF flag and make TSS limit valid) in the update path unconditionally. If the bitmap was already active this is redundant but harmless.
Preparatory change for later optimizations in the context switch code.
--- arch/x86/kernel/ioport.c | 55 ++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 29 deletions(-)
arch/x86/kernel/ioport.c +26 -29
@@ -18,9 +18,10 @@*/ long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) { +unsigned int i, max_long, bytes, bytes_updated;struct thread_struct *t = ¤t->thread;struct tss_struct *tss; -unsigned int i, max_long, bytes, bytes_updated; +unsigned long *bitmap;if ((from + num <= from) || (from + num >IO_BITMAP_BITS ))return -EINVAL ;
@@ -33,59 +34,55 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)* IO bitmap up. ioperm() is much less timing critical than clone(), * this is why we delay this operation until now: */ -if (!t->io_bitmap_ptr) { -unsigned long *bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL ); - + bitmap = t->io_bitmap_ptr; +if (!bitmap) { + bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );if (!bitmap)return -ENOMEM ;memset (bitmap,0xff ,IO_BITMAP_BYTES ); - t->io_bitmap_ptr = bitmap; -set_thread_flag (TIF_IO_BITMAP ); - -/* -* Now that we have an IO bitmap, we need our TSS limit to be -* correct. It's fine if we are preempted after doing this: -* with TIF_IO_BITMAP set, context switches will keep our TSS -* limit correct. -*/ -preempt_disable (); -refresh_tss_limit (); -preempt_enable (); }/* -* do it in the per-thread copy and in the TSS ... -* -* Disable preemption via get_cpu() - we must not switch away -* because the ->io_bitmap_max value must match the bitmap -* contents: +* Update the bitmap and the TSS copy with preemption disabled to +* prevent a race against context switch. */ - tss = &per_cpu (cpu_tss_rw,get_cpu ()); - +preempt_disable ();if (turn_on) -bitmap_clear (t->io_bitmap_ptr, from, num); +bitmap_clear (bitmap, from, num);else -bitmap_set (t->io_bitmap_ptr, from, num); +bitmap_set (bitmap, from, num);/* * Search for a (possibly new) maximum. This is simple and stupid, * to keep it obviously correct: */ max_long =0 ; -for (i =0 ; i <IO_BITMAP_LONGS ; i++) -if (t->io_bitmap_ptr[i] != ~0UL ) +for (i =0 ; i <IO_BITMAP_LONGS ; i++) { +if (bitmap[i] != ~0UL ) max_long = i; + } bytes = (max_long +1 ) *sizeof (unsigned long ); bytes_updated =max (bytes, t->io_bitmap_max); +/* Update the thread data */ t->io_bitmap_max = bytes; +/* +* Store the bitmap pointer (might be the same if the task already +* head one). Set the TIF flag, just in case this is the first +* invocation. +*/ + t->io_bitmap_ptr = bitmap; +set_thread_flag (TIF_IO_BITMAP ); -/* Update the TSS: */ +/* Update the TSS */ + tss =this_cpu_ptr (&cpu_tss_rw);memcpy (tss->io_bitmap, t->io_bitmap_ptr, bytes_updated); +/* Make sure the TSS limit covers the I/O bitmap. */ +refresh_tss_limit (); -put_cpu (); +preempt_enable ();return 0 ; }
PATCH v3 7/20 x86/ioperm: Avoid bitmap allocation if no permissions are set
In reply to Thomas GleixnerIf ioperm() is invoked the first time and the @turn_on argument is 0, then there is no point to allocate a bitmap just to clear permissions which are not set.
---
V3: Split out from large combo patch
--- arch/x86/kernel/ioport.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kernel/ioport.c +3 -0
@@ -36,6 +36,9 @@ long ksys_ioperm(unsigned long from, uns*/ bitmap = t->io_bitmap_ptr;if (!bitmap) { +/* No point to allocate a bitmap just to clear permissions */ +if (!turn_on) +return 0 ; bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );if (!bitmap)return -ENOMEM ;
[tip: x86/iopl] x86/ioperm: Avoid bitmap allocation if no permissions are set
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 5487da6bf0775d996352442b89a8defbd671b4ae Gitweb: https://git.kernel.org/tip/5487da6bf0775d996352442b89a8defbd671b4ae Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:47 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:02 +01:00
x86/ioperm: Avoid bitmap allocation if no permissions are set
If ioperm() is invoked the first time and the @turn_on argument is 0, then there is no point to allocate a bitmap just to clear permissions which are not set.
--- arch/x86/kernel/ioport.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kernel/ioport.c +3 -0
@@ -36,6 +36,9 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)*/ bitmap = t->io_bitmap_ptr;if (!bitmap) { +/* No point to allocate a bitmap just to clear permissions */ +if (!turn_on) +return 0 ; bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL );if (!bitmap)return -ENOMEM ;
PATCH v3 8/20 x86/io: Speedup schedule out of I/O bitmap user
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
There is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the io_bitmap_base member of the tss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.
If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
---
V3: Moved the changes required to make the first invocation of ioperm()
correct into a seperate patch. (Incorrectness pointed out by Andy)
--- arch/x86/include/asm/processor.h | 38 ++++++++++++++++------- arch/x86/kernel/cpu/common.c | 3 + arch/x86/kernel/doublefault.c | 2 - arch/x86/kernel/ioport.c | 2 + arch/x86/kernel/process.c | 63 ++++++++++++++++++++++----------------- 5 files changed, 67 insertions(+), 41 deletions(-)
arch/x86/include/asm/processor.h +26 -12
@@ -330,8 +330,23 @@ struct x86_hw_tss {#define IO_BITMAP_BITS 65536#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) -#define IO_BITMAP_OFFSET (offsetof(struct tss_struct, io_bitmap) - offsetof(struct tss_struct, x86_tss)) -#define INVALID_IO_BITMAP_OFFSET 0x8000 + +#define IO_BITMAP_OFFSET_VALID \ + (offsetof(struct tss_struct, io_bitmap) - \ + offsetof(struct tss_struct, x86_tss)) + +/* +* sizeof(unsigned long) coming from an extra "long" at the end +* of the iobitmap. +* +* -1? seg base+limit should be pointing to the address of the +* last valid byte +*/ +#define __KERNEL_TSS_LIMIT \ + (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + +/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ +#define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)struct entry_stack {unsigned long words[64 ];
@@ -350,6 +365,15 @@ struct tss_struct {struct x86_hw_tss x86_tss;/* +* Store the dirty size of the last io bitmap offender. The next +* one will have to do the cleanup as the switch out to a non io +* bitmap user will just set x86_tss.io_bitmap_base to a value +* outside of the TSS limit. So for sane tasks there is no need to +* actually touch the io_bitmap at all. +*/ +unsigned int io_bitmap_prev_max; + +/* * The extra 1 is there because the CPU will access an * additional byte beyond the end of the IO permission * bitmap. The extra byte must be all 1 bits, and must
@@ -360,16 +384,6 @@ struct tss_struct { DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw); -/* -* sizeof(unsigned long) coming from an extra "long" at the end -* of the iobitmap. -* -* -1? seg base+limit should be pointing to the address of the -* last valid byte -*/ -#define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) -/* Per CPU interrupt stacks */ struct irq_stack {char stack[IRQ_STACK_SIZE ];
arch/x86/kernel/cpu/common.c +2 -1
@@ -1860,7 +1860,8 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + tss->io_bitmap_prev_max =0 ;memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);
arch/x86/kernel/doublefault.c +1 -1
@@ -54,7 +54,7 @@ struct x86_hw_tss doublefault_tss __cach .sp0 = STACK_START, .ss0 = __KERNEL_DS, .ldt = 0, - .io_bitmap_base = INVALID_IO_BITMAP_OFFSET, + .io_bitmap_base = IO_BITMAP_OFFSET_INVALID, .ip = (unsigned long) doublefault_fn,/* 0x2 bit is always set */
arch/x86/kernel/ioport.c +2 -0
@@ -84,6 +84,8 @@ long ksys_ioperm(unsigned long from, unsmemcpy (tss->io_bitmap, t->io_bitmap_ptr, bytes_updated);/* Store the new end of the zero bits */ tss->io_bitmap_prev_max = bytes; +/* Make the bitmap base in the TSS valid */ + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/* Make sure the TSS limit covers the I/O bitmap. */ refresh_tss_limit ();
arch/x86/kernel/process.c +36 -27
108 lines not on this page · raw message
[tip: x86/iopl] x86/io: Speedup schedule out of I/O bitmap user
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 40ba6822b4b396fa3a1490dd63f8f18ab34ad5df Gitweb: https://git.kernel.org/tip/40ba6822b4b396fa3a1490dd63f8f18ab34ad5df Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:48 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:02 +01:00
x86/io: Speedup schedule out of I/O bitmap user
There is no requirement to update the TSS I/O bitmap when a thread using it is scheduled out and the incoming thread does not use it.
For the permission check based on the TSS I/O bitmap the CPU calculates the memory location of the I/O bitmap by the address of the TSS and the io_bitmap_base member of the tss_struct. The easiest way to invalidate the I/O bitmap is to switch the offset to an address outside of the TSS limit.
If an I/O instruction is issued from user space the TSS limit causes #GP to be raised in the same was as valid I/O bitmap with all bits set to 1 would do.
This removes the extra work when an I/O bitmap using task is scheduled out and puts the burden on the rare I/O bitmap users when they are scheduled in.
--- arch/x86/include/asm/processor.h | 38 +++++++++++++------ arch/x86/kernel/cpu/common.c | 3 +- arch/x86/kernel/doublefault.c | 2 +- arch/x86/kernel/ioport.c | 4 ++- arch/x86/kernel/process.c | 63 +++++++++++++++++-------------- 5 files changed, 69 insertions(+), 41 deletions(-)
arch/x86/include/asm/processor.h +26 -12
@@ -330,8 +330,23 @@ struct x86_hw_tss {#define IO_BITMAP_BITS 65536#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) -#define IO_BITMAP_OFFSET (offsetof(struct tss_struct, io_bitmap) - offsetof(struct tss_struct, x86_tss)) -#define INVALID_IO_BITMAP_OFFSET 0x8000 + +#define IO_BITMAP_OFFSET_VALID \ + (offsetof(struct tss_struct, io_bitmap) - \ + offsetof(struct tss_struct, x86_tss)) + +/* +* sizeof(unsigned long) coming from an extra "long" at the end +* of the iobitmap. +* +* -1? seg base+limit should be pointing to the address of the +* last valid byte +*/ +#define __KERNEL_TSS_LIMIT \ + (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + +/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ +#define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)struct entry_stack {unsigned long words[64 ];
@@ -350,6 +365,15 @@ struct tss_struct {struct x86_hw_tss x86_tss;/* +* Store the dirty size of the last io bitmap offender. The next +* one will have to do the cleanup as the switch out to a non io +* bitmap user will just set x86_tss.io_bitmap_base to a value +* outside of the TSS limit. So for sane tasks there is no need to +* actually touch the io_bitmap at all. +*/ +unsigned int io_bitmap_prev_max; + +/* * The extra 1 is there because the CPU will access an * additional byte beyond the end of the IO permission * bitmap. The extra byte must be all 1 bits, and must
@@ -360,16 +384,6 @@ struct tss_struct { DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw); -/* -* sizeof(unsigned long) coming from an extra "long" at the end -* of the iobitmap. -* -* -1? seg base+limit should be pointing to the address of the -* last valid byte -*/ -#define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) -/* Per CPU interrupt stacks */ struct irq_stack {char stack[IRQ_STACK_SIZE ];
arch/x86/kernel/cpu/common.c +2 -1
@@ -1860,7 +1860,8 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET ; + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + tss->io_bitmap_prev_max =0 ;memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);
arch/x86/kernel/doublefault.c +1 -1
@@ -54,7 +54,7 @@ struct x86_hw_tss doublefault_tss __cacheline_aligned = { .sp0 = STACK_START, .ss0 = __KERNEL_DS, .ldt = 0, - .io_bitmap_base = INVALID_IO_BITMAP_OFFSET, + .io_bitmap_base = IO_BITMAP_OFFSET_INVALID, .ip = (unsigned long) doublefault_fn,/* 0x2 bit is always set */
arch/x86/kernel/ioport.c +4 -0
@@ -82,6 +82,10 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)/* Update the TSS */ tss =this_cpu_ptr (&cpu_tss_rw);memcpy (tss->io_bitmap, t->io_bitmap_ptr, bytes_updated); +/* Store the new end of the zero bits */ + tss->io_bitmap_prev_max = bytes; +/* Make the bitmap base in the TSS valid */ + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/* Make sure the TSS limit covers the I/O bitmap. */ refresh_tss_limit ();
arch/x86/kernel/process.c +36 -27
108 lines not on this page · raw message
PATCH v3 9/20 x86/tss: Move I/O bitmap data into a seperate struct
In reply to Thomas GleixnerMove the non hardware portion of I/O bitmap data into a seperate struct for readability sake.
---
V3: New patch
--- arch/x86/include/asm/processor.h | 35 +++++++++++++++++++++-------------- arch/x86/kernel/cpu/common.c | 4 ++-- arch/x86/kernel/ioport.c | 4 ++-- arch/x86/kernel/process.c | 6 +++--- 4 files changed, 28 insertions(+), 21 deletions(-)
arch/x86/include/asm/processor.h +21 -14
@@ -328,11 +328,11 @@ struct x86_hw_tss {* IO-bitmap sizes: */ #define IO_BITMAP_BITS 65536 -#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8) -#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) +#define IO_BITMAP_BYTES (IO_BITMAP_BITS / BITS_PER_BYTE) +#define IO_BITMAP_LONGS (IO_BITMAP_BYTES / sizeof(long)) -#define IO_BITMAP_OFFSET_VALID \ - (offsetof(struct tss_struct, io_bitmap) - \ +#define IO_BITMAP_OFFSET_VALID \ + (offsetof(struct tss_struct, io_bitmap.bitmap) - \ offsetof(struct tss_struct, x86_tss))/*
@@ -356,14 +356,10 @@ struct entry_stack_page {struct entry_stack stack; }__aligned(PAGE_SIZE) ; -struct tss_struct { -/* -* The fixed hardware portion. This must not cross a page boundary -* at risk of violating the SDM's advice and potentially triggering -* errata. -*/ -struct x86_hw_tss x86_tss; - +/* +* All IO bitmap related data stored in the TSS: +*/ +struct x86_io_bitmap {/* * Store the dirty size of the last io bitmap offender. The next * one will have to do the cleanup as the switch out to a non io
@@ -371,7 +367,7 @@ struct tss_struct {* outside of the TSS limit. So for sane tasks there is no need to * actually touch the io_bitmap at all. */ -unsigned int io_bitmap_prev_max; +unsigned int prev_max;/* * The extra 1 is there because the CPU will access an
@@ -379,7 +375,18 @@ struct tss_struct {* bitmap. The extra byte must be all 1 bits, and must * be within the limit. */ -unsigned long io_bitmap[IO_BITMAP_LONGS +1 ]; +unsigned long bitmap[IO_BITMAP_LONGS +1 ]; +}; + +struct tss_struct { +/* +* The fixed hardware portion. This must not cross a page boundary +* at risk of violating the SDM's advice and potentially triggering +* errata. +*/ +struct x86_hw_tss x86_tss; + +struct x86_io_bitmap io_bitmap; }__aligned(PAGE_SIZE) ; DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw);
arch/x86/kernel/cpu/common.c +2 -2
@@ -1861,8 +1861,8 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; - tss->io_bitmap_prev_max =0 ; -memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap)); + tss->io_bitmap.prev_max =0 ; +memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();
arch/x86/kernel/ioport.c +2 -2
@@ -81,9 +81,9 @@ long ksys_ioperm(unsigned long from, uns/* Update the TSS */ tss =this_cpu_ptr (&cpu_tss_rw); -memcpy (tss->io_bitmap, t->io_bitmap_ptr, bytes_updated); +memcpy (tss->io_bitmap.bitmap, t->io_bitmap_ptr, bytes_updated);/* Store the new end of the zero bits */ - tss->io_bitmap_prev_max = bytes; + tss->io_bitmap.prev_max = bytes;/* Make the bitmap base in the TSS valid */ tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/* Make sure the TSS limit covers the I/O bitmap. */
arch/x86/kernel/process.c +3 -3
@@ -368,11 +368,11 @@ static inline void switch_to_bitmap(stru* bits permitted, then the copy needs to cover those as * well so they get turned off. */ -memcpy (tss->io_bitmap, next->io_bitmap_ptr, -max (tss->io_bitmap_prev_max, next->io_bitmap_max)); +memcpy (tss->io_bitmap.bitmap, next->io_bitmap_ptr, +max (tss->io_bitmap.prev_max, next->io_bitmap_max));/* Store the new max and set io_bitmap_base valid */ - tss->io_bitmap_prev_max = next->io_bitmap_max; + tss->io_bitmap.prev_max = next->io_bitmap_max; tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/*
[tip: x86/iopl] x86/tss: Move I/O bitmap data into a seperate struct
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 98336a94ec192ef45ba66e2c86dd3d7a3ef520a1 Gitweb: https://git.kernel.org/tip/98336a94ec192ef45ba66e2c86dd3d7a3ef520a1 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:49 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:03 +01:00
x86/tss: Move I/O bitmap data into a seperate struct
Move the non hardware portion of I/O bitmap data into a seperate struct for readability sake.
--- arch/x86/include/asm/processor.h | 35 ++++++++++++++++++------------- arch/x86/kernel/cpu/common.c | 4 ++-- arch/x86/kernel/ioport.c | 4 ++-- arch/x86/kernel/process.c | 6 ++--- 4 files changed, 28 insertions(+), 21 deletions(-)
arch/x86/include/asm/processor.h +21 -14
@@ -328,11 +328,11 @@ struct x86_hw_tss {* IO-bitmap sizes: */ #define IO_BITMAP_BITS 65536 -#define IO_BITMAP_BYTES (IO_BITMAP_BITS/8) -#define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) +#define IO_BITMAP_BYTES (IO_BITMAP_BITS / BITS_PER_BYTE) +#define IO_BITMAP_LONGS (IO_BITMAP_BYTES / sizeof(long)) -#define IO_BITMAP_OFFSET_VALID \ - (offsetof(struct tss_struct, io_bitmap) - \ +#define IO_BITMAP_OFFSET_VALID \ + (offsetof(struct tss_struct, io_bitmap.bitmap) - \ offsetof(struct tss_struct, x86_tss))/*
@@ -356,14 +356,10 @@ struct entry_stack_page {struct entry_stack stack; }__aligned(PAGE_SIZE) ; -struct tss_struct { -/* -* The fixed hardware portion. This must not cross a page boundary -* at risk of violating the SDM's advice and potentially triggering -* errata. -*/ -struct x86_hw_tss x86_tss; - +/* +* All IO bitmap related data stored in the TSS: +*/ +struct x86_io_bitmap {/* * Store the dirty size of the last io bitmap offender. The next * one will have to do the cleanup as the switch out to a non io
@@ -371,7 +367,7 @@ struct tss_struct {* outside of the TSS limit. So for sane tasks there is no need to * actually touch the io_bitmap at all. */ -unsigned int io_bitmap_prev_max; +unsigned int prev_max;/* * The extra 1 is there because the CPU will access an
@@ -379,7 +375,18 @@ struct tss_struct {* bitmap. The extra byte must be all 1 bits, and must * be within the limit. */ -unsigned long io_bitmap[IO_BITMAP_LONGS +1 ]; +unsigned long bitmap[IO_BITMAP_LONGS +1 ]; +}; + +struct tss_struct { +/* +* The fixed hardware portion. This must not cross a page boundary +* at risk of violating the SDM's advice and potentially triggering +* errata. +*/ +struct x86_hw_tss x86_tss; + +struct x86_io_bitmap io_bitmap; }__aligned(PAGE_SIZE) ; DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw);
arch/x86/kernel/cpu/common.c +2 -2
@@ -1861,8 +1861,8 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; - tss->io_bitmap_prev_max =0 ; -memset (tss->io_bitmap,0xff ,sizeof (tss->io_bitmap)); + tss->io_bitmap.prev_max =0 ; +memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();
arch/x86/kernel/ioport.c +2 -2
@@ -81,9 +81,9 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)/* Update the TSS */ tss =this_cpu_ptr (&cpu_tss_rw); -memcpy (tss->io_bitmap, t->io_bitmap_ptr, bytes_updated); +memcpy (tss->io_bitmap.bitmap, t->io_bitmap_ptr, bytes_updated);/* Store the new end of the zero bits */ - tss->io_bitmap_prev_max = bytes; + tss->io_bitmap.prev_max = bytes;/* Make the bitmap base in the TSS valid */ tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/* Make sure the TSS limit covers the I/O bitmap. */
arch/x86/kernel/process.c +3 -3
@@ -368,11 +368,11 @@ static inline void switch_to_bitmap(struct thread_struct *next,* bits permitted, then the copy needs to cover those as * well so they get turned off. */ -memcpy (tss->io_bitmap, next->io_bitmap_ptr, -max (tss->io_bitmap_prev_max, next->io_bitmap_max)); +memcpy (tss->io_bitmap.bitmap, next->io_bitmap_ptr, +max (tss->io_bitmap.prev_max, next->io_bitmap_max));/* Store the new max and set io_bitmap_base valid */ - tss->io_bitmap_prev_max = next->io_bitmap_max; + tss->io_bitmap.prev_max = next->io_bitmap_max; tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/*
PATCH v3 10/20 x86/ioperm: Move iobitmap data into a struct
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
No point in having all the data in thread_struct, especially as upcoming changes add more.
Make the bitmap in the new struct accessible as array of longs and as array of characters via a union, so both the bitmap functions and the update logic can avoid type casts.
---
V2: New patch
--- arch/x86/include/asm/io_bitmap.h | 13 +++++++++++++ arch/x86/include/asm/processor.h | 6 ++---- arch/x86/kernel/ioport.c | 27 ++++++++++++++------------- arch/x86/kernel/process.c | 38 ++++++++++++++++++++------------------ arch/x86/kernel/ptrace.c | 12 ++++++++---- 5 files changed, 57 insertions(+), 39 deletions(-)
arch/x86/include/asm/io_bitmap.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_IOBITMAP_H +#define _ASM_X86_IOBITMAP_H + +#include <asm/processor.h> + +struct io_bitmap { +/* The maximum number of bytes to copy so all zero bits are covered */ +unsigned int max; +unsigned long bitmap[IO_BITMAP_LONGS ]; +}; + +#endif
arch/x86/include/asm/processor.h +2 -4
@@ -7,6 +7,7 @@/* Forward declaration, a strange C thing */ struct task_struct ;struct mm_struct ; +struct io_bitmap ;struct vm86 ;#include <asm/math_emu.h>
@@ -501,10 +502,8 @@ struct thread_struct {struct vm86 *vm86;#endif /* IO permissions: */ -unsigned long *io_bitmap_ptr; +struct io_bitmap *io_bitmap;unsigned long iopl; -/* Max allowed port in the bitmap, in bytes: */ -unsigned io_bitmap_max;mm_segment_t addr_limit;
@@ -862,7 +861,6 @@ static inline void spin_lock_prefetch(co#define INIT_THREAD { \ .sp0 = TOP_OF_INIT_STACK, \ .sysenter_cs = __KERNEL_CS, \ - .io_bitmap_ptr = NULL, \ .addr_limit = KERNEL_DS, \ }
arch/x86/kernel/ioport.c +14 -13
@@ -11,6 +11,7 @@#include <linux/sched.h> #include <linux/slab.h> +#include <asm/io_bitmap.h> #include <asm/desc.h> /*
@@ -21,7 +22,7 @@ long ksys_ioperm(unsigned long from, unsunsigned int i, max_long, bytes, bytes_updated;struct thread_struct *t = ¤t->thread;struct tss_struct *tss; -unsigned long *bitmap; +struct io_bitmap *iobm;if ((from + num <= from) || (from + num >IO_BITMAP_BITS ))return -EINVAL ;
@@ -34,16 +35,16 @@ long ksys_ioperm(unsigned long from, uns* IO bitmap up. ioperm() is much less timing critical than clone(), * this is why we delay this operation until now: */ - bitmap = t->io_bitmap_ptr; -if (!bitmap) { + iobm = t->io_bitmap; +if (!iobm) {/* No point to allocate a bitmap just to clear permissions */ if (!turn_on)return 0 ; - bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL ); -if (!bitmap) + iobm =kmalloc (sizeof (*iobm),GFP_KERNEL ); +if (!iobm)return -ENOMEM ; -memset (bitmap,0xff ,IO_BITMAP_BYTES ); +memset (iobm->bitmap,0xff ,sizeof (iobm->bitmap)); }/*
@@ -52,9 +53,9 @@ long ksys_ioperm(unsigned long from, uns*/ preempt_disable ();if (turn_on) -bitmap_clear (bitmap, from, num); +bitmap_clear (iobm->bitmap, from, num);else -bitmap_set (bitmap, from, num); +bitmap_set (iobm->bitmap, from, num);/* * Search for a (possibly new) maximum. This is simple and stupid,
@@ -62,26 +63,26 @@ long ksys_ioperm(unsigned long from, uns*/ max_long =0 ;for (i =0 ; i <IO_BITMAP_LONGS ; i++) { -if (bitmap[i] != ~0UL ) +if (iobm->bitmap[i] != ~0UL ) max_long = i; } bytes = (max_long +1 ) *sizeof (unsigned long ); - bytes_updated =max (bytes, t->io_bitmap_max); + bytes_updated =max (bytes, t->io_bitmap->max);/* Update the thread data */ - t->io_bitmap_max = bytes; + iobm->max = bytes;/* * Store the bitmap pointer (might be the same if the task already * head one). Set the TIF flag, just in case this is the first * invocation. */ - t->io_bitmap_ptr = bitmap; + t->io_bitmap = iobm;set_thread_flag (TIF_IO_BITMAP );/* Update the TSS */ tss =this_cpu_ptr (&cpu_tss_rw); -memcpy (tss->io_bitmap.bitmap, t->io_bitmap_ptr, bytes_updated); +memcpy (tss->io_bitmap.bitmap, iobm->bitmap, bytes_updated);/* Store the new end of the zero bits */ tss->io_bitmap.prev_max = bytes;/* Make the bitmap base in the TSS valid */
arch/x86/kernel/process.c +20 -18
@@ -41,6 +41,7 @@#include <asm/desc.h> #include <asm/prctl.h> #include <asm/spec-ctrl.h> +#include <asm/io_bitmap.h> #include <asm/proto.h> #include "process.h"
@@ -101,21 +102,20 @@ int arch_dup_task_struct(struct task_strvoid exit_thread (struct task_struct *tsk) {struct thread_struct *t = &tsk->thread; -unsigned long *bp = t->io_bitmap_ptr; +struct io_bitmap *iobm = t->io_bitmap;struct fpu *fpu = &t->fpu;struct tss_struct *tss; -if (bp) { +if (iobm) {preempt_disable (); tss =this_cpu_ptr (&cpu_tss_rw); - t->io_bitmap_ptr =NULL ; - t->io_bitmap_max =0 ; + t->io_bitmap =NULL ;clear_thread_flag (TIF_IO_BITMAP );/* Invalidate the io bitmap base in the TSS */ tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ;preempt_enable (); -kfree (bp); +kfree (iobm); }free_vm86 (t);
@@ -135,25 +135,25 @@ static int set_new_tls(struct task_strucstatic inline int copy_io_bitmap (struct task_struct *tsk) { +struct io_bitmap *iobm = current->thread.io_bitmap; +if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP )))return 0 ; - tsk->thread.io_bitmap_ptr =kmemdup (current->thread.io_bitmap_ptr, -IO_BITMAP_BYTES ,GFP_KERNEL ); -if (!tsk->thread.io_bitmap_ptr) { - tsk->thread.io_bitmap_max =0 ; + tsk->thread.io_bitmap =kmemdup (iobm,sizeof (*iobm),GFP_KERNEL ); + +if (!tsk->thread.io_bitmap)return -ENOMEM ; - } +set_tsk_thread_flag (tsk,TIF_IO_BITMAP );return 0 ; }static inline void free_io_bitmap (struct task_struct *tsk) { -if (tsk->thread.io_bitmap_ptr) { -kfree (tsk->thread.io_bitmap_ptr); - tsk->thread.io_bitmap_ptr =NULL ; - tsk->thread.io_bitmap_max =0 ; +if (tsk->thread.io_bitmap) { +kfree (tsk->thread.io_bitmap); + tsk->thread.io_bitmap =NULL ; } }
@@ -172,7 +172,7 @@ int copy_thread_tls(unsigned long clone_ frame->bp =0 ; frame->ret_addr = (unsigned long ) ret_from_fork; p->thread.sp = (unsigned long ) fork_frame; - p->thread.io_bitmap_ptr =NULL ; + p->thread.io_bitmap =NULL ;memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));#ifdef CONFIG_X86_64
@@ -360,6 +360,8 @@ static inline void switch_to_bitmap(strustruct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw);if (tifn & _TIF_IO_BITMAP) { +struct io_bitmap *iobm = next->io_bitmap; +/* * Copy at least the size of the incoming tasks bitmap * which covers the last permitted I/O port.
@@ -368,11 +370,11 @@ static inline void switch_to_bitmap(stru* bits permitted, then the copy needs to cover those as * well so they get turned off. */ -memcpy (tss->io_bitmap.bitmap, next->io_bitmap_ptr, -max (tss->io_bitmap.prev_max, next->io_bitmap_max)); +memcpy (tss->io_bitmap.bitmap, next->io_bitmap->bitmap, +max (tss->io_bitmap.prev_max, next->io_bitmap->max));/* Store the new max and set io_bitmap_base valid */ - tss->io_bitmap.prev_max = next->io_bitmap_max; + tss->io_bitmap.prev_max = next->io_bitmap->max; tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/*
arch/x86/kernel/ptrace.c +8 -4
@@ -42,6 +42,7 @@#include <asm/traps.h> #include <asm/syscall.h> #include <asm/fsgsbase.h> +#include <asm/io_bitmap.h> #include "tls.h"
@@ -697,7 +698,9 @@ static int ptrace_set_debugreg(struct tastatic int ioperm_active (struct task_struct *target,const struct user_regset *regset) { -return DIV_ROUND_UP (target->thread.io_bitmap_max, regset->size); +struct io_bitmap *iobm = target->thread.io_bitmap; + +return iobm ?DIV_ROUND_UP (iobm->max, regset->size) :0 ; } static int ioperm_get(struct task_struct *target,
@@ -705,12 +708,13 @@ static int ioperm_get(struct task_struct unsigned int pos, unsigned int count, void *kbuf, void __user *ubuf) { - if (!target->thread.io_bitmap_ptr) + struct io_bitmap *iobm = target->thread.io_bitmap; + +if (!iobm)return -ENXIO ;return user_regset_copyout (&pos, &count, &kbuf, &ubuf, - target->thread.io_bitmap_ptr, -0 ,IO_BITMAP_BYTES ); + iobm->bitmap,0 ,IO_BITMAP_BYTES ); }/*
[tip: x86/iopl] x86/ioperm: Move iobitmap data into a struct
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: a975eef7788ec94ee548a4f182592b425759b7db Gitweb: https://git.kernel.org/tip/a975eef7788ec94ee548a4f182592b425759b7db Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:50 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:03 +01:00
x86/ioperm: Move iobitmap data into a struct
No point in having all the data in thread_struct, especially as upcoming changes add more.
Make the bitmap in the new struct accessible as array of longs and as array of characters via a union, so both the bitmap functions and the update logic can avoid type casts.
--- arch/x86/include/asm/io_bitmap.h | 13 +++++++++++- arch/x86/include/asm/processor.h | 6 +---- arch/x86/kernel/ioport.c | 27 +++++++++++----------- arch/x86/kernel/process.c | 38 ++++++++++++++++--------------- arch/x86/kernel/ptrace.c | 12 ++++++---- 5 files changed, 57 insertions(+), 39 deletions(-) create mode 100644 arch/x86/include/asm/io_bitmap.h
arch/x86/include/asm/io_bitmap.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_IOBITMAP_H +#define _ASM_X86_IOBITMAP_H + +#include <asm/processor.h> + +struct io_bitmap { +/* The maximum number of bytes to copy so all zero bits are covered */ +unsigned int max; +unsigned long bitmap[IO_BITMAP_LONGS ]; +}; + +#endif
arch/x86/include/asm/processor.h +2 -4
@@ -7,6 +7,7 @@/* Forward declaration, a strange C thing */ struct task_struct ;struct mm_struct ; +struct io_bitmap ;struct vm86 ;#include <asm/math_emu.h>
@@ -501,10 +502,8 @@ struct thread_struct {struct vm86 *vm86;#endif /* IO permissions: */ -unsigned long *io_bitmap_ptr; +struct io_bitmap *io_bitmap;unsigned long iopl; -/* Max allowed port in the bitmap, in bytes: */ -unsigned io_bitmap_max;mm_segment_t addr_limit;
@@ -862,7 +861,6 @@ static inline void spin_lock_prefetch(const void *x)#define INIT_THREAD { \ .sp0 = TOP_OF_INIT_STACK, \ .sysenter_cs = __KERNEL_CS, \ - .io_bitmap_ptr = NULL, \ .addr_limit = KERNEL_DS, \ }
arch/x86/kernel/ioport.c +14 -13
@@ -11,6 +11,7 @@#include <linux/sched.h> #include <linux/slab.h> +#include <asm/io_bitmap.h> #include <asm/desc.h> /*
@@ -21,7 +22,7 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)unsigned int i, max_long, bytes, bytes_updated;struct thread_struct *t = ¤t->thread;struct tss_struct *tss; -unsigned long *bitmap; +struct io_bitmap *iobm;if ((from + num <= from) || (from + num >IO_BITMAP_BITS ))return -EINVAL ;
@@ -34,16 +35,16 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)* IO bitmap up. ioperm() is much less timing critical than clone(), * this is why we delay this operation until now: */ - bitmap = t->io_bitmap_ptr; -if (!bitmap) { + iobm = t->io_bitmap; +if (!iobm) {/* No point to allocate a bitmap just to clear permissions */ if (!turn_on)return 0 ; - bitmap =kmalloc (IO_BITMAP_BYTES ,GFP_KERNEL ); -if (!bitmap) + iobm =kmalloc (sizeof (*iobm),GFP_KERNEL ); +if (!iobm)return -ENOMEM ; -memset (bitmap,0xff ,IO_BITMAP_BYTES ); +memset (iobm->bitmap,0xff ,sizeof (iobm->bitmap)); }/*
@@ -52,9 +53,9 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)*/ preempt_disable ();if (turn_on) -bitmap_clear (bitmap, from, num); +bitmap_clear (iobm->bitmap, from, num);else -bitmap_set (bitmap, from, num); +bitmap_set (iobm->bitmap, from, num);/* * Search for a (possibly new) maximum. This is simple and stupid,
@@ -62,26 +63,26 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)*/ max_long =0 ;for (i =0 ; i <IO_BITMAP_LONGS ; i++) { -if (bitmap[i] != ~0UL ) +if (iobm->bitmap[i] != ~0UL ) max_long = i; } bytes = (max_long +1 ) *sizeof (unsigned long ); - bytes_updated =max (bytes, t->io_bitmap_max); + bytes_updated =max (bytes, t->io_bitmap->max);/* Update the thread data */ - t->io_bitmap_max = bytes; + iobm->max = bytes;/* * Store the bitmap pointer (might be the same if the task already * head one). Set the TIF flag, just in case this is the first * invocation. */ - t->io_bitmap_ptr = bitmap; + t->io_bitmap = iobm;set_thread_flag (TIF_IO_BITMAP );/* Update the TSS */ tss =this_cpu_ptr (&cpu_tss_rw); -memcpy (tss->io_bitmap.bitmap, t->io_bitmap_ptr, bytes_updated); +memcpy (tss->io_bitmap.bitmap, iobm->bitmap, bytes_updated);/* Store the new end of the zero bits */ tss->io_bitmap.prev_max = bytes;/* Make the bitmap base in the TSS valid */
arch/x86/kernel/process.c +20 -18
@@ -41,6 +41,7 @@#include <asm/desc.h> #include <asm/prctl.h> #include <asm/spec-ctrl.h> +#include <asm/io_bitmap.h> #include <asm/proto.h> #include "process.h"
@@ -101,21 +102,20 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)void exit_thread (struct task_struct *tsk) {struct thread_struct *t = &tsk->thread; -unsigned long *bp = t->io_bitmap_ptr; +struct io_bitmap *iobm = t->io_bitmap;struct fpu *fpu = &t->fpu;struct tss_struct *tss; -if (bp) { +if (iobm) {preempt_disable (); tss =this_cpu_ptr (&cpu_tss_rw); - t->io_bitmap_ptr =NULL ; - t->io_bitmap_max =0 ; + t->io_bitmap =NULL ;clear_thread_flag (TIF_IO_BITMAP );/* Invalidate the io bitmap base in the TSS */ tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ;preempt_enable (); -kfree (bp); +kfree (iobm); }free_vm86 (t);
@@ -135,25 +135,25 @@ static int set_new_tls(struct task_struct *p, unsigned long tls)static inline int copy_io_bitmap (struct task_struct *tsk) { +struct io_bitmap *iobm = current->thread.io_bitmap; +if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP )))return 0 ; - tsk->thread.io_bitmap_ptr =kmemdup (current->thread.io_bitmap_ptr, -IO_BITMAP_BYTES ,GFP_KERNEL ); -if (!tsk->thread.io_bitmap_ptr) { - tsk->thread.io_bitmap_max =0 ; + tsk->thread.io_bitmap =kmemdup (iobm,sizeof (*iobm),GFP_KERNEL ); + +if (!tsk->thread.io_bitmap)return -ENOMEM ; - } +set_tsk_thread_flag (tsk,TIF_IO_BITMAP );return 0 ; }static inline void free_io_bitmap (struct task_struct *tsk) { -if (tsk->thread.io_bitmap_ptr) { -kfree (tsk->thread.io_bitmap_ptr); - tsk->thread.io_bitmap_ptr =NULL ; - tsk->thread.io_bitmap_max =0 ; +if (tsk->thread.io_bitmap) { +kfree (tsk->thread.io_bitmap); + tsk->thread.io_bitmap =NULL ; } }
@@ -172,7 +172,7 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long sp, frame->bp =0 ; frame->ret_addr = (unsigned long ) ret_from_fork; p->thread.sp = (unsigned long ) fork_frame; - p->thread.io_bitmap_ptr =NULL ; + p->thread.io_bitmap =NULL ;memset (p->thread.ptrace_bps,0 ,sizeof (p->thread.ptrace_bps));#ifdef CONFIG_X86_64
@@ -360,6 +360,8 @@ static inline void switch_to_bitmap(struct thread_struct *next,struct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw);if (tifn & _TIF_IO_BITMAP) { +struct io_bitmap *iobm = next->io_bitmap; +/* * Copy at least the size of the incoming tasks bitmap * which covers the last permitted I/O port.
@@ -368,11 +370,11 @@ static inline void switch_to_bitmap(struct thread_struct *next,* bits permitted, then the copy needs to cover those as * well so they get turned off. */ -memcpy (tss->io_bitmap.bitmap, next->io_bitmap_ptr, -max (tss->io_bitmap.prev_max, next->io_bitmap_max)); +memcpy (tss->io_bitmap.bitmap, next->io_bitmap->bitmap, +max (tss->io_bitmap.prev_max, next->io_bitmap->max));/* Store the new max and set io_bitmap_base valid */ - tss->io_bitmap.prev_max = next->io_bitmap_max; + tss->io_bitmap.prev_max = next->io_bitmap->max; tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/*
arch/x86/kernel/ptrace.c +8 -4
@@ -42,6 +42,7 @@#include <asm/traps.h> #include <asm/syscall.h> #include <asm/fsgsbase.h> +#include <asm/io_bitmap.h> #include "tls.h"
@@ -697,7 +698,9 @@ static int ptrace_set_debugreg(struct task_struct *tsk, int n,static int ioperm_active (struct task_struct *target,const struct user_regset *regset) { -return DIV_ROUND_UP (target->thread.io_bitmap_max, regset->size); +struct io_bitmap *iobm = target->thread.io_bitmap; + +return iobm ?DIV_ROUND_UP (iobm->max, regset->size) :0 ; } static int ioperm_get(struct task_struct *target,
@@ -705,12 +708,13 @@ static int ioperm_get(struct task_struct *target, unsigned int pos, unsigned int count, void *kbuf, void __user *ubuf) { - if (!target->thread.io_bitmap_ptr) + struct io_bitmap *iobm = target->thread.io_bitmap; + +if (!iobm)return -ENXIO ;return user_regset_copyout (&pos, &count, &kbuf, &ubuf, - target->thread.io_bitmap_ptr, -0 ,IO_BITMAP_BYTES ); + iobm->bitmap,0 ,IO_BITMAP_BYTES ); }/*
PATCH v3 11/20 x86/ioperm: Add bitmap sequence number
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
Add a globally unique sequence number which is incremented when ioperm() is changing the I/O bitmap of a task. Store the new sequence number in the io_bitmap structure and compare it with the sequence number of the I/O bitmap which was last loaded on a CPU. Only update the bitmap if the sequence is different.
That should further reduce the overhead of I/O bitmap scheduling when there are only a few I/O bitmap users on the system.
The 64bit sequence counter is sufficient. A wraparound of the sequence counter assuming an ioperm() call every nanosecond would require about 584 years of uptime.
---
V3: Remove the pointer comparison (Peter, Andy)
V2: New patch
--- arch/x86/include/asm/io_bitmap.h | 1 + arch/x86/include/asm/processor.h | 3 +++ arch/x86/kernel/cpu/common.c | 1 + arch/x86/kernel/ioport.c | 5 +++++ arch/x86/kernel/process.c | 38 ++++++++++++++++++++++++++++---------- 5 files changed, 38 insertions(+), 10 deletions(-)
arch/x86/include/asm/io_bitmap.h +1 -0
@@ -5,6 +5,7 @@#include <asm/processor.h> struct io_bitmap { +u64 sequence;/* The maximum number of bytes to copy so all zero bits are covered */ unsigned int max;unsigned long bitmap[IO_BITMAP_LONGS ];
arch/x86/include/asm/processor.h +3 -0
@@ -361,6 +361,9 @@ struct entry_stack_page {* All IO bitmap related data stored in the TSS: */ struct x86_io_bitmap { +/* The sequence number of the last active bitmap. */ +u64 prev_sequence; +/* * Store the dirty size of the last io bitmap offender. The next * one will have to do the cleanup as the switch out to a non io
arch/x86/kernel/cpu/common.c +1 -0
@@ -1862,6 +1862,7 @@ void cpu_init(void)tss_setup_ist (tss); tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; tss->io_bitmap.prev_max =0 ; + tss->io_bitmap.prev_sequence =0 ;memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);
arch/x86/kernel/ioport.c +5 -0
@@ -14,6 +14,8 @@#include <asm/io_bitmap.h> #include <asm/desc.h> +static atomic64_t io_bitmap_sequence; +/* * this changes the io permissions bitmap in the current task. */
@@ -72,6 +74,9 @@ long ksys_ioperm(unsigned long from, uns/* Update the thread data */ iobm->max = bytes; +/* Update the sequence number to force an update in switch_to() */ + iobm->sequence =atomic64_add_return (1 , &io_bitmap_sequence); +/* * Store the bitmap pointer (might be the same if the task already * head one). Set the TIF flag, just in case this is the first
arch/x86/kernel/process.c +28 -10
@@ -354,6 +354,28 @@ void arch_setup_new_exec(void) } } +static void switch_to_update_io_bitmap (struct tss_struct *tss, +struct io_bitmap *iobm) +{ +/* +* Copy at least the byte range of the incoming tasks bitmap which +* covers the permitted I/O ports. +* +* If the previous task which used an I/O bitmap had more bits +* permitted, then the copy needs to cover those as well so they +* get turned off. +*/ +memcpy (tss->io_bitmap.bitmap, iobm->bitmap, +max (tss->io_bitmap.prev_max, iobm->max)); + +/* +* Store the new max and the sequence number of this bitmap +* and a pointer to the bitmap itself. +*/ + tss->io_bitmap.prev_max = iobm->max; + tss->io_bitmap.prev_sequence = iobm->sequence; +} +static inline void switch_to_bitmap (struct thread_struct *next,unsigned long tifp,unsigned long tifn) {
@@ -363,18 +385,14 @@ static inline void switch_to_bitmap(strustruct io_bitmap *iobm = next->io_bitmap;/* -* Copy at least the size of the incoming tasks bitmap -* which covers the last permitted I/O port. -* -* If the previous task which used an io bitmap had more -* bits permitted, then the copy needs to cover those as -* well so they get turned off. +* Only copy bitmap data when the sequence number +* differs. The update time is accounted to the incoming +* task. */ -memcpy (tss->io_bitmap.bitmap, next->io_bitmap->bitmap, -max (tss->io_bitmap.prev_max, next->io_bitmap->max)); +if (tss->io_bitmap.prev_sequence != iobm->sequence) +switch_to_update_io_bitmap (tss, iobm); -/* Store the new max and set io_bitmap_base valid */ - tss->io_bitmap.prev_max = next->io_bitmap->max; +/* Enable the bitmap */ tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/*
[tip: x86/iopl] x86/ioperm: Add bitmap sequence number
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 209c34d3c2de01a5dfe428b8b79fdb913dd76334 Gitweb: https://git.kernel.org/tip/209c34d3c2de01a5dfe428b8b79fdb913dd76334 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:51 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:04 +01:00
x86/ioperm: Add bitmap sequence number
Add a globally unique sequence number which is incremented when ioperm() is changing the I/O bitmap of a task. Store the new sequence number in the io_bitmap structure and compare it with the sequence number of the I/O bitmap which was last loaded on a CPU. Only update the bitmap if the sequence is different.
That should further reduce the overhead of I/O bitmap scheduling when there are only a few I/O bitmap users on the system.
The 64bit sequence counter is sufficient. A wraparound of the sequence counter assuming an ioperm() call every nanosecond would require about 584 years of uptime.
--- arch/x86/include/asm/io_bitmap.h | 1 +- arch/x86/include/asm/processor.h | 3 ++- arch/x86/kernel/cpu/common.c | 1 +- arch/x86/kernel/ioport.c | 5 ++++- arch/x86/kernel/process.c | 38 ++++++++++++++++++++++--------- 5 files changed, 38 insertions(+), 10 deletions(-)
arch/x86/include/asm/io_bitmap.h +1 -0
@@ -5,6 +5,7 @@#include <asm/processor.h> struct io_bitmap { +u64 sequence;/* The maximum number of bytes to copy so all zero bits are covered */ unsigned int max;unsigned long bitmap[IO_BITMAP_LONGS ];
arch/x86/include/asm/processor.h +3 -0
@@ -361,6 +361,9 @@ struct entry_stack_page {* All IO bitmap related data stored in the TSS: */ struct x86_io_bitmap { +/* The sequence number of the last active bitmap. */ +u64 prev_sequence; +/* * Store the dirty size of the last io bitmap offender. The next * one will have to do the cleanup as the switch out to a non io
arch/x86/kernel/cpu/common.c +1 -0
@@ -1862,6 +1862,7 @@ void cpu_init(void)tss_setup_ist (tss); tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; tss->io_bitmap.prev_max =0 ; + tss->io_bitmap.prev_sequence =0 ;memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap));set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);
arch/x86/kernel/ioport.c +5 -0
@@ -14,6 +14,8 @@#include <asm/io_bitmap.h> #include <asm/desc.h> +static atomic64_t io_bitmap_sequence; +/* * this changes the io permissions bitmap in the current task. */
@@ -72,6 +74,9 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)/* Update the thread data */ iobm->max = bytes; +/* Update the sequence number to force an update in switch_to() */ + iobm->sequence =atomic64_add_return (1 , &io_bitmap_sequence); +/* * Store the bitmap pointer (might be the same if the task already * head one). Set the TIF flag, just in case this is the first
arch/x86/kernel/process.c +28 -10
@@ -354,6 +354,28 @@ void arch_setup_new_exec(void) } } +static void switch_to_update_io_bitmap (struct tss_struct *tss, +struct io_bitmap *iobm) +{ +/* +* Copy at least the byte range of the incoming tasks bitmap which +* covers the permitted I/O ports. +* +* If the previous task which used an I/O bitmap had more bits +* permitted, then the copy needs to cover those as well so they +* get turned off. +*/ +memcpy (tss->io_bitmap.bitmap, iobm->bitmap, +max (tss->io_bitmap.prev_max, iobm->max)); + +/* +* Store the new max and the sequence number of this bitmap +* and a pointer to the bitmap itself. +*/ + tss->io_bitmap.prev_max = iobm->max; + tss->io_bitmap.prev_sequence = iobm->sequence; +} +static inline void switch_to_bitmap (struct thread_struct *next,unsigned long tifp,unsigned long tifn) {
@@ -363,18 +385,14 @@ static inline void switch_to_bitmap(struct thread_struct *next,struct io_bitmap *iobm = next->io_bitmap;/* -* Copy at least the size of the incoming tasks bitmap -* which covers the last permitted I/O port. -* -* If the previous task which used an io bitmap had more -* bits permitted, then the copy needs to cover those as -* well so they get turned off. +* Only copy bitmap data when the sequence number +* differs. The update time is accounted to the incoming +* task. */ -memcpy (tss->io_bitmap.bitmap, next->io_bitmap->bitmap, -max (tss->io_bitmap.prev_max, next->io_bitmap->max)); +if (tss->io_bitmap.prev_sequence != iobm->sequence) +switch_to_update_io_bitmap (tss, iobm); -/* Store the new max and set io_bitmap_base valid */ - tss->io_bitmap.prev_max = next->io_bitmap->max; +/* Enable the bitmap */ tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ;/*
PATCH v3 12/20 x86/ioperm: Move TSS bitmap update to exit to user work
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
There is no point to update the TSS bitmap for tasks which use I/O bitmaps on every context switch. It's enough to update it right before exiting to user space.
That reduces the context switch bitmap handling to invalidating the io bitmap base offset in the TSS when the outgoing task has TIF_IO_BITMAP set. The invaldiation is done on purpose when a task with an IO bitmap switches out to prevent any possible leakage of an activated IO bitmap.
It also removes the requirement to update the tasks bitmap atomically in ioperm().
---
V3: Updated comment and changelog to clarify the invalidation logic
Removed _TIF_IO_BITMAP from _TIF_WORK_CTXSW_NEXT as it is only required for the outgoing task.
V2: New patch
--- arch/x86/entry/common.c | 4 ++ arch/x86/include/asm/io_bitmap.h | 2 + arch/x86/include/asm/thread_info.h | 9 +++-- arch/x86/kernel/ioport.c | 25 ++------------- arch/x86/kernel/process.c | 59 ++++++++++++++++++++++++------------- 5 files changed, 54 insertions(+), 45 deletions(-)
arch/x86/entry/common.c +4 -0
@@ -33,6 +33,7 @@#include <asm/cpufeature.h> #include <asm/fpu/api.h> #include <asm/nospec-branch.h> +#include <asm/io_bitmap.h> #define CREATE_TRACE_POINTS #include <trace/events/syscalls.h>
@@ -196,6 +197,9 @@ static void exit_to_usermode_loop(struct/* Reload ti->flags; we may have rescheduled above. */ cached_flags =READ_ONCE (ti->flags); +if (unlikely (cached_flags & _TIF_IO_BITMAP)) +tss_update_io_bitmap (); +fpregs_assert_state_consistent ();if (unlikely (cached_flags & _TIF_NEED_FPU_LOAD))switch_fpu_return ();
arch/x86/include/asm/io_bitmap.h +2 -0
@@ -11,4 +11,6 @@ struct io_bitmap { unsigned long bitmap[IO_BITMAP_LONGS]; }; +void tss_update_io_bitmap (void ); +#endif
arch/x86/include/asm/thread_info.h +5 -4
@@ -143,8 +143,8 @@ struct thread_info { _TIF_NOHZ)/* flags to check in __switch_to() */ -#define _TIF_WORK_CTXSW_BASE \ - (_TIF_IO_BITMAP|_TIF_NOCPUID|_TIF_NOTSC|_TIF_BLOCKSTEP| \ +#define _TIF_WORK_CTXSW_BASE \ + (_TIF_NOCPUID | _TIF_NOTSC | _TIF_BLOCKSTEP | \ _TIF_SSBD | _TIF_SPEC_FORCE_UPDATE)/*
@@ -156,8 +156,9 @@ struct thread_info {# define _TIF_WORK_CTXSW (_TIF_WORK_CTXSW_BASE)#endif -#define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW|_TIF_USER_RETURN_NOTIFY) -#define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW) +#define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY | \ + _TIF_IO_BITMAP) +#define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW)#define STACK_WARN (THREAD_SIZE/8)
arch/x86/kernel/ioport.c +4 -21
@@ -21,9 +21,8 @@ static atomic64_t io_bitmap_sequence;*/ long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) { -unsigned int i, max_long, bytes, bytes_updated;struct thread_struct *t = ¤t->thread; -struct tss_struct *tss; +unsigned int i, max_long;struct io_bitmap *iobm;if ((from + num <= from) || (from + num >IO_BITMAP_BITS ))
@@ -50,10 +49,9 @@ long ksys_ioperm(unsigned long from, uns }/* -* Update the bitmap and the TSS copy with preemption disabled to -* prevent a race against context switch. +* Update the tasks bitmap. The update of the TSS bitmap happens on +* exit to user mode. So this needs no protection. */ -preempt_disable ();if (turn_on)bitmap_clear (iobm->bitmap, from, num);else
@@ -69,11 +67,8 @@ long ksys_ioperm(unsigned long from, uns max_long = i; } - bytes = (max_long +1 ) *sizeof (unsigned long ); - bytes_updated =max (bytes, t->io_bitmap->max); + iobm->max = (max_long +1 ) *sizeof (unsigned long ); -/* Update the thread data */ - iobm->max = bytes;/* Update the sequence number to force an update in switch_to() */ iobm->sequence =atomic64_add_return (1 , &io_bitmap_sequence);
@@ -85,18 +80,6 @@ long ksys_ioperm(unsigned long from, uns t->io_bitmap = iobm;set_thread_flag (TIF_IO_BITMAP ); -/* Update the TSS */ - tss =this_cpu_ptr (&cpu_tss_rw); -memcpy (tss->io_bitmap.bitmap, iobm->bitmap, bytes_updated); -/* Store the new end of the zero bits */ - tss->io_bitmap.prev_max = bytes; -/* Make the bitmap base in the TSS valid */ - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ; -/* Make sure the TSS limit covers the I/O bitmap. */ -refresh_tss_limit (); - -preempt_enable (); -return 0 ; }
arch/x86/kernel/process.c +39 -20
92 lines not on this page · raw message
Re: [patch V3 12/20] x86/ioperm: Move TSS bitmap update to exit to user work
Quoted 6 lines · Thomas Gleixner
From: Thomas Gleixner <[email protected]>
There is no point to update the TSS bitmap for tasks which use I/O bitmaps on every context switch. It's enough to update it right before exiting to user space.
Hmm.
I wonder if it might make sense to delay it even more: just always invalidate the bitmap on task switch, and leave it at that.
And then on GP fault, just add trivial logic like
if (I_have_an_io_bitmap && it_isnt_installed) {
install_io_bitmap();
return;
}and now you do get that extra GP fault if you actually use IO accesses, but the normal case has zero overhead.
Even processes that do ioperm may not be *using* it all the time.
Linus
Re: [patch V3 12/20] x86/ioperm: Move TSS bitmap update to exit to user work
Quoted 2 lines · Linus Torvalds
I wonder if it might make sense to delay it even more: [..]
Btw, don't take that as a suggestion to make further changes. It was more of an idle observation, and it probably doesn't matter one whit.
Linus
Re: [patch V3 12/20] x86/ioperm: Move TSS bitmap update to exit to user work
Quoted 7 lines · Linus Torvalds
Quoted 2 lines · Linus Torvalds
I wonder if it might make sense to delay it even more: [..]
Btw, don't take that as a suggestion to make further changes. It was more of an idle observation, and it probably doesn't matter one whit.
Hehe. I wont, but yes the idea is sneaky enough to be really tempting :)
Thanks,
tglx
[tip: x86/iopl] x86/ioperm: Move TSS bitmap update to exit to user work
In reply to Thomas GleixnerThe following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 8ef04e97f1b49edd583e7b6fe81a3e24a3916786 Gitweb: https://git.kernel.org/tip/8ef04e97f1b49edd583e7b6fe81a3e24a3916786 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:52 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:04 +01:00
x86/ioperm: Move TSS bitmap update to exit to user work
There is no point to update the TSS bitmap for tasks which use I/O bitmaps on every context switch. It's enough to update it right before exiting to user space.
That reduces the context switch bitmap handling to invalidating the io bitmap base offset in the TSS when the outgoing task has TIF_IO_BITMAP set. The invaldiation is done on purpose when a task with an IO bitmap switches out to prevent any possible leakage of an activated IO bitmap.
It also removes the requirement to update the tasks bitmap atomically in ioperm().
--- arch/x86/entry/common.c | 4 ++- arch/x86/include/asm/io_bitmap.h | 2 +- arch/x86/include/asm/thread_info.h | 9 ++-- arch/x86/kernel/ioport.c | 25 +----------- arch/x86/kernel/process.c | 59 +++++++++++++++++++---------- 5 files changed, 54 insertions(+), 45 deletions(-)
arch/x86/entry/common.c +4 -0
@@ -33,6 +33,7 @@#include <asm/cpufeature.h> #include <asm/fpu/api.h> #include <asm/nospec-branch.h> +#include <asm/io_bitmap.h> #define CREATE_TRACE_POINTS #include <trace/events/syscalls.h>
@@ -196,6 +197,9 @@ __visible inline void prepare_exit_to_usermode(struct pt_regs *regs)/* Reload ti->flags; we may have rescheduled above. */ cached_flags =READ_ONCE (ti->flags); +if (unlikely (cached_flags & _TIF_IO_BITMAP)) +tss_update_io_bitmap (); +fpregs_assert_state_consistent ();if (unlikely (cached_flags & _TIF_NEED_FPU_LOAD))switch_fpu_return ();
arch/x86/include/asm/io_bitmap.h +2 -0
@@ -11,4 +11,6 @@ struct io_bitmap { unsigned long bitmap[IO_BITMAP_LONGS]; }; +void tss_update_io_bitmap (void ); +#endif
arch/x86/include/asm/thread_info.h +5 -4
@@ -143,8 +143,8 @@ struct thread_info { _TIF_NOHZ)/* flags to check in __switch_to() */ -#define _TIF_WORK_CTXSW_BASE \ - (_TIF_IO_BITMAP|_TIF_NOCPUID|_TIF_NOTSC|_TIF_BLOCKSTEP| \ +#define _TIF_WORK_CTXSW_BASE \ + (_TIF_NOCPUID | _TIF_NOTSC | _TIF_BLOCKSTEP | \ _TIF_SSBD | _TIF_SPEC_FORCE_UPDATE)/*
@@ -156,8 +156,9 @@ struct thread_info {# define _TIF_WORK_CTXSW (_TIF_WORK_CTXSW_BASE)#endif -#define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW|_TIF_USER_RETURN_NOTIFY) -#define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW) +#define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY | \ + _TIF_IO_BITMAP) +#define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW)#define STACK_WARN (THREAD_SIZE/8)
arch/x86/kernel/ioport.c +4 -21
@@ -21,9 +21,8 @@ static atomic64_t io_bitmap_sequence;*/ long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) { -unsigned int i, max_long, bytes, bytes_updated;struct thread_struct *t = ¤t->thread; -struct tss_struct *tss; +unsigned int i, max_long;struct io_bitmap *iobm;if ((from + num <= from) || (from + num >IO_BITMAP_BITS ))
@@ -50,10 +49,9 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on) }/* -* Update the bitmap and the TSS copy with preemption disabled to -* prevent a race against context switch. +* Update the tasks bitmap. The update of the TSS bitmap happens on +* exit to user mode. So this needs no protection. */ -preempt_disable ();if (turn_on)bitmap_clear (iobm->bitmap, from, num);else
@@ -69,11 +67,8 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on) max_long = i; } - bytes = (max_long +1 ) *sizeof (unsigned long ); - bytes_updated =max (bytes, t->io_bitmap->max); + iobm->max = (max_long +1 ) *sizeof (unsigned long ); -/* Update the thread data */ - iobm->max = bytes;/* Update the sequence number to force an update in switch_to() */ iobm->sequence =atomic64_add_return (1 , &io_bitmap_sequence);
@@ -85,18 +80,6 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on) t->io_bitmap = iobm;set_thread_flag (TIF_IO_BITMAP ); -/* Update the TSS */ - tss =this_cpu_ptr (&cpu_tss_rw); -memcpy (tss->io_bitmap.bitmap, iobm->bitmap, bytes_updated); -/* Store the new end of the zero bits */ - tss->io_bitmap.prev_max = bytes; -/* Make the bitmap base in the TSS valid */ - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ; -/* Make sure the TSS limit covers the I/O bitmap. */ -refresh_tss_limit (); - -preempt_enable (); -return 0 ; }
arch/x86/kernel/process.c +39 -20
92 lines not on this page · raw message
PATCH v3 13/20 x86/ioperm: Remove bitmap if all permissions dropped
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
If ioperm() results in a bitmap with all bits set (no permissions to any I/O port), then handling that bitmap on context switch and exit to user mode is pointless. Drop it.
Move the bitmap exit handling to the ioport code and reuse it for both the thread exit path and dropping it. This allows to reuse this code for the upcoming iopl() emulation.
---
V3: Reduced the preempt disable scope (Ingo)
V2: New patch
--- arch/x86/include/asm/io_bitmap.h | 2 ++ arch/x86/kernel/ioport.c | 19 ++++++++++++++++++- arch/x86/kernel/process.c | 15 ++------------- 3 files changed, 22 insertions(+), 14 deletions(-)
arch/x86/include/asm/io_bitmap.h +2 -0
@@ -11,6 +11,8 @@ struct io_bitmap { unsigned long bitmap[IO_BITMAP_LONGS]; }; +void io_bitmap_exit (void ); +void tss_update_io_bitmap (void );#endif
arch/x86/kernel/ioport.c +18 -1
@@ -16,6 +16,18 @@static atomic64_t io_bitmap_sequence; +void io_bitmap_exit (void ) +{ +struct io_bitmap *iobm = current->thread.io_bitmap; + + current->thread.io_bitmap =NULL ; +clear_thread_flag (TIF_IO_BITMAP ); +preempt_disable (); +tss_update_io_bitmap (); +preempt_enable (); +kfree (iobm); +} +/* * this changes the io permissions bitmap in the current task. */
@@ -61,11 +73,16 @@ long ksys_ioperm(unsigned long from, uns* Search for a (possibly new) maximum. This is simple and stupid, * to keep it obviously correct: */ - max_long =0 ; + max_long =UINT_MAX ;for (i =0 ; i <IO_BITMAP_LONGS ; i++) {if (iobm->bitmap[i] != ~0UL ) max_long = i; } +/* All permissions dropped? */ +if (max_long ==UINT_MAX ) { +io_bitmap_exit (); +return 0 ; + } iobm->max = (max_long +1 ) *sizeof (unsigned long );
arch/x86/kernel/process.c +2 -13
@@ -102,21 +102,10 @@ int arch_dup_task_struct(struct task_strvoid exit_thread (struct task_struct *tsk) {struct thread_struct *t = &tsk->thread; -struct io_bitmap *iobm = t->io_bitmap;struct fpu *fpu = &t->fpu; -struct tss_struct *tss; -if (iobm) { -preempt_disable (); - tss =this_cpu_ptr (&cpu_tss_rw); - - t->io_bitmap =NULL ; -clear_thread_flag (TIF_IO_BITMAP ); -/* Invalidate the io bitmap base in the TSS */ - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; -preempt_enable (); -kfree (iobm); - } +if (test_thread_flag (TIF_IO_BITMAP )) +io_bitmap_exit ();free_vm86 (t);
[tip: x86/iopl] x86/ioperm: Remove bitmap if all permissions dropped
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: f16c41c9db8460874662b46f0db5c8af2a568157 Gitweb: https://git.kernel.org/tip/f16c41c9db8460874662b46f0db5c8af2a568157 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:53 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:05 +01:00
x86/ioperm: Remove bitmap if all permissions dropped
If ioperm() results in a bitmap with all bits set (no permissions to any I/O port), then handling that bitmap on context switch and exit to user mode is pointless. Drop it.
Move the bitmap exit handling to the ioport code and reuse it for both the thread exit path and dropping it. This allows to reuse this code for the upcoming iopl() emulation.
--- arch/x86/include/asm/io_bitmap.h | 2 ++ arch/x86/kernel/ioport.c | 19 ++++++++++++++++++- arch/x86/kernel/process.c | 17 +++-------------- 3 files changed, 23 insertions(+), 15 deletions(-)
arch/x86/include/asm/io_bitmap.h +2 -0
@@ -11,6 +11,8 @@ struct io_bitmap { unsigned long bitmap[IO_BITMAP_LONGS]; }; +void io_bitmap_exit (void ); +void tss_update_io_bitmap (void );#endif
arch/x86/kernel/ioport.c +18 -1
@@ -16,6 +16,18 @@static atomic64_t io_bitmap_sequence; +void io_bitmap_exit (void ) +{ +struct io_bitmap *iobm = current->thread.io_bitmap; + + current->thread.io_bitmap =NULL ; +clear_thread_flag (TIF_IO_BITMAP ); +preempt_disable (); +tss_update_io_bitmap (); +preempt_enable (); +kfree (iobm); +} +/* * this changes the io permissions bitmap in the current task. */
@@ -61,11 +73,16 @@ long ksys_ioperm(unsigned long from, unsigned long num, int turn_on)* Search for a (possibly new) maximum. This is simple and stupid, * to keep it obviously correct: */ - max_long =0 ; + max_long =UINT_MAX ;for (i =0 ; i <IO_BITMAP_LONGS ; i++) {if (iobm->bitmap[i] != ~0UL ) max_long = i; } +/* All permissions dropped? */ +if (max_long ==UINT_MAX ) { +io_bitmap_exit (); +return 0 ; + } iobm->max = (max_long +1 ) *sizeof (unsigned long );
arch/x86/kernel/process.c +3 -14
@@ -102,21 +102,10 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)void exit_thread (struct task_struct *tsk) {struct thread_struct *t = &tsk->thread; -struct io_bitmap *iobm = t->io_bitmap;struct fpu *fpu = &t->fpu; -struct tss_struct *tss; - -if (iobm) { -preempt_disable (); - tss =this_cpu_ptr (&cpu_tss_rw); - - t->io_bitmap =NULL ; -clear_thread_flag (TIF_IO_BITMAP ); -/* Invalidate the io bitmap base in the TSS */ - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; -preempt_enable (); -kfree (iobm); - } + +if (test_thread_flag (TIF_IO_BITMAP )) +io_bitmap_exit ();free_vm86 (t);
PATCH v3 14/20 x86/ioperm: Share I/O bitmap if identical
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
The I/O bitmap is duplicated on fork. That's wasting memory and slows down fork. There is no point to do so. As long as the bitmap is not modified it can be shared between threads and processes.
Add a refcount and just share it on fork. If a task modifies the bitmap then it has to do the duplication if and only if it is shared.
---
V3: Reset the ref count after duplicating io_bitmap. (Peter, Andy)
V2: New patch
--- arch/x86/include/asm/io_bitmap.h | 5 ++++ arch/x86/kernel/ioport.c | 48 +++++++++++++++++++++++++++++++-------- arch/x86/kernel/process.c | 39 ++++--------------------------- 3 files changed, 50 insertions(+), 42 deletions(-)
arch/x86/include/asm/io_bitmap.h +5 -0
@@ -2,15 +2,20 @@#ifndef _ASM_X86_IOBITMAP_H#define _ASM_X86_IOBITMAP_H +#include <linux/refcount.h> #include <asm/processor.h> struct io_bitmap {u64 sequence; +refcount_t refcnt;/* The maximum number of bytes to copy so all zero bits are covered */ unsigned int max;unsigned long bitmap[IO_BITMAP_LONGS ]; }; +struct task_struct ; + +void io_bitmap_share (struct task_struct *tsk);void io_bitmap_exit (void );void tss_update_io_bitmap (void );
arch/x86/kernel/ioport.c +39 -9
77 lines not on this page · raw message
arch/x86/kernel/process.c +6 -33
@@ -122,37 +122,13 @@ static int set_new_tls(struct task_strucreturn do_set_thread_area_64 (p,ARCH_SET_FS , tls); } -static inline int copy_io_bitmap (struct task_struct *tsk) -{ -struct io_bitmap *iobm = current->thread.io_bitmap; - -if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP ))) -return 0 ; - - tsk->thread.io_bitmap =kmemdup (iobm,sizeof (*iobm),GFP_KERNEL ); - -if (!tsk->thread.io_bitmap) -return -ENOMEM ; - -set_tsk_thread_flag (tsk,TIF_IO_BITMAP ); -return 0 ; -} - -static inline void free_io_bitmap (struct task_struct *tsk) -{ -if (tsk->thread.io_bitmap) { -kfree (tsk->thread.io_bitmap); - tsk->thread.io_bitmap =NULL ; - } -} -int copy_thread_tls (unsigned long clone_flags,unsigned long sp,unsigned long arg,struct task_struct *p,unsigned long tls) {struct inactive_task_frame *frame;struct fork_frame *fork_frame;struct pt_regs *childregs; -int ret; +int ret =0 ; childregs =task_pt_regs (p); fork_frame = container_of(childregs, struct fork_frame, regs);
@@ -193,16 +169,13 @@ int copy_thread_tls(unsigned long clone_task_user_gs (p) =get_user_gs (current_pt_regs ());#endif - ret =copy_io_bitmap (p); -if (ret) -return ret; -/* Set a new TLS for the child thread? */ -if (clone_flags &CLONE_SETTLS ) { +if (clone_flags &CLONE_SETTLS ) ret =set_new_tls (p, tls); -if (ret) -free_io_bitmap (p); - } + +if (!ret &&unlikely (test_tsk_thread_flag (current,TIF_IO_BITMAP ))) +io_bitmap_share (p); +return ret; }
[tip: x86/iopl] x86/ioperm: Share I/O bitmap if identical
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: b9027e7c44922bf1499adb76f87da7a374a48c22 Gitweb: https://git.kernel.org/tip/b9027e7c44922bf1499adb76f87da7a374a48c22 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:54 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:05 +01:00
x86/ioperm: Share I/O bitmap if identical
The I/O bitmap is duplicated on fork. That's wasting memory and slows down fork. There is no point to do so. As long as the bitmap is not modified it can be shared between threads and processes.
Add a refcount and just share it on fork. If a task modifies the bitmap then it has to do the duplication if and only if it is shared.
--- arch/x86/include/asm/io_bitmap.h | 5 +++- arch/x86/kernel/ioport.c | 48 +++++++++++++++++++++++++------ arch/x86/kernel/process.c | 39 +++---------------------- 3 files changed, 50 insertions(+), 42 deletions(-)
arch/x86/include/asm/io_bitmap.h +5 -0
@@ -2,15 +2,20 @@#ifndef _ASM_X86_IOBITMAP_H#define _ASM_X86_IOBITMAP_H +#include <linux/refcount.h> #include <asm/processor.h> struct io_bitmap {u64 sequence; +refcount_t refcnt;/* The maximum number of bytes to copy so all zero bits are covered */ unsigned int max;unsigned long bitmap[IO_BITMAP_LONGS ]; }; +struct task_struct ; + +void io_bitmap_share (struct task_struct *tsk);void io_bitmap_exit (void );void tss_update_io_bitmap (void );
arch/x86/kernel/ioport.c +39 -9
77 lines not on this page · raw message
arch/x86/kernel/process.c +6 -33
@@ -122,37 +122,13 @@ static int set_new_tls(struct task_struct *p, unsigned long tls)return do_set_thread_area_64 (p,ARCH_SET_FS , tls); } -static inline int copy_io_bitmap (struct task_struct *tsk) -{ -struct io_bitmap *iobm = current->thread.io_bitmap; - -if (likely (!test_tsk_thread_flag (current,TIF_IO_BITMAP ))) -return 0 ; - - tsk->thread.io_bitmap =kmemdup (iobm,sizeof (*iobm),GFP_KERNEL ); - -if (!tsk->thread.io_bitmap) -return -ENOMEM ; - -set_tsk_thread_flag (tsk,TIF_IO_BITMAP ); -return 0 ; -} - -static inline void free_io_bitmap (struct task_struct *tsk) -{ -if (tsk->thread.io_bitmap) { -kfree (tsk->thread.io_bitmap); - tsk->thread.io_bitmap =NULL ; - } -} -int copy_thread_tls (unsigned long clone_flags,unsigned long sp,unsigned long arg,struct task_struct *p,unsigned long tls) {struct inactive_task_frame *frame;struct fork_frame *fork_frame;struct pt_regs *childregs; -int ret; +int ret =0 ; childregs =task_pt_regs (p); fork_frame = container_of(childregs, struct fork_frame, regs);
@@ -193,16 +169,13 @@ int copy_thread_tls(unsigned long clone_flags, unsigned long sp,task_user_gs (p) =get_user_gs (current_pt_regs ());#endif - ret =copy_io_bitmap (p); -if (ret) -return ret; -/* Set a new TLS for the child thread? */ -if (clone_flags &CLONE_SETTLS ) { +if (clone_flags &CLONE_SETTLS ) ret =set_new_tls (p, tls); -if (ret) -free_io_bitmap (p); - } + +if (!ret &&unlikely (test_tsk_thread_flag (current,TIF_IO_BITMAP ))) +io_bitmap_share (p); +return ret; }
PATCH v3 15/20 selftests/x86/ioperm: Extend testing so the shared bitmap is exercised
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
Add code to the fork path which forces the shared bitmap to be duplicated and the reference count to be dropped. Verify that the child modifications did not affect the parent.
---
V2: New patch
--- tools/testing/selftests/x86/ioperm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
tools/testing/selftests/x86/ioperm.c +15 -1
@@ -131,6 +131,17 @@ int main(void)printf ("[RUN]\tchild: check that we inherited permissions\n" );expect_ok (0x80 );expect_gp (0xed ); +printf ("[RUN]\tchild: Extend permissions to 0x81\n" ); +if (ioperm (0x81 ,1 ,1 ) !=0 ) { +printf ("[FAIL]\tioperm(0x81, 1, 1) failed (%d)" , errno); +return 1 ; + } +printf ("[RUN]\tchild: Drop permissions to 0x80\n" ); +if (ioperm (0x80 ,1 ,0 ) !=0 ) { +printf ("[FAIL]\tioperm(0x80, 1, 0) failed (%d)" , errno); +return 1 ; + } +expect_gp (0x80 );return 0 ; }else {int status;
@@ -146,8 +157,11 @@ int main(void) } } -/* Test the capability checks. */ +/* Verify that the child dropping 0x80 did not affect the parent */ +printf ("\tVerify that unsharing the bitmap worked\n" ); +expect_ok (0x80 ); +/* Test the capability checks. */ printf ("\tDrop privileges\n" );if (setresuid (1 ,1 ,1 ) !=0 ) {printf ("[WARN]\tDropping privileges failed\n" );
[tip: x86/iopl] selftests/x86/ioperm: Extend testing so the shared bitmap is exercised
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: b7091d239d19ef5cf69b139902fda665f74fd7c1 Gitweb: https://git.kernel.org/tip/b7091d239d19ef5cf69b139902fda665f74fd7c1 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:55 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:06 +01:00
selftests/x86/ioperm: Extend testing so the shared bitmap is exercised
Add code to the fork path which forces the shared bitmap to be duplicated and the reference count to be dropped. Verify that the child modifications did not affect the parent.
--- tools/testing/selftests/x86/ioperm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
tools/testing/selftests/x86/ioperm.c +15 -1
@@ -131,6 +131,17 @@ int main(void)printf ("[RUN]\tchild: check that we inherited permissions\n" );expect_ok (0x80 );expect_gp (0xed ); +printf ("[RUN]\tchild: Extend permissions to 0x81\n" ); +if (ioperm (0x81 ,1 ,1 ) !=0 ) { +printf ("[FAIL]\tioperm(0x81, 1, 1) failed (%d)" , errno); +return 1 ; + } +printf ("[RUN]\tchild: Drop permissions to 0x80\n" ); +if (ioperm (0x80 ,1 ,0 ) !=0 ) { +printf ("[FAIL]\tioperm(0x80, 1, 0) failed (%d)" , errno); +return 1 ; + } +expect_gp (0x80 );return 0 ; }else {int status;
@@ -146,8 +157,11 @@ int main(void) } } -/* Test the capability checks. */ +/* Verify that the child dropping 0x80 did not affect the parent */ +printf ("\tVerify that unsharing the bitmap worked\n" ); +expect_ok (0x80 ); +/* Test the capability checks. */ printf ("\tDrop privileges\n" );if (setresuid (1 ,1 ,1 ) !=0 ) {printf ("[WARN]\tDropping privileges failed\n" );
PATCH v3 16/20 x86/iopl: Fixup misleading comment
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
The comment for the sys_iopl() implementation is outdated and actively misleading in some parts. Fix it up.
--- arch/x86/kernel/ioport.c | 35 +++++++++++++++++++++++++++-------- 1 file changed, 27 insertions(+), 8 deletions(-)
arch/x86/kernel/ioport.c +27 -8
@@ -41,7 +41,7 @@ void io_bitmap_exit(void) }/* -* this changes the io permissions bitmap in the current task. +* This changes the io permissions bitmap in the current task. */ long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) {
@@ -136,14 +136,24 @@ SYSCALL_DEFINE3(ioperm, unsigned long, f }/* -* sys_iopl has to be used when you want to access the IO ports -* beyond the 0x3ff range: to get the full 65536 ports bitmapped -* you'd need 8kB of bitmaps/process, which is a bit excessive. +* The sys_iopl functionality depends on the level argument, which if +* granted for the task is used by the CPU to check I/O instruction and +* CLI/STI against the current priviledge level (CPL). If CPL is less than +* or equal the tasks IOPL level the instructions take effect. If not a #GP +* is raised. The default IOPL is 0, i.e. no permissions. * -* Here we just change the flags value on the stack: we allow -* only the super-user to do it. This depends on the stack-layout -* on system-call entry - see also fork() and the signal handling -* code. +* Setting IOPL to level 0-2 is disabling the userspace access. Only level +* 3 enables it. If set it allows the user space thread: +* +* - Unrestricted access to all 65535 I/O ports +* - The usage of CLI/STI instructions +* +* The advantage over ioperm is that the context switch does not require to +* update the I/O bitmap which is especially true when a large number of +* ports is accessed. But the allowance of CLI/STI in userspace is +* considered a major problem. +* +* IOPL is strictly per thread and inherited on fork. */ SYSCALL_DEFINE1(iopl, unsigned int, level) {
@@ -164,9 +174,18 @@ SYSCALL_DEFINE1(iopl, unsigned int, leve security_locked_down(LOCKDOWN_IOPORT))return -EPERM ; } +/* +* Change the flags value on the return stack, which has been set +* up on system-call entry. See also the fork and signal handling +* code how this is handled. +*/ regs->flags = (regs->flags & ~X86_EFLAGS_IOPL ) | (level <<X86_EFLAGS_IOPL_BIT ); +/* Store the new level in the thread struct */ t->iopl = level <<X86_EFLAGS_IOPL_BIT ; +/* +* X86_32 switches immediately and XEN handles it via emulation. +*/ set_iopl_mask (t->iopl);return 0 ;
[tip: x86/iopl] x86/iopl: Fixup misleading comment
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 7543e168c1d3c135d21747e494d99e60550c3828 Gitweb: https://git.kernel.org/tip/7543e168c1d3c135d21747e494d99e60550c3828 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:56 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:06 +01:00
x86/iopl: Fixup misleading comment
The comment for the sys_iopl() implementation is outdated and actively misleading in some parts. Fix it up.
--- arch/x86/kernel/ioport.c | 35 +++++++++++++++++++++++++++-------- 1 file changed, 27 insertions(+), 8 deletions(-)
arch/x86/kernel/ioport.c +27 -8
@@ -41,7 +41,7 @@ void io_bitmap_exit(void) }/* -* this changes the io permissions bitmap in the current task. +* This changes the io permissions bitmap in the current task. */ long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) {
@@ -136,14 +136,24 @@ SYSCALL_DEFINE3(ioperm, unsigned long, from, unsigned long, num, int, turn_on) }/* -* sys_iopl has to be used when you want to access the IO ports -* beyond the 0x3ff range: to get the full 65536 ports bitmapped -* you'd need 8kB of bitmaps/process, which is a bit excessive. +* The sys_iopl functionality depends on the level argument, which if +* granted for the task is used by the CPU to check I/O instruction and +* CLI/STI against the current priviledge level (CPL). If CPL is less than +* or equal the tasks IOPL level the instructions take effect. If not a #GP +* is raised. The default IOPL is 0, i.e. no permissions. * -* Here we just change the flags value on the stack: we allow -* only the super-user to do it. This depends on the stack-layout -* on system-call entry - see also fork() and the signal handling -* code. +* Setting IOPL to level 0-2 is disabling the userspace access. Only level +* 3 enables it. If set it allows the user space thread: +* +* - Unrestricted access to all 65535 I/O ports +* - The usage of CLI/STI instructions +* +* The advantage over ioperm is that the context switch does not require to +* update the I/O bitmap which is especially true when a large number of +* ports is accessed. But the allowance of CLI/STI in userspace is +* considered a major problem. +* +* IOPL is strictly per thread and inherited on fork. */ SYSCALL_DEFINE1(iopl, unsigned int, level) {
@@ -164,9 +174,18 @@ SYSCALL_DEFINE1(iopl, unsigned int, level) security_locked_down(LOCKDOWN_IOPORT))return -EPERM ; } +/* +* Change the flags value on the return stack, which has been set +* up on system-call entry. See also the fork and signal handling +* code how this is handled. +*/ regs->flags = (regs->flags & ~X86_EFLAGS_IOPL ) | (level <<X86_EFLAGS_IOPL_BIT ); +/* Store the new level in the thread struct */ t->iopl = level <<X86_EFLAGS_IOPL_BIT ; +/* +* X86_32 switches immediately and XEN handles it via emulation. +*/ set_iopl_mask (t->iopl);return 0 ;
PATCH v3 17/20 x86/iopl: Restrict iopl() permission scope
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
The access to the full I/O port range can be also provided by the TSS I/O bitmap, but that would require to copy 8k of data on scheduling in the task. As shown with the sched out optimization TSS.io_bitmap_base can be used to switch the incoming task to a preallocated I/O bitmap which has all bits zero, i.e. allows access to all I/O ports.
Implementing this allows to provide an iopl() emulation mode which restricts the IOPL level 3 permissions to I/O port access but removes the STI/CLI permission which is coming with the hardware IOPL mechansim.
Provide a config option to switch IOPL to emulation mode, make it the default and while at it also provide an option to disable IOPL completely.
---
V3: Folded the missing NULL pointer check, reduced preempt disable
scope (Ingo)
V2: Fixed the 32bit build fail by increasing the cpu entry area size
Move the TSS update out of the iopl() emulation code.
--- arch/x86/Kconfig | 32 +++++++++++ arch/x86/include/asm/pgtable_32_types.h | 2 arch/x86/include/asm/processor.h | 28 +++++++--- arch/x86/kernel/cpu/common.c | 5 + arch/x86/kernel/ioport.c | 87 ++++++++++++++++++++++---------- arch/x86/kernel/process.c | 28 ++++++---- 6 files changed, 137 insertions(+), 45 deletions(-)
arch/x86/Kconfig +32 -0
@@ -1254,6 +1254,38 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. +choice + prompt "IOPL" + default X86_IOPL_EMULATION + +config X86_IOPL_EMULATION + bool "IOPL Emulation" + ---help--- + Legacy IOPL support is an overbroad mechanism which allows user + space aside of accessing all 65536 I/O ports also to disable + interrupts. To gain this access the caller needs CAP_SYS_RAWIO + capabilities and permission from eventually active security + modules. + + The emulation restricts the functionality of the syscall to + only allowing the full range I/O port access, but prevents the + ability to disable interrupts from user space. + +config X86_IOPL_LEGACY + bool "IOPL Legacy" + ---help--- + Allow the full IOPL permissions, i.e. user space access to all + 65536 I/O ports and also the ability to disable interrupts, which + is overbroad and can result in system lockups. + +config X86_IOPL_NONE + bool "IOPL None" + ---help--- + Disable the IOPL permission syscall. That's the safest option as + no sane application should depend on this functionality. + +endchoice + config TOSHIBA tristate "Toshiba Laptop support" depends on X86_32
arch/x86/include/asm/pgtable_32_types.h +1 -1
@@ -44,7 +44,7 @@ extern bool __vmalloc_start_set; /* set* Define this here and validate with BUILD_BUG_ON() in pgtable_32.c * to avoid include recursion hell */ -#define CPU_ENTRY_AREA_PAGES (NR_CPUS * 40) +#define CPU_ENTRY_AREA_PAGES (NR_CPUS * 41)#define CPU_ENTRY_AREA_BASE \ ((FIXADDR_TOT_START - PAGE_SIZE * (CPU_ENTRY_AREA_PAGES + 1)) \
arch/x86/include/asm/processor.h +21 -7
@@ -332,19 +332,21 @@ struct x86_hw_tss {#define IO_BITMAP_BYTES (IO_BITMAP_BITS / BITS_PER_BYTE)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES / sizeof(long)) -#define IO_BITMAP_OFFSET_VALID \ +#define IO_BITMAP_OFFSET_VALID_MAP \ (offsetof(struct tss_struct, io_bitmap.bitmap) - \ offsetof(struct tss_struct, x86_tss)) +#define IO_BITMAP_OFFSET_VALID_ALL \ + (offsetof(struct tss_struct, io_bitmap.mapall) - \ + offsetof(struct tss_struct, x86_tss)) +/* -* sizeof(unsigned long) coming from an extra "long" at the end -* of the iobitmap. -* -* -1? seg base+limit should be pointing to the address of the -* last valid byte +* sizeof(unsigned long) coming from an extra "long" at the end of the +* iobitmap. The limit is inclusive, i.e. the last valid byte. */ #define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + (IO_BITMAP_OFFSET_VALID_ALL + IO_BITMAP_BYTES + \ + sizeof(unsigned long) - 1)/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ #define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)
@@ -380,6 +382,12 @@ struct x86_io_bitmap {* be within the limit. */ unsigned long bitmap[IO_BITMAP_LONGS +1 ]; + +/* +* Special I/O bitmap to emulate IOPL(3). All bytes zero, +* except the additional byte at the end. +*/ +unsigned long mapall[IO_BITMAP_LONGS +1 ]; };struct tss_struct {
@@ -506,7 +514,13 @@ struct thread_struct {#endif /* IO permissions: */ struct io_bitmap *io_bitmap; + +/* +* IOPL. Priviledge level dependent I/O permission which includes +* user space CLI/STI when granted. +*/ unsigned long iopl; +unsigned long iopl_emul;mm_segment_t addr_limit;
arch/x86/kernel/cpu/common.c +5 -0
@@ -1864,6 +1864,11 @@ void cpu_init(void) tss->io_bitmap.prev_max =0 ; tss->io_bitmap.prev_sequence =0 ;memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap)); +/* +* Invalidate the extra array entry past the end of the all +* permission bitmap as required by the hardware. +*/ + tss->io_bitmap.mapall[IO_BITMAP_LONGS ] = ~0UL ;set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();
arch/x86/kernel/ioport.c +61 -26
122 lines not on this page · raw message
arch/x86/kernel/process.c +17 -11
@@ -370,21 +370,27 @@ static void tss_copy_io_bitmap(struct tsvoid tss_update_io_bitmap (void ) {struct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw); +u16 *base = &tss->x86_tss.io_bitmap_base;if (test_thread_flag (TIF_IO_BITMAP )) { -struct io_bitmap *iobm = current->thread.io_bitmap; +struct thread_struct *t = ¤t->thread; -/* -* Only copy bitmap data when the sequence number -* differs. The update time is accounted to the incoming -* task. -*/ -if (tss->io_bitmap.prev_sequence != iobm->sequence) -tss_copy_io_bitmap (tss, iobm); - -/* Enable the bitmap */ - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ; +if (IS_ENABLED (CONFIG_X86_IOPL_EMULATION ) && + t->iopl_emul ==3 ) { + *base =IO_BITMAP_OFFSET_VALID_ALL ; + }else { +struct io_bitmap *iobm = t->io_bitmap; +/* +* Only copy bitmap data when the sequence number +* differs. The update time is accounted to the +* incoming task. +*/ +if (tss->io_bitmap.prev_sequence != iobm->sequence) +tss_copy_io_bitmap (tss, iobm); +/* Enable the bitmap */ + *base =IO_BITMAP_OFFSET_VALID_MAP ; + }/* * Make sure that the TSS limit is covering the io bitmap. * It might have been cut down by a VMEXIT to 0x67 which
Borislav Petkov
maintainerRe: [patch V3 17/20] x86/iopl: Restrict iopl() permission scope
Quoted 39 lines · Thomas Gleixner
From: Thomas Gleixner <[email protected]>
The access to the full I/O port range can be also provided by the TSS I/O bitmap, but that would require to copy 8k of data on scheduling in the task. As shown with the sched out optimization TSS.io_bitmap_base can be used to switch the incoming task to a preallocated I/O bitmap which has all bits zero, i.e. allows access to all I/O ports.
Implementing this allows to provide an
iopl()emulation mode which restricts the IOPL level 3 permissions to I/O port access but removes the STI/CLI permission which is coming with the hardware IOPL mechansim.Provide a config option to switch IOPL to emulation mode, make it the default and while at it also provide an option to disable IOPL completely.
Signed-off-by Thomas Gleixner <[email protected]>Acked-by Andy Lutomirski <[email protected]>---
V3: Folded the missing NULL pointer check, reduced preempt disable
scope (Ingo)V2: Fixed the 32bit build fail by increasing the cpu entry area size
Move the TSS update out of theiopl()emulation code.--- arch/x86/Kconfig | 32 +++++++++++ arch/x86/include/asm/pgtable_32_types.h | 2 arch/x86/include/asm/processor.h | 28 +++++++--- arch/x86/kernel/cpu/common.c | 5 + arch/x86/kernel/ioport.c | 87 ++++++++++++++++++++++---------- arch/x86/kernel/process.c | 28 ++++++---- 6 files changed, 137 insertions(+), 45 deletions(-)arch/x86/Kconfig +1
-0@@ -1254,6 +1254,38 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. +choice
arch/x86/Kconfig config X86_VSYSCALL_EMULATION · line 1267
Quoted 10 lines
arch/x86/Kconfig +10
-0+ prompt "IOPL" + default X86_IOPL_EMULATION + +config X86_IOPL_EMULATION + bool "IOPL Emulation" + ---help--- + Legacy IOPL support is an overbroad mechanism which allows user + space aside of accessing all 65536 I/O ports also to disable + interrupts. To gain this access the caller needs CAP_SYS_RAWIO + capabilities and permission from eventually active security
I think you mean here: s/eventually/potentially/ or so. "eventually" is one of the false friends. :)
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRe: [patch V3 17/20] x86/iopl: Restrict iopl() permission scope
Quoted 11 lines · Borislav Petkov
arch/x86/Kconfig config X86_VSYSCALL_EMULATION · line 1267
Quoted 7 lines · Thomas Gleixner
arch/x86/Kconfig +7
-0+config X86_IOPL_EMULATION + bool "IOPL Emulation" + ---help--- + Legacy IOPL support is an overbroad mechanism which allows user + space aside of accessing all 65536 I/O ports also to disable + interrupts. To gain this access the caller needs CAP_SYS_RAWIO + capabilities and permission from eventually active securityI think you mean here: s/eventually/potentially/ or so. "eventually" is one of the false friends. :)
Fixed
[tip: x86/iopl] x86/iopl: Restrict iopl() permission scope
In reply to Thomas GleixnerThe following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: ede65f28e61e5a5ccf6c46d9c8169abbd89fe24d Gitweb: https://git.kernel.org/tip/ede65f28e61e5a5ccf6c46d9c8169abbd89fe24d Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:57 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:07 +01:00
x86/iopl: Restrict iopl() permission scope
The access to the full I/O port range can be also provided by the TSS I/O bitmap, but that would require to copy 8k of data on scheduling in the task. As shown with the sched out optimization TSS.io_bitmap_base can be used to switch the incoming task to a preallocated I/O bitmap which has all bits zero, i.e. allows access to all I/O ports.
Implementing this allows to provide an iopl() emulation mode which restricts the IOPL level 3 permissions to I/O port access but removes the STI/CLI permission which is coming with the hardware IOPL mechansim.
Provide a config option to switch IOPL to emulation mode, make it the default and while at it also provide an option to disable IOPL completely.
--- arch/x86/Kconfig | 32 +++++++++- arch/x86/include/asm/pgtable_32_types.h | 2 +- arch/x86/include/asm/processor.h | 28 ++++++-- arch/x86/kernel/cpu/common.c | 5 +- arch/x86/kernel/ioport.c | 87 ++++++++++++++++-------- arch/x86/kernel/process.c | 32 +++++---- 6 files changed, 139 insertions(+), 47 deletions(-)
arch/x86/Kconfig +32 -0
@@ -1254,6 +1254,38 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. +choice + prompt "IOPL" + default X86_IOPL_EMULATION + +config X86_IOPL_EMULATION + bool "IOPL Emulation" + ---help--- + Legacy IOPL support is an overbroad mechanism which allows user + space aside of accessing all 65536 I/O ports also to disable + interrupts. To gain this access the caller needs CAP_SYS_RAWIO + capabilities and permission from potentially active security + modules. + + The emulation restricts the functionality of the syscall to + only allowing the full range I/O port access, but prevents the + ability to disable interrupts from user space. + +config X86_IOPL_LEGACY + bool "IOPL Legacy" + ---help--- + Allow the full IOPL permissions, i.e. user space access to all + 65536 I/O ports and also the ability to disable interrupts, which + is overbroad and can result in system lockups. + +config X86_IOPL_NONE + bool "IOPL None" + ---help--- + Disable the IOPL permission syscall. That's the safest option as + no sane application should depend on this functionality. + +endchoice + config TOSHIBA tristate "Toshiba Laptop support" depends on X86_32
arch/x86/include/asm/pgtable_32_types.h +1 -1
@@ -44,7 +44,7 @@ extern bool __vmalloc_start_set; /* set once high_memory is set */* Define this here and validate with BUILD_BUG_ON() in pgtable_32.c * to avoid include recursion hell */ -#define CPU_ENTRY_AREA_PAGES (NR_CPUS * 40) +#define CPU_ENTRY_AREA_PAGES (NR_CPUS * 41)#define CPU_ENTRY_AREA_BASE \ ((FIXADDR_TOT_START - PAGE_SIZE * (CPU_ENTRY_AREA_PAGES + 1)) \
arch/x86/include/asm/processor.h +21 -7
@@ -332,19 +332,21 @@ struct x86_hw_tss {#define IO_BITMAP_BYTES (IO_BITMAP_BITS / BITS_PER_BYTE)#define IO_BITMAP_LONGS (IO_BITMAP_BYTES / sizeof(long)) -#define IO_BITMAP_OFFSET_VALID \ +#define IO_BITMAP_OFFSET_VALID_MAP \ (offsetof(struct tss_struct, io_bitmap.bitmap) - \ offsetof(struct tss_struct, x86_tss)) +#define IO_BITMAP_OFFSET_VALID_ALL \ + (offsetof(struct tss_struct, io_bitmap.mapall) - \ + offsetof(struct tss_struct, x86_tss)) +/* -* sizeof(unsigned long) coming from an extra "long" at the end -* of the iobitmap. -* -* -1? seg base+limit should be pointing to the address of the -* last valid byte +* sizeof(unsigned long) coming from an extra "long" at the end of the +* iobitmap. The limit is inclusive, i.e. the last valid byte. */ #define __KERNEL_TSS_LIMIT \ - (IO_BITMAP_OFFSET_VALID + IO_BITMAP_BYTES + sizeof(unsigned long) - 1) + (IO_BITMAP_OFFSET_VALID_ALL + IO_BITMAP_BYTES + \ + sizeof(unsigned long) - 1)/* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ #define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)
@@ -380,6 +382,12 @@ struct x86_io_bitmap {* be within the limit. */ unsigned long bitmap[IO_BITMAP_LONGS +1 ]; + +/* +* Special I/O bitmap to emulate IOPL(3). All bytes zero, +* except the additional byte at the end. +*/ +unsigned long mapall[IO_BITMAP_LONGS +1 ]; };struct tss_struct {
@@ -506,7 +514,13 @@ struct thread_struct {#endif /* IO permissions: */ struct io_bitmap *io_bitmap; + +/* +* IOPL. Priviledge level dependent I/O permission which includes +* user space CLI/STI when granted. +*/ unsigned long iopl; +unsigned long iopl_emul;mm_segment_t addr_limit;
arch/x86/kernel/cpu/common.c +5 -0
@@ -1864,6 +1864,11 @@ void cpu_init(void) tss->io_bitmap.prev_max =0 ; tss->io_bitmap.prev_sequence =0 ;memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap)); +/* +* Invalidate the extra array entry past the end of the all +* permission bitmap as required by the hardware. +*/ + tss->io_bitmap.mapall[IO_BITMAP_LONGS ] = ~0UL ;set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();
arch/x86/kernel/ioport.c +61 -26
122 lines not on this page · raw message
arch/x86/kernel/process.c +19 -13
@@ -370,21 +370,27 @@ static void tss_copy_io_bitmap(struct tss_struct *tss, struct io_bitmap *iobm)void tss_update_io_bitmap (void ) {struct tss_struct *tss =this_cpu_ptr (&cpu_tss_rw); +u16 *base = &tss->x86_tss.io_bitmap_base;if (test_thread_flag (TIF_IO_BITMAP )) { -struct io_bitmap *iobm = current->thread.io_bitmap; - -/* -* Only copy bitmap data when the sequence number -* differs. The update time is accounted to the incoming -* task. -*/ -if (tss->io_bitmap.prev_sequence != iobm->sequence) -tss_copy_io_bitmap (tss, iobm); - -/* Enable the bitmap */ - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_VALID ; - +struct thread_struct *t = ¤t->thread; + +if (IS_ENABLED (CONFIG_X86_IOPL_EMULATION ) && + t->iopl_emul ==3 ) { + *base =IO_BITMAP_OFFSET_VALID_ALL ; + }else { +struct io_bitmap *iobm = t->io_bitmap; +/* +* Only copy bitmap data when the sequence number +* differs. The update time is accounted to the +* incoming task. +*/ +if (tss->io_bitmap.prev_sequence != iobm->sequence) +tss_copy_io_bitmap (tss, iobm); + +/* Enable the bitmap */ + *base =IO_BITMAP_OFFSET_VALID_MAP ; + }/* * Make sure that the TSS limit is covering the io bitmap. * It might have been cut down by a VMEXIT to 0x67 which
PATCH v3 18/20 x86/iopl: Remove legacy IOPL option
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
The IOPL emulation via the I/O bitmap is sufficient. Remove the legacy cruft dealing with the (e)flags based IOPL mechanism.
---
V3: Simplified config option (removed the choice)
V2: Adapted to changes in the previous patches.
--- arch/x86/Kconfig | 23 ++-------------- arch/x86/include/asm/paravirt.h | 4 -- arch/x86/include/asm/paravirt_types.h | 2 - arch/x86/include/asm/processor.h | 26 ++---------------- arch/x86/include/asm/xen/hypervisor.h | 2 - arch/x86/kernel/ioport.c | 47 +++++++--------------------------- arch/x86/kernel/paravirt.c | 2 - arch/x86/kernel/process_32.c | 9 ------ arch/x86/kernel/process_64.c | 11 ------- arch/x86/xen/enlighten_pv.c | 10 ------- 10 files changed, 17 insertions(+), 119 deletions(-)
arch/x86/Kconfig +3 -20
@@ -1254,12 +1254,9 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. -choice - prompt "IOPL" - default X86_IOPL_EMULATION - config X86_IOPL_EMULATION bool "IOPL Emulation" + default y ---help--- Legacy IOPL support is an overbroad mechanism which allows user space aside of accessing all 65536 I/O ports also to disable
@@ -1269,22 +1266,8 @@ config X86_IOPL_EMULATION The emulation restricts the functionality of the syscall to only allowing the full range I/O port access, but prevents the - ability to disable interrupts from user space. - -config X86_IOPL_LEGACY - bool "IOPL Legacy" - ---help--- - Allow the full IOPL permissions, i.e. user space access to all - 65536 I/O ports and also the ability to disable interrupts, which - is overbroad and can result in system lockups. - -config X86_IOPL_NONE - bool "IOPL None" - ---help--- - Disable the IOPL permission syscall. That's the safest option as - no sane application should depend on this functionality. - -endchoice + ability to disable interrupts from user space which would be + granted if the hardware IOPL mechanism would be used. config TOSHIBA tristate "Toshiba Laptop support"
arch/x86/include/asm/paravirt.h +0 -4
@@ -294,10 +294,6 @@ static inline void write_idt_entry(gate_ {PVOP_VCALL3 (cpu.write_idt_entry, dt, entry, g); } -static inline void set_iopl_mask (unsigned mask) -{ -PVOP_VCALL1 (cpu.set_iopl_mask, mask); -}static inline void paravirt_activate_mm (struct mm_struct *prev,struct mm_struct *next)
arch/x86/include/asm/paravirt_types.h +0 -2
@@ -140,8 +140,6 @@ struct pv_cpu_ops {void (*load_sp0)(unsigned long sp0); -void (*set_iopl_mask)(unsigned mask); -void (*wbinvd)(void );/* cpuid emulation, mostly so that caps bits can be disabled */
arch/x86/include/asm/processor.h +3 -23
@@ -516,10 +516,10 @@ struct thread_struct {struct io_bitmap *io_bitmap;/* -* IOPL. Priviledge level dependent I/O permission which includes -* user space CLI/STI when granted. +* IOPL. Priviledge level dependent I/O permission which is +* emulated via the I/O bitmap to prevent user space from disabling +* interrupts. */ -unsigned long iopl;unsigned long iopl_emul;mm_segment_t addr_limit;
@@ -552,25 +552,6 @@ static inline void arch_thread_struct_wh*/ #define TS_COMPAT 0x0002/* 32bit syscall active (64BIT)*/ -/* -* Set IOPL bits in EFLAGS from given mask -*/ -static inline void native_set_iopl_mask (unsigned mask) -{ -#ifdef CONFIG_X86_32 -unsigned int reg; - - asmvolatile ("pushfl;" -"popl %0;" -"andl %1, %0;" -"orl %2, %0;" -"pushl %0;" -"popfl" - :"=&r" (reg) - :"i" (~X86_EFLAGS_IOPL ),"r" (mask)); -#endif -} -static inline void native_load_sp0 (unsigned long sp0) {
@@ -610,7 +591,6 @@ static inline void load_sp0(unsigned lon native_load_sp0(sp0); } -#define set_iopl_mask native_set_iopl_mask#endif /* CONFIG_PARAVIRT_XXL */ /* Free all resources held by a thread. */
arch/x86/include/asm/xen/hypervisor.h +0 -2
@@ -62,6 +62,4 @@ void xen_arch_register_cpu(int num);void xen_arch_unregister_cpu (int num);#endif -extern void xen_set_iopl_mask (unsigned mask); -#endif /* _ASM_X86_XEN_HYPERVISOR_H */
arch/x86/kernel/ioport.c +11 -36
74 lines not on this page · raw message
arch/x86/kernel/paravirt.c +0 -2
@@ -341,8 +341,6 @@ struct paravirt_patch_template pv_ops = .cpu.iret = native_iret, .cpu.swapgs = native_swapgs, - .cpu.set_iopl_mask = native_set_iopl_mask, - .cpu.start_context_switch = paravirt_nop, .cpu.end_context_switch = paravirt_nop,
arch/x86/kernel/process_32.c +0 -9
@@ -187,15 +187,6 @@ EXPORT_SYMBOL_GPL(start_thread);*/ load_TLS (next, cpu); -/* -* Restore IOPL if needed. In normal use, the flags restore -* in the switch assembly will handle this. But if the kernel -* is running virtualized at a non-zero CPL, the popf will -* not restore flags, so it must be done in a separate step. -*/ -if (get_kernel_rpl () &&unlikely (prev->iopl != next->iopl)) -set_iopl_mask (next->iopl); -switch_to_extra (prev_p, next_p);/*
arch/x86/kernel/process_64.c +0 -11
@@ -497,17 +497,6 @@ void compat_start_thread(struct pt_regsswitch_to_extra (prev_p, next_p); -#ifdef CONFIG_XEN_PV -/* -* On Xen PV, IOPL bits in pt_regs->flags have no effect, and -* current_pt_regs()->flags may not match the current task's -* intended IOPL. We need to switch it manually. -*/ -if (unlikely (static_cpu_has (X86_FEATURE_XENPV ) && - prev->iopl != next->iopl)) -xen_set_iopl_mask (next->iopl); -#endif -if (static_cpu_has_bug (X86_BUG_SYSRET_SS_ATTRS )) {/* * AMD CPUs have a misfeature: SYSRET sets the SS selector but
arch/x86/xen/enlighten_pv.c +0 -10
@@ -837,15 +837,6 @@ static void xen_load_sp0(unsigned long sthis_cpu_write (cpu_tss_rw.x86_tss.sp0, sp0); } -void xen_set_iopl_mask (unsigned mask) -{ -struct physdev_set_iopl set_iopl; - -/* Force the change at ring 0. */ - set_iopl.iopl = (mask ==0 ) ?1 : (mask >>12 ) &3 ; -HYPERVISOR_physdev_op (PHYSDEVOP_set_iopl, &set_iopl); -} -static void xen_io_delay (void ) { }
@@ -1055,7 +1046,6 @@ static const struct pv_cpu_ops xen_cpu_o .write_idt_entry = xen_write_idt_entry, .load_sp0 = xen_load_sp0, - .set_iopl_mask = xen_set_iopl_mask, .io_delay = xen_io_delay,/* Xen takes care of %gs when switching to usermode for us */
[tip: x86/iopl] x86/iopl: Remove legacy IOPL option
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 5650e933f0fcc7f76c13878eb944c04bcf1fda32 Gitweb: https://git.kernel.org/tip/5650e933f0fcc7f76c13878eb944c04bcf1fda32 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:58 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:07 +01:00
x86/iopl: Remove legacy IOPL option
The IOPL emulation via the I/O bitmap is sufficient. Remove the legacy cruft dealing with the (e)flags based IOPL mechanism.
--- arch/x86/Kconfig | 23 +------------ arch/x86/include/asm/paravirt.h | 4 +-- arch/x86/include/asm/paravirt_types.h | 2 +- arch/x86/include/asm/processor.h | 26 +------------- arch/x86/include/asm/xen/hypervisor.h | 2 +- arch/x86/kernel/ioport.c | 47 ++++++-------------------- arch/x86/kernel/paravirt.c | 2 +- arch/x86/kernel/process_32.c | 9 +----- arch/x86/kernel/process_64.c | 11 +------ arch/x86/xen/enlighten_pv.c | 10 +------ 10 files changed, 17 insertions(+), 119 deletions(-)
arch/x86/Kconfig +3 -20
@@ -1254,12 +1254,9 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. -choice - prompt "IOPL" - default X86_IOPL_EMULATION - config X86_IOPL_EMULATION bool "IOPL Emulation" + default y ---help--- Legacy IOPL support is an overbroad mechanism which allows user space aside of accessing all 65536 I/O ports also to disable
@@ -1269,22 +1266,8 @@ config X86_IOPL_EMULATION The emulation restricts the functionality of the syscall to only allowing the full range I/O port access, but prevents the - ability to disable interrupts from user space. - -config X86_IOPL_LEGACY - bool "IOPL Legacy" - ---help--- - Allow the full IOPL permissions, i.e. user space access to all - 65536 I/O ports and also the ability to disable interrupts, which - is overbroad and can result in system lockups. - -config X86_IOPL_NONE - bool "IOPL None" - ---help--- - Disable the IOPL permission syscall. That's the safest option as - no sane application should depend on this functionality. - -endchoice + ability to disable interrupts from user space which would be + granted if the hardware IOPL mechanism would be used. config TOSHIBA tristate "Toshiba Laptop support"
arch/x86/include/asm/paravirt.h +0 -4
@@ -294,10 +294,6 @@ static inline void write_idt_entry(gate_desc *dt, int entry, const gate_desc *g) {PVOP_VCALL3 (cpu.write_idt_entry, dt, entry, g); } -static inline void set_iopl_mask (unsigned mask) -{ -PVOP_VCALL1 (cpu.set_iopl_mask, mask); -}static inline void paravirt_activate_mm (struct mm_struct *prev,struct mm_struct *next)
arch/x86/include/asm/paravirt_types.h +0 -2
@@ -140,8 +140,6 @@ struct pv_cpu_ops {void (*load_sp0)(unsigned long sp0); -void (*set_iopl_mask)(unsigned mask); -void (*wbinvd)(void );/* cpuid emulation, mostly so that caps bits can be disabled */
arch/x86/include/asm/processor.h +3 -23
@@ -516,10 +516,10 @@ struct thread_struct {struct io_bitmap *io_bitmap;/* -* IOPL. Priviledge level dependent I/O permission which includes -* user space CLI/STI when granted. +* IOPL. Priviledge level dependent I/O permission which is +* emulated via the I/O bitmap to prevent user space from disabling +* interrupts. */ -unsigned long iopl;unsigned long iopl_emul;mm_segment_t addr_limit;
@@ -552,25 +552,6 @@ static inline void arch_thread_struct_whitelist(unsigned long *offset,*/ #define TS_COMPAT 0x0002/* 32bit syscall active (64BIT)*/ -/* -* Set IOPL bits in EFLAGS from given mask -*/ -static inline void native_set_iopl_mask (unsigned mask) -{ -#ifdef CONFIG_X86_32 -unsigned int reg; - - asmvolatile ("pushfl;" -"popl %0;" -"andl %1, %0;" -"orl %2, %0;" -"pushl %0;" -"popfl" - :"=&r" (reg) - :"i" (~X86_EFLAGS_IOPL ),"r" (mask)); -#endif -} -static inline void native_load_sp0 (unsigned long sp0) {
@@ -610,7 +591,6 @@ static inline void load_sp0(unsigned long sp0) native_load_sp0(sp0); } -#define set_iopl_mask native_set_iopl_mask#endif /* CONFIG_PARAVIRT_XXL */ /* Free all resources held by a thread. */
arch/x86/include/asm/xen/hypervisor.h +0 -2
@@ -62,6 +62,4 @@ void xen_arch_register_cpu(int num);void xen_arch_unregister_cpu (int num);#endif -extern void xen_set_iopl_mask (unsigned mask); -#endif /* _ASM_X86_XEN_HYPERVISOR_H */
arch/x86/kernel/ioport.c +11 -36
74 lines not on this page · raw message
arch/x86/kernel/paravirt.c +0 -2
@@ -341,8 +341,6 @@ struct paravirt_patch_template pv_ops = { .cpu.iret = native_iret, .cpu.swapgs = native_swapgs, - .cpu.set_iopl_mask = native_set_iopl_mask, - .cpu.start_context_switch = paravirt_nop, .cpu.end_context_switch = paravirt_nop,
arch/x86/kernel/process_32.c +0 -9
@@ -187,15 +187,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)*/ load_TLS (next, cpu); -/* -* Restore IOPL if needed. In normal use, the flags restore -* in the switch assembly will handle this. But if the kernel -* is running virtualized at a non-zero CPL, the popf will -* not restore flags, so it must be done in a separate step. -*/ -if (get_kernel_rpl () &&unlikely (prev->iopl != next->iopl)) -set_iopl_mask (next->iopl); -switch_to_extra (prev_p, next_p);/*
arch/x86/kernel/process_64.c +0 -11
@@ -497,17 +497,6 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p)switch_to_extra (prev_p, next_p); -#ifdef CONFIG_XEN_PV -/* -* On Xen PV, IOPL bits in pt_regs->flags have no effect, and -* current_pt_regs()->flags may not match the current task's -* intended IOPL. We need to switch it manually. -*/ -if (unlikely (static_cpu_has (X86_FEATURE_XENPV ) && - prev->iopl != next->iopl)) -xen_set_iopl_mask (next->iopl); -#endif -if (static_cpu_has_bug (X86_BUG_SYSRET_SS_ATTRS )) {/* * AMD CPUs have a misfeature: SYSRET sets the SS selector but
arch/x86/xen/enlighten_pv.c +0 -10
@@ -837,15 +837,6 @@ static void xen_load_sp0(unsigned long sp0)this_cpu_write (cpu_tss_rw.x86_tss.sp0, sp0); } -void xen_set_iopl_mask (unsigned mask) -{ -struct physdev_set_iopl set_iopl; - -/* Force the change at ring 0. */ - set_iopl.iopl = (mask ==0 ) ?1 : (mask >>12 ) &3 ; -HYPERVISOR_physdev_op (PHYSDEVOP_set_iopl, &set_iopl); -} -static void xen_io_delay (void ) { }
@@ -1055,7 +1046,6 @@ static const struct pv_cpu_ops xen_cpu_ops __initconst = { .write_idt_entry = xen_write_idt_entry, .load_sp0 = xen_load_sp0, - .set_iopl_mask = xen_set_iopl_mask, .io_delay = xen_io_delay,/* Xen takes care of %gs when switching to usermode for us */
PATCH v3 19/20 x86/ioperm: Extend IOPL config to control ioperm() as well
In reply to Thomas GleixnerIf iopl() is disabled, then providing ioperm() does not make much sense.
Rename the config option and disable/enable both syscalls with it. Guard the code with #ifdefs where appropriate.
---
V3: New patch
--- arch/x86/Kconfig | 7 +++++-- arch/x86/include/asm/io_bitmap.h | 6 ++++++ arch/x86/include/asm/processor.h | 9 ++++++++- arch/x86/include/asm/thread_info.h | 7 ++++++- arch/x86/kernel/cpu/common.c | 26 +++++++++++++++++--------- arch/x86/kernel/ioport.c | 26 +++++++++++++++++++------- arch/x86/kernel/process.c | 4 ++++ 7 files changed, 65 insertions(+), 20 deletions(-)
arch/x86/Kconfig +5 -2
@@ -1254,10 +1254,13 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. -config X86_IOPL_EMULATION - bool "IOPL Emulation" +config X86_IOPL_IOPERM + bool "IOPERM and IOPL Emulation" default y ---help--- + This enables the ioperm() and iopl() syscalls which are necessary + for legacy applications. + Legacy IOPL support is an overbroad mechanism which allows user space aside of accessing all 65536 I/O ports also to disable interrupts. To gain this access the caller needs CAP_SYS_RAWIO
arch/x86/include/asm/io_bitmap.h +6 -0
@@ -15,9 +15,15 @@ struct io_bitmap {struct task_struct ; +#ifdef CONFIG_X86_IOPL_IOPERM void io_bitmap_share (struct task_struct *tsk);void io_bitmap_exit (void );void tss_update_io_bitmap (void ); +#else +static inline void io_bitmap_share (struct task_struct *tsk) { } +static inline void io_bitmap_exit (void ) { } +static inline void tss_update_io_bitmap (void ) { } +#endif #endif
arch/x86/include/asm/processor.h +8 -1
@@ -340,13 +340,18 @@ struct x86_hw_tss { (offsetof(struct tss_struct, io_bitmap.mapall) - \ offsetof(struct tss_struct, x86_tss)) +#ifdef CONFIG_X86_IOPL_IOPERM /* * sizeof(unsigned long) coming from an extra "long" at the end of the * iobitmap. The limit is inclusive, i.e. the last valid byte. */ -#define __KERNEL_TSS_LIMIT \ +# define __KERNEL_TSS_LIMIT \ (IO_BITMAP_OFFSET_VALID_ALL + IO_BITMAP_BYTES + \ sizeof(unsigned long) - 1) +#else +# define __KERNEL_TSS_LIMIT \ + (offsetof(struct tss_struct, x86_tss) + sizeof(struct x86_hw_tss) - 1) +#endif /* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ #define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)
@@ -398,7 +403,9 @@ struct tss_struct {*/ struct x86_hw_tss x86_tss; +#ifdef CONFIG_X86_IOPL_IOPERM struct x86_io_bitmap io_bitmap; +#endif }__aligned(PAGE_SIZE) ; DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw);
arch/x86/include/asm/thread_info.h +6 -1
@@ -156,8 +156,13 @@ struct thread_info {# define _TIF_WORK_CTXSW (_TIF_WORK_CTXSW_BASE)#endif -#define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY | \ +#ifdef CONFIG_X86_IOPL_IOPERM +# define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY | \ _TIF_IO_BITMAP) +#else +# define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY) +#endif +#define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW)#define STACK_WARN (THREAD_SIZE/8)
arch/x86/kernel/cpu/common.c +17 -9
@@ -1804,6 +1804,22 @@ static inline void gdt_setup_doublefault }#endif /* !CONFIG_X86_64 */ +static inline void tss_setup_io_bitmap (struct tss_struct *tss) +{ + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + +#ifdef CONFIG_X86_IOPL_IOPERM + tss->io_bitmap.prev_max =0 ; + tss->io_bitmap.prev_sequence =0 ; +memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap)); +/* +* Invalidate the extra array entry past the end of the all +* permission bitmap as required by the hardware. +*/ + tss->io_bitmap.mapall[IO_BITMAP_LONGS ] = ~0UL ; +#endif +} +/* * cpu_init() initializes state that is per-CPU. Some data is already * initialized (naturally) in the bootstrap process, such as the GDT
@@ -1860,15 +1876,7 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; - tss->io_bitmap.prev_max =0 ; - tss->io_bitmap.prev_sequence =0 ; -memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap)); -/* -* Invalidate the extra array entry past the end of the all -* permission bitmap as required by the hardware. -*/ - tss->io_bitmap.mapall[IO_BITMAP_LONGS ] = ~0UL ; +tss_setup_io_bitmap (tss);set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();
arch/x86/kernel/ioport.c +19 -7
@@ -14,6 +14,8 @@#include <asm/io_bitmap.h> #include <asm/desc.h> +#ifdef CONFIG_X86_IOPL_IOPERM +static atomic64_t io_bitmap_sequence;void io_bitmap_share (struct task_struct *tsk)
@@ -172,13 +174,6 @@ SYSCALL_DEFINE1(iopl, unsigned int, levestruct thread_struct *t = ¤t->thread;unsigned int old; -/* -* Careful: the IOPL bits in regs->flags are undefined under Xen PV -* and changing them has no effect. -*/ -if (IS_ENABLED (CONFIG_X86_IOPL_NONE )) -return -ENOSYS ; -if (level >3 )return -EINVAL ;
@@ -200,3 +195,20 @@ SYSCALL_DEFINE1(iopl, unsigned int, levereturn 0 ; } + +#else /* CONFIG_X86_IOPL_IOPERM */ + +long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) +{ +return -ENOSYS ; +} +SYSCALL_DEFINE3(ioperm, unsigned long, from, unsigned long, num, int, turn_on) +{ +return -ENOSYS ; +} + +SYSCALL_DEFINE1(iopl, unsigned int, level) +{ +return -ENOSYS ; +} +#endif
arch/x86/kernel/process.c +4 -0
@@ -316,6 +316,7 @@ void arch_setup_new_exec(void) } } +#ifdef CONFIG_X86_IOPL_IOPERM static inline void tss_invalidate_io_bitmap (struct tss_struct *tss) {/*
@@ -403,6 +404,9 @@ void tss_update_io_bitmap(void) tss_invalidate_io_bitmap(tss); } } +#else /* CONFIG_X86_IOPL_IOPERM */ +static inline void switch_to_bitmap (unsigned long tifp) { } +#endif #ifdef CONFIG_SMP
[tip: x86/iopl] x86/ioperm: Extend IOPL config to control ioperm() as well
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 845fa6a960f45e07ab22b660bee7f7644dd29610 Gitweb: https://git.kernel.org/tip/845fa6a960f45e07ab22b660bee7f7644dd29610 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:42:59 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:08 +01:00
x86/ioperm: Extend IOPL config to control ioperm() as well
If iopl() is disabled, then providing ioperm() does not make much sense.
Rename the config option and disable/enable both syscalls with it. Guard the code with #ifdefs where appropriate.
--- arch/x86/Kconfig | 7 +++++-- arch/x86/include/asm/io_bitmap.h | 6 ++++++ arch/x86/include/asm/processor.h | 9 ++++++++- arch/x86/include/asm/thread_info.h | 7 ++++++- arch/x86/kernel/cpu/common.c | 26 +++++++++++++++++--------- arch/x86/kernel/ioport.c | 26 +++++++++++++++++++------- arch/x86/kernel/process.c | 4 ++++ 7 files changed, 65 insertions(+), 20 deletions(-)
arch/x86/Kconfig +5 -2
@@ -1254,10 +1254,13 @@ config X86_VSYSCALL_EMULATION Disabling this option saves about 7K of kernel size and possibly 4K of additional runtime pagetable memory. -config X86_IOPL_EMULATION - bool "IOPL Emulation" +config X86_IOPL_IOPERM + bool "IOPERM and IOPL Emulation" default y ---help--- + This enables the ioperm() and iopl() syscalls which are necessary + for legacy applications. + Legacy IOPL support is an overbroad mechanism which allows user space aside of accessing all 65536 I/O ports also to disable interrupts. To gain this access the caller needs CAP_SYS_RAWIO
arch/x86/include/asm/io_bitmap.h +6 -0
@@ -15,9 +15,15 @@ struct io_bitmap {struct task_struct ; +#ifdef CONFIG_X86_IOPL_IOPERM void io_bitmap_share (struct task_struct *tsk);void io_bitmap_exit (void );void tss_update_io_bitmap (void ); +#else +static inline void io_bitmap_share (struct task_struct *tsk) { } +static inline void io_bitmap_exit (void ) { } +static inline void tss_update_io_bitmap (void ) { } +#endif #endif
arch/x86/include/asm/processor.h +8 -1
@@ -340,13 +340,18 @@ struct x86_hw_tss { (offsetof(struct tss_struct, io_bitmap.mapall) - \ offsetof(struct tss_struct, x86_tss)) +#ifdef CONFIG_X86_IOPL_IOPERM /* * sizeof(unsigned long) coming from an extra "long" at the end of the * iobitmap. The limit is inclusive, i.e. the last valid byte. */ -#define __KERNEL_TSS_LIMIT \ +# define __KERNEL_TSS_LIMIT \ (IO_BITMAP_OFFSET_VALID_ALL + IO_BITMAP_BYTES + \ sizeof(unsigned long) - 1) +#else +# define __KERNEL_TSS_LIMIT \ + (offsetof(struct tss_struct, x86_tss) + sizeof(struct x86_hw_tss) - 1) +#endif /* Base offset outside of TSS_LIMIT so unpriviledged IO causes #GP */ #define IO_BITMAP_OFFSET_INVALID (__KERNEL_TSS_LIMIT + 1)
@@ -398,7 +403,9 @@ struct tss_struct {*/ struct x86_hw_tss x86_tss; +#ifdef CONFIG_X86_IOPL_IOPERM struct x86_io_bitmap io_bitmap; +#endif }__aligned(PAGE_SIZE) ; DECLARE_PER_CPU_PAGE_ALIGNED(struct tss_struct, cpu_tss_rw);
arch/x86/include/asm/thread_info.h +6 -1
@@ -156,8 +156,13 @@ struct thread_info {# define _TIF_WORK_CTXSW (_TIF_WORK_CTXSW_BASE)#endif -#define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY | \ +#ifdef CONFIG_X86_IOPL_IOPERM +# define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY | \ _TIF_IO_BITMAP) +#else +# define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW| _TIF_USER_RETURN_NOTIFY) +#endif +#define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW)#define STACK_WARN (THREAD_SIZE/8)
arch/x86/kernel/cpu/common.c +17 -9
@@ -1804,6 +1804,22 @@ static inline void gdt_setup_doublefault_tss(int cpu) }#endif /* !CONFIG_X86_64 */ +static inline void tss_setup_io_bitmap (struct tss_struct *tss) +{ + tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; + +#ifdef CONFIG_X86_IOPL_IOPERM + tss->io_bitmap.prev_max =0 ; + tss->io_bitmap.prev_sequence =0 ; +memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap)); +/* +* Invalidate the extra array entry past the end of the all +* permission bitmap as required by the hardware. +*/ + tss->io_bitmap.mapall[IO_BITMAP_LONGS ] = ~0UL ; +#endif +} +/* * cpu_init() initializes state that is per-CPU. Some data is already * initialized (naturally) in the bootstrap process, such as the GDT
@@ -1860,15 +1876,7 @@ void cpu_init(void)/* Initialize the TSS. */ tss_setup_ist (tss); - tss->x86_tss.io_bitmap_base =IO_BITMAP_OFFSET_INVALID ; - tss->io_bitmap.prev_max =0 ; - tss->io_bitmap.prev_sequence =0 ; -memset (tss->io_bitmap.bitmap,0xff ,sizeof (tss->io_bitmap.bitmap)); -/* -* Invalidate the extra array entry past the end of the all -* permission bitmap as required by the hardware. -*/ - tss->io_bitmap.mapall[IO_BITMAP_LONGS ] = ~0UL ; +tss_setup_io_bitmap (tss);set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);load_TR_desc ();
arch/x86/kernel/ioport.c +19 -7
@@ -14,6 +14,8 @@#include <asm/io_bitmap.h> #include <asm/desc.h> +#ifdef CONFIG_X86_IOPL_IOPERM +static atomic64_t io_bitmap_sequence;void io_bitmap_share (struct task_struct *tsk)
@@ -172,13 +174,6 @@ SYSCALL_DEFINE1(iopl, unsigned int, level)struct thread_struct *t = ¤t->thread;unsigned int old; -/* -* Careful: the IOPL bits in regs->flags are undefined under Xen PV -* and changing them has no effect. -*/ -if (IS_ENABLED (CONFIG_X86_IOPL_NONE )) -return -ENOSYS ; -if (level >3 )return -EINVAL ;
@@ -200,3 +195,20 @@ SYSCALL_DEFINE1(iopl, unsigned int, level)return 0 ; } + +#else /* CONFIG_X86_IOPL_IOPERM */ + +long ksys_ioperm (unsigned long from,unsigned long num,int turn_on) +{ +return -ENOSYS ; +} +SYSCALL_DEFINE3(ioperm, unsigned long, from, unsigned long, num, int, turn_on) +{ +return -ENOSYS ; +} + +SYSCALL_DEFINE1(iopl, unsigned int, level) +{ +return -ENOSYS ; +} +#endif
arch/x86/kernel/process.c +4 -0
@@ -316,6 +316,7 @@ void arch_setup_new_exec(void) } } +#ifdef CONFIG_X86_IOPL_IOPERM static inline void tss_invalidate_io_bitmap (struct tss_struct *tss) {/*
@@ -403,6 +404,9 @@ void tss_update_io_bitmap(void) tss_invalidate_io_bitmap(tss); } } +#else /* CONFIG_X86_IOPL_IOPERM */ +static inline void switch_to_bitmap (unsigned long tifp) { } +#endif #ifdef CONFIG_SMP
PATCH v3 20/20 selftests/x86/iopl: Extend test to cover IOPL emulation
In reply to Thomas GleixnerFrom: Thomas Gleixner <[email protected]>
Add tests that the now emulated iopl() functionality:
- does not longer allow user space to disable interrupts.
- does restore a I/O bitmap when IOPL is dropped
--- tools/testing/selftests/x86/iopl.c | 129 +++++++++++++++++++++++++++++++++---- 1 file changed, 118 insertions(+), 11 deletions(-)
tools/testing/selftests/x86/iopl.c +118 -11
@@ -35,6 +35,16 @@ static void sethandler(int sig, void (*h } +static void clearhandler (int sig) +{ +struct sigaction sa; +memset (&sa,0 ,sizeof (sa)); + sa.sa_handler =SIG_DFL ; +sigemptyset (&sa.sa_mask); +if (sigaction (sig, &sa,0 )) +err (1 ,"sigaction" ); +} +static jmp_buf jmpbuf;static void sigsegv (int sig,siginfo_t *si,void *ctx_void)
@@ -42,25 +52,128 @@ static void sigsegv(int sig, siginfo_t *siglongjmp (jmpbuf,1 ); } +static bool try_outb (unsigned short port) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("outb %%al, %w[port]" + : : [port]"Nd" (port),"a" (0 )); +return true; + } +clearhandler (SIGSEGV ); +} + +static void expect_ok_outb (unsigned short port) +{ +if (!try_outb (port)) { +printf ("[FAIL]\toutb to 0x%02hx failed\n" , port); +exit (1 ); + } + +printf ("[OK]\toutb to 0x%02hx worked\n" , port); +} + +static void expect_gp_outb (unsigned short port) +{ +if (try_outb (port)) { +printf ("[FAIL]\toutb to 0x%02hx worked\n" , port); + nerrs++; + } + +printf ("[OK]\toutb to 0x%02hx failed\n" , port); +} + +static bool try_cli (void ) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("cli" ); +return true; + } +clearhandler (SIGSEGV ); +} + +static bool try_sti (void ) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("sti" ); +return true; + } +clearhandler (SIGSEGV ); +} + +static void expect_gp_sti (void ) +{ +if (try_sti ()) { +printf ("[FAIL]\tSTI worked\n" ); + nerrs++; + }else { +printf ("[OK]\tSTI faulted\n" ); + } +} + +static void expect_gp_cli (void ) +{ +if (try_cli ()) { +printf ("[FAIL]\tCLI worked\n" ); + nerrs++; + }else { +printf ("[OK]\tCLI faulted\n" ); + } +} +int main (void ) {cpu_set_t cpuset; +CPU_ZERO (&cpuset);CPU_SET (0 , &cpuset);if (sched_setaffinity (0 ,sizeof (cpuset), &cpuset) !=0 )err (1 ,"sched_setaffinity to CPU 0" );/* Probe for iopl support. Note that iopl(0) works even as nonroot. */ -if (iopl (3 ) !=0 ) { +switch (iopl (3 )) { +case 0 : +break ; +case -ENOSYS : +printf ("[OK]\tiopl() nor supported\n" ); +return 0 ; +default :printf ("[OK]\tiopl(3) failed (%d) -- try running as root\n" , errno);return 0 ; } -/* Restore our original state prior to starting the test. */ +/* Make sure that CLI/STI are blocked even with IOPL level 3 */ +expect_gp_cli (); +expect_gp_sti (); +expect_ok_outb (0x80 ); + +/* Establish an I/O bitmap to test the restore */ +if (ioperm (0x80 ,1 ,1 ) !=0 ) +err (1 ,"ioperm(0x80, 1, 1) failed\n" ); + +/* Restore our original state prior to starting the fork test. */ if (iopl (0 ) !=0 )err (1 ,"iopl(0)" ); +/* +* Verify that IOPL emulation is disabled and the I/O bitmap still +* works. +*/ +expect_ok_outb (0x80 ); +expect_gp_outb (0xed ); +/* Drop the I/O bitmap */ +if (ioperm (0x80 ,1 ,0 ) !=0 ) +err (1 ,"ioperm(0x80, 1, 0) failed\n" ); +pid_t child =fork ();if (child ==-1 )err (1 ,"fork" );
@@ -90,14 +203,9 @@ int main(void)printf ("[RUN]\tparent: write to 0x80 (should fail)\n" ); -sethandler (SIGSEGV , sigsegv,0 ); -if (sigsetjmp (jmpbuf,1 ) !=0 ) { -printf ("[OK]\twrite was denied\n" ); - }else { - asmvolatile ("outb %%al, $0x80" : :"a" (0 )); -printf ("[FAIL]\twrite was allowed\n" ); - nerrs++; - } +expect_gp_outb (0x80 ); +expect_gp_cli (); +expect_gp_sti ();/* Test the capability checks. */ printf ("\tiopl(3)\n" );
@@ -133,4 +241,3 @@ int main(void) done:return nerrs ?1 :0 ; } -
[tip: x86/iopl] selftests/x86/iopl: Extend test to cover IOPL emulation
The following commit has been merged into the x86/iopl branch of tip:
The following commit has been merged into the x86/iopl branch of tip:
Commit-ID: 04f822767cb1387a277b96246e1f7f5f93fa35f0 Gitweb: https://git.kernel.org/tip/04f822767cb1387a277b96246e1f7f5f93fa35f0 Author: Thomas Gleixner <[email protected]> AuthorDate: Wed, 13 Nov 2019 21:43:00 +01:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Thu, 14 Nov 2019 20:15:08 +01:00
selftests/x86/iopl: Extend test to cover IOPL emulation
Add tests that the now emulated iopl() functionality:
- does not longer allow user space to disable interrupts.
- does restore a I/O bitmap when IOPL is dropped
--- tools/testing/selftests/x86/iopl.c | 129 +++++++++++++++++++++++++--- 1 file changed, 118 insertions(+), 11 deletions(-)
tools/testing/selftests/x86/iopl.c +118 -11
@@ -35,6 +35,16 @@ static void sethandler(int sig, void (*handler)(int, siginfo_t *, void *), } +static void clearhandler (int sig) +{ +struct sigaction sa; +memset (&sa,0 ,sizeof (sa)); + sa.sa_handler =SIG_DFL ; +sigemptyset (&sa.sa_mask); +if (sigaction (sig, &sa,0 )) +err (1 ,"sigaction" ); +} +static jmp_buf jmpbuf;static void sigsegv (int sig,siginfo_t *si,void *ctx_void)
@@ -42,25 +52,128 @@ static void sigsegv(int sig, siginfo_t *si, void *ctx_void)siglongjmp (jmpbuf,1 ); } +static bool try_outb (unsigned short port) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("outb %%al, %w[port]" + : : [port]"Nd" (port),"a" (0 )); +return true; + } +clearhandler (SIGSEGV ); +} + +static void expect_ok_outb (unsigned short port) +{ +if (!try_outb (port)) { +printf ("[FAIL]\toutb to 0x%02hx failed\n" , port); +exit (1 ); + } + +printf ("[OK]\toutb to 0x%02hx worked\n" , port); +} + +static void expect_gp_outb (unsigned short port) +{ +if (try_outb (port)) { +printf ("[FAIL]\toutb to 0x%02hx worked\n" , port); + nerrs++; + } + +printf ("[OK]\toutb to 0x%02hx failed\n" , port); +} + +static bool try_cli (void ) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("cli" ); +return true; + } +clearhandler (SIGSEGV ); +} + +static bool try_sti (void ) +{ +sethandler (SIGSEGV , sigsegv,SA_RESETHAND ); +if (sigsetjmp (jmpbuf,1 ) !=0 ) { +return false; + }else { + asmvolatile ("sti" ); +return true; + } +clearhandler (SIGSEGV ); +} + +static void expect_gp_sti (void ) +{ +if (try_sti ()) { +printf ("[FAIL]\tSTI worked\n" ); + nerrs++; + }else { +printf ("[OK]\tSTI faulted\n" ); + } +} + +static void expect_gp_cli (void ) +{ +if (try_cli ()) { +printf ("[FAIL]\tCLI worked\n" ); + nerrs++; + }else { +printf ("[OK]\tCLI faulted\n" ); + } +} +int main (void ) {cpu_set_t cpuset; +CPU_ZERO (&cpuset);CPU_SET (0 , &cpuset);if (sched_setaffinity (0 ,sizeof (cpuset), &cpuset) !=0 )err (1 ,"sched_setaffinity to CPU 0" );/* Probe for iopl support. Note that iopl(0) works even as nonroot. */ -if (iopl (3 ) !=0 ) { +switch (iopl (3 )) { +case 0 : +break ; +case -ENOSYS : +printf ("[OK]\tiopl() nor supported\n" ); +return 0 ; +default :printf ("[OK]\tiopl(3) failed (%d) -- try running as root\n" , errno);return 0 ; } -/* Restore our original state prior to starting the test. */ +/* Make sure that CLI/STI are blocked even with IOPL level 3 */ +expect_gp_cli (); +expect_gp_sti (); +expect_ok_outb (0x80 ); + +/* Establish an I/O bitmap to test the restore */ +if (ioperm (0x80 ,1 ,1 ) !=0 ) +err (1 ,"ioperm(0x80, 1, 1) failed\n" ); + +/* Restore our original state prior to starting the fork test. */ if (iopl (0 ) !=0 )err (1 ,"iopl(0)" ); +/* +* Verify that IOPL emulation is disabled and the I/O bitmap still +* works. +*/ +expect_ok_outb (0x80 ); +expect_gp_outb (0xed ); +/* Drop the I/O bitmap */ +if (ioperm (0x80 ,1 ,0 ) !=0 ) +err (1 ,"ioperm(0x80, 1, 0) failed\n" ); +pid_t child =fork ();if (child ==-1 )err (1 ,"fork" );
@@ -90,14 +203,9 @@ int main(void)printf ("[RUN]\tparent: write to 0x80 (should fail)\n" ); -sethandler (SIGSEGV , sigsegv,0 ); -if (sigsetjmp (jmpbuf,1 ) !=0 ) { -printf ("[OK]\twrite was denied\n" ); - }else { - asmvolatile ("outb %%al, $0x80" : :"a" (0 )); -printf ("[FAIL]\twrite was allowed\n" ); - nerrs++; - } +expect_gp_outb (0x80 ); +expect_gp_cli (); +expect_gp_sti ();/* Test the capability checks. */ printf ("\tiopl(3)\n" );
@@ -133,4 +241,3 @@ int main(void) done:return nerrs ?1 :0 ; } -
Quoted 32 lines · Thomas Gleixner
iopl(level = 3) enables aside of access to all 65536 I/O ports also the usage of CLI/STI in user space.
Disabling interrupts in user space can lead to system lockups and breaks assumptions in the kernel that userspace always runs with interrupts enabled.
iopl()is often preferred overioperm()as it avoids the overhead of copying the tasks I/O bitmap to the TSS bitmap on context switch. This overhead can be avoided by providing a all zeroes bitmap in the TSS and switching the TSS bitmap offset to this permit all IO bitmap. It's marginally slower thaniopl()which is a one time setup, but prevents the usage of CLI/STI in user space.--- arch/x86/Kconfig | 18 ++ arch/x86/entry/common.c | 4 arch/x86/include/asm/io_bitmap.h | 29 ++++ arch/x86/include/asm/paravirt.h | 4 arch/x86/include/asm/paravirt_types.h | 2 arch/x86/include/asm/pgtable_32_types.h | 2 arch/x86/include/asm/processor.h | 113 ++++++++++------- arch/x86/include/asm/ptrace.h | 6 arch/x86/include/asm/switch_to.h | 10 + arch/x86/include/asm/thread_info.h | 14 +- arch/x86/include/asm/xen/hypervisor.h | 2 arch/x86/kernel/cpu/common.c | 188 ++++++++++++---------------- arch/x86/kernel/doublefault.c | 2 arch/x86/kernel/ioport.c | 209 +++++++++++++++++++++----------- arch/x86/kernel/paravirt.c | 2 arch/x86/kernel/process.c | 200 ++++++++++++++++++++++++------ arch/x86/kernel/process_32.c | 77 -----------
Quoted 8 lines
arch/x86/kernel/process_64.c | 86 ------------- arch/x86/kernel/ptrace.c | 12 + arch/x86/kvm/vmx/vmx.c | 8 - arch/x86/mm/cpu_entry_area.c | 8 + arch/x86/xen/enlighten_pv.c | 10 - tools/testing/selftests/x86/ioperm.c | 16 ++ tools/testing/selftests/x86/iopl.c | 129 ++++++++++++++++++- 24 files changed, 674 insertions(+), 477 deletions(-)