Skip to content

x86/entry: Entry/exception code rework - the leftovers

Thomas Gleixner lkml 2 versions – Applied

Status from this message: The following commit has been merged into the core/rcu branch of tip:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest38 patches 14 repliesApplied
  2. v431 patches 17 repliesSuperseded

Patches in v5 Applied

#Subject and filesSizeRepliesApplied
1/38x86/kvm/svm: Use uninstrumented wrmsrl() to restore GSarch/x86/kvm/svm/svm.c+1 -11 reply
2/38x86/entry/64: Use native swapgs in asm_native_load_gs_index()arch/x86/entry/entry_64.S+3 -33 replies
3/38nmi, tracing: Provide nmi_enter/exit_notrace()include/linux/hardirq.h+14 -43 replies
4/38x86: Make hardware latency tracing explicitarch/x86/kernel/cpu/mce/core.c arch/x86/kernel/nmi.c arch/x86/kernel/traps.c+11 -92 replies
5/38genirq: Provide irq_enter/exit_rcu()include/linux/hardirq.h kernel/softirq.c+38 -100 replies
6/38x86/entry: Provide helpers for execute on irqstackarch/x86/entry/entry_64.S arch/x86/include/asm/irq_stack.h+79 -01 reply
7/38x86/entry/64: Move do_softirq_own_stack() to Carch/x86/entry/entry_64.S arch/x86/kernel/irq_64.c+9 -130 replies
8/38x86/entry: Split idtentry_enter/exit()arch/x86/entry/common.c+21 -160 replies
9/38x86/entry: Switch XEN/PV hypercall entry to IDTENTRYarch/x86/entry/common.c arch/x86/entry/entry_32.S arch/x86/entry/entry_64.S +8 more+103 -371 reply
10/38x86/entry/64: Simplify idtentry_bodyarch/x86/entry/entry_64.S+0 -20 replies
11/38rcu: Provide __rcu_is_watching()include/linux/rcutiny.h include/linux/rcutree.h kernel/rcu/tree.c+7 -00 replies
12/38x86/entry: Provide idtentry_entry/exit_cond_rcu()arch/x86/entry/common.c arch/x86/include/asm/idtentry.h+128 -170 replies
13/38x86/entry: Switch page fault exception to IDTENTRY_RAWarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/idtentry.h +6 more+62 -920 replies
14/38x86/entry: Remove the transition leftoversarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/idtentry.h+8 -270 replies
15/38x86/entry: Change exit path of xen_failsafe_callbackarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S+2 -20 replies
16/38x86/entry/64: Remove error_exitarch/x86/entry/entry_64.S+0 -90 replies
17/38x86/entry/32: Remove common_exceptionarch/x86/entry/entry_32.S+0 -210 replies
18/38x86/irq: Use generic irq_regs implementationarch/x86/include/asm/irq_regs.h arch/x86/kernel/irq.c+0 -350 replies
19/38x86/irq: Convey vector as argument and not in ptregsarch/x86/entry/calling.h arch/x86/entry/entry_32.S arch/x86/entry/entry_64.S +8 more+95 -780 replies
20/38x86/irq/64: Provide handle_irq()arch/x86/kernel/irq_64.c+8 -00 replies
21/38x86/entry: Add IRQENTRY_IRQ macroarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/idtentry.h+75 -00 replies
22/38x86/entry: Use idtentry for interruptsarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/hw_irq.h +6 more+28 -1010 replies
23/38genirq: Provde __irq_enter/exit_raw()include/linux/hardirq.h+20 -00 replies
24/38x86/entry: Provide IDTENTRY_SYSVECarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/idtentry.h+98 -02 replies
25/38x86/entry: Convert APIC interrupts to IDTENTRY_SYSVECarch/x86/entry/entry_64.S arch/x86/include/asm/entry_arch.h arch/x86/include/asm/hw_irq.h +6 more+20 -440 replies
26/38x86/entry: Convert SMP system vectors to IDTENTRY_SYSVECarch/x86/entry/entry_64.S arch/x86/include/asm/entry_arch.h arch/x86/include/asm/hw_irq.h +5 more+21 -370 replies
27/38x86/entry: Convert various system vectorsarch/x86/entry/entry_64.S arch/x86/include/asm/apic.h arch/x86/include/asm/entry_arch.h +11 more+48 -1020 replies
28/38x86/entry: Convert KVM vectors to IDTENTRY_SYSVECarch/x86/entry/entry_64.S arch/x86/include/asm/entry_arch.h arch/x86/include/asm/hw_irq.h +4 more+15 -420 replies
29/38x86/entry: Convert various hypervisor vectors to IDTENTRY_SYSVECarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/hyperv/hv_init.c +6 more+27 -980 replies
30/38x86/entry: Convert XEN hypercall vector to IDTENTRY_SYSVECarch/x86/entry/entry_32.S arch/x86/entry/entry_64.S arch/x86/include/asm/idtentry.h +3 more+18 -210 replies
31/38x86/entry: Convert reschedule interrupt to IDTENTRY_SYSVEC_SIMPLEarch/x86/entry/entry_64.S arch/x86/include/asm/entry_arch.h arch/x86/include/asm/hw_irq.h +6 more+7 -630 replies
32/38x86/entry: Remove the apic/BUILD interrupt leftoversarch/x86/entry/calling.h arch/x86/entry/entry_32.S arch/x86/entry/entry_64.S +2 more+4 -1960 replies
33/38x86/entry/64: Remove IRQ stack switching ASMarch/x86/entry/entry_64.S+0 -960 replies
34/38x86/entry: Make enter_from_user_mode() staticarch/x86/entry/common.c+1 -10 replies
35/38x86/entry/32: Remove redundant irq disable codearch/x86/entry/entry_32.S+0 -760 replies
36/38x86/entry/64: Remove TRACE_IRQS_*_DEBUGarch/x86/entry/entry_64.S+3 -450 replies
37/38x86/entry: Move paranoid irq tracing out of ASM codearch/x86/entry/entry_64.S arch/x86/kernel/nmi.c+3 -130 replies
38/38x86/entry: Remove the TRACE_IRQS cruftarch/x86/entry/entry_64.S arch/x86/entry/thunk_64.S arch/x86/include/asm/irqflags.h+1 -310 replies

Reviews trailers per patch

6 of 38 patches have a Reviewed-by or Acked-by.

PatchAndy LutomirskiSteven Rostedt (VMware)Juergen GrossJosh PoimboeufBoris Ostrovsky
1/38A-b
2/38R-b A-b
3/38A-b
4/38
5/38
6/38A-b
7/38
8/38
9/38R-b
10/38
11/38
12/38
13/38
14/38
15/38
16/38
17/38
18/38
19/38Originally-by
20/38
21/38
22/38
23/38
24/38
25/38
26/38
27/38
28/38
29/38
30/38
31/38
32/38
33/38
34/38
35/38
36/38R-b
37/38
38/38

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.

Read the v5 thread

Cover letter [patch V5 00/38] x86/entry: Entry/exception code rework - the leftovers

Folks!

This is V5 of the rework series. V4 can be found here:

https://lore.kernel.org/r/[email protected]

Some of the non-x86 specific parts from part 1- 4 have been applied to the relevant tip branches and I merged them all together to base the rest on.

I have applied most of the patches from part 1-4 on top after fixing up the small review comments all over the place. The result can be found here:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git entry-base-v5

I dropped:

[patch V4 part 1 33/36] x86,tracing: Robustify ftrace_nmi_enter() [patch V4 part 2 10/18] x86/entry/64: Check IF in __preempt_enable_notrace() [patch V4 part 2 11/18] x86/entry/64: Mark ___preempt_schedule_notrace()

from V4 because Peter's attempt to sanitize ftrace_nmi_enter/exit() and the underlying callchains turned out to be incomplete and cause the whole __preempt_schedule_notrace() hackery which is caused by a gazillion of preempt_disable_notrace() / preempt_enable_notrace() invocations in the full call chain. I found a better way to do that and that's in part 5 now.

The remaining patches are mostly from part 5 of version 4 dealing with the #PF, device interrupts and the system vectors.

The main differences to V4 part 5 are:

- Provide nmi_enter/exit_notrace() which do not contain the ftrace_nmi_enter/exit() calls, use them in x86 and make the ftrace invocation explicit at a safe place.

- A fix for the reworked SVM code

- An optimization of asm_native_load_gs_index() which I found while staring at stuff in more detail

- The removal of the x86 specific irq_regs implementation which was aside of two underscores in the variable name exactly the same thing as the generic variant.

- Overhaul of the IRQ stack switching mechanics to avoid the objtool and unwinder nastiness of doing it in C inline ASM. As discussed here:

https://lore.kernel.org/r/[email protected]

We agreed on having the actual stack switch still in ASM and utilize it for device interrupts, soft interrupts and system vectors.

This comes for the price of an indirect call, but the device interrupt handling has an indirect call and soft interrupts and most system vectors are indirect call laden already.

There is a lightweight variant for the system vectors which do basically nothing, i.e. reschedule IPI and the KVM posted interrupt vectors. That variant runs on the interrupted stack and avoids the full entry/exit handling overhead by utilizing the conditional RCU variant of idtentry_enter/exit().

- Addressing a few review comments.

The series is also available from git:

git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git entry-v5-the-rest

Running the new objtool checks results in two warning types:

- The MCE code has an issue in the non-instrumentable section which is still under investigation and discussion

- When paravirt is enabled then there are a few complaints about invoking indirect calls (PVOPS) which leave the noinstr sections but they are halfways straight forward to fix.

Thanks,

tglx

8<--------------

 arch/x86/include/asm/acrn.h                |   11 
 arch/x86/include/asm/entry_arch.h          |   56 --
 arch/x86/include/asm/irq_regs.h            |   32 -
 b/arch/x86/entry/calling.h                 |   25 -
 b/arch/x86/entry/common.c                  |  216 +++++++++-
 b/arch/x86/entry/entry_32.S                |  260 +-----------
 b/arch/x86/entry/entry_64.S                |  581 +++--------------------------
 b/arch/x86/entry/thunk_64.S                |    9 
 b/arch/x86/hyperv/hv_init.c                |    9 
 b/arch/x86/include/asm/apic.h              |   33 -
 b/arch/x86/include/asm/hw_irq.h            |   22 -
 b/arch/x86/include/asm/idtentry.h          |  262 ++++++++++++-
 b/arch/x86/include/asm/irq.h               |    6 
 b/arch/x86/include/asm/irq_stack.h         |   40 +
 b/arch/x86/include/asm/irq_work.h          |    1 
 b/arch/x86/include/asm/irqflags.h          |   10 
 b/arch/x86/include/asm/mshyperv.h          |   13 
 b/arch/x86/include/asm/trace/common.h      |    4 
 b/arch/x86/include/asm/trace/irq_vectors.h |   17 
 b/arch/x86/include/asm/traps.h             |   21 -
 b/arch/x86/include/asm/uv/uv_bau.h         |    8 
 b/arch/x86/kernel/apic/apic.c              |   39 -
 b/arch/x86/kernel/apic/msi.c               |    3 
 b/arch/x86/kernel/apic/vector.c            |    5 
 b/arch/x86/kernel/cpu/acrn.c               |    9 
 b/arch/x86/kernel/cpu/mce/amd.c            |    5 
 b/arch/x86/kernel/cpu/mce/core.c           |    4 
 b/arch/x86/kernel/cpu/mce/therm_throt.c    |    5 
 b/arch/x86/kernel/cpu/mce/threshold.c      |    5 
 b/arch/x86/kernel/cpu/mshyperv.c           |   22 -
 b/arch/x86/kernel/head_64.S                |    7 
 b/arch/x86/kernel/idt.c                    |   42 +-
 b/arch/x86/kernel/irq.c                    |   58 --
 b/arch/x86/kernel/irq_64.c                 |   17 
 b/arch/x86/kernel/irq_work.c               |    6 
 b/arch/x86/kernel/kvm.c                    |   14 
 b/arch/x86/kernel/nmi.c                    |    9 
 b/arch/x86/kernel/smp.c                    |   37 -
 b/arch/x86/kernel/tracepoint.c             |   17 
 b/arch/x86/kernel/traps.c                  |   10 
 b/arch/x86/kvm/svm/svm.c                   |    2 
 b/arch/x86/mm/fault.c                      |   69 ++-
 b/arch/x86/platform/uv/tlb_uv.c            |    2 
 b/arch/x86/xen/enlighten_hvm.c             |   12 
 b/arch/x86/xen/enlighten_pv.c              |    2 
 b/arch/x86/xen/setup.c                     |    4 
 b/arch/x86/xen/smp_pv.c                    |    3 
 b/arch/x86/xen/xen-asm_32.S                |   12 
 b/arch/x86/xen/xen-asm_64.S                |    4 
 b/arch/x86/xen/xen-ops.h                   |    1 
 b/drivers/xen/events/events_base.c         |    6 
 b/drivers/xen/preempt.c                    |    2 
 b/include/linux/hardirq.h                  |   51 ++
 b/include/linux/rcutiny.h                  |    1 
 b/include/linux/rcutree.h                  |    1 
 b/include/xen/events.h                     |    7 
 b/include/xen/xen-ops.h                    |    7 
 b/kernel/rcu/tree.c                        |    5 
 b/kernel/softirq.c                         |   35 +
 59 files changed, 906 insertions(+), 1270 deletions(-)