trace,x86: irq vector tracepoint support
Subsystems (5)
PATCH v14 1/3 tracing: Add DEFINE_EVENT_FN() macro
In reply to Seiji AguchiFrom: Steven Rostedt <[email protected]>
Each TRACE_EVENT() adds several helper functions. If two or more trace events share the same structure and print format, they can also share most of these helper functions and save a lot of space from duplicate code. This is why the DECLARE_EVENT_CLASS() and DEFINE_EVENT() were created.
Some events require a trigger to be called at registering and unregistering of the event and to do so they use TRACE_EVENT_FN().
If multiple events require a trigger, they currently have no choice but to use TRACE_EVENT_FN() as there's no DEFINE_EVENT_FN() available. This unfortunately causes a lot of wasted duplicate code created.
By adding a DEFINE_EVENT_FN(), these events can still use a DECLARE_EVENT_CLASS() and then define their own triggers.
--- include/linux/tracepoint.h | 2 ++ include/trace/define_trace.h | 5 +++++ include/trace/ftrace.h | 4 ++++ 3 files changed, 11 insertions(+), 0 deletions(-)
include/linux/tracepoint.h +2 -0
@@ -378,6 +378,8 @@ static inline void tracepoint_synchronize_unregister(void)#define DECLARE_EVENT_CLASS (name, proto, args, tstruct, assign, print)#define DEFINE_EVENT (template, name, proto, args) \ DECLARE_TRACE(name, PARAMS(proto), PARAMS(args)) +#define DEFINE_EVENT_FN (template, name, proto, args, reg, unreg)\ + DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))#define DEFINE_EVENT_PRINT (template, name, proto, args, print) \ DECLARE_TRACE(name, PARAMS(proto), PARAMS(args)) #define DEFINE_EVENT_CONDITION(template, name, proto, \
include/trace/define_trace.h +5 -0
@@ -44,6 +44,10 @@#define DEFINE_EVENT (template, name, proto, args) \ DEFINE_TRACE(name) +#undef DEFINE_EVENT_FN +#define DEFINE_EVENT_FN (template, name, proto, args, reg, unreg) \ + DEFINE_TRACE_FN(name, reg, unreg) +#undef DEFINE_EVENT_PRINT#define DEFINE_EVENT_PRINT (template, name, proto, args, print) \ DEFINE_TRACE(name)
@@ -91,6 +95,7 @@#undef TRACE_EVENT_CONDITION#undef DECLARE_EVENT_CLASS#undef DEFINE_EVENT +#undef DEFINE_EVENT_FN#undef DEFINE_EVENT_PRINT#undef DEFINE_EVENT_CONDITION#undef TRACE_HEADER_MULTI_READ
include/trace/ftrace.h +4 -0
@@ -71,6 +71,10 @@ static struct ftrace_event_call __used \ __attribute__((__aligned__(4))) event_##name +#undef DEFINE_EVENT_FN +#define DEFINE_EVENT_FN (template, name, proto, args, reg, unreg) \ + DEFINE_EVENT(template, name, PARAMS(proto), PARAMS(args)) +#undef DEFINE_EVENT_PRINT#define DEFINE_EVENT_PRINT (template, name, proto, args, print) \ DEFINE_EVENT(template, name, PARAMS(proto), PARAMS(args))
Signature · 1 line
1.7.1
v14 cover trace,x86: irq vector tracepoint support
Change log
v13 -> v14
- Rebase to 3.10-rc4
- Patch 3/3
- Add comments above load_current_idt() in arch/x86/include/asm/desc.h.
- Introduce a logic selecting IDT that we currently should be using, based on the current state instead of current_idt_desc_ptr.
- Rename some "nmi" variables to "debug".
- Introduce local_irq_save/restore() in load_current_idt() to avoid races regardless of process/interrupt context.
v12 -> v13
- Rebase to 3.10-rc3
- Patch 2/3
- Separate a patch introducing entering/exiting_irq() from patch-v12 3/3.
- Patch 3/3
- Introduce write_trace_idt_entry() to remove "ifdef CONFIG_TRACING"
from _set_gate().
- Change DEFINE_IRQ_VECTOR_EVENT to save on repeat code.
- Add a path of irq_vector.h to Makefile.
- Introduce current_idt_descr_ptr and load_current_idt() to switch IDT in a generic way with handling Debug traps/NMI and cpu hotplug.
v11 -> v12
- Rebase to 3.9-rc5
v10 -> v11
- Rebase to 3.9-rc2
- Add a modification for hyperv_callback vector. (patch 2/3)
- Change a way to switch idt to check the table in use instead of saving/restoring it, because saving/restoring functions will break if we have to add another one. (patch 2/3)
v9 -> v10
- Add an explanation the reason why tracepoints has to place inside irq enter/exit handling. (patch 3/3)
v8 -> v9
- Rebase to 3.8-rc6
- Add Steven's email address at the top of the message and
move my signed-off-by below Steven's one because it is
originally created by Steven. (patch 1/3)
- Introduce a irq_vector_mutex to avoid a race at registering/unregistering time. (patch 2/3)
- Use a per_cpu data to orig_idt_descr because IDT descritor is needed to each cpu and the appropriate data type is per_cpu data. It is suggested by Steven. (patch 2/3)
v7 -> v8
- Rebase to 3.8-rc4
- Add a patch 1 introducing DEFINE_EVENT_FN() macro.
- Rename original patches 1 and 2 to 2 and 3.
- Change a definition of tracepoint to use DEFINE_EVENT_FN(). (patch 2)
- Change alloc_intr_gate() to use do{}while(0) to avoid a warning of checkpatch.pl. (patch 2)
- Move entering_irq()/exiting_irq() to arch/x86/include/asm/apic.h (patch 3)
v6 -> v7
- Divide into two patches to make a code review easier. Summery of each patch is as follows.
- Patch 1/2
- Add an irq_vector tracing infrastructure.
- Create idt_table for tracing. It is refactored to avoid duplicating existing logic.
- Duplicate new irq handlers inserted tracepoints.
- Patch 2/2
- Share a common logic among irq handlers to make them manageable and readable.
v5 -> v6
- Rebased to 3.7
v4 -> v5
- Rebased to 3.6.0
- Introduce a logic switching IDT at enabling/disabling TP time so that a time penalty makes a zero when tracepoints are disabled. This IDT is created only when CONFIG_TRACEPOINTS is enabled.
- Remove arch_irq_vector_entry/exit and add followings again so that we can add each tracepoint in a generic way.
- error_apic_vector
- thermal_apic_vector
- threshold_apic_vector
- spurious_apic_vector
- x86_platform_ipi_vector
- Drop nmi tracepoints to begin with apic interrupts and discuss a logic switching IDT first.
- Move irq_vectors.h in the directory of arch/x86/include/asm/trace because I'm not sure if a logic switching IDT is sharable with other architectures.
v3 -> v4
- Add a latency measurement of each tracepoint
- Rebased to 3.6-rc6
v2 -> v3
- Remove an invalidate_tlb_vector event because it was replaced by a call function vector in a following commit.
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commit;h=52aec3308db85f4e9f5c8b9f5dc4fbd0138c6fa4
v1 -> v2
- Modify variable name from irq to vector.
- Merge arch-specific tracepoints below to an arch_irq_vector_entry/exit.
- error_apic_vector
- thermal_apic_vector
- threshold_apic_vector
- spurious_apic_vector
- x86_platform_ipi_vector
[Purpose of this patch]
As Vaibhav explained in the thread below, tracepoints for irq vectors are useful.
http://www.spinics.net/lists/mm-commits/msg85707.html
<snip>
The current interrupt traces from irq_handler_entry and irq_handler_exit provide when an interrupt is handled.
They provide good data about when the system has switched to kernel space and how it affects the currently running processes.
There are some IRQ vectors which trigger the system into kernel space, which are not handled in generic IRQ handlers.
Tracing such events gives us the information about IRQ interaction with other system events.
The trace also tells where the system is spending its time. We want to know which cores are handling interrupts and
how they are affecting other processes in the system. Also, the trace provides information about when the cores are
idle and which interrupts are changing that state.
<snip>
On the other hand, my usecase is tracing just local timer event and getting
a value of instruction pointer.
I suggested to add an argument local timer event to get instruction pointer before. But there is another way to get it with external module like systemtap. So, I don't need to add any argument to irq vector tracepoints now.
[Patch Description]
Vaibhav's patch shared a trace point ,irq_vector_entry/irq_vector_exit, in all events. But there is an above use case to trace specific irq_vector rather than tracing all events. In this case, we are concerned about overhead due to unwanted events.
This patch adds following tracepoints instead of introducing irq_vector_entry/exit. so that we can enable them independently.
- local_timer_vector
- reschedule_vector
- call_function_vector
- call_function_single_vector
- irq_work_entry_vector
- error_apic_vector
- thermal_apic_vector
- threshold_apic_vector
- spurious_apic_vector
- x86_platform_ipi_vector
Please see descriptions in each patch.
Seiji Aguchi (2):
trace,x86: Introduce entering/exiting_irq()
trace,x86: Add irq vector tracepoints
Steven Rostedt (1):
tracing: Add DEFINE_EVENT_FN() macro
arch/x86/include/asm/apic.h | 27 +++++++ arch/x86/include/asm/desc.h | 119 ++++++++++++++++++++++++++++- arch/x86/include/asm/entry_arch.h | 8 +- arch/x86/include/asm/hw_irq.h | 17 ++++ arch/x86/include/asm/mshyperv.h | 1 + arch/x86/include/asm/trace/irq_vectors.h | 104 ++++++++++++++++++++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/apic/Makefile | 1 + arch/x86/kernel/apic/apic.c | 71 +++++++++++++++--- arch/x86/kernel/cpu/common.c | 18 ++-- arch/x86/kernel/cpu/mcheck/therm_throt.c | 24 +++++-- arch/x86/kernel/cpu/mcheck/threshold.c | 24 +++++-- arch/x86/kernel/entry_32.S | 12 +++- arch/x86/kernel/entry_64.S | 31 ++++++-- arch/x86/kernel/head_64.S | 8 ++- arch/x86/kernel/irq.c | 31 +++++--- arch/x86/kernel/irq_work.c | 24 ++++++- arch/x86/kernel/smp.c | 65 ++++++++++++++-- arch/x86/kernel/tracepoint.c | 57 ++++++++++++++ arch/x86/kernel/traps.c | 2 +- include/linux/tracepoint.h | 2 + include/trace/define_trace.h | 5 + include/trace/ftrace.h | 4 + include/xen/events.h | 3 + 24 files changed, 588 insertions(+), 71 deletions(-) create mode 100644 arch/x86/include/asm/trace/irq_vectors.h create mode 100644 arch/x86/kernel/tracepoint.c
PATCH v14 2/3 trace,x86: Introduce entering/exiting_irq()
When implementing tracepoints in interrupt handers, if the tracepoints are simply added in the performance sensitive path of interrupt handers, it may cause potential performance problem due to the time penalty.
To solve the problem, an idea is to prepare non-trace/trace irq handers and switch their IDTs at the enabling/disabling time.
To do this, this patch introduces entering_irq()/exiting_irq() for pre/post- processing of each irq handler.
A way to use them is as follows.
Non-trace irq handler:smp_irq_handler()
{
entering_irq(); /* pre-processing of this handler */
__smp_irq_handler(); /*
* common logic between non-trace and trace handlers
* in a vector.
*/
exiting_irq(); /* post-processing of this handler */}
Trace irq_handler:smp_trace_irq_handler()
{
entering_irq(); /* pre-processing of this handler */
trace_irq_entry(); /* tracepoint for irq entry */
__smp_irq_handler(); /*
* common logic between non-trace and trace handlers
* in a vector.
*/
trace_irq_exit(); /* tracepoint for irq exit */
exiting_irq(); /* post-processing of this handler */}
If tracepoints can place outside entering_irq()/exiting_irq() as follows, it looks cleaner.
smp_trace_irq_handler()
{
trace_irq_entry();
smp_irq_handler();
trace_irq_exit();
}But it doesn't work.
The problem is with irq_enter/exit() being called. They must be called before trace_irq_enter/exit(), because of the rcu_irq_enter() must be called before any tracepoints are used, as tracepoints use rcu to synchronize.
As a possible alternative, we may be able to call irq_enter() first as follows if irq_enter() can nest.
smp_trace_irq_hander()
{
irq_entry();
trace_irq_entry();
smp_irq_handler();
trace_irq_exit();
irq_exit();
}But it doesn't work, either.
If irq_enter() is nested, it may have a time penalty because it has to check if it was already called or not. The time penalty is not desired in performance sensitive paths even if it is tiny.
--- arch/x86/include/asm/apic.h | 27 +++++++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-)
arch/x86/include/asm/apic.h +27 -0
@@ -12,6 +12,7 @@#include <asm/fixmap.h> #include <asm/mpspec.h> #include <asm/msr.h> +#include <asm/idle.h> #define ARCH_APICTIMER_STOPS_ON_C3 1
@@ -687,5 +688,31 @@ extern int default_check_phys_apicid_present(int phys_apicid);#endif #endif /* CONFIG_X86_LOCAL_APIC */ +extern void irq_enter (void ); +extern void irq_exit (void ); + +static inline void entering_irq (void ) +{ +irq_enter (); +exit_idle (); +} + +static inline void entering_ack_irq (void ) +{ +ack_APIC_irq (); +entering_irq (); +} + +static inline void exiting_irq (void ) +{ +irq_exit (); +} + +static inline void exiting_ack_irq (void ) +{ +irq_exit (); +/* Ack only at the end to avoid potential reentry */ +ack_APIC_irq (); +}#endif /* _ASM_X86_APIC_H */
Signature · 1 line
1.7.1
PATCH v14 3/3 trace,x86: Add irq vector tracepoints
In reply to Seiji Aguchi[Purpose of this patch]
As Vaibhav explained in the thread below, tracepoints for irq vectors are useful.
http://www.spinics.net/lists/mm-commits/msg85707.html
<snip>
The current interrupt traces from irq_handler_entry and irq_handler_exit provide when an interrupt is handled. They provide good data about when the system has switched to kernel space and how it affects the currently running processes.
There are some IRQ vectors which trigger the system into kernel space, which are not handled in generic IRQ handlers. Tracing such events gives us the information about IRQ interaction with other system events.
The trace also tells where the system is spending its time. We want to know which cores are handling interrupts and how they are affecting other processes in the system. Also, the trace provides information about when the cores are idle and which interrupts are changing that state. <snip>
On the other hand, my usecase is tracing just local timer event and getting a value of instruction pointer.
I suggested to add an argument local timer event to get instruction pointer before. But there is another way to get it with external module like systemtap. So, I don't need to add any argument to irq vector tracepoints now.
[Patch Description]
Vaibhav's patch shared a trace point ,irq_vector_entry/irq_vector_exit, in all events. But there is an above use case to trace specific irq_vector rather than tracing all events. In this case, we are concerned about overhead due to unwanted events.
This patch adds following tracepoints instead of introducing irq_vector_entry/exit. so that we can enable them independently.
- local_timer_vector
- reschedule_vector
- call_function_vector
- call_function_single_vector
- irq_work_entry_vector
- error_apic_vector
- thermal_apic_vector
- threshold_apic_vector
- spurious_apic_vector
- x86_platform_ipi_vector
It introduces a logic switching IDT at enabling/disabling time so that a time penalty makes a zero when tracepoints are disabled. Detailed explanations are as follows.
- Create non-trace and trace irq handlers with entering_irq()/exiting_irq().
- Create a new IDT, trace_idt_table, at boot time by adding a logic to _set_gate(). It is just a copy of original idt table.
- Register the new handlers for tracpoints to the new IDT by introducing macros to alloc_intr_gate() called at registering time of irq_vector handlers.
- When loading IDT in irq-vector trace on/off, NMI or debug on/off and cpu hotplug events, load_current_idt(), which selects the IDT that we currently should be using, based on the current state, is called.
Also, rename nmi_idt_descr, nmi_idt_table and debug_stack_use_ctr to debug_idt_descr, debug_idt_table, and debug_idt_ctr to describe meaning of the variables precisely.
In addtion, the new IDT is created only when CONFIG_TRACING is enabled to avoid being used for other purposes.
--- arch/x86/include/asm/desc.h | 119 ++++++++++++++++++++++++++++- arch/x86/include/asm/entry_arch.h | 8 +- arch/x86/include/asm/hw_irq.h | 17 ++++ arch/x86/include/asm/mshyperv.h | 1 + arch/x86/include/asm/trace/irq_vectors.h | 104 ++++++++++++++++++++++++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/apic/Makefile | 1 + arch/x86/kernel/apic/apic.c | 71 +++++++++++++++--- arch/x86/kernel/cpu/common.c | 18 ++-- arch/x86/kernel/cpu/mcheck/therm_throt.c | 24 +++++-- arch/x86/kernel/cpu/mcheck/threshold.c | 24 +++++-- arch/x86/kernel/entry_32.S | 12 +++- arch/x86/kernel/entry_64.S | 31 ++++++-- arch/x86/kernel/head_64.S | 8 ++- arch/x86/kernel/irq.c | 31 +++++--- arch/x86/kernel/irq_work.c | 24 ++++++- arch/x86/kernel/smp.c | 65 ++++++++++++++-- arch/x86/kernel/tracepoint.c | 57 ++++++++++++++ arch/x86/kernel/traps.c | 2 +- include/xen/events.h | 3 + 20 files changed, 550 insertions(+), 71 deletions(-) create mode 100644 arch/x86/include/asm/trace/irq_vectors.h create mode 100644 arch/x86/kernel/tracepoint.c
arch/x86/include/asm/desc.h +114 -5
@@ -36,8 +36,8 @@ static inline void fill_ldt(struct desc_struct *desc, const struct user_desc *inextern struct desc_ptr idt_descr;extern gate_desc idt_table[]; -extern struct desc_ptr nmi_idt_descr; -extern gate_desc nmi_idt_table[]; +extern struct desc_ptr debug_idt_descr; +extern gate_desc debug_idt_table[];struct gdt_page {struct desc_struct gdt[GDT_ENTRIES ];
@@ -316,7 +316,20 @@ static inline void set_nmi_gate(int gate, void *addr)gate_desc s;pack_gate (&s,GATE_INTERRUPT , (unsigned long )addr,0 ,0 , __KERNEL_CS); - write_idt_entry(nmi_idt_table, gate, &s); + write_idt_entry(debug_idt_table, gate, &s); +} +#endif + +#ifdef CONFIG_TRACING +extern struct desc_ptr trace_idt_descr; +extern gate_desc trace_idt_table[]; +static inline void write_trace_idt_entry (int entry,const gate_desc *gate) +{ +write_idt_entry (trace_idt_table, entry, gate); +} +#else +static inline void write_trace_idt_entry (int entry,const gate_desc *gate) +{ }#endif
@@ -331,6 +344,7 @@ static inline void _set_gate(int gate, unsigned type, void *addr,* setup time */ write_idt_entry (idt_table, gate, &s); +write_trace_idt_entry (gate, &s); }/*
@@ -360,12 +374,39 @@ static inline void alloc_system_vector(int vector) } } -static inline void alloc_intr_gate (unsigned int n,void *addr) +#ifdef CONFIG_TRACING +static inline void trace_set_intr_gate (unsigned int gate,void *addr) +{ +gate_desc s; + +pack_gate (&s,GATE_INTERRUPT , (unsigned long )addr,0 ,0 , __KERNEL_CS); +write_idt_entry (trace_idt_table, gate, &s); +} + +static inline void __trace_alloc_intr_gate (unsigned int n,void *addr) +{ +trace_set_intr_gate (n, addr); +} +#else +static inline void trace_set_intr_gate (unsigned int gate,void *addr) +{ +} + +#define __trace_alloc_intr_gate (n, addr) +#endif + +static inline void __alloc_intr_gate (unsigned int n,void *addr) { -alloc_system_vector (n);set_intr_gate (n, addr); } +#define alloc_intr_gate (n, addr) \ + do { \ + alloc_system_vector(n); \ + __alloc_intr_gate(n, addr); \ + __trace_alloc_intr_gate(n, trace_##addr); \ + } while (0) +/* * This routine sets up an interrupt gate at directory privilege level 3. */
@@ -405,4 +446,72 @@ static inline void set_system_intr_gate_ist(int n, void *addr, unsigned ist)_set_gate (n,GATE_INTERRUPT , addr,0x3 , ist, __KERNEL_CS); } +#ifdef CONFIG_X86_64 +DECLARE_PER_CPU (u32, debug_idt_ctr); +static inline bool is_debug_idt_enabled (void ) +{ +if (this_cpu_read (debug_idt_ctr)) +return true; + +return false; +} + +static inline void load_debug_idt (void ) +{ +load_idt ((const struct desc_ptr *)&debug_idt_descr); +} +#else +static inline bool is_debug_idt_enabled (void ) +{ +return false; +} + +static inline void load_debug_idt (void ) +{ +} +#endif + +#ifdef CONFIG_TRACING +extern atomic_t trace_idt_ctr; +static inline bool is_trace_idt_enabled (void ) +{ +if (atomic_read (&trace_idt_ctr)) +return true; + +return false; +} + +static inline void load_trace_idt (void ) +{ +load_idt ((const struct desc_ptr *)&trace_idt_descr); +} +#else +static inline bool is_trace_idt_enabled (void ) +{ +return false; +} + +static inline void load_trace_idt (void ) +{ +} +#endif + +/* +* the load_current_idt() is called with interrupt disabled by local_irq_save() +* to avoid races. That way the IDT will always be set back to the expected +* descriptor. +*/ +static inline void load_current_idt (void ) +{ +unsigned long flags; + +local_irq_save (flags); +if (is_debug_idt_enabled ()) +load_debug_idt (); +else if (is_trace_idt_enabled ()) +load_trace_idt (); +else +load_idt ((const struct desc_ptr *)&idt_descr); +local_irq_restore (flags); +}#endif /* _ASM_X86_DESC_H */
arch/x86/include/asm/entry_arch.h +5 -3
@@ -13,14 +13,16 @@ BUILD_INTERRUPT(reschedule_interrupt,RESCHEDULE_VECTOR) BUILD_INTERRUPT(call_function_interrupt,CALL_FUNCTION_VECTOR) BUILD_INTERRUPT(call_function_single_interrupt,CALL_FUNCTION_SINGLE_VECTOR) -BUILD_INTERRUPT(irq_move_cleanup_interrupt,IRQ_MOVE_CLEANUP_VECTOR) -BUILD_INTERRUPT(reboot_interrupt,REBOOT_VECTOR) +BUILD_INTERRUPT3 (irq_move_cleanup_interrupt,IRQ_MOVE_CLEANUP_VECTOR , + smp_irq_move_cleanup_interrupt) +BUILD_INTERRUPT3(reboot_interrupt, REBOOT_VECTOR, smp_reboot_interrupt)#endif BUILD_INTERRUPT(x86_platform_ipi, X86_PLATFORM_IPI_VECTOR)#ifdef CONFIG_HAVE_KVM -BUILD_INTERRUPT(kvm_posted_intr_ipi, POSTED_INTR_VECTOR) +BUILD_INTERRUPT3 (kvm_posted_intr_ipi,POSTED_INTR_VECTOR , + smp_kvm_posted_intr_ipi)#endif /*
arch/x86/include/asm/hw_irq.h +17 -0
@@ -77,6 +77,23 @@ extern void threshold_interrupt(void);extern void call_function_interrupt (void );extern void call_function_single_interrupt (void ); +#ifdef CONFIG_TRACING +/* Interrupt handlers registered during init_IRQ */ +extern void trace_apic_timer_interrupt (void ); +extern void trace_x86_platform_ipi (void ); +extern void trace_error_interrupt (void ); +extern void trace_irq_work_interrupt (void ); +extern void trace_spurious_interrupt (void ); +extern void trace_thermal_interrupt (void ); +extern void trace_reschedule_interrupt (void ); +extern void trace_threshold_interrupt (void ); +extern void trace_call_function_interrupt (void ); +extern void trace_call_function_single_interrupt (void ); +#define trace_irq_move_cleanup_interrupt irq_move_cleanup_interrupt +#define trace_reboot_interrupt reboot_interrupt +#define trace_kvm_posted_intr_ipi kvm_posted_intr_ipi +#endif /* CONFIG_TRACING */ +/* IOAPIC */ #define IO_APIC_IRQ (x) (((x) >= NR_IRQS_LEGACY) || ((1<<(x)) & io_apic_irqs))extern unsigned long io_apic_irqs;
arch/x86/include/asm/mshyperv.h +1 -0
@@ -12,6 +12,7 @@ struct ms_hyperv_info {extern struct ms_hyperv_info ms_hyperv;void hyperv_callback_vector (void ); +#define trace_hyperv_callback_vector hyperv_callback_vectorvoid hyperv_vector_handler (struct pt_regs *regs);void hv_register_vmbus_handler (int irq,irq_handler_t handler);
arch/x86/include/asm/trace/irq_vectors.h new file +104 -0
104 lines not on this page · raw message
arch/x86/kernel/Makefile +1 -0
@@ -102,6 +102,7 @@ obj-$(CONFIG_OF) += devicetree.o obj-$(CONFIG_UPROBES) += uprobes.o obj-$(CONFIG_PERF_EVENTS) += perf_regs.o +obj-$(CONFIG_TRACING) += tracepoint.o ### # 64 bit specific files
arch/x86/kernel/apic/Makefile +1 -0
@@ -2,6 +2,7 @@ # Makefile for local APIC drivers and for the IO-APIC code # +CFLAGS_apic.o := -I$(src)/../../include/asm/trace obj-$(CONFIG_X86_LOCAL_APIC) += apic.o apic_noop.o ipi.o obj-y += hw_nmi.o
arch/x86/kernel/apic/apic.c +59 -12
122 lines not on this page · raw message
arch/x86/kernel/cpu/common.c +9 -9
@@ -1071,8 +1071,8 @@ __setup("clearcpuid=", setup_disablecpuid);#ifdef CONFIG_X86_64 struct desc_ptr idt_descr = {NR_VECTORS *16 -1 , (unsigned long ) idt_table }; -struct desc_ptr nmi_idt_descr = {NR_VECTORS *16 -1 , - (unsigned long ) nmi_idt_table }; +struct desc_ptr debug_idt_descr = {NR_VECTORS *16 -1 , + (unsigned long ) debug_idt_table }; DEFINE_PER_CPU_FIRST(union irq_stack_union, irq_stack_union)__aligned(PAGE_SIZE) ;
@@ -1148,20 +1148,20 @@ int is_debug_stack(unsigned long addr) addr > (__get_cpu_var(debug_stack_addr) - DEBUG_STKSZ)); } -static DEFINE_PER_CPU(u32, debug_stack_use_ctr); +DEFINE_PER_CPU (u32, debug_idt_ctr);void debug_stack_set_zero (void ) { -this_cpu_inc (debug_stack_use_ctr); -load_idt ((const struct desc_ptr *)&nmi_idt_descr); +this_cpu_inc (debug_idt_ctr); +load_current_idt (); }void debug_stack_reset (void ) { -if (WARN_ON (!this_cpu_read (debug_stack_use_ctr))) +if (WARN_ON (!this_cpu_read (debug_idt_ctr)))return ; -if (this_cpu_dec_return (debug_stack_use_ctr) ==0 ) -load_idt ((const struct desc_ptr *)&idt_descr); +if (this_cpu_dec_return (debug_idt_ctr) ==0 ) +load_current_idt (); }#else /* CONFIG_X86_64 */
@@ -1257,7 +1257,7 @@ void __cpuinit cpu_init(void)switch_to_new_gdt (cpu);loadsegment (fs,0 ); -load_idt ((const struct desc_ptr *)&idt_descr); +load_current_idt ();memset (me->thread.tls_array,0 ,GDT_ENTRY_TLS_ENTRIES *8 );syscall_init ();
arch/x86/kernel/cpu/mcheck/therm_throt.c +18 -6
@@ -29,6 +29,7 @@#include <asm/idle.h> #include <asm/mce.h> #include <asm/msr.h> +#include <asm/trace/irq_vectors.h> /* How long to wait between reporting thermal events */ #define CHECK_INTERVAL (300 * HZ)
@@ -378,15 +379,26 @@ static void unexpected_thermal_interrupt(void)static void (*smp_thermal_vector)(void ) = unexpected_thermal_interrupt; -asmlinkage void smp_thermal_interrupt(struct pt_regs *regs) +static inline void __smp_thermal_interrupt (void ) { -irq_enter (); -exit_idle ();inc_irq_stat (irq_thermal_count);smp_thermal_vector (); -irq_exit (); -/* Ack only at the end to avoid potential reentry */ -ack_APIC_irq (); +} + +asmlinkage void smp_thermal_interrupt(struct pt_regs *regs) +{ +entering_irq (); +__smp_thermal_interrupt (); +exiting_ack_irq (); +} + +asmlinkage void smp_trace_thermal_interrupt(struct pt_regs *regs) +{ +entering_irq (); +trace_thermal_apic_entry (THERMAL_APIC_VECTOR ); +__smp_thermal_interrupt (); +trace_thermal_apic_exit (THERMAL_APIC_VECTOR ); +exiting_ack_irq (); }/* Thermal monitoring depends on APIC, ACPI and clock modulation */
arch/x86/kernel/cpu/mcheck/threshold.c +18 -6
@@ -8,6 +8,7 @@#include <asm/apic.h> #include <asm/idle.h> #include <asm/mce.h> +#include <asm/trace/irq_vectors.h> static void default_threshold_interrupt (void ) {
@@ -17,13 +18,24 @@ static void default_threshold_interrupt(void)void (*mce_threshold_vector)(void ) = default_threshold_interrupt; -asmlinkage void smp_threshold_interrupt(void) +static inline void __smp_threshold_interrupt (void ) { -irq_enter (); -exit_idle ();inc_irq_stat (irq_threshold_count);mce_threshold_vector (); -irq_exit (); -/* Ack only at the end to avoid potential reentry */ -ack_APIC_irq (); +} + +asmlinkage void smp_threshold_interrupt(void) +{ +entering_irq (); +__smp_threshold_interrupt (); +exiting_ack_irq (); +} + +asmlinkage void smp_trace_threshold_interrupt(void) +{ +entering_irq (); +trace_threshold_apic_entry (THRESHOLD_APIC_VECTOR ); +__smp_threshold_interrupt (); +trace_threshold_apic_exit (THRESHOLD_APIC_VECTOR ); +exiting_ack_irq (); }
arch/x86/kernel/entry_32.S +11 -1
@@ -801,7 +801,17 @@ ENTRY(name) \ CFI_ENDPROC; \ ENDPROC(name) -#define BUILD_INTERRUPT(name, nr) BUILD_INTERRUPT3(name, nr, smp_##name) + +#ifdef CONFIG_TRACING +#define TRACE_BUILD_INTERRUPT(name, nr) \ + BUILD_INTERRUPT3(trace_##name, nr, smp_trace_##name) +#else +#define TRACE_BUILD_INTERRUPT(name, nr) +#endif + +#define BUILD_INTERRUPT(name, nr) \ + BUILD_INTERRUPT3(name, nr, smp_##name); \ + TRACE_BUILD_INTERRUPT(name, nr) /* The include is where all of the SMP etc. interrupts come from */ #include <asm/entry_arch.h>
arch/x86/kernel/entry_64.S +24 -7
@@ -1138,7 +1138,7 @@ END(common_interrupt) /* * APIC interrupts. */ -.macro apicinterrupt num sym do_sym +.macro apicinterrupt3 num sym do_sym ENTRY(\sym) INTR_FRAME ASM_CLAC
@@ -1150,15 +1150,32 @@ ENTRY(\sym) END(\sym) .endm +#ifdef CONFIG_TRACING +#define trace(sym) trace_##sym +#define smp_trace(sym) smp_trace_##sym + +.macro trace_apicinterrupt num sym +apicinterrupt3 \num trace(\sym) smp_trace(\sym) +.endm +#else +.macro trace_apicinterrupt num sym do_sym +.endm +#endif + +.macro apicinterrupt num sym do_sym +apicinterrupt3 \num \sym \do_sym +trace_apicinterrupt \num \sym +.endm + #ifdef CONFIG_SMP -apicinterrupt IRQ_MOVE_CLEANUP_VECTOR \ +apicinterrupt3 IRQ_MOVE_CLEANUP_VECTOR \ irq_move_cleanup_interrupt smp_irq_move_cleanup_interrupt -apicinterrupt REBOOT_VECTOR \ +apicinterrupt3 REBOOT_VECTOR \ reboot_interrupt smp_reboot_interrupt #endif #ifdef CONFIG_X86_UV -apicinterrupt UV_BAU_MESSAGE \ +apicinterrupt3 UV_BAU_MESSAGE \ uv_bau_message_intr1 uv_bau_message_interrupt #endif apicinterrupt LOCAL_TIMER_VECTOR \
@@ -1167,7 +1184,7 @@ apicinterrupt X86_PLATFORM_IPI_VECTOR \ x86_platform_ipi smp_x86_platform_ipi #ifdef CONFIG_HAVE_KVM -apicinterrupt POSTED_INTR_VECTOR \ +apicinterrupt3 POSTED_INTR_VECTOR \ kvm_posted_intr_ipi smp_kvm_posted_intr_ipi #endif
@@ -1451,13 +1468,13 @@ ENTRY(xen_failsafe_callback) CFI_ENDPROC END(xen_failsafe_callback) -apicinterrupt HYPERVISOR_CALLBACK_VECTOR \ +apicinterrupt3 HYPERVISOR_CALLBACK_VECTOR \ xen_hvm_callback_vector xen_evtchn_do_upcall #endif /* CONFIG_XEN */ #if IS_ENABLED(CONFIG_HYPERV) -apicinterrupt HYPERVISOR_CALLBACK_VECTOR \ +apicinterrupt3 HYPERVISOR_CALLBACK_VECTOR \ hyperv_callback_vector hyperv_vector_handler #endif /* CONFIG_HYPERV */
arch/x86/kernel/head_64.S +7 -1
@@ -518,9 +518,15 @@ ENTRY(idt_table) .skip IDT_ENTRIES * 16 .align L1_CACHE_BYTES -ENTRY(nmi_idt_table) +ENTRY(debug_idt_table) .skip IDT_ENTRIES * 16 +#ifdef CONFIG_TRACING + .align L1_CACHE_BYTES +ENTRY(trace_idt_table) + .skip IDT_ENTRIES * 16 +#endif + __PAGE_ALIGNED_BSS NEXT_PAGE(empty_zero_page) .skip PAGE_SIZE
arch/x86/kernel/irq.c +21 -10
@@ -17,6 +17,7 @@#include <asm/idle.h> #include <asm/mce.h> #include <asm/hw_irq.h> +#include <asm/trace/irq_vectors.h> atomic_t irq_err_count;
@@ -204,23 +205,21 @@ unsigned int __irq_entry do_IRQ(struct pt_regs *regs)/* * Handler for X86_PLATFORM_IPI_VECTOR. */ -void smp_x86_platform_ipi (struct pt_regs *regs) +void __smp_x86_platform_ipi (void ) { -struct pt_regs *old_regs =set_irq_regs (regs); - -ack_APIC_irq (); - -irq_enter (); - -exit_idle (); -inc_irq_stat (x86_platform_ipis);if (x86_platform_ipi_callback)x86_platform_ipi_callback (); +} -irq_exit (); +void smp_x86_platform_ipi (struct pt_regs *regs) +{ +struct pt_regs *old_regs =set_irq_regs (regs); +entering_ack_irq (); +__smp_x86_platform_ipi (); +exiting_irq ();set_irq_regs (old_regs); }
@@ -246,6 +245,18 @@ void smp_kvm_posted_intr_ipi(struct pt_regs *regs) }#endif +void smp_trace_x86_platform_ipi (struct pt_regs *regs) +{ +struct pt_regs *old_regs =set_irq_regs (regs); + +entering_ack_irq (); +trace_x86_platform_ipi_entry (X86_PLATFORM_IPI_VECTOR ); +__smp_x86_platform_ipi (); +trace_x86_platform_ipi_exit (X86_PLATFORM_IPI_VECTOR ); +exiting_irq (); +set_irq_regs (old_regs); +} +EXPORT_SYMBOL_GPL (vector_used_by_percpu_irq);#ifdef CONFIG_HOTPLUG_CPU
arch/x86/kernel/irq_work.c +22 -2
@@ -8,14 +8,34 @@#include <linux/irq_work.h> #include <linux/hardirq.h> #include <asm/apic.h> +#include <asm/trace/irq_vectors.h> -void smp_irq_work_interrupt (struct pt_regs *regs) +static inline void irq_work_entering_irq (void ) {irq_enter ();ack_APIC_irq (); +} + +static inline void __smp_irq_work_interrupt (void ) +{inc_irq_stat (apic_irq_work_irqs);irq_work_run (); -irq_exit (); +} + +void smp_irq_work_interrupt (struct pt_regs *regs) +{ +irq_work_entering_irq (); +__smp_irq_work_interrupt (); +exiting_irq (); +} + +void smp_trace_irq_work_interrupt (struct pt_regs *regs) +{ +irq_work_entering_irq (); +trace_irq_work_entry (IRQ_WORK_VECTOR ); +__smp_irq_work_interrupt (); +trace_irq_work_exit (IRQ_WORK_VECTOR ); +exiting_irq (); }void arch_irq_work_raise (void )
arch/x86/kernel/smp.c +57 -8
95 lines not on this page · raw message
arch/x86/kernel/tracepoint.c new file +57 -0
57 lines not on this page · raw message
arch/x86/kernel/traps.c +1 -1
@@ -785,7 +785,7 @@ void __init trap_init(void) x86_init.irqs.trap_init ();#ifdef CONFIG_X86_64 -memcpy (&nmi_idt_table, &idt_table,IDT_ENTRIES *16 ); +memcpy (&debug_idt_table, &idt_table,IDT_ENTRIES *16 );set_nmi_gate (X86_TRAP_DB , &debug);set_nmi_gate (X86_TRAP_BP , &int3);#endif
include/xen/events.h +3 -0
@@ -76,6 +76,9 @@ unsigned irq_from_evtchn(unsigned int evtchn);/* Xen HVM evtchn vector callback */ void xen_hvm_callback_vector (void ); +#ifdef CONFIG_TRACING +#define trace_xen_hvm_callback_vector xen_hvm_callback_vector +#endif extern int xen_have_vector_callback;int xen_set_callback_via (uint64_t via);void xen_evtchn_do_upcall (struct pt_regs *regs);
Signature · 1 line
1.7.1
Steven Rostedt
maintainerRe: [PATCH v14 2/3] trace,x86: Introduce entering/exiting_irq()
In reply to Seiji AguchiQuoted 38 lines · Seiji Aguchi
--- arch/x86/include/asm/apic.h | 27 +++++++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-)arch/x86/include/asm/apic.h +19
-0@@ -12,6 +12,7 @@#include <asm/fixmap.h> #include <asm/mpspec.h> #include <asm/msr.h> +#include <asm/idle.h> #define ARCH_APICTIMER_STOPS_ON_C3 1@@ -687,5 +688,31 @@ extern int default_check_phys_apicid_present(int phys_apicid);#endif #endif /* CONFIG_X86_LOCAL_APIC */ +extern void irq_enter (void ); +extern void irq_exit (void ); + +static inline void entering_irq (void ) +{ +irq_enter (); +exit_idle (); +} + +static inline void entering_ack_irq (void ) +{ +ack_APIC_irq (); +entering_irq (); +} + +static inline void exiting_irq (void ) +{ +irq_exit ();
arch/x86/include/asm/apic.h extern int default_check_phys_apicid_present(int phys_apicid); · line 718
Quoted 10 lines
arch/x86/include/asm/apic.h +8
-0+} + +static inline void exiting_ack_irq (void ) +{ +irq_exit (); +/* Ack only at the end to avoid potential reentry */ +ack_APIC_irq (); +}#endif /* _ASM_X86_APIC_H */
I think it also makes sense to use these in this patch as well. That is, go ahead and make the changes likes this:
@@ -919,17 +919,14 @@ void __irq_entry smp_apic_timer_interrupt(struct pt_regs *regs)
/*
* NOTE! We'd better ACK the irq immediately,
* because timer handling can be slow.
- */
- ack_APIC_irq();
- /*
+ *
* update_process_times() expects us to have done irq_enter().
* Besides, if we don't timer interrupts ignore the global
* interrupt lock, which is the WrongThing (tm) to do.
*/
- irq_enter();
- exit_idle();
+ entering_ack_irq();
local_apic_timer_interrupt();
- irq_exit();
+ exiting_irq();
set_irq_regs(old_regs);
}
And do that with all the interrupts.
I think this is getting close for adding to 3.11 queue.
-- Steve
Steven Rostedt
maintainerRe: [PATCH v14 3/3] trace,x86: Add irq vector tracepoints
In reply to Seiji AguchiQuoted 24 lines · Seiji Aguchi
arch/x86/include/asm/desc.h +8
-3@@ -36,8 +36,8 @@ static inline void fill_ldt(struct desc_struct *desc, const struct user_desc *inextern struct desc_ptr idt_descr;extern gate_desc idt_table[]; -extern struct desc_ptr nmi_idt_descr; -extern gate_desc nmi_idt_table[]; +extern struct desc_ptr debug_idt_descr; +extern gate_desc debug_idt_table[];struct gdt_page {struct desc_struct gdt[GDT_ENTRIES ];@@ -316,7 +316,20 @@ static inline void set_nmi_gate(int gate, void *addr)gate_desc s;pack_gate (&s,GATE_INTERRUPT , (unsigned long )addr,0 ,0 , __KERNEL_CS); -write_idt_entry (nmi_idt_table, gate, &s); + write_idt_entry(debug_idt_table, gate, &s); +} +#endif + +#ifdef CONFIG_TRACING +extern struct desc_ptr trace_idt_descr;
arch/x86/include/asm/desc.h static inline void set_nmi_gate(int gate, void *addr) · line 334
Quoted 10 lines
arch/x86/include/asm/desc.h +8
-0+extern gate_desc trace_idt_table[]; +static inline void write_trace_idt_entry (int entry,const gate_desc *gate) +{ +write_idt_entry (trace_idt_table, entry, gate); +} +#else +static inline void write_trace_idt_entry (int entry,const gate_desc *gate) +{ }#endif
Since we are basically down to cosmetic changes, can you make the renames of these variables into a separate patch. As they are now more about the debug IDT than a NMI idt, it should have been done before, and is a separate change to what you are doing here.
arch/x86/include/asm/desc.h static inline void alloc_system_vector(int vector) · line 376
Quoted 13 lines
arch/x86/include/asm/desc.h +1
-0@@ -331,6 +344,7 @@ static inline void _set_gate(int gate, unsigned type, void *addr,* setup time */ write_idt_entry (idt_table, gate, &s); +write_trace_idt_entry (gate, &s); }/* @@ -360,12 +374,39 @@ static inline void alloc_system_vector(int vector) } }
-- Steve
Steven Rostedt
maintainerRe: [PATCH v14 3/3] trace,x86: Add irq vector tracepoints
Quoted 6 lines · Steven Rostedt
On Thu, 2013-06-06 at 11:24 -0400, Seiji Aguchi wrote:
Since we are basically down to cosmetic changes, can you make the renames of these variables into a separate patch. As they are now more about the debug IDT than a NMI idt, it should have been done before, and is a separate change to what you are doing here.
I would even add the is_debug_idt_enabled() and load_debug_idt() interface to that patch as well.
-- Steve
RE: [PATCH v14 2/3] trace,x86: Introduce entering/exiting_irq()
In reply to Steven RostedtQuoted 85 lines
-----Original Message-----
From: Steven Rostedt [mailto:[email protected]]
Sent: Thursday, June 06, 2013 2:26 PM
To: Seiji Aguchi
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Tomoki Sekiyama
Subject: Re: [PATCH v14 2/3] trace,x86: Introduce entering/exiting_irq()Quoted 38 lines · Seiji Aguchi
--- arch/x86/include/asm/apic.h | 27 +++++++++++++++++++++++++++ 1 files changed, 27 insertions(+), 0 deletions(-)arch/x86/include/asm/apic.h +19
-0@@ -12,6 +12,7 @@#include <asm/fixmap.h> #include <asm/mpspec.h> #include <asm/msr.h> +#include <asm/idle.h> #define ARCH_APICTIMER_STOPS_ON_C3 1@@ -687,5 +688,31 @@ extern int default_check_phys_apicid_present(int phys_apicid);#endif #endif /* CONFIG_X86_LOCAL_APIC */ +extern void irq_enter (void ); +extern void irq_exit (void ); + +static inline void entering_irq (void ) +{ +irq_enter (); +exit_idle (); +} + +static inline void entering_ack_irq (void ) +{ +ack_APIC_irq (); +entering_irq (); +} + +static inline void exiting_irq (void ) +{ +irq_exit ();arch/x86/include/asm/apic.h extern int default_check_phys_apicid_present(int phys_apicid); · line 718
Quoted 10 lines
arch/x86/include/asm/apic.h +8
-0+} + +static inline void exiting_ack_irq (void ) +{ +irq_exit (); +/* Ack only at the end to avoid potential reentry */ +ack_APIC_irq (); +}#endif /* _ASM_X86_APIC_H */ I think it also makes sense to use these in this patch as well. That is, go ahead and make the changes likes this:
(quoted diff) +3
-6@@ -919,17 +919,14 @@ void __irq_entry smp_apic_timer_interrupt(struct pt_regs *regs) /* * NOTE! We'd better ACK the irq immediately, * because timer handling can be slow. - */ - ack_APIC_irq(); - /* + * * update_process_times() expects us to have done irq_enter(). * Besides, if we don't timer interrupts ignore the global * interrupt lock, which is the WrongThing (tm) to do. */ - irq_enter(); - exit_idle(); + entering_ack_irq(); local_apic_timer_interrupt(); - irq_exit(); + exiting_irq(); set_irq_regs(old_regs); }
Quoted 1 line
And do that with all the interrupts.
OK. Will do.
Seiji
Quoted 5 lines
I think this is getting close for adding to 3.11 queue.
-- Steve
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
RE: [PATCH v14 3/3] trace,x86: Add irq vector tracepoints
In reply to Steven RostedtSteven,
Thank you for reviewing.
I will create a patch just renaming variables and adding the is_debug_idt_enabled() and load_debug_idt().
Seiji
Quoted 16 lines
-----Original Message-----
From: Steven Rostedt [mailto:[email protected]]
Sent: Thursday, June 06, 2013 4:01 PM
To: Seiji Aguchi
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Tomoki Sekiyama
Subject: Re: [PATCH v14 3/3] trace,x86: Add irq vector tracepointsQuoted 6 lines · Steven Rostedt
On Thu, 2013-06-06 at 11:24 -0400, Seiji Aguchi wrote:
Since we are basically down to cosmetic changes, can you make the renames of these variables into a separate patch. As they are now more about the debug IDT than a NMI idt, it should have been done before, and is a separate change to what you are doing here.
Quoted 5 lines
I would even add the
is_debug_idt_enabled()andload_debug_idt()interface to that patch as well.-- Steve
���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m����������?�I�
Steven Rostedt
maintainerRe: [PATCH v14 3/3] trace,x86: Add irq vector tracepoints
Quoted 6 lines · Seiji Aguchi
Steven,
Thank you for reviewing.
I will create a patch just renaming variables and adding the
is_debug_idt_enabled()andload_debug_idt().
Thanks, and you can have the code use those functions as well.
-- Steve