Skip to contentSkip to conversation

kprobes for s390 architecture]

PATCH Michael Grundy 3 participants 18 messages Jun 23, 2006 – Jul 11, 2006 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Heiko Carstens <[email protected]> wrote on 06/23/2006 08:03:44 AM:

Quoted 1 line

This won't solve anything. What Martin probably meant is something like a

poor

Quoted 1 line

man's stop_machine_run() implemented by using smp_call_function(). This

way

Quoted 1 line

you synchronize all cpus and when all cpus are in a known state, you

change

Quoted 1 line

the instruction in question and make sure that serialization happens

before

Quoted 5 lines

cpus leave the handler again... Except for the cpu that called smp_call_function() you get the serialization for free, since the last instruction of the handler is always an lpsw/lpswe instruction.

Otherwise there is still the possibility that a different cpu is fetching

the

Quoted 1 line

instruction concurrently while you change it. This doesn't sound very

good,

Quoted 4 lines

especially if you take this paragraph of the Principles of Operation into account (p.5-89 of SA22-7832-04):

"It is possible, if another CPU or a channel program concurrently

modifies

Quoted 1 line

the instruction, for one CPU to recognize the changes to some but not all

bit

Quoted 1 line

positions of an instruction."

(link to doc for anyone following along at home:
http://publibz.boulder.ibm.com/epubs/pdf/a2278324.pdf)

On the same page it says "All copies of a prefetched instruction are discarded
when: * A serializing function is performed" Would something like this in a smp_call_function do it? :

bcr 15,0

if (*p->addr != breakpoint_instruction)

      *p->addr = breakpoint_instruction;

Alternatively, if we did a compare and swap on that location (serializing instruction) would that be acceptable?

Thanks
Michael

=========================================
If at first you don't succeed, call in an air strike.

#2 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 8 lines

On the same page it says "All copies of a prefetched instruction are discarded
when: * A serializing function is performed" Would something like this in a smp_call_function do it? :

bcr 15,0

if (*p->addr != breakpoint_instruction)

Quoted 8 lines
      *p->addr = breakpoint_instruction;

Alternatively, if we did a compare and swap on that location (serializing instruction) would that be acceptable?

Thanks
Michael

The crap below is something that could solve your problem (assumes that "a" is the address of the instruction to be replaced and 0x42 is the opcode of the new instruction):

- generates an irq on all other cpus -> prefetched stuff on them discarded
- catches all cpus
- writes the new instruction
- the atomic_inc(&cap.done) is a compare and swap instruction -> serialization

At least this is something that could work... completely untested and might have some problems that I didn't think of ;)

struct capture_data {
	atomic_t cpus;
	atomic_t done;
};

void capture_wait(void *data)

{ 
	struct capture_data *cap = data;

	atomic_inc(&cap->cpus);
	while(!atomic_read(&cap->done))
		cpu_relax();
	atomic_dec(&cap->cpus);
}

void replace_instr(int *a)

{
	struct capture_data cap;

	preempt_disable();
	atomic_set(&cap.cpus, 0);
	atomic_set(&cap.done, 0);
	smp_call_function(capture_wait, (void *)&cap, 0, 0);
	while (atomic_read(&cap.cpus) != num_online_cpus() - 1)
		cpu_relax();
	*a = 0x42;
	atomic_inc(&cap.done);
	while (atomic_read(&cap.cpus))
		cpu_relax();
	preempt_enable();
}
#3 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 19 lines

At least this is something that could work... completely untested and might have some problems that I didn't think of ;)

struct capture_data {
	atomic_t cpus;
	atomic_t done;
};

void capture_wait(void *data)

{ 
	struct capture_data *cap = data;

	atomic_inc(&cap->cpus);
	while(!atomic_read(&cap->done))
		cpu_relax();
	atomic_dec(&cap->cpus);
}

void replace_instr(int *a)

Quoted 15 lines
{
	struct capture_data cap;

	preempt_disable();
	atomic_set(&cap.cpus, 0);
	atomic_set(&cap.done, 0);
	smp_call_function(capture_wait, (void *)&cap, 0, 0);
	while (atomic_read(&cap.cpus) != num_online_cpus() - 1)
		cpu_relax();
	*a = 0x42;
	atomic_inc(&cap.done);
	while (atomic_read(&cap.cpus))
		cpu_relax();
	preempt_enable();
}

Forget this crap. It can easily cause deadlocks with more than two cpus.

Just do a compare and swap operation on the instruction you want to replace, then do an smp_call_function() with the wait parameter set to 1 and passing a pointer to a function that does nothing but return.

The cs/csg instruction will make sure that your cpu has exclusive access to the memory region in question and will invalidate the cache lines on all other cpus.
With the following smp_call_function() you can make sure that all other cpus discard everything they have prefetched. Hence there is only a small window between the cs/csg and the return of smp_call_function() where you do not know if other cpus are executing the old or the new instruction.

#4 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 3 lines

Just do a compare and swap operation on the instruction you want to replace, then do an smp_call_function() with the wait parameter set to 1 and passing a pointer to a function that does nothing but return.

Setting wait to 1 isn't necessary as well. Wondering if I get anything right today...

#5 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 3 lines · Heiko Carstens
Quoted 3 lines

Just do a compare and swap operation on the instruction you want to replace, then do an smp_call_function() with the wait parameter set to 1 and passing a pointer to a function that does nothing but return.

Here's what I came up with Friday before I jumped timezones back east:

void smp_replace_instruction(void *info)

{
	struct ins_replace_args *parms;

	parms = (struct ins_replace_args *) info;
	cmpxchg(parms->addr, parms->oinsn, parms->ninsn);
}

void __kprobes arch_arm_kprobe(struct kprobe *p)

{
	struct ins_replace_args parms;
	parms.addr = p->addr;
	parms.ninsn = BREAKPOINT_INSTRUCTION;
	parms.oinsn = p->opcode;

	on_each_cpu(smp_replace_instruction, &parms, 0, 1);
} etc...

After reading your notes it's probably overkill doing the cs on each cpu, since the interrupt will discard the prefetched instructions.

Signature · 10 lines
Thanks
Mike

=========================================
Michael Grundy - [email protected]
Advanced Linux Response Team (ALRT)
http://ltc.linux.ibm.com/teamweb/alrt/
845-435-8842 (T/L 295)

If at first you don't succeed, call in an air strike.
#6 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 20 lines

Here's what I came up with Friday before I jumped timezones back east:

void smp_replace_instruction(void *info)

{
	struct ins_replace_args *parms;

	parms = (struct ins_replace_args *) info;
	cmpxchg(parms->addr, parms->oinsn, parms->ninsn);
}

void __kprobes arch_arm_kprobe(struct kprobe *p)

{
	struct ins_replace_args parms;
	parms.addr = p->addr;
	parms.ninsn = BREAKPOINT_INSTRUCTION;
	parms.oinsn = p->opcode;

	on_each_cpu(smp_replace_instruction, &parms, 0, 1);
} etc...
Quoted 2 lines

After reading your notes it's probably overkill doing the cs on each cpu, since the interrupt will discard the prefetched instructions.

Indeed. Another thing that should not be forgotten: it could be that the whole kernel text segment resides in a shared read only segment. So it can be shared by multiple z/VM guests.
In that case the cs instruction will fail. Looks like you need to write the part that replaces the instruction in assembly and supply a fixup section which in turn makes sure that -EFAULT is returned.

#7 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 9 lines · Heiko Carstens
Quoted 2 lines

After reading your notes it's probably overkill doing the cs on each cpu, since the interrupt will discard the prefetched instructions.

Indeed. Another thing that should not be forgotten: it could be that the whole kernel text segment resides in a shared read only segment. So it can be shared by multiple z/VM guests.
In that case the cs instruction will fail. Looks like you need to write the part that replaces the instruction in assembly and supply a fixup section which in turn makes sure that -EFAULT is returned.

If it fails, won't it will generate a program interrupt, 5 (access exception)?

#8 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 4 lines · Mike Grundy
Quoted 3 lines · Heiko Carstens
Quoted 2 lines

After reading your notes it's probably overkill doing the cs on each cpu, since the interrupt will discard the prefetched instructions.

Quoted 8 lines
Quoted 6 lines

Indeed. Another thing that should not be forgotten: it could be that the whole kernel text segment resides in a shared read only segment. So it can be shared by multiple z/VM guests.
In that case the cs instruction will fail. Looks like you need to write the part that replaces the instruction in assembly and supply a fixup section which in turn makes sure that -EFAULT is returned.

If it fails, won't it will generate a program interrupt, 5 (access exception)?

Yes, that's why you need a fixup section :)

#9 lore raw

Re: [PATCH] kprobes for s390 architecture

In reply to Heiko Carstens
Quoted 35 lines · Heiko Carstens
Quoted 19 lines

At least this is something that could work... completely untested and might have some problems that I didn't think of ;)

struct capture_data {
	atomic_t cpus;
	atomic_t done;
};

void capture_wait(void *data)

{ 
	struct capture_data *cap = data;

	atomic_inc(&cap->cpus);
	while(!atomic_read(&cap->done))
		cpu_relax();
	atomic_dec(&cap->cpus);
}

void replace_instr(int *a)

Quoted 15 lines
{
	struct capture_data cap;

	preempt_disable();
	atomic_set(&cap.cpus, 0);
	atomic_set(&cap.done, 0);
	smp_call_function(capture_wait, (void *)&cap, 0, 0);
	while (atomic_read(&cap.cpus) != num_online_cpus() - 1)
		cpu_relax();
	*a = 0x42;
	atomic_inc(&cap.done);
	while (atomic_read(&cap.cpus))
		cpu_relax();
	preempt_enable();
}
Quoted 1 line

Forget this crap. It can easily cause deadlocks with more than two cpus.

It is not that bad. Instead of preempt_disable/preempt_enable we need a spinlock. Then only one cpu can do this particular smp_call_function which will "stop" all other cpus until cap->done has been set.

Quoted 3 lines

Just do a compare and swap operation on the instruction you want to replace, then do an smp_call_function() with the wait parameter set to 1 and passing a pointer to a function that does nothing but return.

Not good enough. An instruction can be fetched multiple times for a single execution (see the other mail). So you have a half executed instruction, the cache line is invalidated, a new instruction is written and the cache line is recreated to finished the half executed instruction. That can easiliy happen on millicoded instructions.

Quoted 3 lines

The cs/csg instruction will make sure that your cpu has exclusive access to the memory region in question and will invalidate the cache lines on all other cpus.

That the cache line is invalidated does not mean that you are safe..

Quoted 4 lines

With the following smp_call_function() you can make sure that all other cpus discard everything they have prefetched. Hence there is only a small window between the cs/csg and the return of smp_call_function() where you do not know if other cpus are executing the old or the new instruction.

The serialization is indeed done by the smp_call_function(). No need to have a "bcr 15,0" in the called function, the lpsw at the end of the interrupt already does the serialization.

Signature · 8 lines
blue skies,
  Martin.

Martin Schwidefsky
Linux for zSeries Development & Services
IBM Deutschland Entwicklung GmbH

"Reality continues to ruin my life." - Calvin.
#10 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 36 lines · Martin Schwidefsky
Quoted 35 lines · Heiko Carstens
Quoted 34 lines

At least this is something that could work... completely untested and might have some problems that I didn't think of ;)

struct capture_data {
	atomic_t cpus;
	atomic_t done;
};

void capture_wait(void *data)

{ 
	struct capture_data *cap = data;

	atomic_inc(&cap->cpus);
	while(!atomic_read(&cap->done))
		cpu_relax();
	atomic_dec(&cap->cpus);
}

void replace_instr(int *a)

{
	struct capture_data cap;

	preempt_disable();
	atomic_set(&cap.cpus, 0);
	atomic_set(&cap.done, 0);
	smp_call_function(capture_wait, (void *)&cap, 0, 0);
	while (atomic_read(&cap.cpus) != num_online_cpus() - 1)
		cpu_relax();
	*a = 0x42;
	atomic_inc(&cap.done);
	while (atomic_read(&cap.cpus))
		cpu_relax();
	preempt_enable();
}
Quoted 5 lines
Quoted 1 line

Forget this crap. It can easily cause deadlocks with more than two cpus.

It is not that bad. Instead of preempt_disable/preempt_enable we need a spinlock. Then only one cpu can do this particular smp_call_function which will "stop" all other cpus until cap->done has been set.

CPU0: smp_call_function() -> loops and waits for other cpus
CPU1: [irqs_enabled] - spin_lock(somelock) -> irq -> capture_wait() -> loop
CPU2: [irqs_enabled] ----- spin_lock_irqsave(a,..) -> toasted

CPU2 ends up trying to grab the same lock that CPU1 holds, but has interrupts disabled and a pending external interrupt because of the smp_call_function()..

Quoted 8 lines
Quoted 3 lines

Just do a compare and swap operation on the instruction you want to replace, then do an smp_call_function() with the wait parameter set to 1 and passing a pointer to a function that does nothing but return.

Not good enough. An instruction can be fetched multiple times for a single execution (see the other mail). So you have a half executed instruction, the cache line is invalidated, a new instruction is written and the cache line is recreated to finished the half executed instruction. That can easiliy happen on millicoded instructions.

Yes, looks like I was too optimistic. Seems like we really have to go for stop_machine_run() unless somebody comes up with a better idea...

#11 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 5 lines · Heiko Carstens
Quoted 4 lines · Martin Schwidefsky
Quoted 3 lines · Heiko Carstens

Just do a compare and swap operation on the instruction you want to replace, then do an smp_call_function() with the wait parameter set to 1 and passing a pointer to a function that does nothing but return.

Quoted 8 lines
Quoted 5 lines

Not good enough. An instruction can be fetched multiple times for a single execution (see the other mail). So you have a half executed instruction, the cache line is invalidated, a new instruction is written and the cache line is recreated to finished the half executed instruction. That can easiliy happen on millicoded instructions.

Yes, looks like I was too optimistic. Seems like we really have to go for stop_machine_run() unless somebody comes up with a better idea...

ok, I tried, but my "better ideas" made things worse. stop_machine_run() wins:

void __kprobes arch_arm_kprobe(struct kprobe *p)

{
        struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
        unsigned long status = kcb->kprobe_status;
        struct ins_replace_args args;

        args.ptr = p->addr;
        args.old = p->opcode;
        args.new = BREAKPOINT_INSTRUCTION;

        kcb->kprobe_status = KPROBE_SWAP_INST;
        stop_machine_run(swap_instruction, &args, NR_CPUS);
        kcb->kprobe_status = status;
}

It works, and I guess at this point is the only way to do it. I'll send out a full patch with this and the other cleanups later.

Mike

#12 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 17 lines

ok, I tried, but my "better ideas" made things worse. stop_machine_run() wins:

void __kprobes arch_arm_kprobe(struct kprobe *p)

{
        struct kprobe_ctlblk *kcb = get_kprobe_ctlblk();
        unsigned long status = kcb->kprobe_status;
        struct ins_replace_args args;

        args.ptr = p->addr;
        args.old = p->opcode;
        args.new = BREAKPOINT_INSTRUCTION;

        kcb->kprobe_status = KPROBE_SWAP_INST;
        stop_machine_run(swap_instruction, &args, NR_CPUS);
        kcb->kprobe_status = status;
}
Quoted 2 lines

It works, and I guess at this point is the only way to do it. I'll send out a full patch with this and the other cleanups later.

How fast is this if you have to exchange several hundred instructions?

#13 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 1 line · Heiko Carstens

How fast is this if you have to exchange several hundred instructions?

I'll have to do some timing tests to measure how much overhead it puts on the system. It's a trade off of safety vs speed. Each increment of safety we've put on the swap has had inherent performance penalties. The plus side is the swaps are only done on activation and deactivation, and don't have this kind of overhead during operations.

The latest patch is attached.

Thanks
Mike

 arch/s390/Kconfig              |   14
 arch/s390/kernel/Makefile      |    1
 arch/s390/kernel/entry.S       |   12
 arch/s390/kernel/entry64.S     |   12
 arch/s390/kernel/kprobes.c     |  692 +++++++++++++++++++++++++++++++++++++++++
 arch/s390/kernel/traps.c       |   42 ++
 arch/s390/kernel/vmlinux.lds.S |    1
 arch/s390/mm/fault.c           |   40 ++
 include/asm-s390/kdebug.h      |   57 +++
 include/asm-s390/kprobes.h     |  111 ++++++
 10 files changed, 976 insertions(+), 6 deletions(-)
arch/s390/Kconfig +14 -0
diff -Nurp linux-2.6.17-git25/arch/s390/Kconfig linux-2.6.17-git25-kp390/arch/s390/Kconfig
@@ -490,8 +490,22 @@ source "drivers/net/Kconfig"
 
 source "fs/Kconfig"
 
+menu "Instrumentation Support"
+	depends on EXPERIMENTAL
+
 source "arch/s390/oprofile/Kconfig"
 
+config KPROBES
+	bool "Kprobes (EXPERIMENTAL)"
+	depends on EXPERIMENTAL && MODULES
+	help
+	  Kprobes allows you to trap at almost any kernel address and
+	  execute a callback function.  register_kprobe() establishes
+	  a probepoint and specifies the callback.  Kprobes is useful
+	  for kernel debugging, non-intrusive instrumentation and testing.
+	  If in doubt, say "N".
+endmenu
+
 source "arch/s390/Kconfig.debug"
 
 source "security/Kconfig"
arch/s390/kernel/entry64.S +12 -0
diff -Nurp linux-2.6.17-git25/arch/s390/kernel/entry64.S linux-2.6.17-git25-kp390/arch/s390/kernel/entry64.S
@@ -518,6 +518,8 @@ pgm_no_vtime2:
 #endif
 	lg	%r9,__LC_THREAD_INFO	# load pointer to thread_info struct
 	lg	%r1,__TI_task(%r9)
+	tm	__LC_PGM_OLD_PSW+1(%r15),0x01	# kernel per event ?
+	jz	kernel_per
 	mvc	__THREAD_per+__PER_atmid(2,%r1),__LC_PER_ATMID
 	mvc	__THREAD_per+__PER_address(8,%r1),__LC_PER_ADDRESS
 	mvc	__THREAD_per+__PER_access_id(1,%r1),__LC_PER_ACCESS_ID
@@ -553,6 +555,16 @@ pgm_no_vtime3:
 	stosm	__SF_EMPTY(%r15),0x03	# reenable interrupts
 	j	sysc_do_svc
 
+#
+# per was called from kernel, must be kprobes
+#
+kernel_per:
+	lhi	%r0,__LC_PGM_OLD_PSW
+	sth	%r0,SP_TRAP(%r15)	# set trap indication to pgm check
+	la	%r2,SP_PTREGS(%r15)	# address of register-save area
+	larl	%r14,sysc_leave		# load adr. of system ret, no work
+	jg	do_single_step		# branch to do_single_step
+
 /*
  * IO interrupt handler routine
  */
arch/s390/kernel/entry.S +12 -0
diff -Nurp linux-2.6.17-git25/arch/s390/kernel/entry.S linux-2.6.17-git25-kp390/arch/s390/kernel/entry.S
@@ -505,6 +505,8 @@ pgm_no_vtime2:
 	mvc	__THREAD_per+__PER_address(4,%r1),__LC_PER_ADDRESS
 	mvc	__THREAD_per+__PER_access_id(1,%r1),__LC_PER_ACCESS_ID
 	oi	__TI_flags+3(%r9),_TIF_SINGLE_STEP # set TIF_SINGLE_STEP
+	tm	__LC_PGM_OLD_PSW+1(%r15),0x01	# kernel per event ?
+	bz	BASED(kernel_per)
 	l	%r3,__LC_PGM_ILC	 # load program interruption code
 	la	%r8,0x7f
 	nr	%r8,%r3                  # clear per-event-bit and ilc
@@ -536,6 +538,16 @@ pgm_no_vtime3:
 	stosm	__SF_EMPTY(%r15),0x03	# reenable interrupts
 	b	BASED(sysc_do_svc)
 
+#
+# per was called from kernel, must be kprobes
+#
+kernel_per:
+	mvi	SP_TRAP+1(%r15),0x28	# set trap indication to pgm check
+	la	%r2,SP_PTREGS(%r15)	# address of register-save area
+	l	%r1,BASED(.Lhandle_per)	# load adr. of per handler
+	la	%r14,BASED(sysc_leave)	# load adr. of system return
+	br	%r1			# branch to do_single_step
+
 /*
  * IO interrupt handler routine
  */
arch/s390/kernel/kprobes.c +692 -0

692 lines not on this page · raw message

arch/s390/kernel/Makefile +1 -0
diff -Nurp linux-2.6.17-git25/arch/s390/kernel/Makefile linux-2.6.17-git25-kp390/arch/s390/kernel/Makefile
@@ -22,6 +22,7 @@ obj-$(CONFIG_BINFMT_ELF32)	+= binfmt_elf
 
 obj-$(CONFIG_VIRT_TIMER)	+= vtime.o
 obj-$(CONFIG_STACKTRACE)	+= stacktrace.o
+obj-$(CONFIG_KPROBES)		+= kprobes.o
 
 # Kexec part
 S390_KEXEC_OBJS := machine_kexec.o crash.o
arch/s390/kernel/traps.c +37 -5

85 lines not on this page · raw message

arch/s390/kernel/vmlinux.lds.S +1 -0
diff -Nurp linux-2.6.17-git25/arch/s390/kernel/vmlinux.lds.S linux-2.6.17-git25-kp390/arch/s390/kernel/vmlinux.lds.S
@@ -24,6 +24,7 @@ SECTIONS
 	*(.text)
 	SCHED_TEXT
 	LOCK_TEXT
+	KPROBES_TEXT
 	*(.fixup)
 	*(.gnu.warning)
 	} = 0x0700
arch/s390/mm/fault.c +39 -1
diff -Nurp linux-2.6.17-git25/arch/s390/mm/fault.c linux-2.6.17-git25-kp390/arch/s390/mm/fault.c
@@ -25,10 +25,12 @@
 #include <linux/console.h>
 #include <linux/module.h>
 #include <linux/hardirq.h>
+#include <linux/kprobes.h>
 
 #include <asm/system.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>
+#include <asm/kdebug.h>
 
 #ifndef CONFIG_64BIT
 #define __FAIL_ADDR_MASK 0x7ffff000
@@ -48,6 +50,38 @@ extern int sysctl_userprocess_debug;
 
 extern void die(const char *,struct pt_regs *,long);
 
+#ifdef CONFIG_KPROBES
+ATOMIC_NOTIFIER_HEAD(notify_page_fault_chain);
+int register_page_fault_notifier(struct notifier_block *nb)
+{
+	return atomic_notifier_chain_register(&notify_page_fault_chain, nb);
+}
+
+int unregister_page_fault_notifier(struct notifier_block *nb)
+{
+	return atomic_notifier_chain_unregister(&notify_page_fault_chain, nb);
+}
+
+static inline int notify_page_fault(enum die_val val, const char *str,
+			struct pt_regs *regs, long err, int trap, int sig)
+{
+	struct die_args args = {
+		.regs = regs,
+		.str = str,
+		.err = err,
+		.trapnr = trap,
+		.signr = sig
+	};
+	return atomic_notifier_call_chain(&notify_page_fault_chain, val, &args);
+}
+#else
+static inline int notify_page_fault(enum die_val val, const char *str,
+			struct pt_regs *regs, long err, int trap, int sig)
+{
+	return NOTIFY_DONE;
+}
+#endif
+
 extern spinlock_t timerlist_lock;
 
 /*
@@ -159,7 +193,7 @@ static void do_sigsegv(struct pt_regs *r
  *   11       Page translation     ->  Not present       (nullification)
  *   3b       Region third trans.  ->  Not present       (nullification)
  */
-static inline void
+static inline void __kprobes
 do_exception(struct pt_regs *regs, unsigned long error_code, int is_protection)
 {
         struct task_struct *tsk;
@@ -173,6 +207,10 @@ do_exception(struct pt_regs *regs, unsig
         tsk = current;
         mm = tsk->mm;
 	
+	if (notify_page_fault(DIE_PAGE_FAULT, "page fault", regs, error_code, 14,
+					SIGSEGV) == NOTIFY_STOP)
+		return;
+
 	/* 
          * Check for low-address protection.  This needs to be treated
 	 * as a special case because the translation exception code 
include/asm-s390/kdebug.h +57 -0

57 lines not on this page · raw message

include/asm-s390/kprobes.h +111 -0

111 lines not on this page · raw message

#14 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 1 line · Michael Grundy

The latest patch is attached.

Yeah, I wasn't paying attention, the 2.6.18-rc1 version is attached, no difference really (other than testing it with the -rc1 code)

Thanks
Mike

--

 arch/s390/Kconfig              |   14
 arch/s390/kernel/Makefile      |    1
 arch/s390/kernel/entry.S       |   12
 arch/s390/kernel/entry64.S     |   12
 arch/s390/kernel/kprobes.c     |  692 +++++++++++++++++++++++++++++++++++++++++
 arch/s390/kernel/traps.c       |   42 ++
 arch/s390/kernel/vmlinux.lds.S |    1
 arch/s390/mm/fault.c           |   40 ++
 include/asm-s390/kdebug.h      |   57 +++
 include/asm-s390/kprobes.h     |  111 ++++++
 10 files changed, 976 insertions(+), 6 deletions(-)
arch/s390/Kconfig +14 -0
diff -Nurp linux-2.6.18-rc1/arch/s390/Kconfig linux-2.6.18-rc1-kp390/arch/s390/Kconfig
@@ -490,8 +490,22 @@ source "drivers/net/Kconfig"
 
 source "fs/Kconfig"
 
+menu "Instrumentation Support"
+	depends on EXPERIMENTAL
+
 source "arch/s390/oprofile/Kconfig"
 
+config KPROBES
+	bool "Kprobes (EXPERIMENTAL)"
+	depends on EXPERIMENTAL && MODULES
+	help
+	  Kprobes allows you to trap at almost any kernel address and
+	  execute a callback function.  register_kprobe() establishes
+	  a probepoint and specifies the callback.  Kprobes is useful
+	  for kernel debugging, non-intrusive instrumentation and testing.
+	  If in doubt, say "N".
+endmenu
+
 source "arch/s390/Kconfig.debug"
 
 source "security/Kconfig"
arch/s390/kernel/entry64.S +12 -0
diff -Nurp linux-2.6.18-rc1/arch/s390/kernel/entry64.S linux-2.6.18-rc1-kp390/arch/s390/kernel/entry64.S
@@ -518,6 +518,8 @@ pgm_no_vtime2:
 #endif
 	lg	%r9,__LC_THREAD_INFO	# load pointer to thread_info struct
 	lg	%r1,__TI_task(%r9)
+	tm	__LC_PGM_OLD_PSW+1(%r15),0x01	# kernel per event ?
+	jz	kernel_per
 	mvc	__THREAD_per+__PER_atmid(2,%r1),__LC_PER_ATMID
 	mvc	__THREAD_per+__PER_address(8,%r1),__LC_PER_ADDRESS
 	mvc	__THREAD_per+__PER_access_id(1,%r1),__LC_PER_ACCESS_ID
@@ -553,6 +555,16 @@ pgm_no_vtime3:
 	stosm	__SF_EMPTY(%r15),0x03	# reenable interrupts
 	j	sysc_do_svc
 
+#
+# per was called from kernel, must be kprobes
+#
+kernel_per:
+	lhi	%r0,__LC_PGM_OLD_PSW
+	sth	%r0,SP_TRAP(%r15)	# set trap indication to pgm check
+	la	%r2,SP_PTREGS(%r15)	# address of register-save area
+	larl	%r14,sysc_leave		# load adr. of system ret, no work
+	jg	do_single_step		# branch to do_single_step
+
 /*
  * IO interrupt handler routine
  */
arch/s390/kernel/entry.S +12 -0
diff -Nurp linux-2.6.18-rc1/arch/s390/kernel/entry.S linux-2.6.18-rc1-kp390/arch/s390/kernel/entry.S
@@ -505,6 +505,8 @@ pgm_no_vtime2:
 	mvc	__THREAD_per+__PER_address(4,%r1),__LC_PER_ADDRESS
 	mvc	__THREAD_per+__PER_access_id(1,%r1),__LC_PER_ACCESS_ID
 	oi	__TI_flags+3(%r9),_TIF_SINGLE_STEP # set TIF_SINGLE_STEP
+	tm	__LC_PGM_OLD_PSW+1(%r15),0x01	# kernel per event ?
+	bz	BASED(kernel_per)
 	l	%r3,__LC_PGM_ILC	 # load program interruption code
 	la	%r8,0x7f
 	nr	%r8,%r3                  # clear per-event-bit and ilc
@@ -536,6 +538,16 @@ pgm_no_vtime3:
 	stosm	__SF_EMPTY(%r15),0x03	# reenable interrupts
 	b	BASED(sysc_do_svc)
 
+#
+# per was called from kernel, must be kprobes
+#
+kernel_per:
+	mvi	SP_TRAP+1(%r15),0x28	# set trap indication to pgm check
+	la	%r2,SP_PTREGS(%r15)	# address of register-save area
+	l	%r1,BASED(.Lhandle_per)	# load adr. of per handler
+	la	%r14,BASED(sysc_leave)	# load adr. of system return
+	br	%r1			# branch to do_single_step
+
 /*
  * IO interrupt handler routine
  */
arch/s390/kernel/kprobes.c +692 -0

692 lines not on this page · raw message

arch/s390/kernel/Makefile +1 -0
diff -Nurp linux-2.6.18-rc1/arch/s390/kernel/Makefile linux-2.6.18-rc1-kp390/arch/s390/kernel/Makefile
@@ -22,6 +22,7 @@ obj-$(CONFIG_BINFMT_ELF32)	+= binfmt_elf
 
 obj-$(CONFIG_VIRT_TIMER)	+= vtime.o
 obj-$(CONFIG_STACKTRACE)	+= stacktrace.o
+obj-$(CONFIG_KPROBES)		+= kprobes.o
 
 # Kexec part
 S390_KEXEC_OBJS := machine_kexec.o crash.o
arch/s390/kernel/traps.c +37 -5

85 lines not on this page · raw message

arch/s390/kernel/vmlinux.lds.S +1 -0
diff -Nurp linux-2.6.18-rc1/arch/s390/kernel/vmlinux.lds.S linux-2.6.18-rc1-kp390/arch/s390/kernel/vmlinux.lds.S
@@ -24,6 +24,7 @@ SECTIONS
 	*(.text)
 	SCHED_TEXT
 	LOCK_TEXT
+	KPROBES_TEXT
 	*(.fixup)
 	*(.gnu.warning)
 	} = 0x0700
arch/s390/mm/fault.c +39 -1
diff -Nurp linux-2.6.18-rc1/arch/s390/mm/fault.c linux-2.6.18-rc1-kp390/arch/s390/mm/fault.c
@@ -25,10 +25,12 @@
 #include <linux/console.h>
 #include <linux/module.h>
 #include <linux/hardirq.h>
+#include <linux/kprobes.h>
 
 #include <asm/system.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>
+#include <asm/kdebug.h>
 
 #ifndef CONFIG_64BIT
 #define __FAIL_ADDR_MASK 0x7ffff000
@@ -48,6 +50,38 @@ extern int sysctl_userprocess_debug;
 
 extern void die(const char *,struct pt_regs *,long);
 
+#ifdef CONFIG_KPROBES
+ATOMIC_NOTIFIER_HEAD(notify_page_fault_chain);
+int register_page_fault_notifier(struct notifier_block *nb)
+{
+	return atomic_notifier_chain_register(&notify_page_fault_chain, nb);
+}
+
+int unregister_page_fault_notifier(struct notifier_block *nb)
+{
+	return atomic_notifier_chain_unregister(&notify_page_fault_chain, nb);
+}
+
+static inline int notify_page_fault(enum die_val val, const char *str,
+			struct pt_regs *regs, long err, int trap, int sig)
+{
+	struct die_args args = {
+		.regs = regs,
+		.str = str,
+		.err = err,
+		.trapnr = trap,
+		.signr = sig
+	};
+	return atomic_notifier_call_chain(&notify_page_fault_chain, val, &args);
+}
+#else
+static inline int notify_page_fault(enum die_val val, const char *str,
+			struct pt_regs *regs, long err, int trap, int sig)
+{
+	return NOTIFY_DONE;
+}
+#endif
+
 extern spinlock_t timerlist_lock;
 
 /*
@@ -159,7 +193,7 @@ static void do_sigsegv(struct pt_regs *r
  *   11       Page translation     ->  Not present       (nullification)
  *   3b       Region third trans.  ->  Not present       (nullification)
  */
-static inline void
+static inline void __kprobes
 do_exception(struct pt_regs *regs, unsigned long error_code, int is_protection)
 {
         struct task_struct *tsk;
@@ -173,6 +207,10 @@ do_exception(struct pt_regs *regs, unsig
         tsk = current;
         mm = tsk->mm;
 	
+	if (notify_page_fault(DIE_PAGE_FAULT, "page fault", regs, error_code, 14,
+					SIGSEGV) == NOTIFY_STOP)
+		return;
+
 	/* 
          * Check for low-address protection.  This needs to be treated
 	 * as a special case because the translation exception code 
include/asm-s390/kdebug.h +57 -0

57 lines not on this page · raw message

include/asm-s390/kprobes.h +111 -0

111 lines not on this page · raw message

#15 lore raw

Re: [PATCH] kprobes for s390 architecture

arch/s390/kernel/kprobes.c line 168

Quoted 13 lines
arch/s390/kernel/kprobes.c +13 -0
+static int __kprobes swap_instruction(void *aref)
+{
+	unsigned long addr, prev, tmp;
+	int shift;
+	struct ins_replace_args *args = aref;
+
+	addr = (unsigned long) args->ptr;
+	shift = (2 ^ (addr & 2)) << 3;
+	addr ^= addr & 2;
+	asm volatile(
+		"    l   %0,0(%4)\n"
+		"    nr  %0,%5\n"
+                "    lr  %1,%0\n"

Whitespace :)

Quoted 25 lines
arch/s390/kernel/kprobes.c +25 -0
+		"    or  %0,%2\n"
+		"    or  %1,%3\n"
+		"0:  cs  %0,%1,0(%4)\n"
+		"    jnl 1f\n"
+		"    xr  %1,%0\n"
+		"    nr  %1,%5\n"
+		"    jnz 0b\n"
+		"1:"
+#ifndef __s390x__
+		".section .fixup,\"ax\"\n"
+		"2: lhi    %0,%6\n"
+		"   bras   1,3f\n"
+		"   .long  1b\n"
+		"3: l      1,0(1)\n"
+		"   br     1\n"
+		".previous\n"
+		".section __ex_table,\"a\"\n"
+		"   .align 4\n"
+		"   .long  0b,2b\n"
+		".previous"
+#else /* __s390x__ */
+		".section .fixup,\"ax\"\n"
+		"2: lghi   %0,%6\n"
+		"   jg     1b\n"
+		".previous\n"

arch/s390/kernel/kprobes.c line 203

Quoted 10 lines
arch/s390/kernel/kprobes.c +10 -0
+		".section __ex_table,\"a\"\n"
+		"   .align 8\n"
+		"   .quad  0b,2b\n"
+		".previous"
+#endif /* __s390x__ */
+		: "=&d" (prev), "=&d" (tmp)
+		: "d" (args->old << shift), "d" (args->new << shift),
+		  "a" (args->ptr), "d" (~(65535 << shift)), "K" (-EFAULT)
+		: "memory", "cc" );
+	return prev >> shift;

You need a label behind the cs instruction and put that into the __ex_table, since the PSW will point to the instruction after cs if it fails.

Also, on failure this function seems to return -EFAULT >> shift, which seems to be wrong.

arch/s390/kernel/traps.c static int kstack_depth_to_print = 12; · line 91

Quoted 2 lines
arch/s390/kernel/traps.c +2 -0
+EXPORT_SYMBOL(register_die_notifier);
+EXPORT_SYMBOL(unregister_die_notifier);

_GPL?

#16 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 1 line · Heiko Carstens

Whitespace :)

d'oh.

Quoted 2 lines

You need a label behind the cs instruction and put that into the __ex_table, since the PSW will point to the instruction after cs if it fails.

Yeah, thought that was a nullify not a terminate. d'oh. d'oh.

Quoted 2 lines

Also, on failure this function seems to return -EFAULT >> shift, which seems to be wrong.

Yeah. I think just returning the value without the shift would be ok. kprobes never checks to see if the instruction swap was successful (which seems even more wrong)

Quoted 3 lines

arch/s390/kernel/traps.c static int kstack_depth_to_print = 12; · line 91

Quoted 2 lines
arch/s390/kernel/traps.c +2 -0
+EXPORT_SYMBOL(register_die_notifier);
+EXPORT_SYMBOL(unregister_die_notifier);

_GPL?

Makes sense, but I kept it consistent with the rest of kprobes.

Signature · 7 lines
Thanks
Mike

=========================================
Michael Grundy - [email protected]

If at first you don't succeed, call in an air strike.
#17 lore raw

Re: [PATCH] kprobes for s390 architecture

In reply to Heiko Carstens
Quoted 2 lines · Heiko Carstens
Quoted 1 line

ok, I tried, but my "better ideas" made things worse. stop_machine_run() wins:

How fast is this if you have to exchange several hundred instructions?

I did a little measuring. On average stop_machine_run() adds 8.7 msec of overhead on a 4-way config. Of that %57 was sub-msec overhead. For the times where overhead was measurable, the average was 20.2 msec, lowest at 10msec highest at 100msec. That's on a z800 under vm and I have no idea how many real cpus the machine has :-)

Signature · 10 lines
Thanks
Mike

=========================================
Michael Grundy - [email protected]
Advanced Linux Response Team (ALRT)
http://ltc.linux.ibm.com/teamweb/alrt/
845-435-8842 (T/L 295)

If at first you don't succeed, call in an air strike.
#18 lore raw

Re: [PATCH] kprobes for s390 architecture

Quoted 5 lines · Mike Grundy

I did a little measuring. On average stop_machine_run() adds 8.7 msec of overhead on a 4-way config. Of that %57 was sub-msec overhead. For the times where overhead was measurable, the average was 20.2 msec, lowest at 10msec highest at 100msec. That's on a z800 under vm and I have no idea how many real cpus the machine has :-)

So adding 100 probes will roughly take 1 second. Not too bad, I expected it to take longer.

Signature · 8 lines
blue skies,
  Martin.

Martin Schwidefsky
Linux for zSeries Development & Services
IBM Deutschland Entwicklung GmbH

"Reality continues to ruin my life." - Calvin.