Skip to contentSkip to conversation

: Sparc64 immediate values

PATCH In review David Miller 2 participants 4 messages May 13, 2008 – Jun 19, 2008

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 : Sparc64 immediate values

As described in the commit log I think that two instruction sequences can be done properly, and we'd need that to support 16-bit and 32-bit values on sparc64 since the available instructions are "load high 22-bits" and "or in signed low 13 bits".

One idea is to capture all cpus other than the immediate value updating cpu.

These other cpus look to see if their program counter falls on an immediate value instruction sequence. Much like how we lookup exceptions we can sort the table and use binary search so that it isn't too slow.

If they find themselves in such a sequence, they examine the destination register in the last instruction, load that register with the proper immediate value, and advance the program counter. Then they wait to be released.

The updater does the instruction update unimpeded, flushes the instruction cache or whatever needs to be done, and then releases the other cpus.

PowerPC could use this scheme too, if it does in fact work.

Please add to the ftrace tree, thanks.

commit f2b14974b823a9cd9b6f5c0d423945caa15de8a2

Author: David S. Miller <[email protected]>
Date:   Tue May 13 04:29:30 2008 -0700

sparc64: Optimized immediate value implementation.

We can only do byte sized values currently.

In order to support even 16-bit immediates we would need a 2 instruction sequence.

I believe that can be made to work with a suitable breakpoint or some other kind of special patching sequence, but that isn't attempted here.

arch/sparc64/Kconfig +1 -0
index eb36f3b..4c40862 100644
@@ -14,6 +14,7 @@ config SPARC64
 	select HAVE_IDE
 	select HAVE_LMB
 	select HAVE_ARCH_KGDB
+	select HAVE_IMMEDIATE
 
 config GENERIC_TIME
 	bool
arch/sparc64/kernel/Makefile +1 -0
index ec4f5eb..311c797 100644
@@ -25,6 +25,7 @@ obj-$(CONFIG_MODULES) += module.o
 obj-$(CONFIG_US3_FREQ) += us3_cpufreq.o
 obj-$(CONFIG_US2E_FREQ) += us2e_cpufreq.o
 obj-$(CONFIG_KPROBES) += kprobes.o
+obj-$(CONFIG_IMMEDIATE) += immediate.o
 obj-$(CONFIG_SUN_LDOMS) += ldc.o vio.o viohs.o ds.o
 obj-$(CONFIG_AUDIT) += audit.o
 obj-$(CONFIG_AUDIT)$(CONFIG_COMPAT) += compat_audit.o
arch/sparc64/kernel/immediate.c new file +48 -0

48 lines not on this page · raw message

include/asm-sparc64/immediate.h new file +37 -0
new file mode 100644
index 0000000..3673afd
@@ -0,0 +1,37 @@
+#ifndef _ASM_SPARC64_IMMEDIATE_H
+#define _ASM_SPARC64_IMMEDIATE_H
+
+struct __imv {
+	unsigned int var;
+	unsigned int imv;
+	unsigned char size;
+} __attribute__ ((packed));
+
+#define imv_read(name)							\
+	({								\
+		__typeof__(name##__imv) value;				\
+		BUILD_BUG_ON(sizeof(value) > 8);			\
+		switch (sizeof(value)) {				\
+		case 1:							\
+			asm(".section __imv,\"aw\",@progbits\n\t"	\
+					".uaword %c1, 1f\n\t"		\
+					".byte 1\n\t"			\
+					".previous\n\t"			\
+					"1: mov 0, %0\n\t"		\
+				: "=r" (value)				\
+				: "i" (&name##__imv));			\
+			break;						\
+		case 2:							\
+		case 4:							\
+		case 8:	value = name##__imv;				\
+			break;						\
+		};							\
+		value;							\
+	})
+
+#define imv_cond(name)	imv_read(name)
+#define imv_cond_end()
+
+extern int arch_imv_update(const struct __imv *imv, int early);
+
+#endif /* _ASM_SPARC64_IMMEDIATE_H */
kernel/immediate.c +2 -2
index 3668a7f..1b4c5cf 100644
@@ -62,8 +62,8 @@ void imv_update_range(struct __imv *begin,
 				"Invalid immediate value. "
 				"Variable at %p, "
 				"instruction at %p, size %hu\n",
-				(void *)iter->imv,
-				(void *)iter->var, iter->size);
+			        (void *)(long)iter->imv,
+			        (void *)(long)iter->var, iter->size);
 skip:
 		mutex_unlock(&imv_mutex);
 	}
Acked-by on v1 1/1
#1 lore raw
Quoted 19 lines · * David Miller ([email protected]) wrote:

As described in the commit log I think that two instruction sequences can be done properly, and we'd need that to support 16-bit and 32-bit values on sparc64 since the available instructions are "load high 22-bits" and "or in signed low 13 bits".

One idea is to capture all cpus other than the immediate value updating cpu.

These other cpus look to see if their program counter falls on an immediate value instruction sequence. Much like how we lookup exceptions we can sort the table and use binary search so that it isn't too slow.

If they find themselves in such a sequence, they examine the destination register in the last instruction, load that register with the proper immediate value, and advance the program counter. Then they wait to be released.

Quoted 6 lines

The updater does the instruction update unimpeded, flushes the instruction cache or whatever needs to be done, and then releases the other cpus.

PowerPC could use this scheme too, if it does in fact work.

I did use a "stop_machine" with interrupts disabled implementation in the "simple immediate values version". It is really heavy; it busy-loops all cpus with interrupts disabled except one while the immediate value is updated. This is required so we stop interrupt contexts from executing those instructions. However, it does not protect from having a thread preempted in the middle of this instruction sequence and therefore to see incoherent values. Are there non-maskable interrupts on sparc64 ? That would make the stop_machine approach a bit more fragile against this execution context.

The one thing we could do to allow such updates without using stop_machine is to create something similar to the read seqlock using immediate values.

In this case, the read-side would look like (pseudo-code) :

1:
mov 22-bits MSB-lock to r2 (1)
mov 13-bits LSB to r1 low (2)
mov 22-bits MSB to r1 high (3)
mov 22-bits MSB-lock to r3 (4)
compare r2,r3 (5)
jne 1b

Write side :

Increment MSB-lock value in instruction (1)
smp_wmb()
update instructions 2 and 3
smp_wmb()
Increment MSB-lock value in instruction (2)

We would then have a 22-bits detection power of updates done while a context is interrupted or preempted. It means that, in the worse case, we could have a kernel thread preempted right in the middle of (2) and (3), which comes back after exactly 2^22 write-side executions and thinks the lock is correct when it actually isn't. We have the same problem with the read seqlock anyway. One way to fix this would be to put all threads in the freezer, which is a really-really heavy lock, each time the MSB-lock counter reaches overflow. The performance impact of this approach should be carefully considered though : added branch, added instruction cache impact..

If it makes sense, we could put all threads in the freezer and use stop_machine, but I guess it's a bit heavy. In any case, I would like the 8-bit immediate value update not to depend on such heavy lock and to support nmi-like contexts.

Quoted 2 lines

Please add to the ftrace tree, thanks.

The following patch looks good! Although I have not looked at the sparc64 instruction set details, it follows the general immediate values mindset.

Quoted 153 lines

commit f2b14974b823a9cd9b6f5c0d423945caa15de8a2

Author: David S. Miller <[email protected]>
Date:   Tue May 13 04:29:30 2008 -0700

sparc64: Optimized immediate value implementation.

We can only do byte sized values currently.

In order to support even 16-bit immediates we would need a 2 instruction sequence.

I believe that can be made to work with a suitable breakpoint or some other kind of special patching sequence, but that isn't attempted here.

arch/sparc64/Kconfig +1 -0
index eb36f3b..4c40862 100644
@@ -14,6 +14,7 @@ config SPARC64
 	select HAVE_IDE
 	select HAVE_LMB
 	select HAVE_ARCH_KGDB
+	select HAVE_IMMEDIATE
 
 config GENERIC_TIME
 	bool
arch/sparc64/kernel/Makefile +1 -0
index ec4f5eb..311c797 100644
@@ -25,6 +25,7 @@ obj-$(CONFIG_MODULES) += module.o
 obj-$(CONFIG_US3_FREQ) += us3_cpufreq.o
 obj-$(CONFIG_US2E_FREQ) += us2e_cpufreq.o
 obj-$(CONFIG_KPROBES) += kprobes.o
+obj-$(CONFIG_IMMEDIATE) += immediate.o
 obj-$(CONFIG_SUN_LDOMS) += ldc.o vio.o viohs.o ds.o
 obj-$(CONFIG_AUDIT) += audit.o
 obj-$(CONFIG_AUDIT)$(CONFIG_COMPAT) += compat_audit.o
arch/sparc64/kernel/immediate.c new file +48 -0
new file mode 100644
index 0000000..be76f28
@@ -0,0 +1,48 @@
+#include <linux/module.h>
+#include <linux/immediate.h>
+#include <linux/string.h>
+#include <linux/kprobes.h>
+
+#include <asm/system.h>
+
+int arch_imv_update(const struct __imv *imv, int early)
+{
+	unsigned long imv_vaddr = imv->imv;
+	unsigned long var_vaddr = imv->var;
+	u32 insn, *ip = (u32 *) imv_vaddr;
+
+	insn = *ip;
+
+#ifdef CONFIG_KPROBES
+	switch (imv->size) {
+	case 1:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	if (unlikely(!early &&
+		     (insn == BREAKPOINT_INSTRUCTION ||
+		      insn == BREAKPOINT_INSTRUCTION_2))) {
+		printk(KERN_WARNING "Immediate value in conflict with kprobe. "
+				    "Variable at %p, "
+				    "instruction at %p, size %u\n",
+				    ip, (void *)var_vaddr, imv->size);
+		return -EBUSY;
+	}
+#endif
+
+	switch (imv->size) {
+	case 1:
+		if ((insn & 0x1fff) == *(uint8_t *)var_vaddr)
+			return 0;
+		insn &= ~0x00001fff;
+		insn |= (u32) (*(uint8_t *)var_vaddr);
+		break;
+	default:
+		return -EINVAL;
+	}
+	*ip = insn;
+	flushi(ip);
+	return 0;
+}
include/asm-sparc64/immediate.h new file +37 -0
new file mode 100644
index 0000000..3673afd
@@ -0,0 +1,37 @@
+#ifndef _ASM_SPARC64_IMMEDIATE_H
+#define _ASM_SPARC64_IMMEDIATE_H
+
+struct __imv {
+	unsigned int var;
+	unsigned int imv;
+	unsigned char size;
+} __attribute__ ((packed));
+
+#define imv_read(name)							\
+	({								\
+		__typeof__(name##__imv) value;				\
+		BUILD_BUG_ON(sizeof(value) > 8);			\
+		switch (sizeof(value)) {				\
+		case 1:							\
+			asm(".section __imv,\"aw\",@progbits\n\t"	\
+					".uaword %c1, 1f\n\t"		\
+					".byte 1\n\t"			\
+					".previous\n\t"			\
+					"1: mov 0, %0\n\t"		\
+				: "=r" (value)				\
+				: "i" (&name##__imv));			\
+			break;						\
+		case 2:							\
+		case 4:							\
+		case 8:	value = name##__imv;				\
+			break;						\
+		};							\
+		value;							\
+	})
+
+#define imv_cond(name)	imv_read(name)
+#define imv_cond_end()
+
+extern int arch_imv_update(const struct __imv *imv, int early);
+
+#endif /* _ASM_SPARC64_IMMEDIATE_H */
kernel/immediate.c +2 -2
index 3668a7f..1b4c5cf 100644
@@ -62,8 +62,8 @@ void imv_update_range(struct __imv *begin,
 				"Invalid immediate value. "
 				"Variable at %p, "
 				"instruction at %p, size %hu\n",
-				(void *)iter->imv,
-				(void *)iter->var, iter->size);
+			        (void *)(long)iter->imv,
+			        (void *)(long)iter->var, iter->size);
 skip:
 		mutex_unlock(&imv_mutex);
 	}
Signature · 2 lines
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68
#2 lore raw

From: Mathieu Desnoyers <[email protected]>
Date: Tue, 13 May 2008 08:36:15 -0400

Quoted 3 lines

However, it does not protect from having a thread preempted in the middle of this instruction sequence and therefore to see incoherent values.

Yes, that makes such schemes unworkable, how hum...

Quoted 1 line

Are there non-maskable interrupts on sparc64 ?

Yes, and no. When the PSTATE_IE bit is cleared in the processor state register, no interrupts whatsoever are recognized by the processor. This is off only during trap entry/exit sequences, and some other special bits of code.

Quoted 3 lines

The one thing we could do to allow such updates without using stop_machine is to create something similar to the read seqlock using immediate values.

Yes I saw such suggestions in the comments of the immediate code, you don't have to describe such things all over again.

Doing something so heavy like this in the "fast path" is completely pointless in my opinion.

Better to keep brainstorming on a scheme that works without adding any instructions to the immediate load sequence. If you add instructions, fetching the instructions themselves become just as expensive, if not moreso, than the load we are eliminating.

#3 lore raw

Hi David,

I'm picking this patch up in my LTTng patchset for testing.

Thanks,

Mathieu

Quoted 31 lines · * David Miller ([email protected]) wrote:

From: Mathieu Desnoyers <[email protected]>
Date: Tue, 13 May 2008 08:36:15 -0400

Quoted 3 lines

However, it does not protect from having a thread preempted in the middle of this instruction sequence and therefore to see incoherent values.

Yes, that makes such schemes unworkable, how hum...

Quoted 1 line

Are there non-maskable interrupts on sparc64 ?

Yes, and no. When the PSTATE_IE bit is cleared in the processor state register, no interrupts whatsoever are recognized by the processor. This is off only during trap entry/exit sequences, and some other special bits of code.

Quoted 3 lines

The one thing we could do to allow such updates without using stop_machine is to create something similar to the read seqlock using immediate values.

Yes I saw such suggestions in the comments of the immediate code, you don't have to describe such things all over again.

Doing something so heavy like this in the "fast path" is completely pointless in my opinion.

Better to keep brainstorming on a scheme that works without adding any instructions to the immediate load sequence. If you add instructions, fetching the instructions themselves become just as expensive, if not moreso, than the load we are eliminating.

Signature · 2 lines
Mathieu Desnoyers
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68