Skip to contentSkip to conversation

powerpc: Hardware transactional memory support for POWER8

PATCH New Michael Neuling 2 participants 77 messages Nov 27, 2012 – Feb 14, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover powerpc: Hardware transactional memory support for POWER8

POWER8 implements hardware transactional memory support. This patch series adds kernel support so that user programs can use this hardware transactional memory and the new state is properly context switched. It is not currently used by the kernel itself.

This patch series was originally developed by Matt Evans.

Basic overview of a POWER8 hardware transaction memory
=====================================================
Hardware transactional memory is a feature that enables a different form of atomic memory access. Several new instructions are presented to delimit transactions; transactions are guaranteed to either complete atomically or roll back and undo any partial changes.

A simple transaction looks like this:

begin_move_money:
tbegin
beq abort_handler

  ld    r4, SAVINGS_ACCT(r3)
  ld    r5, CURRENT_ACCT(r3)
  subi  r5, r5, 1
  addi  r4, r4, 1
  std   r4, SAVINGS_ACCT(r3)
  std   r5, CURRENT_ACCT(r3)

tend

b continue

abort_handler:
... test for odd failures ...

/* Retry the transaction if it failed because it conflicted with
* someone else: */
b begin_move_money

The 'tbegin' instruction denotes the start point, and 'tend' the end point. Between these points the processor is in 'Transactional' state; any memory references will complete in one go if there are no conflicts with other transactional or non-transactional accesses within the system. In this example, the transaction completes as though it were normal straight-line code IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an atomic move of money from the current account to the savings account has been performed. Even though the normal ld/std instructions are used (note no lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be updated, or neither will be updated.

If, in the meantime, there is a conflict with the locations accessed by the transaction, the transaction will be aborted by the CPU. Register and memory state will roll back to that at the 'tbegin', and control will continue from 'tbegin+4'. The branch to abort_handler will be taken this second time; the abort handler can check the cause of the failure, and retry.

Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR and a few other status/flag regs;

Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/16 powerpc: Add new CPU feature bit for transactional memory

---
 arch/powerpc/include/asm/cputable.h |    8 ++++++++
 1 file changed, 8 insertions(+)
arch/powerpc/include/asm/cputable.h +8 -0
index fc4d2c5..f2163da 100644
@@ -171,6 +171,7 @@ extern const char *powerpc_base_platform;
 #define CPU_FTR_POPCNTD			LONG_ASM_CONST(0x0800000000000000)
 #define CPU_FTR_ICSWX			LONG_ASM_CONST(0x1000000000000000)
 #define CPU_FTR_VMX_COPY		LONG_ASM_CONST(0x2000000000000000)
+#define CPU_FTR_TM			LONG_ASM_CONST(0x4000000000000000)
 
 #ifndef __ASSEMBLY__
 
@@ -216,6 +217,13 @@ extern const char *powerpc_base_platform;
 #define PPC_FEATURE_HAS_EFP_DOUBLE_COMP 0
 #endif
 
+/* We only set the TM feature if the kernel was compiled with TM supprt */
+#ifdef CONFIG_TRANSACTIONAL_MEM
+#define CPU_FTR_TM_COMP		CPU_FTR_TM
+#else
+#define CPU_FTR_TM_COMP		0
+#endif
+
 /* We need to mark all pages as being coherent if we're SMP or we have a
  * 74[45]x and an MPC107 host bridge. Also 83xx and PowerQUICC II
  * require it for PCI "streaming/prefetch" to work properly.
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH 2/16 powerpc: Add new instructions for transactional memory

In reply to Michael Neuling

Here we define the new instructions we need for transactional memory in the kernel. This is so we can support compiling with binutils that don't support the new transactional memory instructions.

Transactional memory results in two sets of architected state (GPRs/VSRs etc).

treclaim allows us to read the checkpointed state (from the tbegin) so that we can store it away on a context switch. It does this by overwriting the exiting architected state, so you have to save that away before you treclaim. treclaim will also abort a transaction, so you can give a register value which contains an abort reason.

trecheckpoint allows us to inject into the checkpointed state as if it were at the tbegin. It does this by copying the current architected state into the checkpointed state.

---
 arch/powerpc/include/asm/ppc-opcode.h |    7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/include/asm/ppc-opcode.h +7 -0
index 45fd394..3674ffc 100644
@@ -129,6 +129,8 @@
 #define PPC_INST_TLBSRX_DOT		0x7c0006a5
 #define PPC_INST_XXLOR			0xf0000510
 #define PPC_INST_XVCPSGNDP		0xf0000780
+#define PPC_INST_TRECHKPT		0x7c0007dd
+#define PPC_INST_TRECLAIM		0x7c00075d
 
 #define PPC_INST_NAP			0x4c000364
 #define PPC_INST_SLEEP			0x4c0003a4
@@ -291,4 +293,9 @@
 #define PPC_NAP			stringify_in_c(.long PPC_INST_NAP)
 #define PPC_SLEEP		stringify_in_c(.long PPC_INST_SLEEP)
 
+/* Transactional memory instructions */
+#define TRECHKPT		stringify_in_c(.long PPC_INST_TRECHKPT)
+#define TRECLAIM(r)		stringify_in_c(.long PPC_INST_TRECLAIM \
+					       | __PPC_RA(r))
+
 #endif /* _ASM_POWERPC_PPC_OPCODE_H */
Signature · 1 line
1.7.9.5
#3 lore raw

PATCH 3/16 powerpc: Add additional state needed for transactional memory to thread struct

In reply to Michael Neuling

Set of new archtected state for saving away on context switch.

---
 arch/powerpc/include/asm/processor.h |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
arch/powerpc/include/asm/processor.h +28 -0
index 8750204..0d1c188 100644
@@ -236,6 +236,34 @@ struct thread_struct {
 	unsigned long	spefscr;	/* SPE & eFP status */
 	int		used_spe;	/* set if process has used spe */
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	u64		tm_tfhar;	/* Transaction fail handler addr */
+	u64		tm_texasr;	/* Transaction exception & summary */
+	u64		tm_tfiar;	/* Transaction fail instr address reg */
+	unsigned long	tm_orig_msr;	/* Thread's MSR on ctx switch */
+	struct pt_regs	ckpt_regs;	/* Checkpointed registers */
+
+	/*
+	 * Transactional FP and VSX 0-31 register set.
+	 * NOTE: the sense of these is the opposite of the integer ckpt_regs!
+	 *
+	 * When a transaction is active/signalled/scheduled etc., *regs is the
+	 * most recent set of/speculated GPRs with ckpt_regs being the older
+	 * checkpointed regs to which we roll back if transaction aborts.
+	 *
+	 * However, fpr[] is the checkpointed 'base state' of FP regs, and
+	 * transact_fpr[] is the new set of transactional values.
+	 * VRs work the same way.
+	 */
+	double		transact_fpr[32][TS_FPRWIDTH];
+	struct {
+		unsigned int pad;
+		unsigned int val;	/* Floating point status */
+	} transact_fpscr;
+	vector128	transact_vr[32] __attribute__((aligned(16)));
+	vector128	transact_vscr __attribute__((aligned(16)));
+	unsigned long	transact_vrsave;
+#endif /* CONFIG_TRANSACTIONAL_MEM */
 #ifdef CONFIG_KVM_BOOK3S_32_HANDLER
 	void*		kvm_shadow_vcpu; /* KVM internal data */
 #endif /* CONFIG_KVM_BOOK3S_32_HANDLER */
Signature · 1 line
1.7.9.5
#4 lore raw

PATCH 4/16 powerpc: New macros for transactional memory support

In reply to Michael Neuling

This adds new macros for saving and restoring checkpointed architected state from and to the thread_struct.

It also adds some debugging macros for when your brain explodes trying to debug your transactional memory enabled kernel.

---
 arch/powerpc/include/asm/ppc_asm.h   |   83 ++++++++++++++++++++++++++++++++++
 arch/powerpc/include/asm/processor.h |    1 +
 arch/powerpc/kernel/asm-offsets.c    |   24 ++++++++++
 arch/powerpc/kernel/fpu.S            |   12 +++++
 arch/powerpc/kernel/process.c        |   10 ++++
 arch/powerpc/kernel/traps.c          |   11 +++++
 6 files changed, 141 insertions(+)
arch/powerpc/include/asm/ppc_asm.h +83 -0
index ea2a86e..a17a598 100644
@@ -125,6 +125,89 @@ END_FW_FTR_SECTION_IFSET(FW_FEATURE_SPLPAR)
 #define REST_16VRS(n,b,base)	REST_8VRS(n,b,base); REST_8VRS(n+8,b,base)
 #define REST_32VRS(n,b,base)	REST_16VRS(n,b,base); REST_16VRS(n+16,b,base)
 
+/* Save/restore FPRs, VRs and VSRs from their checkpointed backups in
+ * thread_struct:
+ */
+#define SAVE_FPR_TRANSACT(n, base)	stfd n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define SAVE_2FPRS_TRANSACT(n, base)	SAVE_FPR_TRANSACT(n, base);	\
+					SAVE_FPR_TRANSACT(n+1, base)
+#define SAVE_4FPRS_TRANSACT(n, base)	SAVE_2FPRS_TRANSACT(n, base);	\
+					SAVE_2FPRS_TRANSACT(n+2, base)
+#define SAVE_8FPRS_TRANSACT(n, base)	SAVE_4FPRS_TRANSACT(n, base);	\
+					SAVE_4FPRS_TRANSACT(n+4, base)
+#define SAVE_16FPRS_TRANSACT(n, base)	SAVE_8FPRS_TRANSACT(n, base);	\
+					SAVE_8FPRS_TRANSACT(n+8, base)
+#define SAVE_32FPRS_TRANSACT(n, base)	SAVE_16FPRS_TRANSACT(n, base);	\
+					SAVE_16FPRS_TRANSACT(n+16, base)
+
+#define REST_FPR_TRANSACT(n, base)	lfd	n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define REST_2FPRS_TRANSACT(n, base)	REST_FPR_TRANSACT(n, base);	\
+					REST_FPR_TRANSACT(n+1, base)
+#define REST_4FPRS_TRANSACT(n, base)	REST_2FPRS_TRANSACT(n, base);	\
+					REST_2FPRS_TRANSACT(n+2, base)
+#define REST_8FPRS_TRANSACT(n, base)	REST_4FPRS_TRANSACT(n, base);	\
+					REST_4FPRS_TRANSACT(n+4, base)
+#define REST_16FPRS_TRANSACT(n, base)	REST_8FPRS_TRANSACT(n, base);	\
+					REST_8FPRS_TRANSACT(n+8, base)
+#define REST_32FPRS_TRANSACT(n, base)	REST_16FPRS_TRANSACT(n, base);	\
+					REST_16FPRS_TRANSACT(n+16, base)
+
+
+#define SAVE_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					stvx n,b,base
+#define SAVE_2VRS_TRANSACT(n,b,base)	SAVE_VR_TRANSACT(n,b,base);	\
+					SAVE_VR_TRANSACT(n+1,b,base)
+#define SAVE_4VRS_TRANSACT(n,b,base)	SAVE_2VRS_TRANSACT(n,b,base);	\
+					SAVE_2VRS_TRANSACT(n+2,b,base)
+#define SAVE_8VRS_TRANSACT(n,b,base)	SAVE_4VRS_TRANSACT(n,b,base);	\
+					SAVE_4VRS_TRANSACT(n+4,b,base)
+#define SAVE_16VRS_TRANSACT(n,b,base)	SAVE_8VRS_TRANSACT(n,b,base);	\
+					SAVE_8VRS_TRANSACT(n+8,b,base)
+#define SAVE_32VRS_TRANSACT(n,b,base)	SAVE_16VRS_TRANSACT(n,b,base);	\
+					SAVE_16VRS_TRANSACT(n+16,b,base)
+
+#define REST_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					lvx n,b,base
+#define REST_2VRS_TRANSACT(n,b,base)	REST_VR_TRANSACT(n,b,base);	\
+					REST_VR_TRANSACT(n+1,b,base)
+#define REST_4VRS_TRANSACT(n,b,base)	REST_2VRS_TRANSACT(n,b,base);	\
+					REST_2VRS_TRANSACT(n+2,b,base)
+#define REST_8VRS_TRANSACT(n,b,base)	REST_4VRS_TRANSACT(n,b,base);	\
+					REST_4VRS_TRANSACT(n+4,b,base)
+#define REST_16VRS_TRANSACT(n,b,base)	REST_8VRS_TRANSACT(n,b,base);	\
+					REST_8VRS_TRANSACT(n+8,b,base)
+#define REST_32VRS_TRANSACT(n,b,base)	REST_16VRS_TRANSACT(n,b,base);	\
+					REST_16VRS_TRANSACT(n+16,b,base)
+
+
+#define SAVE_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					STXVD2X(n,R##base,R##b)
+#define SAVE_2VSRS_TRANSACT(n,b,base)	SAVE_VSR_TRANSACT(n,b,base);	\
+	                                SAVE_VSR_TRANSACT(n+1,b,base)
+#define SAVE_4VSRS_TRANSACT(n,b,base)	SAVE_2VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_2VSRS_TRANSACT(n+2,b,base)
+#define SAVE_8VSRS_TRANSACT(n,b,base)	SAVE_4VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_4VSRS_TRANSACT(n+4,b,base)
+#define SAVE_16VSRS_TRANSACT(n,b,base)	SAVE_8VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_8VSRS_TRANSACT(n+8,b,base)
+#define SAVE_32VSRS_TRANSACT(n,b,base)	SAVE_16VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_16VSRS_TRANSACT(n+16,b,base)
+
+#define REST_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					LXVD2X(n,R##base,R##b)
+#define REST_2VSRS_TRANSACT(n,b,base)	REST_VSR_TRANSACT(n,b,base);    \
+	                                REST_VSR_TRANSACT(n+1,b,base)
+#define REST_4VSRS_TRANSACT(n,b,base)	REST_2VSRS_TRANSACT(n,b,base);	\
+	                                REST_2VSRS_TRANSACT(n+2,b,base)
+#define REST_8VSRS_TRANSACT(n,b,base)	REST_4VSRS_TRANSACT(n,b,base);	\
+	                                REST_4VSRS_TRANSACT(n+4,b,base)
+#define REST_16VSRS_TRANSACT(n,b,base)	REST_8VSRS_TRANSACT(n,b,base);	\
+	                                REST_8VSRS_TRANSACT(n+8,b,base)
+#define REST_32VSRS_TRANSACT(n,b,base)	REST_16VSRS_TRANSACT(n,b,base);	\
+	                                REST_16VSRS_TRANSACT(n+16,b,base)
+
 /* Save the lower 32 VSRs in the thread VSR region */
 #define SAVE_VSR(n,b,base)	li b,THREAD_VSR0+(16*(n));  STXVD2X(n,R##base,R##b)
 #define SAVE_2VSRS(n,b,base)	SAVE_VSR(n,b,base); SAVE_VSR(n+1,b,base)
arch/powerpc/include/asm/processor.h +1 -0
index 0d1c188..8b2bf7a 100644
@@ -141,6 +141,7 @@ typedef struct {
 #define TS_FPROFFSET 0
 #define TS_VSRLOWOFFSET 1
 #define TS_FPR(i) fpr[i][TS_FPROFFSET]
+#define TS_TRANS_FPR(i) transact_fpr[i][TS_FPROFFSET]
 
 struct thread_struct {
 	unsigned long	ksp;		/* Kernel stack pointer */
arch/powerpc/kernel/asm-offsets.c +24 -0
index 7523539..1a70f02 100644
@@ -121,6 +121,30 @@ int main(void)
 	DEFINE(THREAD_KVM_VCPU, offsetof(struct thread_struct, kvm_vcpu));
 #endif
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
+	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
+	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
+	DEFINE(PT_CKPT_REGS, offsetof(struct thread_struct, ckpt_regs));
+	DEFINE(THREAD_TRANSACT_VR0, offsetof(struct thread_struct,
+					 transact_vr[0]));
+	DEFINE(THREAD_TRANSACT_VSCR, offsetof(struct thread_struct,
+					  transact_vscr));
+	DEFINE(THREAD_TRANSACT_VRSAVE, offsetof(struct thread_struct,
+					    transact_vrsave));
+	DEFINE(THREAD_TRANSACT_FPR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+	DEFINE(THREAD_TRANSACT_FPSCR, offsetof(struct thread_struct,
+					   transact_fpscr));
+#ifdef CONFIG_VSX
+	DEFINE(THREAD_TRANSACT_VSR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+#endif
+	/* Local pt_regs on stack for Transactional Memory funcs. */
+	DEFINE(TM_FRAME_SIZE, STACK_FRAME_OVERHEAD +
+	       sizeof(struct pt_regs) + 16);
+#endif /* CONFIG_TRANSACTIONAL_MEM */
+
 	DEFINE(TI_FLAGS, offsetof(struct thread_info, flags));
 	DEFINE(TI_LOCAL_FLAGS, offsetof(struct thread_info, local_flags));
 	DEFINE(TI_PREEMPT, offsetof(struct thread_info, preempt_count));
arch/powerpc/kernel/fpu.S +12 -0
index e0ada05..adb1551 100644
@@ -35,6 +35,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 2:	REST_32VSRS(n,c,base);						\
 3:
 
+#define __REST_32FPVSRS_TRANSACT(n,c,base)				\
+BEGIN_FTR_SECTION							\
+	b	2f;							\
+END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
+	REST_32FPRS_TRANSACT(n,base);					\
+	b	3f;							\
+2:	REST_32VSRS_TRANSACT(n,c,base);					\
+3:
+
 #define __SAVE_32FPVSRS(n,c,base)					\
 BEGIN_FTR_SECTION							\
 	b	2f;							\
@@ -45,9 +54,12 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 3:
 #else
 #define __REST_32FPVSRS(n,b,base)	REST_32FPRS(n, base)
+#define __REST_32FPVSRS_TRANSACT(n,b,base)	REST_32FPRS(n, base)
 #define __SAVE_32FPVSRS(n,b,base)	SAVE_32FPRS(n, base)
 #endif
 #define REST_32FPVSRS(n,c,base) __REST_32FPVSRS(n,__REG_##c,__REG_##base)
+#define REST_32FPVSRS_TRANSACT(n,c,base) \
+	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
 /*
arch/powerpc/kernel/process.c +10 -0
index ba48233..6d66a68 100644
@@ -57,6 +57,16 @@
 #include <linux/kprobes.h>
 #include <linux/kdebug.h>
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+/* Transactional Memory debug */
+// #define TM_DEBUG_SW
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+#endif
+
 extern unsigned long _get_SP(void);
 
 #ifndef CONFIG_SMP
arch/powerpc/kernel/traps.c +11 -0
index 3251840..8fed874 100644
@@ -78,6 +78,16 @@ EXPORT_SYMBOL(__debugger_dabr_match);
 EXPORT_SYMBOL(__debugger_fault_handler);
 #endif
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+/* Transactional Memory trap debug */
+// #define TM_DEBUG_SW
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+#endif
+
 /*
  * Trap & Exception support
  */
@@ -350,6 +360,7 @@ static inline int check_io_access(struct pt_regs *regs)
    exception is in the MSR. */
 #define get_reason(regs)	((regs)->msr)
 #define get_mc_reason(regs)	((regs)->msr)
+#define REASON_TM		0x200000
 #define REASON_FP		0x100000
 #define REASON_ILLEGAL		0x80000
 #define REASON_PRIVILEGED	0x40000
Signature · 1 line
1.7.9.5
#5 lore raw

PATCH 5/16 powerpc: Register defines for various transactional memory registers

In reply to Michael Neuling

Defines for MSR bits and transactional memory related SPRs TFIAR, TEXASR and TEXASRU.

---
 arch/powerpc/include/asm/reg.h |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
arch/powerpc/include/asm/reg.h +21 -0
index 736c6af..bea823f 100644
@@ -29,6 +29,8 @@
 #define MSR_SF_LG	63              /* Enable 64 bit mode */
 #define MSR_ISF_LG	61              /* Interrupt 64b mode valid on 630 */
 #define MSR_HV_LG 	60              /* Hypervisor state */
+#define MSR_TS_LG	33		/* Transactional Mem State (2 bits) */
+#define MSR_TM_LG	32		/* Transactional Mem Available */
 #define MSR_VEC_LG	25	        /* Enable AltiVec */
 #define MSR_VSX_LG	23		/* Enable VSX */
 #define MSR_POW_LG	18		/* Enable Power Management */
@@ -98,6 +100,21 @@
 #define MSR_RI		__MASK(MSR_RI_LG)	/* Recoverable Exception */
 #define MSR_LE		__MASK(MSR_LE_LG)	/* Little Endian */
 
+#define MSR_TM	        __MASK(MSR_TM_LG)	/* Transactional Mem Available */
+#define MSR_TS_MASK     (__MASK(MSR_TS_LG) | \
+		         __MASK(MSR_TS_LG+1))   /* Transaction State bits */
+#define MSR_TM_ACTIVE(x) (((x) & MSR_TS_MASK) != 0) /* Transaction active? */
+
+/* Reason codes describing kernel causes for transaction aborts.  By
+   convention, bit0 is copied to TEXASR[56] (IBM bit 7) which is set if
+   the failure is persistent.
+*/
+#define TM_CAUSE_RESCHED	0xfe
+#define TM_CAUSE_TLBI		0xfc
+#define TM_CAUSE_FAC_UNAV	0xfa
+#define TM_CAUSE_SYSCALL	0xf9 /* Persistent */
+#define TM_CAUSE_MISC		0xf6
+
 #if defined(CONFIG_PPC_BOOK3S_64)
 #define MSR_64BIT	MSR_SF
 
@@ -193,6 +210,10 @@
 #define SPRN_UAMOR	0x9d	/* User Authority Mask Override Register */
 #define SPRN_AMOR	0x15d	/* Authority Mask Override Register */
 #define SPRN_ACOP	0x1F	/* Available Coprocessor Register */
+#define SPRN_TFIAR	0x81	/* Transaction Failure Inst Addr   */
+#define SPRN_TEXASR	0x82	/* Transaction EXception & Summary */
+#define SPRN_TEXASRU	0x83	/* ''	   ''	   ''	 Upper 32  */
+#define SPRN_TFHAR	0x80	/* Transaction Failure Handler Addr */
 #define SPRN_CTRLF	0x088
 #define SPRN_CTRLT	0x098
 #define   CTRL_CT	0xc0000000	/* current thread */
Signature · 1 line
1.7.9.5
#6 lore raw

PATCH 6/16 powerpc: Add transactional memory paca scratch register to show_regs

In reply to Michael Neuling

Add transactional memory paca scratch register to show_regs. This is useful for debugging.

---
 arch/powerpc/include/asm/paca.h   |    1 +
 arch/powerpc/kernel/asm-offsets.c |    1 +
 arch/powerpc/kernel/entry_64.S    |    4 ++++
 arch/powerpc/kernel/process.c     |    3 +++
 4 files changed, 9 insertions(+)
arch/powerpc/include/asm/paca.h +1 -0
index e9e7a69..0168516 100644
@@ -137,6 +137,7 @@ struct paca_struct {
 	u8 irq_work_pending;		/* IRQ_WORK interrupt while soft-disable */
 	u8 nap_state_lost;		/* NV GPR values lost in power7_idle */
 	u64 sprg3;			/* Saved user-visible sprg */
+	u64 tm_scratch;                 /* TM scratch area for reclaim */
 
 #ifdef CONFIG_PPC_POWERNV
 	/* Pointer to OPAL machine check event structure set by the
arch/powerpc/kernel/asm-offsets.c +1 -0
index 1a70f02..42a4243 100644
@@ -122,6 +122,7 @@ int main(void)
 #endif
 
 #ifdef CONFIG_TRANSACTIONAL_MEM
+	DEFINE(PACATMSCRATCH, offsetof(struct paca_struct, tm_scratch));
 	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
 	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
 	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
arch/powerpc/kernel/entry_64.S +4 -0
index df6857f..5ae8e51 100644
@@ -755,6 +755,10 @@ fast_exception_return:
 	andc	r4,r4,r0	 /* r0 contains MSR_RI here */
 	mtmsrd	r4,1
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	/* TM debug */
+	std	r3, PACATMSCRATCH(r13) /* Stash returned-to MSR */
+#endif
 	/*
 	 * r13 is our per cpu area, only restore it if we are returning to
 	 * userspace the value stored in the stack frame may belong to
arch/powerpc/kernel/process.c +3 -0
index 6d66a68..fd5ce1b 100644
@@ -693,6 +693,9 @@ void show_regs(struct pt_regs * regs)
 	printk("NIP ["REG"] %pS\n", regs->nip, (void *)regs->nip);
 	printk("LR ["REG"] %pS\n", regs->link, (void *)regs->link);
 #endif
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	printk("PACATMSCRATCH [%llx]\n", get_paca()->tm_scratch);
+#endif
 	show_stack(current, (unsigned long *) regs->gpr[1]);
 	if (!user_mode(regs))
 		show_instructions(regs);
Signature · 1 line
1.7.9.5
#7 lore raw

PATCH 7/16 powerpc: Add helper functions for transactional memory context switching

In reply to Michael Neuling

Here we add the helper functions to be used when context switching. These allow us to fully reclaim and recheckpoint a transaction.

---
 arch/powerpc/include/asm/reg.h |    2 +-
 arch/powerpc/include/asm/tm.h  |   19 ++
 arch/powerpc/kernel/Makefile   |    2 +
 arch/powerpc/kernel/tm.S       |  378 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 400 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/include/asm/tm.h
 create mode 100644 arch/powerpc/kernel/tm.S
arch/powerpc/include/asm/reg.h +1 -1
index bea823f..895020f 100644
@@ -784,7 +784,7 @@
  *        HV mode in which case it is HSPRG0
  *
  * 64-bit server:
- *	- SPRG0 unused (reserved for HV on Power4)
+ *	- SPRG0 scratch for TM recheckpoint/reclaim (reserved for HV on Power4)
  *	- SPRG2 scratch for exception vectors
  *	- SPRG3 CPU and NUMA node for VDSO getcpu (user visible)
  *      - HSPRG0 stores PACA in HV mode
arch/powerpc/include/asm/tm.h new file +19 -0
new file mode 100644
index 0000000..18f6bf7
@@ -0,0 +1,19 @@
+/*
+ * Transactional memory support routines to reclaim and recheckpoint
+ * transactional process state.
+ *
+ * Copyright 2012 Matt Evans & Michael Neuling, IBM Corporation.
+ */
+
+#ifdef CONFIG_TRANSACTIONAL_MEM
+extern void do_load_up_transact_fpu(struct thread_struct *thread);
+extern void do_load_up_transact_altivec(struct thread_struct *thread);
+#endif
+
+extern void tm_enable(void);
+extern void tm_reclaim(struct thread_struct *thread,
+		       unsigned long orig_msr, uint8_t cause);
+extern void tm_recheckpoint(struct thread_struct *thread,
+			    unsigned long orig_msr);
+extern void tm_save_sprs(struct thread_struct *thread);
+extern void tm_restore_sprs(struct thread_struct *thread);
arch/powerpc/kernel/Makefile +2 -0
index 44fbbea..abf8469 100644
@@ -122,6 +122,8 @@ ifneq ($(CONFIG_PPC_INDIRECT_IO),y)
 obj-y				+= iomap.o
 endif
 
+obj64-$(CONFIG_TRANSACTIONAL_MEM)	+= tm.o
+
 obj-$(CONFIG_PPC64)		+= $(obj64-y)
 obj-$(CONFIG_PPC32)		+= $(obj32-y)
 
arch/powerpc/kernel/tm.S new file +378 -0

378 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#8 lore raw

PATCH 8/16 powerpc: Add FP/VSX and VMX register load functions for transactional memory

In reply to Michael Neuling

This adds functions to restore the state of the FP/VSX registers from what's stored in the thread_struct. Two version for FP/VSX are required since one restores them from transactional/checkpoint side of the thread_struct and the other from the speculated side.

Similar functions are added for VMX registers.

---
 arch/powerpc/kernel/fpu.S    |   54 ++++++++++++++++++++++++++++++++++++++++++
 arch/powerpc/kernel/vector.S |   51 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+)
arch/powerpc/kernel/fpu.S +54 -0
index adb1551..6ab0e87 100644
@@ -62,6 +62,60 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+/*
+ * Wrapper to call load_up_fpu from C.
+ * void do_load_up_fpu(struct pt_regs *regs);
+ */
+_GLOBAL(do_load_up_fpu)
+	mflr	r0
+	std	r0, 16(r1)
+	stdu	r1, -112(r1)
+
+	subi	r6, r3, STACK_FRAME_OVERHEAD
+	/* load_up_fpu expects r12=MSR, r13=PACA, and returns
+	 * with r12 = new MSR.
+	 */
+	ld	r12,_MSR(r6)
+	GET_PACA(r13)
+
+	bl	load_up_fpu
+	std	r12,_MSR(r6)
+
+	ld	r0, 112+16(r1)
+	addi	r1, r1, 112
+	mtlr	r0
+	blr
+
+
+/* void do_load_up_fpu(struct thread_struct *thread)
+ *
+ * This is similar to load_up_fpu but for the transactional version of the FP
+ * register set.  It doesn't mess with the task MSR or valid flags.
+ * Furthermore, we don't do lazy FP with TM currently.
+ */
+_GLOBAL(do_load_up_transact_fpu)
+	mfmsr	r6
+	ori	r5,r6,MSR_FP
+#ifdef CONFIG_VSX
+BEGIN_FTR_SECTION
+	oris	r5,r5,MSR_VSX@h
+END_FTR_SECTION_IFSET(CPU_FTR_VSX)
+#endif
+	SYNC
+	MTMSRD(r5)
+
+	lfd	fr0,THREAD_TRANSACT_FPSCR(r3)
+	MTFSF_L(fr0)
+	REST_32FPVSRS_TRANSACT(0, R4, R3)
+
+	/* FP/VSX off again */
+	MTMSRD(r6)
+	SYNC
+
+	blr
+#endif /* CONFIG_TRANSACTIONAL_MEM */
+
 /*
  * This task wants to use the FPU now.
  * On UP, disable FP for the task which had the FPU previously,
arch/powerpc/kernel/vector.S +51 -0

57 lines not on this page · raw message

Signature · 1 line
1.7.9.5
#9 lore raw

PATCH 9/16 powerpc: Add reclaim and recheckpoint functions for context switching transactional memory processes

In reply to Michael Neuling

When we switch out a task, we need to save both the checkpointed and the speculated state into the thread struct.

Similarly when we are switching in a task we need to load both the checkpointed and speculated state. If the task was using FP, we non-lazily reload both the original and the speculative FP register states. This is because the kernel doesn't see if/when a TM rollback occurs, so if we take an FP unavoidable later, we are unable to determine which set of FP regs need to be restored.

This simply adds these functions. It doesn't hook them into the existing code yet.

---
 arch/powerpc/kernel/process.c |  113 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)
arch/powerpc/kernel/process.c +113 -0
index fd5ce1b..1bf2c6c7 100644
@@ -50,6 +50,7 @@
 #include <asm/runlatch.h>
 #include <asm/syscalls.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 #ifdef CONFIG_PPC64
 #include <asm/firmware.h>
@@ -407,6 +408,118 @@ int set_dabr(unsigned long dabr, unsigned long dabrx)
 DEFINE_PER_CPU(struct cpu_usage, cpu_usage_array);
 #endif
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+static inline void tm_reclaim_task(struct task_struct *tsk)
+{
+	/* We have to work out if we're switching from/to a task that's in the
+	 * middle of a transaction.
+	 *
+	 * In switching we need to maintain a 2nd register state as
+	 * oldtask->thread.ckpt_regs.  We tm_reclaim(oldproc); this saves the
+	 * checkpointed (tbegin) state in ckpt_regs and saves the transactional
+	 * (current) FPRs into oldtask->thread.transact_fpr[].
+	 *
+	 * We also context switch (save) TFHAR/TEXASR/TFIAR in here.
+	 */
+	struct thread_struct *thr = &tsk->thread;
+
+	if (!thr->regs)
+		return;
+
+	if (!MSR_TM_ACTIVE(thr->regs->msr))
+		goto out_and_saveregs;
+
+	/* Stash the original thread MSR, as giveup_fpu et al will
+	 * modify it.  We hold onto it to see whether the task used
+	 * FP & vector regs.
+	 */
+	thr->tm_orig_msr = thr->regs->msr;
+
+	TM_DEBUG("--- tm_reclaim on pid %d (NIP=%lx, "
+		 "ccr=%lx, msr=%lx, trap=%lx)\n",
+		 tsk->pid, thr->regs->nip,
+		 thr->regs->ccr, thr->regs->msr,
+		 thr->regs->trap);
+
+	tm_reclaim(thr, thr->regs->msr, TM_CAUSE_RESCHED);
+
+	TM_DEBUG("--- tm_reclaim on pid %d complete\n",
+		 tsk->pid);
+
+out_and_saveregs:
+	/* Always save the regs here, even if a transaction's not active.
+	 * This context-switches a thread's TM info SPRs.  We do it here to
+	 * be consistent with the restore path (in recheckpoint) which
+	 * cannot happen later in _switch().
+	 */
+	tm_save_sprs(thr);
+}
+
+static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+{
+	unsigned long msr;
+
+	if (!cpu_has_feature(CPU_FTR_TM))
+		return;
+
+	/* Recheckpoint the registers of the thread we're about to switch to.
+	 *
+	 * If the task was using FP, we non-lazily reload both the original and
+	 * the speculative FP register states.  This is because the kernel
+	 * doesn't see if/when a TM rollback occurs, so if we take an FP
+	 * unavoidable later, we are unable to determine which set of FP regs
+	 * need to be restored.
+	 */
+	if (!new->thread.regs)
+		return;
+
+	/* The TM SPRs are restored here, so that TEXASR.FS can be set
+	 * before the trecheckpoint and no explosion occurs.
+	 */
+	tm_restore_sprs(&new->thread);
+
+	if (!MSR_TM_ACTIVE(new->thread.regs->msr))
+		return;
+	msr = new->thread.tm_orig_msr;
+	/* Recheckpoint to restore original checkpointed register state. */
+	TM_DEBUG("*** tm_recheckpoint of pid %d "
+		 "(new->msr 0x%lx, new->origmsr 0x%lx)\n",
+		 new->pid, new->thread.regs->msr, msr);
+
+	/* This loads the checkpointed FP/VEC state, if used */
+	tm_recheckpoint(&new->thread, msr);
+
+	/* This loads the speculative FP/VEC state, if used */
+	if (msr & MSR_FP) {
+		do_load_up_transact_fpu(&new->thread);
+		new->thread.regs->msr |=
+			(MSR_FP | new->thread.fpexc_mode);
+	}
+	if (msr & MSR_VEC) {
+		do_load_up_transact_altivec(&new->thread);
+		new->thread.regs->msr |= MSR_VEC;
+	}
+	/* We may as well turn on VSX too since all the state is restored now */
+	if (msr & MSR_VSX)
+		new->thread.regs->msr |= MSR_VSX;
+
+	TM_DEBUG("*** tm_recheckpoint of pid %d complete "
+		 "(kernel msr 0x%lx)\n",
+		 new->pid, mfmsr());
+}
+
+static inline void __switch_to_tm(struct task_struct *prev)
+{
+	if (cpu_has_feature(CPU_FTR_TM)) {
+		tm_enable();
+		tm_reclaim_task(prev);
+	}
+}
+#else
+#define tm_recheckpoint_new_task(new)
+#define __switch_to_tm(prev)
+#endif /* CONFIG_TRANSACTIONAL_MEM */
+
 struct task_struct *__switch_to(struct task_struct *prev,
 	struct task_struct *new)
 {
Signature · 1 line
1.7.9.5
#10 lore raw

PATCH 10/16 powerpc: Add transactional memory unavaliable execption handler

In reply to Michael Neuling

These should never happen since we always turn on MSR TM when in userspace. We don't do lazy TM.

Hence if we hit this, we barf and kill the task as something's gone horribly wrong.

---
 arch/powerpc/kernel/exceptions-64s.S |   19 +++++++++++++++++++
 arch/powerpc/kernel/traps.c          |   21 +++++++++++++++++++++
 2 files changed, 40 insertions(+)
arch/powerpc/kernel/exceptions-64s.S +19 -0
index 32fc04f..d97cea4 100644
@@ -313,6 +313,9 @@ vsx_unavailable_pSeries_1:
 	. = 0xf40
 	b	vsx_unavailable_pSeries
 
+	. = 0xf60
+	b	tm_unavailable_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_EXCEPTION_HV(0x1200, 0x1202, cbe_system_error)
 	KVM_HANDLER_SKIP(PACA_EXGEN, EXC_HV, 0x1202)
@@ -526,6 +529,8 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_206)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf20)
 	STD_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf40)
+	STD_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
+	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf60)
 
 /*
  * An interrupt came in while soft-disabled. We set paca->irq_happened, then:
@@ -815,6 +820,10 @@ vsx_unavailable_relon_pSeries_1:
 	. = 0x4f40
 	b	vsx_unavailable_relon_pSeries
 
+tm_unavailable_relon_pSeries_1:
+	. = 0x4f60
+	b	tm_unavailable_relon_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_RELON_EXCEPTION_HV(0x5200, 0x1202, cbe_system_error)
 #endif /* CONFIG_CBE_RAS */
@@ -1174,6 +1183,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 	b	.ret_from_except
 
 	.align	7
+	.globl tm_unavailable_common
+tm_unavailable_common:
+	EXCEPTION_PROLOG_COMMON(0xf60, PACA_EXGEN)
+	bl	.save_nvgprs
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.tm_unavailable_exception
+	b	.ret_from_except
+
+	.align	7
 	.globl	__end_handlers
 __end_handlers:
 
@@ -1387,6 +1405,7 @@ _GLOBAL(do_stab_bolted)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf00, performance_monitor)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf20, altivec_unavailable)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
+	STD_RELON_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
 
 #if defined(CONFIG_PPC_PSERIES) || defined(CONFIG_PPC_POWERNV)
 /*
arch/powerpc/kernel/traps.c +21 -0
index 8fed874..4acd98d 100644
@@ -1171,6 +1171,27 @@ void vsx_unavailable_exception(struct pt_regs *regs)
 	die("Unrecoverable VSX Unavailable Exception", regs, SIGABRT);
 }
 
+void tm_unavailable_exception(struct pt_regs *regs)
+{
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	/* Currently we never expect a TMU exception.  Catch
+	 * this and kill the process!
+	 */
+	printk(KERN_EMERG "Unexpected TM unavailable exception at %lx "
+	       "(msr %lx)\n",
+	       regs->nip, regs->msr);
+
+	if (user_mode(regs)) {
+		_exception(SIGILL, regs, ILL_ILLOPC, regs->nip);
+		return;
+	}
+
+	die("Unexpected TM unavailable exception", regs, SIGABRT);
+}
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.9.5
#11 lore raw

PATCH 11/16 powerpc: Assembler routines for FP/VSX/VMX unavailable during a transaction

In reply to Michael Neuling

We do lazy FP but not lazy TM (ie. userspace starts with MSR TM=1 FP=0). Hence if userspace does an FP instruction during a transaction, we'll take an fp unavailable exception.

This adds functions needed to handle this case. We have to inject the current FP state into the checkpoint so that the hardware can decide what to do with the transaction. We can't inject only the FP so we have to do a full treclaim and recheckpoint to inject just the FP state. This will cause the transaction to be marked as aborted by the hardware.

This just add the routines needed to do this for FP, VMX and VSX. It doesn't hook them into the rest of the code yet.

---
 arch/powerpc/kernel/traps.c |   95 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 95 insertions(+)
arch/powerpc/kernel/traps.c +95 -0
index 4acd98d..7b9f160 100644
@@ -58,6 +58,7 @@
 #include <asm/rio.h>
 #include <asm/fadump.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 
 #if defined(CONFIG_DEBUGGER) || defined(CONFIG_KEXEC)
@@ -1192,6 +1193,100 @@ void tm_unavailable_exception(struct pt_regs *regs)
 	die("Unexpected TM unavailable exception", regs, SIGABRT);
 }
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+
+extern void do_load_up_fpu(struct pt_regs *regs);
+
+void fp_unavailable_tm(struct pt_regs *regs)
+{
+	/* Note:  This does not handle any kind of FP laziness. */
+
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	TM_DEBUG("FP Unavailable trap whilst transactional at 0x%lx, MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+
+        /* We can only have got here if the task started using FP after
+         * beginning the transaction.  So, the transactional regs are just a
+         * copy of the checkpointed ones.  But, we still need to recheckpoint
+         * as we're enabling FP for the process; it will return, abort the
+         * transaction, and probably retry but now with FP enabled.  So the
+         * checkpointed FP registers need to be loaded.
+	 */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	/* Reclaim didn't save out any FPRs to transact_fprs. */
+
+	/* Enable FP for the task: */
+	regs->msr |= (MSR_FP | current->thread.fpexc_mode);
+
+	/* This loads and recheckpoints the FP registers from
+	 * thread.fpr[].  They will remain in registers after the
+	 * checkpoint so we don't need to reload them after.
+	 */
+	tm_recheckpoint(&current->thread, regs->msr);
+}
+
+#ifdef CONFIG_ALTIVEC
+extern void do_load_up_altivec(struct pt_regs *regs);
+
+void altivec_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This function operates
+	 * the same way.
+	 */
+
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	TM_DEBUG("Vector Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	regs->msr |= MSR_VEC;
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vr = 1;
+}
+#endif
+
+#ifdef CONFIG_VSX
+void vsx_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This works similarly,
+	 * though we're loading both FP and VEC registers in here.
+	 *
+	 * If FP isn't in use, load FP regs.  If VEC isn't in use, load VEC
+	 * regs.  Either way, set MSR_VSX.
+	 */
+
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	TM_DEBUG("VSX Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+
+	tm_enable();
+	/* This reclaims FP and/or VR regs if they're already enabled */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+
+	regs->msr |= MSR_VEC | MSR_FP | current->thread.fpexc_mode |
+		MSR_VSX;
+	/* This loads & recheckpoints FP and VRs. */
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vsr = 1;
+}
+#endif
+#endif /* CONFIG_TRANSACTIONAL_MEM */
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.9.5
#12 lore raw

PATCH 12/16 powerpc: Hook in new transactional memory code

In reply to Michael Neuling

This hooks the new transactional memory code into context switching, FP/VMX/VMX unavailable and exception return.

---
 arch/powerpc/kernel/entry_64.S       |   22 ++++++++++++++++
 arch/powerpc/kernel/exceptions-64s.S |   48 ++++++++++++++++++++++++++++++++--
 arch/powerpc/kernel/fpu.S            |    1 -
 arch/powerpc/kernel/process.c        |   15 +++++++++--
 arch/powerpc/kernel/traps.c          |   32 +++++++++++++++++++++++
 arch/powerpc/kernel/vector.S         |    1 -
 6 files changed, 113 insertions(+), 6 deletions(-)
arch/powerpc/kernel/entry_64.S +22 -0
index 5ae8e51..b3590c3 100644
@@ -602,7 +602,29 @@ _GLOBAL(ret_from_except_lite)
 	beq	1f
 	bl	.restore_interrupts
 	bl	.schedule
+#ifdef CONFIG_TRANSACTIONAL_MEM
+BEGIN_FTR_SECTION
+	/* If TIF_RESTOREALL was set by switch_to, we MUST clear it before
+	 * any return to userspace -- no one else is going to.  TRAP.0 has been
+	 * cleared to flag full regs to ret_from_except.
+	 * Try to avoid the slow atomic clear if the flag isn't set.
+	 * (This is OK as no one else will be clearing this flag.)
+         */
+	clrrdi	r9,r1,THREAD_SHIFT	/* current_thread_info() */
+	li	r4,_TIF_RESTOREALL
+	addi	r9, r9, TI_FLAGS
+	ld	r3, 0(r9)		/* Test TIF_RESTOREALL first! */
+	and.	r0, r3, r4
+	beq	.ret_from_except
+3:	ldarx	r10, 0, r9		/* If set, clear. */
+	andc	r10, r10, r4
+	stdcx.	r10, 0, r9
+	bne	3b
+	b	.ret_from_except	/* Not _lite; we may have full regs! */
+END_FTR_SECTION_IFSET(CPU_FTR_TM)
+#else
 	b	.ret_from_except_lite
+#endif
 
 1:	bl	.save_nvgprs
 	bl	.restore_interrupts
arch/powerpc/kernel/exceptions-64s.S +46 -2

70 lines not on this page · raw message

arch/powerpc/kernel/fpu.S +0 -1
index 6ab0e87..08b6a12f 100644
@@ -170,7 +170,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 	lwz	r4,THREAD_FPEXC_MODE(r5)
 	ori	r12,r12,MSR_FP
 	or	r12,r12,r4
-	std	r12,_MSR(r1)
 #endif
 	lfd	fr0,THREAD_FPSCR(r5)
 	MTFSF_L(fr0)
arch/powerpc/kernel/process.c +13 -2
index 1bf2c6c7..a0bfd97 100644
@@ -455,7 +455,7 @@ out_and_saveregs:
 	tm_save_sprs(thr);
 }
 
-static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+static inline void tm_recheckpoint_new_task(struct task_struct *new)
 {
 	unsigned long msr;
 
@@ -530,6 +530,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	struct ppc64_tlb_batch *batch;
 #endif
 
+	__switch_to_tm(prev);
+
 #ifdef CONFIG_SMP
 	/* avoid complexity of lazy save/restore of fpu
 	 * by just saving it every time we switch out if
@@ -645,6 +647,9 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	 * of sync. Hard disable here.
 	 */
 	hard_irq_disable();
+
+	tm_recheckpoint_new_task(new);
+
 	last = _switch(old_thread, new_thread);
 
 #ifdef CONFIG_PPC_BOOK3S_64
@@ -1019,7 +1024,6 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 		regs->msr = MSR_USER32;
 	}
 #endif
-
 	discard_lazy_cpu_state();
 #ifdef CONFIG_VSX
 	current->thread.used_vsr = 0;
@@ -1039,6 +1043,13 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.spefscr = 0;
 	current->thread.used_spe = 0;
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	if (cpu_has_feature(CPU_FTR_TM))
+		regs->msr |= MSR_TM;
+	current->thread.tm_tfhar = 0;
+	current->thread.tm_texasr = 0;
+	current->thread.tm_tfiar = 0;
+#endif /* CONFIG_TRANSACTIONAL_MEM */
 }
 
 #define PR_FP_ALL_EXCEPT (PR_FP_EXC_DIV | PR_FP_EXC_OVF | PR_FP_EXC_UND \
arch/powerpc/kernel/traps.c +32 -0
index 7b9f160..cb04fc7 100644
@@ -1032,6 +1032,38 @@ void __kprobes program_check_exception(struct pt_regs *regs)
 		_exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip);
 		return;
 	}
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	if (reason & REASON_TM) {
+		/* This is a TM "Bad Thing Exception" program check.
+		 * This occurs when:
+		 * -  An rfid/hrfid/mtmsrd attempts to cause an illegal
+		 *    transition in TM states.
+		 * -  A trechkpt is attempted when transactional.
+		 * -  A treclaim is attempted when non transactional.
+		 * -  A tend is illegally attempted.
+		 * -  writing a TM SPR when transactional.
+		 */
+		if (!user_mode(regs) &&
+		    report_bug(regs->nip, regs) == BUG_TRAP_TYPE_WARN) {
+			regs->nip += 4;
+			return;
+		}
+		/* If usermode caused this, it's done something illegal and
+		 * gets a SIGILL slap on the wrist.  We call it an illegal
+		 * operand to distinguish from the instruction just being bad
+		 * (e.g. executing a 'tend' on a CPU without TM!); it's an
+		 * illegal /placement/ of a valid instruction.
+		 */
+		if (user_mode(regs)) {
+			_exception(SIGILL, regs, ILL_ILLOPN, regs->nip);
+			return;
+		} else {
+			printk(KERN_EMERG "Unexpected TM Bad Thing exception "
+			       "at %lx (msr 0x%x)\n", regs->nip, reason);
+			die("Unrecoverable exception", regs, SIGABRT);
+		}
+	}
+#endif
 
 	/* We restore the interrupt state now */
 	if (!arch_irq_disabled_regs(regs))
arch/powerpc/kernel/vector.S +0 -1
index 330fc8c..9b47306 100644
@@ -123,7 +123,6 @@ _GLOBAL(load_up_altivec)
 	ld	r4,PACACURRENT(r13)
 	addi	r5,r4,THREAD		/* Get THREAD */
 	oris	r12,r12,MSR_VEC@h
-	std	r12,_MSR(r1)
 #endif
 	li	r4,1
 	li	r10,THREAD_VSCR
Signature · 1 line
1.7.9.5
#13 lore raw

PATCH 13/16 powerpc: Add transactional memory to POWER8 cpu features

In reply to Michael Neuling
---
 arch/powerpc/include/asm/cputable.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/cputable.h +1 -1
index f2163da..74458e69 100644
@@ -417,7 +417,7 @@ extern const char *powerpc_base_platform;
 	    CPU_FTR_DSCR | CPU_FTR_SAO  | \
 	    CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \
 	    CPU_FTR_ICSWX | CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \
-	    CPU_FTR_DBELL)
+	    CPU_FTR_DBELL | CPU_FTR_TM_COMP)
 #define CPU_FTRS_CELL	(CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \
 	    CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
Signature · 1 line
1.7.9.5
#14 lore raw

PATCH 14/16 powerpc: Add config option for transactional memory

In reply to Michael Neuling

Kconfig option for transactional memory on powerpc.

---
 arch/powerpc/Kconfig |    8 ++++++++
 1 file changed, 8 insertions(+)
arch/powerpc/Kconfig +8 -0
index a902a5c..ece67ca 100644
@@ -308,6 +308,14 @@ config MATH_EMULATION
 	  unit, which will allow programs that use floating-point
 	  instructions to run.
 
+config TRANSACTIONAL_MEM
+       bool "Transactional Memory support"
+       depends on PPC64
+       depends on SMP
+       default n
+       ---help---
+         Support user-mode Transactional Memory.
+
 config 8XX_MINIMAL_FPEMU
 	bool "Minimal math emulation for 8xx"
 	depends on 8xx && !MATH_EMULATION
Signature · 1 line
1.7.9.5
#15 lore raw

PATCH 15/16 powerpc: Add transactional memory to pseries and ppc64 defconfigs

In reply to Michael Neuling
---
 arch/powerpc/configs/ppc64_defconfig   |    1 +
 arch/powerpc/configs/pseries_defconfig |    1 +
 2 files changed, 2 insertions(+)
arch/powerpc/configs/ppc64_defconfig +1 -0
index 6d03530..26000f6 100644
@@ -46,6 +46,7 @@ CONFIG_NO_HZ=y
 CONFIG_HIGH_RES_TIMERS=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_TRANSACTIONAL_MEM=y
 CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
arch/powerpc/configs/pseries_defconfig +1 -0
index 5b8e1e5..b5f94b7 100644
@@ -40,6 +40,7 @@ CONFIG_NO_HZ=y
 CONFIG_HIGH_RES_TIMERS=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_TRANSACTIONAL_MEM=y
 CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
Signature · 1 line
1.7.9.5
#16 lore raw

PATCH 16/16 powerpc: Documentation for transactional memory on powerpc

In reply to Michael Neuling
---
 Documentation/powerpc/transactional_memory.txt |  119 ++++++++++++++++++++++++
 1 file changed, 119 insertions(+)
 create mode 100644 Documentation/powerpc/transactional_memory.txt
Documentation/powerpc/transactional_memory.txt new file +119 -0
new file mode 100644
index 0000000..881df6b
@@ -0,0 +1,119 @@
+Transactional Memory support
+============================
+
+POWER kernel support for this feature is currently limited to supporting
+its use by user programs.  It is not currently used by the kernel itself.
+
+This file aims to sum up how it is supported by Linux and what behaviour you
+can expect from your user programs.
+
+
+Basic overview
+==============
+
+Hardware Transactional Memory is supported on POWER8 processors, and is a
+feature that enables a different form of atomic memory access.  Several new
+instructions are presented to delimit transactions; transactions are
+guaranteed to either complete atomically or roll back and undo any partial
+changes.
+
+A simple transaction looks like this:
+
+begin_move_money:
+  tbegin
+  beq   abort_handler
+
+  ld    r4, SAVINGS_ACCT(r3)
+  ld    r5, CURRENT_ACCT(r3)
+  subi  r5, r5, 1
+  addi  r4, r4, 1
+  std   r4, SAVINGS_ACCT(r3)
+  std   r5, CURRENT_ACCT(r3)
+
+  tend
+
+  b     continue
+
+abort_handler:
+  ... test for odd failures ...
+
+  /* Retry the transaction if it failed because it conflicted with
+   * someone else: */
+  b     begin_move_money
+
+
+The 'tbegin' instruction denotes the start point, and 'tend' the end point.
+Between these points the processor is in 'Transactional' state; any memory
+references will complete in one go if there are no conflicts with other
+transactional or non-transactional accesses within the system.  In this
+example, the transaction completes as though it were normal straight-line code
+IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an
+atomic move of money from the current account to the savings account has been
+performed.  Even though the normal ld/std instructions are used (note no
+lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be
+updated, or neither will be updated.
+
+If, in the meantime, there is a conflict with the locations accessed by the
+transaction, the transaction will be aborted by the CPU.  Register and memory
+state will roll back to that at the 'tbegin', and control will continue from
+'tbegin+4'.  The branch to abort_handler will be taken this second time; the
+abort handler can check the cause of the failure, and retry.
+
+Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR
+and a few other status/flag regs; see the ISA for details.
+
+Causes of transaction aborts
+============================
+
+- Conflicts with cache lines used by other processors
+- Signals
+- Context switches
+- See the ISA for full documentation of everything that will abort transactions.
+
+
+Syscalls
+========
+
+Performing syscalls from within transaction is not recommended, and can lead
+to unpredictable results.
+
+Syscalls do not by design abort transactions, but beware: The kernel code will
+not be running in transactional state.  The effect of syscalls will always
+remain visible, but depending on the call they may abort your transaction as a
+side-effect, read soon-to-be-aborted transactional data that should not remain
+invisible, etc.  If you constantly retry a transaction that constantly aborts
+itself by calling a syscall, you'll have a livelock & make no progress.
+
+Simple syscalls (e.g. sigprocmask()) "could" be OK.  Even things like write()
+from, say, printf() should be OK as long as the kernel does not access any
+memory that was accessed transactionally.
+
+Consider any syscalls that happen to work as debug-only -- not recommended for
+production use.  Best to queue them up till after the transaction is over.
+
+
+Failure cause codes used by kernel
+==================================
+
+These are defined in <asm/reg.h>, and distinguish different reasons why the
+kernel aborted a transaction:
+
+ TM_CAUSE_RESCHED       Thread was rescheduled.
+ TM_CAUSE_FAC_UNAV      FP/VEC/VSX unavailable trap.
+ TM_CAUSE_SYSCALL       Currently unused; future syscalls that must abort
+                        transactions for consistency will use this.
+ TM_CAUSE_SIGNAL        Signal delivered.
+ TM_CAUSE_MISC          Currently unused.
+
+These can be checked by the user program's abort handler as TEXASR[0:7].
+
+
+GDB
+===
+
+GDB and ptrace are not currently TM-aware.  If one stops during a transaction,
+it looks like the transaction has just started (the checkpointed state is
+presented).  The transaction cannot then be continued and will take the failure
+handler route.  Furthermore, the transactional 2nd register state will be
+inaccessible.  GDB can currently be used on programs using TM, but not sensibly
+in parts within transactions.
Signature · 1 line
1.7.9.5
#17 lore raw

cover powerpc: Hardware transactional memory support for POWER8

In reply to Michael Neuling

POWER8 implements hardware transactional memory support. This patch series adds kernel support so that user programs can use this hardware transactional memory and the new state is properly context switched. It is not currently used by the kernel itself.

This patch series was originally developed by Matt Evans.

v2 changes
Adds signal handling
Removes some code for detecting when to restore all register or not Added tabort for local tlb invalidates
Rebased on benh's next tree

Basic overview of a POWER8 hardware transaction memory
=====================================================
Hardware transactional memory is a feature that enables a different form of atomic memory access. Several new instructions are presented to delimit transactions; transactions are guaranteed to either complete atomically or roll back and undo any partial changes.

A simple transaction looks like this:

begin_move_money:
tbegin
beq abort_handler

  ld    r4, SAVINGS_ACCT(r3)
  ld    r5, CURRENT_ACCT(r3)
  subi  r5, r5, 1
  addi  r4, r4, 1
  std   r4, SAVINGS_ACCT(r3)
  std   r5, CURRENT_ACCT(r3)

tend

b continue

abort_handler:
... test for odd failures ...

/* Retry the transaction if it failed because it conflicted with
* someone else: */
b begin_move_money

The 'tbegin' instruction denotes the start point, and 'tend' the end point. Between these points the processor is in 'Transactional' state; any memory references will complete in one go if there are no conflicts with other transactional or non-transactional accesses within the system. In this example, the transaction completes as though it were normal straight-line code IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an atomic move of money from the current account to the savings account has been performed. Even though the normal ld/std instructions are used (note no lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be updated, or neither will be updated.

If, in the meantime, there is a conflict with the locations accessed by the transaction, the transaction will be aborted by the CPU. Register and memory state will roll back to that at the 'tbegin', and control will continue from 'tbegin+4'. The branch to abort_handler will be taken this second time; the abort handler can check the cause of the failure, and retry.

Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR and a few other status/flag regs;

#18 lore raw

PATCH 1/17 powerpc: Add new CPU feature bit for transactional memory

---
 arch/powerpc/include/asm/cputable.h |    7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/include/asm/cputable.h +7 -0
index 06f7fb9..71a498b 100644
@@ -220,6 +220,13 @@ extern const char *powerpc_base_platform;
 #define PPC_FEATURE_HAS_EFP_DOUBLE_COMP 0
 #endif
 
+/* We only set the TM feature if the kernel was compiled with TM supprt */
+#ifdef CONFIG_TRANSACTIONAL_MEM
+#define CPU_FTR_TM_COMP		CPU_FTR_TM
+#else
+#define CPU_FTR_TM_COMP		0
+#endif
+
 /* We need to mark all pages as being coherent if we're SMP or we have a
  * 74[45]x and an MPC107 host bridge. Also 83xx and PowerQUICC II
  * require it for PCI "streaming/prefetch" to work properly.
Signature · 1 line
1.7.10.4
#19 lore raw

PATCH 2/17 powerpc: Add new instructions for transactional memory

In reply to Michael Neuling

Here we define the new instructions we need for transactional memory in the kernel. This is so we can support compiling with binutils that don't support the new transactional memory instructions.

Transactional memory results in two sets of architected state (GPRs/VSRs etc).

treclaim allows us to read the checkpointed state (from the tbegin) so that we can store it away on a context switch. It does this by overwriting the exiting architected state, so you have to save that away before you treclaim. treclaim will also abort a transaction, so you can give a register value which contains an abort reason.

trecheckpoint allows us to inject into the checkpointed state as if it were at the tbegin. It does this by copying the current architected state into the checkpointed state.

---
 arch/powerpc/include/asm/ppc-opcode.h |   10 ++++++++++
 1 file changed, 10 insertions(+)
arch/powerpc/include/asm/ppc-opcode.h +10 -0
index 0fd1928..8752bc8 100644
@@ -129,6 +129,9 @@
 #define PPC_INST_TLBSRX_DOT		0x7c0006a5
 #define PPC_INST_XXLOR			0xf0000510
 #define PPC_INST_XVCPSGNDP		0xf0000780
+#define PPC_INST_TRECHKPT		0x7c0007dd
+#define PPC_INST_TRECLAIM		0x7c00075d
+#define PPC_INST_TABORT			0x7c00071d
 
 #define PPC_INST_NAP			0x4c000364
 #define PPC_INST_SLEEP			0x4c0003a4
@@ -294,4 +297,11 @@
 #define PPC_NAP			stringify_in_c(.long PPC_INST_NAP)
 #define PPC_SLEEP		stringify_in_c(.long PPC_INST_SLEEP)
 
+/* Transactional memory instructions */
+#define TRECHKPT		stringify_in_c(.long PPC_INST_TRECHKPT)
+#define TRECLAIM(r)		stringify_in_c(.long PPC_INST_TRECLAIM \
+					       | __PPC_RA(r))
+#define TABORT(r)		stringify_in_c(.long PPC_INST_TABORT \
+					       | __PPC_RA(r))
+
 #endif /* _ASM_POWERPC_PPC_OPCODE_H */
Signature · 1 line
1.7.10.4
#20 lore raw

PATCH 3/17 powerpc: Add additional state needed for transactional memory to thread struct

In reply to Michael Neuling

Set of new archtected state for saving away on context switch.

---
 arch/powerpc/include/asm/processor.h |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
arch/powerpc/include/asm/processor.h +28 -0
index 7938658..1ffed17 100644
@@ -246,6 +246,34 @@ struct thread_struct {
 	unsigned long	spefscr;	/* SPE & eFP status */
 	int		used_spe;	/* set if process has used spe */
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	u64		tm_tfhar;	/* Transaction fail handler addr */
+	u64		tm_texasr;	/* Transaction exception & summary */
+	u64		tm_tfiar;	/* Transaction fail instr address reg */
+	unsigned long	tm_orig_msr;	/* Thread's MSR on ctx switch */
+	struct pt_regs	ckpt_regs;	/* Checkpointed registers */
+
+	/*
+	 * Transactional FP and VSX 0-31 register set.
+	 * NOTE: the sense of these is the opposite of the integer ckpt_regs!
+	 *
+	 * When a transaction is active/signalled/scheduled etc., *regs is the
+	 * most recent set of/speculated GPRs with ckpt_regs being the older
+	 * checkpointed regs to which we roll back if transaction aborts.
+	 *
+	 * However, fpr[] is the checkpointed 'base state' of FP regs, and
+	 * transact_fpr[] is the new set of transactional values.
+	 * VRs work the same way.
+	 */
+	double		transact_fpr[32][TS_FPRWIDTH];
+	struct {
+		unsigned int pad;
+		unsigned int val;	/* Floating point status */
+	} transact_fpscr;
+	vector128	transact_vr[32] __attribute__((aligned(16)));
+	vector128	transact_vscr __attribute__((aligned(16)));
+	unsigned long	transact_vrsave;
+#endif /* CONFIG_TRANSACTIONAL_MEM */
 #ifdef CONFIG_KVM_BOOK3S_32_HANDLER
 	void*		kvm_shadow_vcpu; /* KVM internal data */
 #endif /* CONFIG_KVM_BOOK3S_32_HANDLER */
Signature · 1 line
1.7.10.4
#21 lore raw

PATCH 4/17 powerpc: New macros for transactional memory support

In reply to Michael Neuling

This adds new macros for saving and restoring checkpointed architected state from and to the thread_struct.

It also adds some debugging macros for when your brain explodes trying to debug your transactional memory enabled kernel.

---
 arch/powerpc/include/asm/ppc_asm.h   |   83 ++++++++++++++++++++++++++++++++++
 arch/powerpc/include/asm/processor.h |    1 +
 arch/powerpc/kernel/asm-offsets.c    |   24 ++++++++++
 arch/powerpc/kernel/fpu.S            |   12 +++++
 arch/powerpc/kernel/process.c        |   10 ++++
 arch/powerpc/kernel/traps.c          |   11 +++++
 6 files changed, 141 insertions(+)
arch/powerpc/include/asm/ppc_asm.h +83 -0
index c2d0e58..54219ce 100644
@@ -123,6 +123,89 @@ END_FW_FTR_SECTION_IFSET(FW_FEATURE_SPLPAR)
 #define REST_16VRS(n,b,base)	REST_8VRS(n,b,base); REST_8VRS(n+8,b,base)
 #define REST_32VRS(n,b,base)	REST_16VRS(n,b,base); REST_16VRS(n+16,b,base)
 
+/* Save/restore FPRs, VRs and VSRs from their checkpointed backups in
+ * thread_struct:
+ */
+#define SAVE_FPR_TRANSACT(n, base)	stfd n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define SAVE_2FPRS_TRANSACT(n, base)	SAVE_FPR_TRANSACT(n, base);	\
+					SAVE_FPR_TRANSACT(n+1, base)
+#define SAVE_4FPRS_TRANSACT(n, base)	SAVE_2FPRS_TRANSACT(n, base);	\
+					SAVE_2FPRS_TRANSACT(n+2, base)
+#define SAVE_8FPRS_TRANSACT(n, base)	SAVE_4FPRS_TRANSACT(n, base);	\
+					SAVE_4FPRS_TRANSACT(n+4, base)
+#define SAVE_16FPRS_TRANSACT(n, base)	SAVE_8FPRS_TRANSACT(n, base);	\
+					SAVE_8FPRS_TRANSACT(n+8, base)
+#define SAVE_32FPRS_TRANSACT(n, base)	SAVE_16FPRS_TRANSACT(n, base);	\
+					SAVE_16FPRS_TRANSACT(n+16, base)
+
+#define REST_FPR_TRANSACT(n, base)	lfd	n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define REST_2FPRS_TRANSACT(n, base)	REST_FPR_TRANSACT(n, base);	\
+					REST_FPR_TRANSACT(n+1, base)
+#define REST_4FPRS_TRANSACT(n, base)	REST_2FPRS_TRANSACT(n, base);	\
+					REST_2FPRS_TRANSACT(n+2, base)
+#define REST_8FPRS_TRANSACT(n, base)	REST_4FPRS_TRANSACT(n, base);	\
+					REST_4FPRS_TRANSACT(n+4, base)
+#define REST_16FPRS_TRANSACT(n, base)	REST_8FPRS_TRANSACT(n, base);	\
+					REST_8FPRS_TRANSACT(n+8, base)
+#define REST_32FPRS_TRANSACT(n, base)	REST_16FPRS_TRANSACT(n, base);	\
+					REST_16FPRS_TRANSACT(n+16, base)
+
+
+#define SAVE_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					stvx n,b,base
+#define SAVE_2VRS_TRANSACT(n,b,base)	SAVE_VR_TRANSACT(n,b,base);	\
+					SAVE_VR_TRANSACT(n+1,b,base)
+#define SAVE_4VRS_TRANSACT(n,b,base)	SAVE_2VRS_TRANSACT(n,b,base);	\
+					SAVE_2VRS_TRANSACT(n+2,b,base)
+#define SAVE_8VRS_TRANSACT(n,b,base)	SAVE_4VRS_TRANSACT(n,b,base);	\
+					SAVE_4VRS_TRANSACT(n+4,b,base)
+#define SAVE_16VRS_TRANSACT(n,b,base)	SAVE_8VRS_TRANSACT(n,b,base);	\
+					SAVE_8VRS_TRANSACT(n+8,b,base)
+#define SAVE_32VRS_TRANSACT(n,b,base)	SAVE_16VRS_TRANSACT(n,b,base);	\
+					SAVE_16VRS_TRANSACT(n+16,b,base)
+
+#define REST_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					lvx n,b,base
+#define REST_2VRS_TRANSACT(n,b,base)	REST_VR_TRANSACT(n,b,base);	\
+					REST_VR_TRANSACT(n+1,b,base)
+#define REST_4VRS_TRANSACT(n,b,base)	REST_2VRS_TRANSACT(n,b,base);	\
+					REST_2VRS_TRANSACT(n+2,b,base)
+#define REST_8VRS_TRANSACT(n,b,base)	REST_4VRS_TRANSACT(n,b,base);	\
+					REST_4VRS_TRANSACT(n+4,b,base)
+#define REST_16VRS_TRANSACT(n,b,base)	REST_8VRS_TRANSACT(n,b,base);	\
+					REST_8VRS_TRANSACT(n+8,b,base)
+#define REST_32VRS_TRANSACT(n,b,base)	REST_16VRS_TRANSACT(n,b,base);	\
+					REST_16VRS_TRANSACT(n+16,b,base)
+
+
+#define SAVE_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					STXVD2X(n,R##base,R##b)
+#define SAVE_2VSRS_TRANSACT(n,b,base)	SAVE_VSR_TRANSACT(n,b,base);	\
+	                                SAVE_VSR_TRANSACT(n+1,b,base)
+#define SAVE_4VSRS_TRANSACT(n,b,base)	SAVE_2VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_2VSRS_TRANSACT(n+2,b,base)
+#define SAVE_8VSRS_TRANSACT(n,b,base)	SAVE_4VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_4VSRS_TRANSACT(n+4,b,base)
+#define SAVE_16VSRS_TRANSACT(n,b,base)	SAVE_8VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_8VSRS_TRANSACT(n+8,b,base)
+#define SAVE_32VSRS_TRANSACT(n,b,base)	SAVE_16VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_16VSRS_TRANSACT(n+16,b,base)
+
+#define REST_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					LXVD2X(n,R##base,R##b)
+#define REST_2VSRS_TRANSACT(n,b,base)	REST_VSR_TRANSACT(n,b,base);    \
+	                                REST_VSR_TRANSACT(n+1,b,base)
+#define REST_4VSRS_TRANSACT(n,b,base)	REST_2VSRS_TRANSACT(n,b,base);	\
+	                                REST_2VSRS_TRANSACT(n+2,b,base)
+#define REST_8VSRS_TRANSACT(n,b,base)	REST_4VSRS_TRANSACT(n,b,base);	\
+	                                REST_4VSRS_TRANSACT(n+4,b,base)
+#define REST_16VSRS_TRANSACT(n,b,base)	REST_8VSRS_TRANSACT(n,b,base);	\
+	                                REST_8VSRS_TRANSACT(n+8,b,base)
+#define REST_32VSRS_TRANSACT(n,b,base)	REST_16VSRS_TRANSACT(n,b,base);	\
+	                                REST_16VSRS_TRANSACT(n+16,b,base)
+
 /* Save the lower 32 VSRs in the thread VSR region */
 #define SAVE_VSR(n,b,base)	li b,THREAD_VSR0+(16*(n));  STXVD2X(n,R##base,R##b)
 #define SAVE_2VSRS(n,b,base)	SAVE_VSR(n,b,base); SAVE_VSR(n+1,b,base)
arch/powerpc/include/asm/processor.h +1 -0
index 1ffed17..af420e8 100644
@@ -152,6 +152,7 @@ typedef struct {
 #define TS_FPROFFSET 0
 #define TS_VSRLOWOFFSET 1
 #define TS_FPR(i) fpr[i][TS_FPROFFSET]
+#define TS_TRANS_FPR(i) transact_fpr[i][TS_FPROFFSET]
 
 struct thread_struct {
 	unsigned long	ksp;		/* Kernel stack pointer */
arch/powerpc/kernel/asm-offsets.c +24 -0
index e39ca55..3b81960 100644
@@ -122,6 +122,30 @@ int main(void)
 	DEFINE(THREAD_KVM_VCPU, offsetof(struct thread_struct, kvm_vcpu));
 #endif
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
+	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
+	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
+	DEFINE(PT_CKPT_REGS, offsetof(struct thread_struct, ckpt_regs));
+	DEFINE(THREAD_TRANSACT_VR0, offsetof(struct thread_struct,
+					 transact_vr[0]));
+	DEFINE(THREAD_TRANSACT_VSCR, offsetof(struct thread_struct,
+					  transact_vscr));
+	DEFINE(THREAD_TRANSACT_VRSAVE, offsetof(struct thread_struct,
+					    transact_vrsave));
+	DEFINE(THREAD_TRANSACT_FPR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+	DEFINE(THREAD_TRANSACT_FPSCR, offsetof(struct thread_struct,
+					   transact_fpscr));
+#ifdef CONFIG_VSX
+	DEFINE(THREAD_TRANSACT_VSR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+#endif
+	/* Local pt_regs on stack for Transactional Memory funcs. */
+	DEFINE(TM_FRAME_SIZE, STACK_FRAME_OVERHEAD +
+	       sizeof(struct pt_regs) + 16);
+#endif /* CONFIG_TRANSACTIONAL_MEM */
+
 	DEFINE(TI_FLAGS, offsetof(struct thread_info, flags));
 	DEFINE(TI_LOCAL_FLAGS, offsetof(struct thread_info, local_flags));
 	DEFINE(TI_PREEMPT, offsetof(struct thread_info, preempt_count));
arch/powerpc/kernel/fpu.S +12 -0
index e0ada05..adb1551 100644
@@ -35,6 +35,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 2:	REST_32VSRS(n,c,base);						\
 3:
 
+#define __REST_32FPVSRS_TRANSACT(n,c,base)				\
+BEGIN_FTR_SECTION							\
+	b	2f;							\
+END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
+	REST_32FPRS_TRANSACT(n,base);					\
+	b	3f;							\
+2:	REST_32VSRS_TRANSACT(n,c,base);					\
+3:
+
 #define __SAVE_32FPVSRS(n,c,base)					\
 BEGIN_FTR_SECTION							\
 	b	2f;							\
@@ -45,9 +54,12 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 3:
 #else
 #define __REST_32FPVSRS(n,b,base)	REST_32FPRS(n, base)
+#define __REST_32FPVSRS_TRANSACT(n,b,base)	REST_32FPRS(n, base)
 #define __SAVE_32FPVSRS(n,b,base)	SAVE_32FPRS(n, base)
 #endif
 #define REST_32FPVSRS(n,c,base) __REST_32FPVSRS(n,__REG_##c,__REG_##base)
+#define REST_32FPVSRS_TRANSACT(n,c,base) \
+	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
 /*
arch/powerpc/kernel/process.c +10 -0
index 99550d3..1ab3a2c 100644
@@ -57,6 +57,16 @@
 #include <linux/kprobes.h>
 #include <linux/kdebug.h>
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+/* Transactional Memory debug */
+// #define TM_DEBUG_SW
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+#endif
+
 extern unsigned long _get_SP(void);
 
 #ifndef CONFIG_SMP
arch/powerpc/kernel/traps.c +11 -0
index a008cf5..2a62f54 100644
@@ -78,6 +78,16 @@ EXPORT_SYMBOL(__debugger_break_match);
 EXPORT_SYMBOL(__debugger_fault_handler);
 #endif
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+/* Transactional Memory trap debug */
+// #define TM_DEBUG_SW
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+#endif
+
 /*
  * Trap & Exception support
  */
@@ -350,6 +360,7 @@ static inline int check_io_access(struct pt_regs *regs)
    exception is in the MSR. */
 #define get_reason(regs)	((regs)->msr)
 #define get_mc_reason(regs)	((regs)->msr)
+#define REASON_TM		0x200000
 #define REASON_FP		0x100000
 #define REASON_ILLEGAL		0x80000
 #define REASON_PRIVILEGED	0x40000
Signature · 1 line
1.7.10.4
#22 lore raw

PATCH 5/17 powerpc: Register defines for various transactional memory registers

In reply to Michael Neuling

Defines for MSR bits and transactional memory related SPRs TFIAR, TEXASR and TEXASRU.

---
 arch/powerpc/include/asm/reg.h |   27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
arch/powerpc/include/asm/reg.h +27 -0
index 1f59fbb..3acef78 100644
@@ -29,6 +29,10 @@
 #define MSR_SF_LG	63              /* Enable 64 bit mode */
 #define MSR_ISF_LG	61              /* Interrupt 64b mode valid on 630 */
 #define MSR_HV_LG 	60              /* Hypervisor state */
+#define MSR_TS_T_LG	34		/* Trans Mem state: Transactional */
+#define MSR_TS_S_LG	33		/* Trans Mem state: Suspended */
+#define MSR_TS_LG	33		/* Trans Mem state (2 bits) */
+#define MSR_TM_LG	32		/* Trans Mem Available */
 #define MSR_VEC_LG	25	        /* Enable AltiVec */
 #define MSR_VSX_LG	23		/* Enable VSX */
 #define MSR_POW_LG	18		/* Enable Power Management */
@@ -98,6 +102,25 @@
 #define MSR_RI		__MASK(MSR_RI_LG)	/* Recoverable Exception */
 #define MSR_LE		__MASK(MSR_LE_LG)	/* Little Endian */
 
+#define MSR_TM		__MASK(MSR_TM_LG)	/* Transactional Mem Available */
+#define MSR_TS_N	0			/*  Non-transactional */
+#define MSR_TS_S	__MASK(MSR_TS_S_LG)	/*  Transaction Suspended */
+#define MSR_TS_T	__MASK(MSR_TS_T_LG)	/*  Transaction Transactional */
+#define MSR_TS_MASK	(MSR_TS_T | MSR_TS_S)   /* Transaction State bits */
+#define MSR_TM_ACTIVE(x) (((x) & MSR_TS_MASK) != 0) /* Transaction active? */
+#define MSR_TM_TRANSACTIONAL(x)	(((x) & MSR_TS_MASK) == MSR_TS_T)
+#define MSR_TM_SUSPENDED(x)	(((x) & MSR_TS_MASK) == MSR_TS_S)
+
+/* Reason codes describing kernel causes for transaction aborts.  By
+   convention, bit0 is copied to TEXASR[56] (IBM bit 7) which is set if
+   the failure is persistent.
+*/
+#define TM_CAUSE_RESCHED	0xfe
+#define TM_CAUSE_TLBI		0xfc
+#define TM_CAUSE_FAC_UNAV	0xfa
+#define TM_CAUSE_SYSCALL	0xf9 /* Persistent */
+#define TM_CAUSE_MISC		0xf6
+
 #if defined(CONFIG_PPC_BOOK3S_64)
 #define MSR_64BIT	MSR_SF
 
@@ -193,6 +216,10 @@
 #define SPRN_UAMOR	0x9d	/* User Authority Mask Override Register */
 #define SPRN_AMOR	0x15d	/* Authority Mask Override Register */
 #define SPRN_ACOP	0x1F	/* Available Coprocessor Register */
+#define SPRN_TFIAR	0x81	/* Transaction Failure Inst Addr   */
+#define SPRN_TEXASR	0x82	/* Transaction EXception & Summary */
+#define SPRN_TEXASRU	0x83	/* ''	   ''	   ''	 Upper 32  */
+#define SPRN_TFHAR	0x80	/* Transaction Failure Handler Addr */
 #define SPRN_CTRLF	0x088
 #define SPRN_CTRLT	0x098
 #define   CTRL_CT	0xc0000000	/* current thread */
Signature · 1 line
1.7.10.4
#23 lore raw

PATCH 6/17 powerpc: Add transactional memory paca scratch register to show_regs

In reply to Michael Neuling

Add transactional memory paca scratch register to show_regs. This is useful for debugging.

---
 arch/powerpc/include/asm/paca.h   |    1 +
 arch/powerpc/kernel/asm-offsets.c |    1 +
 arch/powerpc/kernel/entry_64.S    |    4 ++++
 arch/powerpc/kernel/process.c     |    3 +++
 4 files changed, 9 insertions(+)
arch/powerpc/include/asm/paca.h +1 -0
index c47d687..07e9851 100644
@@ -137,6 +137,7 @@ struct paca_struct {
 	u8 irq_work_pending;		/* IRQ_WORK interrupt while soft-disable */
 	u8 nap_state_lost;		/* NV GPR values lost in power7_idle */
 	u64 sprg3;			/* Saved user-visible sprg */
+	u64 tm_scratch;                 /* TM scratch area for reclaim */
 
 #ifdef CONFIG_PPC_POWERNV
 	/* Pointer to OPAL machine check event structure set by the
arch/powerpc/kernel/asm-offsets.c +1 -0
index 3b81960..c9cdec4 100644
@@ -123,6 +123,7 @@ int main(void)
 #endif
 
 #ifdef CONFIG_TRANSACTIONAL_MEM
+	DEFINE(PACATMSCRATCH, offsetof(struct paca_struct, tm_scratch));
 	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
 	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
 	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
arch/powerpc/kernel/entry_64.S +4 -0
index 6faf51e..283bc61 100644
@@ -752,6 +752,10 @@ fast_exception_return:
 	andc	r4,r4,r0	 /* r0 contains MSR_RI here */
 	mtmsrd	r4,1
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	/* TM debug */
+	std	r3, PACATMSCRATCH(r13) /* Stash returned-to MSR */
+#endif
 	/*
 	 * r13 is our per cpu area, only restore it if we are returning to
 	 * userspace the value stored in the stack frame may belong to
arch/powerpc/kernel/process.c +3 -0
index 1ab3a2c..cb11fed 100644
@@ -749,6 +749,9 @@ void show_regs(struct pt_regs * regs)
 	printk("NIP ["REG"] %pS\n", regs->nip, (void *)regs->nip);
 	printk("LR ["REG"] %pS\n", regs->link, (void *)regs->link);
 #endif
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	printk("PACATMSCRATCH [%llx]\n", get_paca()->tm_scratch);
+#endif
 	show_stack(current, (unsigned long *) regs->gpr[1]);
 	if (!user_mode(regs))
 		show_instructions(regs);
Signature · 1 line
1.7.10.4
#24 lore raw

PATCH 7/17 powerpc: Add helper functions for transactional memory context switching

In reply to Michael Neuling

Here we add the helper functions to be used when context switching. These allow us to fully reclaim and recheckpoint a transaction.

---
 arch/powerpc/include/asm/reg.h |    2 +-
 arch/powerpc/include/asm/tm.h  |   20 +++
 arch/powerpc/kernel/Makefile   |    2 +
 arch/powerpc/kernel/tm.S       |  383 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 406 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/include/asm/tm.h
 create mode 100644 arch/powerpc/kernel/tm.S
arch/powerpc/include/asm/reg.h +1 -1
index 3acef78..d19581ae4 100644
@@ -794,7 +794,7 @@
  *        HV mode in which case it is HSPRG0
  *
  * 64-bit server:
- *	- SPRG0 unused (reserved for HV on Power4)
+ *	- SPRG0 scratch for TM recheckpoint/reclaim (reserved for HV on Power4)
  *	- SPRG2 scratch for exception vectors
  *	- SPRG3 CPU and NUMA node for VDSO getcpu (user visible)
  *      - HSPRG0 stores PACA in HV mode
arch/powerpc/include/asm/tm.h new file +20 -0
new file mode 100644
index 0000000..e9ad010
@@ -0,0 +1,20 @@
+/*
+ * Transactional memory support routines to reclaim and recheckpoint
+ * transactional process state.
+ *
+ * Copyright 2012 Matt Evans & Michael Neuling, IBM Corporation.
+ */
+
+#ifdef CONFIG_TRANSACTIONAL_MEM
+extern void do_load_up_transact_fpu(struct thread_struct *thread);
+extern void do_load_up_transact_altivec(struct thread_struct *thread);
+#endif
+
+extern void tm_enable(void);
+extern void tm_reclaim(struct thread_struct *thread,
+		       unsigned long orig_msr, uint8_t cause);
+extern void tm_recheckpoint(struct thread_struct *thread,
+			    unsigned long orig_msr);
+extern void tm_abort(uint8_t cause);
+extern void tm_save_sprs(struct thread_struct *thread);
+extern void tm_restore_sprs(struct thread_struct *thread);
arch/powerpc/kernel/Makefile +2 -0
index b4f0e36..7554968 100644
@@ -121,6 +121,8 @@ ifneq ($(CONFIG_PPC_INDIRECT_IO),y)
 obj-y				+= iomap.o
 endif
 
+obj64-$(CONFIG_TRANSACTIONAL_MEM)	+= tm.o
+
 obj-$(CONFIG_PPC64)		+= $(obj64-y)
 obj-$(CONFIG_PPC32)		+= $(obj32-y)
 
arch/powerpc/kernel/tm.S new file +383 -0

383 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#25 lore raw

PATCH 8/17 powerpc: Add FP/VSX and VMX register load functions for transactional memory

In reply to Michael Neuling

This adds functions to restore the state of the FP/VSX registers from what's stored in the thread_struct. Two version for FP/VSX are required since one restores them from transactional/checkpoint side of the thread_struct and the other from the speculated side.

Similar functions are added for VMX registers.

---
 arch/powerpc/kernel/fpu.S    |   54 ++++++++++++++++++++++++++++++++++++++++++
 arch/powerpc/kernel/vector.S |   51 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+)
arch/powerpc/kernel/fpu.S +54 -0
index adb1551..6ab0e87 100644
@@ -62,6 +62,60 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+/*
+ * Wrapper to call load_up_fpu from C.
+ * void do_load_up_fpu(struct pt_regs *regs);
+ */
+_GLOBAL(do_load_up_fpu)
+	mflr	r0
+	std	r0, 16(r1)
+	stdu	r1, -112(r1)
+
+	subi	r6, r3, STACK_FRAME_OVERHEAD
+	/* load_up_fpu expects r12=MSR, r13=PACA, and returns
+	 * with r12 = new MSR.
+	 */
+	ld	r12,_MSR(r6)
+	GET_PACA(r13)
+
+	bl	load_up_fpu
+	std	r12,_MSR(r6)
+
+	ld	r0, 112+16(r1)
+	addi	r1, r1, 112
+	mtlr	r0
+	blr
+
+
+/* void do_load_up_fpu(struct thread_struct *thread)
+ *
+ * This is similar to load_up_fpu but for the transactional version of the FP
+ * register set.  It doesn't mess with the task MSR or valid flags.
+ * Furthermore, we don't do lazy FP with TM currently.
+ */
+_GLOBAL(do_load_up_transact_fpu)
+	mfmsr	r6
+	ori	r5,r6,MSR_FP
+#ifdef CONFIG_VSX
+BEGIN_FTR_SECTION
+	oris	r5,r5,MSR_VSX@h
+END_FTR_SECTION_IFSET(CPU_FTR_VSX)
+#endif
+	SYNC
+	MTMSRD(r5)
+
+	lfd	fr0,THREAD_TRANSACT_FPSCR(r3)
+	MTFSF_L(fr0)
+	REST_32FPVSRS_TRANSACT(0, R4, R3)
+
+	/* FP/VSX off again */
+	MTMSRD(r6)
+	SYNC
+
+	blr
+#endif /* CONFIG_TRANSACTIONAL_MEM */
+
 /*
  * This task wants to use the FPU now.
  * On UP, disable FP for the task which had the FPU previously,
arch/powerpc/kernel/vector.S +51 -0

57 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#26 lore raw

PATCH 9/17 powerpc: Add reclaim and recheckpoint functions for context switching transactional memory processes

In reply to Michael Neuling

When we switch out a task, we need to save both the checkpointed and the speculated state into the thread struct.

Similarly when we are switching in a task we need to load both the checkpointed and speculated state. If the task was using FP, we non-lazily reload both the original and the speculative FP register states. This is because the kernel doesn't see if/when a TM rollback occurs, so if we take an FP unavoidable later, we are unable to determine which set of FP regs need to be restored.

This simply adds these functions. It doesn't hook them into the existing code yet.

---
 arch/powerpc/kernel/process.c |  112 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 112 insertions(+)
arch/powerpc/kernel/process.c +112 -0
index cb11fed..14fabd1 100644
@@ -50,6 +50,7 @@
 #include <asm/runlatch.h>
 #include <asm/syscalls.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 #ifdef CONFIG_PPC64
 #include <asm/firmware.h>
@@ -462,6 +463,117 @@ static inline bool hw_brk_match(struct arch_hw_breakpoint *a,
 		return false;
 	return true;
 }
+#ifdef CONFIG_TRANSACTIONAL_MEM
+static inline void tm_reclaim_task(struct task_struct *tsk)
+{
+	/* We have to work out if we're switching from/to a task that's in the
+	 * middle of a transaction.
+	 *
+	 * In switching we need to maintain a 2nd register state as
+	 * oldtask->thread.ckpt_regs.  We tm_reclaim(oldproc); this saves the
+	 * checkpointed (tbegin) state in ckpt_regs and saves the transactional
+	 * (current) FPRs into oldtask->thread.transact_fpr[].
+	 *
+	 * We also context switch (save) TFHAR/TEXASR/TFIAR in here.
+	 */
+	struct thread_struct *thr = &tsk->thread;
+
+	if (!thr->regs)
+		return;
+
+	if (!MSR_TM_ACTIVE(thr->regs->msr))
+		goto out_and_saveregs;
+
+	/* Stash the original thread MSR, as giveup_fpu et al will
+	 * modify it.  We hold onto it to see whether the task used
+	 * FP & vector regs.
+	 */
+	thr->tm_orig_msr = thr->regs->msr;
+
+	TM_DEBUG("--- tm_reclaim on pid %d (NIP=%lx, "
+		 "ccr=%lx, msr=%lx, trap=%lx)\n",
+		 tsk->pid, thr->regs->nip,
+		 thr->regs->ccr, thr->regs->msr,
+		 thr->regs->trap);
+
+	tm_reclaim(thr, thr->regs->msr, TM_CAUSE_RESCHED);
+
+	TM_DEBUG("--- tm_reclaim on pid %d complete\n",
+		 tsk->pid);
+
+out_and_saveregs:
+	/* Always save the regs here, even if a transaction's not active.
+	 * This context-switches a thread's TM info SPRs.  We do it here to
+	 * be consistent with the restore path (in recheckpoint) which
+	 * cannot happen later in _switch().
+	 */
+	tm_save_sprs(thr);
+}
+
+static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+{
+	unsigned long msr;
+
+	if (!cpu_has_feature(CPU_FTR_TM))
+		return;
+
+	/* Recheckpoint the registers of the thread we're about to switch to.
+	 *
+	 * If the task was using FP, we non-lazily reload both the original and
+	 * the speculative FP register states.  This is because the kernel
+	 * doesn't see if/when a TM rollback occurs, so if we take an FP
+	 * unavoidable later, we are unable to determine which set of FP regs
+	 * need to be restored.
+	 */
+	if (!new->thread.regs)
+		return;
+
+	/* The TM SPRs are restored here, so that TEXASR.FS can be set
+	 * before the trecheckpoint and no explosion occurs.
+	 */
+	tm_restore_sprs(&new->thread);
+
+	if (!MSR_TM_ACTIVE(new->thread.regs->msr))
+		return;
+	msr = new->thread.tm_orig_msr;
+	/* Recheckpoint to restore original checkpointed register state. */
+	TM_DEBUG("*** tm_recheckpoint of pid %d "
+		 "(new->msr 0x%lx, new->origmsr 0x%lx)\n",
+		 new->pid, new->thread.regs->msr, msr);
+
+	/* This loads the checkpointed FP/VEC state, if used */
+	tm_recheckpoint(&new->thread, msr);
+
+	/* This loads the speculative FP/VEC state, if used */
+	if (msr & MSR_FP) {
+		do_load_up_transact_fpu(&new->thread);
+		new->thread.regs->msr |=
+			(MSR_FP | new->thread.fpexc_mode);
+	}
+	if (msr & MSR_VEC) {
+		do_load_up_transact_altivec(&new->thread);
+		new->thread.regs->msr |= MSR_VEC;
+	}
+	/* We may as well turn on VSX too since all the state is restored now */
+	if (msr & MSR_VSX)
+		new->thread.regs->msr |= MSR_VSX;
+
+	TM_DEBUG("*** tm_recheckpoint of pid %d complete "
+		 "(kernel msr 0x%lx)\n",
+		 new->pid, mfmsr());
+}
+
+static inline void __switch_to_tm(struct task_struct *prev)
+{
+	if (cpu_has_feature(CPU_FTR_TM)) {
+		tm_enable();
+		tm_reclaim_task(prev);
+	}
+}
+#else
+#define tm_recheckpoint_new_task(new)
+#define __switch_to_tm(prev)
+#endif /* CONFIG_TRANSACTIONAL_MEM */
 
 struct task_struct *__switch_to(struct task_struct *prev,
 	struct task_struct *new)
Signature · 1 line
1.7.10.4
#27 lore raw

PATCH 10/17 powerpc: Add transactional memory unavaliable execption handler

In reply to Michael Neuling

These should never happen since we always turn on MSR TM when in userspace. We don't do lazy TM.

Hence if we hit this, we barf and kill the task as something's gone horribly wrong.

---
 arch/powerpc/kernel/exceptions-64s.S |   19 +++++++++++++++++++
 arch/powerpc/kernel/traps.c          |   21 +++++++++++++++++++++
 2 files changed, 40 insertions(+)
arch/powerpc/kernel/exceptions-64s.S +19 -0
index 7a1c87c..d0cc657 100644
@@ -313,6 +313,9 @@ vsx_unavailable_pSeries_1:
 	. = 0xf40
 	b	vsx_unavailable_pSeries
 
+	. = 0xf60
+	b	tm_unavailable_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_EXCEPTION_HV(0x1200, 0x1202, cbe_system_error)
 	KVM_HANDLER_SKIP(PACA_EXGEN, EXC_HV, 0x1202)
@@ -528,6 +531,8 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_206)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf20)
 	STD_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf40)
+	STD_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
+	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf60)
 
 /*
  * An interrupt came in while soft-disabled. We set paca->irq_happened, then:
@@ -817,6 +822,10 @@ vsx_unavailable_relon_pSeries_1:
 	. = 0x4f40
 	b	vsx_unavailable_relon_pSeries
 
+tm_unavailable_relon_pSeries_1:
+	. = 0x4f60
+	b	tm_unavailable_relon_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_RELON_EXCEPTION_HV(0x5200, 0x1202, cbe_system_error)
 #endif /* CONFIG_CBE_RAS */
@@ -1177,6 +1186,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 	b	.ret_from_except
 
 	.align	7
+	.globl tm_unavailable_common
+tm_unavailable_common:
+	EXCEPTION_PROLOG_COMMON(0xf60, PACA_EXGEN)
+	bl	.save_nvgprs
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.tm_unavailable_exception
+	b	.ret_from_except
+
+	.align	7
 	.globl	__end_handlers
 __end_handlers:
 
@@ -1195,6 +1213,7 @@ __end_handlers:
 	STD_RELON_EXCEPTION_PSERIES(., 0xf00, performance_monitor)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf20, altivec_unavailable)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
+	STD_RELON_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
 
 #if defined(CONFIG_PPC_PSERIES) || defined(CONFIG_PPC_POWERNV)
 /*
arch/powerpc/kernel/traps.c +21 -0
index 2a62f54..dda0517 100644
@@ -1171,6 +1171,27 @@ void vsx_unavailable_exception(struct pt_regs *regs)
 	die("Unrecoverable VSX Unavailable Exception", regs, SIGABRT);
 }
 
+void tm_unavailable_exception(struct pt_regs *regs)
+{
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	/* Currently we never expect a TMU exception.  Catch
+	 * this and kill the process!
+	 */
+	printk(KERN_EMERG "Unexpected TM unavailable exception at %lx "
+	       "(msr %lx)\n",
+	       regs->nip, regs->msr);
+
+	if (user_mode(regs)) {
+		_exception(SIGILL, regs, ILL_ILLOPC, regs->nip);
+		return;
+	}
+
+	die("Unexpected TM unavailable exception", regs, SIGABRT);
+}
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.10.4
#28 lore raw

PATCH 11/17 powerpc: Assembler routines for FP/VSX/VMX unavailable during a transaction

In reply to Michael Neuling

We do lazy FP but not lazy TM (ie. userspace starts with MSR TM=1 FP=0). Hence if userspace does an FP instruction during a transaction, we'll take an fp unavailable exception.

This adds functions needed to handle this case. We have to inject the current FP state into the checkpoint so that the hardware can decide what to do with the transaction. We can't inject only the FP so we have to do a full treclaim and recheckpoint to inject just the FP state. This will cause the transaction to be marked as aborted by the hardware.

This just add the routines needed to do this for FP, VMX and VSX. It doesn't hook them into the rest of the code yet.

---
 arch/powerpc/kernel/traps.c |   95 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 95 insertions(+)
arch/powerpc/kernel/traps.c +95 -0
index dda0517..311770d 100644
@@ -58,6 +58,7 @@
 #include <asm/rio.h>
 #include <asm/fadump.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 
 #if defined(CONFIG_DEBUGGER) || defined(CONFIG_KEXEC)
@@ -1192,6 +1193,100 @@ void tm_unavailable_exception(struct pt_regs *regs)
 	die("Unexpected TM unavailable exception", regs, SIGABRT);
 }
 
+#ifdef CONFIG_TRANSACTIONAL_MEM
+
+extern void do_load_up_fpu(struct pt_regs *regs);
+
+void fp_unavailable_tm(struct pt_regs *regs)
+{
+	/* Note:  This does not handle any kind of FP laziness. */
+
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	TM_DEBUG("FP Unavailable trap whilst transactional at 0x%lx, MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+
+        /* We can only have got here if the task started using FP after
+         * beginning the transaction.  So, the transactional regs are just a
+         * copy of the checkpointed ones.  But, we still need to recheckpoint
+         * as we're enabling FP for the process; it will return, abort the
+         * transaction, and probably retry but now with FP enabled.  So the
+         * checkpointed FP registers need to be loaded.
+	 */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	/* Reclaim didn't save out any FPRs to transact_fprs. */
+
+	/* Enable FP for the task: */
+	regs->msr |= (MSR_FP | current->thread.fpexc_mode);
+
+	/* This loads and recheckpoints the FP registers from
+	 * thread.fpr[].  They will remain in registers after the
+	 * checkpoint so we don't need to reload them after.
+	 */
+	tm_recheckpoint(&current->thread, regs->msr);
+}
+
+#ifdef CONFIG_ALTIVEC
+extern void do_load_up_altivec(struct pt_regs *regs);
+
+void altivec_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This function operates
+	 * the same way.
+	 */
+
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	TM_DEBUG("Vector Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	regs->msr |= MSR_VEC;
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vr = 1;
+}
+#endif
+
+#ifdef CONFIG_VSX
+void vsx_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This works similarly,
+	 * though we're loading both FP and VEC registers in here.
+	 *
+	 * If FP isn't in use, load FP regs.  If VEC isn't in use, load VEC
+	 * regs.  Either way, set MSR_VSX.
+	 */
+
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	TM_DEBUG("VSX Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+
+	tm_enable();
+	/* This reclaims FP and/or VR regs if they're already enabled */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+
+	regs->msr |= MSR_VEC | MSR_FP | current->thread.fpexc_mode |
+		MSR_VSX;
+	/* This loads & recheckpoints FP and VRs. */
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vsr = 1;
+}
+#endif
+#endif /* CONFIG_TRANSACTIONAL_MEM */
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.10.4
#29 lore raw

PATCH 12/17 powerpc: Hook in new transactional memory code

In reply to Michael Neuling

This hooks the new transactional memory code into context switching, FP/VMX/VMX unavailable and exception return.

---
 arch/powerpc/kernel/exceptions-64s.S |   48 ++++++++++++++++++++++++++++++++--
 arch/powerpc/kernel/fpu.S            |    1 -
 arch/powerpc/kernel/process.c        |   15 +++++++++--
 arch/powerpc/kernel/traps.c          |   32 +++++++++++++++++++++++
 arch/powerpc/kernel/vector.S         |    1 -
 arch/powerpc/mm/hash_utils_64.c      |   13 +++++++++
 6 files changed, 104 insertions(+), 6 deletions(-)
arch/powerpc/kernel/exceptions-64s.S +46 -2
index d0cc657..c4a20d1 100644
@@ -1147,9 +1147,24 @@ fp_unavailable_common:
 	addi	r3,r1,STACK_FRAME_OVERHEAD
 	bl	.kernel_fp_unavailable_exception
 	BUG_OPCODE
-1:	bl	.load_up_fpu
+1:
+#ifdef CONFIG_TRANSACTIONAL_MEM
+BEGIN_FTR_SECTION
+	srdi	r0, r12, MSR_TS_LG
+	andi.	r0, r0, 3
+	bne-	2f
+END_FTR_SECTION_IFSET(CPU_FTR_TM)
+#endif
+	bl	.load_up_fpu
+	std     r12,_MSR(r1)
 	b	fast_exception_return
-
+#ifdef CONFIG_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.fp_unavailable_tm
+	b	.ret_from_except
+#endif
 	.align	7
 	.globl altivec_unavailable_common
 altivec_unavailable_common:
@@ -1157,8 +1172,23 @@ altivec_unavailable_common:
 #ifdef CONFIG_ALTIVEC
 BEGIN_FTR_SECTION
 	beq	1f
+#ifdef CONFIG_TRANSACTIONAL_MEM
+  BEGIN_FTR_SECTION_NESTED(69)
+	srdi	r0, r12, MSR_TS_LG
+	andi.	r0, r0, 3
+	bne-	2f
+  END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
+#endif
 	bl	.load_up_altivec
+	std	r12,_MSR(r1)
 	b	fast_exception_return
+#ifdef CONFIG_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.altivec_unavailable_tm
+	b	.ret_from_except
+#endif
 1:
 END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
 #endif
@@ -1175,7 +1205,21 @@ vsx_unavailable_common:
 #ifdef CONFIG_VSX
 BEGIN_FTR_SECTION
 	beq	1f
+#ifdef CONFIG_TRANSACTIONAL_MEM
+  BEGIN_FTR_SECTION_NESTED(69)
+	srdi	r0, r12, MSR_TS_LG
+	andi.	r0, r0, 3
+	bne-	2f
+  END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
+#endif
 	b	.load_up_vsx
+#ifdef CONFIG_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.vsx_unavailable_tm
+	b	.ret_from_except
+#endif
 1:
 END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 #endif
arch/powerpc/kernel/fpu.S +0 -1
index 6ab0e87..08b6a12f 100644
@@ -170,7 +170,6 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 	lwz	r4,THREAD_FPEXC_MODE(r5)
 	ori	r12,r12,MSR_FP
 	or	r12,r12,r4
-	std	r12,_MSR(r1)
 #endif
 	lfd	fr0,THREAD_FPSCR(r5)
 	MTFSF_L(fr0)
arch/powerpc/kernel/process.c +13 -2
index 14fabd1..bccc81b 100644
@@ -510,7 +510,7 @@ out_and_saveregs:
 	tm_save_sprs(thr);
 }
 
-static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+static inline void tm_recheckpoint_new_task(struct task_struct *new)
 {
 	unsigned long msr;
 
@@ -585,6 +585,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	struct ppc64_tlb_batch *batch;
 #endif
 
+	__switch_to_tm(prev);
+
 #ifdef CONFIG_SMP
 	/* avoid complexity of lazy save/restore of fpu
 	 * by just saving it every time we switch out if
@@ -700,6 +702,9 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	 * of sync. Hard disable here.
 	 */
 	hard_irq_disable();
+
+	tm_recheckpoint_new_task(new);
+
 	last = _switch(old_thread, new_thread);
 
 #ifdef CONFIG_PPC_BOOK3S_64
@@ -1075,7 +1080,6 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 		regs->msr = MSR_USER32;
 	}
 #endif
-
 	discard_lazy_cpu_state();
 #ifdef CONFIG_VSX
 	current->thread.used_vsr = 0;
@@ -1095,6 +1099,13 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.spefscr = 0;
 	current->thread.used_spe = 0;
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	if (cpu_has_feature(CPU_FTR_TM))
+		regs->msr |= MSR_TM;
+	current->thread.tm_tfhar = 0;
+	current->thread.tm_texasr = 0;
+	current->thread.tm_tfiar = 0;
+#endif /* CONFIG_TRANSACTIONAL_MEM */
 }
 
 #define PR_FP_ALL_EXCEPT (PR_FP_EXC_DIV | PR_FP_EXC_OVF | PR_FP_EXC_UND \
arch/powerpc/kernel/traps.c +32 -0
index 311770d..486712b 100644
@@ -1032,6 +1032,38 @@ void __kprobes program_check_exception(struct pt_regs *regs)
 		_exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip);
 		return;
 	}
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	if (reason & REASON_TM) {
+		/* This is a TM "Bad Thing Exception" program check.
+		 * This occurs when:
+		 * -  An rfid/hrfid/mtmsrd attempts to cause an illegal
+		 *    transition in TM states.
+		 * -  A trechkpt is attempted when transactional.
+		 * -  A treclaim is attempted when non transactional.
+		 * -  A tend is illegally attempted.
+		 * -  writing a TM SPR when transactional.
+		 */
+		if (!user_mode(regs) &&
+		    report_bug(regs->nip, regs) == BUG_TRAP_TYPE_WARN) {
+			regs->nip += 4;
+			return;
+		}
+		/* If usermode caused this, it's done something illegal and
+		 * gets a SIGILL slap on the wrist.  We call it an illegal
+		 * operand to distinguish from the instruction just being bad
+		 * (e.g. executing a 'tend' on a CPU without TM!); it's an
+		 * illegal /placement/ of a valid instruction.
+		 */
+		if (user_mode(regs)) {
+			_exception(SIGILL, regs, ILL_ILLOPN, regs->nip);
+			return;
+		} else {
+			printk(KERN_EMERG "Unexpected TM Bad Thing exception "
+			       "at %lx (msr 0x%x)\n", regs->nip, reason);
+			die("Unrecoverable exception", regs, SIGABRT);
+		}
+	}
+#endif
 
 	/* We restore the interrupt state now */
 	if (!arch_irq_disabled_regs(regs))
arch/powerpc/kernel/vector.S +0 -1
index 330fc8c..9b47306 100644
@@ -123,7 +123,6 @@ _GLOBAL(load_up_altivec)
 	ld	r4,PACACURRENT(r13)
 	addi	r5,r4,THREAD		/* Get THREAD */
 	oris	r12,r12,MSR_VEC@h
-	std	r12,_MSR(r1)
 #endif
 	li	r4,1
 	li	r10,THREAD_VSCR
arch/powerpc/mm/hash_utils_64.c +13 -0
index 3a292be..03a4c4a 100644
@@ -55,6 +55,7 @@
 #include <asm/code-patching.h>
 #include <asm/fadump.h>
 #include <asm/firmware.h>
+#include <asm/tm.h>
 
 #ifdef DEBUG
 #define DBG(fmt...) udbg_printf(fmt)
@@ -1171,6 +1172,18 @@ void flush_hash_page(unsigned long vpn, real_pte_t pte, int psize, int ssize,
 		DBG_LOW(" sub %ld: hash=%lx, hidx=%lx\n", index, slot, hidx);
 		ppc_md.hpte_invalidate(slot, vpn, psize, ssize, local);
 	} pte_iterate_hashed_end();
+
+#ifdef CONFIG_TRANSACTIONAL_MEM
+	/* Transactions are not aborted by tlbiel, only tlbie.
+	 * Without, syncing a page back to a block device w/ PIO could pick up
+	 * transactional data (bad!) so we force an abort here.  Before the
+	 * sync the page will be made read-only, which will flush_hash_page.
+	 */
+	if (local && cpu_has_feature(CPU_FTR_TM)) {
+		tm_enable();
+		tm_abort(TM_CAUSE_TLBI);
+	}
+#endif
 }
 
 void flush_hash_range(unsigned long number, int local)
Signature · 1 line
1.7.10.4
#30 lore raw

PATCH 13/17 powerpc: Add new transactional memory state to the signal context

In reply to Michael Neuling

This adds the new transactional memory archtected state to the signal context in both 32 and 64 bit.

---
 arch/powerpc/include/asm/reg.h  |    1 +
 arch/powerpc/kernel/signal.h    |    8 +
 arch/powerpc/kernel/signal_32.c |  500 ++++++++++++++++++++++++++++++++++++++-
 arch/powerpc/kernel/signal_64.c |  337 +++++++++++++++++++++++++-
 4 files changed, 830 insertions(+), 16 deletions(-)
arch/powerpc/include/asm/reg.h +1 -0
index d19581ae4..41894dc 100644
@@ -120,6 +120,7 @@
 #define TM_CAUSE_FAC_UNAV	0xfa
 #define TM_CAUSE_SYSCALL	0xf9 /* Persistent */
 #define TM_CAUSE_MISC		0xf6
+#define TM_CAUSE_SIGNAL		0xf4
 
 #if defined(CONFIG_PPC_BOOK3S_64)
 #define MSR_64BIT	MSR_SF
arch/powerpc/kernel/signal.h +8 -0
index e00acb4..ec84c90 100644
@@ -25,13 +25,21 @@ extern int handle_rt_signal32(unsigned long sig, struct k_sigaction *ka,
 
 extern unsigned long copy_fpr_to_user(void __user *to,
 				      struct task_struct *task);
+extern unsigned long copy_transact_fpr_to_user(void __user *to,
+					       struct task_struct *task);
 extern unsigned long copy_fpr_from_user(struct task_struct *task,
 					void __user *from);
+extern unsigned long copy_transact_fpr_from_user(struct task_struct *task,
+						 void __user *from);
 #ifdef CONFIG_VSX
 extern unsigned long copy_vsx_to_user(void __user *to,
 				      struct task_struct *task);
+extern unsigned long copy_transact_vsx_to_user(void __user *to,
+					       struct task_struct *task);
 extern unsigned long copy_vsx_from_user(struct task_struct *task,
 					void __user *from);
+extern unsigned long copy_transact_vsx_from_user(struct task_struct *task,
+						 void __user *from);
 #endif
 
 #ifdef CONFIG_PPC64
arch/powerpc/kernel/signal_32.c +487 -13

608 lines not on this page · raw message

arch/powerpc/kernel/signal_64.c +334 -3

389 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#31 lore raw

PATCH 14/17 powerpc: Add transactional memory to POWER8 cpu features

In reply to Michael Neuling
---
 arch/powerpc/include/asm/cputable.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/powerpc/include/asm/cputable.h +1 -1
index 71a498b..5ab77f4 100644
@@ -421,7 +421,7 @@ extern const char *powerpc_base_platform;
 	    CPU_FTR_DSCR | CPU_FTR_SAO  | \
 	    CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \
 	    CPU_FTR_ICSWX | CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \
-	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_DAWR)
+	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_DAWR | CPU_FTR_TM_COMP)
 #define CPU_FTRS_CELL	(CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \
 	    CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
Signature · 1 line
1.7.10.4
#32 lore raw

PATCH 15/17 powerpc: Add config option for transactional memory

In reply to Michael Neuling

Kconfig option for transactional memory on powerpc.

---
 arch/powerpc/Kconfig |    8 ++++++++
 1 file changed, 8 insertions(+)
arch/powerpc/Kconfig +8 -0
index 684fa64..349ed12 100644
@@ -309,6 +309,14 @@ config MATH_EMULATION
 	  unit, which will allow programs that use floating-point
 	  instructions to run.
 
+config TRANSACTIONAL_MEM
+       bool "Transactional Memory support"
+       depends on PPC64
+       depends on SMP
+       default n
+       ---help---
+         Support user-mode Transactional Memory.
+
 config 8XX_MINIMAL_FPEMU
 	bool "Minimal math emulation for 8xx"
 	depends on 8xx && !MATH_EMULATION
Signature · 1 line
1.7.10.4
#33 lore raw

PATCH 16/17 powerpc: Add transactional memory to pseries and ppc64 defconfigs

In reply to Michael Neuling
---
 arch/powerpc/configs/ppc64_defconfig   |    2 ++
 arch/powerpc/configs/pseries_defconfig |    2 ++
 2 files changed, 4 insertions(+)
arch/powerpc/configs/ppc64_defconfig +2 -0
index 6cbdeb4..01e2e86 100644
@@ -49,6 +49,8 @@ CONFIG_CPU_FREQ_GOV_USERSPACE=y
 CONFIG_CPU_FREQ_PMAC64=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_TRANSACTIONAL_MEM=y
+CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
 CONFIG_MEMORY_HOTREMOVE=y
arch/powerpc/configs/pseries_defconfig +2 -0
index 04fa7f2..a50484f 100644
@@ -43,6 +43,8 @@ CONFIG_RTAS_FLASH=m
 CONFIG_IBMEBUS=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_TRANSACTIONAL_MEM=y
+CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
 CONFIG_MEMORY_HOTPLUG=y
Signature · 1 line
1.7.10.4
#34 lore raw

PATCH 17/17 powerpc: Documentation for transactional memory on powerpc

In reply to Michael Neuling
---
 Documentation/powerpc/transactional_memory.txt |  175 ++++++++++++++++++++++++
 1 file changed, 175 insertions(+)
 create mode 100644 Documentation/powerpc/transactional_memory.txt
Documentation/powerpc/transactional_memory.txt new file +175 -0
new file mode 100644
index 0000000..c907be4
@@ -0,0 +1,175 @@
+Transactional Memory support
+============================
+
+POWER kernel support for this feature is currently limited to supporting
+its use by user programs.  It is not currently used by the kernel itself.
+
+This file aims to sum up how it is supported by Linux and what behaviour you
+can expect from your user programs.
+
+
+Basic overview
+==============
+
+Hardware Transactional Memory is supported on POWER8 processors, and is a
+feature that enables a different form of atomic memory access.  Several new
+instructions are presented to delimit transactions; transactions are
+guaranteed to either complete atomically or roll back and undo any partial
+changes.
+
+A simple transaction looks like this:
+
+begin_move_money:
+  tbegin
+  beq   abort_handler
+
+  ld    r4, SAVINGS_ACCT(r3)
+  ld    r5, CURRENT_ACCT(r3)
+  subi  r5, r5, 1
+  addi  r4, r4, 1
+  std   r4, SAVINGS_ACCT(r3)
+  std   r5, CURRENT_ACCT(r3)
+
+  tend
+
+  b     continue
+
+abort_handler:
+  ... test for odd failures ...
+
+  /* Retry the transaction if it failed because it conflicted with
+   * someone else: */
+  b     begin_move_money
+
+
+The 'tbegin' instruction denotes the start point, and 'tend' the end point.
+Between these points the processor is in 'Transactional' state; any memory
+references will complete in one go if there are no conflicts with other
+transactional or non-transactional accesses within the system.  In this
+example, the transaction completes as though it were normal straight-line code
+IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an
+atomic move of money from the current account to the savings account has been
+performed.  Even though the normal ld/std instructions are used (note no
+lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be
+updated, or neither will be updated.
+
+If, in the meantime, there is a conflict with the locations accessed by the
+transaction, the transaction will be aborted by the CPU.  Register and memory
+state will roll back to that at the 'tbegin', and control will continue from
+'tbegin+4'.  The branch to abort_handler will be taken this second time; the
+abort handler can check the cause of the failure, and retry.
+
+Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR
+and a few other status/flag regs; see the ISA for details.
+
+Causes of transaction aborts
+============================
+
+- Conflicts with cache lines used by other processors
+- Signals
+- Context switches
+- See the ISA for full documentation of everything that will abort transactions.
+
+
+Syscalls
+========
+
+Performing syscalls from within transaction is not recommended, and can lead
+to unpredictable results.
+
+Syscalls do not by design abort transactions, but beware: The kernel code will
+not be running in transactional state.  The effect of syscalls will always
+remain visible, but depending on the call they may abort your transaction as a
+side-effect, read soon-to-be-aborted transactional data that should not remain
+invisible, etc.  If you constantly retry a transaction that constantly aborts
+itself by calling a syscall, you'll have a livelock & make no progress.
+
+Simple syscalls (e.g. sigprocmask()) "could" be OK.  Even things like write()
+from, say, printf() should be OK as long as the kernel does not access any
+memory that was accessed transactionally.
+
+Consider any syscalls that happen to work as debug-only -- not recommended for
+production use.  Best to queue them up till after the transaction is over.
+
+
+Signals
+=======
+
+Delivery of signals (both sync and async) during transactions provides a second
+thread state (ucontext/mcontext) to represent the second transactional register
+state.  Signal delivery 'treclaim's to capture both register states, so signals
+abort transactions.  The usual ucontext_t passed to the signal handler
+represents the checkpointed/original register state; the signal appears to have
+arisen at 'tbegin+4'.
+
+If the sighandler ucontext has uc_link set, a second ucontext has been
+delivered.  For future compatibility the MSR.TS field should be checked to
+determine the transactional state -- if so, the second ucontext in uc->uc_link
+represents the active transactional registers at the point of the signal.
+
+For 64-bit processes, uc->uc_mcontext.regs->msr is a full 64-bit MSR and its TS
+field shows the transactional mode.
+
+For 32-bit processes, the mcontext's MSR register is only 32 bits; the top 32
+bits are stored in the MSR of the second ucontext, i.e. in
+uc->uc_link->uc_mcontext.regs->msr.  The top word contains the transactional
+state TS.
+
+However, basic signal handlers don't need to be aware of transactions
+and simply returning from the handler will deal with things correctly:
+
+Transaction-aware signal handlers can read the transactional register state
+from the second ucontext.  This will be necessary for crash handlers to
+determine, for example, the address of the instruction causing the SIGSEGV.
+
+Example signal handler:
+
+    void crash_handler(int sig, siginfo_t *si, void *uc)
+    {
+      ucontext_t *ucp = uc;
+      ucontext_t *transactional_ucp = ucp->uc_link;
+
+      if (ucp_link) {
+        u64 msr = ucp->uc_mcontext.regs->msr;
+        /* May have transactional ucontext! */
+#ifndef __powerpc64__
+        msr |= ((u64)transactional_ucp->uc_mcontext.regs->msr) << 32;
+#endif
+        if (MSR_TM_ACTIVE(msr)) {
+           /* Yes, we crashed during a transaction.  Oops. */
+   fprintf(stderr, "Transaction to be restarted at 0x%llx, but "
+                           "crashy instruction was at 0x%llx\n",
+                           ucp->uc_mcontext.regs->nip,
+                           transactional_ucp->uc_mcontext.regs->nip);
+        }
+      }
+
+      fix_the_problem(ucp->dar);
+    }
+
+
+Failure cause codes used by kernel
+==================================
+
+These are defined in <asm/reg.h>, and distinguish different reasons why the
+kernel aborted a transaction:
+
+ TM_CAUSE_RESCHED       Thread was rescheduled.
+ TM_CAUSE_FAC_UNAV      FP/VEC/VSX unavailable trap.
+ TM_CAUSE_SYSCALL       Currently unused; future syscalls that must abort
+                        transactions for consistency will use this.
+ TM_CAUSE_SIGNAL        Signal delivered.
+ TM_CAUSE_MISC          Currently unused.
+
+These can be checked by the user program's abort handler as TEXASR[0:7].
+
+
+GDB
+===
+
+GDB and ptrace are not currently TM-aware.  If one stops during a transaction,
+it looks like the transaction has just started (the checkpointed state is
+presented).  The transaction cannot then be continued and will take the failure
+handler route.  Furthermore, the transactional 2nd register state will be
+inaccessible.  GDB can currently be used on programs using TM, but not sensibly
+in parts within transactions.
Signature · 1 line
1.7.10.4
#35 lore raw

cover powerpc: Hardware transactional memory support for POWER8

In reply to Michael Neuling

POWER8 implements hardware transactional memory support. This patch series adds kernel support so that user programs can use this hardware transactional memory and the new state is properly context switched. It is not currently used by the kernel itself.

This patch series was originally developed by Matt Evans.

v3 changes:
Incorporate review from benh (offline conversation):
- Improve IRQ disabling for tm_reclaim
- Improve IRQ disabling for early fp/vmx/vsx unavail handling
- Change CONFIG option to add PPC (now PPC_TRANSACTIONAL_MEM)
- strip some unneeded #defines
- micro optimisation to asm in early fp/vmx/vsx

      -       srdi    r0, r12, MSR_TS_LG
      -       andi.   r0, r0, 3
      +       rldicl. r0, r12, (64-MSR_TS_LG), (64-2)

- do tabort on unmap only when current is using TM
- removed unneeded patch which moved when writing MSR back to the stack

v2 changes:
Adds signal handling
Removes some code for detecting when to restore all register or not Added tabort for local tlb invalidates
Rebased on benh's next tree

Basic overview of a POWER8 hardware transaction memory
=====================================================
Hardware transactional memory is a feature that enables a different form of atomic memory access. Several new instructions are presented to delimit transactions; transactions are guaranteed to either complete atomically or roll back and undo any partial changes.

A simple transaction looks like this:

begin_move_money:
tbegin
beq abort_handler

  ld    r4, SAVINGS_ACCT(r3)
  ld    r5, CURRENT_ACCT(r3)
  subi  r5, r5, 1
  addi  r4, r4, 1
  std   r4, SAVINGS_ACCT(r3)
  std   r5, CURRENT_ACCT(r3)

tend

b continue

abort_handler:
... test for odd failures ...

/* Retry the transaction if it failed because it conflicted with
* someone else: */
b begin_move_money

The 'tbegin' instruction denotes the start point, and 'tend' the end point. Between these points the processor is in 'Transactional' state; any memory references will complete in one go if there are no conflicts with other transactional or non-transactional accesses within the system. In this example, the transaction completes as though it were normal straight-line code IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an atomic move of money from the current account to the savings account has been performed. Even though the normal ld/std instructions are used (note no lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be updated, or neither will be updated.

If, in the meantime, there is a conflict with the locations accessed by the transaction, the transaction will be aborted by the CPU. Register and memory state will roll back to that at the 'tbegin', and control will continue from 'tbegin+4'. The branch to abort_handler will be taken this second time; the abort handler can check the cause of the failure, and retry.

Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR and a few other status/flag regs;

A preliminary ISA for TM can be found here:
https://www.power.org/documentation/power-isa-transactional-memory/

Signature · 1 line
1.7.10.4
#36 lore raw

PATCH 1/17 powerpc: Add new CPU feature bit for transactional memory

---
 arch/powerpc/include/asm/cputable.h |    7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/include/asm/cputable.h +7 -0
index 5f1938f..3636081 100644
@@ -220,6 +220,13 @@ extern const char *powerpc_base_platform;
 #define PPC_FEATURE_HAS_EFP_DOUBLE_COMP 0
 #endif
 
+/* We only set the TM feature if the kernel was compiled with TM supprt */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+#define CPU_FTR_TM_COMP		CPU_FTR_TM
+#else
+#define CPU_FTR_TM_COMP		0
+#endif
+
 /* We need to mark all pages as being coherent if we're SMP or we have a
  * 74[45]x and an MPC107 host bridge. Also 83xx and PowerQUICC II
  * require it for PCI "streaming/prefetch" to work properly.
Signature · 1 line
1.7.10.4
#37 lore raw

PATCH 2/17 powerpc: Add new instructions for transactional memory

In reply to Michael Neuling

Here we define the new instructions we need for transactional memory in the kernel. This is so we can support compiling with binutils that don't support the new transactional memory instructions.

Transactional memory results in two sets of architected state (GPRs/VSRs etc).

treclaim allows us to read the checkpointed state (from the tbegin) so that we can store it away on a context switch. It does this by overwriting the exiting architected state, so you have to save that away before you treclaim. treclaim will also abort a transaction, so you can give a register value which contains an abort reason.

trecheckpoint allows us to inject into the checkpointed state as if it were at the tbegin. It does this by copying the current architected state into the checkpointed state.

---
 arch/powerpc/include/asm/ppc-opcode.h |   10 ++++++++++
 1 file changed, 10 insertions(+)
arch/powerpc/include/asm/ppc-opcode.h +10 -0
index 0fd1928..8752bc8 100644
@@ -129,6 +129,9 @@
 #define PPC_INST_TLBSRX_DOT		0x7c0006a5
 #define PPC_INST_XXLOR			0xf0000510
 #define PPC_INST_XVCPSGNDP		0xf0000780
+#define PPC_INST_TRECHKPT		0x7c0007dd
+#define PPC_INST_TRECLAIM		0x7c00075d
+#define PPC_INST_TABORT			0x7c00071d
 
 #define PPC_INST_NAP			0x4c000364
 #define PPC_INST_SLEEP			0x4c0003a4
@@ -294,4 +297,11 @@
 #define PPC_NAP			stringify_in_c(.long PPC_INST_NAP)
 #define PPC_SLEEP		stringify_in_c(.long PPC_INST_SLEEP)
 
+/* Transactional memory instructions */
+#define TRECHKPT		stringify_in_c(.long PPC_INST_TRECHKPT)
+#define TRECLAIM(r)		stringify_in_c(.long PPC_INST_TRECLAIM \
+					       | __PPC_RA(r))
+#define TABORT(r)		stringify_in_c(.long PPC_INST_TABORT \
+					       | __PPC_RA(r))
+
 #endif /* _ASM_POWERPC_PPC_OPCODE_H */
Signature · 1 line
1.7.10.4
#38 lore raw

PATCH 3/17 powerpc: Add additional state needed for transactional memory to thread struct

In reply to Michael Neuling

Set of new archtected state for saving away on context switch.

---
 arch/powerpc/include/asm/processor.h |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
arch/powerpc/include/asm/processor.h +28 -0
index 42ac53c..fc41ab3 100644
@@ -246,6 +246,34 @@ struct thread_struct {
 	unsigned long	spefscr;	/* SPE & eFP status */
 	int		used_spe;	/* set if process has used spe */
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	u64		tm_tfhar;	/* Transaction fail handler addr */
+	u64		tm_texasr;	/* Transaction exception & summary */
+	u64		tm_tfiar;	/* Transaction fail instr address reg */
+	unsigned long	tm_orig_msr;	/* Thread's MSR on ctx switch */
+	struct pt_regs	ckpt_regs;	/* Checkpointed registers */
+
+	/*
+	 * Transactional FP and VSX 0-31 register set.
+	 * NOTE: the sense of these is the opposite of the integer ckpt_regs!
+	 *
+	 * When a transaction is active/signalled/scheduled etc., *regs is the
+	 * most recent set of/speculated GPRs with ckpt_regs being the older
+	 * checkpointed regs to which we roll back if transaction aborts.
+	 *
+	 * However, fpr[] is the checkpointed 'base state' of FP regs, and
+	 * transact_fpr[] is the new set of transactional values.
+	 * VRs work the same way.
+	 */
+	double		transact_fpr[32][TS_FPRWIDTH];
+	struct {
+		unsigned int pad;
+		unsigned int val;	/* Floating point status */
+	} transact_fpscr;
+	vector128	transact_vr[32] __attribute__((aligned(16)));
+	vector128	transact_vscr __attribute__((aligned(16)));
+	unsigned long	transact_vrsave;
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 #ifdef CONFIG_KVM_BOOK3S_32_HANDLER
 	void*		kvm_shadow_vcpu; /* KVM internal data */
 #endif /* CONFIG_KVM_BOOK3S_32_HANDLER */
Signature · 1 line
1.7.10.4
#39 lore raw

PATCH 4/17 powerpc: New macros for transactional memory support

In reply to Michael Neuling

This adds new macros for saving and restoring checkpointed architected state from and to the thread_struct.

It also adds some debugging macros for when your brain explodes trying to debug your transactional memory enabled kernel.

---
 arch/powerpc/include/asm/ppc_asm.h   |   83 ++++++++++++++++++++++++++++++++++
 arch/powerpc/include/asm/processor.h |    1 +
 arch/powerpc/kernel/asm-offsets.c    |   23 ++++++++++
 arch/powerpc/kernel/fpu.S            |   12 +++++
 arch/powerpc/kernel/process.c        |    7 +++
 arch/powerpc/kernel/traps.c          |    8 ++++
 6 files changed, 134 insertions(+)
arch/powerpc/include/asm/ppc_asm.h +83 -0
index c2d0e58..54219ce 100644
@@ -123,6 +123,89 @@ END_FW_FTR_SECTION_IFSET(FW_FEATURE_SPLPAR)
 #define REST_16VRS(n,b,base)	REST_8VRS(n,b,base); REST_8VRS(n+8,b,base)
 #define REST_32VRS(n,b,base)	REST_16VRS(n,b,base); REST_16VRS(n+16,b,base)
 
+/* Save/restore FPRs, VRs and VSRs from their checkpointed backups in
+ * thread_struct:
+ */
+#define SAVE_FPR_TRANSACT(n, base)	stfd n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define SAVE_2FPRS_TRANSACT(n, base)	SAVE_FPR_TRANSACT(n, base);	\
+					SAVE_FPR_TRANSACT(n+1, base)
+#define SAVE_4FPRS_TRANSACT(n, base)	SAVE_2FPRS_TRANSACT(n, base);	\
+					SAVE_2FPRS_TRANSACT(n+2, base)
+#define SAVE_8FPRS_TRANSACT(n, base)	SAVE_4FPRS_TRANSACT(n, base);	\
+					SAVE_4FPRS_TRANSACT(n+4, base)
+#define SAVE_16FPRS_TRANSACT(n, base)	SAVE_8FPRS_TRANSACT(n, base);	\
+					SAVE_8FPRS_TRANSACT(n+8, base)
+#define SAVE_32FPRS_TRANSACT(n, base)	SAVE_16FPRS_TRANSACT(n, base);	\
+					SAVE_16FPRS_TRANSACT(n+16, base)
+
+#define REST_FPR_TRANSACT(n, base)	lfd	n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define REST_2FPRS_TRANSACT(n, base)	REST_FPR_TRANSACT(n, base);	\
+					REST_FPR_TRANSACT(n+1, base)
+#define REST_4FPRS_TRANSACT(n, base)	REST_2FPRS_TRANSACT(n, base);	\
+					REST_2FPRS_TRANSACT(n+2, base)
+#define REST_8FPRS_TRANSACT(n, base)	REST_4FPRS_TRANSACT(n, base);	\
+					REST_4FPRS_TRANSACT(n+4, base)
+#define REST_16FPRS_TRANSACT(n, base)	REST_8FPRS_TRANSACT(n, base);	\
+					REST_8FPRS_TRANSACT(n+8, base)
+#define REST_32FPRS_TRANSACT(n, base)	REST_16FPRS_TRANSACT(n, base);	\
+					REST_16FPRS_TRANSACT(n+16, base)
+
+
+#define SAVE_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					stvx n,b,base
+#define SAVE_2VRS_TRANSACT(n,b,base)	SAVE_VR_TRANSACT(n,b,base);	\
+					SAVE_VR_TRANSACT(n+1,b,base)
+#define SAVE_4VRS_TRANSACT(n,b,base)	SAVE_2VRS_TRANSACT(n,b,base);	\
+					SAVE_2VRS_TRANSACT(n+2,b,base)
+#define SAVE_8VRS_TRANSACT(n,b,base)	SAVE_4VRS_TRANSACT(n,b,base);	\
+					SAVE_4VRS_TRANSACT(n+4,b,base)
+#define SAVE_16VRS_TRANSACT(n,b,base)	SAVE_8VRS_TRANSACT(n,b,base);	\
+					SAVE_8VRS_TRANSACT(n+8,b,base)
+#define SAVE_32VRS_TRANSACT(n,b,base)	SAVE_16VRS_TRANSACT(n,b,base);	\
+					SAVE_16VRS_TRANSACT(n+16,b,base)
+
+#define REST_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					lvx n,b,base
+#define REST_2VRS_TRANSACT(n,b,base)	REST_VR_TRANSACT(n,b,base);	\
+					REST_VR_TRANSACT(n+1,b,base)
+#define REST_4VRS_TRANSACT(n,b,base)	REST_2VRS_TRANSACT(n,b,base);	\
+					REST_2VRS_TRANSACT(n+2,b,base)
+#define REST_8VRS_TRANSACT(n,b,base)	REST_4VRS_TRANSACT(n,b,base);	\
+					REST_4VRS_TRANSACT(n+4,b,base)
+#define REST_16VRS_TRANSACT(n,b,base)	REST_8VRS_TRANSACT(n,b,base);	\
+					REST_8VRS_TRANSACT(n+8,b,base)
+#define REST_32VRS_TRANSACT(n,b,base)	REST_16VRS_TRANSACT(n,b,base);	\
+					REST_16VRS_TRANSACT(n+16,b,base)
+
+
+#define SAVE_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					STXVD2X(n,R##base,R##b)
+#define SAVE_2VSRS_TRANSACT(n,b,base)	SAVE_VSR_TRANSACT(n,b,base);	\
+	                                SAVE_VSR_TRANSACT(n+1,b,base)
+#define SAVE_4VSRS_TRANSACT(n,b,base)	SAVE_2VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_2VSRS_TRANSACT(n+2,b,base)
+#define SAVE_8VSRS_TRANSACT(n,b,base)	SAVE_4VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_4VSRS_TRANSACT(n+4,b,base)
+#define SAVE_16VSRS_TRANSACT(n,b,base)	SAVE_8VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_8VSRS_TRANSACT(n+8,b,base)
+#define SAVE_32VSRS_TRANSACT(n,b,base)	SAVE_16VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_16VSRS_TRANSACT(n+16,b,base)
+
+#define REST_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					LXVD2X(n,R##base,R##b)
+#define REST_2VSRS_TRANSACT(n,b,base)	REST_VSR_TRANSACT(n,b,base);    \
+	                                REST_VSR_TRANSACT(n+1,b,base)
+#define REST_4VSRS_TRANSACT(n,b,base)	REST_2VSRS_TRANSACT(n,b,base);	\
+	                                REST_2VSRS_TRANSACT(n+2,b,base)
+#define REST_8VSRS_TRANSACT(n,b,base)	REST_4VSRS_TRANSACT(n,b,base);	\
+	                                REST_4VSRS_TRANSACT(n+4,b,base)
+#define REST_16VSRS_TRANSACT(n,b,base)	REST_8VSRS_TRANSACT(n,b,base);	\
+	                                REST_8VSRS_TRANSACT(n+8,b,base)
+#define REST_32VSRS_TRANSACT(n,b,base)	REST_16VSRS_TRANSACT(n,b,base);	\
+	                                REST_16VSRS_TRANSACT(n+16,b,base)
+
 /* Save the lower 32 VSRs in the thread VSR region */
 #define SAVE_VSR(n,b,base)	li b,THREAD_VSR0+(16*(n));  STXVD2X(n,R##base,R##b)
 #define SAVE_2VSRS(n,b,base)	SAVE_VSR(n,b,base); SAVE_VSR(n+1,b,base)
arch/powerpc/include/asm/processor.h +1 -0
index fc41ab3..7ff9eaa 100644
@@ -152,6 +152,7 @@ typedef struct {
 #define TS_FPROFFSET 0
 #define TS_VSRLOWOFFSET 1
 #define TS_FPR(i) fpr[i][TS_FPROFFSET]
+#define TS_TRANS_FPR(i) transact_fpr[i][TS_FPROFFSET]
 
 struct thread_struct {
 	unsigned long	ksp;		/* Kernel stack pointer */
arch/powerpc/kernel/asm-offsets.c +23 -0
index beddba4..b67e4c3 100644
@@ -125,6 +125,29 @@ int main(void)
 #ifdef CONFIG_PPC_BOOK3S_64
 	DEFINE(THREAD_TAR, offsetof(struct thread_struct, tar));
 #endif
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
+	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
+	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
+	DEFINE(PT_CKPT_REGS, offsetof(struct thread_struct, ckpt_regs));
+	DEFINE(THREAD_TRANSACT_VR0, offsetof(struct thread_struct,
+					 transact_vr[0]));
+	DEFINE(THREAD_TRANSACT_VSCR, offsetof(struct thread_struct,
+					  transact_vscr));
+	DEFINE(THREAD_TRANSACT_VRSAVE, offsetof(struct thread_struct,
+					    transact_vrsave));
+	DEFINE(THREAD_TRANSACT_FPR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+	DEFINE(THREAD_TRANSACT_FPSCR, offsetof(struct thread_struct,
+					   transact_fpscr));
+#ifdef CONFIG_VSX
+	DEFINE(THREAD_TRANSACT_VSR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+#endif
+	/* Local pt_regs on stack for Transactional Memory funcs. */
+	DEFINE(TM_FRAME_SIZE, STACK_FRAME_OVERHEAD +
+	       sizeof(struct pt_regs) + 16);
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 
 	DEFINE(TI_FLAGS, offsetof(struct thread_info, flags));
 	DEFINE(TI_LOCAL_FLAGS, offsetof(struct thread_info, local_flags));
arch/powerpc/kernel/fpu.S +12 -0
index e0ada05..adb1551 100644
@@ -35,6 +35,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 2:	REST_32VSRS(n,c,base);						\
 3:
 
+#define __REST_32FPVSRS_TRANSACT(n,c,base)				\
+BEGIN_FTR_SECTION							\
+	b	2f;							\
+END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
+	REST_32FPRS_TRANSACT(n,base);					\
+	b	3f;							\
+2:	REST_32VSRS_TRANSACT(n,c,base);					\
+3:
+
 #define __SAVE_32FPVSRS(n,c,base)					\
 BEGIN_FTR_SECTION							\
 	b	2f;							\
@@ -45,9 +54,12 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 3:
 #else
 #define __REST_32FPVSRS(n,b,base)	REST_32FPRS(n, base)
+#define __REST_32FPVSRS_TRANSACT(n,b,base)	REST_32FPRS(n, base)
 #define __SAVE_32FPVSRS(n,b,base)	SAVE_32FPRS(n, base)
 #endif
 #define REST_32FPVSRS(n,c,base) __REST_32FPVSRS(n,__REG_##c,__REG_##base)
+#define REST_32FPVSRS_TRANSACT(n,c,base) \
+	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
 /*
arch/powerpc/kernel/process.c +7 -0
index 96e31de..b0a0321 100644
@@ -57,6 +57,13 @@
 #include <linux/kprobes.h>
 #include <linux/kdebug.h>
 
+/* Transactional Memory debug */
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+
 extern unsigned long _get_SP(void);
 
 #ifndef CONFIG_SMP
arch/powerpc/kernel/traps.c +8 -0
index a008cf5..bd5de5d 100644
@@ -78,6 +78,13 @@ EXPORT_SYMBOL(__debugger_break_match);
 EXPORT_SYMBOL(__debugger_fault_handler);
 #endif
 
+/* Transactional Memory trap debug */
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+
 /*
  * Trap & Exception support
  */
@@ -350,6 +357,7 @@ static inline int check_io_access(struct pt_regs *regs)
    exception is in the MSR. */
 #define get_reason(regs)	((regs)->msr)
 #define get_mc_reason(regs)	((regs)->msr)
+#define REASON_TM		0x200000
 #define REASON_FP		0x100000
 #define REASON_ILLEGAL		0x80000
 #define REASON_PRIVILEGED	0x40000
Signature · 1 line
1.7.10.4
#40 lore raw

PATCH 5/17 powerpc: Register defines for various transactional memory registers

In reply to Michael Neuling

Defines for MSR bits and transactional memory related SPRs TFIAR, TEXASR and TEXASRU.

---
 arch/powerpc/include/asm/reg.h |   27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
arch/powerpc/include/asm/reg.h +27 -0
index e09ac51..7844c28 100644
@@ -29,6 +29,10 @@
 #define MSR_SF_LG	63              /* Enable 64 bit mode */
 #define MSR_ISF_LG	61              /* Interrupt 64b mode valid on 630 */
 #define MSR_HV_LG 	60              /* Hypervisor state */
+#define MSR_TS_T_LG	34		/* Trans Mem state: Transactional */
+#define MSR_TS_S_LG	33		/* Trans Mem state: Suspended */
+#define MSR_TS_LG	33		/* Trans Mem state (2 bits) */
+#define MSR_TM_LG	32		/* Trans Mem Available */
 #define MSR_VEC_LG	25	        /* Enable AltiVec */
 #define MSR_VSX_LG	23		/* Enable VSX */
 #define MSR_POW_LG	18		/* Enable Power Management */
@@ -98,6 +102,25 @@
 #define MSR_RI		__MASK(MSR_RI_LG)	/* Recoverable Exception */
 #define MSR_LE		__MASK(MSR_LE_LG)	/* Little Endian */
 
+#define MSR_TM		__MASK(MSR_TM_LG)	/* Transactional Mem Available */
+#define MSR_TS_N	0			/*  Non-transactional */
+#define MSR_TS_S	__MASK(MSR_TS_S_LG)	/*  Transaction Suspended */
+#define MSR_TS_T	__MASK(MSR_TS_T_LG)	/*  Transaction Transactional */
+#define MSR_TS_MASK	(MSR_TS_T | MSR_TS_S)   /* Transaction State bits */
+#define MSR_TM_ACTIVE(x) (((x) & MSR_TS_MASK) != 0) /* Transaction active? */
+#define MSR_TM_TRANSACTIONAL(x)	(((x) & MSR_TS_MASK) == MSR_TS_T)
+#define MSR_TM_SUSPENDED(x)	(((x) & MSR_TS_MASK) == MSR_TS_S)
+
+/* Reason codes describing kernel causes for transaction aborts.  By
+   convention, bit0 is copied to TEXASR[56] (IBM bit 7) which is set if
+   the failure is persistent.
+*/
+#define TM_CAUSE_RESCHED	0xfe
+#define TM_CAUSE_TLBI		0xfc
+#define TM_CAUSE_FAC_UNAV	0xfa
+#define TM_CAUSE_SYSCALL	0xf9 /* Persistent */
+#define TM_CAUSE_MISC		0xf6
+
 #if defined(CONFIG_PPC_BOOK3S_64)
 #define MSR_64BIT	MSR_SF
 
@@ -193,6 +216,10 @@
 #define SPRN_UAMOR	0x9d	/* User Authority Mask Override Register */
 #define SPRN_AMOR	0x15d	/* Authority Mask Override Register */
 #define SPRN_ACOP	0x1F	/* Available Coprocessor Register */
+#define SPRN_TFIAR	0x81	/* Transaction Failure Inst Addr   */
+#define SPRN_TEXASR	0x82	/* Transaction EXception & Summary */
+#define SPRN_TEXASRU	0x83	/* ''	   ''	   ''	 Upper 32  */
+#define SPRN_TFHAR	0x80	/* Transaction Failure Handler Addr */
 #define SPRN_CTRLF	0x088
 #define SPRN_CTRLT	0x098
 #define   CTRL_CT	0xc0000000	/* current thread */
Signature · 1 line
1.7.10.4
#41 lore raw

PATCH 6/17 powerpc: Add transactional memory paca scratch register to show_regs

In reply to Michael Neuling

Add transactional memory paca scratch register to show_regs. This is useful for debugging.

---
 arch/powerpc/include/asm/paca.h   |    1 +
 arch/powerpc/kernel/asm-offsets.c |    1 +
 arch/powerpc/kernel/entry_64.S    |    4 ++++
 arch/powerpc/kernel/process.c     |    3 +++
 4 files changed, 9 insertions(+)
arch/powerpc/include/asm/paca.h +1 -0
index c47d687..07e9851 100644
@@ -137,6 +137,7 @@ struct paca_struct {
 	u8 irq_work_pending;		/* IRQ_WORK interrupt while soft-disable */
 	u8 nap_state_lost;		/* NV GPR values lost in power7_idle */
 	u64 sprg3;			/* Saved user-visible sprg */
+	u64 tm_scratch;                 /* TM scratch area for reclaim */
 
 #ifdef CONFIG_PPC_POWERNV
 	/* Pointer to OPAL machine check event structure set by the
arch/powerpc/kernel/asm-offsets.c +1 -0
index b67e4c3..f9654b6 100644
@@ -126,6 +126,7 @@ int main(void)
 	DEFINE(THREAD_TAR, offsetof(struct thread_struct, tar));
 #endif
 #ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	DEFINE(PACATMSCRATCH, offsetof(struct paca_struct, tm_scratch));
 	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
 	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
 	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
arch/powerpc/kernel/entry_64.S +4 -0
index 9ae8451..612ea13 100644
@@ -785,6 +785,10 @@ fast_exception_return:
 	andc	r4,r4,r0	 /* r0 contains MSR_RI here */
 	mtmsrd	r4,1
 
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	/* TM debug */
+	std	r3, PACATMSCRATCH(r13) /* Stash returned-to MSR */
+#endif
 	/*
 	 * r13 is our per cpu area, only restore it if we are returning to
 	 * userspace the value stored in the stack frame may belong to
arch/powerpc/kernel/process.c +3 -0
index b0a0321..1cc4053 100644
@@ -754,6 +754,9 @@ void show_regs(struct pt_regs * regs)
 	printk("NIP ["REG"] %pS\n", regs->nip, (void *)regs->nip);
 	printk("LR ["REG"] %pS\n", regs->link, (void *)regs->link);
 #endif
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	printk("PACATMSCRATCH [%llx]\n", get_paca()->tm_scratch);
+#endif
 	show_stack(current, (unsigned long *) regs->gpr[1]);
 	if (!user_mode(regs))
 		show_instructions(regs);
Signature · 1 line
1.7.10.4
#42 lore raw

Re: [PATCH 06/17] powerpc: Add transactional memory paca scratch register to show_regs

Quoted 1 line · Michael Neuling

Add transactional memory paca scratch register to show_regs. This is =

useful

=

Quoted 12 lines

for debugging.
=20

---

arch/powerpc/include/asm/paca.h   |    1 +
arch/powerpc/kernel/asm-offsets.c |    1 +
arch/powerpc/kernel/entry_64.S    |    4 ++++
arch/powerpc/kernel/process.c     |    3 +++
4 files changed, 9 insertions(+)

=20

= renamed from arch/powerpc/include/asm/paca.h +0 -0

b/arch/powerpc/include/asm/paca.h

arch/powerpc/include/asm/paca.h struct paca_struct { · line 137

Quoted 5 lines

index c47d687..07e9851 100644

arch/powerpc/include/asm/paca.h +0 -0
@@ -137,6 +137,7 @@ struct paca_struct {
 	u8 irq_work_pending;		/* IRQ_WORK interrupt while =

soft-disable */

arch/powerpc/include/asm/paca.h struct paca_struct { · line 138

Quoted 1 line
arch/powerpc/include/asm/paca.h +0 -0
 	u8 nap_state_lost;		/* NV GPR values lost in =

power7_idle */

arch/powerpc/include/asm/paca.h struct paca_struct { · line 139

Quoted 1 line
arch/powerpc/include/asm/paca.h +0 -0
 	u64 sprg3;			/* Saved user-visible sprg */

#ifdef CONFIG_PPC_TRANSACTIONAL_MEM

arch/powerpc/include/asm/paca.h struct paca_struct { · line 140

Quoted 1 line
arch/powerpc/include/asm/paca.h +1 -0
+	u64 tm_scratch;                 /* TM scratch area for reclaim =

*/

#endif

=

Quoted 4 lines

=20

#ifdef CONFIG_PPC_POWERNV
	/* Pointer to OPAL machine check event structure set by the
= renamed from arch/powerpc/kernel/asm-offsets.c +0 -0

b/arch/powerpc/kernel/asm-offsets.c

arch/powerpc/kernel/asm-offsets.c int main(void) · line 130

Quoted 9 lines

index b67e4c3..f9654b6 100644

arch/powerpc/kernel/asm-offsets.c +1 -0
@@ -126,6 +126,7 @@ int main(void)
 	DEFINE(THREAD_TAR, offsetof(struct thread_struct, tar));
 #endif
 #ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	DEFINE(PACATMSCRATCH, offsetof(struct paca_struct, tm_scratch));
 	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, =

tm_tfhar));

arch/powerpc/kernel/asm-offsets.c int main(void) · line 131

Quoted 1 line
arch/powerpc/kernel/asm-offsets.c +0 -0
 	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, =

tm_texasr));

arch/powerpc/kernel/asm-offsets.c int main(void) · line 132

Quoted 1 line
arch/powerpc/kernel/asm-offsets.c +0 -0
 	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, =

tm_tfiar));

=

Quoted 1 line
= renamed from arch/powerpc/kernel/entry_64.S +0 -0

b/arch/powerpc/kernel/entry_64.S

Quoted 11 lines

index 9ae8451..612ea13 100644

arch/powerpc/kernel/entry_64.S +0 -0
@@ -785,6 +785,10 @@ fast_exception_return:
 	andc	r4,r4,r0	 /* r0 contains MSR_RI here */
 	mtmsrd	r4,1

=20

arch/powerpc/kernel/entry_64.S +4 -0
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	/* TM debug */
+	std	r3, PACATMSCRATCH(r13) /* Stash returned-to MSR */
+#endif
Quoted 2 lines
	/*
	 * r13 is our per cpu area, only restore it if we are returning =

to

=

Quoted 2 lines

* userspace the value stored in the stack frame may belong to

= renamed from arch/powerpc/kernel/process.c +0 -0

b/arch/powerpc/kernel/process.c

Quoted 20 lines

index b0a0321..1cc4053 100644

arch/powerpc/kernel/process.c +3 -1
@@ -754,6 +754,9 @@ void show_regs(struct pt_regs * regs)
 	printk("NIP ["REG"] %pS\n", regs->nip, (void *)regs->nip);
 	printk("LR ["REG"] %pS\n", regs->link, (void *)regs->link);
 #endif
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	printk("PACATMSCRATCH [%llx]\n", get_paca()->tm_scratch);
+#endif
 	show_stack(current, (unsigned long *) regs->gpr[1]);
 	if (!user_mode(regs))
 		show_instructions(regs);
--=20

1.7.10.4
=20
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev

#43 lore raw

Re: [PATCH 06/17] powerpc: Add transactional memory paca scratch register to show_regs

Quoted 26 lines · Kumar Gala
Quoted 17 lines · Michael Neuling

Add transactional memory paca scratch register to show_regs. This is useful for debugging.

---

arch/powerpc/include/asm/paca.h   |    1 +
arch/powerpc/kernel/asm-offsets.c |    1 +
arch/powerpc/kernel/entry_64.S    |    4 ++++
arch/powerpc/kernel/process.c     |    3 +++
4 files changed, 9 insertions(+)
arch/powerpc/include/asm/paca.h +0 -0
index c47d687..07e9851 100644

arch/powerpc/include/asm/paca.h struct paca_struct { · line 139

Quoted 3 lines
arch/powerpc/include/asm/paca.h +0 -0
@@ -137,6 +137,7 @@ struct paca_struct {
 	u8 irq_work_pending;		/* IRQ_WORK interrupt while soft-disable */
 	u8 nap_state_lost;		/* NV GPR values lost in power7_idle */
 	u64 sprg3;			/* Saved user-visible sprg */

#ifdef CONFIG_PPC_TRANSACTIONAL_MEM

Quoted 3 lines

arch/powerpc/include/asm/paca.h struct paca_struct { · line 140

Quoted 1 line
arch/powerpc/include/asm/paca.h +1 -0
+	u64 tm_scratch;                 /* TM scratch area for reclaim */

#endif

Yep.. thanks again.

Mikey

#44 lore raw

PATCH 7/17 powerpc: Add helper functions for transactional memory context switching

In reply to Michael Neuling

Here we add the helper functions to be used when context switching. These allow us to fully reclaim and recheckpoint a transaction.

We introduce a new paca field called tm_scratch to help us store away register values when doing the low level tm reclaim register save.

---
 arch/powerpc/include/asm/reg.h |    2 +-
 arch/powerpc/include/asm/tm.h  |   20 +++
 arch/powerpc/kernel/Makefile   |    2 +
 arch/powerpc/kernel/tm.S       |  388 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 411 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/include/asm/tm.h
 create mode 100644 arch/powerpc/kernel/tm.S
arch/powerpc/include/asm/reg.h +1 -1
index 7844c28..eee2a60 100644
@@ -797,7 +797,7 @@
  *        HV mode in which case it is HSPRG0
  *
  * 64-bit server:
- *	- SPRG0 unused (reserved for HV on Power4)
+ *	- SPRG0 scratch for TM recheckpoint/reclaim (reserved for HV on Power4)
  *	- SPRG2 scratch for exception vectors
  *	- SPRG3 CPU and NUMA node for VDSO getcpu (user visible)
  *      - HSPRG0 stores PACA in HV mode
arch/powerpc/include/asm/tm.h new file +20 -0
new file mode 100644
index 0000000..4b4449a
@@ -0,0 +1,20 @@
+/*
+ * Transactional memory support routines to reclaim and recheckpoint
+ * transactional process state.
+ *
+ * Copyright 2012 Matt Evans & Michael Neuling, IBM Corporation.
+ */
+
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+extern void do_load_up_transact_fpu(struct thread_struct *thread);
+extern void do_load_up_transact_altivec(struct thread_struct *thread);
+#endif
+
+extern void tm_enable(void);
+extern void tm_reclaim(struct thread_struct *thread,
+		       unsigned long orig_msr, uint8_t cause);
+extern void tm_recheckpoint(struct thread_struct *thread,
+			    unsigned long orig_msr);
+extern void tm_abort(uint8_t cause);
+extern void tm_save_sprs(struct thread_struct *thread);
+extern void tm_restore_sprs(struct thread_struct *thread);
arch/powerpc/kernel/Makefile +2 -0
index b4f0e36..f960a79 100644
@@ -121,6 +121,8 @@ ifneq ($(CONFIG_PPC_INDIRECT_IO),y)
 obj-y				+= iomap.o
 endif
 
+obj64-$(CONFIG_PPC_TRANSACTIONAL_MEM)	+= tm.o
+
 obj-$(CONFIG_PPC64)		+= $(obj64-y)
 obj-$(CONFIG_PPC32)		+= $(obj32-y)
 
arch/powerpc/kernel/tm.S new file +388 -0

388 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#45 lore raw

PATCH 8/17 powerpc: Add FP/VSX and VMX register load functions for transactional memory

In reply to Michael Neuling

This adds functions to restore the state of the FP/VSX registers from what's stored in the thread_struct. Two version for FP/VSX are required since one restores them from transactional/checkpoint side of the thread_struct and the other from the speculated side.

Similar functions are added for VMX registers.

---
 arch/powerpc/kernel/fpu.S    |   54 ++++++++++++++++++++++++++++++++++++++++++
 arch/powerpc/kernel/vector.S |   51 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+)
arch/powerpc/kernel/fpu.S +54 -0
index adb1551..0441ba6 100644
@@ -62,6 +62,60 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+/*
+ * Wrapper to call load_up_fpu from C.
+ * void do_load_up_fpu(struct pt_regs *regs);
+ */
+_GLOBAL(do_load_up_fpu)
+	mflr	r0
+	std	r0, 16(r1)
+	stdu	r1, -112(r1)
+
+	subi	r6, r3, STACK_FRAME_OVERHEAD
+	/* load_up_fpu expects r12=MSR, r13=PACA, and returns
+	 * with r12 = new MSR.
+	 */
+	ld	r12,_MSR(r6)
+	GET_PACA(r13)
+
+	bl	load_up_fpu
+	std	r12,_MSR(r6)
+
+	ld	r0, 112+16(r1)
+	addi	r1, r1, 112
+	mtlr	r0
+	blr
+
+
+/* void do_load_up_fpu(struct thread_struct *thread)
+ *
+ * This is similar to load_up_fpu but for the transactional version of the FP
+ * register set.  It doesn't mess with the task MSR or valid flags.
+ * Furthermore, we don't do lazy FP with TM currently.
+ */
+_GLOBAL(do_load_up_transact_fpu)
+	mfmsr	r6
+	ori	r5,r6,MSR_FP
+#ifdef CONFIG_VSX
+BEGIN_FTR_SECTION
+	oris	r5,r5,MSR_VSX@h
+END_FTR_SECTION_IFSET(CPU_FTR_VSX)
+#endif
+	SYNC
+	MTMSRD(r5)
+
+	lfd	fr0,THREAD_TRANSACT_FPSCR(r3)
+	MTFSF_L(fr0)
+	REST_32FPVSRS_TRANSACT(0, R4, R3)
+
+	/* FP/VSX off again */
+	MTMSRD(r6)
+	SYNC
+
+	blr
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+
 /*
  * This task wants to use the FPU now.
  * On UP, disable FP for the task which had the FPU previously,
arch/powerpc/kernel/vector.S +51 -0

57 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#46 lore raw

Re: [PATCH 08/17] powerpc: Add FP/VSX and VMX register load functions for transactional memory

Quoted 2 lines · Michael Neuling

This adds functions to restore the state of the FP/VSX registers from what's stored in the thread_struct. Two version for FP/VSX are =

required

Quoted 9 lines

since one restores them from transactional/checkpoint side of the thread_struct and the other from the speculated side. =20
Similar functions are added for VMX registers.
=20

---
arch/powerpc/kernel/fpu.S | 54 =

++++++++++++++++++++++++++++++++++++++++++

Quoted 1 line

arch/powerpc/kernel/vector.S | 51 =

+++++++++++++++++++++++++++++++++++++++

arch/powerpc/kernel/fpu.S END_FTR_SECTION_IFSET(CPU_FTR_VSX); =

Quoted 7 lines
2 files changed, 105 insertions(+)

=20

arch/powerpc/kernel/fpu.S +0 -0
index adb1551..0441ba6 100644
@@ -62,6 +62,60 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);			=
		\

arch/powerpc/kernel/fpu.S END_FTR_SECTION_IFSET(CPU_FTR_VSX); = · line 63

Quoted 2 lines
arch/powerpc/kernel/fpu.S +0 -0
 	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) =

__SAVE_32FPVSRS(n,__REG_##c,__REG_##base)

Quoted 18 lines

=20

arch/powerpc/kernel/fpu.S +17 -0
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+/*
+ * Wrapper to call load_up_fpu from C.
+ * void do_load_up_fpu(struct pt_regs *regs);
+ */
+_GLOBAL(do_load_up_fpu)
+	mflr	r0
+	std	r0, 16(r1)
+	stdu	r1, -112(r1)
+
+	subi	r6, r3, STACK_FRAME_OVERHEAD
+	/* load_up_fpu expects r12=3DMSR, r13=3DPACA, and returns
+	 * with r12 =3D new MSR.
+	 */
+	ld	r12,_MSR(r6)
+	GET_PACA(r13)
+

arch/powerpc/kernel/fpu.S END_FTR_SECTION_IFSET(CPU_FTR_VSX); \ · line 91

Quoted 10 lines
arch/powerpc/kernel/fpu.S +10 -0
+	bl	load_up_fpu
+	std	r12,_MSR(r6)
+
+	ld	r0, 112+16(r1)
+	addi	r1, r1, 112
+	mtlr	r0
+	blr
+
+
+/* void do_load_up_fpu(struct thread_struct *thread)

Comment is wrong, should be void do_load_up_transact_fpu(...)

Quoted 2 lines

+ *
+ * This is similar to load_up_fpu but for the transactional version =

of the FP

=

Quoted 29 lines

+ * register set. It doesn't mess with the task MSR or valid flags.
+ * Furthermore, we don't do lazy FP with TM currently.
+ */

arch/powerpc/kernel/fpu.S +22 -0
+_GLOBAL(do_load_up_transact_fpu)
+	mfmsr	r6
+	ori	r5,r6,MSR_FP
+#ifdef CONFIG_VSX
+BEGIN_FTR_SECTION
+	oris	r5,r5,MSR_VSX@h
+END_FTR_SECTION_IFSET(CPU_FTR_VSX)
+#endif
+	SYNC
+	MTMSRD(r5)
+
+	lfd	fr0,THREAD_TRANSACT_FPSCR(r3)
+	MTFSF_L(fr0)
+	REST_32FPVSRS_TRANSACT(0, R4, R3)
+
+	/* FP/VSX off again */
+	MTMSRD(r6)
+	SYNC
+
+	blr
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+

/*
* This task wants to use the FPU now.
* On UP, disable FP for the task which had the FPU previously,

= renamed from arch/powerpc/kernel/vector.S +0 -0

b/arch/powerpc/kernel/vector.S

Quoted 22 lines

index e830289..7112a24 100644

arch/powerpc/kernel/vector.S +0 -0
@@ -7,6 +7,57 @@
 #include <asm/page.h>
 #include <asm/ptrace.h>

=20

arch/powerpc/kernel/vector.S +15 -0
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+/*
+ * Wrapper to call load_up_altivec from C.
+ * void do_load_up_altivec(struct pt_regs *regs);
+ */
+_GLOBAL(do_load_up_altivec)
+	mflr	r0
+	std	r0, 16(r1)
+	stdu	r1, -112(r1)
+
+	subi	r6, r3, STACK_FRAME_OVERHEAD
+	/* load_up_altivec expects r12=3DMSR, r13=3DPACA, and returns
+	 * with r12 =3D new MSR.
+	 */
+	ld	r12,_MSR(r6)

arch/powerpc/kernel/vector.S line 34

Quoted 10 lines
arch/powerpc/kernel/vector.S +10 -0
+	GET_PACA(r13)
+	bl	load_up_altivec
+	std	r12,_MSR(r6)
+
+	ld	r0, 112+16(r1)
+	addi	r1, r1, 112
+	mtlr	r0
+	blr
+
+/* void do_load_up_altivec(struct thread_struct *thread)

comment wrong.

Quoted 2 lines

+ *
+ * This is similar to load_up_altivec but for the transactional =

version of the

Quoted 34 lines

+ * vector regs. It doesn't mess with the task MSR or valid flags.
+ * Furthermore, VEC laziness is not supported with TM currently.
+ */

arch/powerpc/kernel/vector.S +21 -0
+_GLOBAL(do_load_up_transact_altivec)
+	mfmsr	r6
+	oris	r5,r6,MSR_VEC@h
+	MTMSRD(r5)
+	isync
+
+	li	r4,1
+	stw	r4,THREAD_USED_VR(r3)
+
+	li	r10,THREAD_TRANSACT_VSCR
+	lvx	vr0,r10,r3
+	mtvscr	vr0
+	REST_32VRS_TRANSACT(0,r4,r3)
+
+	/* Disable VEC again. */
+	MTMSRD(r6)
+	isync
+
+	blr
+#endif
+

/*
* load_up_altivec(unused, unused, tsk)
* Disable VMX for the task which had it previously,
--=20
1.7.10.4
=20
_______________________________________________
Linuxppc-dev mailing list [email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev

#47 lore raw

Re: [PATCH 08/17] powerpc: Add FP/VSX and VMX register load functions for transactional memory

Quoted 3 lines
Quoted 1 line

+/* void do_load_up_fpu(struct thread_struct *thread)

Comment is wrong, should be void do_load_up_transact_fpu(...)

Ooops, thanks for the catch.

Quoted 3 lines
Quoted 1 line

+/* void do_load_up_altivec(struct thread_struct *thread)

comment wrong.

Ooops again.

FYI I'm in Austin next week if you wanna grab a beer.

Mikey

#48 lore raw

PATCH 9/17 powerpc: Add reclaim and recheckpoint functions for context switching transactional memory processes

In reply to Michael Neuling

When we switch out a task, we need to save both the checkpointed and the speculated state into the thread struct.

Similarly when we are switching in a task we need to load both the checkpointed and speculated state. If the task was using FP, we non-lazily reload both the original and the speculative FP register states. This is because the kernel doesn't see if/when a TM rollback occurs, so if we take an FP unavoidable later, we are unable to determine which set of FP regs need to be restored.

This simply adds these functions. It doesn't hook them into the existing code yet.

---
 arch/powerpc/kernel/process.c |  112 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 112 insertions(+)
arch/powerpc/kernel/process.c +112 -0
index 1cc4053..48a9875 100644
@@ -50,6 +50,7 @@
 #include <asm/runlatch.h>
 #include <asm/syscalls.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 #ifdef CONFIG_PPC64
 #include <asm/firmware.h>
@@ -467,6 +468,117 @@ static inline bool hw_brk_match(struct arch_hw_breakpoint *a,
 		return false;
 	return true;
 }
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+static inline void tm_reclaim_task(struct task_struct *tsk)
+{
+	/* We have to work out if we're switching from/to a task that's in the
+	 * middle of a transaction.
+	 *
+	 * In switching we need to maintain a 2nd register state as
+	 * oldtask->thread.ckpt_regs.  We tm_reclaim(oldproc); this saves the
+	 * checkpointed (tbegin) state in ckpt_regs and saves the transactional
+	 * (current) FPRs into oldtask->thread.transact_fpr[].
+	 *
+	 * We also context switch (save) TFHAR/TEXASR/TFIAR in here.
+	 */
+	struct thread_struct *thr = &tsk->thread;
+
+	if (!thr->regs)
+		return;
+
+	if (!MSR_TM_ACTIVE(thr->regs->msr))
+		goto out_and_saveregs;
+
+	/* Stash the original thread MSR, as giveup_fpu et al will
+	 * modify it.  We hold onto it to see whether the task used
+	 * FP & vector regs.
+	 */
+	thr->tm_orig_msr = thr->regs->msr;
+
+	TM_DEBUG("--- tm_reclaim on pid %d (NIP=%lx, "
+		 "ccr=%lx, msr=%lx, trap=%lx)\n",
+		 tsk->pid, thr->regs->nip,
+		 thr->regs->ccr, thr->regs->msr,
+		 thr->regs->trap);
+
+	tm_reclaim(thr, thr->regs->msr, TM_CAUSE_RESCHED);
+
+	TM_DEBUG("--- tm_reclaim on pid %d complete\n",
+		 tsk->pid);
+
+out_and_saveregs:
+	/* Always save the regs here, even if a transaction's not active.
+	 * This context-switches a thread's TM info SPRs.  We do it here to
+	 * be consistent with the restore path (in recheckpoint) which
+	 * cannot happen later in _switch().
+	 */
+	tm_save_sprs(thr);
+}
+
+static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+{
+	unsigned long msr;
+
+	if (!cpu_has_feature(CPU_FTR_TM))
+		return;
+
+	/* Recheckpoint the registers of the thread we're about to switch to.
+	 *
+	 * If the task was using FP, we non-lazily reload both the original and
+	 * the speculative FP register states.  This is because the kernel
+	 * doesn't see if/when a TM rollback occurs, so if we take an FP
+	 * unavoidable later, we are unable to determine which set of FP regs
+	 * need to be restored.
+	 */
+	if (!new->thread.regs)
+		return;
+
+	/* The TM SPRs are restored here, so that TEXASR.FS can be set
+	 * before the trecheckpoint and no explosion occurs.
+	 */
+	tm_restore_sprs(&new->thread);
+
+	if (!MSR_TM_ACTIVE(new->thread.regs->msr))
+		return;
+	msr = new->thread.tm_orig_msr;
+	/* Recheckpoint to restore original checkpointed register state. */
+	TM_DEBUG("*** tm_recheckpoint of pid %d "
+		 "(new->msr 0x%lx, new->origmsr 0x%lx)\n",
+		 new->pid, new->thread.regs->msr, msr);
+
+	/* This loads the checkpointed FP/VEC state, if used */
+	tm_recheckpoint(&new->thread, msr);
+
+	/* This loads the speculative FP/VEC state, if used */
+	if (msr & MSR_FP) {
+		do_load_up_transact_fpu(&new->thread);
+		new->thread.regs->msr |=
+			(MSR_FP | new->thread.fpexc_mode);
+	}
+	if (msr & MSR_VEC) {
+		do_load_up_transact_altivec(&new->thread);
+		new->thread.regs->msr |= MSR_VEC;
+	}
+	/* We may as well turn on VSX too since all the state is restored now */
+	if (msr & MSR_VSX)
+		new->thread.regs->msr |= MSR_VSX;
+
+	TM_DEBUG("*** tm_recheckpoint of pid %d complete "
+		 "(kernel msr 0x%lx)\n",
+		 new->pid, mfmsr());
+}
+
+static inline void __switch_to_tm(struct task_struct *prev)
+{
+	if (cpu_has_feature(CPU_FTR_TM)) {
+		tm_enable();
+		tm_reclaim_task(prev);
+	}
+}
+#else
+#define tm_recheckpoint_new_task(new)
+#define __switch_to_tm(prev)
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 
 struct task_struct *__switch_to(struct task_struct *prev,
 	struct task_struct *new)
Signature · 1 line
1.7.10.4
#49 lore raw

PATCH 10/17 powerpc: Add transactional memory unavaliable execption handler

In reply to Michael Neuling

These should never happen since we always turn on MSR TM when in userspace. We don't do lazy TM.

Hence if we hit this, we barf and kill the task as something's gone horribly wrong.

---
 arch/powerpc/kernel/exceptions-64s.S |   19 +++++++++++++++++++
 arch/powerpc/kernel/traps.c          |   21 +++++++++++++++++++++
 2 files changed, 40 insertions(+)
arch/powerpc/kernel/exceptions-64s.S +19 -0
index 7a1c87c..d0cc657 100644
@@ -313,6 +313,9 @@ vsx_unavailable_pSeries_1:
 	. = 0xf40
 	b	vsx_unavailable_pSeries
 
+	. = 0xf60
+	b	tm_unavailable_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_EXCEPTION_HV(0x1200, 0x1202, cbe_system_error)
 	KVM_HANDLER_SKIP(PACA_EXGEN, EXC_HV, 0x1202)
@@ -528,6 +531,8 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_206)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf20)
 	STD_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf40)
+	STD_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
+	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf60)
 
 /*
  * An interrupt came in while soft-disabled. We set paca->irq_happened, then:
@@ -817,6 +822,10 @@ vsx_unavailable_relon_pSeries_1:
 	. = 0x4f40
 	b	vsx_unavailable_relon_pSeries
 
+tm_unavailable_relon_pSeries_1:
+	. = 0x4f60
+	b	tm_unavailable_relon_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_RELON_EXCEPTION_HV(0x5200, 0x1202, cbe_system_error)
 #endif /* CONFIG_CBE_RAS */
@@ -1177,6 +1186,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 	b	.ret_from_except
 
 	.align	7
+	.globl tm_unavailable_common
+tm_unavailable_common:
+	EXCEPTION_PROLOG_COMMON(0xf60, PACA_EXGEN)
+	bl	.save_nvgprs
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.tm_unavailable_exception
+	b	.ret_from_except
+
+	.align	7
 	.globl	__end_handlers
 __end_handlers:
 
@@ -1195,6 +1213,7 @@ __end_handlers:
 	STD_RELON_EXCEPTION_PSERIES(., 0xf00, performance_monitor)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf20, altivec_unavailable)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
+	STD_RELON_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
 
 #if defined(CONFIG_PPC_PSERIES) || defined(CONFIG_PPC_POWERNV)
 /*
arch/powerpc/kernel/traps.c +21 -0
index bd5de5d..f829e05 100644
@@ -1168,6 +1168,27 @@ void vsx_unavailable_exception(struct pt_regs *regs)
 	die("Unrecoverable VSX Unavailable Exception", regs, SIGABRT);
 }
 
+void tm_unavailable_exception(struct pt_regs *regs)
+{
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	/* Currently we never expect a TMU exception.  Catch
+	 * this and kill the process!
+	 */
+	printk(KERN_EMERG "Unexpected TM unavailable exception at %lx "
+	       "(msr %lx)\n",
+	       regs->nip, regs->msr);
+
+	if (user_mode(regs)) {
+		_exception(SIGILL, regs, ILL_ILLOPC, regs->nip);
+		return;
+	}
+
+	die("Unexpected TM unavailable exception", regs, SIGABRT);
+}
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.10.4
#50 lore raw

PATCH 11/17 powerpc: Routines for FP/VSX/VMX unavailable during a transaction

In reply to Michael Neuling

We do lazy FP but not lazy TM (ie. userspace starts with MSR TM=1 FP=0). Hence if userspace does an FP instruction during a transaction, we'll take an fp unavailable exception.

This adds functions needed to handle this case. We have to inject the current FP state into the checkpoint so that the hardware can decide what to do with the transaction. We can't inject only the FP so we have to do a full treclaim and recheckpoint to inject just the FP state. This will cause the transaction to be marked as aborted by the hardware.

This just add the routines needed to do this for FP, VMX and VSX. It doesn't hook them into the rest of the code yet.

---
 arch/powerpc/kernel/traps.c |   83 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
arch/powerpc/kernel/traps.c +83 -0
index f829e05..5c30469 100644
@@ -58,6 +58,7 @@
 #include <asm/rio.h>
 #include <asm/fadump.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 
 #if defined(CONFIG_DEBUGGER) || defined(CONFIG_KEXEC)
@@ -1189,6 +1190,88 @@ void tm_unavailable_exception(struct pt_regs *regs)
 	die("Unexpected TM unavailable exception", regs, SIGABRT);
 }
 
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+
+extern void do_load_up_fpu(struct pt_regs *regs);
+
+void fp_unavailable_tm(struct pt_regs *regs)
+{
+	/* Note:  This does not handle any kind of FP laziness. */
+
+	TM_DEBUG("FP Unavailable trap whilst transactional at 0x%lx, MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+
+        /* We can only have got here if the task started using FP after
+         * beginning the transaction.  So, the transactional regs are just a
+         * copy of the checkpointed ones.  But, we still need to recheckpoint
+         * as we're enabling FP for the process; it will return, abort the
+         * transaction, and probably retry but now with FP enabled.  So the
+         * checkpointed FP registers need to be loaded.
+	 */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	/* Reclaim didn't save out any FPRs to transact_fprs. */
+
+	/* Enable FP for the task: */
+	regs->msr |= (MSR_FP | current->thread.fpexc_mode);
+
+	/* This loads and recheckpoints the FP registers from
+	 * thread.fpr[].  They will remain in registers after the
+	 * checkpoint so we don't need to reload them after.
+	 */
+	tm_recheckpoint(&current->thread, regs->msr);
+}
+
+#ifdef CONFIG_ALTIVEC
+extern void do_load_up_altivec(struct pt_regs *regs);
+
+void altivec_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This function operates
+	 * the same way.
+	 */
+
+	TM_DEBUG("Vector Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	regs->msr |= MSR_VEC;
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vr = 1;
+}
+#endif
+
+#ifdef CONFIG_VSX
+void vsx_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This works similarly,
+	 * though we're loading both FP and VEC registers in here.
+	 *
+	 * If FP isn't in use, load FP regs.  If VEC isn't in use, load VEC
+	 * regs.  Either way, set MSR_VSX.
+	 */
+
+	TM_DEBUG("VSX Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+
+	tm_enable();
+	/* This reclaims FP and/or VR regs if they're already enabled */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+
+	regs->msr |= MSR_VEC | MSR_FP | current->thread.fpexc_mode |
+		MSR_VSX;
+	/* This loads & recheckpoints FP and VRs. */
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vsr = 1;
+}
+#endif
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.10.4
#51 lore raw

PATCH 12/17 powerpc: Hook in new transactional memory code

In reply to Michael Neuling

This hooks the new transactional memory code into context switching, FP/VMX/VMX unavailable and exception return.

---
 arch/powerpc/kernel/exceptions-64s.S |   56 ++++++++++++++++++++++++++++++++--
 arch/powerpc/kernel/process.c        |   15 +++++++--
 arch/powerpc/kernel/traps.c          |   32 +++++++++++++++++++
 arch/powerpc/mm/hash_utils_64.c      |   16 ++++++++++
 4 files changed, 115 insertions(+), 4 deletions(-)
arch/powerpc/kernel/exceptions-64s.S +54 -2
index d0cc657..85b923c 100644
@@ -1147,9 +1147,26 @@ fp_unavailable_common:
 	addi	r3,r1,STACK_FRAME_OVERHEAD
 	bl	.kernel_fp_unavailable_exception
 	BUG_OPCODE
-1:	bl	.load_up_fpu
+1:
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+BEGIN_FTR_SECTION
+	/* Test if 2 TM state bits are zero.  If non-zero (ie. userspace was in
+	 * transaction), go do TM stuff
+	 */
+	rldicl.	r0, r12, (64-MSR_TS_LG), (64-2)
+	bne-	2f
+END_FTR_SECTION_IFSET(CPU_FTR_TM)
+#endif
+	bl	.load_up_fpu
 	b	fast_exception_return
-
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	DISABLE_INTS
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.fp_unavailable_tm
+	b	.ret_from_except
+#endif
 	.align	7
 	.globl altivec_unavailable_common
 altivec_unavailable_common:
@@ -1157,8 +1174,25 @@ altivec_unavailable_common:
 #ifdef CONFIG_ALTIVEC
 BEGIN_FTR_SECTION
 	beq	1f
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+  BEGIN_FTR_SECTION_NESTED(69)
+	/* Test if 2 TM state bits are zero.  If non-zero (ie. userspace was in
+	 * transaction), go do TM stuff
+	 */
+	rldicl.	r0, r12, (64-MSR_TS_LG), (64-2)
+	bne-	2f
+  END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
+#endif
 	bl	.load_up_altivec
 	b	fast_exception_return
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	DISABLE_INTS
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.altivec_unavailable_tm
+	b	.ret_from_except
+#endif
 1:
 END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
 #endif
@@ -1175,7 +1209,24 @@ vsx_unavailable_common:
 #ifdef CONFIG_VSX
 BEGIN_FTR_SECTION
 	beq	1f
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+  BEGIN_FTR_SECTION_NESTED(69)
+	/* Test if 2 TM state bits are zero.  If non-zero (ie. userspace was in
+	 * transaction), go do TM stuff
+	 */
+	rldicl.	r0, r12, (64-MSR_TS_LG), (64-2)
+	bne-	2f
+  END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
+#endif
 	b	.load_up_vsx
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	DISABLE_INTS
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.vsx_unavailable_tm
+	b	.ret_from_except
+#endif
 1:
 END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 #endif
@@ -1190,6 +1241,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 tm_unavailable_common:
 	EXCEPTION_PROLOG_COMMON(0xf60, PACA_EXGEN)
 	bl	.save_nvgprs
+	DISABLE_INTS
 	addi	r3,r1,STACK_FRAME_OVERHEAD
 	bl	.tm_unavailable_exception
 	b	.ret_from_except
arch/powerpc/kernel/process.c +13 -2
index 48a9875..59dd545 100644
@@ -515,7 +515,7 @@ out_and_saveregs:
 	tm_save_sprs(thr);
 }
 
-static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+static inline void tm_recheckpoint_new_task(struct task_struct *new)
 {
 	unsigned long msr;
 
@@ -590,6 +590,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	struct ppc64_tlb_batch *batch;
 #endif
 
+	__switch_to_tm(prev);
+
 #ifdef CONFIG_SMP
 	/* avoid complexity of lazy save/restore of fpu
 	 * by just saving it every time we switch out if
@@ -705,6 +707,9 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	 * of sync. Hard disable here.
 	 */
 	hard_irq_disable();
+
+	tm_recheckpoint_new_task(new);
+
 	last = _switch(old_thread, new_thread);
 
 #ifdef CONFIG_PPC_BOOK3S_64
@@ -1080,7 +1085,6 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 		regs->msr = MSR_USER32;
 	}
 #endif
-
 	discard_lazy_cpu_state();
 #ifdef CONFIG_VSX
 	current->thread.used_vsr = 0;
@@ -1100,6 +1104,13 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.spefscr = 0;
 	current->thread.used_spe = 0;
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	if (cpu_has_feature(CPU_FTR_TM))
+		regs->msr |= MSR_TM;
+	current->thread.tm_tfhar = 0;
+	current->thread.tm_texasr = 0;
+	current->thread.tm_tfiar = 0;
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 }
 
 #define PR_FP_ALL_EXCEPT (PR_FP_EXC_DIV | PR_FP_EXC_OVF | PR_FP_EXC_UND \
arch/powerpc/kernel/traps.c +32 -0
index 5c30469..f9b751b 100644
@@ -1029,6 +1029,38 @@ void __kprobes program_check_exception(struct pt_regs *regs)
 		_exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip);
 		return;
 	}
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	if (reason & REASON_TM) {
+		/* This is a TM "Bad Thing Exception" program check.
+		 * This occurs when:
+		 * -  An rfid/hrfid/mtmsrd attempts to cause an illegal
+		 *    transition in TM states.
+		 * -  A trechkpt is attempted when transactional.
+		 * -  A treclaim is attempted when non transactional.
+		 * -  A tend is illegally attempted.
+		 * -  writing a TM SPR when transactional.
+		 */
+		if (!user_mode(regs) &&
+		    report_bug(regs->nip, regs) == BUG_TRAP_TYPE_WARN) {
+			regs->nip += 4;
+			return;
+		}
+		/* If usermode caused this, it's done something illegal and
+		 * gets a SIGILL slap on the wrist.  We call it an illegal
+		 * operand to distinguish from the instruction just being bad
+		 * (e.g. executing a 'tend' on a CPU without TM!); it's an
+		 * illegal /placement/ of a valid instruction.
+		 */
+		if (user_mode(regs)) {
+			_exception(SIGILL, regs, ILL_ILLOPN, regs->nip);
+			return;
+		} else {
+			printk(KERN_EMERG "Unexpected TM Bad Thing exception "
+			       "at %lx (msr 0x%x)\n", regs->nip, reason);
+			die("Unrecoverable exception", regs, SIGABRT);
+		}
+	}
+#endif
 
 	/* We restore the interrupt state now */
 	if (!arch_irq_disabled_regs(regs))
arch/powerpc/mm/hash_utils_64.c +16 -0
index 3a292be..1b6e127 100644
@@ -55,6 +55,7 @@
 #include <asm/code-patching.h>
 #include <asm/fadump.h>
 #include <asm/firmware.h>
+#include <asm/tm.h>
 
 #ifdef DEBUG
 #define DBG(fmt...) udbg_printf(fmt)
@@ -1171,6 +1172,21 @@ void flush_hash_page(unsigned long vpn, real_pte_t pte, int psize, int ssize,
 		DBG_LOW(" sub %ld: hash=%lx, hidx=%lx\n", index, slot, hidx);
 		ppc_md.hpte_invalidate(slot, vpn, psize, ssize, local);
 	} pte_iterate_hashed_end();
+
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	/* Transactions are not aborted by tlbiel, only tlbie.
+	 * Without, syncing a page back to a block device w/ PIO could pick up
+	 * transactional data (bad!) so we force an abort here.  Before the
+	 * sync the page will be made read-only, which will flush_hash_page.
+	 * BIG ISSUE here: if the kernel uses a page from userspace without
+	 * unmapping it first, it may see the speculated version.
+	 */
+	if (local && cpu_has_feature(CPU_FTR_TM) &&
+	    MSR_TM_ACTIVE(current->thread.regs->msr)) {
+		tm_enable();
+		tm_abort(TM_CAUSE_TLBI);
+	}
+#endif
 }
 
 void flush_hash_range(unsigned long number, int local)
Signature · 1 line
1.7.10.4
#52 lore raw

PATCH 13/17 powerpc: Add new transactional memory state to the signal context

In reply to Michael Neuling

This adds the new transactional memory archtected state to the signal context in both 32 and 64 bit.

---
 arch/powerpc/include/asm/reg.h  |    1 +
 arch/powerpc/kernel/signal.h    |    8 +
 arch/powerpc/kernel/signal_32.c |  500 ++++++++++++++++++++++++++++++++++++++-
 arch/powerpc/kernel/signal_64.c |  337 +++++++++++++++++++++++++-
 4 files changed, 830 insertions(+), 16 deletions(-)
arch/powerpc/include/asm/reg.h +1 -0
index eee2a60..7035e60 100644
@@ -120,6 +120,7 @@
 #define TM_CAUSE_FAC_UNAV	0xfa
 #define TM_CAUSE_SYSCALL	0xf9 /* Persistent */
 #define TM_CAUSE_MISC		0xf6
+#define TM_CAUSE_SIGNAL		0xf4
 
 #if defined(CONFIG_PPC_BOOK3S_64)
 #define MSR_64BIT	MSR_SF
arch/powerpc/kernel/signal.h +8 -0
index e00acb4..ec84c90 100644
@@ -25,13 +25,21 @@ extern int handle_rt_signal32(unsigned long sig, struct k_sigaction *ka,
 
 extern unsigned long copy_fpr_to_user(void __user *to,
 				      struct task_struct *task);
+extern unsigned long copy_transact_fpr_to_user(void __user *to,
+					       struct task_struct *task);
 extern unsigned long copy_fpr_from_user(struct task_struct *task,
 					void __user *from);
+extern unsigned long copy_transact_fpr_from_user(struct task_struct *task,
+						 void __user *from);
 #ifdef CONFIG_VSX
 extern unsigned long copy_vsx_to_user(void __user *to,
 				      struct task_struct *task);
+extern unsigned long copy_transact_vsx_to_user(void __user *to,
+					       struct task_struct *task);
 extern unsigned long copy_vsx_from_user(struct task_struct *task,
 					void __user *from);
+extern unsigned long copy_transact_vsx_from_user(struct task_struct *task,
+						 void __user *from);
 #endif
 
 #ifdef CONFIG_PPC64
arch/powerpc/kernel/signal_32.c +487 -13

608 lines not on this page · raw message

arch/powerpc/kernel/signal_64.c +334 -3

389 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#53 lore raw

PATCH 14/17 powerpc: Add transactional memory to POWER8 cpu features

In reply to Michael Neuling
---
 arch/powerpc/include/asm/cputable.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/cputable.h +2 -1
index 3636081..fb3245e 100644
@@ -421,7 +421,8 @@ extern const char *powerpc_base_platform;
 	    CPU_FTR_DSCR | CPU_FTR_SAO  | \
 	    CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \
 	    CPU_FTR_ICSWX | CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \
-	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_DAWR | CPU_FTR_BCTAR)
+	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_DAWR | CPU_FTR_BCTAR | \
+	    CPU_FTR_TM_COMP)
 #define CPU_FTRS_CELL	(CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \
 	    CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
Signature · 1 line
1.7.10.4
#54 lore raw

PATCH 15/17 powerpc: Add config option for transactional memory

In reply to Michael Neuling

Kconfig option for transactional memory on powerpc.

---
 arch/powerpc/Kconfig |    8 ++++++++
 1 file changed, 8 insertions(+)
arch/powerpc/Kconfig +8 -0
index 4b27edb..fbeb6d2 100644
@@ -313,6 +313,14 @@ config MATH_EMULATION
 	  unit, which will allow programs that use floating-point
 	  instructions to run.
 
+config PPC_TRANSACTIONAL_MEM
+       bool "Transactional Memory support for POWERPC"
+       depends on PPC64
+       depends on SMP
+       default n
+       ---help---
+         Support user-mode Transactional Memory on POWERPC.
+
 config 8XX_MINIMAL_FPEMU
 	bool "Minimal math emulation for 8xx"
 	depends on 8xx && !MATH_EMULATION
Signature · 1 line
1.7.10.4
#55 lore raw

Re: [PATCH 15/17] powerpc: Add config option for transactional memory

Quoted 13 lines · Michael Neuling

Kconfig option for transactional memory on powerpc.

---
arch/powerpc/Kconfig | 8 ++++++++

1 file changed, 8 insertions(+)
arch/powerpc/Kconfig +0 -0
index 4b27edb..fbeb6d2 100644

arch/powerpc/Kconfig config MATH_EMULATION · line 318

Quoted 6 lines
arch/powerpc/Kconfig +3 -0
@@ -313,6 +313,14 @@ config MATH_EMULATION
 	  unit, which will allow programs that use floating-point
 	  instructions to run.
 
+config PPC_TRANSACTIONAL_MEM
+       bool "Transactional Memory support for POWERPC"
+       depends on PPC64

Should this really depend on PPC_BOOK3S_64 ?

Quoted 15 lines
arch/powerpc/Kconfig +5 -0
+       depends on SMP
+       default n
+       ---help---
+         Support user-mode Transactional Memory on POWERPC.
+

config 8XX_MINIMAL_FPEMU

	bool "Minimal math emulation for 8xx"
	depends on 8xx && !MATH_EMULATION
Signature · 6 lines
1.7.10.4

_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev
#56 lore raw

Re: [PATCH 15/17] powerpc: Add config option for transactional memory

Quoted 19 lines · Kumar Gala
Quoted 16 lines · Michael Neuling

Kconfig option for transactional memory on powerpc.

---
arch/powerpc/Kconfig | 8 ++++++++

1 file changed, 8 insertions(+)
arch/powerpc/Kconfig +0 -0
index 4b27edb..fbeb6d2 100644
@@ -313,6 +313,14 @@ config MATH_EMULATION
 	  unit, which will allow programs that use floating-point
 	  instructions to run.
 
Quoted 5 lines

arch/powerpc/Kconfig config MATH_EMULATION · line 318

Quoted 3 lines
arch/powerpc/Kconfig +3 -0
+config PPC_TRANSACTIONAL_MEM
+       bool "Transactional Memory support for POWERPC"
+       depends on PPC64

Should this really depend on PPC_BOOK3S_64 ?

OK, thanks.

Mikey

#57 lore raw

PATCH 16/17 powerpc: Add transactional memory to pseries and ppc64 defconfigs

In reply to Michael Neuling
---
 arch/powerpc/configs/ppc64_defconfig   |    2 ++
 arch/powerpc/configs/pseries_defconfig |    2 ++
 2 files changed, 4 insertions(+)
arch/powerpc/configs/ppc64_defconfig +2 -0
index 6cbdeb4..aef3f71 100644
@@ -49,6 +49,8 @@ CONFIG_CPU_FREQ_GOV_USERSPACE=y
 CONFIG_CPU_FREQ_PMAC64=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_PPC_TRANSACTIONAL_MEM=y
+CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
 CONFIG_MEMORY_HOTREMOVE=y
arch/powerpc/configs/pseries_defconfig +2 -0
index 04fa7f2..c4dfbaf 100644
@@ -43,6 +43,8 @@ CONFIG_RTAS_FLASH=m
 CONFIG_IBMEBUS=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_PPC_TRANSACTIONAL_MEM=y
+CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
 CONFIG_MEMORY_HOTPLUG=y
Signature · 1 line
1.7.10.4
#58 lore raw

PATCH 17/17 powerpc: Documentation for transactional memory on powerpc

In reply to Michael Neuling
---
 Documentation/powerpc/transactional_memory.txt |  175 ++++++++++++++++++++++++
 1 file changed, 175 insertions(+)
 create mode 100644 Documentation/powerpc/transactional_memory.txt
Documentation/powerpc/transactional_memory.txt new file +175 -0
new file mode 100644
index 0000000..c907be4
@@ -0,0 +1,175 @@
+Transactional Memory support
+============================
+
+POWER kernel support for this feature is currently limited to supporting
+its use by user programs.  It is not currently used by the kernel itself.
+
+This file aims to sum up how it is supported by Linux and what behaviour you
+can expect from your user programs.
+
+
+Basic overview
+==============
+
+Hardware Transactional Memory is supported on POWER8 processors, and is a
+feature that enables a different form of atomic memory access.  Several new
+instructions are presented to delimit transactions; transactions are
+guaranteed to either complete atomically or roll back and undo any partial
+changes.
+
+A simple transaction looks like this:
+
+begin_move_money:
+  tbegin
+  beq   abort_handler
+
+  ld    r4, SAVINGS_ACCT(r3)
+  ld    r5, CURRENT_ACCT(r3)
+  subi  r5, r5, 1
+  addi  r4, r4, 1
+  std   r4, SAVINGS_ACCT(r3)
+  std   r5, CURRENT_ACCT(r3)
+
+  tend
+
+  b     continue
+
+abort_handler:
+  ... test for odd failures ...
+
+  /* Retry the transaction if it failed because it conflicted with
+   * someone else: */
+  b     begin_move_money
+
+
+The 'tbegin' instruction denotes the start point, and 'tend' the end point.
+Between these points the processor is in 'Transactional' state; any memory
+references will complete in one go if there are no conflicts with other
+transactional or non-transactional accesses within the system.  In this
+example, the transaction completes as though it were normal straight-line code
+IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an
+atomic move of money from the current account to the savings account has been
+performed.  Even though the normal ld/std instructions are used (note no
+lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be
+updated, or neither will be updated.
+
+If, in the meantime, there is a conflict with the locations accessed by the
+transaction, the transaction will be aborted by the CPU.  Register and memory
+state will roll back to that at the 'tbegin', and control will continue from
+'tbegin+4'.  The branch to abort_handler will be taken this second time; the
+abort handler can check the cause of the failure, and retry.
+
+Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR
+and a few other status/flag regs; see the ISA for details.
+
+Causes of transaction aborts
+============================
+
+- Conflicts with cache lines used by other processors
+- Signals
+- Context switches
+- See the ISA for full documentation of everything that will abort transactions.
+
+
+Syscalls
+========
+
+Performing syscalls from within transaction is not recommended, and can lead
+to unpredictable results.
+
+Syscalls do not by design abort transactions, but beware: The kernel code will
+not be running in transactional state.  The effect of syscalls will always
+remain visible, but depending on the call they may abort your transaction as a
+side-effect, read soon-to-be-aborted transactional data that should not remain
+invisible, etc.  If you constantly retry a transaction that constantly aborts
+itself by calling a syscall, you'll have a livelock & make no progress.
+
+Simple syscalls (e.g. sigprocmask()) "could" be OK.  Even things like write()
+from, say, printf() should be OK as long as the kernel does not access any
+memory that was accessed transactionally.
+
+Consider any syscalls that happen to work as debug-only -- not recommended for
+production use.  Best to queue them up till after the transaction is over.
+
+
+Signals
+=======
+
+Delivery of signals (both sync and async) during transactions provides a second
+thread state (ucontext/mcontext) to represent the second transactional register
+state.  Signal delivery 'treclaim's to capture both register states, so signals
+abort transactions.  The usual ucontext_t passed to the signal handler
+represents the checkpointed/original register state; the signal appears to have
+arisen at 'tbegin+4'.
+
+If the sighandler ucontext has uc_link set, a second ucontext has been
+delivered.  For future compatibility the MSR.TS field should be checked to
+determine the transactional state -- if so, the second ucontext in uc->uc_link
+represents the active transactional registers at the point of the signal.
+
+For 64-bit processes, uc->uc_mcontext.regs->msr is a full 64-bit MSR and its TS
+field shows the transactional mode.
+
+For 32-bit processes, the mcontext's MSR register is only 32 bits; the top 32
+bits are stored in the MSR of the second ucontext, i.e. in
+uc->uc_link->uc_mcontext.regs->msr.  The top word contains the transactional
+state TS.
+
+However, basic signal handlers don't need to be aware of transactions
+and simply returning from the handler will deal with things correctly:
+
+Transaction-aware signal handlers can read the transactional register state
+from the second ucontext.  This will be necessary for crash handlers to
+determine, for example, the address of the instruction causing the SIGSEGV.
+
+Example signal handler:
+
+    void crash_handler(int sig, siginfo_t *si, void *uc)
+    {
+      ucontext_t *ucp = uc;
+      ucontext_t *transactional_ucp = ucp->uc_link;
+
+      if (ucp_link) {
+        u64 msr = ucp->uc_mcontext.regs->msr;
+        /* May have transactional ucontext! */
+#ifndef __powerpc64__
+        msr |= ((u64)transactional_ucp->uc_mcontext.regs->msr) << 32;
+#endif
+        if (MSR_TM_ACTIVE(msr)) {
+           /* Yes, we crashed during a transaction.  Oops. */
+   fprintf(stderr, "Transaction to be restarted at 0x%llx, but "
+                           "crashy instruction was at 0x%llx\n",
+                           ucp->uc_mcontext.regs->nip,
+                           transactional_ucp->uc_mcontext.regs->nip);
+        }
+      }
+
+      fix_the_problem(ucp->dar);
+    }
+
+
+Failure cause codes used by kernel
+==================================
+
+These are defined in <asm/reg.h>, and distinguish different reasons why the
+kernel aborted a transaction:
+
+ TM_CAUSE_RESCHED       Thread was rescheduled.
+ TM_CAUSE_FAC_UNAV      FP/VEC/VSX unavailable trap.
+ TM_CAUSE_SYSCALL       Currently unused; future syscalls that must abort
+                        transactions for consistency will use this.
+ TM_CAUSE_SIGNAL        Signal delivered.
+ TM_CAUSE_MISC          Currently unused.
+
+These can be checked by the user program's abort handler as TEXASR[0:7].
+
+
+GDB
+===
+
+GDB and ptrace are not currently TM-aware.  If one stops during a transaction,
+it looks like the transaction has just started (the checkpointed state is
+presented).  The transaction cannot then be continued and will take the failure
+handler route.  Furthermore, the transactional 2nd register state will be
+inaccessible.  GDB can currently be used on programs using TM, but not sensibly
+in parts within transactions.
Signature · 1 line
1.7.10.4
#59 lore raw

cover powerpc: Hardware transactional memory support for POWER8

In reply to Michael Neuling

POWER8 implements hardware transactional memory support. This patch series adds kernel support so that user programs can use this hardware transactional memory and the new state is properly context switched. It is not currently used by the kernel itself.

This patch series was originally developed by Matt Evans.

v4 changes:
Incorporate reviews from Kumar.

v3 changes:
Incorporate review from benh (offline conversation):
- Improve IRQ disabling for tm_reclaim
- Improve IRQ disabling for early fp/vmx/vsx unavail handling
- Change CONFIG option to add PPC (now PPC_TRANSACTIONAL_MEM)
- strip some unneeded #defines
- micro optimisation to asm in early fp/vmx/vsx

      -       srdi    r0, r12, MSR_TS_LG
      -       andi.   r0, r0, 3
      +       rldicl. r0, r12, (64-MSR_TS_LG), (64-2)

- do tabort on unmap only when current is using TM
- removed unneeded patch which moved when writing MSR back to the stack

v2 changes:
Adds signal handling
Removes some code for detecting when to restore all register or not Added tabort for local tlb invalidates
Rebased on benh's next tree

Basic overview of a POWER8 hardware transaction memory
=====================================================
Hardware transactional memory is a feature that enables a different form of atomic memory access. Several new instructions are presented to delimit transactions; transactions are guaranteed to either complete atomically or roll back and undo any partial changes.

A simple transaction looks like this:

begin_move_money:
tbegin
beq abort_handler

  ld    r4, SAVINGS_ACCT(r3)
  ld    r5, CURRENT_ACCT(r3)
  subi  r5, r5, 1
  addi  r4, r4, 1
  std   r4, SAVINGS_ACCT(r3)
  std   r5, CURRENT_ACCT(r3)

tend

b continue

abort_handler:
... test for odd failures ...

/* Retry the transaction if it failed because it conflicted with
* someone else: */
b begin_move_money

The 'tbegin' instruction denotes the start point, and 'tend' the end point. Between these points the processor is in 'Transactional' state; any memory references will complete in one go if there are no conflicts with other transactional or non-transactional accesses within the system. In this example, the transaction completes as though it were normal straight-line code IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an atomic move of money from the current account to the savings account has been performed. Even though the normal ld/std instructions are used (note no lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be updated, or neither will be updated.

If, in the meantime, there is a conflict with the locations accessed by the transaction, the transaction will be aborted by the CPU. Register and memory state will roll back to that at the 'tbegin', and control will continue from 'tbegin+4'. The branch to abort_handler will be taken this second time; the abort handler can check the cause of the failure, and retry.

Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR and a few other status/flag regs;

A preliminary ISA for TM can be found here:
https://www.power.org/documentation/power-isa-transactional-memory/

Signature · 1 line
1.7.10.4
#60 lore raw

PATCH 1/17 powerpc: Add new CPU feature bit for transactional memory

---
 arch/powerpc/include/asm/cputable.h |    7 +++++++
 1 file changed, 7 insertions(+)
arch/powerpc/include/asm/cputable.h +7 -0
index 5f1938f..3636081 100644
@@ -220,6 +220,13 @@ extern const char *powerpc_base_platform;
 #define PPC_FEATURE_HAS_EFP_DOUBLE_COMP 0
 #endif
 
+/* We only set the TM feature if the kernel was compiled with TM supprt */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+#define CPU_FTR_TM_COMP		CPU_FTR_TM
+#else
+#define CPU_FTR_TM_COMP		0
+#endif
+
 /* We need to mark all pages as being coherent if we're SMP or we have a
  * 74[45]x and an MPC107 host bridge. Also 83xx and PowerQUICC II
  * require it for PCI "streaming/prefetch" to work properly.
Signature · 1 line
1.7.10.4
#61 lore raw

PATCH 2/17 powerpc: Add new instructions for transactional memory

In reply to Michael Neuling

Here we define the new instructions we need for transactional memory in the kernel. This is so we can support compiling with binutils that don't support the new transactional memory instructions.

Transactional memory results in two sets of architected state (GPRs/VSRs etc).

treclaim allows us to read the checkpointed state (from the tbegin) so that we can store it away on a context switch. It does this by overwriting the exiting architected state, so you have to save that away before you treclaim. treclaim will also abort a transaction, so you can give a register value which contains an abort reason.

trecheckpoint allows us to inject into the checkpointed state as if it were at the tbegin. It does this by copying the current architected state into the checkpointed state.

---
 arch/powerpc/include/asm/ppc-opcode.h |   10 ++++++++++
 1 file changed, 10 insertions(+)
arch/powerpc/include/asm/ppc-opcode.h +10 -0
index 0fd1928..8752bc8 100644
@@ -129,6 +129,9 @@
 #define PPC_INST_TLBSRX_DOT		0x7c0006a5
 #define PPC_INST_XXLOR			0xf0000510
 #define PPC_INST_XVCPSGNDP		0xf0000780
+#define PPC_INST_TRECHKPT		0x7c0007dd
+#define PPC_INST_TRECLAIM		0x7c00075d
+#define PPC_INST_TABORT			0x7c00071d
 
 #define PPC_INST_NAP			0x4c000364
 #define PPC_INST_SLEEP			0x4c0003a4
@@ -294,4 +297,11 @@
 #define PPC_NAP			stringify_in_c(.long PPC_INST_NAP)
 #define PPC_SLEEP		stringify_in_c(.long PPC_INST_SLEEP)
 
+/* Transactional memory instructions */
+#define TRECHKPT		stringify_in_c(.long PPC_INST_TRECHKPT)
+#define TRECLAIM(r)		stringify_in_c(.long PPC_INST_TRECLAIM \
+					       | __PPC_RA(r))
+#define TABORT(r)		stringify_in_c(.long PPC_INST_TABORT \
+					       | __PPC_RA(r))
+
 #endif /* _ASM_POWERPC_PPC_OPCODE_H */
Signature · 1 line
1.7.10.4
#62 lore raw

PATCH 3/17 powerpc: Add additional state needed for transactional memory to thread struct

In reply to Michael Neuling

Set of new archtected state for saving away on context switch.

---
 arch/powerpc/include/asm/processor.h |   28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
arch/powerpc/include/asm/processor.h +28 -0
index 42ac53c..fc41ab3 100644
@@ -246,6 +246,34 @@ struct thread_struct {
 	unsigned long	spefscr;	/* SPE & eFP status */
 	int		used_spe;	/* set if process has used spe */
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	u64		tm_tfhar;	/* Transaction fail handler addr */
+	u64		tm_texasr;	/* Transaction exception & summary */
+	u64		tm_tfiar;	/* Transaction fail instr address reg */
+	unsigned long	tm_orig_msr;	/* Thread's MSR on ctx switch */
+	struct pt_regs	ckpt_regs;	/* Checkpointed registers */
+
+	/*
+	 * Transactional FP and VSX 0-31 register set.
+	 * NOTE: the sense of these is the opposite of the integer ckpt_regs!
+	 *
+	 * When a transaction is active/signalled/scheduled etc., *regs is the
+	 * most recent set of/speculated GPRs with ckpt_regs being the older
+	 * checkpointed regs to which we roll back if transaction aborts.
+	 *
+	 * However, fpr[] is the checkpointed 'base state' of FP regs, and
+	 * transact_fpr[] is the new set of transactional values.
+	 * VRs work the same way.
+	 */
+	double		transact_fpr[32][TS_FPRWIDTH];
+	struct {
+		unsigned int pad;
+		unsigned int val;	/* Floating point status */
+	} transact_fpscr;
+	vector128	transact_vr[32] __attribute__((aligned(16)));
+	vector128	transact_vscr __attribute__((aligned(16)));
+	unsigned long	transact_vrsave;
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 #ifdef CONFIG_KVM_BOOK3S_32_HANDLER
 	void*		kvm_shadow_vcpu; /* KVM internal data */
 #endif /* CONFIG_KVM_BOOK3S_32_HANDLER */
Signature · 1 line
1.7.10.4
#63 lore raw

PATCH 4/17 powerpc: New macros for transactional memory support

In reply to Michael Neuling

This adds new macros for saving and restoring checkpointed architected state from and to the thread_struct.

It also adds some debugging macros for when your brain explodes trying to debug your transactional memory enabled kernel.

---
 arch/powerpc/include/asm/ppc_asm.h   |   83 ++++++++++++++++++++++++++++++++++
 arch/powerpc/include/asm/processor.h |    1 +
 arch/powerpc/kernel/asm-offsets.c    |   23 ++++++++++
 arch/powerpc/kernel/fpu.S            |   12 +++++
 arch/powerpc/kernel/process.c        |    7 +++
 arch/powerpc/kernel/traps.c          |    8 ++++
 6 files changed, 134 insertions(+)
arch/powerpc/include/asm/ppc_asm.h +83 -0
index c2d0e58..54219ce 100644
@@ -123,6 +123,89 @@ END_FW_FTR_SECTION_IFSET(FW_FEATURE_SPLPAR)
 #define REST_16VRS(n,b,base)	REST_8VRS(n,b,base); REST_8VRS(n+8,b,base)
 #define REST_32VRS(n,b,base)	REST_16VRS(n,b,base); REST_16VRS(n+16,b,base)
 
+/* Save/restore FPRs, VRs and VSRs from their checkpointed backups in
+ * thread_struct:
+ */
+#define SAVE_FPR_TRANSACT(n, base)	stfd n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define SAVE_2FPRS_TRANSACT(n, base)	SAVE_FPR_TRANSACT(n, base);	\
+					SAVE_FPR_TRANSACT(n+1, base)
+#define SAVE_4FPRS_TRANSACT(n, base)	SAVE_2FPRS_TRANSACT(n, base);	\
+					SAVE_2FPRS_TRANSACT(n+2, base)
+#define SAVE_8FPRS_TRANSACT(n, base)	SAVE_4FPRS_TRANSACT(n, base);	\
+					SAVE_4FPRS_TRANSACT(n+4, base)
+#define SAVE_16FPRS_TRANSACT(n, base)	SAVE_8FPRS_TRANSACT(n, base);	\
+					SAVE_8FPRS_TRANSACT(n+8, base)
+#define SAVE_32FPRS_TRANSACT(n, base)	SAVE_16FPRS_TRANSACT(n, base);	\
+					SAVE_16FPRS_TRANSACT(n+16, base)
+
+#define REST_FPR_TRANSACT(n, base)	lfd	n,THREAD_TRANSACT_FPR0+	\
+					8*TS_FPRWIDTH*(n)(base)
+#define REST_2FPRS_TRANSACT(n, base)	REST_FPR_TRANSACT(n, base);	\
+					REST_FPR_TRANSACT(n+1, base)
+#define REST_4FPRS_TRANSACT(n, base)	REST_2FPRS_TRANSACT(n, base);	\
+					REST_2FPRS_TRANSACT(n+2, base)
+#define REST_8FPRS_TRANSACT(n, base)	REST_4FPRS_TRANSACT(n, base);	\
+					REST_4FPRS_TRANSACT(n+4, base)
+#define REST_16FPRS_TRANSACT(n, base)	REST_8FPRS_TRANSACT(n, base);	\
+					REST_8FPRS_TRANSACT(n+8, base)
+#define REST_32FPRS_TRANSACT(n, base)	REST_16FPRS_TRANSACT(n, base);	\
+					REST_16FPRS_TRANSACT(n+16, base)
+
+
+#define SAVE_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					stvx n,b,base
+#define SAVE_2VRS_TRANSACT(n,b,base)	SAVE_VR_TRANSACT(n,b,base);	\
+					SAVE_VR_TRANSACT(n+1,b,base)
+#define SAVE_4VRS_TRANSACT(n,b,base)	SAVE_2VRS_TRANSACT(n,b,base);	\
+					SAVE_2VRS_TRANSACT(n+2,b,base)
+#define SAVE_8VRS_TRANSACT(n,b,base)	SAVE_4VRS_TRANSACT(n,b,base);	\
+					SAVE_4VRS_TRANSACT(n+4,b,base)
+#define SAVE_16VRS_TRANSACT(n,b,base)	SAVE_8VRS_TRANSACT(n,b,base);	\
+					SAVE_8VRS_TRANSACT(n+8,b,base)
+#define SAVE_32VRS_TRANSACT(n,b,base)	SAVE_16VRS_TRANSACT(n,b,base);	\
+					SAVE_16VRS_TRANSACT(n+16,b,base)
+
+#define REST_VR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VR0+(16*(n)); \
+					lvx n,b,base
+#define REST_2VRS_TRANSACT(n,b,base)	REST_VR_TRANSACT(n,b,base);	\
+					REST_VR_TRANSACT(n+1,b,base)
+#define REST_4VRS_TRANSACT(n,b,base)	REST_2VRS_TRANSACT(n,b,base);	\
+					REST_2VRS_TRANSACT(n+2,b,base)
+#define REST_8VRS_TRANSACT(n,b,base)	REST_4VRS_TRANSACT(n,b,base);	\
+					REST_4VRS_TRANSACT(n+4,b,base)
+#define REST_16VRS_TRANSACT(n,b,base)	REST_8VRS_TRANSACT(n,b,base);	\
+					REST_8VRS_TRANSACT(n+8,b,base)
+#define REST_32VRS_TRANSACT(n,b,base)	REST_16VRS_TRANSACT(n,b,base);	\
+					REST_16VRS_TRANSACT(n+16,b,base)
+
+
+#define SAVE_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					STXVD2X(n,R##base,R##b)
+#define SAVE_2VSRS_TRANSACT(n,b,base)	SAVE_VSR_TRANSACT(n,b,base);	\
+	                                SAVE_VSR_TRANSACT(n+1,b,base)
+#define SAVE_4VSRS_TRANSACT(n,b,base)	SAVE_2VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_2VSRS_TRANSACT(n+2,b,base)
+#define SAVE_8VSRS_TRANSACT(n,b,base)	SAVE_4VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_4VSRS_TRANSACT(n+4,b,base)
+#define SAVE_16VSRS_TRANSACT(n,b,base)	SAVE_8VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_8VSRS_TRANSACT(n+8,b,base)
+#define SAVE_32VSRS_TRANSACT(n,b,base)	SAVE_16VSRS_TRANSACT(n,b,base);	\
+	                                SAVE_16VSRS_TRANSACT(n+16,b,base)
+
+#define REST_VSR_TRANSACT(n,b,base)	li b,THREAD_TRANSACT_VSR0+(16*(n)); \
+					LXVD2X(n,R##base,R##b)
+#define REST_2VSRS_TRANSACT(n,b,base)	REST_VSR_TRANSACT(n,b,base);    \
+	                                REST_VSR_TRANSACT(n+1,b,base)
+#define REST_4VSRS_TRANSACT(n,b,base)	REST_2VSRS_TRANSACT(n,b,base);	\
+	                                REST_2VSRS_TRANSACT(n+2,b,base)
+#define REST_8VSRS_TRANSACT(n,b,base)	REST_4VSRS_TRANSACT(n,b,base);	\
+	                                REST_4VSRS_TRANSACT(n+4,b,base)
+#define REST_16VSRS_TRANSACT(n,b,base)	REST_8VSRS_TRANSACT(n,b,base);	\
+	                                REST_8VSRS_TRANSACT(n+8,b,base)
+#define REST_32VSRS_TRANSACT(n,b,base)	REST_16VSRS_TRANSACT(n,b,base);	\
+	                                REST_16VSRS_TRANSACT(n+16,b,base)
+
 /* Save the lower 32 VSRs in the thread VSR region */
 #define SAVE_VSR(n,b,base)	li b,THREAD_VSR0+(16*(n));  STXVD2X(n,R##base,R##b)
 #define SAVE_2VSRS(n,b,base)	SAVE_VSR(n,b,base); SAVE_VSR(n+1,b,base)
arch/powerpc/include/asm/processor.h +1 -0
index fc41ab3..7ff9eaa 100644
@@ -152,6 +152,7 @@ typedef struct {
 #define TS_FPROFFSET 0
 #define TS_VSRLOWOFFSET 1
 #define TS_FPR(i) fpr[i][TS_FPROFFSET]
+#define TS_TRANS_FPR(i) transact_fpr[i][TS_FPROFFSET]
 
 struct thread_struct {
 	unsigned long	ksp;		/* Kernel stack pointer */
arch/powerpc/kernel/asm-offsets.c +23 -0
index beddba4..b67e4c3 100644
@@ -125,6 +125,29 @@ int main(void)
 #ifdef CONFIG_PPC_BOOK3S_64
 	DEFINE(THREAD_TAR, offsetof(struct thread_struct, tar));
 #endif
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
+	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
+	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
+	DEFINE(PT_CKPT_REGS, offsetof(struct thread_struct, ckpt_regs));
+	DEFINE(THREAD_TRANSACT_VR0, offsetof(struct thread_struct,
+					 transact_vr[0]));
+	DEFINE(THREAD_TRANSACT_VSCR, offsetof(struct thread_struct,
+					  transact_vscr));
+	DEFINE(THREAD_TRANSACT_VRSAVE, offsetof(struct thread_struct,
+					    transact_vrsave));
+	DEFINE(THREAD_TRANSACT_FPR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+	DEFINE(THREAD_TRANSACT_FPSCR, offsetof(struct thread_struct,
+					   transact_fpscr));
+#ifdef CONFIG_VSX
+	DEFINE(THREAD_TRANSACT_VSR0, offsetof(struct thread_struct,
+					  transact_fpr[0]));
+#endif
+	/* Local pt_regs on stack for Transactional Memory funcs. */
+	DEFINE(TM_FRAME_SIZE, STACK_FRAME_OVERHEAD +
+	       sizeof(struct pt_regs) + 16);
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 
 	DEFINE(TI_FLAGS, offsetof(struct thread_info, flags));
 	DEFINE(TI_LOCAL_FLAGS, offsetof(struct thread_info, local_flags));
arch/powerpc/kernel/fpu.S +12 -0
index e0ada05..adb1551 100644
@@ -35,6 +35,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 2:	REST_32VSRS(n,c,base);						\
 3:
 
+#define __REST_32FPVSRS_TRANSACT(n,c,base)				\
+BEGIN_FTR_SECTION							\
+	b	2f;							\
+END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
+	REST_32FPRS_TRANSACT(n,base);					\
+	b	3f;							\
+2:	REST_32VSRS_TRANSACT(n,c,base);					\
+3:
+
 #define __SAVE_32FPVSRS(n,c,base)					\
 BEGIN_FTR_SECTION							\
 	b	2f;							\
@@ -45,9 +54,12 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 3:
 #else
 #define __REST_32FPVSRS(n,b,base)	REST_32FPRS(n, base)
+#define __REST_32FPVSRS_TRANSACT(n,b,base)	REST_32FPRS(n, base)
 #define __SAVE_32FPVSRS(n,b,base)	SAVE_32FPRS(n, base)
 #endif
 #define REST_32FPVSRS(n,c,base) __REST_32FPVSRS(n,__REG_##c,__REG_##base)
+#define REST_32FPVSRS_TRANSACT(n,c,base) \
+	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
 /*
arch/powerpc/kernel/process.c +7 -0
index 96e31de..b0a0321 100644
@@ -57,6 +57,13 @@
 #include <linux/kprobes.h>
 #include <linux/kdebug.h>
 
+/* Transactional Memory debug */
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+
 extern unsigned long _get_SP(void);
 
 #ifndef CONFIG_SMP
arch/powerpc/kernel/traps.c +8 -0
index a008cf5..bd5de5d 100644
@@ -78,6 +78,13 @@ EXPORT_SYMBOL(__debugger_break_match);
 EXPORT_SYMBOL(__debugger_fault_handler);
 #endif
 
+/* Transactional Memory trap debug */
+#ifdef TM_DEBUG_SW
+#define TM_DEBUG(x...) printk(KERN_INFO x)
+#else
+#define TM_DEBUG(x...) do { } while(0)
+#endif
+
 /*
  * Trap & Exception support
  */
@@ -350,6 +357,7 @@ static inline int check_io_access(struct pt_regs *regs)
    exception is in the MSR. */
 #define get_reason(regs)	((regs)->msr)
 #define get_mc_reason(regs)	((regs)->msr)
+#define REASON_TM		0x200000
 #define REASON_FP		0x100000
 #define REASON_ILLEGAL		0x80000
 #define REASON_PRIVILEGED	0x40000
Signature · 1 line
1.7.10.4
#64 lore raw

PATCH 5/17 powerpc: Register defines for various transactional memory registers

In reply to Michael Neuling

Defines for MSR bits and transactional memory related SPRs TFIAR, TEXASR and TEXASRU.

---
 arch/powerpc/include/asm/reg.h |   27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
arch/powerpc/include/asm/reg.h +27 -0
index e09ac51..7844c28 100644
@@ -29,6 +29,10 @@
 #define MSR_SF_LG	63              /* Enable 64 bit mode */
 #define MSR_ISF_LG	61              /* Interrupt 64b mode valid on 630 */
 #define MSR_HV_LG 	60              /* Hypervisor state */
+#define MSR_TS_T_LG	34		/* Trans Mem state: Transactional */
+#define MSR_TS_S_LG	33		/* Trans Mem state: Suspended */
+#define MSR_TS_LG	33		/* Trans Mem state (2 bits) */
+#define MSR_TM_LG	32		/* Trans Mem Available */
 #define MSR_VEC_LG	25	        /* Enable AltiVec */
 #define MSR_VSX_LG	23		/* Enable VSX */
 #define MSR_POW_LG	18		/* Enable Power Management */
@@ -98,6 +102,25 @@
 #define MSR_RI		__MASK(MSR_RI_LG)	/* Recoverable Exception */
 #define MSR_LE		__MASK(MSR_LE_LG)	/* Little Endian */
 
+#define MSR_TM		__MASK(MSR_TM_LG)	/* Transactional Mem Available */
+#define MSR_TS_N	0			/*  Non-transactional */
+#define MSR_TS_S	__MASK(MSR_TS_S_LG)	/*  Transaction Suspended */
+#define MSR_TS_T	__MASK(MSR_TS_T_LG)	/*  Transaction Transactional */
+#define MSR_TS_MASK	(MSR_TS_T | MSR_TS_S)   /* Transaction State bits */
+#define MSR_TM_ACTIVE(x) (((x) & MSR_TS_MASK) != 0) /* Transaction active? */
+#define MSR_TM_TRANSACTIONAL(x)	(((x) & MSR_TS_MASK) == MSR_TS_T)
+#define MSR_TM_SUSPENDED(x)	(((x) & MSR_TS_MASK) == MSR_TS_S)
+
+/* Reason codes describing kernel causes for transaction aborts.  By
+   convention, bit0 is copied to TEXASR[56] (IBM bit 7) which is set if
+   the failure is persistent.
+*/
+#define TM_CAUSE_RESCHED	0xfe
+#define TM_CAUSE_TLBI		0xfc
+#define TM_CAUSE_FAC_UNAV	0xfa
+#define TM_CAUSE_SYSCALL	0xf9 /* Persistent */
+#define TM_CAUSE_MISC		0xf6
+
 #if defined(CONFIG_PPC_BOOK3S_64)
 #define MSR_64BIT	MSR_SF
 
@@ -193,6 +216,10 @@
 #define SPRN_UAMOR	0x9d	/* User Authority Mask Override Register */
 #define SPRN_AMOR	0x15d	/* Authority Mask Override Register */
 #define SPRN_ACOP	0x1F	/* Available Coprocessor Register */
+#define SPRN_TFIAR	0x81	/* Transaction Failure Inst Addr   */
+#define SPRN_TEXASR	0x82	/* Transaction EXception & Summary */
+#define SPRN_TEXASRU	0x83	/* ''	   ''	   ''	 Upper 32  */
+#define SPRN_TFHAR	0x80	/* Transaction Failure Handler Addr */
 #define SPRN_CTRLF	0x088
 #define SPRN_CTRLT	0x098
 #define   CTRL_CT	0xc0000000	/* current thread */
Signature · 1 line
1.7.10.4
#65 lore raw

PATCH 6/17 powerpc: Add transactional memory paca scratch register to show_regs

In reply to Michael Neuling

Add transactional memory paca scratch register to show_regs. This is useful for debugging.

---
 arch/powerpc/include/asm/paca.h   |    3 +++
 arch/powerpc/kernel/asm-offsets.c |    1 +
 arch/powerpc/kernel/entry_64.S    |    4 ++++
 arch/powerpc/kernel/process.c     |    3 +++
 4 files changed, 11 insertions(+)
arch/powerpc/include/asm/paca.h +3 -0
index c47d687..4cbcc7d 100644
@@ -137,6 +137,9 @@ struct paca_struct {
 	u8 irq_work_pending;		/* IRQ_WORK interrupt while soft-disable */
 	u8 nap_state_lost;		/* NV GPR values lost in power7_idle */
 	u64 sprg3;			/* Saved user-visible sprg */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	u64 tm_scratch;                 /* TM scratch area for reclaim */
+#endif
 
 #ifdef CONFIG_PPC_POWERNV
 	/* Pointer to OPAL machine check event structure set by the
arch/powerpc/kernel/asm-offsets.c +1 -0
index b67e4c3..f9654b6 100644
@@ -126,6 +126,7 @@ int main(void)
 	DEFINE(THREAD_TAR, offsetof(struct thread_struct, tar));
 #endif
 #ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	DEFINE(PACATMSCRATCH, offsetof(struct paca_struct, tm_scratch));
 	DEFINE(THREAD_TM_TFHAR, offsetof(struct thread_struct, tm_tfhar));
 	DEFINE(THREAD_TM_TEXASR, offsetof(struct thread_struct, tm_texasr));
 	DEFINE(THREAD_TM_TFIAR, offsetof(struct thread_struct, tm_tfiar));
arch/powerpc/kernel/entry_64.S +4 -0
index 9ae8451..612ea13 100644
@@ -785,6 +785,10 @@ fast_exception_return:
 	andc	r4,r4,r0	 /* r0 contains MSR_RI here */
 	mtmsrd	r4,1
 
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	/* TM debug */
+	std	r3, PACATMSCRATCH(r13) /* Stash returned-to MSR */
+#endif
 	/*
 	 * r13 is our per cpu area, only restore it if we are returning to
 	 * userspace the value stored in the stack frame may belong to
arch/powerpc/kernel/process.c +3 -0
index b0a0321..1cc4053 100644
@@ -754,6 +754,9 @@ void show_regs(struct pt_regs * regs)
 	printk("NIP ["REG"] %pS\n", regs->nip, (void *)regs->nip);
 	printk("LR ["REG"] %pS\n", regs->link, (void *)regs->link);
 #endif
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	printk("PACATMSCRATCH [%llx]\n", get_paca()->tm_scratch);
+#endif
 	show_stack(current, (unsigned long *) regs->gpr[1]);
 	if (!user_mode(regs))
 		show_instructions(regs);
Signature · 1 line
1.7.10.4
#66 lore raw

PATCH 7/17 powerpc: Add helper functions for transactional memory context switching

In reply to Michael Neuling

Here we add the helper functions to be used when context switching. These allow us to fully reclaim and recheckpoint a transaction.

We introduce a new paca field called tm_scratch to help us store away register values when doing the low level tm reclaim register save.

---
 arch/powerpc/include/asm/reg.h |    2 +-
 arch/powerpc/include/asm/tm.h  |   20 +++
 arch/powerpc/kernel/Makefile   |    2 +
 arch/powerpc/kernel/tm.S       |  388 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 411 insertions(+), 1 deletion(-)
 create mode 100644 arch/powerpc/include/asm/tm.h
 create mode 100644 arch/powerpc/kernel/tm.S
arch/powerpc/include/asm/reg.h +1 -1
index 7844c28..eee2a60 100644
@@ -797,7 +797,7 @@
  *        HV mode in which case it is HSPRG0
  *
  * 64-bit server:
- *	- SPRG0 unused (reserved for HV on Power4)
+ *	- SPRG0 scratch for TM recheckpoint/reclaim (reserved for HV on Power4)
  *	- SPRG2 scratch for exception vectors
  *	- SPRG3 CPU and NUMA node for VDSO getcpu (user visible)
  *      - HSPRG0 stores PACA in HV mode
arch/powerpc/include/asm/tm.h new file +20 -0
new file mode 100644
index 0000000..4b4449a
@@ -0,0 +1,20 @@
+/*
+ * Transactional memory support routines to reclaim and recheckpoint
+ * transactional process state.
+ *
+ * Copyright 2012 Matt Evans & Michael Neuling, IBM Corporation.
+ */
+
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+extern void do_load_up_transact_fpu(struct thread_struct *thread);
+extern void do_load_up_transact_altivec(struct thread_struct *thread);
+#endif
+
+extern void tm_enable(void);
+extern void tm_reclaim(struct thread_struct *thread,
+		       unsigned long orig_msr, uint8_t cause);
+extern void tm_recheckpoint(struct thread_struct *thread,
+			    unsigned long orig_msr);
+extern void tm_abort(uint8_t cause);
+extern void tm_save_sprs(struct thread_struct *thread);
+extern void tm_restore_sprs(struct thread_struct *thread);
arch/powerpc/kernel/Makefile +2 -0
index b4f0e36..f960a79 100644
@@ -121,6 +121,8 @@ ifneq ($(CONFIG_PPC_INDIRECT_IO),y)
 obj-y				+= iomap.o
 endif
 
+obj64-$(CONFIG_PPC_TRANSACTIONAL_MEM)	+= tm.o
+
 obj-$(CONFIG_PPC64)		+= $(obj64-y)
 obj-$(CONFIG_PPC32)		+= $(obj32-y)
 
arch/powerpc/kernel/tm.S new file +388 -0

388 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#67 lore raw

PATCH 8/17 powerpc: Add FP/VSX and VMX register load functions for transactional memory

In reply to Michael Neuling

This adds functions to restore the state of the FP/VSX registers from what's stored in the thread_struct. Two version for FP/VSX are required since one restores them from transactional/checkpoint side of the thread_struct and the other from the speculated side.

Similar functions are added for VMX registers.

---
 arch/powerpc/kernel/fpu.S    |   54 ++++++++++++++++++++++++++++++++++++++++++
 arch/powerpc/kernel/vector.S |   51 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+)
arch/powerpc/kernel/fpu.S +54 -0
index adb1551..caeaabf 100644
@@ -62,6 +62,60 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX);					\
 	__REST_32FPVSRS_TRANSACT(n,__REG_##c,__REG_##base)
 #define SAVE_32FPVSRS(n,c,base) __SAVE_32FPVSRS(n,__REG_##c,__REG_##base)
 
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+/*
+ * Wrapper to call load_up_fpu from C.
+ * void do_load_up_fpu(struct pt_regs *regs);
+ */
+_GLOBAL(do_load_up_fpu)
+	mflr	r0
+	std	r0, 16(r1)
+	stdu	r1, -112(r1)
+
+	subi	r6, r3, STACK_FRAME_OVERHEAD
+	/* load_up_fpu expects r12=MSR, r13=PACA, and returns
+	 * with r12 = new MSR.
+	 */
+	ld	r12,_MSR(r6)
+	GET_PACA(r13)
+
+	bl	load_up_fpu
+	std	r12,_MSR(r6)
+
+	ld	r0, 112+16(r1)
+	addi	r1, r1, 112
+	mtlr	r0
+	blr
+
+
+/* void do_load_up_transact_fpu(struct thread_struct *thread)
+ *
+ * This is similar to load_up_fpu but for the transactional version of the FP
+ * register set.  It doesn't mess with the task MSR or valid flags.
+ * Furthermore, we don't do lazy FP with TM currently.
+ */
+_GLOBAL(do_load_up_transact_fpu)
+	mfmsr	r6
+	ori	r5,r6,MSR_FP
+#ifdef CONFIG_VSX
+BEGIN_FTR_SECTION
+	oris	r5,r5,MSR_VSX@h
+END_FTR_SECTION_IFSET(CPU_FTR_VSX)
+#endif
+	SYNC
+	MTMSRD(r5)
+
+	lfd	fr0,THREAD_TRANSACT_FPSCR(r3)
+	MTFSF_L(fr0)
+	REST_32FPVSRS_TRANSACT(0, R4, R3)
+
+	/* FP/VSX off again */
+	MTMSRD(r6)
+	SYNC
+
+	blr
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+
 /*
  * This task wants to use the FPU now.
  * On UP, disable FP for the task which had the FPU previously,
arch/powerpc/kernel/vector.S +51 -0

57 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#68 lore raw

PATCH 9/17 powerpc: Add reclaim and recheckpoint functions for context switching transactional memory processes

In reply to Michael Neuling

When we switch out a task, we need to save both the checkpointed and the speculated state into the thread struct.

Similarly when we are switching in a task we need to load both the checkpointed and speculated state. If the task was using FP, we non-lazily reload both the original and the speculative FP register states. This is because the kernel doesn't see if/when a TM rollback occurs, so if we take an FP unavoidable later, we are unable to determine which set of FP regs need to be restored.

This simply adds these functions. It doesn't hook them into the existing code yet.

---
 arch/powerpc/kernel/process.c |  112 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 112 insertions(+)
arch/powerpc/kernel/process.c +112 -0
index 1cc4053..48a9875 100644
@@ -50,6 +50,7 @@
 #include <asm/runlatch.h>
 #include <asm/syscalls.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 #ifdef CONFIG_PPC64
 #include <asm/firmware.h>
@@ -467,6 +468,117 @@ static inline bool hw_brk_match(struct arch_hw_breakpoint *a,
 		return false;
 	return true;
 }
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+static inline void tm_reclaim_task(struct task_struct *tsk)
+{
+	/* We have to work out if we're switching from/to a task that's in the
+	 * middle of a transaction.
+	 *
+	 * In switching we need to maintain a 2nd register state as
+	 * oldtask->thread.ckpt_regs.  We tm_reclaim(oldproc); this saves the
+	 * checkpointed (tbegin) state in ckpt_regs and saves the transactional
+	 * (current) FPRs into oldtask->thread.transact_fpr[].
+	 *
+	 * We also context switch (save) TFHAR/TEXASR/TFIAR in here.
+	 */
+	struct thread_struct *thr = &tsk->thread;
+
+	if (!thr->regs)
+		return;
+
+	if (!MSR_TM_ACTIVE(thr->regs->msr))
+		goto out_and_saveregs;
+
+	/* Stash the original thread MSR, as giveup_fpu et al will
+	 * modify it.  We hold onto it to see whether the task used
+	 * FP & vector regs.
+	 */
+	thr->tm_orig_msr = thr->regs->msr;
+
+	TM_DEBUG("--- tm_reclaim on pid %d (NIP=%lx, "
+		 "ccr=%lx, msr=%lx, trap=%lx)\n",
+		 tsk->pid, thr->regs->nip,
+		 thr->regs->ccr, thr->regs->msr,
+		 thr->regs->trap);
+
+	tm_reclaim(thr, thr->regs->msr, TM_CAUSE_RESCHED);
+
+	TM_DEBUG("--- tm_reclaim on pid %d complete\n",
+		 tsk->pid);
+
+out_and_saveregs:
+	/* Always save the regs here, even if a transaction's not active.
+	 * This context-switches a thread's TM info SPRs.  We do it here to
+	 * be consistent with the restore path (in recheckpoint) which
+	 * cannot happen later in _switch().
+	 */
+	tm_save_sprs(thr);
+}
+
+static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+{
+	unsigned long msr;
+
+	if (!cpu_has_feature(CPU_FTR_TM))
+		return;
+
+	/* Recheckpoint the registers of the thread we're about to switch to.
+	 *
+	 * If the task was using FP, we non-lazily reload both the original and
+	 * the speculative FP register states.  This is because the kernel
+	 * doesn't see if/when a TM rollback occurs, so if we take an FP
+	 * unavoidable later, we are unable to determine which set of FP regs
+	 * need to be restored.
+	 */
+	if (!new->thread.regs)
+		return;
+
+	/* The TM SPRs are restored here, so that TEXASR.FS can be set
+	 * before the trecheckpoint and no explosion occurs.
+	 */
+	tm_restore_sprs(&new->thread);
+
+	if (!MSR_TM_ACTIVE(new->thread.regs->msr))
+		return;
+	msr = new->thread.tm_orig_msr;
+	/* Recheckpoint to restore original checkpointed register state. */
+	TM_DEBUG("*** tm_recheckpoint of pid %d "
+		 "(new->msr 0x%lx, new->origmsr 0x%lx)\n",
+		 new->pid, new->thread.regs->msr, msr);
+
+	/* This loads the checkpointed FP/VEC state, if used */
+	tm_recheckpoint(&new->thread, msr);
+
+	/* This loads the speculative FP/VEC state, if used */
+	if (msr & MSR_FP) {
+		do_load_up_transact_fpu(&new->thread);
+		new->thread.regs->msr |=
+			(MSR_FP | new->thread.fpexc_mode);
+	}
+	if (msr & MSR_VEC) {
+		do_load_up_transact_altivec(&new->thread);
+		new->thread.regs->msr |= MSR_VEC;
+	}
+	/* We may as well turn on VSX too since all the state is restored now */
+	if (msr & MSR_VSX)
+		new->thread.regs->msr |= MSR_VSX;
+
+	TM_DEBUG("*** tm_recheckpoint of pid %d complete "
+		 "(kernel msr 0x%lx)\n",
+		 new->pid, mfmsr());
+}
+
+static inline void __switch_to_tm(struct task_struct *prev)
+{
+	if (cpu_has_feature(CPU_FTR_TM)) {
+		tm_enable();
+		tm_reclaim_task(prev);
+	}
+}
+#else
+#define tm_recheckpoint_new_task(new)
+#define __switch_to_tm(prev)
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 
 struct task_struct *__switch_to(struct task_struct *prev,
 	struct task_struct *new)
Signature · 1 line
1.7.10.4
#69 lore raw

PATCH 10/17 powerpc: Add transactional memory unavaliable execption handler

In reply to Michael Neuling

These should never happen since we always turn on MSR TM when in userspace. We don't do lazy TM.

Hence if we hit this, we barf and kill the task as something's gone horribly wrong.

---
 arch/powerpc/kernel/exceptions-64s.S |   19 +++++++++++++++++++
 arch/powerpc/kernel/traps.c          |   21 +++++++++++++++++++++
 2 files changed, 40 insertions(+)
arch/powerpc/kernel/exceptions-64s.S +19 -0
index 7a1c87c..d0cc657 100644
@@ -313,6 +313,9 @@ vsx_unavailable_pSeries_1:
 	. = 0xf40
 	b	vsx_unavailable_pSeries
 
+	. = 0xf60
+	b	tm_unavailable_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_EXCEPTION_HV(0x1200, 0x1202, cbe_system_error)
 	KVM_HANDLER_SKIP(PACA_EXGEN, EXC_HV, 0x1202)
@@ -528,6 +531,8 @@ ALT_FTR_SECTION_END_IFCLR(CPU_FTR_ARCH_206)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf20)
 	STD_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
 	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf40)
+	STD_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
+	KVM_HANDLER_PR(PACA_EXGEN, EXC_STD, 0xf60)
 
 /*
  * An interrupt came in while soft-disabled. We set paca->irq_happened, then:
@@ -817,6 +822,10 @@ vsx_unavailable_relon_pSeries_1:
 	. = 0x4f40
 	b	vsx_unavailable_relon_pSeries
 
+tm_unavailable_relon_pSeries_1:
+	. = 0x4f60
+	b	tm_unavailable_relon_pSeries
+
 #ifdef CONFIG_CBE_RAS
 	STD_RELON_EXCEPTION_HV(0x5200, 0x1202, cbe_system_error)
 #endif /* CONFIG_CBE_RAS */
@@ -1177,6 +1186,15 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 	b	.ret_from_except
 
 	.align	7
+	.globl tm_unavailable_common
+tm_unavailable_common:
+	EXCEPTION_PROLOG_COMMON(0xf60, PACA_EXGEN)
+	bl	.save_nvgprs
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.tm_unavailable_exception
+	b	.ret_from_except
+
+	.align	7
 	.globl	__end_handlers
 __end_handlers:
 
@@ -1195,6 +1213,7 @@ __end_handlers:
 	STD_RELON_EXCEPTION_PSERIES(., 0xf00, performance_monitor)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf20, altivec_unavailable)
 	STD_RELON_EXCEPTION_PSERIES(., 0xf40, vsx_unavailable)
+	STD_RELON_EXCEPTION_PSERIES(., 0xf60, tm_unavailable)
 
 #if defined(CONFIG_PPC_PSERIES) || defined(CONFIG_PPC_POWERNV)
 /*
arch/powerpc/kernel/traps.c +21 -0
index bd5de5d..f829e05 100644
@@ -1168,6 +1168,27 @@ void vsx_unavailable_exception(struct pt_regs *regs)
 	die("Unrecoverable VSX Unavailable Exception", regs, SIGABRT);
 }
 
+void tm_unavailable_exception(struct pt_regs *regs)
+{
+	/* We restore the interrupt state now */
+	if (!arch_irq_disabled_regs(regs))
+		local_irq_enable();
+
+	/* Currently we never expect a TMU exception.  Catch
+	 * this and kill the process!
+	 */
+	printk(KERN_EMERG "Unexpected TM unavailable exception at %lx "
+	       "(msr %lx)\n",
+	       regs->nip, regs->msr);
+
+	if (user_mode(regs)) {
+		_exception(SIGILL, regs, ILL_ILLOPC, regs->nip);
+		return;
+	}
+
+	die("Unexpected TM unavailable exception", regs, SIGABRT);
+}
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.10.4
#70 lore raw

PATCH 11/17 powerpc: Routines for FP/VSX/VMX unavailable during a transaction

In reply to Michael Neuling

We do lazy FP but not lazy TM (ie. userspace starts with MSR TM=1 FP=0). Hence if userspace does an FP instruction during a transaction, we'll take an fp unavailable exception.

This adds functions needed to handle this case. We have to inject the current FP state into the checkpoint so that the hardware can decide what to do with the transaction. We can't inject only the FP so we have to do a full treclaim and recheckpoint to inject just the FP state. This will cause the transaction to be marked as aborted by the hardware.

This just add the routines needed to do this for FP, VMX and VSX. It doesn't hook them into the rest of the code yet.

---
 arch/powerpc/kernel/traps.c |   83 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
arch/powerpc/kernel/traps.c +83 -0
index f829e05..5c30469 100644
@@ -58,6 +58,7 @@
 #include <asm/rio.h>
 #include <asm/fadump.h>
 #include <asm/switch_to.h>
+#include <asm/tm.h>
 #include <asm/debug.h>
 
 #if defined(CONFIG_DEBUGGER) || defined(CONFIG_KEXEC)
@@ -1189,6 +1190,88 @@ void tm_unavailable_exception(struct pt_regs *regs)
 	die("Unexpected TM unavailable exception", regs, SIGABRT);
 }
 
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+
+extern void do_load_up_fpu(struct pt_regs *regs);
+
+void fp_unavailable_tm(struct pt_regs *regs)
+{
+	/* Note:  This does not handle any kind of FP laziness. */
+
+	TM_DEBUG("FP Unavailable trap whilst transactional at 0x%lx, MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+
+        /* We can only have got here if the task started using FP after
+         * beginning the transaction.  So, the transactional regs are just a
+         * copy of the checkpointed ones.  But, we still need to recheckpoint
+         * as we're enabling FP for the process; it will return, abort the
+         * transaction, and probably retry but now with FP enabled.  So the
+         * checkpointed FP registers need to be loaded.
+	 */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	/* Reclaim didn't save out any FPRs to transact_fprs. */
+
+	/* Enable FP for the task: */
+	regs->msr |= (MSR_FP | current->thread.fpexc_mode);
+
+	/* This loads and recheckpoints the FP registers from
+	 * thread.fpr[].  They will remain in registers after the
+	 * checkpoint so we don't need to reload them after.
+	 */
+	tm_recheckpoint(&current->thread, regs->msr);
+}
+
+#ifdef CONFIG_ALTIVEC
+extern void do_load_up_altivec(struct pt_regs *regs);
+
+void altivec_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This function operates
+	 * the same way.
+	 */
+
+	TM_DEBUG("Vector Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+	tm_enable();
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+	regs->msr |= MSR_VEC;
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vr = 1;
+}
+#endif
+
+#ifdef CONFIG_VSX
+void vsx_unavailable_tm(struct pt_regs *regs)
+{
+	/* See the comments in fp_unavailable_tm().  This works similarly,
+	 * though we're loading both FP and VEC registers in here.
+	 *
+	 * If FP isn't in use, load FP regs.  If VEC isn't in use, load VEC
+	 * regs.  Either way, set MSR_VSX.
+	 */
+
+	TM_DEBUG("VSX Unavailable trap whilst transactional at 0x%lx,"
+		 "MSR=%lx\n",
+		 regs->nip, regs->msr);
+
+	tm_enable();
+	/* This reclaims FP and/or VR regs if they're already enabled */
+	tm_reclaim(&current->thread, current->thread.regs->msr,
+		   TM_CAUSE_FAC_UNAV);
+
+	regs->msr |= MSR_VEC | MSR_FP | current->thread.fpexc_mode |
+		MSR_VSX;
+	/* This loads & recheckpoints FP and VRs. */
+	tm_recheckpoint(&current->thread, regs->msr);
+	current->thread.used_vsr = 1;
+}
+#endif
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
+
 void performance_monitor_exception(struct pt_regs *regs)
 {
 	__get_cpu_var(irq_stat).pmu_irqs++;
Signature · 1 line
1.7.10.4
#71 lore raw

PATCH 12/17 powerpc: Hook in new transactional memory code

In reply to Michael Neuling

This hooks the new transactional memory code into context switching, FP/VMX/VMX unavailable and exception return.

---
 arch/powerpc/kernel/exceptions-64s.S |   56 ++++++++++++++++++++++++++++++++--
 arch/powerpc/kernel/process.c        |   15 +++++++--
 arch/powerpc/kernel/traps.c          |   32 +++++++++++++++++++
 arch/powerpc/mm/hash_utils_64.c      |   16 ++++++++++
 4 files changed, 115 insertions(+), 4 deletions(-)
arch/powerpc/kernel/exceptions-64s.S +54 -2
index d0cc657..85b923c 100644
@@ -1147,9 +1147,26 @@ fp_unavailable_common:
 	addi	r3,r1,STACK_FRAME_OVERHEAD
 	bl	.kernel_fp_unavailable_exception
 	BUG_OPCODE
-1:	bl	.load_up_fpu
+1:
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+BEGIN_FTR_SECTION
+	/* Test if 2 TM state bits are zero.  If non-zero (ie. userspace was in
+	 * transaction), go do TM stuff
+	 */
+	rldicl.	r0, r12, (64-MSR_TS_LG), (64-2)
+	bne-	2f
+END_FTR_SECTION_IFSET(CPU_FTR_TM)
+#endif
+	bl	.load_up_fpu
 	b	fast_exception_return
-
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	DISABLE_INTS
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.fp_unavailable_tm
+	b	.ret_from_except
+#endif
 	.align	7
 	.globl altivec_unavailable_common
 altivec_unavailable_common:
@@ -1157,8 +1174,25 @@ altivec_unavailable_common:
 #ifdef CONFIG_ALTIVEC
 BEGIN_FTR_SECTION
 	beq	1f
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+  BEGIN_FTR_SECTION_NESTED(69)
+	/* Test if 2 TM state bits are zero.  If non-zero (ie. userspace was in
+	 * transaction), go do TM stuff
+	 */
+	rldicl.	r0, r12, (64-MSR_TS_LG), (64-2)
+	bne-	2f
+  END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
+#endif
 	bl	.load_up_altivec
 	b	fast_exception_return
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	DISABLE_INTS
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.altivec_unavailable_tm
+	b	.ret_from_except
+#endif
 1:
 END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC)
 #endif
@@ -1175,7 +1209,24 @@ vsx_unavailable_common:
 #ifdef CONFIG_VSX
 BEGIN_FTR_SECTION
 	beq	1f
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+  BEGIN_FTR_SECTION_NESTED(69)
+	/* Test if 2 TM state bits are zero.  If non-zero (ie. userspace was in
+	 * transaction), go do TM stuff
+	 */
+	rldicl.	r0, r12, (64-MSR_TS_LG), (64-2)
+	bne-	2f
+  END_FTR_SECTION_NESTED(CPU_FTR_TM, CPU_FTR_TM, 69)
+#endif
 	b	.load_up_vsx
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+2:	/* User process was in a transaction */
+	bl	.save_nvgprs
+	DISABLE_INTS
+	addi	r3,r1,STACK_FRAME_OVERHEAD
+	bl	.vsx_unavailable_tm
+	b	.ret_from_except
+#endif
 1:
 END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 #endif
@@ -1190,6 +1241,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
 tm_unavailable_common:
 	EXCEPTION_PROLOG_COMMON(0xf60, PACA_EXGEN)
 	bl	.save_nvgprs
+	DISABLE_INTS
 	addi	r3,r1,STACK_FRAME_OVERHEAD
 	bl	.tm_unavailable_exception
 	b	.ret_from_except
arch/powerpc/kernel/process.c +13 -2
index 48a9875..59dd545 100644
@@ -515,7 +515,7 @@ out_and_saveregs:
 	tm_save_sprs(thr);
 }
 
-static inline void __maybe_unused tm_recheckpoint_new_task(struct task_struct *new)
+static inline void tm_recheckpoint_new_task(struct task_struct *new)
 {
 	unsigned long msr;
 
@@ -590,6 +590,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	struct ppc64_tlb_batch *batch;
 #endif
 
+	__switch_to_tm(prev);
+
 #ifdef CONFIG_SMP
 	/* avoid complexity of lazy save/restore of fpu
 	 * by just saving it every time we switch out if
@@ -705,6 +707,9 @@ struct task_struct *__switch_to(struct task_struct *prev,
 	 * of sync. Hard disable here.
 	 */
 	hard_irq_disable();
+
+	tm_recheckpoint_new_task(new);
+
 	last = _switch(old_thread, new_thread);
 
 #ifdef CONFIG_PPC_BOOK3S_64
@@ -1080,7 +1085,6 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 		regs->msr = MSR_USER32;
 	}
 #endif
-
 	discard_lazy_cpu_state();
 #ifdef CONFIG_VSX
 	current->thread.used_vsr = 0;
@@ -1100,6 +1104,13 @@ void start_thread(struct pt_regs *regs, unsigned long start, unsigned long sp)
 	current->thread.spefscr = 0;
 	current->thread.used_spe = 0;
 #endif /* CONFIG_SPE */
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	if (cpu_has_feature(CPU_FTR_TM))
+		regs->msr |= MSR_TM;
+	current->thread.tm_tfhar = 0;
+	current->thread.tm_texasr = 0;
+	current->thread.tm_tfiar = 0;
+#endif /* CONFIG_PPC_TRANSACTIONAL_MEM */
 }
 
 #define PR_FP_ALL_EXCEPT (PR_FP_EXC_DIV | PR_FP_EXC_OVF | PR_FP_EXC_UND \
arch/powerpc/kernel/traps.c +32 -0
index 5c30469..f9b751b 100644
@@ -1029,6 +1029,38 @@ void __kprobes program_check_exception(struct pt_regs *regs)
 		_exception(SIGTRAP, regs, TRAP_BRKPT, regs->nip);
 		return;
 	}
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	if (reason & REASON_TM) {
+		/* This is a TM "Bad Thing Exception" program check.
+		 * This occurs when:
+		 * -  An rfid/hrfid/mtmsrd attempts to cause an illegal
+		 *    transition in TM states.
+		 * -  A trechkpt is attempted when transactional.
+		 * -  A treclaim is attempted when non transactional.
+		 * -  A tend is illegally attempted.
+		 * -  writing a TM SPR when transactional.
+		 */
+		if (!user_mode(regs) &&
+		    report_bug(regs->nip, regs) == BUG_TRAP_TYPE_WARN) {
+			regs->nip += 4;
+			return;
+		}
+		/* If usermode caused this, it's done something illegal and
+		 * gets a SIGILL slap on the wrist.  We call it an illegal
+		 * operand to distinguish from the instruction just being bad
+		 * (e.g. executing a 'tend' on a CPU without TM!); it's an
+		 * illegal /placement/ of a valid instruction.
+		 */
+		if (user_mode(regs)) {
+			_exception(SIGILL, regs, ILL_ILLOPN, regs->nip);
+			return;
+		} else {
+			printk(KERN_EMERG "Unexpected TM Bad Thing exception "
+			       "at %lx (msr 0x%x)\n", regs->nip, reason);
+			die("Unrecoverable exception", regs, SIGABRT);
+		}
+	}
+#endif
 
 	/* We restore the interrupt state now */
 	if (!arch_irq_disabled_regs(regs))
arch/powerpc/mm/hash_utils_64.c +16 -0
index 3a292be..1b6e127 100644
@@ -55,6 +55,7 @@
 #include <asm/code-patching.h>
 #include <asm/fadump.h>
 #include <asm/firmware.h>
+#include <asm/tm.h>
 
 #ifdef DEBUG
 #define DBG(fmt...) udbg_printf(fmt)
@@ -1171,6 +1172,21 @@ void flush_hash_page(unsigned long vpn, real_pte_t pte, int psize, int ssize,
 		DBG_LOW(" sub %ld: hash=%lx, hidx=%lx\n", index, slot, hidx);
 		ppc_md.hpte_invalidate(slot, vpn, psize, ssize, local);
 	} pte_iterate_hashed_end();
+
+#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
+	/* Transactions are not aborted by tlbiel, only tlbie.
+	 * Without, syncing a page back to a block device w/ PIO could pick up
+	 * transactional data (bad!) so we force an abort here.  Before the
+	 * sync the page will be made read-only, which will flush_hash_page.
+	 * BIG ISSUE here: if the kernel uses a page from userspace without
+	 * unmapping it first, it may see the speculated version.
+	 */
+	if (local && cpu_has_feature(CPU_FTR_TM) &&
+	    MSR_TM_ACTIVE(current->thread.regs->msr)) {
+		tm_enable();
+		tm_abort(TM_CAUSE_TLBI);
+	}
+#endif
 }
 
 void flush_hash_range(unsigned long number, int local)
Signature · 1 line
1.7.10.4
#72 lore raw

PATCH 13/17 powerpc: Add new transactional memory state to the signal context

In reply to Michael Neuling

This adds the new transactional memory archtected state to the signal context in both 32 and 64 bit.

---
 arch/powerpc/include/asm/reg.h  |    1 +
 arch/powerpc/kernel/signal.h    |    8 +
 arch/powerpc/kernel/signal_32.c |  500 ++++++++++++++++++++++++++++++++++++++-
 arch/powerpc/kernel/signal_64.c |  337 +++++++++++++++++++++++++-
 4 files changed, 830 insertions(+), 16 deletions(-)
arch/powerpc/include/asm/reg.h +1 -0
index eee2a60..7035e60 100644
@@ -120,6 +120,7 @@
 #define TM_CAUSE_FAC_UNAV	0xfa
 #define TM_CAUSE_SYSCALL	0xf9 /* Persistent */
 #define TM_CAUSE_MISC		0xf6
+#define TM_CAUSE_SIGNAL		0xf4
 
 #if defined(CONFIG_PPC_BOOK3S_64)
 #define MSR_64BIT	MSR_SF
arch/powerpc/kernel/signal.h +8 -0
index e00acb4..ec84c90 100644
@@ -25,13 +25,21 @@ extern int handle_rt_signal32(unsigned long sig, struct k_sigaction *ka,
 
 extern unsigned long copy_fpr_to_user(void __user *to,
 				      struct task_struct *task);
+extern unsigned long copy_transact_fpr_to_user(void __user *to,
+					       struct task_struct *task);
 extern unsigned long copy_fpr_from_user(struct task_struct *task,
 					void __user *from);
+extern unsigned long copy_transact_fpr_from_user(struct task_struct *task,
+						 void __user *from);
 #ifdef CONFIG_VSX
 extern unsigned long copy_vsx_to_user(void __user *to,
 				      struct task_struct *task);
+extern unsigned long copy_transact_vsx_to_user(void __user *to,
+					       struct task_struct *task);
 extern unsigned long copy_vsx_from_user(struct task_struct *task,
 					void __user *from);
+extern unsigned long copy_transact_vsx_from_user(struct task_struct *task,
+						 void __user *from);
 #endif
 
 #ifdef CONFIG_PPC64
arch/powerpc/kernel/signal_32.c +487 -13

608 lines not on this page · raw message

arch/powerpc/kernel/signal_64.c +334 -3

389 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#73 lore raw

PATCH 14/17 powerpc: Add transactional memory to POWER8 cpu features

In reply to Michael Neuling
---
 arch/powerpc/include/asm/cputable.h |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/powerpc/include/asm/cputable.h +2 -1
index 3636081..fb3245e 100644
@@ -421,7 +421,8 @@ extern const char *powerpc_base_platform;
 	    CPU_FTR_DSCR | CPU_FTR_SAO  | \
 	    CPU_FTR_STCX_CHECKS_ADDRESS | CPU_FTR_POPCNTB | CPU_FTR_POPCNTD | \
 	    CPU_FTR_ICSWX | CPU_FTR_CFAR | CPU_FTR_HVMODE | CPU_FTR_VMX_COPY | \
-	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_DAWR | CPU_FTR_BCTAR)
+	    CPU_FTR_DBELL | CPU_FTR_HAS_PPR | CPU_FTR_DAWR | CPU_FTR_BCTAR | \
+	    CPU_FTR_TM_COMP)
 #define CPU_FTRS_CELL	(CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \
 	    CPU_FTR_PPCAS_ARCH_V2 | CPU_FTR_CTRL | \
 	    CPU_FTR_ALTIVEC_COMP | CPU_FTR_MMCRA | CPU_FTR_SMT | \
Signature · 1 line
1.7.10.4
#74 lore raw

PATCH 15/17 powerpc: Add config option for transactional memory

In reply to Michael Neuling

Kconfig option for transactional memory on powerpc.

---
 arch/powerpc/Kconfig |    8 ++++++++
 1 file changed, 8 insertions(+)
arch/powerpc/Kconfig +8 -0
index 4b27edb..85ff3a0 100644
@@ -313,6 +313,14 @@ config MATH_EMULATION
 	  unit, which will allow programs that use floating-point
 	  instructions to run.
 
+config PPC_TRANSACTIONAL_MEM
+       bool "Transactional Memory support for POWERPC"
+       depends on PPC_BOOK3S_64
+       depends on SMP
+       default n
+       ---help---
+         Support user-mode Transactional Memory on POWERPC.
+
 config 8XX_MINIMAL_FPEMU
 	bool "Minimal math emulation for 8xx"
 	depends on 8xx && !MATH_EMULATION
Signature · 1 line
1.7.10.4
#75 lore raw

PATCH 16/17 powerpc: Add transactional memory to pseries and ppc64 defconfigs

In reply to Michael Neuling
---
 arch/powerpc/configs/ppc64_defconfig   |    2 ++
 arch/powerpc/configs/pseries_defconfig |    2 ++
 2 files changed, 4 insertions(+)
arch/powerpc/configs/ppc64_defconfig +2 -0
index 6cbdeb4..aef3f71 100644
@@ -49,6 +49,8 @@ CONFIG_CPU_FREQ_GOV_USERSPACE=y
 CONFIG_CPU_FREQ_PMAC64=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_PPC_TRANSACTIONAL_MEM=y
+CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
 CONFIG_MEMORY_HOTREMOVE=y
arch/powerpc/configs/pseries_defconfig +2 -0
index 04fa7f2..c4dfbaf 100644
@@ -43,6 +43,8 @@ CONFIG_RTAS_FLASH=m
 CONFIG_IBMEBUS=y
 CONFIG_HZ_100=y
 CONFIG_BINFMT_MISC=m
+CONFIG_PPC_TRANSACTIONAL_MEM=y
+CONFIG_HOTPLUG_CPU=y
 CONFIG_KEXEC=y
 CONFIG_IRQ_ALL_CPUS=y
 CONFIG_MEMORY_HOTPLUG=y
Signature · 1 line
1.7.10.4
#76 lore raw

PATCH 17/17 powerpc: Documentation for transactional memory on powerpc

In reply to Michael Neuling
---
 Documentation/powerpc/transactional_memory.txt |  175 ++++++++++++++++++++++++
 1 file changed, 175 insertions(+)
 create mode 100644 Documentation/powerpc/transactional_memory.txt
Documentation/powerpc/transactional_memory.txt new file +175 -0
new file mode 100644
index 0000000..c907be4
@@ -0,0 +1,175 @@
+Transactional Memory support
+============================
+
+POWER kernel support for this feature is currently limited to supporting
+its use by user programs.  It is not currently used by the kernel itself.
+
+This file aims to sum up how it is supported by Linux and what behaviour you
+can expect from your user programs.
+
+
+Basic overview
+==============
+
+Hardware Transactional Memory is supported on POWER8 processors, and is a
+feature that enables a different form of atomic memory access.  Several new
+instructions are presented to delimit transactions; transactions are
+guaranteed to either complete atomically or roll back and undo any partial
+changes.
+
+A simple transaction looks like this:
+
+begin_move_money:
+  tbegin
+  beq   abort_handler
+
+  ld    r4, SAVINGS_ACCT(r3)
+  ld    r5, CURRENT_ACCT(r3)
+  subi  r5, r5, 1
+  addi  r4, r4, 1
+  std   r4, SAVINGS_ACCT(r3)
+  std   r5, CURRENT_ACCT(r3)
+
+  tend
+
+  b     continue
+
+abort_handler:
+  ... test for odd failures ...
+
+  /* Retry the transaction if it failed because it conflicted with
+   * someone else: */
+  b     begin_move_money
+
+
+The 'tbegin' instruction denotes the start point, and 'tend' the end point.
+Between these points the processor is in 'Transactional' state; any memory
+references will complete in one go if there are no conflicts with other
+transactional or non-transactional accesses within the system.  In this
+example, the transaction completes as though it were normal straight-line code
+IF no other processor has touched SAVINGS_ACCT(r3) or CURRENT_ACCT(r3); an
+atomic move of money from the current account to the savings account has been
+performed.  Even though the normal ld/std instructions are used (note no
+lwarx/stwcx), either *both* SAVINGS_ACCT(r3) and CURRENT_ACCT(r3) will be
+updated, or neither will be updated.
+
+If, in the meantime, there is a conflict with the locations accessed by the
+transaction, the transaction will be aborted by the CPU.  Register and memory
+state will roll back to that at the 'tbegin', and control will continue from
+'tbegin+4'.  The branch to abort_handler will be taken this second time; the
+abort handler can check the cause of the failure, and retry.
+
+Checkpointed registers include all GPRs, FPRs, VRs/VSRs, LR, CCR/CR, CTR, FPCSR
+and a few other status/flag regs; see the ISA for details.
+
+Causes of transaction aborts
+============================
+
+- Conflicts with cache lines used by other processors
+- Signals
+- Context switches
+- See the ISA for full documentation of everything that will abort transactions.
+
+
+Syscalls
+========
+
+Performing syscalls from within transaction is not recommended, and can lead
+to unpredictable results.
+
+Syscalls do not by design abort transactions, but beware: The kernel code will
+not be running in transactional state.  The effect of syscalls will always
+remain visible, but depending on the call they may abort your transaction as a
+side-effect, read soon-to-be-aborted transactional data that should not remain
+invisible, etc.  If you constantly retry a transaction that constantly aborts
+itself by calling a syscall, you'll have a livelock & make no progress.
+
+Simple syscalls (e.g. sigprocmask()) "could" be OK.  Even things like write()
+from, say, printf() should be OK as long as the kernel does not access any
+memory that was accessed transactionally.
+
+Consider any syscalls that happen to work as debug-only -- not recommended for
+production use.  Best to queue them up till after the transaction is over.
+
+
+Signals
+=======
+
+Delivery of signals (both sync and async) during transactions provides a second
+thread state (ucontext/mcontext) to represent the second transactional register
+state.  Signal delivery 'treclaim's to capture both register states, so signals
+abort transactions.  The usual ucontext_t passed to the signal handler
+represents the checkpointed/original register state; the signal appears to have
+arisen at 'tbegin+4'.
+
+If the sighandler ucontext has uc_link set, a second ucontext has been
+delivered.  For future compatibility the MSR.TS field should be checked to
+determine the transactional state -- if so, the second ucontext in uc->uc_link
+represents the active transactional registers at the point of the signal.
+
+For 64-bit processes, uc->uc_mcontext.regs->msr is a full 64-bit MSR and its TS
+field shows the transactional mode.
+
+For 32-bit processes, the mcontext's MSR register is only 32 bits; the top 32
+bits are stored in the MSR of the second ucontext, i.e. in
+uc->uc_link->uc_mcontext.regs->msr.  The top word contains the transactional
+state TS.
+
+However, basic signal handlers don't need to be aware of transactions
+and simply returning from the handler will deal with things correctly:
+
+Transaction-aware signal handlers can read the transactional register state
+from the second ucontext.  This will be necessary for crash handlers to
+determine, for example, the address of the instruction causing the SIGSEGV.
+
+Example signal handler:
+
+    void crash_handler(int sig, siginfo_t *si, void *uc)
+    {
+      ucontext_t *ucp = uc;
+      ucontext_t *transactional_ucp = ucp->uc_link;
+
+      if (ucp_link) {
+        u64 msr = ucp->uc_mcontext.regs->msr;
+        /* May have transactional ucontext! */
+#ifndef __powerpc64__
+        msr |= ((u64)transactional_ucp->uc_mcontext.regs->msr) << 32;
+#endif
+        if (MSR_TM_ACTIVE(msr)) {
+           /* Yes, we crashed during a transaction.  Oops. */
+   fprintf(stderr, "Transaction to be restarted at 0x%llx, but "
+                           "crashy instruction was at 0x%llx\n",
+                           ucp->uc_mcontext.regs->nip,
+                           transactional_ucp->uc_mcontext.regs->nip);
+        }
+      }
+
+      fix_the_problem(ucp->dar);
+    }
+
+
+Failure cause codes used by kernel
+==================================
+
+These are defined in <asm/reg.h>, and distinguish different reasons why the
+kernel aborted a transaction:
+
+ TM_CAUSE_RESCHED       Thread was rescheduled.
+ TM_CAUSE_FAC_UNAV      FP/VEC/VSX unavailable trap.
+ TM_CAUSE_SYSCALL       Currently unused; future syscalls that must abort
+                        transactions for consistency will use this.
+ TM_CAUSE_SIGNAL        Signal delivered.
+ TM_CAUSE_MISC          Currently unused.
+
+These can be checked by the user program's abort handler as TEXASR[0:7].
+
+
+GDB
+===
+
+GDB and ptrace are not currently TM-aware.  If one stops during a transaction,
+it looks like the transaction has just started (the checkpointed state is
+presented).  The transaction cannot then be continued and will take the failure
+handler route.  Furthermore, the transactional 2nd register state will be
+inaccessible.  GDB can currently be used on programs using TM, but not sensibly
+in parts within transactions.
Signature · 1 line
1.7.10.4