Add support OProfile for profiling Cell BE SPUs

RFC PATCH In review Maynard Johnson 1 participant 2 messages Dec 15, 2006 – Jan 2, 2007

Subsystems (3)

PowerPC Cell broadband engine architecture SPU file system

lore

Share & tools

lore

Maynard Johnson

author

2006-12-15 19:24 UTC #0 lore raw

RFC PATCH 1/1 Add support OProfile for profiling Cell BE SPUs

The attached patch extends OProfile's Cell support (committed into 2.6.20-rc1), adding the capability to do time-based profiling of the SPUs.

This is a preliminary patch we're posting for comments. Development is not 100% complete yet, but very close. This patch is dependent on two outstanding patches that have not yet been committed to mainline: 1) the cleanup patch for the initial OProfile Cell PPU support (posted on Nov 27); and 2) the spu notifier patch (posted on Dec 1), which exports the [un]register functions in SPUFS. It is also dependent on an internal patch, which will be submitted once the cleanup patch is committed. So please don't try to apply this patch to a source tree yet.

The code is functional and passing all test scenarios, except one: if an SPU task is already active before OProfile is started, the notification we receive for this already-active task happens in the wrong context, so we're unable to collect the information we need about the SPU binary being executed. The spu notifer patch mentioned above was meant to solve this problem, but in fact, it does not (which is why that patch hasn't been committed yet). We're currently investigating other options.

All comments are welcome.

NOTE: The availability of the developers of this patch is limited between now and Jan 2, 2007, so replies to comments may be delayed until then.

Thanks.
Maynard Johnson
IBM LTC Toolchain

Subject: Add support to OProfile for profiling Cell BE SPUs

From: Maynard Johnson <[email protected]>

This patch updates the existing arch/powerpc/oprofile/op_model_cell.c to add in the SPU profiling capabilities. In addition, a 'cell' subdirectory was added to arch/powerpc/oprofile to hold Cell-specific SPU profiling code.

Signed-off-by Carl Love <[email protected]>

Signed-off-by Maynard Johnson <[email protected]>

arch/powerpc/configs/cell_defconfig +1 -1

Index: linux-2.6.19-rc6-arnd1+patches/arch/powerpc/configs/cell_defconfig
===================================================================
@@ -1136,7 +1136,7 @@
 # Instrumentation Support
 #
 CONFIG_PROFILING=y
-CONFIG_OPROFILE=y
+CONFIG_OPROFILE=m
 # CONFIG_KPROBES is not set
 
 #

arch/powerpc/oprofile/cell/pr_util.h new file +74 -0

74 lines not on this page · raw message

arch/powerpc/oprofile/cell/spu_profiler.c new file +203 -0

203 lines not on this page · raw message

arch/powerpc/oprofile/cell/spu_task_sync.c new file +486 -0

486 lines not on this page · raw message

arch/powerpc/oprofile/cell/vma_map.c new file +233 -0

233 lines not on this page · raw message

arch/powerpc/oprofile/common.c +2 -0

Index: linux-2.6.19-rc6-arnd1+patches/arch/powerpc/oprofile/common.c
===================================================================
@@ -150,6 +150,8 @@
 #ifdef CONFIG_PPC_CELL
 		case PPC_OPROFILE_CELL:
 			model = &op_model_cell;
+			ops->sync_start = model->sync_start;
+			ops->sync_stop = model->sync_stop;
 			break;
 #endif
 		case PPC_OPROFILE_RS64:

arch/powerpc/oprofile/Kconfig +2 -1

Index: linux-2.6.19-rc6-arnd1+patches/arch/powerpc/oprofile/Kconfig
===================================================================
@@ -7,7 +7,8 @@
 
 config OPROFILE
 	tristate "OProfile system profiling (EXPERIMENTAL)"
-	depends on PROFILING
+	default m
+	depends on SPU_FS && PROFILING && CBE_CPUFREQ
 	help
 	  OProfile is a profiling system capable of profiling the
 	  whole system, include the kernel, kernel modules, libraries,

arch/powerpc/oprofile/Makefile +2 -1

Index: linux-2.6.19-rc6-arnd1+patches/arch/powerpc/oprofile/Makefile
===================================================================
@@ -11,7 +11,8 @@
 		timer_int.o )
 
 oprofile-y := $(DRIVER_OBJS) common.o backtrace.o
-oprofile-$(CONFIG_PPC_CELL) += op_model_cell.o
+oprofile-$(CONFIG_PPC_CELL) += op_model_cell.o \
+				cell/spu_profiler.o cell/vma_map.o cell/spu_task_sync.o
 oprofile-$(CONFIG_PPC64) += op_model_rs64.o op_model_power4.o
 oprofile-$(CONFIG_FSL_BOOKE) += op_model_fsl_booke.o
 oprofile-$(CONFIG_6xx) += op_model_7450.o

arch/powerpc/oprofile/op_model_cell.c +236 -7

330 lines not on this page · raw message

arch/powerpc/platforms/cell/spufs/sched.c +2 -0

Index: linux-2.6.19-rc6-arnd1+patches/arch/powerpc/platforms/cell/spufs/sched.c
===================================================================
@@ -125,6 +125,7 @@
 	ctx->spu = spu;
 	ctx->ops = &spu_hw_ops;
 	spu->pid = current->pid;
+	spu->tgid = current->tgid;
 	spu->prio = current->prio;
 	spu->mm = ctx->owner;
 	mm_needs_global_tlbie(spu->mm);

@@ -157,6 +158,7 @@
 	spu->dma_callback = NULL;
 	spu->mm = NULL;
 	spu->pid = 0;
+	spu->tgid = 0;
 	spu->prio = MAX_PRIO;
 	ctx->ops = &spu_backing_ops;
 	ctx->spu = NULL;

drivers/oprofile/buffer_sync.c +1 -0

Index: linux-2.6.19-rc6-arnd1+patches/drivers/oprofile/buffer_sync.c
===================================================================
@@ -26,6 +26,7 @@
 #include <linux/profile.h>
 #include <linux/module.h>
 #include <linux/fs.h>
+#include <linux/oprofile.h>
  
 #include "oprofile_stats.h"
 #include "event_buffer.h"

drivers/oprofile/event_buffer.h +1 ~~-19~~

Index: linux-2.6.19-rc6-arnd1+patches/drivers/oprofile/event_buffer.h
===================================================================
@@ -19,28 +19,10 @@
  
 /* wake up the process sleeping on the event file */
 void wake_up_buffer_waiter(void);
- 
-/* Each escaped entry is prefixed by ESCAPE_CODE
- * then one of the following codes, then the
- * relevant data.
- */
-#define ESCAPE_CODE			~0UL
-#define CTX_SWITCH_CODE 		1
-#define CPU_SWITCH_CODE 		2
-#define COOKIE_SWITCH_CODE 		3
-#define KERNEL_ENTER_SWITCH_CODE	4
-#define KERNEL_EXIT_SWITCH_CODE		5
-#define MODULE_LOADED_CODE		6
-#define CTX_TGID_CODE			7
-#define TRACE_BEGIN_CODE		8
-#define TRACE_END_CODE			9
- 
+  
 #define INVALID_COOKIE ~0UL
 #define NO_COOKIE 0UL
 
-/* add data to the event buffer */
-void add_event_entry(unsigned long data);
- 
 extern struct file_operations event_buffer_fops;
  
 /* mutex between sync_cpu_buffers() and the

drivers/oprofile/oprof.c +26 -0

Index: linux-2.6.19-rc6-arnd1+patches/drivers/oprofile/oprof.c
===================================================================
@@ -53,9 +53,23 @@
 	 * us missing task deaths and eventually oopsing
 	 * when trying to process the event buffer.
 	 */
+	if (oprofile_ops.sync_start) {
+		int sync_ret = oprofile_ops.sync_start();
+		switch (sync_ret) {
+			case 0: goto post_sync;
+				break;
+			case 1: goto do_generic;
+				break;
+			case -1: goto out3;
+				break;
+			default: goto out3;
+		}
+	}
+do_generic:
 	if ((err = sync_start()))
 		goto out3;
 
+post_sync:
 	is_setup = 1;
 	mutex_unlock(&start_mutex);
 	return 0;

@@ -118,7 +132,19 @@
 void oprofile_shutdown(void)
 {
 	mutex_lock(&start_mutex);
+        if (oprofile_ops.sync_stop) {
+                int sync_ret = oprofile_ops.sync_stop();
+                switch (sync_ret) {
+                        case 0: goto post_sync;
+                                break;
+                        case 1: goto do_generic;
+                                break;
+			default: goto post_sync;
+                }
+        }
+do_generic:
 	sync_stop();
+post_sync:
 	if (oprofile_ops.shutdown)
 		oprofile_ops.shutdown();
 	is_setup = 0;

include/asm-powerpc/oprofile_impl.h +2 -0

Index: linux-2.6.19-rc6-arnd1+patches/include/asm-powerpc/oprofile_impl.h
===================================================================
@@ -47,6 +47,8 @@
         void (*global_start) (struct op_counter_config *);
 	void (*stop) (void);
 	void (*global_stop) (void);
+	int (*sync_start)(void);
+	int (*sync_stop)(void);
 	void (*handle_interrupt) (struct pt_regs *,
 				  struct op_counter_config *);
 	int num_counters;

include/asm-powerpc/spu.h +1 -0

Index: linux-2.6.19-rc6-arnd1+patches/include/asm-powerpc/spu.h
===================================================================
@@ -130,6 +130,7 @@
 	struct spu_runqueue *rq;
 	unsigned long long timestamp;
 	pid_t pid;
+	pid_t tgid;
 	int prio;
 	int class_0_pending;
 	spinlock_t register_lock;

include/linux/oprofile.h +37 -0

Index: linux-2.6.19-rc6-arnd1+patches/include/linux/oprofile.h
===================================================================
@@ -17,6 +17,28 @@
 #include <linux/spinlock.h>
 #include <asm/atomic.h>
  
+/* Each escaped entry is prefixed by ESCAPE_CODE
+ * then one of the following codes, then the
+ * relevant data.
+ * These #defines live in this file so that arch-specific
+ * buffer sync'ing code can access them.  
+ */
+#define ESCAPE_CODE                     ~0UL
+#define CTX_SWITCH_CODE                 1
+#define CPU_SWITCH_CODE                 2
+#define COOKIE_SWITCH_CODE              3
+#define KERNEL_ENTER_SWITCH_CODE        4
+#define KERNEL_EXIT_SWITCH_CODE         5
+#define MODULE_LOADED_CODE              6
+#define CTX_TGID_CODE                   7
+#define TRACE_BEGIN_CODE                8
+#define TRACE_END_CODE                  9
+#define XEN_ENTER_SWITCH_CODE          10
+#define SPU_PROFILING_CODE             11
+#define SPU_CTX_SWITCH_CODE            12
+#define SPU_OFFSET_CODE                13
+#define SPU_COOKIE_CODE                14
+
 struct super_block;
 struct dentry;
 struct file_operations;

@@ -35,6 +57,14 @@
 	int (*start)(void);
 	/* Stop delivering interrupts. */
 	void (*stop)(void);
+	/* Arch-specific buffer sync functions.
+	 * Return value = 0:  Success
+	 * Return value = -1: Failure
+	 * Return value = 1:  Run generic sync function
+	 */
+	int (*sync_start)(void);
+	int (*sync_stop)(void);
+
 	/* Initiate a stack backtrace. Optional. */
 	void (*backtrace)(struct pt_regs * const regs, unsigned int depth);
 	/* CPU identification string. */

@@ -56,6 +86,13 @@
 void oprofile_arch_exit(void);
 
 /**
+ * Add data to the event buffer.
+ * The data passed is free-form, but typically consists of
+ * file offsets, dcookies, context information, and ESCAPE codes.
+ */
+void add_event_entry(unsigned long data);
+ 
+/**
  * Add a sample. This may be called from any context. Pass
  * smp_processor_id() as cpu.
  */

Maynard Johnson

author

2007-01-02 18:26 UTC #1 lore raw

Re: [PATCH -- RFC] Add support to OProfile for profiling Cell BE SPUs

Any comments on the attached patch would be appreciated. Thank you.

-Maynard

-------------------------------

Quoted 32 lines · Maynard Johnson

The attached patch extends OProfile's Cell support (committed into 2.6.20-rc1), adding the capability to do time-based profiling of the SPUs.
This is a preliminary patch we're posting for comments. Development is not 100% complete yet, but very close. This patch is dependent on two outstanding patches that have not yet been committed to mainline: 1) the cleanup patch for the initial OProfile Cell PPU support (posted on Nov 27); and 2) the spu notifier patch (posted on Dec 1), which exports the [un]register functions in SPUFS. It is also dependent on an internal patch, which will be submitted once the cleanup patch is committed. So please don't try to apply this patch to a source tree yet.
The code is functional and passing all test scenarios, except one: if an SPU task is already active before OProfile is started, the notification we receive for this already-active task happens in the wrong context, so we're unable to collect the information we need about the SPU binary being executed. The spu notifer patch mentioned above was meant to solve this problem, but in fact, it does not (which is why that patch hasn't been committed yet). We're currently investigating other options.
All comments are welcome.
NOTE: The availability of the developers of this patch is limited between now and Jan 2, 2007, so replies to comments may be delayed until then.
Thanks.
Maynard Johnson
IBM LTC Toolchain

Quoted 4 lines

_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-dev