Skip to contentSkip to conversation

habanalabs: add new IOCTL for debug, tracing and profiling

PATCH In review Oded Gabbay 2 participants 4 messages Apr 2, 2019 Char and misc drivers

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 habanalabs: add new IOCTL for debug, tracing and profiling

From: Omer Shpigelman <[email protected]>

Habanalabs ASICs use the ARM coresight infrastructure to support debug, tracing and profiling of neural networks topologies.

Because the coresight is configured using register writes and reads, and some of the registers hold sensitive information (e.g. the address in the device's DRAM where the trace data is written to), the user must go through the kernel driver to configure this mechanism.

This patch implements the common code of the IOCTL and calls the ASIC-specific function for the actual H/W configuration.

The IOCTL supports configuration of seven coresight components:
ETR, ETF, STM, FUNNEL, BMON, SPMU and TIMESTAMP

The user specifies which component he wishes to configure and provides a pointer to a structure (located in its process space) that contains the relevant configuration.

The common code copies the relevant data from the user-space to kernel space and then calls the ASIC-specific function to do the H/W configuration.

After the configuration is done, which is usually composed of several IOCTL calls depending on what the user wanted to trace, the user can start executing the topology. The trace data will be written to the user's area in the device's DRAM.

After the tracing operation is complete, and user will call the IOCTL again to disable the tracing operation. The user also need to read values from registers for some of the components (e.g. the size of the trace data in the device's DRAM). In that case, the user will provide a pointer to an "output" structure in user-space, which the IOCTL code will fill according the to selected component.

---
 drivers/misc/habanalabs/device.c              |   2 +-
 drivers/misc/habanalabs/goya/Makefile         |   3 +-
 drivers/misc/habanalabs/goya/goya.c           |   1 +
 drivers/misc/habanalabs/goya/goyaP.h          |   1 +
 drivers/misc/habanalabs/goya/goya_coresight.c |  13 ++
 drivers/misc/habanalabs/habanalabs.h          |  25 ++++
 drivers/misc/habanalabs/habanalabs_ioctl.c    | 123 +++++++++++++++++-
 include/uapi/misc/habanalabs.h                | 116 ++++++++++++++++-
 8 files changed, 280 insertions(+), 4 deletions(-)
 create mode 100644 drivers/misc/habanalabs/goya/goya_coresight.c
drivers/misc/habanalabs/device.c +1 -1
index e3797f582436..6cbfd560721e 100644
@@ -1044,7 +1044,7 @@ void hl_device_fini(struct hl_device *hdev)
 			WARN(1, "Failed to remove device because reset function did not finish\n");
 			return;
 		}
-	};
+	}
 
 	/* Mark device as disabled */
 	hdev->disabled = true;
drivers/misc/habanalabs/goya/Makefile +2 -1
index e458e5ba500b..131432f677e2 100644
@@ -1,3 +1,4 @@
 subdir-ccflags-y += -I$(src)
 
-HL_GOYA_FILES :=  goya/goya.o goya/goya_security.o goya/goya_hwmgr.o
+HL_GOYA_FILES :=  goya/goya.o goya/goya_security.o goya/goya_hwmgr.o \
+	goya/goya_coresight.o
drivers/misc/habanalabs/goya/goya.c +1 -0
index 5fdc8c3cf141..0c2a902facb6 100644
@@ -4814,6 +4814,7 @@ static const struct hl_asic_funcs goya_funcs = {
 	.send_heartbeat = goya_send_heartbeat,
 	.enable_clock_gating = goya_init_clock_gating,
 	.disable_clock_gating = goya_disable_clock_gating,
+	.debug_coresight = goya_debug_coresight,
 	.is_device_idle = goya_is_device_idle,
 	.soft_reset_late_init = goya_soft_reset_late_init,
 	.hw_queues_lock = goya_hw_queues_lock,
drivers/misc/habanalabs/goya/goyaP.h +1 -0
index b99d92f197eb..6f1f7715075d 100644
@@ -175,6 +175,7 @@ void goya_add_device_attr(struct hl_device *hdev,
 			struct attribute_group *dev_attr_grp);
 int goya_armcp_info_get(struct hl_device *hdev);
 void goya_init_security(struct hl_device *hdev);
+int goya_debug_coresight(struct hl_device *hdev, void *data);
 u64 goya_get_max_power(struct hl_device *hdev);
 void goya_set_max_power(struct hl_device *hdev, u64 value);
 
drivers/misc/habanalabs/goya/goya_coresight.c new file +13 -0
new file mode 100644
index 000000000000..8957b8b1c0d6
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Copyright 2016-2019 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ */
+
+#include "goyaP.h"
+
+int goya_debug_coresight(struct hl_device *hdev, void *data)
+{
+	return -ENOTTY;
+}
drivers/misc/habanalabs/habanalabs.h +25 -0
index fd7fcdf7050f..18a03657780a 100644
@@ -483,6 +483,7 @@ enum hl_pll_frequency {
  * @send_heartbeat: send is-alive packet to ArmCP and verify response.
  * @enable_clock_gating: enable clock gating for reducing power consumption.
  * @disable_clock_gating: disable clock for accessing registers on HBW.
+ * @debug_coresight: perform certain actions on Coresight for debugging.
  * @is_device_idle: return true if device is idle, false otherwise.
  * @soft_reset_late_init: perform certain actions needed after soft reset.
  * @hw_queues_lock: acquire H/W queues lock.
@@ -557,6 +558,7 @@ struct hl_asic_funcs {
 	int (*send_heartbeat)(struct hl_device *hdev);
 	void (*enable_clock_gating)(struct hl_device *hdev);
 	void (*disable_clock_gating)(struct hl_device *hdev);
+	int (*debug_coresight)(struct hl_device *hdev, void *data);
 	bool (*is_device_idle)(struct hl_device *hdev, char *buf, size_t size);
 	int (*soft_reset_late_init)(struct hl_device *hdev);
 	void (*hw_queues_lock)(struct hl_device *hdev);
@@ -867,6 +869,29 @@ struct hl_vm {
 	u8			init_done;
 };
 
+
+/*
+ * DEBUG, PROFILING STRUCTURE
+ */
+
+/**
+ * struct hl_debug_params - Coresight debug parameters.
+ * @input: pointer to component specific input parameters.
+ * @output: pointer to component specific output parameters.
+ * @output_size: size of output buffer.
+ * @reg_idx: relevant register ID.
+ * @op: component operation to execute.
+ * @enable: true if to enable component debugging, false otherwise.
+ */
+struct hl_debug_params {
+	void *input;
+	void *output;
+	u32 output_size;
+	u32 reg_idx;
+	u32 op;
+	bool enable;
+};
+
 /*
  * FILE PRIVATE STRUCTURE
  */
drivers/misc/habanalabs/habanalabs_ioctl.c +122 -1

147 lines not on this page · raw message

include/uapi/misc/habanalabs.h +115 -1

128 lines not on this page · raw message

Signature · 1 line
2.17.1
#1 lore raw

PATCH 2/2 habanalabs: add goya implementation for debug configuration

From: Omer Shpigelman <[email protected]>

This patch adds the ASIC-specific function for GOYA to configure the coresight components.

Most of the components have an enabled/disabled flag, depending on whether the user wants to enable the component or disable it.

For some of the components, such as ETR and SPMU, the user can also request to read values from them. Those values are needed for the user to parse the trace data.

The ETR configuration is also checked for security purposes, to make sure the trace data is written to the device's DRAM.

---
 drivers/misc/habanalabs/goya/goya.c           |   9 +-
 drivers/misc/habanalabs/goya/goyaP.h          |   2 +
 drivers/misc/habanalabs/goya/goya_coresight.c | 609 +++++++++++++++++-
 .../include/goya/asic_reg/goya_regs.h         |   3 +-
 .../include/goya/asic_reg/pcie_wrap_regs.h    | 306 +++++++++
 .../habanalabs/include/goya/goya_coresight.h  | 199 ++++++
 6 files changed, 1122 insertions(+), 6 deletions(-)
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_wrap_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_coresight.h
drivers/misc/habanalabs/goya/goya.c +5 -4
index 0c2a902facb6..cf4e66abf77e 100644
@@ -69,7 +69,7 @@
  *
  */
 
-#define GOYA_MMU_REGS_NUM		61
+#define GOYA_MMU_REGS_NUM		63
 
 #define GOYA_DMA_POOL_BLK_SIZE		0x100		/* 256 bytes */
 
@@ -85,8 +85,6 @@
 
 #define GOYA_QMAN0_FENCE_VAL		0xD169B243
 
-#define GOYA_MAX_INITIATORS		20
-
 #define GOYA_MAX_STRING_LEN		20
 
 #define GOYA_CB_POOL_CB_CNT		512
@@ -171,7 +169,9 @@ static u64 goya_mmu_regs[GOYA_MMU_REGS_NUM] = {
 	mmMME_SBA_CONTROL_DATA,
 	mmMME_SBB_CONTROL_DATA,
 	mmMME_SBC_CONTROL_DATA,
-	mmMME_WBC_CONTROL_DATA
+	mmMME_WBC_CONTROL_DATA,
+	mmPCIE_WRAP_PSOC_ARUSER,
+	mmPCIE_WRAP_PSOC_AWUSER
 };
 
 static u32 goya_all_events[] = {
@@ -1435,6 +1435,7 @@ static void goya_init_golden_registers(struct hl_device *hdev)
 	 * from Host using DMA CH 1
 	 */
 	WREG32(mmDMA_CH_1_CFG0, 0x0fff00F0);
+
 	WREG32(mmTPC_PLL_CLK_RLX_0, 0x200020);
 
 	goya->hw_cap_initialized |= HW_CAP_GOLDEN;
drivers/misc/habanalabs/goya/goyaP.h +2 -0
index 6f1f7715075d..2a6cdca744dd 100644
@@ -43,6 +43,8 @@
 
 #define QMAN_STOP_TIMEOUT_USEC		100000	/* 100 ms */
 
+#define CORESIGHT_TIMEOUT_USEC		100000	/* 100 ms */
+
 #define TPC_ENABLED_MASK		0xFF
 
 #define PLL_HIGH_DEFAULT		1575000000	/* 1.575 GHz */
drivers/misc/habanalabs/goya/goya_coresight.c +608 -1

616 lines not on this page · raw message

drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h +2 -1
index 6cb0b6e54d41..506e71e201e1 100644
@@ -1,6 +1,6 @@
 /* SPDX-License-Identifier: GPL-2.0
  *
- * Copyright 2016-2018 HabanaLabs, Ltd.
+ * Copyright 2016-2019 HabanaLabs, Ltd.
  * All Rights Reserved.
  *
  */
@@ -12,6 +12,7 @@
 #include "stlb_regs.h"
 #include "mmu_regs.h"
 #include "pcie_aux_regs.h"
+#include "pcie_wrap_regs.h"
 #include "psoc_global_conf_regs.h"
 #include "psoc_spi_regs.h"
 #include "psoc_mme_pll_regs.h"
drivers/misc/habanalabs/include/goya/asic_reg/pcie_wrap_regs.h new file +306 -0

306 lines not on this page · raw message

drivers/misc/habanalabs/include/goya/goya_coresight.h new file +199 -0

199 lines not on this page · raw message

Signature · 1 line
2.17.1
maintainer
#2 lore raw
In reply to Oded Gabbay
Quoted 55 lines · Oded Gabbay

From: Omer Shpigelman <[email protected]>

Habanalabs ASICs use the ARM coresight infrastructure to support debug, tracing and profiling of neural networks topologies.

Because the coresight is configured using register writes and reads, and some of the registers hold sensitive information (e.g. the address in the device's DRAM where the trace data is written to), the user must go through the kernel driver to configure this mechanism.

This patch implements the common code of the IOCTL and calls the ASIC-specific function for the actual H/W configuration.

The IOCTL supports configuration of seven coresight components:
ETR, ETF, STM, FUNNEL, BMON, SPMU and TIMESTAMP

The user specifies which component he wishes to configure and provides a pointer to a structure (located in its process space) that contains the relevant configuration.

The common code copies the relevant data from the user-space to kernel space and then calls the ASIC-specific function to do the H/W configuration.

After the configuration is done, which is usually composed of several IOCTL calls depending on what the user wanted to trace, the user can start executing the topology. The trace data will be written to the user's area in the device's DRAM.

After the tracing operation is complete, and user will call the IOCTL again to disable the tracing operation. The user also need to read values from registers for some of the components (e.g. the size of the trace data in the device's DRAM). In that case, the user will provide a pointer to an "output" structure in user-space, which the IOCTL code will fill according the to selected component.

---
 drivers/misc/habanalabs/device.c              |   2 +-
 drivers/misc/habanalabs/goya/Makefile         |   3 +-
 drivers/misc/habanalabs/goya/goya.c           |   1 +
 drivers/misc/habanalabs/goya/goyaP.h          |   1 +
 drivers/misc/habanalabs/goya/goya_coresight.c |  13 ++
 drivers/misc/habanalabs/habanalabs.h          |  25 ++++
 drivers/misc/habanalabs/habanalabs_ioctl.c    | 123 +++++++++++++++++-
 include/uapi/misc/habanalabs.h                | 116 ++++++++++++++++-
 8 files changed, 280 insertions(+), 4 deletions(-)
 create mode 100644 drivers/misc/habanalabs/goya/goya_coresight.c
drivers/misc/habanalabs/device.c +0 -0
index e3797f582436..6cbfd560721e 100644

drivers/misc/habanalabs/device.c void hl_device_fini(struct hl_device *hdev) · line 1050

Quoted 8 lines
drivers/misc/habanalabs/device.c +1 -1
@@ -1044,7 +1044,7 @@ void hl_device_fini(struct hl_device *hdev)
 			WARN(1, "Failed to remove device because reset function did not finish\n");
 			return;
 		}
-	};
+	}
 
 	/* Mark device as disabled */
 	hdev->disabled = true;

This change isn't part of this new feature :(

#3 lore raw
Quoted 63 lines · Greg KH
Quoted 61 lines · Oded Gabbay

From: Omer Shpigelman <[email protected]>

Habanalabs ASICs use the ARM coresight infrastructure to support debug, tracing and profiling of neural networks topologies.

Because the coresight is configured using register writes and reads, and some of the registers hold sensitive information (e.g. the address in the device's DRAM where the trace data is written to), the user must go through the kernel driver to configure this mechanism.

This patch implements the common code of the IOCTL and calls the ASIC-specific function for the actual H/W configuration.

The IOCTL supports configuration of seven coresight components:
ETR, ETF, STM, FUNNEL, BMON, SPMU and TIMESTAMP

The user specifies which component he wishes to configure and provides a pointer to a structure (located in its process space) that contains the relevant configuration.

The common code copies the relevant data from the user-space to kernel space and then calls the ASIC-specific function to do the H/W configuration.

After the configuration is done, which is usually composed of several IOCTL calls depending on what the user wanted to trace, the user can start executing the topology. The trace data will be written to the user's area in the device's DRAM.

After the tracing operation is complete, and user will call the IOCTL again to disable the tracing operation. The user also need to read values from registers for some of the components (e.g. the size of the trace data in the device's DRAM). In that case, the user will provide a pointer to an "output" structure in user-space, which the IOCTL code will fill according the to selected component.

---
 drivers/misc/habanalabs/device.c              |   2 +-
 drivers/misc/habanalabs/goya/Makefile         |   3 +-
 drivers/misc/habanalabs/goya/goya.c           |   1 +
 drivers/misc/habanalabs/goya/goyaP.h          |   1 +
 drivers/misc/habanalabs/goya/goya_coresight.c |  13 ++
 drivers/misc/habanalabs/habanalabs.h          |  25 ++++
 drivers/misc/habanalabs/habanalabs_ioctl.c    | 123 +++++++++++++++++-
 include/uapi/misc/habanalabs.h                | 116 ++++++++++++++++-
 8 files changed, 280 insertions(+), 4 deletions(-)
 create mode 100644 drivers/misc/habanalabs/goya/goya_coresight.c
drivers/misc/habanalabs/device.c +1 -1
index e3797f582436..6cbfd560721e 100644
@@ -1044,7 +1044,7 @@ void hl_device_fini(struct hl_device *hdev)
                      WARN(1, "Failed to remove device because reset function did not finish\n");
                      return;
              }
-     };
+     }
 
Quoted 5 lines

drivers/misc/habanalabs/device.c void hl_device_fini(struct hl_device *hdev) · line 1050

Quoted 2 lines
drivers/misc/habanalabs/device.c +0 -0
      /* Mark device as disabled */
      hdev->disabled = true;

This change isn't part of this new feature :(

I'll split it up
Oded