Skip to contentSkip to conversation

Habana Labs kernel driver

PATCH Superseded Oded Gabbay 4 participants 29 messages Feb 4, 2019 – Feb 9, 2019

Subsystems (2)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover Habana Labs kernel driver

Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.

Link to v2 cover letter: https://lkml.org/lkml/2019/1/30/1003

Link to v1 cover letter: https://lwn.net/Articles/777342/

I would appricate any feedback, question and/or review.

Thanks,
Oded

p.s. for those who prefer to clone the tree instead of looking at the emails, you can grab a copy from our company's page in GitHub:

https://github.com/HabanaAI/linux/releases/tag/hl_patchset_v3_20190204

Oded Gabbay (14):
habanalabs: add skeleton driver
habanalabs: add Goya registers header files
habanalabs: add basic Goya support
habanalabs: add context and ASID modules
habanalabs: add command buffer module
habanalabs: add basic Goya h/w initialization
habanalabs: add h/w queues module
habanalabs: add event queue and interrupts
habanalabs: add sysfs and hwmon support
habanalabs: add device reset support
habanalabs: add command submission module
habanalabs: implement INFO IOCTL
habanalabs: add debugfs support
Update MAINTAINERS and CREDITS with habanalabs info

Omer Shpigelman (1):
habanalabs: add virtual memory and MMU modules

 CREDITS                                       |    2 +-
 .../ABI/testing/debugfs-driver-habanalabs     |  127 +
 .../ABI/testing/sysfs-driver-habanalabs       |  190 +
 MAINTAINERS                                   |    9 +
 drivers/misc/Kconfig                          |    1 +
 drivers/misc/Makefile                         |    1 +
 drivers/misc/habanalabs/Kconfig               |   22 +
 drivers/misc/habanalabs/Makefile              |   15 +
 drivers/misc/habanalabs/asid.c                |   58 +
 drivers/misc/habanalabs/command_buffer.c      |  432 ++
 drivers/misc/habanalabs/command_submission.c  |  782 +++
 drivers/misc/habanalabs/context.c             |  216 +
 drivers/misc/habanalabs/debugfs.c             | 1071 ++++
 drivers/misc/habanalabs/device.c              | 1111 ++++
 drivers/misc/habanalabs/goya/Makefile         |    4 +
 drivers/misc/habanalabs/goya/goya.c           | 5344 +++++++++++++++++
 drivers/misc/habanalabs/goya/goyaP.h          |  193 +
 drivers/misc/habanalabs/goya/goya_hwmgr.c     |  306 +
 drivers/misc/habanalabs/goya/goya_security.c  | 2999 +++++++++
 drivers/misc/habanalabs/habanalabs.h          | 1441 +++++
 drivers/misc/habanalabs/habanalabs_drv.c      |  465 ++
 drivers/misc/habanalabs/habanalabs_ioctl.c    |  234 +
 drivers/misc/habanalabs/hw_queue.c            |  636 ++
 drivers/misc/habanalabs/hwmon.c               |  449 ++
 drivers/misc/habanalabs/include/armcp_if.h    |  335 ++
 .../include/goya/asic_reg/cpu_ca53_cfg_regs.h |  213 +
 .../include/goya/asic_reg/cpu_if_regs.h       |  110 +
 .../include/goya/asic_reg/cpu_pll_regs.h      |  186 +
 .../include/goya/asic_reg/dma_ch_0_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_1_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_2_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_3_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_4_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_macro_regs.h    |  242 +
 .../include/goya/asic_reg/dma_nrtr_regs.h     |  380 ++
 .../include/goya/asic_reg/dma_qm_0_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_1_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_2_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_3_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_4_regs.h     |  543 ++
 .../include/goya/asic_reg/goya_blocks.h       | 1372 +++++
 .../include/goya/asic_reg/goya_masks.h        |  262 +
 .../include/goya/asic_reg/goya_regs.h         |   99 +
 .../include/goya/asic_reg/ic_pll_regs.h       |  186 +
 .../include/goya/asic_reg/mc_pll_regs.h       |  186 +
 .../include/goya/asic_reg/mme1_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme2_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme3_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme4_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme5_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme6_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme_cmdq_regs.h     |  431 ++
 .../include/goya/asic_reg/mme_qm_regs.h       |  543 ++
 .../include/goya/asic_reg/mme_regs.h          | 2422 ++++++++
 .../include/goya/asic_reg/mmu_regs.h          |  158 +
 .../include/goya/asic_reg/pci_nrtr_regs.h     |  380 ++
 .../include/goya/asic_reg/pcie_aux_regs.h     |  476 ++
 .../goya/asic_reg/psoc_emmc_pll_regs.h        |  186 +
 .../goya/asic_reg/psoc_global_conf_regs.h     | 1119 ++++
 .../include/goya/asic_reg/psoc_mme_pll_regs.h |  186 +
 .../include/goya/asic_reg/psoc_pci_pll_regs.h |  186 +
 .../include/goya/asic_reg/psoc_spi_regs.h     |  427 ++
 .../goya/asic_reg/sram_y0_x0_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x1_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x2_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x3_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x4_rtr_regs.h       |  215 +
 .../include/goya/asic_reg/stlb_regs.h         |  133 +
 .../include/goya/asic_reg/tpc0_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc0_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc0_eml_cfg_regs.h |  580 ++
 .../include/goya/asic_reg/tpc0_nrtr_regs.h    |  380 ++
 .../include/goya/asic_reg/tpc0_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc1_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc1_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc1_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc1_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc2_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc2_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc2_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc2_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc3_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc3_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc3_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc3_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc4_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc4_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc4_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc4_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc5_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc5_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc5_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc5_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc6_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc6_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc6_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc6_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc7_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc7_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc7_nrtr_regs.h    |  380 ++
 .../include/goya/asic_reg/tpc7_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc_pll_regs.h      |  186 +
 drivers/misc/habanalabs/include/goya/goya.h   |   41 +
 .../include/goya/goya_async_events.h          |  186 +
 .../misc/habanalabs/include/goya/goya_fw_if.h |   28 +
 .../habanalabs/include/goya/goya_packets.h    |  129 +
 drivers/misc/habanalabs/include/hl_boot_if.h  |   30 +
 .../include/hw_ip/mmu/mmu_general.h           |   45 +
 .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h   |   15 +
 drivers/misc/habanalabs/include/qman_if.h     |   56 +
 drivers/misc/habanalabs/irq.c                 |  325 +
 drivers/misc/habanalabs/memory.c              | 1722 ++++++
 drivers/misc/habanalabs/mmu.c                 |  604 ++
 drivers/misc/habanalabs/sysfs.c               |  691 +++
 include/uapi/misc/habanalabs.h                |  444 ++
 115 files changed, 73533 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs
 create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/Kconfig
 create mode 100644 drivers/misc/habanalabs/Makefile
 create mode 100644 drivers/misc/habanalabs/asid.c
 create mode 100644 drivers/misc/habanalabs/command_buffer.c
 create mode 100644 drivers/misc/habanalabs/command_submission.c
 create mode 100644 drivers/misc/habanalabs/context.c
 create mode 100644 drivers/misc/habanalabs/debugfs.c
 create mode 100644 drivers/misc/habanalabs/device.c
 create mode 100644 drivers/misc/habanalabs/goya/Makefile
 create mode 100644 drivers/misc/habanalabs/goya/goya.c
 create mode 100644 drivers/misc/habanalabs/goya/goyaP.h
 create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c
 create mode 100644 drivers/misc/habanalabs/goya/goya_security.c
 create mode 100644 drivers/misc/habanalabs/habanalabs.h
 create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c
 create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c
 create mode 100644 drivers/misc/habanalabs/hw_queue.c
 create mode 100644 drivers/misc/habanalabs/hwmon.c
 create mode 100644 drivers/misc/habanalabs/include/armcp_if.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h
 create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h
 create mode 100644 drivers/misc/habanalabs/include/qman_if.h
 create mode 100644 drivers/misc/habanalabs/irq.c
 create mode 100644 drivers/misc/habanalabs/memory.c
 create mode 100644 drivers/misc/habanalabs/mmu.c
 create mode 100644 drivers/misc/habanalabs/sysfs.c
 create mode 100644 include/uapi/misc/habanalabs.h
Signature · 1 line
2.17.1
#1 lore raw

PATCH v3 1/15 habanalabs: add skeleton driver

This patch adds the habanalabs skeleton driver. The driver does nothing at this stage except very basic operations. It contains the minimal code to insmod and rmmod the driver and to create a /dev/hlX file per PCI device.

---
 drivers/misc/Kconfig                     |   1 +
 drivers/misc/Makefile                    |   1 +
 drivers/misc/habanalabs/Kconfig          |  22 ++
 drivers/misc/habanalabs/Makefile         |   7 +
 drivers/misc/habanalabs/device.c         | 336 +++++++++++++++++++++
 drivers/misc/habanalabs/habanalabs.h     | 138 +++++++++
 drivers/misc/habanalabs/habanalabs_drv.c | 362 +++++++++++++++++++++++
 7 files changed, 867 insertions(+)
 create mode 100644 drivers/misc/habanalabs/Kconfig
 create mode 100644 drivers/misc/habanalabs/Makefile
 create mode 100644 drivers/misc/habanalabs/device.c
 create mode 100644 drivers/misc/habanalabs/habanalabs.h
 create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c
drivers/misc/Kconfig +1 -0
index f417b06e11c5..fecab53c4f21 100644
@@ -535,4 +535,5 @@ source "drivers/misc/echo/Kconfig"
 source "drivers/misc/cxl/Kconfig"
 source "drivers/misc/ocxl/Kconfig"
 source "drivers/misc/cardreader/Kconfig"
+source "drivers/misc/habanalabs/Kconfig"
 endmenu
drivers/misc/Makefile +1 -0
index e39ccbbc1b3a..ae77dfd790a4 100644
@@ -59,3 +59,4 @@ obj-$(CONFIG_PCI_ENDPOINT_TEST)	+= pci_endpoint_test.o
 obj-$(CONFIG_OCXL)		+= ocxl/
 obj-y				+= cardreader/
 obj-$(CONFIG_PVPANIC)   	+= pvpanic.o
+obj-$(CONFIG_HABANA_AI)		+= habanalabs/
drivers/misc/habanalabs/Kconfig new file +22 -0
new file mode 100644
index 000000000000..b7f38a14caf5
@@ -0,0 +1,22 @@
+#
+# HabanaLabs AI accelerators driver
+#
+
+config HABANA_AI
+	tristate "HabanaAI accelerators (habanalabs)"
+	depends on PCI
+	select FRAME_VECTOR
+	help
+	  Enables PCIe card driver for Habana's AI Processors (AIP) that are
+	  designed to accelerate Deep Learning inference and training workloads.
+
+	  The driver manages the PCIe devices and provides IOCTL interface for
+	  the user to submit workloads to the devices.
+
+	  The user-space interface is described in
+	  include/uapi/misc/habanalabs.h
+
+	  If unsure, say N.
+
+	  To compile this driver as a module, choose M here: the
+	  module will be called habanalabs.
drivers/misc/habanalabs/Makefile new file +7 -0
new file mode 100644
index 000000000000..0910f7fa34ec
@@ -0,0 +1,7 @@
+#
+# Makefile for HabanaLabs AI accelerators driver
+#
+
+obj-m	:= habanalabs.o
+
+habanalabs-y := habanalabs_drv.o device.o
drivers/misc/habanalabs/device.c new file +336 -0

336 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs.h new file +138 -0

138 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c new file +362 -0

362 lines not on this page · raw message

Signature · 1 line
2.17.1
#3 lore raw

PATCH v3 4/15 habanalabs: add context and ASID modules

In reply to Oded Gabbay

This patch adds two modules - ASID and context.

Each user process that opens a device's file must have at least one context before it is able to "work" with the device. Each context has its own device address-space and contains information about its runtime state (its active command submissions).

To have address-space separation between contexts, each context is assigned a unique ASID, which stands for "address-space id". Goya supports up to 1024 ASIDs.

Currently, the driver doesn't support multiple contexts. Therefore, the user doesn't need to actively create a context. A "primary context" is created automatically when the user opens the device's file.

---
 drivers/misc/habanalabs/Makefile         |   2 +-
 drivers/misc/habanalabs/asid.c           |  58 +++++++++
 drivers/misc/habanalabs/context.c        | 155 +++++++++++++++++++++++
 drivers/misc/habanalabs/device.c         |  47 +++++++
 drivers/misc/habanalabs/habanalabs.h     |  71 +++++++++++
 drivers/misc/habanalabs/habanalabs_drv.c |  46 ++++++-
 6 files changed, 376 insertions(+), 3 deletions(-)
 create mode 100644 drivers/misc/habanalabs/asid.c
 create mode 100644 drivers/misc/habanalabs/context.c
drivers/misc/habanalabs/Makefile +1 -1
index 64bad3ca21c6..4944d6ccf8fe 100644
@@ -4,7 +4,7 @@
 
 obj-m	:= habanalabs.o
 
-habanalabs-y := habanalabs_drv.o device.o
+habanalabs-y := habanalabs_drv.o device.o context.o asid.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/asid.c new file +58 -0

58 lines not on this page · raw message

drivers/misc/habanalabs/context.c new file +155 -0

155 lines not on this page · raw message

drivers/misc/habanalabs/device.c +47 -0

92 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs.h +71 -0

131 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +44 -2

68 lines not on this page · raw message

Signature · 1 line
2.17.1
#2 lore raw

PATCH v3 3/15 habanalabs: add basic Goya support

In reply to Oded Gabbay

This patch adds a basic support for the Goya device. The code initializes the device's PCI controller and PCI bars. It also initializes various S/W structures and adds some basic helper functions.

---
 drivers/misc/habanalabs/Makefile            |   4 +
 drivers/misc/habanalabs/device.c            |  71 +++
 drivers/misc/habanalabs/goya/Makefile       |   3 +
 drivers/misc/habanalabs/goya/goya.c         | 639 ++++++++++++++++++++
 drivers/misc/habanalabs/goya/goyaP.h        | 155 +++++
 drivers/misc/habanalabs/habanalabs.h        | 137 +++++
 drivers/misc/habanalabs/habanalabs_drv.c    |   3 +
 drivers/misc/habanalabs/include/goya/goya.h |  41 ++
 include/uapi/misc/habanalabs.h              |  20 +
 9 files changed, 1073 insertions(+)
 create mode 100644 drivers/misc/habanalabs/goya/Makefile
 create mode 100644 drivers/misc/habanalabs/goya/goya.c
 create mode 100644 drivers/misc/habanalabs/goya/goyaP.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya.h
 create mode 100644 include/uapi/misc/habanalabs.h
drivers/misc/habanalabs/Makefile +4 -0
index 0910f7fa34ec..64bad3ca21c6 100644
@@ -5,3 +5,7 @@
 obj-m	:= habanalabs.o
 
 habanalabs-y := habanalabs_drv.o device.o
+
+include $(src)/goya/Makefile
+habanalabs-y += $(HL_GOYA_FILES)
+
drivers/misc/habanalabs/device.c +71 -0

125 lines not on this page · raw message

drivers/misc/habanalabs/goya/Makefile new file +3 -0
new file mode 100644
index 000000000000..38d43006386d
@@ -0,0 +1,3 @@
+subdir-ccflags-y += -I$(src)
+
+HL_GOYA_FILES :=  goya/goya.o
drivers/misc/habanalabs/goya/goya.c new file +639 -0

639 lines not on this page · raw message

drivers/misc/habanalabs/goya/goyaP.h new file +155 -0

155 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs.h +137 -0

182 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +3 -0
index 573349eedce4..1596cf38f6c0 100644
@@ -126,6 +126,9 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 
 	hdev->major = hl_major;
 
+	/* Parameters for bring-up - set them to defaults */
+	hdev->reset_pcilink = 0;
+
 	hdev->disabled = true;
 	hdev->pdev = pdev; /* can be NULL in case of simulator device */
 
drivers/misc/habanalabs/include/goya/goya.h new file +41 -0

41 lines not on this page · raw message

include/uapi/misc/habanalabs.h new file +20 -0
new file mode 100644
index 000000000000..a0ec23adf8f5
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note
+ *
+ * Copyright 2016-2018 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ *
+ */
+
+#ifndef HABANALABS_H_
+#define HABANALABS_H_
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+/*
+ * Defines that are asic-specific but constitutes as ABI between kernel driver
+ * and userspace
+ */
+#define GOYA_KMD_SRAM_RESERVED_SIZE_FROM_START	0x8000	/* 32KB */
+
+#endif /* HABANALABS_H_ */
Signature · 1 line
2.17.1
#10 lore raw

PATCH v3 8/15 habanalabs: add event queue and interrupts

In reply to Oded Gabbay

This patch adds support for receiving events from Goya's control CPU and for receiving MSI-X interrupts from Goya's DMA engines and CPU.

Goya's PCI controller supports up to 8 MSI-X interrupts, which only 6 of them are currently used. The first 5 interrupts are dedicated for Goya's DMA engine queues. The 6th interrupt is dedicated for Goya's control CPU.

The DMA queue will signal its MSI-X entry upon each completion of a command buffer that was placed on its primary queue. The driver will then mark that CB as completed and free the related resources. It will also update the command submission object which that CB belongs to.

There is a dedicated event queue (EQ) between the driver and Goya's control CPU. The EQ is located on the Host memory. The control CPU writes a new entry to the EQ for various reasons, such as ECC error, MMU page fault, Hot temperature. After writing the new entry to the EQ, the control CPU will trigger its dedicated MSI-X entry to signal the driver that there is a new entry in the EQ. The driver will then read the entry and act accordingly.

---
 drivers/misc/habanalabs/device.c           |  34 +-
 drivers/misc/habanalabs/goya/goya.c        | 459 ++++++++++++++++++++-
 drivers/misc/habanalabs/goya/goyaP.h       |   1 +
 drivers/misc/habanalabs/habanalabs.h       |  39 +-
 drivers/misc/habanalabs/include/armcp_if.h |  24 ++
 drivers/misc/habanalabs/irq.c              | 144 +++++++
 6 files changed, 689 insertions(+), 12 deletions(-)
drivers/misc/habanalabs/device.c +33 -1
index 3f807e1419a6..e4c939899042 100644
@@ -182,6 +182,13 @@ static int device_early_init(struct hl_device *hdev)
 		goto asid_fini;
 	}
 
+	hdev->eq_wq = alloc_workqueue("hl-events", WQ_UNBOUND, 0);
+	if (hdev->eq_wq == NULL) {
+		dev_err(hdev->dev, "Failed to allocate EQ workqueue\n");
+		rc = -ENOMEM;
+		goto free_cq_wq;
+	}
+
 	hl_cb_mgr_init(&hdev->kernel_cb_mgr);
 
 	mutex_init(&hdev->fd_open_cnt_lock);
@@ -190,6 +197,8 @@ static int device_early_init(struct hl_device *hdev)
 
 	return 0;
 
+free_cq_wq:
+	destroy_workqueue(hdev->cq_wq);
 asid_fini:
 	hl_asid_fini(hdev);
 early_fini:
@@ -211,6 +220,7 @@ static void device_early_fini(struct hl_device *hdev)
 
 	hl_cb_mgr_fini(hdev, &hdev->kernel_cb_mgr);
 
+	destroy_workqueue(hdev->eq_wq);
 	destroy_workqueue(hdev->cq_wq);
 
 	hl_asid_fini(hdev);
@@ -349,11 +359,22 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass)
 		}
 	}
 
+	/*
+	 * Initialize the event queue. Must be done before hw_init,
+	 * because there the address of the event queue is being
+	 * passed as argument to request_irq
+	 */
+	rc = hl_eq_init(hdev, &hdev->event_queue);
+	if (rc) {
+		dev_err(hdev->dev, "failed to initialize event queue\n");
+		goto cq_fini;
+	}
+
 	/* Allocate the kernel context */
 	hdev->kernel_ctx = kzalloc(sizeof(*hdev->kernel_ctx), GFP_KERNEL);
 	if (!hdev->kernel_ctx) {
 		rc = -ENOMEM;
-		goto cq_fini;
+		goto eq_fini;
 	}
 
 	hdev->user_ctx = NULL;
@@ -398,6 +419,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass)
 			"kernel ctx is still alive on initialization failure\n");
 free_ctx:
 	kfree(hdev->kernel_ctx);
+eq_fini:
+	hl_eq_fini(hdev, &hdev->event_queue);
 cq_fini:
 	for (i = 0 ; i < cq_ready_cnt ; i++)
 		hl_cq_fini(hdev, &hdev->completion_queue[i]);
@@ -439,6 +462,13 @@ void hl_device_fini(struct hl_device *hdev)
 	/* Mark device as disabled */
 	hdev->disabled = true;
 
+	/*
+	 * Halt the engines and disable interrupts so we won't get any more
+	 * completions from H/W and we won't have any accesses from the
+	 * H/W to the host machine
+	 */
+	hdev->asic_funcs->halt_engines(hdev, true);
+
 	hl_cb_pool_fini(hdev);
 
 	/* Release kernel context */
@@ -448,6 +478,8 @@ void hl_device_fini(struct hl_device *hdev)
 	/* Reset the H/W. It will be in idle state after this returns */
 	hdev->asic_funcs->hw_fini(hdev, true);
 
+	hl_eq_fini(hdev, &hdev->event_queue);
+
 	for (i = 0 ; i < hdev->asic_prop.completion_queues_count ; i++)
 		hl_cq_fini(hdev, &hdev->completion_queue[i]);
 	kfree(hdev->completion_queue);
drivers/misc/habanalabs/goya/goya.c +450 -9

550 lines not on this page · raw message

drivers/misc/habanalabs/goya/goyaP.h +1 -0
index 7badde68aef0..58f1e316f8d9 100644
@@ -155,6 +155,7 @@ struct goya_device {
 	/* TODO: remove hw_queues_lock after moving to scheduler code */
 	spinlock_t	hw_queues_lock;
 	u64		ddr_bar_cur_addr;
+	u32		events_stat[GOYA_ASYNC_EVENT_ID_SIZE];
 	u32		hw_cap_initialized;
 };
 
drivers/misc/habanalabs/habanalabs.h +37 -2
index 58319df24fc5..11f0c0492c79 100644
@@ -86,6 +86,7 @@ struct hw_queue_properties {
  * @cfg_size: configuration space size on SRAM.
  * @sram_size: total size of SRAM.
  * @max_asid: maximum number of open contexts (ASIDs).
+ * @num_of_events: number of possible internal H/W IRQs.
  * @completion_queues_count: number of completion queues.
  * @high_pll: high PLL frequency used by the device.
  * @cb_pool_cb_cnt: number of CBs in the CB pool.
@@ -112,6 +113,7 @@ struct asic_fixed_properties {
 	u32			cfg_size;
 	u32			sram_size;
 	u32			max_asid;
+	u32			num_of_events;
 	u32			high_pll;
 	u32			cb_pool_cb_cnt;
 	u32			cb_pool_cb_size;
@@ -204,6 +206,9 @@ struct hl_cs_job;
 #define HL_CQ_LENGTH			HL_QUEUE_LENGTH
 #define HL_CQ_SIZE_IN_BYTES		(HL_CQ_LENGTH * HL_CQ_ENTRY_SIZE)
 
+/* Must be power of 2 (HL_PAGE_SIZE / HL_EQ_ENTRY_SIZE) */
+#define HL_EQ_LENGTH			64
+#define HL_EQ_SIZE_IN_BYTES		(HL_EQ_LENGTH * HL_EQ_ENTRY_SIZE)
 
 
 /**
@@ -251,6 +256,20 @@ struct hl_cq {
 	atomic_t		free_slots_cnt;
 };
 
+/**
+ * struct hl_eq - describes the event queue (single one per device)
+ * @hdev: pointer to the device structure
+ * @kernel_address: holds the queue's kernel virtual address
+ * @bus_address: holds the queue's DMA address
+ * @ci: ci inside the queue
+ */
+struct hl_eq {
+	struct hl_device	*hdev;
+	u64			kernel_address;
+	dma_addr_t		bus_address;
+	u32			ci;
+};
+
 
 /*
  * ASICs
@@ -277,6 +296,9 @@ enum hl_asic_type {
  * @sw_fini: tears down driver state, does not configure H/W.
  * @hw_init: sets up the H/W state.
  * @hw_fini: tears down the H/W state.
+ * @halt_engines: halt engines, needed for reset sequence. This also disables
+ *                interrupts from the device. Should be called before
+ *                hw_fini and before CS rollback.
  * @suspend: handles IP specific H/W or SW changes for suspend.
  * @resume: handles IP specific H/W or SW changes for resume.
  * @mmap: mmap function, does nothing.
@@ -298,6 +320,9 @@ enum hl_asic_type {
  * @dma_pool_free: free small DMA allocation from pool.
  * @cpu_accessible_dma_pool_alloc: allocate CPU PQ packet from DMA pool.
  * @cpu_accessible_dma_pool_free: free CPU PQ packet from DMA pool.
+ * @update_eq_ci: update event queue CI.
+ * @handle_eqe: handle event queue entry (IRQ) from ArmCP.
+ * @get_events_stat: retrieve event queue entries histogram.
  * @hw_queues_lock: acquire H/W queues lock.
  * @hw_queues_unlock: release H/W queues lock.
  * @send_cpu_message: send buffer to ArmCP.
@@ -309,6 +334,7 @@ struct hl_asic_funcs {
 	int (*sw_fini)(struct hl_device *hdev);
 	int (*hw_init)(struct hl_device *hdev);
 	void (*hw_fini)(struct hl_device *hdev, bool hard_reset);
+	void (*halt_engines)(struct hl_device *hdev, bool hard_reset);
 	int (*suspend)(struct hl_device *hdev);
 	int (*resume)(struct hl_device *hdev);
 	int (*mmap)(struct hl_fpriv *hpriv, struct vm_area_struct *vma);
@@ -331,6 +357,10 @@ struct hl_asic_funcs {
 				size_t size, dma_addr_t *dma_handle);
 	void (*cpu_accessible_dma_pool_free)(struct hl_device *hdev,
 				size_t size, void *vaddr);
+	void (*update_eq_ci)(struct hl_device *hdev, u32 val);
+	void (*handle_eqe)(struct hl_device *hdev,
+				struct hl_eq_entry *eq_entry);
+	void* (*get_events_stat)(struct hl_device *hdev, u32 *size);
 	void (*hw_queues_lock)(struct hl_device *hdev);
 	void (*hw_queues_unlock)(struct hl_device *hdev);
 	int (*send_cpu_message)(struct hl_device *hdev, u32 *msg,
@@ -370,8 +400,6 @@ struct hl_ctx_mgr {
 };
 
 
-
-
 /**
  * struct hl_cs_job - command submission job.
  * @finish_work: workqueue object to run when job is completed.
@@ -461,6 +489,7 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);
  * @kernel_ctx: KMD context structure.
  * @kernel_queues: array of hl_hw_queue.
  * @kernel_cb_mgr: command buffer manager for creating/destroying/handling CGs.
+ * @event_queue: event queue for IRQ from ArmCP.
  * @dma_pool: DMA pool for small allocations.
  * @cpu_accessible_dma_mem: KMD <-> ArmCP shared memory CPU address.
  * @cpu_accessible_dma_address: KMD <-> ArmCP shared memory DMA address.
@@ -495,9 +524,11 @@ struct hl_device {
 	enum hl_asic_type		asic_type;
 	struct hl_cq			*completion_queue;
 	struct workqueue_struct		*cq_wq;
+	struct workqueue_struct		*eq_wq;
 	struct hl_ctx			*kernel_ctx;
 	struct hl_hw_queue		*kernel_queues;
 	struct hl_cb_mgr		kernel_cb_mgr;
+	struct hl_eq			event_queue;
 	struct dma_pool			*dma_pool;
 	void				*cpu_accessible_dma_mem;
 	dma_addr_t			cpu_accessible_dma_address;
@@ -579,6 +610,10 @@ void hl_hw_queue_inc_ci_kernel(struct hl_device *hdev, u32 hw_queue_id);
 
 int hl_cq_init(struct hl_device *hdev, struct hl_cq *q, u32 hw_queue_id);
 void hl_cq_fini(struct hl_device *hdev, struct hl_cq *q);
+int hl_eq_init(struct hl_device *hdev, struct hl_eq *q);
+void hl_eq_fini(struct hl_device *hdev, struct hl_eq *q);
+irqreturn_t hl_irq_handler_cq(int irq, void *arg);
+irqreturn_t hl_irq_handler_eq(int irq, void *arg);
 int hl_asid_init(struct hl_device *hdev);
 void hl_asid_fini(struct hl_device *hdev);
 unsigned long hl_asid_alloc(struct hl_device *hdev);
drivers/misc/habanalabs/include/armcp_if.h +24 -0
index cc37003aa6b7..9dddb917e72c 100644
@@ -10,6 +10,30 @@
 
 #include <linux/types.h>
 
+/*
+ * EVENT QUEUE
+ */
+
+struct hl_eq_header {
+	__le32 reserved;
+	__le32 ctl;
+};
+
+struct hl_eq_entry {
+	struct hl_eq_header hdr;
+	__le64 data[7];
+};
+
+#define HL_EQ_ENTRY_SIZE		sizeof(struct hl_eq_entry)
+
+#define EQ_CTL_READY_SHIFT		31
+#define EQ_CTL_READY_MASK		0x80000000
+
+#define EQ_CTL_EVENT_TYPE_SHIFT		16
+#define EQ_CTL_EVENT_TYPE_MASK		0x03FF0000
+
+#define EVENT_QUEUE_MSIX_IDX		5
+
 enum pq_init_status {
 	PQ_INIT_STATUS_NA = 0,
 	PQ_INIT_STATUS_READY_FOR_CP,
drivers/misc/habanalabs/irq.c +144 -0

165 lines not on this page · raw message

Signature · 1 line
2.17.1
#9 lore raw

PATCH v3 7/15 habanalabs: add h/w queues module

In reply to Oded Gabbay

This patch adds the H/W queues module and the code to initialize Goya's various compute and DMA engines and their queues.

Goya has 5 DMA channels, 8 TPC engines and a single MME engine. For each channel/engine, there is a H/W queue logic which is used to pass commands from the user to the H/W. That logic is called QMAN.

There are two types of QMANs: external and internal. The DMA QMANs are considered external while the TPC and MME QMANs are considered internal. For each external queue there is a completion queue, which is located on the Host memory.

The differences between external and internal QMANs are:

1. The location of the queue's memory. External QMANs are located on the Host memory while internal QMANs are located on the on-chip memory.

2. The external QMAN write an entry to a completion queue and sends an MSI-X interrupt upon completion of a command buffer that was given to it. The internal QMAN doesn't do that.

---
Changes in v3:
- Fix the return value in goya_stop_external_queues()
- Remove hdev->ifh field as it was used for debugging

 drivers/misc/habanalabs/Makefile              |    2 +-
 drivers/misc/habanalabs/device.c              |   75 +-
 drivers/misc/habanalabs/goya/goya.c           | 1525 +++++++++++++++--
 drivers/misc/habanalabs/goya/goyaP.h          |    7 +
 drivers/misc/habanalabs/habanalabs.h          |  174 +-
 drivers/misc/habanalabs/habanalabs_drv.c      |    5 +
 drivers/misc/habanalabs/hw_queue.c            |  404 +++++
 drivers/misc/habanalabs/include/armcp_if.h    |  292 ++++
 .../include/goya/goya_async_events.h          |  186 ++
 .../habanalabs/include/goya/goya_packets.h    |  129 ++
 drivers/misc/habanalabs/include/qman_if.h     |   56 +
 drivers/misc/habanalabs/irq.c                 |  150 ++
 include/uapi/misc/habanalabs.h                |   29 +
 13 files changed, 2918 insertions(+), 116 deletions(-)
 create mode 100644 drivers/misc/habanalabs/hw_queue.c
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h
 create mode 100644 drivers/misc/habanalabs/include/qman_if.h
 create mode 100644 drivers/misc/habanalabs/irq.c
drivers/misc/habanalabs/Makefile +1 -1
index 06c886289049..1e2a0c1b3890 100644
@@ -5,7 +5,7 @@
 obj-m	:= habanalabs.o
 
 habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
-		command_buffer.o
+		command_buffer.o hw_queue.o irq.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/device.c +73 -2

137 lines not on this page · raw message

drivers/misc/habanalabs/goya/goya.c +1413 -112

1,639 lines not on this page · raw message

drivers/misc/habanalabs/goya/goyaP.h +7 -0
index c3e17de41ab3..7badde68aef0 100644
@@ -11,7 +11,9 @@
 #include <uapi/misc/habanalabs.h>
 #include "habanalabs.h"
 #include "include/hl_boot_if.h"
+#include "include/goya/goya_packets.h"
 #include "include/goya/goya.h"
+#include "include/goya/goya_async_events.h"
 #include "include/goya/goya_fw_if.h"
 
 #define NUMBER_OF_CMPLT_QUEUES		5
@@ -148,12 +150,17 @@ enum goya_fw_component {
 };
 
 struct goya_device {
+	int (*test_cpu_queue)(struct hl_device *hdev);
+
 	/* TODO: remove hw_queues_lock after moving to scheduler code */
 	spinlock_t	hw_queues_lock;
 	u64		ddr_bar_cur_addr;
 	u32		hw_cap_initialized;
 };
 
+int goya_test_cpu_queue(struct hl_device *hdev);
+int goya_send_cpu_message(struct hl_device *hdev, u32 *msg, u16 len,
+				u32 timeout, long *result);
 void goya_init_security(struct hl_device *hdev);
 
 #endif /* GOYAP_H_ */
drivers/misc/habanalabs/habanalabs.h +173 -1

268 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +5 -0
index 2611883eab11..14775a7022f0 100644
@@ -173,6 +173,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 	/* Parameters for bring-up - set them to defaults */
 	hdev->cpu_enable = 1;
 	hdev->reset_pcilink = 0;
+	hdev->cpu_queues_enable = 1;
 	hdev->fw_loading = 1;
 	hdev->pldm = 0;
 
@@ -180,6 +181,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 	if (!hdev->cpu_enable)
 		hdev->fw_loading = 0;
 
+	/* If we don't load FW, no need to initialize CPU queues */
+	if (!hdev->fw_loading)
+		hdev->cpu_queues_enable = 0;
+
 	hdev->disabled = true;
 	hdev->pdev = pdev; /* can be NULL in case of simulator device */
 
drivers/misc/habanalabs/hw_queue.c new file +404 -0

404 lines not on this page · raw message

drivers/misc/habanalabs/include/armcp_if.h +292 -0

302 lines not on this page · raw message

drivers/misc/habanalabs/include/goya/goya_async_events.h new file +186 -0

186 lines not on this page · raw message

drivers/misc/habanalabs/include/goya/goya_packets.h new file +129 -0

13 lines not on this page · raw message

drivers/misc/habanalabs/include/qman_if.h new file +56 -0

56 lines not on this page · raw message

drivers/misc/habanalabs/irq.c new file +150 -0

150 lines not on this page · raw message

include/uapi/misc/habanalabs.h +29 -0
index a8edfd3e9c95..756266cf0416 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#14 lore raw

PATCH v3 10/15 habanalabs: add device reset support

In reply to Oded Gabbay

This patch adds support for doing various on-the-fly reset of Goya.

The driver supports two types of resets:
1. soft-reset
2. hard-reset

Soft-reset is done when the device detects a timeout of a command submission that was given to the device. The soft-reset process only resets the engines that are relevant for the submission of compute jobs, i.e. the DMA channels, the TPCs and the MME. The purpose is to bring the device as fast as possible to a working state.

Hard-reset is done in several cases:
1. After soft-reset is done but the device is not responding
2. When fatal errors occur inside the device, e.g. ECC error
3. When the driver is removed

Hard-reset performs a reset of the entire chip except for the PCI controller and the PLLs. It is a much longer process then soft-reset but it helps to recover the device without the need to reboot the Host.

After hard-reset, the driver will restore the max power attribute and in case of manual power management, the frequencies that were set.

This patch also adds two entries to the sysfs, which allows the root user to initiate a soft or hard reset.

---
Changes in v3:
- Remove use of three exclamation marks
- Add "init_done" field to hl_device to mark that initialization is done. This field is used in the reset function to avoid doing reset during initialization phase. This is instead of setting the in_reset atomic field to 1 during initialization, which is conceptually not accurate.

 drivers/misc/habanalabs/command_buffer.c   |   9 +-
 drivers/misc/habanalabs/device.c           | 317 ++++++++++++++++++++-
 drivers/misc/habanalabs/goya/goya.c        | 210 ++++++++++++++
 drivers/misc/habanalabs/goya/goya_hwmgr.c  |  18 +-
 drivers/misc/habanalabs/habanalabs.h       |  38 +++
 drivers/misc/habanalabs/habanalabs_drv.c   |   9 +-
 drivers/misc/habanalabs/habanalabs_ioctl.c |   6 +
 drivers/misc/habanalabs/hwmon.c            |   4 +-
 drivers/misc/habanalabs/irq.c              |  31 ++
 drivers/misc/habanalabs/sysfs.c            | 120 +++++++-
 10 files changed, 737 insertions(+), 25 deletions(-)
drivers/misc/habanalabs/command_buffer.c +7 -2
index 20adba5f9d0c..c70b548a7b8a 100644
@@ -89,9 +89,14 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr,
 	bool alloc_new_cb = true;
 	int rc;
 
-	if (hdev->disabled) {
+	/*
+	 * Can't use generic function to check this because of special case
+	 * where we create a CB as part of the reset process
+	 */
+	if ((hdev->disabled) || ((atomic_read(&hdev->in_reset)) &&
+					(ctx_id != HL_KERNEL_ASID_ID))) {
 		dev_warn_ratelimited(hdev->dev,
-			"Device is disabled. Can't create new CBs\n");
+			"Device is disabled or in reset. Can't create new CBs\n");
 		rc = -EBUSY;
 		goto out_err;
 	}
drivers/misc/habanalabs/device.c +316 -1

367 lines not on this page · raw message

drivers/misc/habanalabs/goya/goya.c +210 -0

248 lines not on this page · raw message

drivers/misc/habanalabs/goya/goya_hwmgr.c +9 -9
index 866d1774b2e4..68f057c4b224 100644
@@ -38,7 +38,7 @@ static ssize_t mme_clk_show(struct device *dev, struct device_attribute *attr,
 	struct hl_device *hdev = dev_get_drvdata(dev);
 	long value;
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	value = hl_get_frequency(hdev, MME_PLL, false);
@@ -57,7 +57,7 @@ static ssize_t mme_clk_store(struct device *dev, struct device_attribute *attr,
 	int rc;
 	long value;
 
-	if (hdev->disabled) {
+	if (hl_device_disabled_or_in_reset(hdev)) {
 		count = -ENODEV;
 		goto fail;
 	}
@@ -87,7 +87,7 @@ static ssize_t tpc_clk_show(struct device *dev, struct device_attribute *attr,
 	struct hl_device *hdev = dev_get_drvdata(dev);
 	long value;
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	value = hl_get_frequency(hdev, TPC_PLL, false);
@@ -106,7 +106,7 @@ static ssize_t tpc_clk_store(struct device *dev, struct device_attribute *attr,
 	int rc;
 	long value;
 
-	if (hdev->disabled) {
+	if (hl_device_disabled_or_in_reset(hdev)) {
 		count = -ENODEV;
 		goto fail;
 	}
@@ -136,7 +136,7 @@ static ssize_t ic_clk_show(struct device *dev, struct device_attribute *attr,
 	struct hl_device *hdev = dev_get_drvdata(dev);
 	long value;
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	value = hl_get_frequency(hdev, IC_PLL, false);
@@ -155,7 +155,7 @@ static ssize_t ic_clk_store(struct device *dev, struct device_attribute *attr,
 	int rc;
 	long value;
 
-	if (hdev->disabled) {
+	if (hl_device_disabled_or_in_reset(hdev)) {
 		count = -ENODEV;
 		goto fail;
 	}
@@ -185,7 +185,7 @@ static ssize_t mme_clk_curr_show(struct device *dev,
 	struct hl_device *hdev = dev_get_drvdata(dev);
 	long value;
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	value = hl_get_frequency(hdev, MME_PLL, true);
@@ -202,7 +202,7 @@ static ssize_t tpc_clk_curr_show(struct device *dev,
 	struct hl_device *hdev = dev_get_drvdata(dev);
 	long value;
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	value = hl_get_frequency(hdev, TPC_PLL, true);
@@ -219,7 +219,7 @@ static ssize_t ic_clk_curr_show(struct device *dev,
 	struct hl_device *hdev = dev_get_drvdata(dev);
 	long value;
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	value = hl_get_frequency(hdev, IC_PLL, true);
drivers/misc/habanalabs/habanalabs.h +38 -0
index 5e0f20b92bde..402309180b40 100644
@@ -26,8 +26,12 @@
 
 #define HL_MMAP_CB_MASK			(0x8000000000000000ull >> PAGE_SHIFT)
 
+#define HL_PENDING_RESET_PER_SEC	5
+
 #define HL_DEVICE_TIMEOUT_USEC		1000000 /* 1 s */
 
+#define HL_HEARTBEAT_PER_USEC		5000000 /* 5 s */
+
 #define HL_PLL_LOW_JOB_FREQ_USEC	5000000 /* 5 s */
 
 #define HL_MAX_QUEUES			128
@@ -367,8 +371,10 @@ enum hl_pll_frequency {
  * @handle_eqe: handle event queue entry (IRQ) from ArmCP.
  * @set_pll_profile: change PLL profile (manual/automatic).
  * @get_events_stat: retrieve event queue entries histogram.
+ * @send_heartbeat: send is-alive packet to ArmCP and verify response.
  * @enable_clock_gating: enable clock gating for reducing power consumption.
  * @disable_clock_gating: disable clock for accessing registers on HBW.
+ * @soft_reset_late_init: perform certain actions needed after soft reset.
  * @hw_queues_lock: acquire H/W queues lock.
  * @hw_queues_unlock: release H/W queues lock.
  * @get_eeprom_data: retrieve EEPROM data from F/W.
@@ -414,8 +420,10 @@ struct hl_asic_funcs {
 	void (*set_pll_profile)(struct hl_device *hdev,
 			enum hl_pll_frequency freq);
 	void* (*get_events_stat)(struct hl_device *hdev, u32 *size);
+	int (*send_heartbeat)(struct hl_device *hdev);
 	void (*enable_clock_gating)(struct hl_device *hdev);
 	void (*disable_clock_gating)(struct hl_device *hdev);
+	int (*soft_reset_late_init)(struct hl_device *hdev);
 	void (*hw_queues_lock)(struct hl_device *hdev);
 	void (*hw_queues_unlock)(struct hl_device *hdev);
 	int (*get_eeprom_data)(struct hl_device *hdev, void *data,
@@ -533,6 +541,16 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);
 	WREG32(mm##reg, (RREG32(mm##reg) & ~REG_FIELD_MASK(reg, field)) | \
 			(val) << REG_FIELD_SHIFT(reg, field))
 
+/**
+ * struct hl_device_reset_work - reset workqueue task wrapper.
+ * @reset_work: reset work to be done.
+ * @hdev: habanalabs device structure.
+ */
+struct hl_device_reset_work {
+	struct work_struct		reset_work;
+	struct hl_device		*hdev;
+};
+
 /**
  * struct hl_device - habanalabs device structure.
  * @pdev: pointer to PCI device, can be NULL in case of simulator device.
@@ -541,6 +559,7 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);
  * @cdev: related char device.
  * @dev: realted kernel basic device structure.
  * @work_freq: delayed work to lower device frequency if possible.
+ * @work_heartbeat: delayed work for ArmCP is-alive check.
  * @asic_name: ASIC specific nmae.
  * @asic_type: ASIC specific type.
  * @completion_queue: array of hl_cq.
@@ -572,6 +591,7 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);
  * @cb_pool: list of preallocated CBs.
  * @cb_pool_lock: protects the CB pool.
  * @user_ctx: current user context executing.
+ * @in_reset: is device in reset flow.
  * @curr_pll_profile: current PLL profile.
  * @fd_open_cnt: number of open user processes.
  * @max_power: the max power of the device, as configured by the sysadmin. This
@@ -579,10 +599,15 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);
  *             value and update the F/W after the re-initialization
  * @major: habanalabs KMD major.
  * @high_pll: high PLL profile frequency.
+ * @soft_reset_cnt: number of soft reset since KMD loading.
+ * @hard_reset_cnt: number of hard reset since KMD loading.
  * @id: device minor.
  * @disabled: is device disabled.
  * @late_init_done: is late init stage was done during initialization.
  * @hwmon_initialized: is H/W monitor sensors was initialized.
+ * @hard_reset_pending: is there a hard reset work pending.
+ * @heartbeat: is heartbeat sanity check towards ArmCP enabled.
+ * @init_done: is the initialization of the device done.
  */
 struct hl_device {
 	struct pci_dev			*pdev;
@@ -591,6 +616,7 @@ struct hl_device {
 	struct cdev			cdev;
 	struct device			*dev;
 	struct delayed_work		work_freq;
+	struct delayed_work		work_heartbeat;
 	char				asic_name[16];
 	enum hl_asic_type		asic_type;
 	struct hl_cq			*completion_queue;
@@ -622,15 +648,21 @@ struct hl_device {
 	/* TODO: remove user_ctx for multiple process support */
 	struct hl_ctx			*user_ctx;
 
+	atomic_t			in_reset;
 	atomic_t			curr_pll_profile;
 	atomic_t			fd_open_cnt;
 	u64				max_power;
 	u32				major;
 	u32				high_pll;
+	u32				soft_reset_cnt;
+	u32				hard_reset_cnt;
 	u16				id;
 	u8				disabled;
 	u8				late_init_done;
 	u8				hwmon_initialized;
+	u8				hard_reset_pending;
+	u8				heartbeat;
+	u8				init_done;
 
 	/* Parameters for bring-up */
 	u8				cpu_enable;
@@ -671,6 +703,7 @@ struct hl_ioctl_desc {
  */
 
 int hl_device_open(struct inode *inode, struct file *filp);
+bool hl_device_disabled_or_in_reset(struct hl_device *hdev);
 int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 		enum hl_asic_type asic_type, int minor);
 void destroy_hdev(struct hl_device *hdev);
@@ -684,6 +717,7 @@ int hl_hw_queue_send_cb_no_cmpl(struct hl_device *hdev, u32 hw_queue_id,
 				u32 cb_size, u64 cb_ptr);
 u32 hl_hw_queue_add_ptr(u32 ptr, u16 val);
 void hl_hw_queue_inc_ci_kernel(struct hl_device *hdev, u32 hw_queue_id);
+void hl_hw_queue_reset(struct hl_device *hdev, bool hard_reset);
 
 #define hl_queue_inc_ptr(p)		hl_hw_queue_add_ptr(p, 1)
 #define hl_pi_2_offset(pi)		((pi) & (HL_QUEUE_LENGTH - 1))
@@ -692,6 +726,8 @@ int hl_cq_init(struct hl_device *hdev, struct hl_cq *q, u32 hw_queue_id);
 void hl_cq_fini(struct hl_device *hdev, struct hl_cq *q);
 int hl_eq_init(struct hl_device *hdev, struct hl_eq *q);
 void hl_eq_fini(struct hl_device *hdev, struct hl_eq *q);
+void hl_cq_reset(struct hl_device *hdev, struct hl_cq *q);
+void hl_eq_reset(struct hl_device *hdev, struct hl_eq *q);
 irqreturn_t hl_irq_handler_cq(int irq, void *arg);
 irqreturn_t hl_irq_handler_eq(int irq, void *arg);
 int hl_asid_init(struct hl_device *hdev);
@@ -709,6 +745,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass);
 void hl_device_fini(struct hl_device *hdev);
 int hl_device_suspend(struct hl_device *hdev);
 int hl_device_resume(struct hl_device *hdev);
+int hl_device_reset(struct hl_device *hdev, bool hard_reset,
+			bool from_hard_reset_thread);
 void hl_hpriv_get(struct hl_fpriv *hpriv);
 void hl_hpriv_put(struct hl_fpriv *hpriv);
 int hl_device_set_frequency(struct hl_device *hdev, enum hl_pll_frequency freq);
drivers/misc/habanalabs/habanalabs_drv.c +7 -2
index 6260d99ef1ca..0702f6f974eb 100644
@@ -88,9 +88,9 @@ int hl_device_open(struct inode *inode, struct file *filp)
 
 	mutex_lock(&hdev->fd_open_cnt_lock);
 
-	if (hdev->disabled) {
+	if (hl_device_disabled_or_in_reset(hdev)) {
 		dev_err_ratelimited(hdev->dev,
-			"Can't open %s because it is disabled\n",
+			"Can't open %s because it is disabled or in reset\n",
 			dev_name(hdev->dev));
 		mutex_unlock(&hdev->fd_open_cnt_lock);
 		return -EPERM;
@@ -183,6 +183,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 	hdev->cpu_queues_enable = 1;
 	hdev->fw_loading = 1;
 	hdev->pldm = 0;
+	hdev->heartbeat = 1;
 
 	/* If CPU is disabled, no point in loading FW */
 	if (!hdev->cpu_enable)
@@ -192,6 +193,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 	if (!hdev->fw_loading)
 		hdev->cpu_queues_enable = 0;
 
+	/* If CPU queues not enabled, no way to do heartbeat */
+	if (!hdev->cpu_queues_enable)
+		hdev->heartbeat = 0;
+
 	hdev->disabled = true;
 	hdev->pdev = pdev; /* can be NULL in case of simulator device */
 
drivers/misc/habanalabs/habanalabs_ioctl.c +6 -0
index 71a7a782fbd1..12b004b92c83 100644
@@ -33,6 +33,12 @@ long hl_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
 	unsigned int usize, asize;
 	int retcode;
 
+	if (hdev->hard_reset_pending) {
+		dev_crit_ratelimited(hdev->dev,
+			"Device HARD reset pending! Please close FD\n");
+		return -ENODEV;
+	}
+
 	if ((nr >= HL_COMMAND_START) && (nr < HL_COMMAND_END)) {
 		u32 hl_size;
 
drivers/misc/habanalabs/hwmon.c +2 -2
index 4d60b36dfa21..61b0a2b26c19 100644
@@ -111,7 +111,7 @@ static int hl_read(struct device *dev, enum hwmon_sensor_types type,
 {
 	struct hl_device *hdev = dev_get_drvdata(dev);
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	switch (type) {
@@ -185,7 +185,7 @@ static int hl_write(struct device *dev, enum hwmon_sensor_types type,
 {
 	struct hl_device *hdev = dev_get_drvdata(dev);
 
-	if (hdev->disabled)
+	if (hl_device_disabled_or_in_reset(hdev))
 		return -ENODEV;
 
 	switch (type) {
drivers/misc/habanalabs/irq.c +31 -0
index 5a7cb85ab50a..29dc24c63606 100644
@@ -250,6 +250,23 @@ void hl_cq_fini(struct hl_device *hdev, struct hl_cq *q)
 			(void *) q->kernel_address, q->bus_address);
 }
 
+void hl_cq_reset(struct hl_device *hdev, struct hl_cq *q)
+{
+	q->ci = 0;
+	q->pi = 0;
+
+	atomic_set(&q->free_slots_cnt, HL_CQ_LENGTH);
+
+	/*
+	 * It's not enough to just reset the PI/CI because the H/W may have
+	 * written valid completion entries before it was halted and therefore
+	 * we need to clean the actual queues so we won't process old entries
+	 * when the device is operational again
+	 */
+
+	memset((void *) q->kernel_address, 0, HL_CQ_SIZE_IN_BYTES);
+}
+
 /*
  * hl_eq_init - main initialization function for an event queue object
  *
@@ -292,3 +309,17 @@ void hl_eq_fini(struct hl_device *hdev, struct hl_eq *q)
 	hdev->asic_funcs->dma_free_coherent(hdev, HL_EQ_SIZE_IN_BYTES,
 			(void *) q->kernel_address, q->bus_address);
 }
+
+void hl_eq_reset(struct hl_device *hdev, struct hl_eq *q)
+{
+	q->ci = 0;
+
+	/*
+	 * It's not enough to just reset the PI/CI because the H/W may have
+	 * written valid completion entries before it was halted and therefore
+	 * we need to clean the actual queues so we won't process old entries
+	 * when the device is operational again
+	 */
+
+	memset((void *) q->kernel_address, 0, HL_EQ_SIZE_IN_BYTES);
+}
drivers/misc/habanalabs/sysfs.c +111 -9

228 lines not on this page · raw message

Signature · 1 line
2.17.1
#11 lore raw

PATCH v3 9/15 habanalabs: add sysfs and hwmon support

In reply to Oded Gabbay

This patch add the sysfs and hwmon entries that are exposed by the driver.

Goya has several sensors, from various categories such as temperature, voltage, current, etc. The driver exposes those sensors in the standard hwmon mechanism.

In addition, the driver exposes a couple of interfaces in sysfs, both for configuration and for providing status of the device or driver.

The configuration attributes is for Power Management:
- Automatic or manual
- Frequency value when moving to high frequency mode
- Maximum power the device is allowed to consume

The rest of the attributes are read-only and provide the following information:
- Versions of the various firmwares running on the device
- Contents of the device's EEPROM
- The device type (currently only Goya is supported)
- PCI address of the device (to allow user-space to connect between /dev/hlX to PCI address)
- Status of the device (operational, malfunction, in_reset)
- How many processes are open on the device's file

---
 .../ABI/testing/sysfs-driver-habanalabs       | 190 ++++++
 drivers/misc/habanalabs/Makefile              |   2 +-
 drivers/misc/habanalabs/device.c              | 146 +++++
 drivers/misc/habanalabs/goya/Makefile         |   2 +-
 drivers/misc/habanalabs/goya/goya.c           | 230 +++++++
 drivers/misc/habanalabs/goya/goyaP.h          |  21 +
 drivers/misc/habanalabs/goya/goya_hwmgr.c     | 306 +++++++++
 drivers/misc/habanalabs/habanalabs.h          | 101 +++
 drivers/misc/habanalabs/habanalabs_drv.c      |   7 +
 drivers/misc/habanalabs/hwmon.c               | 449 +++++++++++++
 drivers/misc/habanalabs/sysfs.c               | 589 ++++++++++++++++++
 11 files changed, 2041 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c
 create mode 100644 drivers/misc/habanalabs/hwmon.c
 create mode 100644 drivers/misc/habanalabs/sysfs.c
Documentation/ABI/testing/sysfs-driver-habanalabs new file +190 -0
new file mode 100644
index 000000000000..19edd4da87c1
@@ -0,0 +1,190 @@
+What:           /sys/class/habanalabs/hl<n>/armcp_kernel_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the Linux kernel running on the device's CPU
+
+What:           /sys/class/habanalabs/hl<n>/armcp_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the application running on the device's CPU
+
+What:           /sys/class/habanalabs/hl<n>/cpld_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the Device's CPLD F/W
+
+What:           /sys/class/habanalabs/hl<n>/device_type
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the code name of the device according to its type.
+                The supported values are: "GOYA"
+
+What:           /sys/class/habanalabs/hl<n>/eeprom
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    A binary file attribute that contains the contents of the
+                on-board EEPROM
+
+What:           /sys/class/habanalabs/hl<n>/fuse_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the device's version from the eFuse
+
+What:           /sys/class/habanalabs/hl<n>/hard_reset
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Interface to trigger a hard-reset operation for the device.
+                Hard-reset will reset ALL internal components of the device
+                except for the PCI interface and the internal PLLs
+
+What:           /sys/class/habanalabs/hl<n>/hard_reset_cnt
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays how many times the device have undergone a hard-reset
+                operation
+
+What:           /sys/class/habanalabs/hl<n>/high_pll
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Allows the user to set the maximum clock frequency for MME, TPC
+                and IC when the power management profile is set to "automatic".
+
+What:           /sys/class/habanalabs/hl<n>/ic_clk
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Allows the user to set the maximum clock frequency of the
+                Interconnect fabric. Writes to this parameter affect the device
+                only when the power management profile is set to "manual" mode.
+                The device IC clock might be set to lower value then the
+                maximum. The user should read the ic_clk_curr to see the actual
+                frequency value of the IC
+
+What:           /sys/class/habanalabs/hl<n>/ic_clk_curr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the current clock frequency of the Interconnect fabric
+
+What:           /sys/class/habanalabs/hl<n>/infineon_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the Device's power supply F/W code
+
+What:           /sys/class/habanalabs/hl<n>/max_power
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Allows the user to set the maximum power consumption of the
+                device in milliwatts.
+
+What:           /sys/class/habanalabs/hl<n>/mme_clk
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Allows the user to set the maximum clock frequency of the
+                MME compute engine. Writes to this parameter affect the device
+                only when the power management profile is set to "manual" mode.
+                The device MME clock might be set to lower value then the
+                maximum. The user should read the mme_clk_curr to see the actual
+                frequency value of the MME
+
+What:           /sys/class/habanalabs/hl<n>/mme_clk_curr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the current clock frequency of the MME compute engine
+
+What:           /sys/class/habanalabs/hl<n>/pci_addr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the PCI address of the device. This is needed so the
+                user would be able to open a device based on its PCI address
+
+What:           /sys/class/habanalabs/hl<n>/pm_mng_profile
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Power management profile. Values are "auto", "manual". In "auto"
+                mode, the driver will set the maximum clock frequency to a high
+                value when a user-space process opens the device's file (unless
+                it was already opened by another process). The driver will set
+                the max clock frequency to a low value when there are no user
+                processes that are opened on the device's file. In "manual"
+                mode, the user sets the maximum clock frequency by writing to
+                ic_clk, mme_clk and tpc_clk
+
+
+What:           /sys/class/habanalabs/hl<n>/preboot_btl_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the device's preboot F/W code
+
+What:           /sys/class/habanalabs/hl<n>/soft_reset
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Interface to trigger a soft-reset operation for the device.
+                Soft-reset will reset only the compute and DMA engines of the
+                device
+
+What:           /sys/class/habanalabs/hl<n>/soft_reset_cnt
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays how many times the device have undergone a soft-reset
+                operation
+
+What:           /sys/class/habanalabs/hl<n>/status
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Status of the card: "Operational", "Malfunction", "In reset".
+
+What:           /sys/class/habanalabs/hl<n>/thermal_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the Device's thermal daemon
+
+What:           /sys/class/habanalabs/hl<n>/tpc_clk
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Allows the user to set the maximum clock frequency of the
+                TPC compute engines. Writes to this parameter affect the device
+                only when the power management profile is set to "manual" mode.
+                The device TPC clock might be set to lower value then the
+                maximum. The user should read the tpc_clk_curr to see the actual
+                frequency value of the TPC
+
+What:           /sys/class/habanalabs/hl<n>/tpc_clk_curr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the current clock frequency of the TPC compute engines
+
+What:           /sys/class/habanalabs/hl<n>/uboot_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the u-boot running on the device's CPU
+
+What:           /sys/class/habanalabs/hl<n>/write_open_cnt
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the total number of user processes that are currently
+                opened on the device's file
drivers/misc/habanalabs/Makefile +1 -1
index 1e2a0c1b3890..2aeadc4cef30 100644
@@ -5,7 +5,7 @@
 obj-m	:= habanalabs.o
 
 habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
-		command_buffer.o hw_queue.o irq.o
+		command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/device.c +146 -0

175 lines not on this page · raw message

drivers/misc/habanalabs/goya/Makefile +1 -1
index b546e9f5ef76..46125e4c2799 100644
@@ -1,4 +1,4 @@
 subdir-ccflags-y += -I$(src)
 
-HL_GOYA_FILES :=  goya/goya.o goya/goya_security.o
+HL_GOYA_FILES :=  goya/goya.o goya/goya_security.o goya/goya_hwmgr.o
 
drivers/misc/habanalabs/goya/goya.c +230 -0

282 lines not on this page · raw message

drivers/misc/habanalabs/goya/goyaP.h +21 -0
index 58f1e316f8d9..6d2da801c155 100644
@@ -50,7 +50,10 @@
 
 #define PLL_HIGH_DEFAULT		1575000000	/* 1.575 GHz */
 
+#define MAX_POWER_DEFAULT		200000		/* 200W */
+
 #define GOYA_ARMCP_INFO_TIMEOUT		10000000	/* 10s */
+#define GOYA_ARMCP_EEPROM_TIMEOUT	10000000	/* 10s */
 
 #define DRAM_PHYS_DEFAULT_SIZE		0x100000000ull	/* 4GB */
 
@@ -151,9 +154,15 @@ enum goya_fw_component {
 
 struct goya_device {
 	int (*test_cpu_queue)(struct hl_device *hdev);
+	int (*armcp_info_get)(struct hl_device *hdev);
 
 	/* TODO: remove hw_queues_lock after moving to scheduler code */
 	spinlock_t	hw_queues_lock;
+
+	u64		mme_clk;
+	u64		tpc_clk;
+	u64		ic_clk;
+
 	u64		ddr_bar_cur_addr;
 	u32		events_stat[GOYA_ASYNC_EVENT_ID_SIZE];
 	u32		hw_cap_initialized;
@@ -162,6 +171,18 @@ struct goya_device {
 int goya_test_cpu_queue(struct hl_device *hdev);
 int goya_send_cpu_message(struct hl_device *hdev, u32 *msg, u16 len,
 				u32 timeout, long *result);
+long goya_get_temperature(struct hl_device *hdev, int sensor_index, u32 attr);
+long goya_get_voltage(struct hl_device *hdev, int sensor_index, u32 attr);
+long goya_get_current(struct hl_device *hdev, int sensor_index, u32 attr);
+long goya_get_fan_speed(struct hl_device *hdev, int sensor_index, u32 attr);
+long goya_get_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr);
+void goya_set_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr,
+			long value);
+void goya_set_pll_profile(struct hl_device *hdev, enum hl_pll_frequency freq);
+int goya_add_device_attr(struct hl_device *hdev);
+void goya_remove_device_attr(struct hl_device *hdev);
 void goya_init_security(struct hl_device *hdev);
+u64 goya_get_max_power(struct hl_device *hdev);
+void goya_set_max_power(struct hl_device *hdev, u64 value);
 
 #endif /* GOYAP_H_ */
drivers/misc/habanalabs/goya/goya_hwmgr.c new file +306 -0

306 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs.h +101 -0

239 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +7 -0
index 14775a7022f0..6260d99ef1ca 100644
@@ -131,6 +131,13 @@ int hl_device_open(struct inode *inode, struct file *filp)
 
 	hpriv->taskpid = find_get_pid(current->pid);
 
+	/*
+	 * Device is IDLE at this point so it is legal to change PLLs. There
+	 * is no need to check anything because if the PLL is already HIGH, the
+	 * set function will return without doing anything
+	 */
+	hl_device_set_frequency(hdev, PLL_HIGH);
+
 	return 0;
 
 out_err:
drivers/misc/habanalabs/hwmon.c new file +449 -0

449 lines not on this page · raw message

drivers/misc/habanalabs/sysfs.c new file +589 -0

589 lines not on this page · raw message

Signature · 1 line
2.17.1
#16 lore raw

PATCH v3 12/15 habanalabs: add virtual memory and MMU modules

In reply to Oded Gabbay

From: Omer Shpigelman <[email protected]>

This patch adds the Virtual Memory and MMU modules.

Goya has an internal MMU which provides process isolation on the internal DDR. The internal MMU also performs translations for transactions that go from Goya to the Host.

The driver is responsible for allocating and freeing memory on the DDR upon user request. It also provides an interface to map and unmap DDR and Host memory to the device address space.

---
Changes in v3:
- Remove use of three exclamation marks
- Add optimization - support mapping for huge page host memory
- Minor bug fixes

 drivers/misc/habanalabs/Makefile              |    2 +-
 drivers/misc/habanalabs/context.c             |   19 +-
 drivers/misc/habanalabs/device.c              |   20 +-
 drivers/misc/habanalabs/goya/goya.c           |  393 +++++
 drivers/misc/habanalabs/habanalabs.h          |  188 +-
 drivers/misc/habanalabs/habanalabs_drv.c      |    2 +-
 drivers/misc/habanalabs/habanalabs_ioctl.c    |    3 +-
 .../include/hw_ip/mmu/mmu_general.h           |   45 +
 .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h   |   15 +
 drivers/misc/habanalabs/memory.c              | 1514 +++++++++++++++++
 drivers/misc/habanalabs/mmu.c                 |  604 +++++++
 include/uapi/misc/habanalabs.h                |  122 +-
 12 files changed, 2919 insertions(+), 8 deletions(-)
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h
 create mode 100644 drivers/misc/habanalabs/mmu.c
drivers/misc/habanalabs/Makefile +1 -1
index 5167699701b9..2698bb6a2355 100644
@@ -6,7 +6,7 @@ obj-m	:= habanalabs.o
 
 habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
 		command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \
-		command_submission.o
+		command_submission.o mmu.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/context.c +18 -1
index 98710646de6c..6ff0f2103d8d 100644
@@ -26,8 +26,10 @@ static void hl_ctx_fini(struct hl_ctx *ctx)
 	for (i = 0 ; i < HL_MAX_PENDING_CS ; i++)
 		dma_fence_put(ctx->cs_pending[i]);
 
-	if (ctx->asid != HL_KERNEL_ASID_ID)
+	if (ctx->asid != HL_KERNEL_ASID_ID) {
+		hl_vm_ctx_fini(ctx);
 		hl_asid_free(hdev, ctx->asid);
+	}
 }
 
 void hl_ctx_do_release(struct kref *ref)
@@ -97,6 +99,8 @@ void hl_ctx_free(struct hl_device *hdev, struct hl_ctx *ctx)
 
 int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx)
 {
+	int rc = 0;
+
 	ctx->hdev = hdev;
 
 	kref_init(&ctx->refcount);
@@ -114,9 +118,22 @@ int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx)
 			dev_err(hdev->dev, "No free ASID, failed to create context\n");
 			return -ENOMEM;
 		}
+
+		rc = hl_vm_ctx_init(ctx);
+		if (rc) {
+			dev_err(hdev->dev, "Failed to init mem ctx module\n");
+			rc = -ENOMEM;
+			goto mem_ctx_err;
+		}
 	}
 
 	return 0;
+
+mem_ctx_err:
+	if (ctx->asid != HL_KERNEL_ASID_ID)
+		hl_asid_free(hdev, ctx->asid);
+
+	return rc;
 }
 
 void hl_ctx_get(struct hl_device *hdev, struct hl_ctx *ctx)
drivers/misc/habanalabs/device.c +19 -1
index ad27d46a03ba..c667b2507c4d 100644
@@ -604,8 +604,10 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
 	/* Reset the H/W. It will be in idle state after this returns */
 	hdev->asic_funcs->hw_fini(hdev, hard_reset);
 
-	if (hard_reset)
+	if (hard_reset) {
+		hl_vm_fini(hdev);
 		hl_eq_reset(hdev, &hdev->event_queue);
+	}
 
 	/* Re-initialize PI,CI to 0 in all queues (hw queue, cq) */
 	hl_hw_queue_reset(hdev, hard_reset);
@@ -666,6 +668,13 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
 			goto out_err;
 		}
 
+		rc = hl_vm_init(hdev);
+		if (rc) {
+			dev_err(hdev->dev,
+				"Failed to init memory module after hard reset\n");
+			goto out_err;
+		}
+
 		hl_set_max_power(hdev, hdev->max_power);
 
 		hdev->hard_reset_pending = false;
@@ -844,6 +853,13 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass)
 		hdev->asic_name,
 		hdev->asic_prop.dram_size / 1024 / 1024 / 1024);
 
+	rc = hl_vm_init(hdev);
+	if (rc) {
+		dev_err(hdev->dev, "Failed to initialize memory module\n");
+		rc = 0;
+		goto out_disabled;
+	}
+
 	rc = hl_hwmon_init(hdev);
 	if (rc) {
 		dev_err(hdev->dev, "Failed to initialize hwmon\n");
@@ -955,6 +971,8 @@ void hl_device_fini(struct hl_device *hdev)
 	/* Reset the H/W. It will be in idle state after this returns */
 	hdev->asic_funcs->hw_fini(hdev, true);
 
+	hl_vm_fini(hdev);
+
 	hl_eq_fini(hdev, &hdev->event_queue);
 
 	for (i = 0 ; i < hdev->asic_prop.completion_queues_count ; i++)
drivers/misc/habanalabs/goya/goya.c +393 -0

466 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs.h +186 -2

321 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +1 -1
index dbc0c9c2c99a..658dc4588a36 100644
@@ -192,7 +192,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 	hdev->reset_on_lockup = reset_on_lockup;
 
 	/* Parameters for bring-up - set them to defaults */
-	hdev->mmu_enable = 0;
+	hdev->mmu_enable = 1;
 	hdev->cpu_enable = 1;
 	hdev->reset_pcilink = 0;
 	hdev->cpu_queues_enable = 1;
drivers/misc/habanalabs/habanalabs_ioctl.c +2 -1
index 35b3df73b752..cd6904e639e9 100644
@@ -18,7 +18,8 @@
 static const struct hl_ioctl_desc hl_ioctls[] = {
 	HL_IOCTL_DEF(HL_IOCTL_CB, hl_cb_ioctl),
 	HL_IOCTL_DEF(HL_IOCTL_CS, hl_cs_ioctl),
-	HL_IOCTL_DEF(HL_IOCTL_WAIT_CS, hl_cs_wait_ioctl)
+	HL_IOCTL_DEF(HL_IOCTL_WAIT_CS, hl_cs_wait_ioctl),
+	HL_IOCTL_DEF(HL_IOCTL_MEMORY, hl_mem_ioctl)
 };
 
 #define HL_CORE_IOCTL_COUNT	ARRAY_SIZE(hl_ioctls)
drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h new file +45 -0

45 lines not on this page · raw message

drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h new file +15 -0
new file mode 100644
index 000000000000..8539dd041f2c
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright 2016-2018 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ *
+ */
+
+#ifndef INCLUDE_MMU_V1_0_H_
+#define INCLUDE_MMU_V1_0_H_
+
+#define MMU_HOP0_PA43_12	0x490004
+#define MMU_HOP0_PA49_44	0x490008
+#define MMU_ASID_BUSY		0x490000
+
+#endif /* INCLUDE_MMU_V1_0_H_ */
drivers/misc/habanalabs/memory.c +1514 -0

1,529 lines not on this page · raw message

drivers/misc/habanalabs/mmu.c new file +604 -0

513 lines not on this page · raw message

include/uapi/misc/habanalabs.h +121 -1

122 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#20 lore raw

PATCH v3 14/15 habanalabs: add debugfs support

In reply to Oded Gabbay

This patch adds debugfs support to the driver. It allows the user-space to display information that is contained in the internal structures of the driver, such as:
- active command submissions
- active user virtual memory mappings
- number of allocated command buffers

It also enables the user to perform reads and writes through Goya's PCI bars.

---
Changes in v3:
- Adjust code to huge page optimization

 .../ABI/testing/debugfs-driver-habanalabs     |  127 ++
 drivers/misc/habanalabs/Makefile              |    2 +
 drivers/misc/habanalabs/command_buffer.c      |    4 +
 drivers/misc/habanalabs/command_submission.c  |   12 +
 drivers/misc/habanalabs/debugfs.c             | 1071 +++++++++++++++++
 drivers/misc/habanalabs/device.c              |    6 +
 drivers/misc/habanalabs/goya/goya.c           |  108 ++
 drivers/misc/habanalabs/goya/goyaP.h          |    5 +
 drivers/misc/habanalabs/habanalabs.h          |  190 +++
 drivers/misc/habanalabs/habanalabs_drv.c      |   16 +-
 drivers/misc/habanalabs/memory.c              |    8 +
 11 files changed, 1547 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/debugfs.c
Documentation/ABI/testing/debugfs-driver-habanalabs new file +127 -0
new file mode 100644
index 000000000000..2b606c84938c
@@ -0,0 +1,127 @@
+What:           /sys/kernel/debug/habanalabs/hl<n>/addr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the device address to be used for read or write through
+                PCI bar. The acceptable value is a string that starts with "0x"
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/command_buffers
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about the currently allocated
+                command buffers
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/command_submission
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about the currently active
+                command submissions
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/command_submission_jobs
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with detailed information about each JOB (CB) of
+                each active command submission
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/data32
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Allows the root user to read or write directly through the
+                device's PCI bar. Writing to this file generates a write
+                transaction while reading from the file generates a read
+                transcation. This custom interface is needed (instead of using
+                the generic Linux user-space PCI mapping) because the DDR bar
+                is very small compared to the DDR memory and only the driver can
+                move the bar before and after the transaction
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/device
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Enables the root user to set the device to specific state.
+                Valid values are "disable", "enable", "suspend", "resume".
+                User can read this property to see the valid values
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_addr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets I2C device address for I2C transaction that is generated
+                by the device's CPU
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_bus
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets I2C bus address for I2C transaction that is generated by
+                the device's CPU
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_data
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Triggers an I2C transaction that is generated by the device's
+                CPU. Writing to this file generates a write transaction while
+                reading from the file generates a read transcation
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_reg
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets I2C register id for I2C transaction that is generated by
+                the device's CPU
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/led0
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the state of the first S/W led on the device
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/led1
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the state of the second S/W led on the device
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/led2
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the state of the third S/W led on the device
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/mmu
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the hop values and physical address for a given ASID
+                and virtual address. The user should write the ASID and VA into
+                the file and then read the file to get the result.
+                e.g. to display info about VA 0x1000 for ASID 1 you need to do:
+                echo "1 0x1000" > /sys/kernel/debug/habanalabs/hl0/mmu
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/set_power_state
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the PCI power state. Valid values are "1" for D0 and "2"
+                for D3Hot
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/userptr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about the currently user
+                pointers (user virtual addresses) that are pinned and mapped
+                to DMA addresses
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/vm
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about all the active virtual
+                address mappings per ASID
+
drivers/misc/habanalabs/Makefile +2 -0
index 2698bb6a2355..625046487aa0 100644
@@ -8,6 +8,8 @@ habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
 		command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \
 		command_submission.o mmu.o
 
+habanalabs-$(CONFIG_DEBUG_FS) += debugfs.o
+
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
 
drivers/misc/habanalabs/command_buffer.c +4 -0
index c70b548a7b8a..607f6abc7f3f 100644
@@ -36,6 +36,8 @@ static void cb_release(struct kref *ref)
 	cb = container_of(ref, struct hl_cb, refcount);
 	hdev = cb->hdev;
 
+	hl_debugfs_remove_cb(cb);
+
 	cb_do_release(hdev, cb);
 }
 
@@ -153,6 +155,8 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr,
 	*handle = cb->id | HL_MMAP_CB_MASK;
 	*handle <<= PAGE_SHIFT;
 
+	hl_debugfs_add_cb(cb);
+
 	return 0;
 
 release_cb:
drivers/misc/habanalabs/command_submission.c +12 -0
index 288dddc14873..e539ac0a4ca3 100644
@@ -153,6 +153,8 @@ static void free_job(struct hl_device *hdev, struct hl_cs_job *job)
 	list_del(&job->cs_node);
 	spin_unlock(&cs->job_lock);
 
+	hl_debugfs_remove_job(hdev, job);
+
 	if (job->ext_queue)
 		cs_put(cs);
 
@@ -216,6 +218,12 @@ static void cs_do_release(struct kref *ref)
 		}
 	}
 
+	/*
+	 * Must be called before hl_ctx_put because inside we use ctx to get
+	 * the device
+	 */
+	hl_debugfs_remove_cs(cs);
+
 	hl_ctx_put(cs->ctx);
 
 	if (cs->timedout)
@@ -484,6 +492,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks,
 
 	*cs_seq = cs->sequence;
 
+	hl_debugfs_add_cs(cs);
+
 	/* Validate ALL the CS chunks before submitting the CS */
 	for (i = 0, parse_cnt = 0 ; i < num_chunks ; i++, parse_cnt++) {
 		struct hl_cs_chunk *chunk = &cs_chunk_array[i];
@@ -532,6 +542,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks,
 		if (job->ext_queue)
 			cs_get(cs);
 
+		hl_debugfs_add_job(hdev, job);
+
 		rc = cs_parser(hpriv, job);
 		if (rc) {
 			dev_err(hdev->dev,
drivers/misc/habanalabs/debugfs.c new file +1071 -0

1,071 lines not on this page · raw message

drivers/misc/habanalabs/device.c +6 -0
index c667b2507c4d..1ec5e10cc36d 100644
@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)
 
 	put_pid(hpriv->taskpid);
 
+	hl_debugfs_remove_file(hpriv);
+
 	mutex_destroy(&hpriv->restore_phase_mutex);
 
 	kfree(hpriv);
@@ -823,6 +825,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass)
 		goto free_cb_pool;
 	}
 
+	hl_debugfs_add_device(hdev);
+
 	rc = hdev->asic_funcs->hw_init(hdev);
 	if (rc) {
 		dev_err(hdev->dev, "failed to initialize the H/W\n");
@@ -950,6 +954,8 @@ void hl_device_fini(struct hl_device *hdev)
 
 	device_late_fini(hdev);
 
+	hl_debugfs_remove_device(hdev);
+
 	hl_sysfs_fini(hdev);
 
 	/*
drivers/misc/habanalabs/goya/goya.c +108 -0

144 lines not on this page · raw message

drivers/misc/habanalabs/goya/goyaP.h +5 -0
index 6d2da801c155..569015857c59 100644
@@ -168,6 +168,10 @@ struct goya_device {
 	u32		hw_cap_initialized;
 };
 
+int goya_debugfs_i2c_read(struct hl_device *hdev, u8 i2c_bus,
+			u8 i2c_addr, u8 i2c_reg, u32 *val);
+int goya_debugfs_i2c_write(struct hl_device *hdev, u8 i2c_bus,
+			u8 i2c_addr, u8 i2c_reg, u32 val);
 int goya_test_cpu_queue(struct hl_device *hdev);
 int goya_send_cpu_message(struct hl_device *hdev, u32 *msg, u16 len,
 				u32 timeout, long *result);
@@ -178,6 +182,7 @@ long goya_get_fan_speed(struct hl_device *hdev, int sensor_index, u32 attr);
 long goya_get_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr);
 void goya_set_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr,
 			long value);
+void goya_debugfs_led_set(struct hl_device *hdev, u8 led, u8 state);
 void goya_set_pll_profile(struct hl_device *hdev, enum hl_pll_frequency freq);
 int goya_add_device_attr(struct hl_device *hdev);
 void goya_remove_device_attr(struct hl_device *hdev);
drivers/misc/habanalabs/habanalabs.h +190 -0

285 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +14 -2
index 658dc4588a36..96e0fc9c401f 100644
@@ -150,6 +150,8 @@ int hl_device_open(struct inode *inode, struct file *filp)
 	 */
 	hl_device_set_frequency(hdev, PLL_HIGH);
 
+	hl_debugfs_add_file(hpriv);
+
 	return 0;
 
 out_err:
@@ -417,17 +419,20 @@ static int __init hl_init(void)
 		goto remove_major;
 	}
 
+	hl_debugfs_init();
+
 	rc = pci_register_driver(&hl_pci_driver);
 	if (rc) {
 		pr_err("failed to register pci device\n");
-		goto remove_class;
+		goto remove_debugfs;
 	}
 
 	pr_debug("driver loaded\n");
 
 	return 0;
 
-remove_class:
+remove_debugfs:
+	hl_debugfs_fini();
 	class_destroy(hl_class);
 remove_major:
 	unregister_chrdev_region(MKDEV(hl_major, 0), HL_MAX_MINORS);
@@ -441,6 +446,13 @@ static void __exit hl_exit(void)
 {
 	pci_unregister_driver(&hl_pci_driver);
 
+	/*
+	 * Removing debugfs must be after all devices or simulator devices
+	 * have been removed because otherwise we get a bug in the
+	 * debugfs module for referencing NULL objects
+	 */
+	hl_debugfs_fini();
+
 	class_destroy(hl_class);
 	unregister_chrdev_region(MKDEV(hl_major, 0), HL_MAX_MINORS);
 
drivers/misc/habanalabs/memory.c +8 -0
index cf0be4f254e0..35f2339310d3 100644
@@ -1289,6 +1289,8 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u32 size,
 		goto free_sgt;
 	}
 
+	hl_debugfs_add_userptr(hdev, userptr);
+
 	return 0;
 
 free_sgt:
@@ -1314,6 +1316,8 @@ int hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
 {
 	struct page **pages;
 
+	hl_debugfs_remove_userptr(hdev, userptr);
+
 	if (userptr->dma_mapped)
 		hdev->asic_funcs->hl_dma_unmap_sg(hdev,
 				userptr->sgt->sgl,
@@ -1475,6 +1479,8 @@ int hl_vm_ctx_init_with_ranges(struct hl_ctx *ctx, u64 host_range_start,
 		goto dram_vm_err;
 	}
 
+	hl_debugfs_add_ctx_mem_hash(hdev, ctx);
+
 	return 0;
 
 dram_vm_err:
@@ -1597,6 +1603,8 @@ void hl_vm_ctx_fini(struct hl_ctx *ctx)
 	struct hlist_node *tmp_node;
 	int i;
 
+	hl_debugfs_remove_ctx_mem_hash(hdev, ctx);
+
 	if (!hash_empty(ctx->mem_hash))
 		dev_notice(hdev->dev, "ctx is freed while it has va in use\n");
 
Signature · 1 line
2.17.1
#24 lore raw

PATCH v3 15/15 Update MAINTAINERS and CREDITS with habanalabs info

In reply to Oded Gabbay

The habanalabs driver was written from scratch from the very first days of Habana and is maintained by Oded Gabbay.

---
 CREDITS     | 2 +-
 MAINTAINERS | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)
CREDITS +1 -1
index e818eb6a3e71..03f3d67126fc 100644
@@ -1222,7 +1222,7 @@ S: Brazil
 
 N: Oded Gabbay
 E: [email protected]
-D: AMD KFD maintainer
+D: HabanaLabs and AMD KFD maintainer
 S: 12 Shraga Raphaeli
 S: Petah-Tikva, 4906418
 S: Israel
MAINTAINERS +9 -0
index 8c68de3cfd80..814170776325 100644
@@ -6642,6 +6642,15 @@ F:	drivers/clocksource/h8300_*.c
 F:	drivers/clk/h8300/
 F:	drivers/irqchip/irq-renesas-h8*.c
 
+HABANALABS PCI DRIVER
+M:	Oded Gabbay <[email protected]>
+T:	git https://github.com/HabanaAI/linux.git
+S:	Supported
+F:	drivers/misc/habanalabs/
+F:	include/uapi/misc/habanalabs.h
+F:	Documentation/ABI/testing/sysfs-driver-habanalabs
+F:	Documentation/ABI/testing/debugfs-driver-habanalabs
+
 HACKRF MEDIA DRIVER
 M:	Antti Palosaari <[email protected]>
 L:	[email protected]
Signature · 1 line
2.17.1
#8 lore raw

PATCH v3 6/15 habanalabs: add basic Goya h/w initialization

In reply to Oded Gabbay

This patch adds the basic part of Goya's H/W initialization. It adds code that initializes Goya's internal CPU, various registers that are related to internal routing, scrambling, workarounds for H/W bugs, etc.

It also initializes Goya's security scheme that prevents the user from abusing Goya to steal data from the host, crash the host, change Goya's F/W, etc.

---
Changes in v3:
- Remove use of three exclamation marks
- Rename boot interface header file

 drivers/misc/habanalabs/device.c              |   12 +
 drivers/misc/habanalabs/goya/Makefile         |    3 +-
 drivers/misc/habanalabs/goya/goya.c           |  878 ++++-
 drivers/misc/habanalabs/goya/goyaP.h          |    4 +
 drivers/misc/habanalabs/goya/goya_security.c  | 2999 +++++++++++++++++
 drivers/misc/habanalabs/habanalabs.h          |   15 +
 drivers/misc/habanalabs/habanalabs_drv.c      |    7 +
 drivers/misc/habanalabs/include/armcp_if.h    |   19 +
 .../misc/habanalabs/include/goya/goya_fw_if.h |   28 +
 drivers/misc/habanalabs/include/hl_boot_if.h  |   30 +
 10 files changed, 3988 insertions(+), 7 deletions(-)
 create mode 100644 drivers/misc/habanalabs/goya/goya_security.c
 create mode 100644 drivers/misc/habanalabs/include/armcp_if.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h
 create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h
drivers/misc/habanalabs/device.c +12 -0
index c5296b068aaf..9dcfe2cd3a0f 100644
@@ -320,6 +320,15 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass)
 		goto release_ctx;
 	}
 
+	rc = hdev->asic_funcs->hw_init(hdev);
+	if (rc) {
+		dev_err(hdev->dev, "failed to initialize the H/W\n");
+		rc = 0;
+		goto out_disabled;
+	}
+
+	hdev->disabled = false;
+
 	dev_notice(hdev->dev,
 		"Successfully added device to habanalabs driver\n");
 
@@ -371,6 +380,9 @@ void hl_device_fini(struct hl_device *hdev)
 	if ((hdev->kernel_ctx) && (hl_ctx_put(hdev->kernel_ctx) != 1))
 		dev_err(hdev->dev, "kernel ctx is still alive\n");
 
+	/* Reset the H/W. It will be in idle state after this returns */
+	hdev->asic_funcs->hw_fini(hdev, true);
+
 	/* Call ASIC S/W finalize function */
 	hdev->asic_funcs->sw_fini(hdev);
 
drivers/misc/habanalabs/goya/Makefile +2 -1
index 38d43006386d..b546e9f5ef76 100644
@@ -1,3 +1,4 @@
 subdir-ccflags-y += -I$(src)
 
-HL_GOYA_FILES :=  goya/goya.o
+HL_GOYA_FILES :=  goya/goya.o goya/goya_security.o
+
drivers/misc/habanalabs/goya/goya.c +872 -6

905 lines not on this page · raw message

drivers/misc/habanalabs/goya/goyaP.h +4 -0
index d65ac54ba89b..c3e17de41ab3 100644
@@ -10,7 +10,9 @@
 
 #include <uapi/misc/habanalabs.h>
 #include "habanalabs.h"
+#include "include/hl_boot_if.h"
 #include "include/goya/goya.h"
+#include "include/goya/goya_fw_if.h"
 
 #define NUMBER_OF_CMPLT_QUEUES		5
 #define NUMBER_OF_EXT_HW_QUEUES		5
@@ -152,4 +154,6 @@ struct goya_device {
 	u32		hw_cap_initialized;
 };
 
+void goya_init_security(struct hl_device *hdev);
+
 #endif /* GOYAP_H_ */
drivers/misc/habanalabs/goya/goya_security.c new file +2999 -0

2,051 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs.h +15 -0
index 0ca174a8dad9..6f1e976a7d08 100644

Lines of this file are not shown. View the raw message.

drivers/misc/habanalabs/habanalabs_drv.c +7 -0
index b07979c599e2..2611883eab11 100644

Lines of this file are not shown. View the raw message.

drivers/misc/habanalabs/include/armcp_if.h new file +19 -0
new file mode 100644
index 000000000000..85fc2efe144b

Lines of this file are not shown. View the raw message.

drivers/misc/habanalabs/include/goya/goya_fw_if.h new file +28 -0
new file mode 100644
index 000000000000..a9920cb4a07b

Lines of this file are not shown. View the raw message.

drivers/misc/habanalabs/include/hl_boot_if.h new file +30 -0
new file mode 100644
index 000000000000..7475732b9996

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#15 lore raw

PATCH v3 11/15 habanalabs: add command submission module

In reply to Oded Gabbay

This patch adds the main flow for the user to submit work to the device.

Each work is described by a command submission object (CS). The CS contains 3 arrays of command buffers: One for execution, and two for context-switch (store and restore).

For each CB, the user specifies on which queue to put that CB. In case of an internal queue, the entry doesn't contain a pointer to the CB but the address in the on-chip memory that the CB resides at.

The driver parses some of the CBs to enforce security restrictions.

The user receives a sequence number that represents the CS object. The user can then query the driver regarding the status of the CS, using that sequence number.

In case the CS doesn't finish before the timeout expires, the driver will perform a soft-reset of the device.

---
Changes in v3:
- Remove use of three exclamation marks
- Remove hdev->ifh field as it was used for debugging

 drivers/misc/habanalabs/Makefile             |    3 +-
 drivers/misc/habanalabs/command_submission.c |  770 ++++++++++++
 drivers/misc/habanalabs/context.c            |   52 +-
 drivers/misc/habanalabs/device.c             |   16 +
 drivers/misc/habanalabs/goya/goya.c          | 1122 ++++++++++++++++++
 drivers/misc/habanalabs/habanalabs.h         |  272 +++++
 drivers/misc/habanalabs/habanalabs_drv.c     |   20 +
 drivers/misc/habanalabs/habanalabs_ioctl.c   |    4 +-
 drivers/misc/habanalabs/hw_queue.c           |  232 ++++
 drivers/misc/habanalabs/memory.c             |  200 ++++
 include/uapi/misc/habanalabs.h               |  158 ++-
 11 files changed, 2842 insertions(+), 7 deletions(-)
 create mode 100644 drivers/misc/habanalabs/command_submission.c
 create mode 100644 drivers/misc/habanalabs/memory.c
drivers/misc/habanalabs/Makefile +2 -1
index 2aeadc4cef30..5167699701b9 100644
@@ -5,7 +5,8 @@
 obj-m	:= habanalabs.o
 
 habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
-		command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o
+		command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \
+		command_submission.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_submission.c new file +770 -0

770 lines not on this page · raw message

drivers/misc/habanalabs/context.c +48 -4

82 lines not on this page · raw message

drivers/misc/habanalabs/device.c +16 -0
index d64c250feb1c..ad27d46a03ba 100644
@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)
 
 	put_pid(hpriv->taskpid);
 
+	mutex_destroy(&hpriv->restore_phase_mutex);
+
 	kfree(hpriv);
 
 	/* Now the FD is really closed */
@@ -201,6 +203,8 @@ static int device_early_init(struct hl_device *hdev)
 
 	mutex_init(&hdev->fd_open_cnt_lock);
 	mutex_init(&hdev->send_cpu_message_lock);
+	INIT_LIST_HEAD(&hdev->hw_queues_mirror_list);
+	spin_lock_init(&hdev->hw_queues_mirror_lock);
 	atomic_set(&hdev->in_reset, 0);
 	atomic_set(&hdev->fd_open_cnt, 0);
 
@@ -582,6 +586,9 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
 	 */
 	hdev->asic_funcs->halt_engines(hdev, hard_reset);
 
+	/* Go over all the queues, release all CS and their jobs */
+	hl_cs_rollback_all(hdev);
+
 	if (hard_reset) {
 		/* Release kernel context */
 		if (hl_ctx_put(hdev->kernel_ctx) != 1) {
@@ -605,6 +612,12 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,
 	for (i = 0 ; i < hdev->asic_prop.completion_queues_count ; i++)
 		hl_cq_reset(hdev, &hdev->completion_queue[i]);
 
+	/* Make sure the setup phase for the user context will run again */
+	if (hdev->user_ctx) {
+		atomic_set(&hdev->user_ctx->thread_restore_token, 1);
+		hdev->user_ctx->thread_restore_wait_token = 0;
+	}
+
 	/* Finished tear-down, starting to re-initialize */
 
 	if (hard_reset) {
@@ -930,6 +943,9 @@ void hl_device_fini(struct hl_device *hdev)
 	 */
 	hdev->asic_funcs->halt_engines(hdev, true);
 
+	/* Go over all the queues, release all CS and their jobs */
+	hl_cs_rollback_all(hdev);
+
 	hl_cb_pool_fini(hdev);
 
 	/* Release kernel context */
drivers/misc/habanalabs/goya/goya.c +1122 -0

1,164 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs.h +272 -0

454 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +20 -0
index 0702f6f974eb..dbc0c9c2c99a 100644
@@ -28,6 +28,17 @@ static struct class *hl_class;
 DEFINE_IDR(hl_devs_idr);
 DEFINE_MUTEX(hl_devs_idr_lock);
 
+static int timeout_locked = 5;
+static int reset_on_lockup = 1;
+
+module_param(timeout_locked, int, 0444);
+MODULE_PARM_DESC(timeout_locked,
+	"Device lockup timeout in seconds (0 = disabled, default 5s)");
+
+module_param(reset_on_lockup, int, 0444);
+MODULE_PARM_DESC(reset_on_lockup,
+	"Do device reset on lockup (0 = no, 1 = yes, default yes)");
+
 #define PCI_VENDOR_ID_HABANALABS	0x1da3
 
 #define PCI_IDS_GOYA			0x0001
@@ -117,6 +128,7 @@ int hl_device_open(struct inode *inode, struct file *filp)
 	hpriv->hdev = hdev;
 	filp->private_data = hpriv;
 	hpriv->filp = filp;
+	mutex_init(&hpriv->restore_phase_mutex);
 	kref_init(&hpriv->refcount);
 	nonseekable_open(inode, filp);
 
@@ -144,6 +156,7 @@ int hl_device_open(struct inode *inode, struct file *filp)
 	filp->private_data = NULL;
 	hl_ctx_mgr_fini(hpriv->hdev, &hpriv->ctx_mgr);
 	hl_cb_mgr_fini(hpriv->hdev, &hpriv->cb_mgr);
+	mutex_destroy(&hpriv->restore_phase_mutex);
 	kfree(hpriv);
 
 close_device:
@@ -176,8 +189,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 		return -ENOMEM;
 
 	hdev->major = hl_major;
+	hdev->reset_on_lockup = reset_on_lockup;
 
 	/* Parameters for bring-up - set them to defaults */
+	hdev->mmu_enable = 0;
 	hdev->cpu_enable = 1;
 	hdev->reset_pcilink = 0;
 	hdev->cpu_queues_enable = 1;
@@ -197,6 +212,11 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,
 	if (!hdev->cpu_queues_enable)
 		hdev->heartbeat = 0;
 
+	if (timeout_locked)
+		hdev->timeout_jiffies = msecs_to_jiffies(timeout_locked * 1000);
+	else
+		hdev->timeout_jiffies = MAX_SCHEDULE_TIMEOUT;
+
 	hdev->disabled = true;
 	hdev->pdev = pdev; /* can be NULL in case of simulator device */
 
drivers/misc/habanalabs/habanalabs_ioctl.c +3 -1
index 12b004b92c83..35b3df73b752 100644
@@ -16,7 +16,9 @@
 	[_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func}
 
 static const struct hl_ioctl_desc hl_ioctls[] = {
-	HL_IOCTL_DEF(HL_IOCTL_CB, hl_cb_ioctl)
+	HL_IOCTL_DEF(HL_IOCTL_CB, hl_cb_ioctl),
+	HL_IOCTL_DEF(HL_IOCTL_CS, hl_cs_ioctl),
+	HL_IOCTL_DEF(HL_IOCTL_WAIT_CS, hl_cs_wait_ioctl)
 };
 
 #define HL_CORE_IOCTL_COUNT	ARRAY_SIZE(hl_ioctls)
drivers/misc/habanalabs/hw_queue.c +232 -0

250 lines not on this page · raw message

drivers/misc/habanalabs/memory.c new file +200 -0

163 lines not on this page · raw message

include/uapi/misc/habanalabs.h +157 -1

158 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#19 lore raw

PATCH v3 13/15 habanalabs: implement INFO IOCTL

In reply to Oded Gabbay

This patch implements the INFO IOCTL. That IOCTL is used by the user to query information that is relevant/needed by the user in order to submit deep learning jobs to Goya.

The information is divided into several categories, such as H/W IP, Events that happened, DDR usage and more.

---
Changes in v3:
- Remove use of three exclamation marks

 drivers/misc/habanalabs/goya/goya.c        |   6 +
 drivers/misc/habanalabs/habanalabs.h       |   2 +
 drivers/misc/habanalabs/habanalabs_ioctl.c | 126 +++++++++++++++++++++
 include/uapi/misc/habanalabs.h             |  75 +++++++++++-
 4 files changed, 208 insertions(+), 1 deletion(-)
drivers/misc/habanalabs/goya/goya.c +6 -0
index e87fe053605d..3e858f499013 100644
@@ -5117,6 +5117,11 @@ static void goya_hw_queues_unlock(struct hl_device *hdev)
 	spin_unlock(&goya->hw_queues_lock);
 }
 
+static u32 goya_get_pci_id(struct hl_device *hdev)
+{
+	return hdev->pdev->device;
+}
+
 int goya_get_eeprom_data(struct hl_device *hdev, void *data, size_t max_size)
 {
 	struct goya_device *goya = hdev->asic_specific;
@@ -5214,6 +5219,7 @@ static const struct hl_asic_funcs goya_funcs = {
 	.soft_reset_late_init = goya_soft_reset_late_init,
 	.hw_queues_lock = goya_hw_queues_lock,
 	.hw_queues_unlock = goya_hw_queues_unlock,
+	.get_pci_id = goya_get_pci_id,
 	.get_eeprom_data = goya_get_eeprom_data,
 	.send_cpu_message = goya_send_cpu_message
 };
drivers/misc/habanalabs/habanalabs.h +2 -0
index c798caae2c09..fc33026974c9 100644
@@ -460,6 +460,7 @@ enum hl_pll_frequency {
  * @soft_reset_late_init: perform certain actions needed after soft reset.
  * @hw_queues_lock: acquire H/W queues lock.
  * @hw_queues_unlock: release H/W queues lock.
+ * @get_pci_id: retrieve PCI ID.
  * @get_eeprom_data: retrieve EEPROM data from F/W.
  * @send_cpu_message: send buffer to ArmCP.
  */
@@ -528,6 +529,7 @@ struct hl_asic_funcs {
 	int (*soft_reset_late_init)(struct hl_device *hdev);
 	void (*hw_queues_lock)(struct hl_device *hdev);
 	void (*hw_queues_unlock)(struct hl_device *hdev);
+	u32 (*get_pci_id)(struct hl_device *hdev);
 	int (*get_eeprom_data)(struct hl_device *hdev, void *data,
 				size_t max_size);
 	int (*send_cpu_message)(struct hl_device *hdev, u32 *msg,
drivers/misc/habanalabs/habanalabs_ioctl.c +126 -0

136 lines not on this page · raw message

include/uapi/misc/habanalabs.h +74 -1

93 lines not on this page · raw message

Signature · 1 line
2.17.1
#4 lore raw

PATCH v3 5/15 habanalabs: add command buffer module

In reply to Oded Gabbay

This patch adds the command buffer (CB) module, which allows the user to create and destroy CBs and to map them to the user's process address-space.

A command buffer is a memory blocks that reside in DMA-able address-space and is physically contiguous so it can be accessed by the device without MMU translation. The command buffer memory is allocated using the coherent DMA API.

When creating a new CB, the IOCTL returns a handle of it, and the user-space process needs to use that handle to mmap the buffer to get a VA in the user's address-space.

Before destroying (freeing) a CB, the user must unmap the CB's VA using the CB handle.

Each CB has a reference counter, which tracks its usage in command submissions and also its mmaps (only a single mmap is allowed).

The driver maintains a pool of pre-allocated CBs in order to reduce latency during command submissions. In case the pool is empty, the driver will go to the slow-path of allocating a new CB, i.e. calling dma_alloc_coherent.

---
Changes in v3:
- Remove use of three exclamation marks

 drivers/misc/habanalabs/Makefile           |   3 +-
 drivers/misc/habanalabs/command_buffer.c   | 423 +++++++++++++++++++++
 drivers/misc/habanalabs/device.c           |  43 ++-
 drivers/misc/habanalabs/goya/goya.c        |  28 ++
 drivers/misc/habanalabs/habanalabs.h       |  86 +++++
 drivers/misc/habanalabs/habanalabs_drv.c   |   2 +
 drivers/misc/habanalabs/habanalabs_ioctl.c |  99 +++++
 include/uapi/misc/habanalabs.h             |  46 +++
 8 files changed, 728 insertions(+), 2 deletions(-)
 create mode 100644 drivers/misc/habanalabs/command_buffer.c
 create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c
drivers/misc/habanalabs/Makefile +2 -1
index 4944d6ccf8fe..06c886289049 100644
@@ -4,7 +4,8 @@
 
 obj-m	:= habanalabs.o
 
-habanalabs-y := habanalabs_drv.o device.o context.o asid.o
+habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
+		command_buffer.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_buffer.c new file +423 -0

423 lines not on this page · raw message

drivers/misc/habanalabs/device.c +42 -1

87 lines not on this page · raw message

drivers/misc/habanalabs/goya/goya.c +28 -0
index 47e7472b7472..6626dead7231 100644
@@ -92,6 +92,9 @@
 
 #define GOYA_MAX_INITIATORS		20
 
+#define GOYA_CB_POOL_CB_CNT		512
+#define GOYA_CB_POOL_CB_SIZE		0x20000		/* 128KB */
+
 static void goya_get_fixed_properties(struct hl_device *hdev)
 {
 	struct asic_fixed_properties *prop = &hdev->asic_prop;
@@ -119,6 +122,8 @@ static void goya_get_fixed_properties(struct hl_device *hdev)
 	prop->tpc_enabled_mask = TPC_ENABLED_MASK;
 
 	prop->high_pll = PLL_HIGH_DEFAULT;
+	prop->cb_pool_cb_cnt = GOYA_CB_POOL_CB_CNT;
+	prop->cb_pool_cb_size = GOYA_CB_POOL_CB_SIZE;
 }
 
 /*
@@ -604,6 +609,27 @@ int goya_resume(struct hl_device *hdev)
 	return 0;
 }
 
+int goya_mmap(struct hl_fpriv *hpriv, struct vm_area_struct *vma)
+{
+	return -EINVAL;
+}
+
+int goya_cb_mmap(struct hl_device *hdev, struct vm_area_struct *vma,
+		u64 kaddress, phys_addr_t paddress, u32 size)
+{
+	int rc;
+
+	vma->vm_flags |= VM_IO | VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP |
+			VM_DONTCOPY | VM_NORESERVE;
+
+	rc = remap_pfn_range(vma, vma->vm_start, paddress >> PAGE_SHIFT,
+				size, vma->vm_page_prot);
+	if (rc)
+		dev_err(hdev->dev, "remap_pfn_range error %d", rc);
+
+	return rc;
+}
+
 void *goya_dma_alloc_coherent(struct hl_device *hdev, size_t size,
 					dma_addr_t *dma_handle, gfp_t flags)
 {
@@ -623,6 +649,8 @@ static const struct hl_asic_funcs goya_funcs = {
 	.sw_fini = goya_sw_fini,
 	.suspend = goya_suspend,
 	.resume = goya_resume,
+	.mmap = goya_mmap,
+	.cb_mmap = goya_cb_mmap,
 	.dma_alloc_coherent = goya_dma_alloc_coherent,
 	.dma_free_coherent = goya_dma_free_coherent,
 };
drivers/misc/habanalabs/habanalabs.h +86 -0

166 lines not on this page · raw message

drivers/misc/habanalabs/habanalabs_drv.c +2 -0
index 8ecb51fbb133..b07979c599e2 100644
@@ -120,6 +120,7 @@ int hl_device_open(struct inode *inode, struct file *filp)
 	kref_init(&hpriv->refcount);
 	nonseekable_open(inode, filp);
 
+	hl_cb_mgr_init(&hpriv->cb_mgr);
 	hl_ctx_mgr_init(&hpriv->ctx_mgr);
 
 	rc = hl_ctx_create(hdev, hpriv);
@@ -135,6 +136,7 @@ int hl_device_open(struct inode *inode, struct file *filp)
 out_err:
 	filp->private_data = NULL;
 	hl_ctx_mgr_fini(hpriv->hdev, &hpriv->ctx_mgr);
+	hl_cb_mgr_fini(hpriv->hdev, &hpriv->cb_mgr);
 	kfree(hpriv);
 
 close_device:
drivers/misc/habanalabs/habanalabs_ioctl.c new file +99 -0

99 lines not on this page · raw message

include/uapi/misc/habanalabs.h +46 -0

50 lines not on this page · raw message

Signature · 1 line
2.17.1
Reviewed-by on the v3 series
#25 lore raw
In reply to Oded Gabbay
Quoted 34 lines · Oded Gabbay

Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.

Link to v2 cover letter: https://lkml.org/lkml/2019/1/30/1003

Link to v1 cover letter: https://lwn.net/Articles/777342/

I would appricate any feedback, question and/or review.

Thanks,
Oded

p.s. for those who prefer to clone the tree instead of looking at the emails, you can grab a copy from our company's page in GitHub:

https://github.com/HabanaAI/linux/releases/tag/hl_patchset_v3_20190204

Oded Gabbay (14):
habanalabs: add skeleton driver
habanalabs: add Goya registers header files
habanalabs: add basic Goya support
habanalabs: add context and ASID modules
habanalabs: add command buffer module
habanalabs: add basic Goya h/w initialization
habanalabs: add h/w queues module
habanalabs: add event queue and interrupts
habanalabs: add sysfs and hwmon support
habanalabs: add device reset support
habanalabs: add command submission module
habanalabs: implement INFO IOCTL
habanalabs: add debugfs support
Update MAINTAINERS and CREDITS with habanalabs info

Quoted 2 lines

Omer Shpigelman (1):
habanalabs: add virtual memory and MMU modules

Nit: git am is unhappy about black lines at EOF, e.g:

Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +

Aside from that, for patches 1, 3-11 and 13 feel free to add

As for the "habanalabs: add virtual memory and MMU modules" and the debugfs patch I'll need more time to review the changes related to 2M pages.

Quoted 231 lines
 CREDITS                                       |    2 +-
 .../ABI/testing/debugfs-driver-habanalabs     |  127 +
 .../ABI/testing/sysfs-driver-habanalabs       |  190 +
 MAINTAINERS                                   |    9 +
 drivers/misc/Kconfig                          |    1 +
 drivers/misc/Makefile                         |    1 +
 drivers/misc/habanalabs/Kconfig               |   22 +
 drivers/misc/habanalabs/Makefile              |   15 +
 drivers/misc/habanalabs/asid.c                |   58 +
 drivers/misc/habanalabs/command_buffer.c      |  432 ++
 drivers/misc/habanalabs/command_submission.c  |  782 +++
 drivers/misc/habanalabs/context.c             |  216 +
 drivers/misc/habanalabs/debugfs.c             | 1071 ++++
 drivers/misc/habanalabs/device.c              | 1111 ++++
 drivers/misc/habanalabs/goya/Makefile         |    4 +
 drivers/misc/habanalabs/goya/goya.c           | 5344 +++++++++++++++++
 drivers/misc/habanalabs/goya/goyaP.h          |  193 +
 drivers/misc/habanalabs/goya/goya_hwmgr.c     |  306 +
 drivers/misc/habanalabs/goya/goya_security.c  | 2999 +++++++++
 drivers/misc/habanalabs/habanalabs.h          | 1441 +++++
 drivers/misc/habanalabs/habanalabs_drv.c      |  465 ++
 drivers/misc/habanalabs/habanalabs_ioctl.c    |  234 +
 drivers/misc/habanalabs/hw_queue.c            |  636 ++
 drivers/misc/habanalabs/hwmon.c               |  449 ++
 drivers/misc/habanalabs/include/armcp_if.h    |  335 ++
 .../include/goya/asic_reg/cpu_ca53_cfg_regs.h |  213 +
 .../include/goya/asic_reg/cpu_if_regs.h       |  110 +
 .../include/goya/asic_reg/cpu_pll_regs.h      |  186 +
 .../include/goya/asic_reg/dma_ch_0_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_1_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_2_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_3_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_4_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_macro_regs.h    |  242 +
 .../include/goya/asic_reg/dma_nrtr_regs.h     |  380 ++
 .../include/goya/asic_reg/dma_qm_0_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_1_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_2_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_3_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_4_regs.h     |  543 ++
 .../include/goya/asic_reg/goya_blocks.h       | 1372 +++++
 .../include/goya/asic_reg/goya_masks.h        |  262 +
 .../include/goya/asic_reg/goya_regs.h         |   99 +
 .../include/goya/asic_reg/ic_pll_regs.h       |  186 +
 .../include/goya/asic_reg/mc_pll_regs.h       |  186 +
 .../include/goya/asic_reg/mme1_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme2_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme3_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme4_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme5_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme6_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme_cmdq_regs.h     |  431 ++
 .../include/goya/asic_reg/mme_qm_regs.h       |  543 ++
 .../include/goya/asic_reg/mme_regs.h          | 2422 ++++++++
 .../include/goya/asic_reg/mmu_regs.h          |  158 +
 .../include/goya/asic_reg/pci_nrtr_regs.h     |  380 ++
 .../include/goya/asic_reg/pcie_aux_regs.h     |  476 ++
 .../goya/asic_reg/psoc_emmc_pll_regs.h        |  186 +
 .../goya/asic_reg/psoc_global_conf_regs.h     | 1119 ++++
 .../include/goya/asic_reg/psoc_mme_pll_regs.h |  186 +
 .../include/goya/asic_reg/psoc_pci_pll_regs.h |  186 +
 .../include/goya/asic_reg/psoc_spi_regs.h     |  427 ++
 .../goya/asic_reg/sram_y0_x0_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x1_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x2_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x3_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x4_rtr_regs.h       |  215 +
 .../include/goya/asic_reg/stlb_regs.h         |  133 +
 .../include/goya/asic_reg/tpc0_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc0_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc0_eml_cfg_regs.h |  580 ++
 .../include/goya/asic_reg/tpc0_nrtr_regs.h    |  380 ++
 .../include/goya/asic_reg/tpc0_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc1_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc1_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc1_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc1_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc2_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc2_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc2_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc2_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc3_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc3_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc3_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc3_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc4_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc4_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc4_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc4_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc5_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc5_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc5_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc5_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc6_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc6_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc6_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc6_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc7_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc7_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc7_nrtr_regs.h    |  380 ++
 .../include/goya/asic_reg/tpc7_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc_pll_regs.h      |  186 +
 drivers/misc/habanalabs/include/goya/goya.h   |   41 +
 .../include/goya/goya_async_events.h          |  186 +
 .../misc/habanalabs/include/goya/goya_fw_if.h |   28 +
 .../habanalabs/include/goya/goya_packets.h    |  129 +
 drivers/misc/habanalabs/include/hl_boot_if.h  |   30 +
 .../include/hw_ip/mmu/mmu_general.h           |   45 +
 .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h   |   15 +
 drivers/misc/habanalabs/include/qman_if.h     |   56 +
 drivers/misc/habanalabs/irq.c                 |  325 +
 drivers/misc/habanalabs/memory.c              | 1722 ++++++
 drivers/misc/habanalabs/mmu.c                 |  604 ++
 drivers/misc/habanalabs/sysfs.c               |  691 +++
 include/uapi/misc/habanalabs.h                |  444 ++
 115 files changed, 73533 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs
 create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/Kconfig
 create mode 100644 drivers/misc/habanalabs/Makefile
 create mode 100644 drivers/misc/habanalabs/asid.c
 create mode 100644 drivers/misc/habanalabs/command_buffer.c
 create mode 100644 drivers/misc/habanalabs/command_submission.c
 create mode 100644 drivers/misc/habanalabs/context.c
 create mode 100644 drivers/misc/habanalabs/debugfs.c
 create mode 100644 drivers/misc/habanalabs/device.c
 create mode 100644 drivers/misc/habanalabs/goya/Makefile
 create mode 100644 drivers/misc/habanalabs/goya/goya.c
 create mode 100644 drivers/misc/habanalabs/goya/goyaP.h
 create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c
 create mode 100644 drivers/misc/habanalabs/goya/goya_security.c
 create mode 100644 drivers/misc/habanalabs/habanalabs.h
 create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c
 create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c
 create mode 100644 drivers/misc/habanalabs/hw_queue.c
 create mode 100644 drivers/misc/habanalabs/hwmon.c
 create mode 100644 drivers/misc/habanalabs/include/armcp_if.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h
 create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h
 create mode 100644 drivers/misc/habanalabs/include/qman_if.h
 create mode 100644 drivers/misc/habanalabs/irq.c
 create mode 100644 drivers/misc/habanalabs/memory.c
 create mode 100644 drivers/misc/habanalabs/mmu.c
 create mode 100644 drivers/misc/habanalabs/sysfs.c
 create mode 100644 include/uapi/misc/habanalabs.h
Signature · 1 line
2.17.1
Signature · 2 lines
Sincerely yours,
Mike.
#17 lore raw

Re: [PATCH v3 12/15] habanalabs: add virtual memory and MMU modules

In reply to Oded Gabbay
Quoted 50 lines · Oded Gabbay

From: Omer Shpigelman <[email protected]>

This patch adds the Virtual Memory and MMU modules.

Goya has an internal MMU which provides process isolation on the internal DDR. The internal MMU also performs translations for transactions that go from Goya to the Host.

The driver is responsible for allocating and freeing memory on the DDR upon user request. It also provides an interface to map and unmap DDR and Host memory to the device address space.

---
Changes in v3:
- Remove use of three exclamation marks
- Add optimization - support mapping for huge page host memory
- Minor bug fixes

 drivers/misc/habanalabs/Makefile              |    2 +-
 drivers/misc/habanalabs/context.c             |   19 +-
 drivers/misc/habanalabs/device.c              |   20 +-
 drivers/misc/habanalabs/goya/goya.c           |  393 +++++
 drivers/misc/habanalabs/habanalabs.h          |  188 +-
 drivers/misc/habanalabs/habanalabs_drv.c      |    2 +-
 drivers/misc/habanalabs/habanalabs_ioctl.c    |    3 +-
 .../include/hw_ip/mmu/mmu_general.h           |   45 +
 .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h   |   15 +
 drivers/misc/habanalabs/memory.c              | 1514 +++++++++++++++++
 drivers/misc/habanalabs/mmu.c                 |  604 +++++++
 include/uapi/misc/habanalabs.h                |  122 +-
 12 files changed, 2919 insertions(+), 8 deletions(-)
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h
 create mode 100644 drivers/misc/habanalabs/mmu.c
drivers/misc/habanalabs/Makefile +1 -1
index 5167699701b9..2698bb6a2355 100644
@@ -6,7 +6,7 @@ obj-m	:= habanalabs.o
 
 habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
 		command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \
-		command_submission.o
+		command_submission.o mmu.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)

[ ... ]

Quoted 18 lines
drivers/misc/habanalabs/memory.c +4 -0
index 6536b40c63e0..cf0be4f254e0 100644
@@ -5,12 +5,1198 @@
  * All Rights Reserved.
  */
 
+#include <uapi/misc/habanalabs.h>
 #include "habanalabs.h"
+#include "include/hw_ip/mmu/mmu_general.h"
 
 #include <linux/sched.h>
 #include <linux/uaccess.h>
 #include <linux/genalloc.h>
 
+#define PGS_IN_HPAGE   (HPAGE_SIZE >> PAGE_SHIFT)
+#define HL_MMU_DEBUG	0

[ ... ]

Quoted 13 lines
drivers/misc/habanalabs/memory.c +13 -0
+/*
+ * init_phys_pg_pack_from_userptr - initialize physical page pack from host
+ *                                   memory
+ *
+ * @ctx                : current context
+ * @userptr            : userptr to initialize from
+ * @pphys_pg_pack      : res pointer
+ *
+ * This function does the following:
+ * - Pin the physical pages related to the given virtual block
+ * - Create a physical page pack from the physical pages related to the given
+ *   virtual block
+ */

drivers/misc/habanalabs/memory.c line 653

Quoted 10 lines
drivers/misc/habanalabs/memory.c +10 -0
+static int init_phys_pg_pack_from_userptr(struct hl_ctx *ctx,
+		struct hl_userptr *userptr,
+		struct hl_vm_phys_pg_pack **pphys_pg_pack)
+{
+	struct hl_vm_phys_pg_pack *phys_pg_pack;
+	struct scatterlist *sg;
+	dma_addr_t dma_addr;
+	u64 page_mask;
+	u32 npages, total_npages, page_size = PAGE_SIZE;
+	bool first = true, is_2mb_opt = true;

Now you've mentioned 2M pages, I've started to wonder what would happen if PAGE_SIZE != 4k and HPAGE_SIZE != 2M?

There are architectures that may have different sizes for both...

Quoted 82 lines
drivers/misc/habanalabs/memory.c +82 -0
+	int rc, i, j;
+
+	phys_pg_pack = kzalloc(sizeof(*phys_pg_pack), GFP_KERNEL);
+	if (!phys_pg_pack)
+		return -ENOMEM;
+
+	phys_pg_pack->vm_type = userptr->vm_type;
+	phys_pg_pack->created_from_userptr = true;
+	phys_pg_pack->asid = ctx->asid;
+	atomic_set(&phys_pg_pack->mapping_cnt, 1);
+
+	/* Only if all dma_addrs are aligned to 2MB and their
+	 * sizes is at least 2MB, we can use huge page mapping.
+	 * We limit the 2MB optimization to this condition,
+	 * since later on we acquire the related VA range as one
+	 * consecutive block.
+	 */
+	total_npages = 0;
+	for_each_sg(userptr->sgt->sgl, sg, userptr->sgt->nents, i) {
+		npages = get_sg_info(sg, &dma_addr);
+
+		total_npages += npages;
+
+		if (first) {
+			first = false;
+			dma_addr &= HPAGE_MASK;
+		}
+
+		if ((npages % PGS_IN_HPAGE) || (dma_addr & (HPAGE_SIZE - 1)))
+			is_2mb_opt = false;
+	}
+
+	if (is_2mb_opt) {
+		page_size = HPAGE_SIZE;
+		total_npages /= PGS_IN_HPAGE;
+	}
+
+	page_mask = ~(((u64) page_size) - 1);
+
+	phys_pg_pack->pages = kcalloc(total_npages, sizeof(u64), GFP_KERNEL);
+	if (!phys_pg_pack->pages) {
+		rc = -ENOMEM;
+		goto page_pack_arr_mem_err;
+	}
+
+	phys_pg_pack->npages = total_npages;
+	phys_pg_pack->page_size = page_size;
+	phys_pg_pack->total_size = total_npages * page_size;
+
+	j = 0;
+	first = true;
+	for_each_sg(userptr->sgt->sgl, sg, userptr->sgt->nents, i) {
+		npages = get_sg_info(sg, &dma_addr);
+
+		/* align down to physical page size and save the offset */
+		if (first) {
+			first = false;
+			phys_pg_pack->offset = dma_addr & (page_size - 1);
+			dma_addr &= page_mask;
+		}
+
+		while (npages) {
+			phys_pg_pack->pages[j++] = dma_addr;
+			dma_addr += page_size;
+
+			if (is_2mb_opt)
+				npages -= PGS_IN_HPAGE;
+			else
+				npages--;
+		}
+	}
+
+	*pphys_pg_pack = phys_pg_pack;
+
+	return 0;
+
+page_pack_arr_mem_err:
+	kfree(phys_pg_pack);
+
+	return rc;
+}
+

[ ... ]

Quoted 81 lines
drivers/misc/habanalabs/mmu.c new file +75 -0
new file mode 100644
index 000000000000..054b72e48a49
@@ -0,0 +1,604 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Copyright 2016-2018 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ */
+
+#include "habanalabs.h"
+#include "include/hw_ip/mmu/mmu_general.h"
+
+#include <linux/genalloc.h>
+
+#define HOP_NOT_FREED	0
+#define HOP_FREED	1
+
+static struct pgt_info *get_pgt_info(struct hl_ctx *ctx, u64 addr)
+{
+	struct pgt_info *pgt_info = NULL;
+
+	hash_for_each_possible(ctx->mmu_hash, pgt_info, node,
+				(unsigned long) addr)
+		if (addr == pgt_info->addr)
+			break;
+
+	return pgt_info;
+}
+
+static void free_hop(struct hl_ctx *ctx, u64 hop_addr)
+{
+	struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr);
+
+	gen_pool_free(pgt_info->ctx->hdev->mmu_pgt_pool, pgt_info->addr,
+			ctx->hdev->asic_prop.mmu_hop_table_size);
+	hash_del(&pgt_info->node);
+
+	kfree(pgt_info);
+}
+
+static u64 alloc_hop(struct hl_ctx *ctx)
+{
+	struct hl_device *hdev = ctx->hdev;
+	struct pgt_info *pgt_info;
+	u64 addr;
+
+	pgt_info = kmalloc(sizeof(*pgt_info), GFP_KERNEL);
+	if (!pgt_info)
+		return ULLONG_MAX;
+
+	addr = (u64) gen_pool_alloc(hdev->mmu_pgt_pool,
+			hdev->asic_prop.mmu_hop_table_size);
+	if (!addr) {
+		dev_err(hdev->dev, "failed to allocate page\n");
+		kfree(pgt_info);
+		return ULLONG_MAX;
+	}
+
+	pgt_info->addr = addr;
+	pgt_info->ctx = ctx;
+	pgt_info->num_of_ptes = 0;
+	hash_add(ctx->mmu_hash, &pgt_info->node, addr);
+
+	return addr;
+}
+
+static inline void clear_pte(struct hl_device *hdev, u64 pte_addr)
+{
+	/* clear the last and present bits */
+	hdev->asic_funcs->write_pte(hdev, pte_addr, 0);
+}
+
+static inline void inc_num_of_ptes(struct hl_ctx *ctx, u64 hop_addr)
+{
+	get_pgt_info(ctx, hop_addr)->num_of_ptes++;
+}
+

drivers/misc/habanalabs/mmu.c line 85

Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10 -0
+/*
+ * dec_num_of_ptes - decrement the num of ptes and free the hop if possible
+ *
+ * @ctx: pointer to the context structure
+ * @hop_addr: addr of the hop
+ *
+ * This function returns HOP_FREED if the hop was freed or HOP_NOT_FREED if the
+ * num of ptes was decremented without freeing the hop.
+ */
+static inline int dec_num_of_ptes(struct hl_ctx *ctx, u64 hop_addr)

I think it's worth emphasizing that this function may free a pte. Maybe even use {get,put}_pte instead of {inc,dec}_num_of_ptes.

drivers/misc/habanalabs/mmu.c line 96

Quoted 11 lines
drivers/misc/habanalabs/mmu.c new file +11 -0
+{
+	struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr);
+
+	pgt_info->num_of_ptes--;
+
+	if (pgt_info->num_of_ptes == 0) {
+		free_hop(ctx, hop_addr);
+		return HOP_FREED;
+	}
+
+	return HOP_NOT_FREED;

Why not simply return pgt_info->num_of_ptes?

Quoted 32 lines
drivers/misc/habanalabs/mmu.c new file +32 -0
+}
+
+static inline u64 get_hop0_addr(struct hl_ctx *ctx)
+{
+	return ctx->hdev->asic_prop.mmu_pgt_addr +
+			(ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size);
+}
+
+static inline u64 get_hop0_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+					u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP0_MASK) >> HOP0_SHIFT);
+}
+
+static inline u64 get_hop1_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+					u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP1_MASK) >> HOP1_SHIFT);
+}
+
+static inline u64 get_hop2_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+					u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP2_MASK) >> HOP2_SHIFT);
+}
+
+static inline u64 get_hop3_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+					u64 virt_addr)
+{

drivers/misc/habanalabs/mmu.c line 138

Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10 -0
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP3_MASK) >> HOP3_SHIFT);
+}
+
+static inline u64 get_hop4_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+					u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP4_MASK) >> HOP4_SHIFT);
+}

These are almost clones of each other. I'd factor out

static inline u64 get_hopN_pte(struct hl_ctx *ctx, u64 hop_addr,

				u64 vaddr, u64 mask, u64 shift);

and alias it for each level.

Quoted 183 lines
drivers/misc/habanalabs/mmu.c new file +183 -0
+static inline u64 get_next_hop_addr(u64 curr_pte)
+{
+	if (curr_pte & PAGE_PRESENT_MASK)
+		return curr_pte & PHYS_ADDR_MASK;
+	else
+		return ULLONG_MAX;
+}
+
+/*
+ * hl_mmu_init - init the mmu module
+ *
+ * @hdev: pointer to the habanalabs device structure
+ *
+ * This function does the following:
+ * - Allocate max_asid zeroed hop0 pgts so no mapping is available
+ * - Enable mmu in hw
+ * - Invalidate the mmu cache
+ * - Create a pool of pages for pgts
+ * - Returns 0 on success
+ *
+ * This function depends on DMA QMAN to be working!
+ */
+int hl_mmu_init(struct hl_device *hdev)
+{
+	struct asic_fixed_properties *prop = &hdev->asic_prop;
+	int rc;
+
+	if (!hdev->mmu_enable)
+		return 0;
+
+	/* MMU HW init was already done in device hw_init() */
+
+	mutex_init(&hdev->mmu_cache_lock);
+
+	hdev->mmu_pgt_pool =
+			gen_pool_create(__ffs(prop->mmu_hop_table_size), -1);
+
+	if (!hdev->mmu_pgt_pool) {
+		dev_err(hdev->dev, "Failed to create page gen pool\n");
+		rc = -ENOMEM;
+		goto err_pool_create;
+	}
+
+	rc = gen_pool_add(hdev->mmu_pgt_pool, prop->mmu_pgt_addr +
+			prop->mmu_hop0_tables_total_size,
+			prop->mmu_pgt_size - prop->mmu_hop0_tables_total_size,
+			-1);
+	if (rc) {
+		dev_err(hdev->dev, "Failed to add memory to page gen pool\n");
+		goto err_pool_add;
+	}
+
+	return 0;
+
+err_pool_add:
+	gen_pool_destroy(hdev->mmu_pgt_pool);
+err_pool_create:
+	mutex_destroy(&hdev->mmu_cache_lock);
+
+	return rc;
+}
+
+/*
+ * hl_mmu_fini - release the mmu module.
+ *
+ * @hdev: pointer to the habanalabs device structure
+ *
+ * This function does the following:
+ * - Disable mmu in hw
+ * - free the pgts pool
+ *
+ * All ctxs should be freed before calling this func
+ */
+void hl_mmu_fini(struct hl_device *hdev)
+{
+	if (!hdev->mmu_enable)
+		return;
+
+	gen_pool_destroy(hdev->mmu_pgt_pool);
+
+	mutex_destroy(&hdev->mmu_cache_lock);
+
+	/* MMU HW fini will be done in device hw_fini() */
+}
+
+/*
+ * hl_mmu_ctx_init - init a ctx for using the mmu module
+ *
+ * @ctx: pointer to the context structure
+ *
+ * This function does the following:
+ * - Init a mutex to protect the concurrent mapping flow
+ * - Init a hash to hold all pgts related to this ctx
+ */
+void hl_mmu_ctx_init(struct hl_ctx *ctx)
+{
+	if (!ctx->hdev->mmu_enable)
+		return;
+
+	mutex_init(&ctx->mmu_lock);
+	hash_init(ctx->mmu_hash);
+}
+
+/*
+ * hl_mmu_ctx_fini - disable a ctx from using the mmu module
+ *
+ * @ctx: pointer to the context structure
+ *
+ * This function does the following:
+ * - Free any pgts which were not freed yet
+ * - Free the mutex
+ */
+void hl_mmu_ctx_fini(struct hl_ctx *ctx)
+{
+	struct pgt_info *pgt_info;
+	struct hlist_node *tmp;
+	int i;
+
+	if (!ctx->hdev->mmu_enable)
+		return;
+
+	if (!hash_empty(ctx->mmu_hash))
+		dev_err(ctx->hdev->dev,
+				"ctx is freed while it has pgts in use\n");
+
+	hash_for_each_safe(ctx->mmu_hash, i, tmp, pgt_info, node) {
+		dev_err(ctx->hdev->dev,
+			"pgt_info of addr 0x%llx of asid %d was not destroyed, num_ptes: %d\n",
+			pgt_info->addr, ctx->asid, pgt_info->num_of_ptes);
+		free_hop(ctx, pgt_info->addr);
+	}
+
+	mutex_destroy(&ctx->mmu_lock);
+}
+
+/*
+ * hl_mmu_map - maps a virtual addr to physical addr
+ *
+ * @ctx: pointer to the context structure
+ * @virt_addr: virt addr to map from
+ * @phys_addr: phys addr to map to
+ * @page_size: physical page size
+ *
+ * This function does the following:
+ * - Check that the virt addr is not mapped
+ * - Allocate pgts as necessary in order to map the virt addr to the phys
+ * - Returns 0 on success, -EINVAL if addr is already mapped, or -ENOMEM.
+ *
+ * Because this function changes the page tables in the device and because it
+ * changes the MMU hash, it must be protected by a lock.
+ * However, because it maps only a single page, the lock should be implemented
+ * in a higher level in order to protect the entire mapping of the memory area
+ */
+int hl_mmu_map(struct hl_ctx *ctx, u64 virt_addr, u64 phys_addr, u32 page_size)
+{
+	struct hl_device *hdev = ctx->hdev;
+	u64 hop0_addr = 0, hop0_pte_addr = 0,
+		hop1_addr = 0, hop1_pte_addr = 0,
+		hop2_addr = 0, hop2_pte_addr = 0,
+		hop3_addr = 0, hop3_pte_addr = 0,
+		hop4_addr = 0, hop4_pte_addr = 0,
+		curr_pte = 0;
+	int hop1_new = 0, hop2_new = 0, hop3_new = 0, hop4_new = 0,
+			rc = -ENOMEM;
+	bool is_huge;
+
+	if (!hdev->mmu_enable)
+		return 0;
+
+	/*
+	 * This mapping function can map a 4KB/2MB page. For 2MB page there are
+	 * only 3 hops rather than 4. Currently the DRAM allocation uses 2MB
+	 * pages only but user memory could have been allocated with one of the
+	 * two page sizes. Since this is a common code for all the three cases,
+	 * we need this hugs page check.
+	 */
+	is_huge = page_size == PAGE_SIZE_2MB;
+
+	hop0_addr = get_hop0_addr(ctx);
+
+	hop0_pte_addr = get_hop0_pte_addr(ctx, hop0_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop0_pte_addr);

drivers/misc/habanalabs/mmu.c line 332

Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10 -0
+
+	hop1_addr = get_next_hop_addr(curr_pte);
+
+	if (hop1_addr == ULLONG_MAX) {
+		hop1_addr = alloc_hop(ctx);
+		if (hop1_addr == ULLONG_MAX)
+			goto err;
+		else
+			hop1_new = 1;
+	}

The allocation can be folded into get_alloc_next_hop_addr() along with get_next_hop_addr().

Quoted 107 lines
drivers/misc/habanalabs/mmu.c new file +107 -0
+
+	hop1_pte_addr = get_hop1_pte_addr(ctx, hop1_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop1_pte_addr);
+
+	hop2_addr = get_next_hop_addr(curr_pte);
+
+	if (hop2_addr == ULLONG_MAX) {
+		hop2_addr = alloc_hop(ctx);
+		if (hop2_addr == ULLONG_MAX)
+			goto err;
+		else
+			hop2_new = 1;
+	}
+
+	hop2_pte_addr = get_hop2_pte_addr(ctx, hop2_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop2_pte_addr);
+
+	hop3_addr = get_next_hop_addr(curr_pte);
+
+	if (hop3_addr == ULLONG_MAX) {
+		hop3_addr = alloc_hop(ctx);
+		if (hop3_addr == ULLONG_MAX)
+			goto err;
+		else
+			hop3_new = 1;
+	}
+
+	hop3_pte_addr = get_hop3_pte_addr(ctx, hop3_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop3_pte_addr);
+
+	if (!is_huge) {
+		hop4_addr = get_next_hop_addr(curr_pte);
+
+		if (hop4_addr == ULLONG_MAX) {
+			hop4_addr = alloc_hop(ctx);
+			if (hop4_addr == ULLONG_MAX)
+				goto err;
+			else
+				hop4_new = 1;
+		}
+
+		hop4_pte_addr = get_hop4_pte_addr(ctx, hop4_addr, virt_addr);
+
+		curr_pte = hdev->asic_funcs->read_pte(hdev, hop4_pte_addr);
+	}
+
+	if (curr_pte & PAGE_PRESENT_MASK) {
+		dev_err(hdev->dev,
+				"mapping already exists for virt_addr 0x%llx\n",
+					virt_addr);
+
+		dev_dbg(hdev->dev, "hop0 pte: 0x%llx (0x%llx)\n",
+				hdev->asic_funcs->read_pte(hdev, hop0_pte_addr),
+				hop0_pte_addr);
+		dev_dbg(hdev->dev, "hop1 pte: 0x%llx (0x%llx)\n",
+				hdev->asic_funcs->read_pte(hdev, hop1_pte_addr),
+				hop1_pte_addr);
+		dev_dbg(hdev->dev, "hop2 pte: 0x%llx (0x%llx)\n",
+				hdev->asic_funcs->read_pte(hdev, hop2_pte_addr),
+				hop2_pte_addr);
+		dev_dbg(hdev->dev, "hop3 pte: 0x%llx (0x%llx)\n",
+				hdev->asic_funcs->read_pte(hdev, hop3_pte_addr),
+				hop3_pte_addr);
+
+		if (!is_huge)
+			dev_dbg(hdev->dev, "hop4 pte: 0x%llx (0x%llx)\n",
+				hdev->asic_funcs->read_pte(hdev,
+							hop4_pte_addr),
+							hop4_pte_addr);
+
+		rc = EINVAL;
+		goto err;
+	}
+
+	curr_pte = (phys_addr & PTE_PHYS_ADDR_MASK) | LAST_MASK
+			| PAGE_PRESENT_MASK;
+
+	hdev->asic_funcs->write_pte(hdev,
+				is_huge ? hop3_pte_addr : hop4_pte_addr,
+				curr_pte);
+
+	if (hop1_new) {
+		curr_pte = (hop1_addr & PTE_PHYS_ADDR_MASK) |
+				PAGE_PRESENT_MASK;
+		ctx->hdev->asic_funcs->write_pte(ctx->hdev, hop0_pte_addr,
+				curr_pte);
+	}
+	if (hop2_new) {
+		curr_pte = (hop2_addr & PTE_PHYS_ADDR_MASK) |
+				PAGE_PRESENT_MASK;
+		ctx->hdev->asic_funcs->write_pte(ctx->hdev, hop1_pte_addr,
+				curr_pte);
+		inc_num_of_ptes(ctx, hop1_addr);
+	}
+	if (hop3_new) {
+		curr_pte = (hop3_addr & PTE_PHYS_ADDR_MASK) |
+				PAGE_PRESENT_MASK;
+		ctx->hdev->asic_funcs->write_pte(ctx->hdev, hop2_pte_addr,
+				curr_pte);
+		inc_num_of_ptes(ctx, hop2_addr);
+	}
+
+	if (!is_huge) {
+		if (hop4_new) {

drivers/misc/habanalabs/mmu.c line 449

Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10 -0
+			curr_pte = (hop4_addr & PTE_PHYS_ADDR_MASK) |
+					PAGE_PRESENT_MASK;
+			ctx->hdev->asic_funcs->write_pte(ctx->hdev,
+					hop3_pte_addr, curr_pte);
+			inc_num_of_ptes(ctx, hop3_addr);
+		}
+
+		inc_num_of_ptes(ctx, hop4_addr);
+	} else
+		inc_num_of_ptes(ctx, hop3_addr);

Isn't hop3 would be incremented twice for 2M pages?

Quoted 108 lines
drivers/misc/habanalabs/mmu.c new file +108 -0
+
+	/* flush all writes from all cores to reach PCI */
+	mb();
+
+	hdev->asic_funcs->read_pte(hdev,
+				is_huge ? hop3_pte_addr : hop4_pte_addr);
+
+	return 0;
+
+err:
+	if (hop4_new)
+		free_hop(ctx, hop4_addr);
+	if (hop3_new)
+		free_hop(ctx, hop3_addr);
+	if (hop2_new)
+		free_hop(ctx, hop2_addr);
+	if (hop1_new)
+		free_hop(ctx, hop1_addr);
+
+	return rc;
+}
+
+/*
+ * hl_mmu_unmap - unmaps a virtual addr
+ *
+ * @ctx: pointer to the context structure
+ * @virt_addr: virt addr to map from
+ *
+ * This function does the following:
+ * - Check that the virt addr is mapped
+ * - Unmap the vurt addr and frees pgts if possible
+ * - Returns 0 on success, -EINVAL if the given addr is not mapped
+ *
+ * Because this function changes the page tables in the device and because it
+ * changes the MMU hash, it must be protected by a lock.
+ * However, because it maps only a single page, the lock should be implemented
+ * in a higher level in order to protect the entire mapping of the memory area
+ */
+int hl_mmu_unmap(struct hl_ctx *ctx, u64 virt_addr)
+{
+	struct hl_device *hdev = ctx->hdev;
+	u64 hop0_addr = 0, hop0_pte_addr = 0,
+		hop1_addr = 0, hop1_pte_addr = 0,
+		hop2_addr = 0, hop2_pte_addr = 0,
+		hop3_addr = 0, hop3_pte_addr = 0,
+		hop4_addr = 0, hop4_pte_addr = 0,
+		curr_pte;
+	int clear_hop3 = 1;
+
+	if (!hdev->mmu_enable)
+		return 0;
+
+	hop0_addr = get_hop0_addr(ctx);
+
+	hop0_pte_addr = get_hop0_pte_addr(ctx, hop0_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop0_pte_addr);
+
+	hop1_addr = get_next_hop_addr(curr_pte);
+
+	if (hop1_addr == ULLONG_MAX)
+		goto not_mapped;
+
+	hop1_pte_addr = get_hop1_pte_addr(ctx, hop1_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop1_pte_addr);
+
+	hop2_addr = get_next_hop_addr(curr_pte);
+
+	if (hop2_addr == ULLONG_MAX)
+		goto not_mapped;
+
+	hop2_pte_addr = get_hop2_pte_addr(ctx, hop2_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop2_pte_addr);
+
+	hop3_addr = get_next_hop_addr(curr_pte);
+
+	if (hop3_addr == ULLONG_MAX)
+		goto not_mapped;
+
+	hop3_pte_addr = get_hop3_pte_addr(ctx, hop3_addr, virt_addr);
+
+	curr_pte = hdev->asic_funcs->read_pte(hdev, hop3_pte_addr);
+
+	if (!(curr_pte & LAST_MASK)) {
+		hop4_addr = get_next_hop_addr(curr_pte);
+
+		if (hop4_addr == ULLONG_MAX)
+			goto not_mapped;
+
+		hop4_pte_addr = get_hop4_pte_addr(ctx, hop4_addr, virt_addr);
+
+		curr_pte = hdev->asic_funcs->read_pte(hdev, hop4_pte_addr);
+
+		clear_hop3 = 0;
+	}
+
+	if (!(curr_pte & PAGE_PRESENT_MASK))
+		goto not_mapped;
+
+	clear_pte(hdev, hop4_addr ? hop4_pte_addr : hop3_pte_addr);
+
+	if (hop4_addr && dec_num_of_ptes(ctx, hop4_addr) == HOP_FREED)
+		clear_hop3 = 1;
+
+	if (clear_hop3) {
+		clear_pte(hdev, hop3_pte_addr);

drivers/misc/habanalabs/mmu.c line 567

Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10 -0
+		if (dec_num_of_ptes(ctx, hop3_addr) == HOP_FREED) {
+			clear_pte(hdev, hop2_pte_addr);
+			if (dec_num_of_ptes(ctx, hop2_addr) == HOP_FREED) {
+				clear_pte(hdev, hop1_pte_addr);
+				if (dec_num_of_ptes(ctx, hop1_addr) ==
+						HOP_FREED)
+					clear_pte(hdev, hop0_pte_addr);
+			}
+		}
+	}

Consider inverting the logic and doing something like

	if (dec_num_of_ptes() != HOP_FREED)
		goto flush;
	clear_pte(...)
	if (...)
		goto flush;

flush:

Quoted 15 lines
drivers/misc/habanalabs/mmu.c new file +15 -0
+
+	/* flush all writes from all cores to reach PCI */
+	mb();
+
+	hdev->asic_funcs->read_pte(hdev,
+				hop4_addr ? hop4_pte_addr : hop3_pte_addr);
+
+	return 0;
+
+not_mapped:
+	dev_err(hdev->dev, "virt addr 0x%llx is not mapped to phys addr\n",
+		virt_addr);
+
+	return -EINVAL;
+}

[ ... ]

Quoted 3 lines
Signature · 1 line
2.17.1
Signature · 2 lines
Sincerely yours,
Mike.
Reviewed-by on v3 14/15
#21 lore raw

Re: [PATCH v3 14/15] habanalabs: add debugfs support

In reply to Oded Gabbay
Quoted 11 lines · Oded Gabbay

This patch adds debugfs support to the driver. It allows the user-space to display information that is contained in the internal structures of the driver, such as:
- active command submissions
- active user virtual memory mappings
- number of allocated command buffers

It also enables the user to perform reads and writes through Goya's PCI bars.

Quoted 1,913 lines

---
Changes in v3:
- Adjust code to huge page optimization

 .../ABI/testing/debugfs-driver-habanalabs     |  127 ++
 drivers/misc/habanalabs/Makefile              |    2 +
 drivers/misc/habanalabs/command_buffer.c      |    4 +
 drivers/misc/habanalabs/command_submission.c  |   12 +
 drivers/misc/habanalabs/debugfs.c             | 1071 +++++++++++++++++
 drivers/misc/habanalabs/device.c              |    6 +
 drivers/misc/habanalabs/goya/goya.c           |  108 ++
 drivers/misc/habanalabs/goya/goyaP.h          |    5 +
 drivers/misc/habanalabs/habanalabs.h          |  190 +++
 drivers/misc/habanalabs/habanalabs_drv.c      |   16 +-
 drivers/misc/habanalabs/memory.c              |    8 +
 11 files changed, 1547 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/debugfs.c
Documentation/ABI/testing/debugfs-driver-habanalabs new file +127 -0
new file mode 100644
index 000000000000..2b606c84938c
@@ -0,0 +1,127 @@
+What:           /sys/kernel/debug/habanalabs/hl<n>/addr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the device address to be used for read or write through
+                PCI bar. The acceptable value is a string that starts with "0x"
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/command_buffers
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about the currently allocated
+                command buffers
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/command_submission
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about the currently active
+                command submissions
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/command_submission_jobs
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with detailed information about each JOB (CB) of
+                each active command submission
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/data32
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Allows the root user to read or write directly through the
+                device's PCI bar. Writing to this file generates a write
+                transaction while reading from the file generates a read
+                transcation. This custom interface is needed (instead of using
+                the generic Linux user-space PCI mapping) because the DDR bar
+                is very small compared to the DDR memory and only the driver can
+                move the bar before and after the transaction
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/device
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Enables the root user to set the device to specific state.
+                Valid values are "disable", "enable", "suspend", "resume".
+                User can read this property to see the valid values
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_addr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets I2C device address for I2C transaction that is generated
+                by the device's CPU
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_bus
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets I2C bus address for I2C transaction that is generated by
+                the device's CPU
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_data
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Triggers an I2C transaction that is generated by the device's
+                CPU. Writing to this file generates a write transaction while
+                reading from the file generates a read transcation
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/i2c_reg
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets I2C register id for I2C transaction that is generated by
+                the device's CPU
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/led0
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the state of the first S/W led on the device
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/led1
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the state of the second S/W led on the device
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/led2
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the state of the third S/W led on the device
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/mmu
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays the hop values and physical address for a given ASID
+                and virtual address. The user should write the ASID and VA into
+                the file and then read the file to get the result.
+                e.g. to display info about VA 0x1000 for ASID 1 you need to do:
+                echo "1 0x1000" > /sys/kernel/debug/habanalabs/hl0/mmu
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/set_power_state
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Sets the PCI power state. Valid values are "1" for D0 and "2"
+                for D3Hot
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/userptr
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about the currently user
+                pointers (user virtual addresses) that are pinned and mapped
+                to DMA addresses
+
+What:           /sys/kernel/debug/habanalabs/hl<n>/vm
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays a list with information about all the active virtual
+                address mappings per ASID
+
drivers/misc/habanalabs/Makefile +2 -0
index 2698bb6a2355..625046487aa0 100644
@@ -8,6 +8,8 @@ habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
 		command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \
 		command_submission.o mmu.o
 
+habanalabs-$(CONFIG_DEBUG_FS) += debugfs.o
+
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)
 
drivers/misc/habanalabs/command_buffer.c +4 -0
index c70b548a7b8a..607f6abc7f3f 100644
@@ -36,6 +36,8 @@ static void cb_release(struct kref *ref)
 	cb = container_of(ref, struct hl_cb, refcount);
 	hdev = cb->hdev;
 
+	hl_debugfs_remove_cb(cb);
+
 	cb_do_release(hdev, cb);
 }
 
@@ -153,6 +155,8 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr,
 	*handle = cb->id | HL_MMAP_CB_MASK;
 	*handle <<= PAGE_SHIFT;
 
+	hl_debugfs_add_cb(cb);
+
 	return 0;
 
 release_cb:
drivers/misc/habanalabs/command_submission.c +12 -0
index 288dddc14873..e539ac0a4ca3 100644
@@ -153,6 +153,8 @@ static void free_job(struct hl_device *hdev, struct hl_cs_job *job)
 	list_del(&job->cs_node);
 	spin_unlock(&cs->job_lock);
 
+	hl_debugfs_remove_job(hdev, job);
+
 	if (job->ext_queue)
 		cs_put(cs);
 
@@ -216,6 +218,12 @@ static void cs_do_release(struct kref *ref)
 		}
 	}
 
+	/*
+	 * Must be called before hl_ctx_put because inside we use ctx to get
+	 * the device
+	 */
+	hl_debugfs_remove_cs(cs);
+
 	hl_ctx_put(cs->ctx);
 
 	if (cs->timedout)
@@ -484,6 +492,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks,
 
 	*cs_seq = cs->sequence;
 
+	hl_debugfs_add_cs(cs);
+
 	/* Validate ALL the CS chunks before submitting the CS */
 	for (i = 0, parse_cnt = 0 ; i < num_chunks ; i++, parse_cnt++) {
 		struct hl_cs_chunk *chunk = &cs_chunk_array[i];
@@ -532,6 +542,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks,
 		if (job->ext_queue)
 			cs_get(cs);
 
+		hl_debugfs_add_job(hdev, job);
+
 		rc = cs_parser(hpriv, job);
 		if (rc) {
 			dev_err(hdev->dev,
drivers/misc/habanalabs/debugfs.c new file +1071 -0
new file mode 100644
index 000000000000..f7f2c1e14dfa
@@ -0,0 +1,1071 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Copyright 2016-2018 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ */
+
+#include "habanalabs.h"
+#include "include/hw_ip/mmu/mmu_general.h"
+
+#include <linux/debugfs.h>
+#include <linux/uaccess.h>
+
+#define MMU_ADDR_BUF_SIZE	40
+#define MMU_ASID_BUF_SIZE	10
+#define MMU_KBUF_SIZE		(MMU_ADDR_BUF_SIZE + MMU_ASID_BUF_SIZE)
+
+static struct dentry *hl_debug_root;
+
+static int hl_debugfs_i2c_read(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr,
+				u8 i2c_reg, u32 *val)
+{
+	struct armcp_packet pkt;
+	int rc;
+
+	if (hl_device_disabled_or_in_reset(hdev))
+		return 0;
+
+	memset(&pkt, 0, sizeof(pkt));
+
+	pkt.ctl = ARMCP_PACKET_I2C_RD << ARMCP_PKT_CTL_OPCODE_SHIFT;
+	pkt.i2c_bus = i2c_bus;
+	pkt.i2c_addr = i2c_addr;
+	pkt.i2c_reg = i2c_reg;
+
+	rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt),
+					HL_DEVICE_TIMEOUT_USEC, (long *) val);
+
+	if (rc)
+		dev_err(hdev->dev, "Failed to read from I2C, error %d\n", rc);
+
+	return rc;
+}
+
+static int hl_debugfs_i2c_write(struct hl_device *hdev, u8 i2c_bus, u8 i2c_addr,
+				u8 i2c_reg, u32 val)
+{
+	struct armcp_packet pkt;
+	int rc;
+
+	if (hl_device_disabled_or_in_reset(hdev))
+		return 0;
+
+	memset(&pkt, 0, sizeof(pkt));
+
+	pkt.ctl = ARMCP_PACKET_I2C_WR << ARMCP_PKT_CTL_OPCODE_SHIFT;
+	pkt.i2c_bus = i2c_bus;
+	pkt.i2c_addr = i2c_addr;
+	pkt.i2c_reg = i2c_reg;
+	pkt.value = val;
+
+	rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt),
+					HL_DEVICE_TIMEOUT_USEC, NULL);
+
+	if (rc)
+		dev_err(hdev->dev, "Failed to write to I2C, error %d\n", rc);
+
+	return rc;
+}
+
+static void hl_debugfs_led_set(struct hl_device *hdev, u8 led, u8 state)
+{
+	struct armcp_packet pkt;
+	int rc;
+
+	if (hl_device_disabled_or_in_reset(hdev))
+		return;
+
+	memset(&pkt, 0, sizeof(pkt));
+
+	pkt.ctl = ARMCP_PACKET_LED_SET << ARMCP_PKT_CTL_OPCODE_SHIFT;
+	pkt.led_index = led;
+	pkt.value = state;
+
+	rc = hdev->asic_funcs->send_cpu_message(hdev, (u32 *) &pkt, sizeof(pkt),
+						HL_DEVICE_TIMEOUT_USEC, NULL);
+
+	if (rc)
+		dev_err(hdev->dev, "Failed to set LED %d, error %d\n", led, rc);
+}
+
+static int command_buffers_show(struct seq_file *s, void *data)
+{
+	struct hl_debugfs_entry *entry = s->private;
+	struct hl_dbg_device_entry *dev_entry = entry->dev_entry;
+	struct hl_cb *cb;
+	bool first = true;
+
+	spin_lock(&dev_entry->cb_spinlock);
+
+	list_for_each_entry(cb, &dev_entry->cb_list, debugfs_list) {
+		if (first) {
+			first = false;
+			seq_puts(s, "\n");
+			seq_puts(s, " CB ID   CTX ID   CB size    CB RefCnt    mmap?   CS counter\n");
+			seq_puts(s, "---------------------------------------------------------------\n");
+		}
+		seq_printf(s,
+			"   %03d        %d    0x%08x      %d          %d          %d\n",
+			cb->id, cb->ctx_id, cb->size,
+			kref_read(&cb->refcount),
+			cb->mmap, cb->cs_cnt);
+	}
+
+	spin_unlock(&dev_entry->cb_spinlock);
+
+	if (!first)
+		seq_puts(s, "\n");
+
+	return 0;
+}
+
+static int command_submission_show(struct seq_file *s, void *data)
+{
+	struct hl_debugfs_entry *entry = s->private;
+	struct hl_dbg_device_entry *dev_entry = entry->dev_entry;
+	struct hl_cs *cs;
+	bool first = true;
+
+	spin_lock(&dev_entry->cs_spinlock);
+
+	list_for_each_entry(cs, &dev_entry->cs_list, debugfs_list) {
+		if (first) {
+			first = false;
+			seq_puts(s, "\n");
+			seq_puts(s, " CS ID   CTX ASID   CS RefCnt   Submitted    Completed\n");
+			seq_puts(s, "------------------------------------------------------\n");
+		}
+		seq_printf(s,
+			"   %llu       %d          %d           %d            %d\n",
+			cs->sequence, cs->ctx->asid,
+			kref_read(&cs->refcount),
+			cs->submitted, cs->completed);
+	}
+
+	spin_unlock(&dev_entry->cs_spinlock);
+
+	if (!first)
+		seq_puts(s, "\n");
+
+	return 0;
+}
+
+static int command_submission_jobs_show(struct seq_file *s, void *data)
+{
+	struct hl_debugfs_entry *entry = s->private;
+	struct hl_dbg_device_entry *dev_entry = entry->dev_entry;
+	struct hl_cs_job *job;
+	bool first = true;
+
+	spin_lock(&dev_entry->cs_job_spinlock);
+
+	list_for_each_entry(job, &dev_entry->cs_job_list, debugfs_list) {
+		if (first) {
+			first = false;
+			seq_puts(s, "\n");
+			seq_puts(s, " JOB ID   CS ID    CTX ASID   H/W Queue\n");
+			seq_puts(s, "---------------------------------------\n");
+		}
+		if (job->cs)
+			seq_printf(s,
+				"    %02d       %llu         %d         %d\n",
+				job->id, job->cs->sequence, job->cs->ctx->asid,
+				job->hw_queue_id);
+		else
+			seq_printf(s,
+				"    %02d       0         %d         %d\n",
+				job->id, HL_KERNEL_ASID_ID, job->hw_queue_id);
+	}
+
+	spin_unlock(&dev_entry->cs_job_spinlock);
+
+	if (!first)
+		seq_puts(s, "\n");
+
+	return 0;
+}
+
+static int userptr_show(struct seq_file *s, void *data)
+{
+	struct hl_debugfs_entry *entry = s->private;
+	struct hl_dbg_device_entry *dev_entry = entry->dev_entry;
+	struct hl_userptr *userptr;
+	char dma_dir[4][30] = {"DMA_BIDIRECTIONAL", "DMA_TO_DEVICE",
+				"DMA_FROM_DEVICE", "DMA_NONE"};
+	bool first = true;
+
+	spin_lock(&dev_entry->userptr_spinlock);
+
+	list_for_each_entry(userptr, &dev_entry->userptr_list, debugfs_list) {
+		if (first) {
+			first = false;
+			seq_puts(s, "\n");
+			seq_puts(s, " user virtual address     size             dma dir\n");
+			seq_puts(s, "----------------------------------------------------------\n");
+		}
+		seq_printf(s,
+			"    0x%-14llx      %-10u    %-30s\n",
+			userptr->addr, userptr->size, dma_dir[userptr->dir]);
+	}
+
+	spin_unlock(&dev_entry->userptr_spinlock);
+
+	if (!first)
+		seq_puts(s, "\n");
+
+	return 0;
+}
+
+static int vm_show(struct seq_file *s, void *data)
+{
+	struct hl_debugfs_entry *entry = s->private;
+	struct hl_dbg_device_entry *dev_entry = entry->dev_entry;
+	struct hl_ctx *ctx;
+	struct hl_vm *vm;
+	struct hl_vm_hash_node *hnode;
+	struct hl_userptr *userptr;
+	struct hl_vm_phys_pg_pack *phys_pg_pack = NULL;
+	enum vm_type_t *vm_type;
+	bool once = true;
+	int i;
+
+	if (!dev_entry->hdev->mmu_enable)
+		return 0;
+
+	spin_lock(&dev_entry->ctx_mem_hash_spinlock);
+
+	list_for_each_entry(ctx, &dev_entry->ctx_mem_hash_list, debugfs_list) {
+		once = false;
+		seq_puts(s, "\n\n----------------------------------------------------");
+		seq_puts(s, "\n----------------------------------------------------\n\n");
+		seq_printf(s, "ctx asid: %u\n", ctx->asid);
+
+		seq_puts(s, "\nmappings:\n\n");
+		seq_puts(s, "    virtual address        size          handle\n");
+		seq_puts(s, "----------------------------------------------------\n");
+		mutex_lock(&ctx->mem_hash_lock);
+		hash_for_each(ctx->mem_hash, i, hnode, node) {
+			vm_type = hnode->ptr;
+
+			if (*vm_type == VM_TYPE_USERPTR) {
+				userptr = hnode->ptr;
+				seq_printf(s,
+					"    0x%-14llx      %-10u\n",
+					hnode->vaddr, userptr->size);
+			} else {
+				phys_pg_pack = hnode->ptr;
+				seq_printf(s,
+					"    0x%-14llx      %-10u       %-4u\n",
+					hnode->vaddr, phys_pg_pack->total_size,
+					phys_pg_pack->handle);
+			}
+		}
+		mutex_unlock(&ctx->mem_hash_lock);
+
+		vm = &ctx->hdev->vm;
+		spin_lock(&vm->idr_lock);
+
+		if (!idr_is_empty(&vm->phys_pg_pack_handles))
+			seq_puts(s, "\n\nallocations:\n");
+
+		idr_for_each_entry(&vm->phys_pg_pack_handles, phys_pg_pack, i) {
+			if (phys_pg_pack->asid != ctx->asid)
+				continue;
+
+			seq_printf(s, "\nhandle: %u\n", phys_pg_pack->handle);
+			seq_printf(s, "page size: %u\n\n",
+						phys_pg_pack->page_size);
+			seq_puts(s, "   physical address\n");
+			seq_puts(s, "---------------------\n");
+			for (i = 0 ; i < phys_pg_pack->npages ; i++) {
+				seq_printf(s, "    0x%-14llx\n",
+						phys_pg_pack->pages[i]);
+			}
+		}
+		spin_unlock(&vm->idr_lock);
+
+	}
+
+	spin_unlock(&dev_entry->ctx_mem_hash_spinlock);
+
+	if (!once)
+		seq_puts(s, "\n");
+
+	return 0;
+}
+
+/* these inline functions are copied from mmu.c */
+static inline u64 get_hop0_addr(struct hl_ctx *ctx)
+{
+	return ctx->hdev->asic_prop.mmu_pgt_addr +
+			(ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size);
+}
+
+static inline u64 get_hop0_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+		u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP0_MASK) >> HOP0_SHIFT);
+}
+
+static inline u64 get_hop1_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+		u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP1_MASK) >> HOP1_SHIFT);
+}
+
+static inline u64 get_hop2_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+		u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP2_MASK) >> HOP2_SHIFT);
+}
+
+static inline u64 get_hop3_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+		u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP3_MASK) >> HOP3_SHIFT);
+}
+
+static inline u64 get_hop4_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+		u64 virt_addr)
+{
+	return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+			((virt_addr & HOP4_MASK) >> HOP4_SHIFT);
+}
+
+static inline u64 get_next_hop_addr(u64 curr_pte)
+{
+	if (curr_pte & PAGE_PRESENT_MASK)
+		return curr_pte & PHYS_ADDR_MASK;
+	else
+		return ULLONG_MAX;
+}
+
+static int mmu_show(struct seq_file *s, void *data)
+{
+	struct hl_debugfs_entry *entry = s->private;
+	struct hl_dbg_device_entry *dev_entry = entry->dev_entry;
+	struct hl_device *hdev = dev_entry->hdev;
+	struct hl_ctx *ctx = hdev->user_ctx;
+
+	u64 hop0_addr = 0, hop0_pte_addr = 0, hop0_pte = 0,
+		hop1_addr = 0, hop1_pte_addr = 0, hop1_pte = 0,
+		hop2_addr = 0, hop2_pte_addr = 0, hop2_pte = 0,
+		hop3_addr = 0, hop3_pte_addr = 0, hop3_pte = 0,
+		hop4_addr = 0, hop4_pte_addr = 0, hop4_pte = 0,
+		virt_addr = dev_entry->mmu_addr;
+
+	if (!hdev->mmu_enable)
+		return 0;
+
+	if (!ctx) {
+		dev_err(hdev->dev, "no ctx available\n");
+		return 0;
+	}
+
+	mutex_lock(&ctx->mmu_lock);
+
+	/* the following lookup is copied from unmap() in mmu.c */
+
+	hop0_addr = get_hop0_addr(ctx);
+	hop0_pte_addr = get_hop0_pte_addr(ctx, hop0_addr, virt_addr);
+	hop0_pte = hdev->asic_funcs->read_pte(hdev, hop0_pte_addr);
+	hop1_addr = get_next_hop_addr(hop0_pte);
+
+	if (hop1_addr == ULLONG_MAX)
+		goto not_mapped;
+
+	hop1_pte_addr = get_hop1_pte_addr(ctx, hop1_addr, virt_addr);
+	hop1_pte = hdev->asic_funcs->read_pte(hdev, hop1_pte_addr);
+	hop2_addr = get_next_hop_addr(hop1_pte);
+
+	if (hop2_addr == ULLONG_MAX)
+		goto not_mapped;
+
+	hop2_pte_addr = get_hop2_pte_addr(ctx, hop2_addr, virt_addr);
+	hop2_pte = hdev->asic_funcs->read_pte(hdev, hop2_pte_addr);
+	hop3_addr = get_next_hop_addr(hop2_pte);
+
+	if (hop3_addr == ULLONG_MAX)
+		goto not_mapped;
+
+	hop3_pte_addr = get_hop3_pte_addr(ctx, hop3_addr, virt_addr);
+	hop3_pte = hdev->asic_funcs->read_pte(hdev, hop3_pte_addr);
+
+	if (!(hop3_pte & LAST_MASK)) {
+		hop4_addr = get_next_hop_addr(hop3_pte);
+
+		if (hop4_addr == ULLONG_MAX)
+			goto not_mapped;
+
+		hop4_pte_addr = get_hop4_pte_addr(ctx, hop4_addr, virt_addr);
+		hop4_pte = hdev->asic_funcs->read_pte(hdev, hop4_pte_addr);
+		if (!(hop4_pte & PAGE_PRESENT_MASK))
+			goto not_mapped;
+	} else {
+		if (!(hop3_pte & PAGE_PRESENT_MASK))
+			goto not_mapped;
+	}
+
+	seq_printf(s, "asid: %u, virt_addr: 0x%llx\n",
+			dev_entry->mmu_asid, dev_entry->mmu_addr);
+
+	seq_printf(s, "hop0_addr: 0x%llx\n", hop0_addr);
+	seq_printf(s, "hop0_pte_addr: 0x%llx\n", hop0_pte_addr);
+	seq_printf(s, "hop0_pte: 0x%llx\n", hop0_pte);
+
+	seq_printf(s, "hop1_addr: 0x%llx\n", hop1_addr);
+	seq_printf(s, "hop1_pte_addr: 0x%llx\n", hop1_pte_addr);
+	seq_printf(s, "hop1_pte: 0x%llx\n", hop1_pte);
+
+	seq_printf(s, "hop2_addr: 0x%llx\n", hop2_addr);
+	seq_printf(s, "hop2_pte_addr: 0x%llx\n", hop2_pte_addr);
+	seq_printf(s, "hop2_pte: 0x%llx\n", hop2_pte);
+
+	seq_printf(s, "hop3_addr: 0x%llx\n", hop3_addr);
+	seq_printf(s, "hop3_pte_addr: 0x%llx\n", hop3_pte_addr);
+	seq_printf(s, "hop3_pte: 0x%llx\n", hop3_pte);
+
+	if (!(hop3_pte & LAST_MASK)) {
+		seq_printf(s, "hop4_addr: 0x%llx\n", hop4_addr);
+		seq_printf(s, "hop4_pte_addr: 0x%llx\n", hop4_pte_addr);
+		seq_printf(s, "hop4_pte: 0x%llx\n", hop4_pte);
+	}
+
+	goto out;
+
+not_mapped:
+	dev_err(hdev->dev, "virt addr 0x%llx is not mapped to phys addr\n",
+			virt_addr);
+out:
+	mutex_unlock(&ctx->mmu_lock);
+
+	return 0;
+}
+
+static ssize_t mmu_write(struct file *file, const char __user *buf,
+		size_t count, loff_t *f_pos)
+{
+	struct seq_file *s = file->private_data;
+	struct hl_debugfs_entry *entry = s->private;
+	struct hl_dbg_device_entry *dev_entry = entry->dev_entry;
+	struct hl_device *hdev = dev_entry->hdev;
+	char kbuf[MMU_KBUF_SIZE], asid_kbuf[MMU_ASID_BUF_SIZE],
+		addr_kbuf[MMU_ADDR_BUF_SIZE];
+	char *c;
+	ssize_t rc;
+
+	if (!hdev->mmu_enable)
+		return count;
+
+	memset(kbuf, 0, sizeof(kbuf));
+	memset(asid_kbuf, 0, sizeof(asid_kbuf));
+	memset(addr_kbuf, 0, sizeof(addr_kbuf));
+
+	if (copy_from_user(kbuf, buf, count))
+		goto err;
+
+	kbuf[MMU_KBUF_SIZE - 1] = 0;
+
+	c = strchr(kbuf, ' ');
+	if (!c)
+		goto err;
+
+	memcpy(asid_kbuf, kbuf, c - kbuf);
+
+	rc = kstrtouint(asid_kbuf, 10, &dev_entry->mmu_asid);
+	if (rc)
+		goto err;
+
+	c = strstr(kbuf, " 0x");
+	if (!c)
+		goto err;
+
+	c += 3;
+	memcpy(addr_kbuf, c, (kbuf + count) - c);
+
+	rc = kstrtoull(addr_kbuf, 16, &dev_entry->mmu_addr);
+	if (rc)
+		goto err;
+
+	return count;
+
+err:
+	dev_err(hdev->dev, "usage: echo <asid> <0xaddr> > mmu\n");
+
+	return -EINVAL;
+}
+
+static ssize_t hl_data_read32(struct file *f, char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	char tmp_buf[32];
+	u32 val;
+	ssize_t rc;
+
+	if (*ppos)
+		return 0;
+
+	rc = hdev->asic_funcs->debugfs_read32(hdev, entry->addr, &val);
+	if (rc) {
+		dev_err(hdev->dev, "Failed to read from 0x%010llx\n",
+			entry->addr);
+		return rc;
+	}
+
+	sprintf(tmp_buf, "0x%08x\n", val);
+	rc = simple_read_from_buffer(buf, strlen(tmp_buf) + 1, ppos, tmp_buf,
+			strlen(tmp_buf) + 1);
+
+	return rc;
+}
+
+static ssize_t hl_data_write32(struct file *f, const char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	u32 value;
+	ssize_t rc;
+
+	rc = kstrtouint_from_user(buf, count, 16, &value);
+	if (rc)
+		return rc;
+
+	rc = hdev->asic_funcs->debugfs_write32(hdev, entry->addr, value);
+	if (rc) {
+		dev_err(hdev->dev, "Failed to write 0x%08x to 0x%010llx\n",
+			value, entry->addr);
+		return rc;
+	}
+
+	return count;
+}
+
+static ssize_t hl_get_power_state(struct file *f, char __user *buf,
+		size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	char tmp_buf[200];
+	ssize_t rc;
+	int i;
+
+	if (*ppos)
+		return 0;
+
+	if (hdev->pdev->current_state == PCI_D0)
+		i = 1;
+	else if (hdev->pdev->current_state == PCI_D3hot)
+		i = 2;
+	else
+		i = 3;
+
+	sprintf(tmp_buf,
+		"current power state: %d\n1 - D0\n2 - D3hot\n3 - Unknown\n", i);
+	rc = simple_read_from_buffer(buf, strlen(tmp_buf) + 1, ppos, tmp_buf,
+			strlen(tmp_buf) + 1);
+
+	return rc;
+}
+
+static ssize_t hl_set_power_state(struct file *f, const char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	u32 value;
+	ssize_t rc;
+
+	rc = kstrtouint_from_user(buf, count, 10, &value);
+	if (rc)
+		return rc;
+
+	if (value == 1) {
+		pci_set_power_state(hdev->pdev, PCI_D0);
+		pci_restore_state(hdev->pdev);
+		rc = pci_enable_device(hdev->pdev);
+	} else if (value == 2) {
+		pci_save_state(hdev->pdev);
+		pci_disable_device(hdev->pdev);
+		pci_set_power_state(hdev->pdev, PCI_D3hot);
+	} else {
+		dev_dbg(hdev->dev, "invalid power state value %u\n", value);
+		return -EINVAL;
+	}
+
+	return count;
+}
+
+static ssize_t hl_i2c_data_read(struct file *f, char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	char tmp_buf[32];
+	u32 val;
+	ssize_t rc;
+
+	if (*ppos)
+		return 0;
+
+	rc = hl_debugfs_i2c_read(hdev, entry->i2c_bus, entry->i2c_addr,
+			entry->i2c_reg, &val);
+	if (rc) {
+		dev_err(hdev->dev,
+			"Failed to read from I2C bus %d, addr %d, reg %d\n",
+			entry->i2c_bus, entry->i2c_addr, entry->i2c_reg);
+		return rc;
+	}
+
+	sprintf(tmp_buf, "0x%02x\n", val);
+	rc = simple_read_from_buffer(buf, strlen(tmp_buf) + 1, ppos, tmp_buf,
+			strlen(tmp_buf) + 1);
+
+	return rc;
+}
+
+static ssize_t hl_i2c_data_write(struct file *f, const char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	u32 value;
+	ssize_t rc;
+
+	rc = kstrtouint_from_user(buf, count, 16, &value);
+	if (rc)
+		return rc;
+
+	rc = hl_debugfs_i2c_write(hdev, entry->i2c_bus, entry->i2c_addr,
+			entry->i2c_reg, value);
+	if (rc) {
+		dev_err(hdev->dev,
+			"Failed to write 0x%02x to I2C bus %d, addr %d, reg %d\n",
+			value, entry->i2c_bus, entry->i2c_addr, entry->i2c_reg);
+		return rc;
+	}
+
+	return count;
+}
+
+static ssize_t hl_led0_write(struct file *f, const char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	u32 value;
+	ssize_t rc;
+
+	rc = kstrtouint_from_user(buf, count, 10, &value);
+	if (rc)
+		return rc;
+
+	value = value ? 1 : 0;
+
+	hl_debugfs_led_set(hdev, 0, value);
+
+	return count;
+}
+
+static ssize_t hl_led1_write(struct file *f, const char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	u32 value;
+	ssize_t rc;
+
+	rc = kstrtouint_from_user(buf, count, 10, &value);
+	if (rc)
+		return rc;
+
+	value = value ? 1 : 0;
+
+	hl_debugfs_led_set(hdev, 1, value);
+
+	return count;
+}
+
+static ssize_t hl_led2_write(struct file *f, const char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+	u32 value;
+	ssize_t rc;
+
+	rc = kstrtouint_from_user(buf, count, 10, &value);
+	if (rc)
+		return rc;
+
+	value = value ? 1 : 0;
+
+	hl_debugfs_led_set(hdev, 2, value);
+
+	return count;
+}
+
+static ssize_t hl_device_read(struct file *f, char __user *buf,
+					size_t count, loff_t *ppos)
+{
+	char tmp_buf[200];
+	ssize_t rc;
+
+	if (*ppos)
+		return 0;
+
+	sprintf(tmp_buf,
+		"Valid values are: disable, enable, suspend, resume\n");
+	rc = simple_read_from_buffer(buf, strlen(tmp_buf) + 1, ppos, tmp_buf,
+			strlen(tmp_buf) + 1);
+
+	return rc;
+}
+
+static ssize_t hl_device_write(struct file *f, const char __user *buf,
+				     size_t count, loff_t *ppos)
+{
+	struct hl_dbg_device_entry *entry = file_inode(f)->i_private;
+	struct hl_device *hdev = entry->hdev;
+
+	if (strncmp("disable", buf, strlen("disable")) == 0) {
+		hdev->disabled = true;
+	} else if (strncmp("enable", buf, strlen("enable")) == 0) {
+		hdev->disabled = false;
+	} else if (strncmp("suspend", buf, strlen("suspend")) == 0) {
+		hdev->asic_funcs->suspend(hdev);
+	} else if (strncmp("resume", buf, strlen("resume")) == 0) {
+		hdev->asic_funcs->resume(hdev);
+	} else {
+		dev_err(hdev->dev,
+			"Valid values are: disable, enable, suspend, resume\n");
+		count = -EINVAL;
+	}
+
+	return count;
+}
+
+static const struct file_operations hl_data32b_fops = {
+	.owner = THIS_MODULE,
+	.read = hl_data_read32,
+	.write = hl_data_write32
+};
+
+static const struct file_operations hl_i2c_data_fops = {
+	.owner = THIS_MODULE,
+	.read = hl_i2c_data_read,
+	.write = hl_i2c_data_write
+};
+
+static const struct file_operations hl_power_fops = {
+	.owner = THIS_MODULE,
+	.read = hl_get_power_state,
+	.write = hl_set_power_state
+};
+
+static const struct file_operations hl_led0_fops = {
+	.owner = THIS_MODULE,
+	.write = hl_led0_write
+};
+
+static const struct file_operations hl_led1_fops = {
+	.owner = THIS_MODULE,
+	.write = hl_led1_write
+};
+
+static const struct file_operations hl_led2_fops = {
+	.owner = THIS_MODULE,
+	.write = hl_led2_write
+};
+
+static const struct file_operations hl_device_fops = {
+	.owner = THIS_MODULE,
+	.read = hl_device_read,
+	.write = hl_device_write
+};
+
+static const struct hl_info_list hl_debugfs_list[] = {
+	{"command_buffers", command_buffers_show, NULL},
+	{"command_submission", command_submission_show, NULL},
+	{"command_submission_jobs", command_submission_jobs_show, NULL},
+	{"userptr", userptr_show, NULL},
+	{"vm", vm_show, NULL},
+	{"mmu", mmu_show, mmu_write},
+};
+
+static int hl_debugfs_open(struct inode *inode, struct file *file)
+{
+	struct hl_debugfs_entry *node = inode->i_private;
+
+	return single_open(file, node->info_ent->show, node);
+}
+
+static ssize_t hl_debugfs_write(struct file *file, const char __user *buf,
+		size_t count, loff_t *f_pos)
+{
+	struct hl_debugfs_entry *node = file->f_inode->i_private;
+
+	if (node->info_ent->write)
+		return node->info_ent->write(file, buf, count, f_pos);
+	else
+		return -EINVAL;
+
+}
+
+static const struct file_operations hl_debugfs_fops = {
+	.owner = THIS_MODULE,
+	.open = hl_debugfs_open,
+	.read = seq_read,
+	.write = hl_debugfs_write,
+	.llseek = seq_lseek,
+	.release = single_release,
+};
+
+void hl_debugfs_add_device(struct hl_device *hdev)
+{
+	struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs;
+	int count = ARRAY_SIZE(hl_debugfs_list);
+	struct hl_debugfs_entry *entry;
+	struct dentry *ent;
+	int i;
+
+	dev_entry->hdev = hdev;
+	dev_entry->entry_arr = kmalloc_array(count,
+					sizeof(struct hl_debugfs_entry),
+					GFP_KERNEL);
+	if (!dev_entry->entry_arr)
+		return;
+
+	INIT_LIST_HEAD(&dev_entry->file_list);
+	INIT_LIST_HEAD(&dev_entry->cb_list);
+	INIT_LIST_HEAD(&dev_entry->cs_list);
+	INIT_LIST_HEAD(&dev_entry->cs_job_list);
+	INIT_LIST_HEAD(&dev_entry->userptr_list);
+	INIT_LIST_HEAD(&dev_entry->ctx_mem_hash_list);
+	mutex_init(&dev_entry->file_mutex);
+	spin_lock_init(&dev_entry->cb_spinlock);
+	spin_lock_init(&dev_entry->cs_spinlock);
+	spin_lock_init(&dev_entry->cs_job_spinlock);
+	spin_lock_init(&dev_entry->userptr_spinlock);
+	spin_lock_init(&dev_entry->ctx_mem_hash_spinlock);
+
+	dev_entry->root = debugfs_create_dir(dev_name(hdev->dev),
+						hl_debug_root);
+
+	debugfs_create_x64("addr",
+				0644,
+				dev_entry->root,
+				&dev_entry->addr);
+
+	debugfs_create_file("data32",
+				0644,
+				dev_entry->root,
+				dev_entry,
+				&hl_data32b_fops);
+
+	debugfs_create_file("set_power_state",
+				0200,
+				dev_entry->root,
+				dev_entry,
+				&hl_power_fops);
+
+	debugfs_create_u8("i2c_bus",
+				0644,
+				dev_entry->root,
+				&dev_entry->i2c_bus);
+
+	debugfs_create_u8("i2c_addr",
+				0644,
+				dev_entry->root,
+				&dev_entry->i2c_addr);
+
+	debugfs_create_u8("i2c_reg",
+				0644,
+				dev_entry->root,
+				&dev_entry->i2c_reg);
+
+	debugfs_create_file("i2c_data",
+				0644,
+				dev_entry->root,
+				dev_entry,
+				&hl_i2c_data_fops);
+
+	debugfs_create_file("led0",
+				0200,
+				dev_entry->root,
+				dev_entry,
+				&hl_led0_fops);
+
+	debugfs_create_file("led1",
+				0200,
+				dev_entry->root,
+				dev_entry,
+				&hl_led1_fops);
+
+	debugfs_create_file("led2",
+				0200,
+				dev_entry->root,
+				dev_entry,
+				&hl_led2_fops);
+
+	debugfs_create_file("device",
+				0200,
+				dev_entry->root,
+				dev_entry,
+				&hl_device_fops);
+
+	for (i = 0, entry = dev_entry->entry_arr ; i < count ; i++, entry++) {
+
+		ent = debugfs_create_file(hl_debugfs_list[i].name,
+					0444,
+					dev_entry->root,
+					entry,
+					&hl_debugfs_fops);
+		entry->dent = ent;
+		entry->info_ent = &hl_debugfs_list[i];
+		entry->dev_entry = dev_entry;
+	}
+}
+
+void hl_debugfs_remove_device(struct hl_device *hdev)
+{
+	struct hl_dbg_device_entry *entry = &hdev->hl_debugfs;
+
+	debugfs_remove_recursive(entry->root);
+
+	mutex_destroy(&entry->file_mutex);
+	kfree(entry->entry_arr);
+}
+
+void hl_debugfs_add_file(struct hl_fpriv *hpriv)
+{
+	struct hl_dbg_device_entry *dev_entry = &hpriv->hdev->hl_debugfs;
+
+	mutex_lock(&dev_entry->file_mutex);
+	list_add(&hpriv->debugfs_list, &dev_entry->file_list);
+	mutex_unlock(&dev_entry->file_mutex);
+}
+
+void hl_debugfs_remove_file(struct hl_fpriv *hpriv)
+{
+	struct hl_dbg_device_entry *dev_entry = &hpriv->hdev->hl_debugfs;
+
+	mutex_lock(&dev_entry->file_mutex);
+	list_del(&hpriv->debugfs_list);
+	mutex_unlock(&dev_entry->file_mutex);
+}
+
+void hl_debugfs_add_cb(struct hl_cb *cb)
+{
+	struct hl_dbg_device_entry *dev_entry = &cb->hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->cb_spinlock);
+	list_add(&cb->debugfs_list, &dev_entry->cb_list);
+	spin_unlock(&dev_entry->cb_spinlock);
+}
+
+void hl_debugfs_remove_cb(struct hl_cb *cb)
+{
+	struct hl_dbg_device_entry *dev_entry = &cb->hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->cb_spinlock);
+	list_del(&cb->debugfs_list);
+	spin_unlock(&dev_entry->cb_spinlock);
+}
+
+void hl_debugfs_add_cs(struct hl_cs *cs)
+{
+	struct hl_dbg_device_entry *dev_entry = &cs->ctx->hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->cs_spinlock);
+	list_add(&cs->debugfs_list, &dev_entry->cs_list);
+	spin_unlock(&dev_entry->cs_spinlock);
+}
+
+void hl_debugfs_remove_cs(struct hl_cs *cs)
+{
+	struct hl_dbg_device_entry *dev_entry = &cs->ctx->hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->cs_spinlock);
+	list_del(&cs->debugfs_list);
+	spin_unlock(&dev_entry->cs_spinlock);
+}
+
+void hl_debugfs_add_job(struct hl_device *hdev, struct hl_cs_job *job)
+{
+	struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->cs_job_spinlock);
+	list_add(&job->debugfs_list, &dev_entry->cs_job_list);
+	spin_unlock(&dev_entry->cs_job_spinlock);
+}
+
+void hl_debugfs_remove_job(struct hl_device *hdev, struct hl_cs_job *job)
+{
+	struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->cs_job_spinlock);
+	list_del(&job->debugfs_list);
+	spin_unlock(&dev_entry->cs_job_spinlock);
+}
+
+void hl_debugfs_add_userptr(struct hl_device *hdev, struct hl_userptr *userptr)
+{
+	struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->userptr_spinlock);
+	list_add(&userptr->debugfs_list, &dev_entry->userptr_list);
+	spin_unlock(&dev_entry->userptr_spinlock);
+}
+
+void hl_debugfs_remove_userptr(struct hl_device *hdev,
+				struct hl_userptr *userptr)
+{
+	struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->userptr_spinlock);
+	list_del(&userptr->debugfs_list);
+	spin_unlock(&dev_entry->userptr_spinlock);
+}
+
+void hl_debugfs_add_ctx_mem_hash(struct hl_device *hdev, struct hl_ctx *ctx)
+{
+	struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->ctx_mem_hash_spinlock);
+	list_add(&ctx->debugfs_list, &dev_entry->ctx_mem_hash_list);
+	spin_unlock(&dev_entry->ctx_mem_hash_spinlock);
+}
+
+void hl_debugfs_remove_ctx_mem_hash(struct hl_device *hdev, struct hl_ctx *ctx)
+{
+	struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs;
+
+	spin_lock(&dev_entry->ctx_mem_hash_spinlock);
+	list_del(&ctx->debugfs_list);
+	spin_unlock(&dev_entry->ctx_mem_hash_spinlock);
+}
+
+int __init hl_debugfs_init(void)
+{
+	hl_debug_root = debugfs_create_dir("habanalabs", NULL);
+	if (IS_ERR_OR_NULL(hl_debug_root)) {
+		pr_err("can not create debugfs directory\n");
+		hl_debug_root = NULL;
+		return -ENOMEM;
+	}
+
+	return 0;
+}
+
+void hl_debugfs_fini(void)
+{
+	debugfs_remove_recursive(hl_debug_root);
+}
drivers/misc/habanalabs/device.c +6 -0
index c667b2507c4d..1ec5e10cc36d 100644
@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)
 
 	put_pid(hpriv->taskpid);
 
+	hl_debugfs_remove_file(hpriv);
+
 	mutex_destroy(&hpriv->restore_phase_mutex);
 
 	kfree(hpriv);
@@ -823,6 +825,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass)
 		goto free_cb_pool;
 	}
 
+	hl_debugfs_add_device(hdev);
+
 	rc = hdev->asic_funcs->hw_init(hdev);
 	if (rc) {
 		dev_err(hdev->dev, "failed to initialize the H/W\n");
@@ -950,6 +954,8 @@ void hl_device_fini(struct hl_device *hdev)
 
 	device_late_fini(hdev);
 
+	hl_debugfs_remove_device(hdev);
+
 	hl_sysfs_fini(hdev);
 
 	/*
drivers/misc/habanalabs/goya/goya.c +108 -0
index 3e858f499013..2db0c4f0f7e0 100644
@@ -4358,6 +4358,8 @@ int goya_context_switch(struct hl_device *hdev, u32 asid)
 	job->user_cb_size = cb_size;
 	job->hw_queue_id = GOYA_QUEUE_ID_DMA_0;
 
+	hl_debugfs_add_job(hdev, job);
+
 	parser.ctx_id = HL_KERNEL_ASID_ID;
 	parser.cs_sequence = 0;
 	parser.job_id = job->id;
@@ -4390,6 +4392,7 @@ int goya_context_switch(struct hl_device *hdev, u32 asid)
 
 free_job:
 	hl_userptr_delete_list(hdev, &job->userptr_list);
+	hl_debugfs_remove_job(hdev, job);
 	kfree(job);
 	cb->cs_cnt--;
 
@@ -4420,6 +4423,106 @@ void goya_restore_phase_topology(struct hl_device *hdev)
 	i = RREG32(mmSYNC_MNGR_SOB_OBJ_0);
 }
 
+/*
+ * goya_debugfs_read32 - read a 32bit value from a given device address
+ *
+ * @hdev:	pointer to hl_device structure
+ * @addr:	address in device
+ * @val:	returned value
+ *
+ * In case of DDR address that is not mapped into the default aperture that
+ * the DDR bar exposes, the function will configure the iATU so that the DDR
+ * bar will be positioned at a base address that allows reading from the
+ * required address. Configuring the iATU during normal operation can
+ * lead to undefined behavior and therefore, should be done with extreme care
+ *
+ */
+int goya_debugfs_read32(struct hl_device *hdev, u64 addr, u32 *val)
+{
+	struct asic_fixed_properties *prop = &hdev->asic_prop;
+	int rc = 0;
+
+	if ((addr >= CFG_BASE) && (addr < CFG_BASE + CFG_SIZE)) {
+		*val = RREG32(addr - CFG_BASE);
+
+	} else if ((addr >= SRAM_BASE_ADDR) &&
+			(addr < SRAM_BASE_ADDR + SRAM_SIZE)) {
+
+		*val = readl(hdev->pcie_bar[SRAM_CFG_BAR_ID] +
+				(addr - SRAM_BASE_ADDR));
+
+	} else if ((addr >= DRAM_PHYS_BASE) &&
+			(addr < DRAM_PHYS_BASE + hdev->asic_prop.dram_size)) {
+
+		u64 bar_base_addr = DRAM_PHYS_BASE +
+				(addr & ~(prop->dram_pci_bar_size - 0x1ull));
+
+		rc = goya_set_ddr_bar_base(hdev, bar_base_addr);
+		if (!rc) {
+			*val = readl(hdev->pcie_bar[DDR_BAR_ID] +
+						(addr - bar_base_addr));
+
+			rc = goya_set_ddr_bar_base(hdev, DRAM_PHYS_BASE +
+				(MMU_PAGE_TABLES_ADDR &
+					~(prop->dram_pci_bar_size - 0x1ull)));
+		}
+	} else {
+		rc = -EFAULT;
+	}
+
+	return rc;
+}
+
+/*
+ * goya_debugfs_write32 - write a 32bit value to a given device address
+ *
+ * @hdev:	pointer to hl_device structure
+ * @addr:	address in device
+ * @val:	returned value
+ *
+ * In case of DDR address that is not mapped into the default aperture that
+ * the DDR bar exposes, the function will configure the iATU so that the DDR
+ * bar will be positioned at a base address that allows writing to the
+ * required address. Configuring the iATU during normal operation can
+ * lead to undefined behavior and therefore, should be done with extreme care
+ *
+ */
+int goya_debugfs_write32(struct hl_device *hdev, u64 addr, u32 val)
+{
+	struct asic_fixed_properties *prop = &hdev->asic_prop;
+	int rc = 0;
+
+	if ((addr >= CFG_BASE) && (addr < CFG_BASE + CFG_SIZE)) {
+		WREG32(addr - CFG_BASE, val);
+
+	} else if ((addr >= SRAM_BASE_ADDR) &&
+			(addr < SRAM_BASE_ADDR + SRAM_SIZE)) {
+
+		writel(val, hdev->pcie_bar[SRAM_CFG_BAR_ID] +
+					(addr - SRAM_BASE_ADDR));
+
+	} else if ((addr >= DRAM_PHYS_BASE) &&
+			(addr < DRAM_PHYS_BASE + hdev->asic_prop.dram_size)) {
+
+		u64 bar_base_addr = DRAM_PHYS_BASE +
+				(addr & ~(prop->dram_pci_bar_size - 0x1ull));
+
+		rc = goya_set_ddr_bar_base(hdev, bar_base_addr);
+		if (!rc) {
+			writel(val, hdev->pcie_bar[DDR_BAR_ID] +
+						(addr - bar_base_addr));
+
+			rc = goya_set_ddr_bar_base(hdev, DRAM_PHYS_BASE +
+				(MMU_PAGE_TABLES_ADDR &
+					~(prop->dram_pci_bar_size - 0x1ull)));
+		}
+	} else {
+		rc = -EFAULT;
+	}
+
+	return rc;
+}
+
 static u64 goya_read_pte(struct hl_device *hdev, u64 addr)
 {
 	struct goya_device *goya = hdev->asic_specific;
@@ -4766,6 +4869,8 @@ static int goya_mmu_clear_pgt_range(struct hl_device *hdev)
 	job->user_cb_size = cb_size;
 	job->hw_queue_id = GOYA_QUEUE_ID_DMA_0;
 
+	hl_debugfs_add_job(hdev, job);
+
 	parser.ctx_id = HL_KERNEL_ASID_ID;
 	parser.cs_sequence = 0;
 	parser.job_id = job->id;
@@ -4794,6 +4899,7 @@ static int goya_mmu_clear_pgt_range(struct hl_device *hdev)
 
 free_job:
 	hl_userptr_delete_list(hdev, &job->userptr_list);
+	hl_debugfs_remove_job(hdev, job);
 	kfree(job);
 	cb->cs_cnt--;
 
@@ -5203,6 +5309,8 @@ static const struct hl_asic_funcs goya_funcs = {
 	.update_eq_ci = goya_update_eq_ci,
 	.context_switch = goya_context_switch,
 	.restore_phase_topology = goya_restore_phase_topology,
+	.debugfs_read32 = goya_debugfs_read32,
+	.debugfs_write32 = goya_debugfs_write32,
 	.add_device_attr = goya_add_device_attr,
 	.remove_device_attr = goya_remove_device_attr,
 	.handle_eqe = goya_handle_eqe,
drivers/misc/habanalabs/goya/goyaP.h +5 -0
index 6d2da801c155..569015857c59 100644
@@ -168,6 +168,10 @@ struct goya_device {
 	u32		hw_cap_initialized;
 };
 
+int goya_debugfs_i2c_read(struct hl_device *hdev, u8 i2c_bus,
+			u8 i2c_addr, u8 i2c_reg, u32 *val);
+int goya_debugfs_i2c_write(struct hl_device *hdev, u8 i2c_bus,
+			u8 i2c_addr, u8 i2c_reg, u32 val);
 int goya_test_cpu_queue(struct hl_device *hdev);
 int goya_send_cpu_message(struct hl_device *hdev, u32 *msg, u16 len,
 				u32 timeout, long *result);
@@ -178,6 +182,7 @@ long goya_get_fan_speed(struct hl_device *hdev, int sensor_index, u32 attr);
 long goya_get_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr);
 void goya_set_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr,
 			long value);
+void goya_debugfs_led_set(struct hl_device *hdev, u8 led, u8 state);
 void goya_set_pll_profile(struct hl_device *hdev, enum hl_pll_frequency freq);
 int goya_add_device_attr(struct hl_device *hdev);
 void goya_remove_device_attr(struct hl_device *hdev);
drivers/misc/habanalabs/habanalabs.h +190 -0
index fc33026974c9..a4258a6847c4 100644
@@ -225,6 +225,7 @@ struct hl_cb_mgr {
  * @refcount: reference counter for usage of the CB.
  * @hdev: pointer to device this CB belongs to.
  * @lock: spinlock to protect mmap/cs flows.
+ * @debugfs_list: node in debugfs list of command buffers.
  * @pool_list: node in pool list of command buffers.
  * @kernel_address: Holds the CB's kernel virtual address.
  * @bus_address: Holds the CB's DMA address.
@@ -241,6 +242,7 @@ struct hl_cb {
 	struct kref		refcount;
 	struct hl_device	*hdev;
 	spinlock_t		lock;
+	struct list_head	debugfs_list;
 	struct list_head	pool_list;
 	u64			kernel_address;
 	dma_addr_t		bus_address;
@@ -442,6 +444,8 @@ enum hl_pll_frequency {
  * @update_eq_ci: update event queue CI.
  * @context_switch: called upon ASID context switch.
  * @restore_phase_topology: clear all SOBs amd MONs.
+ * @debugfs_read32: debug interface for reading u32 from DRAM/SRAM.
+ * @debugfs_write32: debug interface for writing u32 to DRAM/SRAM.
  * @add_device_attr: add ASIC specific device attributes.
  * @remove_device_attr: remove ASIC specific device attributes.
  * @handle_eqe: handle event queue entry (IRQ) from ArmCP.
@@ -510,6 +514,8 @@ struct hl_asic_funcs {
 	void (*update_eq_ci)(struct hl_device *hdev, u32 val);
 	int (*context_switch)(struct hl_device *hdev, u32 asid);
 	void (*restore_phase_topology)(struct hl_device *hdev);
+	int (*debugfs_read32)(struct hl_device *hdev, u64 addr, u32 *val);
+	int (*debugfs_write32)(struct hl_device *hdev, u64 addr, u32 val);
 	int (*add_device_attr)(struct hl_device *hdev);
 	void (*remove_device_attr)(struct hl_device *hdev);
 	void (*handle_eqe)(struct hl_device *hdev,
@@ -572,6 +578,7 @@ struct hl_va_range {
  * @mem_hash_lock: protects the mem_hash.
  * @mmu_lock: protects the MMU page tables. Any change to the PGT, modifing the
  *            MMU hash or walking the PGT requires talking this lock
+ * @debugfs_list: node in debugfs list of contexts.
  * @cs_sequence: sequence number for CS. Value is assigned to a CS and passed
  *			to user so user could inquire about CS. It is used as
  *			index to cs_pending array.
@@ -596,6 +603,7 @@ struct hl_ctx {
 	struct hl_va_range	dram_va_range;
 	struct mutex		mem_hash_lock;
 	struct mutex		mmu_lock;
+	struct list_head	debugfs_list;
 	u64			cs_sequence;
 	spinlock_t		cs_lock;
 	atomic64_t		dram_phys_mem;
@@ -654,6 +662,7 @@ struct hl_userptr {
  * @fence: pointer to the fence object of this CS.
  * @work_tdr: delayed work node for TDR.
  * @mirror_node : node in device mirror list of command submissions.
+ * @debugfs_list: node in debugfs list of command submissions.
  * @sequence: the sequence number of this CS.
  * @submitted: true if CS was submitted to H/W.
  * @completed: true if CS was completed by device.
@@ -671,6 +680,7 @@ struct hl_cs {
 	struct dma_fence	*fence;
 	struct delayed_work	work_tdr;
 	struct list_head	mirror_node;
+	struct list_head	debugfs_list;
 	u64			sequence;
 	u8			submitted;
 	u8			completed;
@@ -689,6 +699,7 @@ struct hl_cs {
  * @finish_work: workqueue object to run when job is completed.
  * @userptr_list: linked-list of userptr mappings that belong to this job and
  *			wait for completion.
+ * @debugfs_list: node in debugfs list of command submission jobs.
  * @id: the id of this job inside a CS.
  * @hw_queue_id: the id of the H/W queue this job is submitted to.
  * @user_cb_size: the actual size of the CB we got from the user.
@@ -702,6 +713,7 @@ struct hl_cs_job {
 	struct hl_cb		*patched_cb;
 	struct work_struct	finish_work;
 	struct list_head	userptr_list;
+	struct list_head	debugfs_list;
 	u32			id;
 	u32			hw_queue_id;
 	u32			user_cb_size;
@@ -832,6 +844,7 @@ struct hl_vm {
  * @ctx: current executing context.
  * @ctx_mgr: context manager to handle multiple context for this FD.
  * @cb_mgr: command buffer manager to handle multiple buffers for this FD.
+ * @debugfs_list: list of relevant ASIC debugfs.
  * @refcount: number of related contexts.
  * @restore_phase_mutex: lock for context switch and restore phase.
  */
@@ -842,11 +855,90 @@ struct hl_fpriv {
 	struct hl_ctx		*ctx; /* TODO: remove for multiple ctx */
 	struct hl_ctx_mgr	ctx_mgr;
 	struct hl_cb_mgr	cb_mgr;
+	struct list_head	debugfs_list;
 	struct kref		refcount;
 	struct mutex		restore_phase_mutex;
 };
 
 
+/*
+ * DebugFS
+ */
+
+/**
+ * struct hl_info_list - debugfs file ops.
+ * @name: file name.
+ * @show: function to output information.
+ * @write: function to write to the file.
+ */
+struct hl_info_list {
+	const char	*name;
+	int		(*show)(struct seq_file *s, void *data);
+	ssize_t		(*write)(struct file *file, const char __user *buf,
+				size_t count, loff_t *f_pos);
+};
+
+/**
+ * struct hl_debugfs_entry - debugfs dentry wrapper.
+ * @dent: base debugfs entry structure.
+ * @info_ent: dentry realted ops.
+ * @dev_entry: ASIC specific debugfs manager.
+ */
+struct hl_debugfs_entry {
+	struct dentry			*dent;
+	const struct hl_info_list	*info_ent;
+	struct hl_dbg_device_entry	*dev_entry;
+};
+
+/**
+ * struct hl_dbg_device_entry - ASIC specific debugfs manager.
+ * @root: root dentry.
+ * @hdev: habanalabs device structure.
+ * @entry_arr: array of available hl_debugfs_entry.
+ * @file_list: list of available debugfs files.
+ * @file_mutex: protects file_list.
+ * @cb_list: list of available CBs.
+ * @cb_spinlock: protects cb_list.
+ * @cs_list: list of available CSs.
+ * @cs_spinlock: protects cs_list.
+ * @cs_job_list: list of available CB jobs.
+ * @cs_job_spinlock: protects cs_job_list.
+ * @userptr_list: list of available userptrs (virtual memory chunk descriptor).
+ * @userptr_spinlock: protects userptr_list.
+ * @ctx_mem_hash_list: list of available contexts with MMU mappings.
+ * @ctx_mem_hash_spinlock: protects cb_list.
+ * @addr: next address to read/write from/to in read/write32.
+ * @mmu_addr: next virtual address to translate to physical address in mmu_show.
+ * @mmu_asid: ASID to use while translating in mmu_show.
+ * @i2c_bus: generic u8 debugfs file for bus value to use in i2c_data_read.
+ * @i2c_bus: generic u8 debugfs file for address value to use in i2c_data_read.
+ * @i2c_bus: generic u8 debugfs file for register value to use in i2c_data_read.
+ */
+struct hl_dbg_device_entry {
+	struct dentry			*root;
+	struct hl_device		*hdev;
+	struct hl_debugfs_entry		*entry_arr;
+	struct list_head		file_list;
+	struct mutex			file_mutex;
+	struct list_head		cb_list;
+	spinlock_t			cb_spinlock;
+	struct list_head		cs_list;
+	spinlock_t			cs_spinlock;
+	struct list_head		cs_job_list;
+	spinlock_t			cs_job_spinlock;
+	struct list_head		userptr_list;
+	spinlock_t			userptr_spinlock;
+	struct list_head		ctx_mem_hash_list;
+	spinlock_t			ctx_mem_hash_spinlock;
+	u64				addr;
+	u64				mmu_addr;
+	u32				mmu_asid;
+	u8				i2c_bus;
+	u8				i2c_addr;
+	u8				i2c_reg;
+};
+
+
 /*
  * DEVICES
  */
@@ -939,6 +1031,7 @@ struct hl_device_reset_work {
  * @hwmon_dev: H/W monitor device.
  * @pm_mng_profile: current power management profile.
  * @hl_chip_info: ASIC's sensors information.
+ * @hl_debugfs: device's debugfs manager.
  * @cb_pool: list of preallocated CBs.
  * @cb_pool_lock: protects the CB pool.
  * @user_ctx: current user context executing.
@@ -1004,6 +1097,8 @@ struct hl_device {
 	enum hl_pm_mng_profile		pm_mng_profile;
 	struct hwmon_chip_info		hl_chip_info;
 
+	struct hl_dbg_device_entry	hl_debugfs;
+
 	struct list_head		cb_pool;
 	spinlock_t			cb_pool_lock;
 
@@ -1241,6 +1336,101 @@ void hl_set_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr,
 u64 hl_get_max_power(struct hl_device *hdev);
 void hl_set_max_power(struct hl_device *hdev, u64 value);
 
+#ifdef CONFIG_DEBUG_FS
+
+int hl_debugfs_init(void);
+void hl_debugfs_fini(void);
+void hl_debugfs_add_device(struct hl_device *hdev);
+void hl_debugfs_remove_device(struct hl_device *hdev);
+void hl_debugfs_add_file(struct hl_fpriv *hpriv);
+void hl_debugfs_remove_file(struct hl_fpriv *hpriv);
+void hl_debugfs_add_cb(struct hl_cb *cb);
+void hl_debugfs_remove_cb(struct hl_cb *cb);
+void hl_debugfs_add_cs(struct hl_cs *cs);
+void hl_debugfs_remove_cs(struct hl_cs *cs);
+void hl_debugfs_add_job(struct hl_device *hdev, struct hl_cs_job *job);
+void hl_debugfs_remove_job(struct hl_device *hdev, struct hl_cs_job *job);
+void hl_debugfs_add_userptr(struct hl_device *hdev, struct hl_userptr *userptr);
+void hl_debugfs_remove_userptr(struct hl_device *hdev,
+				struct hl_userptr *userptr);
+void hl_debugfs_add_ctx_mem_hash(struct hl_device *hdev, struct hl_ctx *ctx);
+void hl_debugfs_remove_ctx_mem_hash(struct hl_device *hdev, struct hl_ctx *ctx);
+
+#else
+
+static inline int __init hl_debugfs_init(void)
+{
+	return 0;
+}
+
+static inline void hl_debugfs_fini(void)
+{
+}
+
+static inline void hl_debugfs_add_device(struct hl_device *hdev)
+{
+}
+
+static inline void hl_debugfs_remove_device(struct hl_device *hdev)
+{
+}
+
+static inline void hl_debugfs_add_file(struct hl_fpriv *hpriv)
+{
+}
+
+static inline void hl_debugfs_remove_file(struct hl_fpriv *hpriv)
+{
+}
+
+static inline void hl_debugfs_add_cb(struct hl_cb *cb)
+{
+}
+
+static inline void hl_debugfs_remove_cb(struct hl_cb *cb)
+{
+}
+
+static inline void hl_debugfs_add_cs(struct hl_cs *cs)
+{
+}
+
+static inline void hl_debugfs_remove_cs(struct hl_cs *cs)
+{
+}
+
+static inline void hl_debugfs_add_job(struct hl_device *hdev,
+					struct hl_cs_job *job)
+{
+}
+
+static inline void hl_debugfs_remove_job(struct hl_device *hdev,
+					struct hl_cs_job *job)
+{
+}
+
+static inline void hl_debugfs_add_userptr(struct hl_device *hdev,
+					struct hl_userptr *userptr)
+{
+}
+
+static inline void hl_debugfs_remove_userptr(struct hl_device *hdev,
+					struct hl_userptr *userptr)
+{
+}
+
+static inline void hl_debugfs_add_ctx_mem_hash(struct hl_device *hdev,
+					struct hl_ctx *ctx)
+{
+}
+
+static inline void hl_debugfs_remove_ctx_mem_hash(struct hl_device *hdev,
+					struct hl_ctx *ctx)
+{
+}
+
+#endif
+
 /* IOCTLs */
 long hl_ioctl(struct file *filep, unsigned int cmd, unsigned long arg);
 int hl_cb_ioctl(struct hl_fpriv *hpriv, void *data);
drivers/misc/habanalabs/habanalabs_drv.c +14 -2
index 658dc4588a36..96e0fc9c401f 100644
@@ -150,6 +150,8 @@ int hl_device_open(struct inode *inode, struct file *filp)
 	 */
 	hl_device_set_frequency(hdev, PLL_HIGH);
 
+	hl_debugfs_add_file(hpriv);
+
 	return 0;
 
 out_err:
@@ -417,17 +419,20 @@ static int __init hl_init(void)
 		goto remove_major;
 	}
 
+	hl_debugfs_init();
+
 	rc = pci_register_driver(&hl_pci_driver);
 	if (rc) {
 		pr_err("failed to register pci device\n");
-		goto remove_class;
+		goto remove_debugfs;
 	}
 
 	pr_debug("driver loaded\n");
 
 	return 0;
 
-remove_class:
+remove_debugfs:
+	hl_debugfs_fini();
 	class_destroy(hl_class);
 remove_major:
 	unregister_chrdev_region(MKDEV(hl_major, 0), HL_MAX_MINORS);
@@ -441,6 +446,13 @@ static void __exit hl_exit(void)
 {
 	pci_unregister_driver(&hl_pci_driver);
 
+	/*
+	 * Removing debugfs must be after all devices or simulator devices
+	 * have been removed because otherwise we get a bug in the
+	 * debugfs module for referencing NULL objects
+	 */
+	hl_debugfs_fini();
+
 	class_destroy(hl_class);
 	unregister_chrdev_region(MKDEV(hl_major, 0), HL_MAX_MINORS);
 
drivers/misc/habanalabs/memory.c +8 -0
index cf0be4f254e0..35f2339310d3 100644
@@ -1289,6 +1289,8 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u32 size,
 		goto free_sgt;
 	}
 
+	hl_debugfs_add_userptr(hdev, userptr);
+
 	return 0;
 
 free_sgt:
@@ -1314,6 +1316,8 @@ int hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr)
 {
 	struct page **pages;
 
+	hl_debugfs_remove_userptr(hdev, userptr);
+
 	if (userptr->dma_mapped)
 		hdev->asic_funcs->hl_dma_unmap_sg(hdev,
 				userptr->sgt->sgl,
@@ -1475,6 +1479,8 @@ int hl_vm_ctx_init_with_ranges(struct hl_ctx *ctx, u64 host_range_start,
 		goto dram_vm_err;
 	}
 
+	hl_debugfs_add_ctx_mem_hash(hdev, ctx);
+
 	return 0;
 
 dram_vm_err:
@@ -1597,6 +1603,8 @@ void hl_vm_ctx_fini(struct hl_ctx *ctx)
 	struct hlist_node *tmp_node;
 	int i;
 
+	hl_debugfs_remove_ctx_mem_hash(hdev, ctx);
+
 	if (!hash_empty(ctx->mem_hash))
 		dev_notice(hdev->dev, "ctx is freed while it has va in use\n");
 
Signature · 1 line
2.17.1
Signature · 2 lines
Sincerely yours,
Mike.
#18 lore raw

Re: [PATCH v3 12/15] habanalabs: add virtual memory and MMU modules

In reply to Mike Rapoport
Quoted 103 lines · Mike Rapoport
Quoted 50 lines · Oded Gabbay

From: Omer Shpigelman <[email protected]>

This patch adds the Virtual Memory and MMU modules.

Goya has an internal MMU which provides process isolation on the internal DDR. The internal MMU also performs translations for transactions that go from Goya to the Host.

The driver is responsible for allocating and freeing memory on the DDR upon user request. It also provides an interface to map and unmap DDR and Host memory to the device address space.

---
Changes in v3:
- Remove use of three exclamation marks
- Add optimization - support mapping for huge page host memory
- Minor bug fixes

 drivers/misc/habanalabs/Makefile              |    2 +-
 drivers/misc/habanalabs/context.c             |   19 +-
 drivers/misc/habanalabs/device.c              |   20 +-
 drivers/misc/habanalabs/goya/goya.c           |  393 +++++
 drivers/misc/habanalabs/habanalabs.h          |  188 +-
 drivers/misc/habanalabs/habanalabs_drv.c      |    2 +-
 drivers/misc/habanalabs/habanalabs_ioctl.c    |    3 +-
 .../include/hw_ip/mmu/mmu_general.h           |   45 +
 .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h   |   15 +
 drivers/misc/habanalabs/memory.c              | 1514 +++++++++++++++++
 drivers/misc/habanalabs/mmu.c                 |  604 +++++++
 include/uapi/misc/habanalabs.h                |  122 +-
 12 files changed, 2919 insertions(+), 8 deletions(-)
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h
 create mode 100644 drivers/misc/habanalabs/mmu.c
drivers/misc/habanalabs/Makefile +1 -1
index 5167699701b9..2698bb6a2355 100644
@@ -6,7 +6,7 @@ obj-m := habanalabs.o
 
 habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \
              command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \
-             command_submission.o
+             command_submission.o mmu.o
 
 include $(src)/goya/Makefile
 habanalabs-y += $(HL_GOYA_FILES)

[ ... ]

Quoted 18 lines
drivers/misc/habanalabs/memory.c +4 -0
index 6536b40c63e0..cf0be4f254e0 100644
@@ -5,12 +5,1198 @@
  * All Rights Reserved.
  */
 
+#include <uapi/misc/habanalabs.h>
 #include "habanalabs.h"
+#include "include/hw_ip/mmu/mmu_general.h"
 
 #include <linux/sched.h>
 #include <linux/uaccess.h>
 #include <linux/genalloc.h>
 
+#define PGS_IN_HPAGE   (HPAGE_SIZE >> PAGE_SHIFT)
+#define HL_MMU_DEBUG 0

[ ... ]

Quoted 13 lines
drivers/misc/habanalabs/memory.c +13 -0
+/*
+ * init_phys_pg_pack_from_userptr - initialize physical page pack from host
+ *                                   memory
+ *
+ * @ctx                : current context
+ * @userptr            : userptr to initialize from
+ * @pphys_pg_pack      : res pointer
+ *
+ * This function does the following:
+ * - Pin the physical pages related to the given virtual block
+ * - Create a physical page pack from the physical pages related to the given
+ *   virtual block
+ */

drivers/misc/habanalabs/memory.c line 653

Quoted 10 lines
drivers/misc/habanalabs/memory.c +10 -0
+static int init_phys_pg_pack_from_userptr(struct hl_ctx *ctx,
+             struct hl_userptr *userptr,
+             struct hl_vm_phys_pg_pack **pphys_pg_pack)
+{
+     struct hl_vm_phys_pg_pack *phys_pg_pack;
+     struct scatterlist *sg;
+     dma_addr_t dma_addr;
+     u64 page_mask;
+     u32 npages, total_npages, page_size = PAGE_SIZE;
+     bool first = true, is_2mb_opt = true;

Now you've mentioned 2M pages, I've started to wonder what would happen if PAGE_SIZE != 4k and HPAGE_SIZE != 2M?

Quoted 2 lines

There are architectures that may have different sizes for both...

Will be fixed but we currently don't have any H/W that we can check this on.

Quoted 171 lines
Quoted 82 lines
drivers/misc/habanalabs/memory.c +82 -0
+     int rc, i, j;
+
+     phys_pg_pack = kzalloc(sizeof(*phys_pg_pack), GFP_KERNEL);
+     if (!phys_pg_pack)
+             return -ENOMEM;
+
+     phys_pg_pack->vm_type = userptr->vm_type;
+     phys_pg_pack->created_from_userptr = true;
+     phys_pg_pack->asid = ctx->asid;
+     atomic_set(&phys_pg_pack->mapping_cnt, 1);
+
+     /* Only if all dma_addrs are aligned to 2MB and their
+      * sizes is at least 2MB, we can use huge page mapping.
+      * We limit the 2MB optimization to this condition,
+      * since later on we acquire the related VA range as one
+      * consecutive block.
+      */
+     total_npages = 0;
+     for_each_sg(userptr->sgt->sgl, sg, userptr->sgt->nents, i) {
+             npages = get_sg_info(sg, &dma_addr);
+
+             total_npages += npages;
+
+             if (first) {
+                     first = false;
+                     dma_addr &= HPAGE_MASK;
+             }
+
+             if ((npages % PGS_IN_HPAGE) || (dma_addr & (HPAGE_SIZE - 1)))
+                     is_2mb_opt = false;
+     }
+
+     if (is_2mb_opt) {
+             page_size = HPAGE_SIZE;
+             total_npages /= PGS_IN_HPAGE;
+     }
+
+     page_mask = ~(((u64) page_size) - 1);
+
+     phys_pg_pack->pages = kcalloc(total_npages, sizeof(u64), GFP_KERNEL);
+     if (!phys_pg_pack->pages) {
+             rc = -ENOMEM;
+             goto page_pack_arr_mem_err;
+     }
+
+     phys_pg_pack->npages = total_npages;
+     phys_pg_pack->page_size = page_size;
+     phys_pg_pack->total_size = total_npages * page_size;
+
+     j = 0;
+     first = true;
+     for_each_sg(userptr->sgt->sgl, sg, userptr->sgt->nents, i) {
+             npages = get_sg_info(sg, &dma_addr);
+
+             /* align down to physical page size and save the offset */
+             if (first) {
+                     first = false;
+                     phys_pg_pack->offset = dma_addr & (page_size - 1);
+                     dma_addr &= page_mask;
+             }
+
+             while (npages) {
+                     phys_pg_pack->pages[j++] = dma_addr;
+                     dma_addr += page_size;
+
+                     if (is_2mb_opt)
+                             npages -= PGS_IN_HPAGE;
+                     else
+                             npages--;
+             }
+     }
+
+     *pphys_pg_pack = phys_pg_pack;
+
+     return 0;
+
+page_pack_arr_mem_err:
+     kfree(phys_pg_pack);
+
+     return rc;
+}
+

[ ... ]

Quoted 86 lines
drivers/misc/habanalabs/mmu.c new file +80 -0
new file mode 100644
index 000000000000..054b72e48a49
@@ -0,0 +1,604 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/*
+ * Copyright 2016-2018 HabanaLabs, Ltd.
+ * All Rights Reserved.
+ */
+
+#include "habanalabs.h"
+#include "include/hw_ip/mmu/mmu_general.h"
+
+#include <linux/genalloc.h>
+
+#define HOP_NOT_FREED        0
+#define HOP_FREED    1
+
+static struct pgt_info *get_pgt_info(struct hl_ctx *ctx, u64 addr)
+{
+     struct pgt_info *pgt_info = NULL;
+
+     hash_for_each_possible(ctx->mmu_hash, pgt_info, node,
+                             (unsigned long) addr)
+             if (addr == pgt_info->addr)
+                     break;
+
+     return pgt_info;
+}
+
+static void free_hop(struct hl_ctx *ctx, u64 hop_addr)
+{
+     struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr);
+
+     gen_pool_free(pgt_info->ctx->hdev->mmu_pgt_pool, pgt_info->addr,
+                     ctx->hdev->asic_prop.mmu_hop_table_size);
+     hash_del(&pgt_info->node);
+
+     kfree(pgt_info);
+}
+
+static u64 alloc_hop(struct hl_ctx *ctx)
+{
+     struct hl_device *hdev = ctx->hdev;
+     struct pgt_info *pgt_info;
+     u64 addr;
+
+     pgt_info = kmalloc(sizeof(*pgt_info), GFP_KERNEL);
+     if (!pgt_info)
+             return ULLONG_MAX;
+
+     addr = (u64) gen_pool_alloc(hdev->mmu_pgt_pool,
+                     hdev->asic_prop.mmu_hop_table_size);
+     if (!addr) {
+             dev_err(hdev->dev, "failed to allocate page\n");
+             kfree(pgt_info);
+             return ULLONG_MAX;
+     }
+
+     pgt_info->addr = addr;
+     pgt_info->ctx = ctx;
+     pgt_info->num_of_ptes = 0;
+     hash_add(ctx->mmu_hash, &pgt_info->node, addr);
+
+     return addr;
+}
+
+static inline void clear_pte(struct hl_device *hdev, u64 pte_addr)
+{
+     /* clear the last and present bits */
+     hdev->asic_funcs->write_pte(hdev, pte_addr, 0);
+}
+
+static inline void inc_num_of_ptes(struct hl_ctx *ctx, u64 hop_addr)
+{
+     get_pgt_info(ctx, hop_addr)->num_of_ptes++;
+}
+
+/*
+ * dec_num_of_ptes - decrement the num of ptes and free the hop if possible
+ *
+ * @ctx: pointer to the context structure
+ * @hop_addr: addr of the hop
Quoted 9 lines

drivers/misc/habanalabs/mmu.c line 85

Quoted 5 lines
drivers/misc/habanalabs/mmu.c new file +5 -0
+ *
+ * This function returns HOP_FREED if the hop was freed or HOP_NOT_FREED if the
+ * num of ptes was decremented without freeing the hop.
+ */
+static inline int dec_num_of_ptes(struct hl_ctx *ctx, u64 hop_addr)

I think it's worth emphasizing that this function may free a pte. Maybe even use {get,put}_pte instead of {inc,dec}_num_of_ptes.

Agreed, will be fixed.

Quoted 5 lines
Quoted 5 lines
drivers/misc/habanalabs/mmu.c new file +5 -0
+{
+     struct pgt_info *pgt_info = get_pgt_info(ctx, hop_addr);
+
+     pgt_info->num_of_ptes--;
+
Quoted 8 lines

drivers/misc/habanalabs/mmu.c line 96

Quoted 6 lines
drivers/misc/habanalabs/mmu.c new file +6 -0
+     if (pgt_info->num_of_ptes == 0) {
+             free_hop(ctx, hop_addr);
+             return HOP_FREED;
+     }
+
+     return HOP_NOT_FREED;

Why not simply return pgt_info->num_of_ptes?

Fixed

Quoted 46 lines
Quoted 33 lines
drivers/misc/habanalabs/mmu.c new file +32 -0
+}
+
+static inline u64 get_hop0_addr(struct hl_ctx *ctx)
+{
+     return ctx->hdev->asic_prop.mmu_pgt_addr +
+                     (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size);
+}
+
+static inline u64 get_hop0_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+                                     u64 virt_addr)
+{
+     return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+                     ((virt_addr & HOP0_MASK) >> HOP0_SHIFT);
+}
+
+static inline u64 get_hop1_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+                                     u64 virt_addr)
+{
+     return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+                     ((virt_addr & HOP1_MASK) >> HOP1_SHIFT);
+}
+
+static inline u64 get_hop2_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+                                     u64 virt_addr)
+{
+     return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+                     ((virt_addr & HOP2_MASK) >> HOP2_SHIFT);
+}
+
+static inline u64 get_hop3_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+                                     u64 virt_addr)
+{

drivers/misc/habanalabs/mmu.c line 138

Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10 -0
+     return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+                     ((virt_addr & HOP3_MASK) >> HOP3_SHIFT);
+}
+
+static inline u64 get_hop4_pte_addr(struct hl_ctx *ctx, u64 hop_addr,
+                                     u64 virt_addr)
+{
+     return hop_addr + ctx->hdev->asic_prop.mmu_pte_size *
+                     ((virt_addr & HOP4_MASK) >> HOP4_SHIFT);
+}

These are almost clones of each other. I'd factor out

Quoted 5 lines

static inline u64 get_hopN_pte(struct hl_ctx *ctx, u64 hop_addr,

                                u64 vaddr, u64 mask, u64 shift);

and alias it for each level.

Fixed, but I'm aliasing them using inline functions, which I prefer to macros.

Quoted 188 lines
Quoted 188 lines
drivers/misc/habanalabs/mmu.c new file +188 -0
+static inline u64 get_next_hop_addr(u64 curr_pte)
+{
+     if (curr_pte & PAGE_PRESENT_MASK)
+             return curr_pte & PHYS_ADDR_MASK;
+     else
+             return ULLONG_MAX;
+}
+
+/*
+ * hl_mmu_init - init the mmu module
+ *
+ * @hdev: pointer to the habanalabs device structure
+ *
+ * This function does the following:
+ * - Allocate max_asid zeroed hop0 pgts so no mapping is available
+ * - Enable mmu in hw
+ * - Invalidate the mmu cache
+ * - Create a pool of pages for pgts
+ * - Returns 0 on success
+ *
+ * This function depends on DMA QMAN to be working!
+ */
+int hl_mmu_init(struct hl_device *hdev)
+{
+     struct asic_fixed_properties *prop = &hdev->asic_prop;
+     int rc;
+
+     if (!hdev->mmu_enable)
+             return 0;
+
+     /* MMU HW init was already done in device hw_init() */
+
+     mutex_init(&hdev->mmu_cache_lock);
+
+     hdev->mmu_pgt_pool =
+                     gen_pool_create(__ffs(prop->mmu_hop_table_size), -1);
+
+     if (!hdev->mmu_pgt_pool) {
+             dev_err(hdev->dev, "Failed to create page gen pool\n");
+             rc = -ENOMEM;
+             goto err_pool_create;
+     }
+
+     rc = gen_pool_add(hdev->mmu_pgt_pool, prop->mmu_pgt_addr +
+                     prop->mmu_hop0_tables_total_size,
+                     prop->mmu_pgt_size - prop->mmu_hop0_tables_total_size,
+                     -1);
+     if (rc) {
+             dev_err(hdev->dev, "Failed to add memory to page gen pool\n");
+             goto err_pool_add;
+     }
+
+     return 0;
+
+err_pool_add:
+     gen_pool_destroy(hdev->mmu_pgt_pool);
+err_pool_create:
+     mutex_destroy(&hdev->mmu_cache_lock);
+
+     return rc;
+}
+
+/*
+ * hl_mmu_fini - release the mmu module.
+ *
+ * @hdev: pointer to the habanalabs device structure
+ *
+ * This function does the following:
+ * - Disable mmu in hw
+ * - free the pgts pool
+ *
+ * All ctxs should be freed before calling this func
+ */
+void hl_mmu_fini(struct hl_device *hdev)
+{
+     if (!hdev->mmu_enable)
+             return;
+
+     gen_pool_destroy(hdev->mmu_pgt_pool);
+
+     mutex_destroy(&hdev->mmu_cache_lock);
+
+     /* MMU HW fini will be done in device hw_fini() */
+}
+
+/*
+ * hl_mmu_ctx_init - init a ctx for using the mmu module
+ *
+ * @ctx: pointer to the context structure
+ *
+ * This function does the following:
+ * - Init a mutex to protect the concurrent mapping flow
+ * - Init a hash to hold all pgts related to this ctx
+ */
+void hl_mmu_ctx_init(struct hl_ctx *ctx)
+{
+     if (!ctx->hdev->mmu_enable)
+             return;
+
+     mutex_init(&ctx->mmu_lock);
+     hash_init(ctx->mmu_hash);
+}
+
+/*
+ * hl_mmu_ctx_fini - disable a ctx from using the mmu module
+ *
+ * @ctx: pointer to the context structure
+ *
+ * This function does the following:
+ * - Free any pgts which were not freed yet
+ * - Free the mutex
+ */
+void hl_mmu_ctx_fini(struct hl_ctx *ctx)
+{
+     struct pgt_info *pgt_info;
+     struct hlist_node *tmp;
+     int i;
+
+     if (!ctx->hdev->mmu_enable)
+             return;
+
+     if (!hash_empty(ctx->mmu_hash))
+             dev_err(ctx->hdev->dev,
+                             "ctx is freed while it has pgts in use\n");
+
+     hash_for_each_safe(ctx->mmu_hash, i, tmp, pgt_info, node) {
+             dev_err(ctx->hdev->dev,
+                     "pgt_info of addr 0x%llx of asid %d was not destroyed, num_ptes: %d\n",
+                     pgt_info->addr, ctx->asid, pgt_info->num_of_ptes);
+             free_hop(ctx, pgt_info->addr);
+     }
+
+     mutex_destroy(&ctx->mmu_lock);
+}
+
+/*
+ * hl_mmu_map - maps a virtual addr to physical addr
+ *
+ * @ctx: pointer to the context structure
+ * @virt_addr: virt addr to map from
+ * @phys_addr: phys addr to map to
+ * @page_size: physical page size
+ *
+ * This function does the following:
+ * - Check that the virt addr is not mapped
+ * - Allocate pgts as necessary in order to map the virt addr to the phys
+ * - Returns 0 on success, -EINVAL if addr is already mapped, or -ENOMEM.
+ *
+ * Because this function changes the page tables in the device and because it
+ * changes the MMU hash, it must be protected by a lock.
+ * However, because it maps only a single page, the lock should be implemented
+ * in a higher level in order to protect the entire mapping of the memory area
+ */
+int hl_mmu_map(struct hl_ctx *ctx, u64 virt_addr, u64 phys_addr, u32 page_size)
+{
+     struct hl_device *hdev = ctx->hdev;
+     u64 hop0_addr = 0, hop0_pte_addr = 0,
+             hop1_addr = 0, hop1_pte_addr = 0,
+             hop2_addr = 0, hop2_pte_addr = 0,
+             hop3_addr = 0, hop3_pte_addr = 0,
+             hop4_addr = 0, hop4_pte_addr = 0,
+             curr_pte = 0;
+     int hop1_new = 0, hop2_new = 0, hop3_new = 0, hop4_new = 0,
+                     rc = -ENOMEM;
+     bool is_huge;
+
+     if (!hdev->mmu_enable)
+             return 0;
+
+     /*
+      * This mapping function can map a 4KB/2MB page. For 2MB page there are
+      * only 3 hops rather than 4. Currently the DRAM allocation uses 2MB
+      * pages only but user memory could have been allocated with one of the
+      * two page sizes. Since this is a common code for all the three cases,
+      * we need this hugs page check.
+      */
+     is_huge = page_size == PAGE_SIZE_2MB;
+
+     hop0_addr = get_hop0_addr(ctx);
+
+     hop0_pte_addr = get_hop0_pte_addr(ctx, hop0_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop0_pte_addr);
+
+     hop1_addr = get_next_hop_addr(curr_pte);
+
+     if (hop1_addr == ULLONG_MAX) {
+             hop1_addr = alloc_hop(ctx);
Quoted 9 lines

drivers/misc/habanalabs/mmu.c line 332

Quoted 5 lines
drivers/misc/habanalabs/mmu.c new file +5 -0
+             if (hop1_addr == ULLONG_MAX)
+                     goto err;
+             else
+                     hop1_new = 1;
+     }

The allocation can be folded into get_alloc_next_hop_addr() along with get_next_hop_addr().

Fixed

Quoted 111 lines
Quoted 111 lines
drivers/misc/habanalabs/mmu.c new file +111 -0
+
+     hop1_pte_addr = get_hop1_pte_addr(ctx, hop1_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop1_pte_addr);
+
+     hop2_addr = get_next_hop_addr(curr_pte);
+
+     if (hop2_addr == ULLONG_MAX) {
+             hop2_addr = alloc_hop(ctx);
+             if (hop2_addr == ULLONG_MAX)
+                     goto err;
+             else
+                     hop2_new = 1;
+     }
+
+     hop2_pte_addr = get_hop2_pte_addr(ctx, hop2_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop2_pte_addr);
+
+     hop3_addr = get_next_hop_addr(curr_pte);
+
+     if (hop3_addr == ULLONG_MAX) {
+             hop3_addr = alloc_hop(ctx);
+             if (hop3_addr == ULLONG_MAX)
+                     goto err;
+             else
+                     hop3_new = 1;
+     }
+
+     hop3_pte_addr = get_hop3_pte_addr(ctx, hop3_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop3_pte_addr);
+
+     if (!is_huge) {
+             hop4_addr = get_next_hop_addr(curr_pte);
+
+             if (hop4_addr == ULLONG_MAX) {
+                     hop4_addr = alloc_hop(ctx);
+                     if (hop4_addr == ULLONG_MAX)
+                             goto err;
+                     else
+                             hop4_new = 1;
+             }
+
+             hop4_pte_addr = get_hop4_pte_addr(ctx, hop4_addr, virt_addr);
+
+             curr_pte = hdev->asic_funcs->read_pte(hdev, hop4_pte_addr);
+     }
+
+     if (curr_pte & PAGE_PRESENT_MASK) {
+             dev_err(hdev->dev,
+                             "mapping already exists for virt_addr 0x%llx\n",
+                                     virt_addr);
+
+             dev_dbg(hdev->dev, "hop0 pte: 0x%llx (0x%llx)\n",
+                             hdev->asic_funcs->read_pte(hdev, hop0_pte_addr),
+                             hop0_pte_addr);
+             dev_dbg(hdev->dev, "hop1 pte: 0x%llx (0x%llx)\n",
+                             hdev->asic_funcs->read_pte(hdev, hop1_pte_addr),
+                             hop1_pte_addr);
+             dev_dbg(hdev->dev, "hop2 pte: 0x%llx (0x%llx)\n",
+                             hdev->asic_funcs->read_pte(hdev, hop2_pte_addr),
+                             hop2_pte_addr);
+             dev_dbg(hdev->dev, "hop3 pte: 0x%llx (0x%llx)\n",
+                             hdev->asic_funcs->read_pte(hdev, hop3_pte_addr),
+                             hop3_pte_addr);
+
+             if (!is_huge)
+                     dev_dbg(hdev->dev, "hop4 pte: 0x%llx (0x%llx)\n",
+                             hdev->asic_funcs->read_pte(hdev,
+                                                     hop4_pte_addr),
+                                                     hop4_pte_addr);
+
+             rc = EINVAL;
+             goto err;
+     }
+
+     curr_pte = (phys_addr & PTE_PHYS_ADDR_MASK) | LAST_MASK
+                     | PAGE_PRESENT_MASK;
+
+     hdev->asic_funcs->write_pte(hdev,
+                             is_huge ? hop3_pte_addr : hop4_pte_addr,
+                             curr_pte);
+
+     if (hop1_new) {
+             curr_pte = (hop1_addr & PTE_PHYS_ADDR_MASK) |
+                             PAGE_PRESENT_MASK;
+             ctx->hdev->asic_funcs->write_pte(ctx->hdev, hop0_pte_addr,
+                             curr_pte);
+     }
+     if (hop2_new) {
+             curr_pte = (hop2_addr & PTE_PHYS_ADDR_MASK) |
+                             PAGE_PRESENT_MASK;
+             ctx->hdev->asic_funcs->write_pte(ctx->hdev, hop1_pte_addr,
+                             curr_pte);
+             inc_num_of_ptes(ctx, hop1_addr);
+     }
+     if (hop3_new) {
+             curr_pte = (hop3_addr & PTE_PHYS_ADDR_MASK) |
+                             PAGE_PRESENT_MASK;
+             ctx->hdev->asic_funcs->write_pte(ctx->hdev, hop2_pte_addr,
+                             curr_pte);
+             inc_num_of_ptes(ctx, hop2_addr);
+     }
+
+     if (!is_huge) {
+             if (hop4_new) {
+                     curr_pte = (hop4_addr & PTE_PHYS_ADDR_MASK) |
+                                     PAGE_PRESENT_MASK;
+                     ctx->hdev->asic_funcs->write_pte(ctx->hdev,
+                                     hop3_pte_addr, curr_pte);
Quoted 9 lines

drivers/misc/habanalabs/mmu.c line 449

Quoted 6 lines
drivers/misc/habanalabs/mmu.c new file +6 -0
+                     inc_num_of_ptes(ctx, hop3_addr);
+             }
+
+             inc_num_of_ptes(ctx, hop4_addr);
+     } else
+             inc_num_of_ptes(ctx, hop3_addr);

Isn't hop3 would be incremented twice for 2M pages?

Why ? 2M pages means is_huge, so you will go to the else part of the if above and inc the ptes of hop3 only once there.
The other inc happens if !is_huge, which is for 4K pages.

Quoted 121 lines
Quoted 108 lines
drivers/misc/habanalabs/mmu.c new file +108 -0
+
+     /* flush all writes from all cores to reach PCI */
+     mb();
+
+     hdev->asic_funcs->read_pte(hdev,
+                             is_huge ? hop3_pte_addr : hop4_pte_addr);
+
+     return 0;
+
+err:
+     if (hop4_new)
+             free_hop(ctx, hop4_addr);
+     if (hop3_new)
+             free_hop(ctx, hop3_addr);
+     if (hop2_new)
+             free_hop(ctx, hop2_addr);
+     if (hop1_new)
+             free_hop(ctx, hop1_addr);
+
+     return rc;
+}
+
+/*
+ * hl_mmu_unmap - unmaps a virtual addr
+ *
+ * @ctx: pointer to the context structure
+ * @virt_addr: virt addr to map from
+ *
+ * This function does the following:
+ * - Check that the virt addr is mapped
+ * - Unmap the vurt addr and frees pgts if possible
+ * - Returns 0 on success, -EINVAL if the given addr is not mapped
+ *
+ * Because this function changes the page tables in the device and because it
+ * changes the MMU hash, it must be protected by a lock.
+ * However, because it maps only a single page, the lock should be implemented
+ * in a higher level in order to protect the entire mapping of the memory area
+ */
+int hl_mmu_unmap(struct hl_ctx *ctx, u64 virt_addr)
+{
+     struct hl_device *hdev = ctx->hdev;
+     u64 hop0_addr = 0, hop0_pte_addr = 0,
+             hop1_addr = 0, hop1_pte_addr = 0,
+             hop2_addr = 0, hop2_pte_addr = 0,
+             hop3_addr = 0, hop3_pte_addr = 0,
+             hop4_addr = 0, hop4_pte_addr = 0,
+             curr_pte;
+     int clear_hop3 = 1;
+
+     if (!hdev->mmu_enable)
+             return 0;
+
+     hop0_addr = get_hop0_addr(ctx);
+
+     hop0_pte_addr = get_hop0_pte_addr(ctx, hop0_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop0_pte_addr);
+
+     hop1_addr = get_next_hop_addr(curr_pte);
+
+     if (hop1_addr == ULLONG_MAX)
+             goto not_mapped;
+
+     hop1_pte_addr = get_hop1_pte_addr(ctx, hop1_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop1_pte_addr);
+
+     hop2_addr = get_next_hop_addr(curr_pte);
+
+     if (hop2_addr == ULLONG_MAX)
+             goto not_mapped;
+
+     hop2_pte_addr = get_hop2_pte_addr(ctx, hop2_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop2_pte_addr);
+
+     hop3_addr = get_next_hop_addr(curr_pte);
+
+     if (hop3_addr == ULLONG_MAX)
+             goto not_mapped;
+
+     hop3_pte_addr = get_hop3_pte_addr(ctx, hop3_addr, virt_addr);
+
+     curr_pte = hdev->asic_funcs->read_pte(hdev, hop3_pte_addr);
+
+     if (!(curr_pte & LAST_MASK)) {
+             hop4_addr = get_next_hop_addr(curr_pte);
+
+             if (hop4_addr == ULLONG_MAX)
+                     goto not_mapped;
+
+             hop4_pte_addr = get_hop4_pte_addr(ctx, hop4_addr, virt_addr);
+
+             curr_pte = hdev->asic_funcs->read_pte(hdev, hop4_pte_addr);
+
+             clear_hop3 = 0;
+     }
+
+     if (!(curr_pte & PAGE_PRESENT_MASK))
+             goto not_mapped;
+
+     clear_pte(hdev, hop4_addr ? hop4_pte_addr : hop3_pte_addr);
+
+     if (hop4_addr && dec_num_of_ptes(ctx, hop4_addr) == HOP_FREED)
+             clear_hop3 = 1;
+
+     if (clear_hop3) {
+             clear_pte(hdev, hop3_pte_addr);

drivers/misc/habanalabs/mmu.c line 567

Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10 -0
+             if (dec_num_of_ptes(ctx, hop3_addr) == HOP_FREED) {
+                     clear_pte(hdev, hop2_pte_addr);
+                     if (dec_num_of_ptes(ctx, hop2_addr) == HOP_FREED) {
+                             clear_pte(hdev, hop1_pte_addr);
+                             if (dec_num_of_ptes(ctx, hop1_addr) ==
+                                             HOP_FREED)
+                                     clear_pte(hdev, hop0_pte_addr);
+                     }
+             }
+     }

Consider inverting the logic and doing something like

Quoted 7 lines
        if (dec_num_of_ptes() != HOP_FREED)
                goto flush;
        clear_pte(...)
        if (...)
                goto flush;

flush:

Yes, it does look more neat that way. Fixed

Quoted 26 lines
Quoted 15 lines
drivers/misc/habanalabs/mmu.c new file +15 -0
+
+     /* flush all writes from all cores to reach PCI */
+     mb();
+
+     hdev->asic_funcs->read_pte(hdev,
+                             hop4_addr ? hop4_pte_addr : hop3_pte_addr);
+
+     return 0;
+
+not_mapped:
+     dev_err(hdev->dev, "virt addr 0x%llx is not mapped to phys addr\n",
+             virt_addr);
+
+     return -EINVAL;
+}

[ ... ]

Quoted 3 lines

--
2.17.1

--
Sincerely yours,
Mike.

maintainer
#5 lore raw

Re: [PATCH v3 05/15] habanalabs: add command buffer module

In reply to Oded Gabbay

drivers/misc/habanalabs/command_buffer.c line 201

Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/command_buffer.c new file +11 -0
+int hl_cb_ioctl(struct hl_fpriv *hpriv, void *data)
+{
+	union hl_cb_args *args = data;
+	struct hl_device *hdev = hpriv->hdev;
+	u64 handle;
+	int rc;
+
+	switch (args->in.op) {
+	case HL_CB_OP_CREATE:
+		rc = hl_cb_create(hdev, &hpriv->cb_mgr, args->in.cb_size,
+					&handle, hpriv->ctx->asid);

so cb_size comes from userspace, ok, you check for the value to be too small, but not too big. That means someone can try to allocate too much memory, possibly crashing things, not good :(

drivers/misc/habanalabs/command_buffer.c line 208

Quoted 7 lines
drivers/misc/habanalabs/command_buffer.c new file +7 -0
+		memset(args, 0, sizeof(*args));
+		args->out.cb_handle = handle;
+		break;
+	case HL_CB_OP_DESTROY:
+		rc = hl_cb_destroy(hdev, &hpriv->cb_mgr,
+					args->in.cb_handle);
+		memset(args, 0, sizeof(*args));

Why zero this if it's not copied back to userspace?

drivers/misc/habanalabs/command_buffer.c line 211

Quoted 3 lines
drivers/misc/habanalabs/command_buffer.c new file +3 -0
+		break;
+	default:
+		rc = -EINVAL;

-ENOTTY is normally the "invalid ioctl value", right?

drivers/misc/habanalabs/command_buffer.c line 216

Quoted 5 lines
drivers/misc/habanalabs/command_buffer.c new file +5 -0
+		break;
+	}
+
+	return rc;
+}

thanks,

greg k-h

maintainer
#12 lore raw

Re: [PATCH v3 09/15] habanalabs: add sysfs and hwmon support

In reply to Oded Gabbay
Quoted 49 lines · Oded Gabbay

This patch add the sysfs and hwmon entries that are exposed by the driver.

Goya has several sensors, from various categories such as temperature, voltage, current, etc. The driver exposes those sensors in the standard hwmon mechanism.

In addition, the driver exposes a couple of interfaces in sysfs, both for configuration and for providing status of the device or driver.

The configuration attributes is for Power Management:
- Automatic or manual
- Frequency value when moving to high frequency mode
- Maximum power the device is allowed to consume

The rest of the attributes are read-only and provide the following information:
- Versions of the various firmwares running on the device
- Contents of the device's EEPROM
- The device type (currently only Goya is supported)
- PCI address of the device (to allow user-space to connect between /dev/hlX to PCI address)
- Status of the device (operational, malfunction, in_reset)
- How many processes are open on the device's file

---
 .../ABI/testing/sysfs-driver-habanalabs       | 190 ++++++
 drivers/misc/habanalabs/Makefile              |   2 +-
 drivers/misc/habanalabs/device.c              | 146 +++++
 drivers/misc/habanalabs/goya/Makefile         |   2 +-
 drivers/misc/habanalabs/goya/goya.c           | 230 +++++++
 drivers/misc/habanalabs/goya/goyaP.h          |  21 +
 drivers/misc/habanalabs/goya/goya_hwmgr.c     | 306 +++++++++
 drivers/misc/habanalabs/habanalabs.h          | 101 +++
 drivers/misc/habanalabs/habanalabs_drv.c      |   7 +
 drivers/misc/habanalabs/hwmon.c               | 449 +++++++++++++
 drivers/misc/habanalabs/sysfs.c               | 589 ++++++++++++++++++
 11 files changed, 2041 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c
 create mode 100644 drivers/misc/habanalabs/hwmon.c
 create mode 100644 drivers/misc/habanalabs/sysfs.c
Documentation/ABI/testing/sysfs-driver-habanalabs new file +0 -0
new file mode 100644
index 000000000000..19edd4da87c1

Documentation/ABI/testing/sysfs-driver-habanalabs line 5

Quoted 5 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5 -0
@@ -0,0 +1,190 @@
+What:           /sys/class/habanalabs/hl<n>/armcp_kernel_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the Linux kernel running on the device's CPU

Hey, nice! We can see how old that kernel gets over time :)

Documentation/ABI/testing/sysfs-driver-habanalabs line 148

Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6 -0
+What:           /sys/class/habanalabs/hl<n>/soft_reset_cnt
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays how many times the device have undergone a soft-reset
+                operation

"how many times" since when? Power on? Kernel boot? Driver load?

Quoted 5 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5 -0
+static ssize_t mme_clk_show(struct device *dev, struct device_attribute *attr,
+				char *buf)
+{
+	struct hl_device *hdev = dev_get_drvdata(dev);
+	long value;

Documentation/ABI/testing/sysfs-driver-habanalabs line 26

Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10 -0
+
+	if (hdev->disabled)
+		return -ENODEV;
+
+	value = hl_get_frequency(hdev, MME_PLL, false);
+
+	if (value < 0)
+		return value;
+
+	return snprintf(buf, PAGE_SIZE, "%lu\n", value);

Meta-comment. You do this in all of your sysfs show functions, and I understand the "I want to be safe!" feeling here, but you should just use sprintf(). The size of a sysfs buffer is PAGE_SIZE, and you should never even get close to it if you are only writing a single numeric value.

So no need to do anything fancy, just use sprintf() please.

Documentation/ABI/testing/sysfs-driver-habanalabs line 32

Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6 -0
+static DEVICE_ATTR_RW(mme_clk);
+static DEVICE_ATTR_RW(tpc_clk);
+static DEVICE_ATTR_RW(ic_clk);
+static DEVICE_ATTR_RO(mme_clk_curr);
+static DEVICE_ATTR_RO(tpc_clk_curr);
+static DEVICE_ATTR_RO(ic_clk_curr);

Some people like to put the macro right under the show/store functions, to keep them all in one place. Makes it easier to change/add things later over time.

Quoted 32 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +32 -0
+int goya_add_device_attr(struct hl_device *hdev)
+{
+	int rc;
+
+	rc = device_create_file(hdev->dev, &dev_attr_mme_clk);
+	if (rc) {
+		dev_err(hdev->dev, "failed to create device file mme_clk\n");
+		return rc;
+	}
+
+	rc = device_create_file(hdev->dev, &dev_attr_tpc_clk);
+	if (rc) {
+		dev_err(hdev->dev, "failed to create device file tpc_clk\n");
+		goto remove_mme_clk;
+	}
+
+	rc = device_create_file(hdev->dev, &dev_attr_ic_clk);
+	if (rc) {
+		dev_err(hdev->dev, "failed to create device file ic_clk\n");
+		goto remove_tpc_clk;
+	}
+
+	rc = device_create_file(hdev->dev, &dev_attr_mme_clk_curr);
+	if (rc) {
+		dev_err(hdev->dev,
+			"failed to create device file mme_clk_curr\n");
+		goto remove_ic_clk;
+	}
+
+	rc = device_create_file(hdev->dev, &dev_attr_tpc_clk_curr);
+	if (rc) {
+		dev_err(hdev->dev,

Documentation/ABI/testing/sysfs-driver-habanalabs line 74

Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10 -0
+			"failed to create device file tpc_clk_curr\n");
+		goto remove_mme_clk_curr;
+	}
+
+	rc = device_create_file(hdev->dev, &dev_attr_ic_clk_curr);
+	if (rc) {
+		dev_err(hdev->dev,
+			"failed to create device file ic_clk_curr\n");
+		goto remove_tpc_clk_curr;
+	}

You do know about attribute groups, right? Please use them so you don't have to hand-roll the "add a bunch of files" logic here and elsewhere. The driver core will just automatically add and remove all of the files when the device is bound properly without you having to do anything. That saves you a lot of code and debugging logic.

Same for your other lists of attributes, they can all be handled automatically, no need for this type of logic.

thanks,

greg k-h

maintainer
#22 lore raw

Re: [PATCH v3 14/15] habanalabs: add debugfs support

In reply to Oded Gabbay

drivers/misc/habanalabs/debugfs.c line 1066

Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/debugfs.c new file +11 -0
+int __init hl_debugfs_init(void)
+{
+	hl_debug_root = debugfs_create_dir("habanalabs", NULL);
+	if (IS_ERR_OR_NULL(hl_debug_root)) {
+		pr_err("can not create debugfs directory\n");
+		hl_debug_root = NULL;
+		return -ENOMEM;
+	}
+
+	return 0;
+}

This function should just be one line:
void __init hl_debugfs_init(void)

{
	hl_debug_root = debugfs_create_dir("habanalabs", NULL);
}

No need to do anything else, or check anything.

thanks,

greg k-h

#6 lore raw

Re: [PATCH v3 05/15] habanalabs: add command buffer module

In reply to Greg KH
Quoted 17 lines · Greg KH

drivers/misc/habanalabs/command_buffer.c line 201

Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/command_buffer.c new file +11 -0
+int hl_cb_ioctl(struct hl_fpriv *hpriv, void *data)
+{
+     union hl_cb_args *args = data;
+     struct hl_device *hdev = hpriv->hdev;
+     u64 handle;
+     int rc;
+
+     switch (args->in.op) {
+     case HL_CB_OP_CREATE:
+             rc = hl_cb_create(hdev, &hpriv->cb_mgr, args->in.cb_size,
+                                     &handle, hpriv->ctx->asid);

so cb_size comes from userspace, ok, you check for the value to be too small, but not too big. That means someone can try to allocate too much memory, possibly crashing things, not good :(

Yes, correct, but even if I limit a single allocation to, let's say, 1MB, what's stopping a userspace process from allocating multiple CBs and draining the system memory ? I'm counting on the oom module to kill that process if it mis-behaves.
And, btw, I assumed there is hard limit of 4MB on a single dma_alloc_coherent. i.e. I was never able to allocate more then 4MB through that API. So I never thought I need to check for max size because of that hard limit.
Am I missing something here ?

Quoted 11 lines

drivers/misc/habanalabs/command_buffer.c line 208

Quoted 9 lines
drivers/misc/habanalabs/command_buffer.c new file +7 -0
+             memset(args, 0, sizeof(*args));
+             args->out.cb_handle = handle;
+             break;
+     case HL_CB_OP_DESTROY:
+             rc = hl_cb_destroy(hdev, &hpriv->cb_mgr,
+                                     args->in.cb_handle);
+             memset(args, 0, sizeof(*args));

Why zero this if it's not copied back to userspace?

Fixed

Quoted 8 lines

drivers/misc/habanalabs/command_buffer.c line 211

Quoted 6 lines
drivers/misc/habanalabs/command_buffer.c new file +3 -0
+             break;
+     default:
+             rc = -EINVAL;

-ENOTTY is normally the "invalid ioctl value", right?

Fixed

Quoted 10 lines

drivers/misc/habanalabs/command_buffer.c line 216

Quoted 6 lines
drivers/misc/habanalabs/command_buffer.c new file +5 -0
+             break;
+     }
+
+     return rc;
+}

thanks,

greg k-h

#13 lore raw

Re: [PATCH v3 09/15] habanalabs: add sysfs and hwmon support

In reply to Greg KH
Quoted 59 lines · Greg KH
Quoted 49 lines · Oded Gabbay

This patch add the sysfs and hwmon entries that are exposed by the driver.

Goya has several sensors, from various categories such as temperature, voltage, current, etc. The driver exposes those sensors in the standard hwmon mechanism.

In addition, the driver exposes a couple of interfaces in sysfs, both for configuration and for providing status of the device or driver.

The configuration attributes is for Power Management:
- Automatic or manual
- Frequency value when moving to high frequency mode
- Maximum power the device is allowed to consume

The rest of the attributes are read-only and provide the following information:
- Versions of the various firmwares running on the device
- Contents of the device's EEPROM
- The device type (currently only Goya is supported)
- PCI address of the device (to allow user-space to connect between /dev/hlX to PCI address)
- Status of the device (operational, malfunction, in_reset)
- How many processes are open on the device's file

---
 .../ABI/testing/sysfs-driver-habanalabs       | 190 ++++++
 drivers/misc/habanalabs/Makefile              |   2 +-
 drivers/misc/habanalabs/device.c              | 146 +++++
 drivers/misc/habanalabs/goya/Makefile         |   2 +-
 drivers/misc/habanalabs/goya/goya.c           | 230 +++++++
 drivers/misc/habanalabs/goya/goyaP.h          |  21 +
 drivers/misc/habanalabs/goya/goya_hwmgr.c     | 306 +++++++++
 drivers/misc/habanalabs/habanalabs.h          | 101 +++
 drivers/misc/habanalabs/habanalabs_drv.c      |   7 +
 drivers/misc/habanalabs/hwmon.c               | 449 +++++++++++++
 drivers/misc/habanalabs/sysfs.c               | 589 ++++++++++++++++++
 11 files changed, 2041 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c
 create mode 100644 drivers/misc/habanalabs/hwmon.c
 create mode 100644 drivers/misc/habanalabs/sysfs.c
Documentation/ABI/testing/sysfs-driver-habanalabs new file +0 -0
new file mode 100644
index 000000000000..19edd4da87c1

Documentation/ABI/testing/sysfs-driver-habanalabs line 5

Quoted 5 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5 -0
@@ -0,0 +1,190 @@
+What:           /sys/class/habanalabs/hl<n>/armcp_kernel_ver
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Version of the Linux kernel running on the device's CPU

Hey, nice! We can see how old that kernel gets over time :)

Quoted 8 lines

Documentation/ABI/testing/sysfs-driver-habanalabs line 148

Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6 -0
+What:           /sys/class/habanalabs/hl<n>/soft_reset_cnt
+Date:           Jan 2019
+KernelVersion:  5.1
+Contact:        [email protected]
+Description:    Displays how many times the device have undergone a soft-reset
+                operation

"how many times" since when? Power on? Kernel boot? Driver load?

Thanks, fixed that (Driver load).

Quoted 23 lines
Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5 -0
+static ssize_t mme_clk_show(struct device *dev, struct device_attribute *attr,
+                             char *buf)
+{
+     struct hl_device *hdev = dev_get_drvdata(dev);
+     long value;

Documentation/ABI/testing/sysfs-driver-habanalabs line 26

Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10 -0
+
+     if (hdev->disabled)
+             return -ENODEV;
+
+     value = hl_get_frequency(hdev, MME_PLL, false);
+
+     if (value < 0)
+             return value;
+
+     return snprintf(buf, PAGE_SIZE, "%lu\n", value);

Meta-comment. You do this in all of your sysfs show functions, and I understand the "I want to be safe!" feeling here, but you should just use sprintf(). The size of a sysfs buffer is PAGE_SIZE, and you should never even get close to it if you are only writing a single numeric value.

Quoted 1 line

So no need to do anything fancy, just use sprintf() please.

Got it, fixed.

Quoted 11 lines

Documentation/ABI/testing/sysfs-driver-habanalabs line 32

Quoted 7 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6 -0
+static DEVICE_ATTR_RW(mme_clk);
+static DEVICE_ATTR_RW(tpc_clk);
+static DEVICE_ATTR_RW(ic_clk);
+static DEVICE_ATTR_RO(mme_clk_curr);
+static DEVICE_ATTR_RO(tpc_clk_curr);
+static DEVICE_ATTR_RO(ic_clk_curr);

Some people like to put the macro right under the show/store functions, to keep them all in one place. Makes it easier to change/add things later over time.

I really prefer it this way, you get a clear picture of all the ATTR you expose without the need to go over the entire file. If you don't mind I want to leave this as it is.

Quoted 51 lines
Quoted 33 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +32 -0
+int goya_add_device_attr(struct hl_device *hdev)
+{
+     int rc;
+
+     rc = device_create_file(hdev->dev, &dev_attr_mme_clk);
+     if (rc) {
+             dev_err(hdev->dev, "failed to create device file mme_clk\n");
+             return rc;
+     }
+
+     rc = device_create_file(hdev->dev, &dev_attr_tpc_clk);
+     if (rc) {
+             dev_err(hdev->dev, "failed to create device file tpc_clk\n");
+             goto remove_mme_clk;
+     }
+
+     rc = device_create_file(hdev->dev, &dev_attr_ic_clk);
+     if (rc) {
+             dev_err(hdev->dev, "failed to create device file ic_clk\n");
+             goto remove_tpc_clk;
+     }
+
+     rc = device_create_file(hdev->dev, &dev_attr_mme_clk_curr);
+     if (rc) {
+             dev_err(hdev->dev,
+                     "failed to create device file mme_clk_curr\n");
+             goto remove_ic_clk;
+     }
+
+     rc = device_create_file(hdev->dev, &dev_attr_tpc_clk_curr);
+     if (rc) {
+             dev_err(hdev->dev,

Documentation/ABI/testing/sysfs-driver-habanalabs line 74

Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10 -0
+                     "failed to create device file tpc_clk_curr\n");
+             goto remove_mme_clk_curr;
+     }
+
+     rc = device_create_file(hdev->dev, &dev_attr_ic_clk_curr);
+     if (rc) {
+             dev_err(hdev->dev,
+                     "failed to create device file ic_clk_curr\n");
+             goto remove_tpc_clk_curr;
+     }

You do know about attribute groups, right? Please use them so you don't have to hand-roll the "add a bunch of files" logic here and elsewhere. The driver core will just automatically add and remove all of the files when the device is bound properly without you having to do anything. That saves you a lot of code and debugging logic.

Quoted 2 lines

Same for your other lists of attributes, they can all be handled automatically, no need for this type of logic.

ah... hmm... oops, didn't know about them. feelsbadman :( I don't think they are used in drm (or at least not in amd) which is where most of my kernel knowledge comes from.
Indeed, much more elegant. Fixed the code.

Quoted 4 lines

thanks,

greg k-h

#23 lore raw

Re: [PATCH v3 14/15] habanalabs: add debugfs support

In reply to Greg KH
Quoted 16 lines · Greg KH

drivers/misc/habanalabs/debugfs.c line 1066

Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/debugfs.c new file +11 -0
+int __init hl_debugfs_init(void)
+{
+     hl_debug_root = debugfs_create_dir("habanalabs", NULL);
+     if (IS_ERR_OR_NULL(hl_debug_root)) {
+             pr_err("can not create debugfs directory\n");
+             hl_debug_root = NULL;
+             return -ENOMEM;
+     }
+
+     return 0;
+}

This function should just be one line:
void __init hl_debugfs_init(void)

Quoted 6 lines
{
        hl_debug_root = debugfs_create_dir("habanalabs", NULL);
}

No need to do anything else, or check anything.

Correct, fixed.

Quoted 3 lines

thanks,

greg k-h

#26 lore raw
In reply to Mike Rapoport
Quoted 36 lines · Mike Rapoport
Quoted 34 lines · Oded Gabbay

Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.

Link to v2 cover letter: https://lkml.org/lkml/2019/1/30/1003

Link to v1 cover letter: https://lwn.net/Articles/777342/

I would appricate any feedback, question and/or review.

Thanks,
Oded

p.s. for those who prefer to clone the tree instead of looking at the emails, you can grab a copy from our company's page in GitHub:

https://github.com/HabanaAI/linux/releases/tag/hl_patchset_v3_20190204

Oded Gabbay (14):
habanalabs: add skeleton driver
habanalabs: add Goya registers header files
habanalabs: add basic Goya support
habanalabs: add context and ASID modules
habanalabs: add command buffer module
habanalabs: add basic Goya h/w initialization
habanalabs: add h/w queues module
habanalabs: add event queue and interrupts
habanalabs: add sysfs and hwmon support
habanalabs: add device reset support
habanalabs: add command submission module
habanalabs: implement INFO IOCTL
habanalabs: add debugfs support
Update MAINTAINERS and CREDITS with habanalabs info

Quoted 8 lines
Quoted 2 lines

Omer Shpigelman (1):
habanalabs: add virtual memory and MMU modules

Nit: git am is unhappy about black lines at EOF, e.g:

Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +

Yes, but on the other hand, joe perches gave me a comment about missing newline at end of Makefile.
So either diff will complain or git am.

Quoted 5 lines

Aside from that, for patches 1, 3-11 and 13 feel free to add

Thanks for the review!
Oded

Quoted 239 lines

As for the "habanalabs: add virtual memory and MMU modules" and the debugfs patch I'll need more time to review the changes related to 2M pages.

Quoted 228 lines
 CREDITS                                       |    2 +-
 .../ABI/testing/debugfs-driver-habanalabs     |  127 +
 .../ABI/testing/sysfs-driver-habanalabs       |  190 +
 MAINTAINERS                                   |    9 +
 drivers/misc/Kconfig                          |    1 +
 drivers/misc/Makefile                         |    1 +
 drivers/misc/habanalabs/Kconfig               |   22 +
 drivers/misc/habanalabs/Makefile              |   15 +
 drivers/misc/habanalabs/asid.c                |   58 +
 drivers/misc/habanalabs/command_buffer.c      |  432 ++
 drivers/misc/habanalabs/command_submission.c  |  782 +++
 drivers/misc/habanalabs/context.c             |  216 +
 drivers/misc/habanalabs/debugfs.c             | 1071 ++++
 drivers/misc/habanalabs/device.c              | 1111 ++++
 drivers/misc/habanalabs/goya/Makefile         |    4 +
 drivers/misc/habanalabs/goya/goya.c           | 5344 +++++++++++++++++
 drivers/misc/habanalabs/goya/goyaP.h          |  193 +
 drivers/misc/habanalabs/goya/goya_hwmgr.c     |  306 +
 drivers/misc/habanalabs/goya/goya_security.c  | 2999 +++++++++
 drivers/misc/habanalabs/habanalabs.h          | 1441 +++++
 drivers/misc/habanalabs/habanalabs_drv.c      |  465 ++
 drivers/misc/habanalabs/habanalabs_ioctl.c    |  234 +
 drivers/misc/habanalabs/hw_queue.c            |  636 ++
 drivers/misc/habanalabs/hwmon.c               |  449 ++
 drivers/misc/habanalabs/include/armcp_if.h    |  335 ++
 .../include/goya/asic_reg/cpu_ca53_cfg_regs.h |  213 +
 .../include/goya/asic_reg/cpu_if_regs.h       |  110 +
 .../include/goya/asic_reg/cpu_pll_regs.h      |  186 +
 .../include/goya/asic_reg/dma_ch_0_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_1_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_2_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_3_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_ch_4_regs.h     |  512 ++
 .../include/goya/asic_reg/dma_macro_regs.h    |  242 +
 .../include/goya/asic_reg/dma_nrtr_regs.h     |  380 ++
 .../include/goya/asic_reg/dma_qm_0_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_1_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_2_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_3_regs.h     |  543 ++
 .../include/goya/asic_reg/dma_qm_4_regs.h     |  543 ++
 .../include/goya/asic_reg/goya_blocks.h       | 1372 +++++
 .../include/goya/asic_reg/goya_masks.h        |  262 +
 .../include/goya/asic_reg/goya_regs.h         |   99 +
 .../include/goya/asic_reg/ic_pll_regs.h       |  186 +
 .../include/goya/asic_reg/mc_pll_regs.h       |  186 +
 .../include/goya/asic_reg/mme1_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme2_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme3_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme4_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme5_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme6_rtr_regs.h     |  876 +++
 .../include/goya/asic_reg/mme_cmdq_regs.h     |  431 ++
 .../include/goya/asic_reg/mme_qm_regs.h       |  543 ++
 .../include/goya/asic_reg/mme_regs.h          | 2422 ++++++++
 .../include/goya/asic_reg/mmu_regs.h          |  158 +
 .../include/goya/asic_reg/pci_nrtr_regs.h     |  380 ++
 .../include/goya/asic_reg/pcie_aux_regs.h     |  476 ++
 .../goya/asic_reg/psoc_emmc_pll_regs.h        |  186 +
 .../goya/asic_reg/psoc_global_conf_regs.h     | 1119 ++++
 .../include/goya/asic_reg/psoc_mme_pll_regs.h |  186 +
 .../include/goya/asic_reg/psoc_pci_pll_regs.h |  186 +
 .../include/goya/asic_reg/psoc_spi_regs.h     |  427 ++
 .../goya/asic_reg/sram_y0_x0_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x1_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x2_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x3_rtr_regs.h       |  215 +
 .../goya/asic_reg/sram_y0_x4_rtr_regs.h       |  215 +
 .../include/goya/asic_reg/stlb_regs.h         |  133 +
 .../include/goya/asic_reg/tpc0_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc0_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc0_eml_cfg_regs.h |  580 ++
 .../include/goya/asic_reg/tpc0_nrtr_regs.h    |  380 ++
 .../include/goya/asic_reg/tpc0_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc1_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc1_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc1_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc1_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc2_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc2_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc2_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc2_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc3_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc3_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc3_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc3_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc4_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc4_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc4_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc4_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc5_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc5_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc5_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc5_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc6_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc6_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc6_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc6_rtr_regs.h     |  848 +++
 .../include/goya/asic_reg/tpc7_cfg_regs.h     | 2110 +++++++
 .../include/goya/asic_reg/tpc7_cmdq_regs.h    |  431 ++
 .../include/goya/asic_reg/tpc7_nrtr_regs.h    |  380 ++
 .../include/goya/asic_reg/tpc7_qm_regs.h      |  543 ++
 .../include/goya/asic_reg/tpc_pll_regs.h      |  186 +
 drivers/misc/habanalabs/include/goya/goya.h   |   41 +
 .../include/goya/goya_async_events.h          |  186 +
 .../misc/habanalabs/include/goya/goya_fw_if.h |   28 +
 .../habanalabs/include/goya/goya_packets.h    |  129 +
 drivers/misc/habanalabs/include/hl_boot_if.h  |   30 +
 .../include/hw_ip/mmu/mmu_general.h           |   45 +
 .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h   |   15 +
 drivers/misc/habanalabs/include/qman_if.h     |   56 +
 drivers/misc/habanalabs/irq.c                 |  325 +
 drivers/misc/habanalabs/memory.c              | 1722 ++++++
 drivers/misc/habanalabs/mmu.c                 |  604 ++
 drivers/misc/habanalabs/sysfs.c               |  691 +++
 include/uapi/misc/habanalabs.h                |  444 ++
 115 files changed, 73533 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs
 create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs
 create mode 100644 drivers/misc/habanalabs/Kconfig
 create mode 100644 drivers/misc/habanalabs/Makefile
 create mode 100644 drivers/misc/habanalabs/asid.c
 create mode 100644 drivers/misc/habanalabs/command_buffer.c
 create mode 100644 drivers/misc/habanalabs/command_submission.c
 create mode 100644 drivers/misc/habanalabs/context.c
 create mode 100644 drivers/misc/habanalabs/debugfs.c
 create mode 100644 drivers/misc/habanalabs/device.c
 create mode 100644 drivers/misc/habanalabs/goya/Makefile
 create mode 100644 drivers/misc/habanalabs/goya/goya.c
 create mode 100644 drivers/misc/habanalabs/goya/goyaP.h
 create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c
 create mode 100644 drivers/misc/habanalabs/goya/goya_security.c
 create mode 100644 drivers/misc/habanalabs/habanalabs.h
 create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c
 create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c
 create mode 100644 drivers/misc/habanalabs/hw_queue.c
 create mode 100644 drivers/misc/habanalabs/hwmon.c
 create mode 100644 drivers/misc/habanalabs/include/armcp_if.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h
 create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h
 create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h
 create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h
 create mode 100644 drivers/misc/habanalabs/include/qman_if.h
 create mode 100644 drivers/misc/habanalabs/irq.c
 create mode 100644 drivers/misc/habanalabs/memory.c
 create mode 100644 drivers/misc/habanalabs/mmu.c
 create mode 100644 drivers/misc/habanalabs/sysfs.c
 create mode 100644 include/uapi/misc/habanalabs.h
Quoted 3 lines

--
2.17.1

--
Sincerely yours,
Mike.

#27 lore raw
Quoted 5 lines · Oded Gabbay
Quoted 4 lines · Mike Rapoport
Quoted 3 lines · Oded Gabbay

Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.

[]

Quoted 8 lines
Quoted 5 lines

Nit: git am is unhappy about black lines at EOF, e.g:

Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +

Yes, but on the other hand, joe perches gave me a comment about missing newline at end of Makefile.
So either diff will complain or git am.

Not really.

The desired modification was the last non-blank line of the Makefile should have a terminating newline, not that there was a blank line at EOF.

#28 lore raw
Quoted 17 lines · Joe Perches
Quoted 5 lines · Oded Gabbay
Quoted 4 lines · Mike Rapoport
Quoted 3 lines · Oded Gabbay

Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.

[]

Quoted 8 lines
Quoted 5 lines

Nit: git am is unhappy about black lines at EOF, e.g:

Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +

Yes, but on the other hand, joe perches gave me a comment about missing newline at end of Makefile.
So either diff will complain or git am.

Quoted 7 lines

Not really.

The desired modification was the last non-blank line of the Makefile should have a terminating newline, not that there was a blank line at EOF.

ok, thanks for the clarification, I'll fix that.
Oded

maintainer
#7 lore raw

Re: [PATCH v3 05/15] habanalabs: add command buffer module

In reply to Oded Gabbay
Quoted 18 lines · Oded Gabbay
Quoted 17 lines · Greg KH

drivers/misc/habanalabs/command_buffer.c line 201

Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/command_buffer.c new file +11 -0
+int hl_cb_ioctl(struct hl_fpriv *hpriv, void *data)
+{
+     union hl_cb_args *args = data;
+     struct hl_device *hdev = hpriv->hdev;
+     u64 handle;
+     int rc;
+
+     switch (args->in.op) {
+     case HL_CB_OP_CREATE:
+             rc = hl_cb_create(hdev, &hpriv->cb_mgr, args->in.cb_size,
+                                     &handle, hpriv->ctx->asid);

so cb_size comes from userspace, ok, you check for the value to be too small, but not too big. That means someone can try to allocate too much memory, possibly crashing things, not good :(

Quoted 9 lines

Yes, correct, but even if I limit a single allocation to, let's say, 1MB, what's stopping a userspace process from allocating multiple CBs and draining the system memory ? I'm counting on the oom module to kill that process if it mis-behaves.
And, btw, I assumed there is hard limit of 4MB on a single dma_alloc_coherent. i.e. I was never able to allocate more then 4MB through that API. So I never thought I need to check for max size because of that hard limit.
Am I missing something here ?

Relying on the oom module to handle driver issues is not always a wise idea ;)

You should put a bounds on your max allocation, if it really is 4MB, then test for that.

thanks,

greg k-h