Habana Labs kernel driver
Subsystems (2)
v3 cover Habana Labs kernel driver
Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.
Link to v2 cover letter: https://lkml.org/lkml/2019/1/30/1003
Link to v1 cover letter: https://lwn.net/Articles/777342/
I would appricate any feedback, question and/or review.
Thanks,
Oded
p.s. for those who prefer to clone the tree instead of looking at the emails, you can grab a copy from our company's page in GitHub:
https://github.com/HabanaAI/linux/releases/tag/hl_patchset_v3_20190204
Oded Gabbay (14):
habanalabs: add skeleton driver
habanalabs: add Goya registers header files
habanalabs: add basic Goya support
habanalabs: add context and ASID modules
habanalabs: add command buffer module
habanalabs: add basic Goya h/w initialization
habanalabs: add h/w queues module
habanalabs: add event queue and interrupts
habanalabs: add sysfs and hwmon support
habanalabs: add device reset support
habanalabs: add command submission module
habanalabs: implement INFO IOCTL
habanalabs: add debugfs support
Update MAINTAINERS and CREDITS with habanalabs info
Omer Shpigelman (1):
habanalabs: add virtual memory and MMU modules
CREDITS | 2 +- .../ABI/testing/debugfs-driver-habanalabs | 127 + .../ABI/testing/sysfs-driver-habanalabs | 190 + MAINTAINERS | 9 + drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/habanalabs/Kconfig | 22 + drivers/misc/habanalabs/Makefile | 15 + drivers/misc/habanalabs/asid.c | 58 + drivers/misc/habanalabs/command_buffer.c | 432 ++ drivers/misc/habanalabs/command_submission.c | 782 +++ drivers/misc/habanalabs/context.c | 216 + drivers/misc/habanalabs/debugfs.c | 1071 ++++ drivers/misc/habanalabs/device.c | 1111 ++++ drivers/misc/habanalabs/goya/Makefile | 4 + drivers/misc/habanalabs/goya/goya.c | 5344 +++++++++++++++++ drivers/misc/habanalabs/goya/goyaP.h | 193 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 306 + drivers/misc/habanalabs/goya/goya_security.c | 2999 +++++++++ drivers/misc/habanalabs/habanalabs.h | 1441 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 465 ++ drivers/misc/habanalabs/habanalabs_ioctl.c | 234 + drivers/misc/habanalabs/hw_queue.c | 636 ++ drivers/misc/habanalabs/hwmon.c | 449 ++ drivers/misc/habanalabs/include/armcp_if.h | 335 ++ .../include/goya/asic_reg/cpu_ca53_cfg_regs.h | 213 + .../include/goya/asic_reg/cpu_if_regs.h | 110 + .../include/goya/asic_reg/cpu_pll_regs.h | 186 + .../include/goya/asic_reg/dma_ch_0_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_1_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_2_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_3_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_4_regs.h | 512 ++ .../include/goya/asic_reg/dma_macro_regs.h | 242 + .../include/goya/asic_reg/dma_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/dma_qm_0_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_1_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_2_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_3_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_4_regs.h | 543 ++ .../include/goya/asic_reg/goya_blocks.h | 1372 +++++ .../include/goya/asic_reg/goya_masks.h | 262 + .../include/goya/asic_reg/goya_regs.h | 99 + .../include/goya/asic_reg/ic_pll_regs.h | 186 + .../include/goya/asic_reg/mc_pll_regs.h | 186 + .../include/goya/asic_reg/mme1_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme2_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme3_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme4_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme5_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme6_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/mme_qm_regs.h | 543 ++ .../include/goya/asic_reg/mme_regs.h | 2422 ++++++++ .../include/goya/asic_reg/mmu_regs.h | 158 + .../include/goya/asic_reg/pci_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/pcie_aux_regs.h | 476 ++ .../goya/asic_reg/psoc_emmc_pll_regs.h | 186 + .../goya/asic_reg/psoc_global_conf_regs.h | 1119 ++++ .../include/goya/asic_reg/psoc_mme_pll_regs.h | 186 + .../include/goya/asic_reg/psoc_pci_pll_regs.h | 186 + .../include/goya/asic_reg/psoc_spi_regs.h | 427 ++ .../goya/asic_reg/sram_y0_x0_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x1_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x2_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x3_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x4_rtr_regs.h | 215 + .../include/goya/asic_reg/stlb_regs.h | 133 + .../include/goya/asic_reg/tpc0_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc0_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc0_eml_cfg_regs.h | 580 ++ .../include/goya/asic_reg/tpc0_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/tpc0_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc1_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc1_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc1_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc1_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc2_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc2_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc2_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc2_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc3_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc3_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc3_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc3_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc4_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc4_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc4_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc4_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc5_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc5_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc5_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc5_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc6_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc6_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc6_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc6_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc7_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc7_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc7_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/tpc7_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc_pll_regs.h | 186 + drivers/misc/habanalabs/include/goya/goya.h | 41 + .../include/goya/goya_async_events.h | 186 + .../misc/habanalabs/include/goya/goya_fw_if.h | 28 + .../habanalabs/include/goya/goya_packets.h | 129 + drivers/misc/habanalabs/include/hl_boot_if.h | 30 + .../include/hw_ip/mmu/mmu_general.h | 45 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/include/qman_if.h | 56 + drivers/misc/habanalabs/irq.c | 325 + drivers/misc/habanalabs/memory.c | 1722 ++++++ drivers/misc/habanalabs/mmu.c | 604 ++ drivers/misc/habanalabs/sysfs.c | 691 +++ include/uapi/misc/habanalabs.h | 444 ++ 115 files changed, 73533 insertions(+), 1 deletion(-) create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/Kconfig create mode 100644 drivers/misc/habanalabs/Makefile create mode 100644 drivers/misc/habanalabs/asid.c create mode 100644 drivers/misc/habanalabs/command_buffer.c create mode 100644 drivers/misc/habanalabs/command_submission.c create mode 100644 drivers/misc/habanalabs/context.c create mode 100644 drivers/misc/habanalabs/debugfs.c create mode 100644 drivers/misc/habanalabs/device.c create mode 100644 drivers/misc/habanalabs/goya/Makefile create mode 100644 drivers/misc/habanalabs/goya/goya.c create mode 100644 drivers/misc/habanalabs/goya/goyaP.h create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/goya/goya_security.c create mode 100644 drivers/misc/habanalabs/habanalabs.h create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c create mode 100644 drivers/misc/habanalabs/hw_queue.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/include/armcp_if.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/goya.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/include/qman_if.h create mode 100644 drivers/misc/habanalabs/irq.c create mode 100644 drivers/misc/habanalabs/memory.c create mode 100644 drivers/misc/habanalabs/mmu.c create mode 100644 drivers/misc/habanalabs/sysfs.c create mode 100644 include/uapi/misc/habanalabs.h
Signature · 1 line
2.17.1
PATCH v3 1/15 habanalabs: add skeleton driver
This patch adds the habanalabs skeleton driver. The driver does nothing at this stage except very basic operations. It contains the minimal code to insmod and rmmod the driver and to create a /dev/hlX file per PCI device.
--- drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/habanalabs/Kconfig | 22 ++ drivers/misc/habanalabs/Makefile | 7 + drivers/misc/habanalabs/device.c | 336 +++++++++++++++++++++ drivers/misc/habanalabs/habanalabs.h | 138 +++++++++ drivers/misc/habanalabs/habanalabs_drv.c | 362 +++++++++++++++++++++++ 7 files changed, 867 insertions(+) create mode 100644 drivers/misc/habanalabs/Kconfig create mode 100644 drivers/misc/habanalabs/Makefile create mode 100644 drivers/misc/habanalabs/device.c create mode 100644 drivers/misc/habanalabs/habanalabs.h create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c
drivers/misc/Kconfig +1 -0
@@ -535,4 +535,5 @@ source "drivers/misc/echo/Kconfig" source "drivers/misc/cxl/Kconfig" source "drivers/misc/ocxl/Kconfig" source "drivers/misc/cardreader/Kconfig" +source "drivers/misc/habanalabs/Kconfig" endmenu
drivers/misc/Makefile +1 -0
@@ -59,3 +59,4 @@ obj-$(CONFIG_PCI_ENDPOINT_TEST) += pci_endpoint_test.o obj-$(CONFIG_OCXL) += ocxl/ obj-y += cardreader/ obj-$(CONFIG_PVPANIC) += pvpanic.o +obj-$(CONFIG_HABANA_AI) += habanalabs/
drivers/misc/habanalabs/Kconfig new file +22 -0
@@ -0,0 +1,22 @@ +# +# HabanaLabs AI accelerators driver +# + +config HABANA_AI + tristate "HabanaAI accelerators (habanalabs)" + depends on PCI + select FRAME_VECTOR + help + Enables PCIe card driver for Habana's AI Processors (AIP) that are + designed to accelerate Deep Learning inference and training workloads. + + The driver manages the PCIe devices and provides IOCTL interface for + the user to submit workloads to the devices. + + The user-space interface is described in + include/uapi/misc/habanalabs.h + + If unsure, say N. + + To compile this driver as a module, choose M here: the + module will be called habanalabs.
drivers/misc/habanalabs/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +# +# Makefile for HabanaLabs AI accelerators driver +# + +obj-m := habanalabs.o + +habanalabs-y := habanalabs_drv.o device.o
drivers/misc/habanalabs/device.c new file +336 -0
336 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h new file +138 -0
138 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c new file +362 -0
362 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 3/15 habanalabs: add basic Goya support
In reply to Oded GabbayThis patch adds a basic support for the Goya device. The code initializes the device's PCI controller and PCI bars. It also initializes various S/W structures and adds some basic helper functions.
--- drivers/misc/habanalabs/Makefile | 4 + drivers/misc/habanalabs/device.c | 71 +++ drivers/misc/habanalabs/goya/Makefile | 3 + drivers/misc/habanalabs/goya/goya.c | 639 ++++++++++++++++++++ drivers/misc/habanalabs/goya/goyaP.h | 155 +++++ drivers/misc/habanalabs/habanalabs.h | 137 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 3 + drivers/misc/habanalabs/include/goya/goya.h | 41 ++ include/uapi/misc/habanalabs.h | 20 + 9 files changed, 1073 insertions(+) create mode 100644 drivers/misc/habanalabs/goya/Makefile create mode 100644 drivers/misc/habanalabs/goya/goya.c create mode 100644 drivers/misc/habanalabs/goya/goyaP.h create mode 100644 drivers/misc/habanalabs/include/goya/goya.h create mode 100644 include/uapi/misc/habanalabs.h
drivers/misc/habanalabs/Makefile +4 -0
@@ -5,3 +5,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o + +include $(src)/goya/Makefile +habanalabs-y += $(HL_GOYA_FILES) +
drivers/misc/habanalabs/device.c +71 -0
125 lines not on this page · raw message
drivers/misc/habanalabs/goya/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +subdir-ccflags-y += -I$(src) + +HL_GOYA_FILES := goya/goya.o
drivers/misc/habanalabs/goya/goya.c new file +639 -0
639 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h new file +155 -0
155 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +137 -0
182 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +3 -0
@@ -126,6 +126,9 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev, hdev->major = hl_major; +/* Parameters for bring-up - set them to defaults */ + hdev->reset_pcilink =0 ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/include/goya/goya.h new file +41 -0
41 lines not on this page · raw message
include/uapi/misc/habanalabs.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note +* +* Copyright 2016-2018 HabanaLabs, Ltd. +* All Rights Reserved. +* +*/ + +#ifndef HABANALABS_H_ +#define HABANALABS_H_ + +#include <linux/types.h> +#include <linux/ioctl.h> + +/* +* Defines that are asic-specific but constitutes as ABI between kernel driver +* and userspace +*/ +#define GOYA_KMD_SRAM_RESERVED_SIZE_FROM_START 0x8000/* 32KB */ + +#endif /* HABANALABS_H_ */
Signature · 1 line
2.17.1
PATCH v3 4/15 habanalabs: add context and ASID modules
In reply to Oded GabbayThis patch adds two modules - ASID and context.
Each user process that opens a device's file must have at least one context before it is able to "work" with the device. Each context has its own device address-space and contains information about its runtime state (its active command submissions).
To have address-space separation between contexts, each context is assigned a unique ASID, which stands for "address-space id". Goya supports up to 1024 ASIDs.
Currently, the driver doesn't support multiple contexts. Therefore, the user doesn't need to actively create a context. A "primary context" is created automatically when the user opens the device's file.
--- drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/asid.c | 58 +++++++++ drivers/misc/habanalabs/context.c | 155 +++++++++++++++++++++++ drivers/misc/habanalabs/device.c | 47 +++++++ drivers/misc/habanalabs/habanalabs.h | 71 +++++++++++ drivers/misc/habanalabs/habanalabs_drv.c | 46 ++++++- 6 files changed, 376 insertions(+), 3 deletions(-) create mode 100644 drivers/misc/habanalabs/asid.c create mode 100644 drivers/misc/habanalabs/context.c
drivers/misc/habanalabs/Makefile +1 -1
@@ -4,7 +4,7 @@ obj-m := habanalabs.o -habanalabs-y := habanalabs_drv.o device.o +habanalabs-y := habanalabs_drv.o device.o context.o asid.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/asid.c new file +58 -0
58 lines not on this page · raw message
drivers/misc/habanalabs/context.c new file +155 -0
155 lines not on this page · raw message
drivers/misc/habanalabs/device.c +47 -0
92 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +71 -0
131 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +44 -2
68 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 5/15 habanalabs: add command buffer module
In reply to Oded GabbayThis patch adds the command buffer (CB) module, which allows the user to create and destroy CBs and to map them to the user's process address-space.
A command buffer is a memory blocks that reside in DMA-able address-space and is physically contiguous so it can be accessed by the device without MMU translation. The command buffer memory is allocated using the coherent DMA API.
When creating a new CB, the IOCTL returns a handle of it, and the user-space process needs to use that handle to mmap the buffer to get a VA in the user's address-space.
Before destroying (freeing) a CB, the user must unmap the CB's VA using the CB handle.
Each CB has a reference counter, which tracks its usage in command submissions and also its mmaps (only a single mmap is allowed).
The driver maintains a pool of pre-allocated CBs in order to reduce latency during command submissions. In case the pool is empty, the driver will go to the slow-path of allocating a new CB, i.e. calling dma_alloc_coherent.
---
Changes in v3:
- Remove use of three exclamation marks
drivers/misc/habanalabs/Makefile | 3 +- drivers/misc/habanalabs/command_buffer.c | 423 +++++++++++++++++++++ drivers/misc/habanalabs/device.c | 43 ++- drivers/misc/habanalabs/goya/goya.c | 28 ++ drivers/misc/habanalabs/habanalabs.h | 86 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 2 + drivers/misc/habanalabs/habanalabs_ioctl.c | 99 +++++ include/uapi/misc/habanalabs.h | 46 +++ 8 files changed, 728 insertions(+), 2 deletions(-) create mode 100644 drivers/misc/habanalabs/command_buffer.c create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c
drivers/misc/habanalabs/Makefile +2 -1
@@ -4,7 +4,8 @@ obj-m := habanalabs.o -habanalabs-y := habanalabs_drv.o device.o context.o asid.o +habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ + command_buffer.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_buffer.c new file +423 -0
423 lines not on this page · raw message
drivers/misc/habanalabs/device.c +42 -1
87 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya.c +28 -0
@@ -92,6 +92,9 @@#define GOYA_MAX_INITIATORS 20 +#define GOYA_CB_POOL_CB_CNT 512 +#define GOYA_CB_POOL_CB_SIZE 0x20000/* 128KB */ +static void goya_get_fixed_properties (struct hl_device *hdev) {struct asic_fixed_properties *prop = &hdev->asic_prop;
@@ -119,6 +122,8 @@ static void goya_get_fixed_properties(struct hl_device *hdev) prop->tpc_enabled_mask =TPC_ENABLED_MASK ; prop->high_pll =PLL_HIGH_DEFAULT ; + prop->cb_pool_cb_cnt =GOYA_CB_POOL_CB_CNT ; + prop->cb_pool_cb_size =GOYA_CB_POOL_CB_SIZE ; }/*
@@ -604,6 +609,27 @@ int goya_resume(struct hl_device *hdev)return 0 ; } +int goya_mmap (struct hl_fpriv *hpriv,struct vm_area_struct *vma) +{ +return -EINVAL ; +} + +int goya_cb_mmap (struct hl_device *hdev,struct vm_area_struct *vma, +u64 kaddress,phys_addr_t paddress,u32 size) +{ +int rc; + + vma->vm_flags |=VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP | +VM_DONTCOPY |VM_NORESERVE ; + + rc =remap_pfn_range (vma, vma->vm_start, paddress >>PAGE_SHIFT , + size, vma->vm_page_prot); +if (rc) +dev_err (hdev->dev,"remap_pfn_range error %d" , rc); + +return rc; +} +void *goya_dma_alloc_coherent (struct hl_device *hdev,size_t size,dma_addr_t *dma_handle,gfp_t flags) {
@@ -623,6 +649,8 @@ static const struct hl_asic_funcs goya_funcs = { .sw_fini = goya_sw_fini, .suspend = goya_suspend, .resume = goya_resume, + .mmap = goya_mmap, + .cb_mmap = goya_cb_mmap, .dma_alloc_coherent = goya_dma_alloc_coherent, .dma_free_coherent = goya_dma_free_coherent, };
drivers/misc/habanalabs/habanalabs.h +86 -0
166 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +2 -0
@@ -120,6 +120,7 @@ int hl_device_open(struct inode *inode, struct file *filp)kref_init (&hpriv->refcount);nonseekable_open (inode, filp); +hl_cb_mgr_init (&hpriv->cb_mgr);hl_ctx_mgr_init (&hpriv->ctx_mgr); rc =hl_ctx_create (hdev, hpriv);
@@ -135,6 +136,7 @@ int hl_device_open(struct inode *inode, struct file *filp) out_err: filp->private_data =NULL ;hl_ctx_mgr_fini (hpriv->hdev, &hpriv->ctx_mgr); +hl_cb_mgr_fini (hpriv->hdev, &hpriv->cb_mgr);kfree (hpriv); close_device:
drivers/misc/habanalabs/habanalabs_ioctl.c new file +99 -0
99 lines not on this page · raw message
include/uapi/misc/habanalabs.h +46 -0
50 lines not on this page · raw message
Signature · 1 line
2.17.1
Greg KH
maintainerRe: [PATCH v3 05/15] habanalabs: add command buffer module
drivers/misc/habanalabs/command_buffer.c line 201
Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/command_buffer.c new file +11
-0+int hl_cb_ioctl (struct hl_fpriv *hpriv,void *data) +{ +union hl_cb_args *args = data; +struct hl_device *hdev = hpriv->hdev; +u64 handle; +int rc; + +switch (args->in.op) { +case HL_CB_OP_CREATE : + rc =hl_cb_create (hdev, &hpriv->cb_mgr, args->in.cb_size, + &handle, hpriv->ctx->asid);
so cb_size comes from userspace, ok, you check for the value to be too small, but not too big. That means someone can try to allocate too much memory, possibly crashing things, not good :(
drivers/misc/habanalabs/command_buffer.c line 208
Quoted 7 lines
drivers/misc/habanalabs/command_buffer.c new file +7
-0+memset (args,0 ,sizeof (*args)); + args->out.cb_handle = handle; +break ; +case HL_CB_OP_DESTROY : + rc =hl_cb_destroy (hdev, &hpriv->cb_mgr, + args->in.cb_handle); +memset (args,0 ,sizeof (*args));
Why zero this if it's not copied back to userspace?
drivers/misc/habanalabs/command_buffer.c line 211
Quoted 3 lines
drivers/misc/habanalabs/command_buffer.c new file +3
-0+break ; +default : + rc = -EINVAL ;
-ENOTTY is normally the "invalid ioctl value", right?
drivers/misc/habanalabs/command_buffer.c line 216
Quoted 5 lines
drivers/misc/habanalabs/command_buffer.c new file +5
-0+break ; + } + +return rc; +}
thanks,
greg k-h
Re: [PATCH v3 05/15] habanalabs: add command buffer module
Quoted 17 lines · Greg KH
drivers/misc/habanalabs/command_buffer.c line 201
Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/command_buffer.c new file +11
-0+int hl_cb_ioctl (struct hl_fpriv *hpriv,void *data) +{ +union hl_cb_args *args = data; +struct hl_device *hdev = hpriv->hdev; +u64 handle; +int rc; + +switch (args->in.op) { +case HL_CB_OP_CREATE : + rc =hl_cb_create (hdev, &hpriv->cb_mgr, args->in.cb_size, + &handle, hpriv->ctx->asid);so
cb_sizecomes from userspace, ok, you check for the value to be too small, but not too big. That means someone can try to allocate too much memory, possibly crashing things, not good :(
Yes, correct, but even if I limit a single allocation to, let's say, 1MB, what's stopping a userspace process from allocating multiple CBs and draining the system memory ? I'm counting on the oom module to kill that process if it mis-behaves.
And, btw, I assumed there is hard limit of 4MB on a single dma_alloc_coherent. i.e. I was never able to allocate more then 4MB through that API. So I never thought I need to check for max size because of that hard limit.
Am I missing something here ?
Quoted 11 lines
drivers/misc/habanalabs/command_buffer.c line 208
Quoted 9 lines
drivers/misc/habanalabs/command_buffer.c new file +7
-0+memset (args,0 ,sizeof (*args)); + args->out.cb_handle = handle; +break ; +case HL_CB_OP_DESTROY : + rc =hl_cb_destroy (hdev, &hpriv->cb_mgr, + args->in.cb_handle); +memset (args,0 ,sizeof (*args));Why zero this if it's not copied back to userspace?
Fixed
Quoted 8 lines
drivers/misc/habanalabs/command_buffer.c line 211
Quoted 6 lines
drivers/misc/habanalabs/command_buffer.c new file +3
-0+break ; +default : + rc = -EINVAL ;
-ENOTTYis normally the "invalid ioctl value", right?
Fixed
Quoted 10 lines
drivers/misc/habanalabs/command_buffer.c line 216
Quoted 6 lines
drivers/misc/habanalabs/command_buffer.c new file +5
-0+break ; + } + +return rc; +}thanks,
greg k-h
Greg KH
maintainerRe: [PATCH v3 05/15] habanalabs: add command buffer module
Quoted 18 lines · Oded Gabbay
Quoted 17 lines · Greg KH
drivers/misc/habanalabs/command_buffer.c line 201
Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/command_buffer.c new file +11
-0+int hl_cb_ioctl (struct hl_fpriv *hpriv,void *data) +{ +union hl_cb_args *args = data; +struct hl_device *hdev = hpriv->hdev; +u64 handle; +int rc; + +switch (args->in.op) { +case HL_CB_OP_CREATE : + rc =hl_cb_create (hdev, &hpriv->cb_mgr, args->in.cb_size, + &handle, hpriv->ctx->asid);so
cb_sizecomes from userspace, ok, you check for the value to be too small, but not too big. That means someone can try to allocate too much memory, possibly crashing things, not good :(
Quoted 9 lines
Yes, correct, but even if I limit a single allocation to, let's say, 1MB, what's stopping a userspace process from allocating multiple CBs and draining the system memory ? I'm counting on the oom module to kill that process if it mis-behaves.
And, btw, I assumed there is hard limit of 4MB on a singledma_alloc_coherent. i.e. I was never able to allocate more then 4MB through that API. So I never thought I need to check for max size because of that hard limit.
Am I missing something here ?
Relying on the oom module to handle driver issues is not always a wise idea ;)
You should put a bounds on your max allocation, if it really is 4MB, then test for that.
thanks,
greg k-h
PATCH v3 6/15 habanalabs: add basic Goya h/w initialization
In reply to Oded GabbayThis patch adds the basic part of Goya's H/W initialization. It adds code that initializes Goya's internal CPU, various registers that are related to internal routing, scrambling, workarounds for H/W bugs, etc.
It also initializes Goya's security scheme that prevents the user from abusing Goya to steal data from the host, crash the host, change Goya's F/W, etc.
---
Changes in v3:
- Remove use of three exclamation marks
- Rename boot interface header file
drivers/misc/habanalabs/device.c | 12 + drivers/misc/habanalabs/goya/Makefile | 3 +- drivers/misc/habanalabs/goya/goya.c | 878 ++++- drivers/misc/habanalabs/goya/goyaP.h | 4 + drivers/misc/habanalabs/goya/goya_security.c | 2999 +++++++++++++++++ drivers/misc/habanalabs/habanalabs.h | 15 + drivers/misc/habanalabs/habanalabs_drv.c | 7 + drivers/misc/habanalabs/include/armcp_if.h | 19 + .../misc/habanalabs/include/goya/goya_fw_if.h | 28 + drivers/misc/habanalabs/include/hl_boot_if.h | 30 + 10 files changed, 3988 insertions(+), 7 deletions(-) create mode 100644 drivers/misc/habanalabs/goya/goya_security.c create mode 100644 drivers/misc/habanalabs/include/armcp_if.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h
drivers/misc/habanalabs/device.c +12 -0
@@ -320,6 +320,15 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) goto release_ctx; } + rc = hdev->asic_funcs->hw_init (hdev); +if (rc) { +dev_err (hdev->dev,"failed to initialize the H/W\n" ); + rc =0 ; + goto out_disabled; + } + + hdev->disabled = false; +dev_notice (hdev->dev,"Successfully added device to habanalabs driver\n" );
@@ -371,6 +380,9 @@ void hl_device_fini(struct hl_device *hdev)if ((hdev->kernel_ctx) && (hl_ctx_put (hdev->kernel_ctx) !=1 ))dev_err (hdev->dev,"kernel ctx is still alive\n" ); +/* Reset the H/W. It will be in idle state after this returns */ + hdev->asic_funcs->hw_fini (hdev, true); +/* Call ASIC S/W finalize function */ hdev->asic_funcs->sw_fini (hdev);
drivers/misc/habanalabs/goya/Makefile +2 -1
@@ -1,3 +1,4 @@ subdir-ccflags-y += -I$(src) -HL_GOYA_FILES := goya/goya.o +HL_GOYA_FILES := goya/goya.o goya/goya_security.o +
drivers/misc/habanalabs/goya/goya.c +872 -6
905 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +4 -0
@@ -10,7 +10,9 @@#include <uapi/misc/habanalabs.h> #include "habanalabs.h" +#include "include/hl_boot_if.h" #include "include/goya/goya.h" +#include "include/goya/goya_fw_if.h" #define NUMBER_OF_CMPLT_QUEUES 5#define NUMBER_OF_EXT_HW_QUEUES 5
@@ -152,4 +154,6 @@ struct goya_device { u32 hw_cap_initialized; }; +void goya_init_security (struct hl_device *hdev); +#endif /* GOYAP_H_ */
drivers/misc/habanalabs/goya/goya_security.c new file +2999 -0
2,051 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +7 -0
Lines of this file are not shown. View the raw message.
drivers/misc/habanalabs/include/armcp_if.h new file +19 -0
Lines of this file are not shown. View the raw message.
drivers/misc/habanalabs/include/goya/goya_fw_if.h new file +28 -0
Lines of this file are not shown. View the raw message.
drivers/misc/habanalabs/include/hl_boot_if.h new file +30 -0
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
PATCH v3 7/15 habanalabs: add h/w queues module
In reply to Oded GabbayThis patch adds the H/W queues module and the code to initialize Goya's various compute and DMA engines and their queues.
Goya has 5 DMA channels, 8 TPC engines and a single MME engine. For each channel/engine, there is a H/W queue logic which is used to pass commands from the user to the H/W. That logic is called QMAN.
There are two types of QMANs: external and internal. The DMA QMANs are considered external while the TPC and MME QMANs are considered internal. For each external queue there is a completion queue, which is located on the Host memory.
The differences between external and internal QMANs are:
1. The location of the queue's memory. External QMANs are located on the Host memory while internal QMANs are located on the on-chip memory.
2. The external QMAN write an entry to a completion queue and sends an MSI-X interrupt upon completion of a command buffer that was given to it. The internal QMAN doesn't do that.
---
Changes in v3:
- Fix the return value in goya_stop_external_queues()
- Remove hdev->ifh field as it was used for debugging
drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/device.c | 75 +- drivers/misc/habanalabs/goya/goya.c | 1525 +++++++++++++++-- drivers/misc/habanalabs/goya/goyaP.h | 7 + drivers/misc/habanalabs/habanalabs.h | 174 +- drivers/misc/habanalabs/habanalabs_drv.c | 5 + drivers/misc/habanalabs/hw_queue.c | 404 +++++ drivers/misc/habanalabs/include/armcp_if.h | 292 ++++ .../include/goya/goya_async_events.h | 186 ++ .../habanalabs/include/goya/goya_packets.h | 129 ++ drivers/misc/habanalabs/include/qman_if.h | 56 + drivers/misc/habanalabs/irq.c | 150 ++ include/uapi/misc/habanalabs.h | 29 + 13 files changed, 2918 insertions(+), 116 deletions(-) create mode 100644 drivers/misc/habanalabs/hw_queue.c create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h create mode 100644 drivers/misc/habanalabs/include/qman_if.h create mode 100644 drivers/misc/habanalabs/irq.c
drivers/misc/habanalabs/Makefile +1 -1
@@ -5,7 +5,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ - command_buffer.o + command_buffer.o hw_queue.o irq.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/device.c +73 -2
137 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya.c +1413 -112
1,639 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +7 -0
@@ -11,7 +11,9 @@#include <uapi/misc/habanalabs.h> #include "habanalabs.h" #include "include/hl_boot_if.h" +#include "include/goya/goya_packets.h" #include "include/goya/goya.h" +#include "include/goya/goya_async_events.h" #include "include/goya/goya_fw_if.h" #define NUMBER_OF_CMPLT_QUEUES 5
@@ -148,12 +150,17 @@ enum goya_fw_component { };struct goya_device { +int (*test_cpu_queue)(struct hl_device *hdev); +/* TODO: remove hw_queues_lock after moving to scheduler code */ spinlock_t hw_queues_lock;u64 ddr_bar_cur_addr;u32 hw_cap_initialized; }; +int goya_test_cpu_queue (struct hl_device *hdev); +int goya_send_cpu_message (struct hl_device *hdev,u32 *msg,u16 len, +u32 timeout,long *result);void goya_init_security (struct hl_device *hdev);#endif /* GOYAP_H_ */
drivers/misc/habanalabs/habanalabs.h +173 -1
268 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +5 -0
@@ -173,6 +173,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,/* Parameters for bring-up - set them to defaults */ hdev->cpu_enable =1 ; hdev->reset_pcilink =0 ; + hdev->cpu_queues_enable =1 ; hdev->fw_loading =1 ; hdev->pldm =0 ;
@@ -180,6 +181,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,if (!hdev->cpu_enable) hdev->fw_loading =0 ; +/* If we don't load FW, no need to initialize CPU queues */ +if (!hdev->fw_loading) + hdev->cpu_queues_enable =0 ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/hw_queue.c new file +404 -0
404 lines not on this page · raw message
drivers/misc/habanalabs/include/armcp_if.h +292 -0
302 lines not on this page · raw message
drivers/misc/habanalabs/include/goya/goya_async_events.h new file +186 -0
186 lines not on this page · raw message
drivers/misc/habanalabs/include/goya/goya_packets.h new file +129 -0
13 lines not on this page · raw message
drivers/misc/habanalabs/include/qman_if.h new file +56 -0
56 lines not on this page · raw message
drivers/misc/habanalabs/irq.c new file +150 -0
150 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
PATCH v3 8/15 habanalabs: add event queue and interrupts
In reply to Oded GabbayThis patch adds support for receiving events from Goya's control CPU and for receiving MSI-X interrupts from Goya's DMA engines and CPU.
Goya's PCI controller supports up to 8 MSI-X interrupts, which only 6 of them are currently used. The first 5 interrupts are dedicated for Goya's DMA engine queues. The 6th interrupt is dedicated for Goya's control CPU.
The DMA queue will signal its MSI-X entry upon each completion of a command buffer that was placed on its primary queue. The driver will then mark that CB as completed and free the related resources. It will also update the command submission object which that CB belongs to.
There is a dedicated event queue (EQ) between the driver and Goya's control CPU. The EQ is located on the Host memory. The control CPU writes a new entry to the EQ for various reasons, such as ECC error, MMU page fault, Hot temperature. After writing the new entry to the EQ, the control CPU will trigger its dedicated MSI-X entry to signal the driver that there is a new entry in the EQ. The driver will then read the entry and act accordingly.
--- drivers/misc/habanalabs/device.c | 34 +- drivers/misc/habanalabs/goya/goya.c | 459 ++++++++++++++++++++- drivers/misc/habanalabs/goya/goyaP.h | 1 + drivers/misc/habanalabs/habanalabs.h | 39 +- drivers/misc/habanalabs/include/armcp_if.h | 24 ++ drivers/misc/habanalabs/irq.c | 144 +++++++ 6 files changed, 689 insertions(+), 12 deletions(-)
drivers/misc/habanalabs/device.c +33 -1
@@ -182,6 +182,13 @@ static int device_early_init(struct hl_device *hdev) goto asid_fini; } + hdev->eq_wq =alloc_workqueue ("hl-events" ,WQ_UNBOUND ,0 ); +if (hdev->eq_wq ==NULL ) { +dev_err (hdev->dev,"Failed to allocate EQ workqueue\n" ); + rc = -ENOMEM ; + goto free_cq_wq; + } +hl_cb_mgr_init (&hdev->kernel_cb_mgr);mutex_init (&hdev->fd_open_cnt_lock);
@@ -190,6 +197,8 @@ static int device_early_init(struct hl_device *hdev)return 0 ; +free_cq_wq: +destroy_workqueue (hdev->cq_wq); asid_fini:hl_asid_fini (hdev); early_fini:
@@ -211,6 +220,7 @@ static void device_early_fini(struct hl_device *hdev)hl_cb_mgr_fini (hdev, &hdev->kernel_cb_mgr); +destroy_workqueue (hdev->eq_wq);destroy_workqueue (hdev->cq_wq);hl_asid_fini (hdev);
@@ -349,11 +359,22 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) } } +/* +* Initialize the event queue. Must be done before hw_init, +* because there the address of the event queue is being +* passed as argument to request_irq +*/ + rc =hl_eq_init (hdev, &hdev->event_queue); +if (rc) { +dev_err (hdev->dev,"failed to initialize event queue\n" ); + goto cq_fini; + } +/* Allocate the kernel context */ hdev->kernel_ctx =kzalloc (sizeof (*hdev->kernel_ctx),GFP_KERNEL );if (!hdev->kernel_ctx) { rc = -ENOMEM ; - goto cq_fini; + goto eq_fini; } hdev->user_ctx =NULL ;
@@ -398,6 +419,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) "kernel ctx is still alive on initialization failure\n"); free_ctx:kfree (hdev->kernel_ctx); +eq_fini: +hl_eq_fini (hdev, &hdev->event_queue); cq_fini:for (i =0 ; i < cq_ready_cnt ; i++)hl_cq_fini (hdev, &hdev->completion_queue[i]);
@@ -439,6 +462,13 @@ void hl_device_fini(struct hl_device *hdev)/* Mark device as disabled */ hdev->disabled = true; +/* +* Halt the engines and disable interrupts so we won't get any more +* completions from H/W and we won't have any accesses from the +* H/W to the host machine +*/ + hdev->asic_funcs->halt_engines (hdev, true); +hl_cb_pool_fini (hdev);/* Release kernel context */
@@ -448,6 +478,8 @@ void hl_device_fini(struct hl_device *hdev)/* Reset the H/W. It will be in idle state after this returns */ hdev->asic_funcs->hw_fini (hdev, true); +hl_eq_fini (hdev, &hdev->event_queue); +for (i =0 ; i < hdev->asic_prop.completion_queues_count ; i++)hl_cq_fini (hdev, &hdev->completion_queue[i]);kfree (hdev->completion_queue);
drivers/misc/habanalabs/goya/goya.c +450 -9
550 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +1 -0
@@ -155,6 +155,7 @@ struct goya_device {/* TODO: remove hw_queues_lock after moving to scheduler code */ spinlock_t hw_queues_lock;u64 ddr_bar_cur_addr; +u32 events_stat[GOYA_ASYNC_EVENT_ID_SIZE ];u32 hw_cap_initialized; };
drivers/misc/habanalabs/habanalabs.h +37 -2
@@ -86,6 +86,7 @@ struct hw_queue_properties {* @cfg_size: configuration space size on SRAM. * @sram_size: total size of SRAM. * @max_asid: maximum number of open contexts (ASIDs). +* @num_of_events: number of possible internal H/W IRQs. * @completion_queues_count: number of completion queues. * @high_pll: high PLL frequency used by the device. * @cb_pool_cb_cnt: number of CBs in the CB pool.
@@ -112,6 +113,7 @@ struct asic_fixed_properties {u32 cfg_size;u32 sram_size;u32 max_asid; +u32 num_of_events;u32 high_pll;u32 cb_pool_cb_cnt;u32 cb_pool_cb_size;
@@ -204,6 +206,9 @@ struct hl_cs_job;#define HL_CQ_LENGTH HL_QUEUE_LENGTH#define HL_CQ_SIZE_IN_BYTES (HL_CQ_LENGTH * HL_CQ_ENTRY_SIZE) +/* Must be power of 2 (HL_PAGE_SIZE / HL_EQ_ENTRY_SIZE) */ +#define HL_EQ_LENGTH 64 +#define HL_EQ_SIZE_IN_BYTES (HL_EQ_LENGTH * HL_EQ_ENTRY_SIZE)/**
@@ -251,6 +256,20 @@ struct hl_cq {atomic_t free_slots_cnt; }; +/** +* struct hl_eq - describes the event queue (single one per device) +* @hdev: pointer to the device structure +* @kernel_address: holds the queue's kernel virtual address +* @bus_address: holds the queue's DMA address +* @ci: ci inside the queue +*/ +struct hl_eq { +struct hl_device *hdev; +u64 kernel_address; +dma_addr_t bus_address; +u32 ci; +}; +/* * ASICs
@@ -277,6 +296,9 @@ enum hl_asic_type {* @sw_fini: tears down driver state, does not configure H/W. * @hw_init: sets up the H/W state. * @hw_fini: tears down the H/W state. +* @halt_engines: halt engines, needed for reset sequence. This also disables +* interrupts from the device. Should be called before +* hw_fini and before CS rollback. * @suspend: handles IP specific H/W or SW changes for suspend. * @resume: handles IP specific H/W or SW changes for resume. * @mmap: mmap function, does nothing.
@@ -298,6 +320,9 @@ enum hl_asic_type {* @dma_pool_free: free small DMA allocation from pool. * @cpu_accessible_dma_pool_alloc: allocate CPU PQ packet from DMA pool. * @cpu_accessible_dma_pool_free: free CPU PQ packet from DMA pool. +* @update_eq_ci: update event queue CI. +* @handle_eqe: handle event queue entry (IRQ) from ArmCP. +* @get_events_stat: retrieve event queue entries histogram. * @hw_queues_lock: acquire H/W queues lock. * @hw_queues_unlock: release H/W queues lock. * @send_cpu_message: send buffer to ArmCP.
@@ -309,6 +334,7 @@ struct hl_asic_funcs {int (*sw_fini)(struct hl_device *hdev);int (*hw_init)(struct hl_device *hdev);void (*hw_fini)(struct hl_device *hdev,bool hard_reset); +void (*halt_engines)(struct hl_device *hdev,bool hard_reset);int (*suspend)(struct hl_device *hdev);int (*resume)(struct hl_device *hdev);int (*mmap)(struct hl_fpriv *hpriv,struct vm_area_struct *vma);
@@ -331,6 +357,10 @@ struct hl_asic_funcs { size_t size, dma_addr_t *dma_handle);void (*cpu_accessible_dma_pool_free)(struct hl_device *hdev,size_t size,void *vaddr); +void (*update_eq_ci)(struct hl_device *hdev,u32 val); +void (*handle_eqe)(struct hl_device *hdev, +struct hl_eq_entry *eq_entry); +void * (*get_events_stat)(struct hl_device *hdev,u32 *size);void (*hw_queues_lock)(struct hl_device *hdev);void (*hw_queues_unlock)(struct hl_device *hdev); int (*send_cpu_message)(struct hl_device *hdev, u32 *msg,
@@ -370,8 +400,6 @@ struct hl_ctx_mgr { }; - -/** * struct hl_cs_job - command submission job. * @finish_work: workqueue object to run when job is completed.
@@ -461,6 +489,7 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);* @kernel_ctx: KMD context structure. * @kernel_queues: array of hl_hw_queue. * @kernel_cb_mgr: command buffer manager for creating/destroying/handling CGs. +* @event_queue: event queue for IRQ from ArmCP. * @dma_pool: DMA pool for small allocations. * @cpu_accessible_dma_mem: KMD <-> ArmCP shared memory CPU address. * @cpu_accessible_dma_address: KMD <-> ArmCP shared memory DMA address.
@@ -495,9 +524,11 @@ struct hl_device {enum hl_asic_type asic_type;struct hl_cq *completion_queue;struct workqueue_struct *cq_wq; +struct workqueue_struct *eq_wq;struct hl_ctx *kernel_ctx;struct hl_hw_queue *kernel_queues;struct hl_cb_mgr kernel_cb_mgr; +struct hl_eq event_queue;struct dma_pool *dma_pool;void *cpu_accessible_dma_mem;dma_addr_t cpu_accessible_dma_address;
@@ -579,6 +610,10 @@ void hl_hw_queue_inc_ci_kernel(struct hl_device *hdev, u32 hw_queue_id);int hl_cq_init (struct hl_device *hdev,struct hl_cq *q,u32 hw_queue_id);void hl_cq_fini (struct hl_device *hdev,struct hl_cq *q); +int hl_eq_init (struct hl_device *hdev,struct hl_eq *q); +void hl_eq_fini (struct hl_device *hdev,struct hl_eq *q); +irqreturn_t hl_irq_handler_cq (int irq,void *arg); +irqreturn_t hl_irq_handler_eq (int irq,void *arg);int hl_asid_init (struct hl_device *hdev);void hl_asid_fini (struct hl_device *hdev);unsigned long hl_asid_alloc (struct hl_device *hdev);
drivers/misc/habanalabs/include/armcp_if.h +24 -0
@@ -10,6 +10,30 @@#include <linux/types.h> +/* +* EVENT QUEUE +*/ + +struct hl_eq_header { +__le32 reserved; +__le32 ctl; +}; + +struct hl_eq_entry { +struct hl_eq_header hdr; +__le64 data[7 ]; +}; + +#define HL_EQ_ENTRY_SIZE sizeof(struct hl_eq_entry) + +#define EQ_CTL_READY_SHIFT 31 +#define EQ_CTL_READY_MASK 0x80000000 + +#define EQ_CTL_EVENT_TYPE_SHIFT 16 +#define EQ_CTL_EVENT_TYPE_MASK 0x03FF0000 + +#define EVENT_QUEUE_MSIX_IDX 5 +enum pq_init_status {PQ_INIT_STATUS_NA =0 ,PQ_INIT_STATUS_READY_FOR_CP ,
drivers/misc/habanalabs/irq.c +144 -0
165 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 9/15 habanalabs: add sysfs and hwmon support
In reply to Oded GabbayThis patch add the sysfs and hwmon entries that are exposed by the driver.
Goya has several sensors, from various categories such as temperature, voltage, current, etc. The driver exposes those sensors in the standard hwmon mechanism.
In addition, the driver exposes a couple of interfaces in sysfs, both for configuration and for providing status of the device or driver.
The configuration attributes is for Power Management:
- Automatic or manual
- Frequency value when moving to high frequency mode
- Maximum power the device is allowed to consume
The rest of the attributes are read-only and provide the following information:
- Versions of the various firmwares running on the device
- Contents of the device's EEPROM
- The device type (currently only Goya is supported)
- PCI address of the device (to allow user-space to connect between /dev/hlX to PCI address)
- Status of the device (operational, malfunction, in_reset)
- How many processes are open on the device's file
--- .../ABI/testing/sysfs-driver-habanalabs | 190 ++++++ drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/device.c | 146 +++++ drivers/misc/habanalabs/goya/Makefile | 2 +- drivers/misc/habanalabs/goya/goya.c | 230 +++++++ drivers/misc/habanalabs/goya/goyaP.h | 21 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 306 +++++++++ drivers/misc/habanalabs/habanalabs.h | 101 +++ drivers/misc/habanalabs/habanalabs_drv.c | 7 + drivers/misc/habanalabs/hwmon.c | 449 +++++++++++++ drivers/misc/habanalabs/sysfs.c | 589 ++++++++++++++++++ 11 files changed, 2041 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/sysfs.c
Documentation/ABI/testing/sysfs-driver-habanalabs new file +190 -0
@@ -0,0 +1,190 @@ +What: /sys/class/habanalabs/hl<n>/armcp_kernel_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Linux kernel running on the device's CPU + +What: /sys/class/habanalabs/hl<n>/armcp_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the application running on the device's CPU + +What: /sys/class/habanalabs/hl<n>/cpld_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Device's CPLD F/W + +What: /sys/class/habanalabs/hl<n>/device_type +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the code name of the device according to its type. + The supported values are: "GOYA" + +What: /sys/class/habanalabs/hl<n>/eeprom +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: A binary file attribute that contains the contents of the + on-board EEPROM + +What: /sys/class/habanalabs/hl<n>/fuse_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the device's version from the eFuse + +What: /sys/class/habanalabs/hl<n>/hard_reset +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Interface to trigger a hard-reset operation for the device. + Hard-reset will reset ALL internal components of the device + except for the PCI interface and the internal PLLs + +What: /sys/class/habanalabs/hl<n>/hard_reset_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays how many times the device have undergone a hard-reset + operation + +What: /sys/class/habanalabs/hl<n>/high_pll +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency for MME, TPC + and IC when the power management profile is set to "automatic". + +What: /sys/class/habanalabs/hl<n>/ic_clk +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency of the + Interconnect fabric. Writes to this parameter affect the device + only when the power management profile is set to "manual" mode. + The device IC clock might be set to lower value then the + maximum. The user should read the ic_clk_curr to see the actual + frequency value of the IC + +What: /sys/class/habanalabs/hl<n>/ic_clk_curr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the current clock frequency of the Interconnect fabric + +What: /sys/class/habanalabs/hl<n>/infineon_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Device's power supply F/W code + +What: /sys/class/habanalabs/hl<n>/max_power +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum power consumption of the + device in milliwatts. + +What: /sys/class/habanalabs/hl<n>/mme_clk +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency of the + MME compute engine. Writes to this parameter affect the device + only when the power management profile is set to "manual" mode. + The device MME clock might be set to lower value then the + maximum. The user should read the mme_clk_curr to see the actual + frequency value of the MME + +What: /sys/class/habanalabs/hl<n>/mme_clk_curr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the current clock frequency of the MME compute engine + +What: /sys/class/habanalabs/hl<n>/pci_addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the PCI address of the device. This is needed so the + user would be able to open a device based on its PCI address + +What: /sys/class/habanalabs/hl<n>/pm_mng_profile +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Power management profile. Values are "auto", "manual". In "auto" + mode, the driver will set the maximum clock frequency to a high + value when a user-space process opens the device's file (unless + it was already opened by another process). The driver will set + the max clock frequency to a low value when there are no user + processes that are opened on the device's file. In "manual" + mode, the user sets the maximum clock frequency by writing to + ic_clk, mme_clk and tpc_clk + + +What: /sys/class/habanalabs/hl<n>/preboot_btl_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the device's preboot F/W code + +What: /sys/class/habanalabs/hl<n>/soft_reset +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Interface to trigger a soft-reset operation for the device. + Soft-reset will reset only the compute and DMA engines of the + device + +What: /sys/class/habanalabs/hl<n>/soft_reset_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays how many times the device have undergone a soft-reset + operation + +What: /sys/class/habanalabs/hl<n>/status +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Status of the card: "Operational", "Malfunction", "In reset". + +What: /sys/class/habanalabs/hl<n>/thermal_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Device's thermal daemon + +What: /sys/class/habanalabs/hl<n>/tpc_clk +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency of the + TPC compute engines. Writes to this parameter affect the device + only when the power management profile is set to "manual" mode. + The device TPC clock might be set to lower value then the + maximum. The user should read the tpc_clk_curr to see the actual + frequency value of the TPC + +What: /sys/class/habanalabs/hl<n>/tpc_clk_curr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the current clock frequency of the TPC compute engines + +What: /sys/class/habanalabs/hl<n>/uboot_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the u-boot running on the device's CPU + +What: /sys/class/habanalabs/hl<n>/write_open_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the total number of user processes that are currently + opened on the device's file
drivers/misc/habanalabs/Makefile +1 -1
@@ -5,7 +5,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ - command_buffer.o hw_queue.o irq.o + command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/device.c +146 -0
175 lines not on this page · raw message
drivers/misc/habanalabs/goya/Makefile +1 -1
@@ -1,4 +1,4 @@ subdir-ccflags-y += -I$(src) -HL_GOYA_FILES := goya/goya.o goya/goya_security.o +HL_GOYA_FILES := goya/goya.o goya/goya_security.o goya/goya_hwmgr.o
drivers/misc/habanalabs/goya/goya.c +230 -0
282 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +21 -0
@@ -50,7 +50,10 @@#define PLL_HIGH_DEFAULT 1575000000/* 1.575 GHz */ +#define MAX_POWER_DEFAULT 200000/* 200W */ +#define GOYA_ARMCP_INFO_TIMEOUT 10000000/* 10s */ +#define GOYA_ARMCP_EEPROM_TIMEOUT 10000000/* 10s */ #define DRAM_PHYS_DEFAULT_SIZE 0x100000000ull/* 4GB */
@@ -151,9 +154,15 @@ enum goya_fw_component {struct goya_device {int (*test_cpu_queue)(struct hl_device *hdev); +int (*armcp_info_get)(struct hl_device *hdev);/* TODO: remove hw_queues_lock after moving to scheduler code */ spinlock_t hw_queues_lock; + +u64 mme_clk; +u64 tpc_clk; +u64 ic_clk; +u64 ddr_bar_cur_addr;u32 events_stat[GOYA_ASYNC_EVENT_ID_SIZE ];u32 hw_cap_initialized;
@@ -162,6 +171,18 @@ struct goya_device {int goya_test_cpu_queue (struct hl_device *hdev);int goya_send_cpu_message (struct hl_device *hdev,u32 *msg,u16 len,u32 timeout,long *result); +long goya_get_temperature (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_voltage (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_current (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_fan_speed (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr); +void goya_set_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr, +long value); +void goya_set_pll_profile (struct hl_device *hdev,enum hl_pll_frequency freq); +int goya_add_device_attr (struct hl_device *hdev); +void goya_remove_device_attr (struct hl_device *hdev);void goya_init_security (struct hl_device *hdev); +u64 goya_get_max_power (struct hl_device *hdev); +void goya_set_max_power (struct hl_device *hdev,u64 value);#endif /* GOYAP_H_ */
drivers/misc/habanalabs/goya/goya_hwmgr.c new file +306 -0
306 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +101 -0
239 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +7 -0
@@ -131,6 +131,13 @@ int hl_device_open(struct inode *inode, struct file *filp) hpriv->taskpid =find_get_pid (current->pid); +/* +* Device is IDLE at this point so it is legal to change PLLs. There +* is no need to check anything because if the PLL is already HIGH, the +* set function will return without doing anything +*/ +hl_device_set_frequency (hdev,PLL_HIGH ); +return 0 ; out_err:
drivers/misc/habanalabs/hwmon.c new file +449 -0
449 lines not on this page · raw message
drivers/misc/habanalabs/sysfs.c new file +589 -0
589 lines not on this page · raw message
Signature · 1 line
2.17.1
Greg KH
maintainerRe: [PATCH v3 09/15] habanalabs: add sysfs and hwmon support
Quoted 49 lines · Oded Gabbay
This patch add the sysfs and hwmon entries that are exposed by the driver.
Goya has several sensors, from various categories such as temperature, voltage, current, etc. The driver exposes those sensors in the standard hwmon mechanism.
In addition, the driver exposes a couple of interfaces in sysfs, both for configuration and for providing status of the device or driver.
The configuration attributes is for Power Management:
- Automatic or manual
- Frequency value when moving to high frequency mode
- Maximum power the device is allowed to consumeThe rest of the attributes are read-only and provide the following information:
- Versions of the various firmwares running on the device
- Contents of the device's EEPROM
- The device type (currently only Goya is supported)
- PCI address of the device (to allow user-space to connect between/dev/hlXto PCI address)
- Status of the device (operational, malfunction,in_reset)
- How many processes are open on the device's fileSigned-off-by Oded Gabbay <[email protected]>--- .../ABI/testing/sysfs-driver-habanalabs | 190 ++++++ drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/device.c | 146 +++++ drivers/misc/habanalabs/goya/Makefile | 2 +- drivers/misc/habanalabs/goya/goya.c | 230 +++++++ drivers/misc/habanalabs/goya/goyaP.h | 21 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 306 +++++++++ drivers/misc/habanalabs/habanalabs.h | 101 +++ drivers/misc/habanalabs/habanalabs_drv.c | 7 + drivers/misc/habanalabs/hwmon.c | 449 +++++++++++++ drivers/misc/habanalabs/sysfs.c | 589 ++++++++++++++++++ 11 files changed, 2041 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/sysfs.cDocumentation/ABI/testing/sysfs-driver-habanalabs new file +0
-0
Documentation/ABI/testing/sysfs-driver-habanalabs line 5
Quoted 5 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5
-0@@ -0,0 +1,190 @@ +What: /sys/class/habanalabs/hl<n>/armcp_kernel_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Linux kernel running on the device's CPU
Hey, nice! We can see how old that kernel gets over time :)
Documentation/ABI/testing/sysfs-driver-habanalabs line 148
Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6
-0+What: /sys/class/habanalabs/hl<n>/soft_reset_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays how many times the device have undergone a soft-reset + operation
"how many times" since when? Power on? Kernel boot? Driver load?
Quoted 5 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5
-0+static ssize_t mme_clk_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct hl_device *hdev = dev_get_drvdata(dev); + long value;
Documentation/ABI/testing/sysfs-driver-habanalabs line 26
Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10
-0+ + if (hdev->disabled) + return -ENODEV; + + value = hl_get_frequency(hdev, MME_PLL, false); + + if (value < 0) + return value; + + return snprintf(buf, PAGE_SIZE, "%lu\n", value);
Meta-comment. You do this in all of your sysfs show functions, and I understand the "I want to be safe!" feeling here, but you should just use sprintf(). The size of a sysfs buffer is PAGE_SIZE, and you should never even get close to it if you are only writing a single numeric value.
So no need to do anything fancy, just use sprintf() please.
Documentation/ABI/testing/sysfs-driver-habanalabs line 32
Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6
-0+static DEVICE_ATTR_RW(mme_clk); +static DEVICE_ATTR_RW(tpc_clk); +static DEVICE_ATTR_RW(ic_clk); +static DEVICE_ATTR_RO(mme_clk_curr); +static DEVICE_ATTR_RO(tpc_clk_curr); +static DEVICE_ATTR_RO(ic_clk_curr);
Some people like to put the macro right under the show/store functions, to keep them all in one place. Makes it easier to change/add things later over time.
Quoted 32 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +32
-0+int goya_add_device_attr(struct hl_device *hdev) +{ + int rc; + + rc = device_create_file(hdev->dev, &dev_attr_mme_clk); + if (rc) { + dev_err(hdev->dev, "failed to create device file mme_clk\n"); + return rc; + } + + rc = device_create_file(hdev->dev, &dev_attr_tpc_clk); + if (rc) { + dev_err(hdev->dev, "failed to create device file tpc_clk\n"); + goto remove_mme_clk; + } + + rc = device_create_file(hdev->dev, &dev_attr_ic_clk); + if (rc) { + dev_err(hdev->dev, "failed to create device file ic_clk\n"); + goto remove_tpc_clk; + } + + rc = device_create_file(hdev->dev, &dev_attr_mme_clk_curr); + if (rc) { + dev_err(hdev->dev, + "failed to create device file mme_clk_curr\n"); + goto remove_ic_clk; + } + + rc = device_create_file(hdev->dev, &dev_attr_tpc_clk_curr); + if (rc) { + dev_err(hdev->dev,
Documentation/ABI/testing/sysfs-driver-habanalabs line 74
Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10
-0+ "failed to create device file tpc_clk_curr\n"); + goto remove_mme_clk_curr; + } + + rc = device_create_file(hdev->dev, &dev_attr_ic_clk_curr); + if (rc) { + dev_err(hdev->dev, + "failed to create device file ic_clk_curr\n"); + goto remove_tpc_clk_curr; + }
You do know about attribute groups, right? Please use them so you don't have to hand-roll the "add a bunch of files" logic here and elsewhere. The driver core will just automatically add and remove all of the files when the device is bound properly without you having to do anything. That saves you a lot of code and debugging logic.
Same for your other lists of attributes, they can all be handled automatically, no need for this type of logic.
thanks,
greg k-h
Re: [PATCH v3 09/15] habanalabs: add sysfs and hwmon support
Quoted 59 lines · Greg KH
Quoted 49 lines · Oded Gabbay
This patch add the sysfs and hwmon entries that are exposed by the driver.
Goya has several sensors, from various categories such as temperature, voltage, current, etc. The driver exposes those sensors in the standard hwmon mechanism.
In addition, the driver exposes a couple of interfaces in sysfs, both for configuration and for providing status of the device or driver.
The configuration attributes is for Power Management:
- Automatic or manual
- Frequency value when moving to high frequency mode
- Maximum power the device is allowed to consumeThe rest of the attributes are read-only and provide the following information:
- Versions of the various firmwares running on the device
- Contents of the device's EEPROM
- The device type (currently only Goya is supported)
- PCI address of the device (to allow user-space to connect between/dev/hlXto PCI address)
- Status of the device (operational, malfunction,in_reset)
- How many processes are open on the device's fileSigned-off-by Oded Gabbay <[email protected]>--- .../ABI/testing/sysfs-driver-habanalabs | 190 ++++++ drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/device.c | 146 +++++ drivers/misc/habanalabs/goya/Makefile | 2 +- drivers/misc/habanalabs/goya/goya.c | 230 +++++++ drivers/misc/habanalabs/goya/goyaP.h | 21 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 306 +++++++++ drivers/misc/habanalabs/habanalabs.h | 101 +++ drivers/misc/habanalabs/habanalabs_drv.c | 7 + drivers/misc/habanalabs/hwmon.c | 449 +++++++++++++ drivers/misc/habanalabs/sysfs.c | 589 ++++++++++++++++++ 11 files changed, 2041 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/sysfs.cDocumentation/ABI/testing/sysfs-driver-habanalabs new file +0
-0Documentation/ABI/testing/sysfs-driver-habanalabs line 5
Quoted 5 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5
-0@@ -0,0 +1,190 @@ +What: /sys/class/habanalabs/hl<n>/armcp_kernel_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Linux kernel running on the device's CPUHey, nice! We can see how old that kernel gets over time :)
Quoted 8 lines
Documentation/ABI/testing/sysfs-driver-habanalabs line 148
Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6
-0+What: /sys/class/habanalabs/hl<n>/soft_reset_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays how many times the device have undergone a soft-reset + operation"how many times" since when? Power on? Kernel boot? Driver load?
Thanks, fixed that (Driver load).
Quoted 23 lines
Quoted 6 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +5
-0+static ssize_t mme_clk_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct hl_device *hdev = dev_get_drvdata(dev); + long value;Documentation/ABI/testing/sysfs-driver-habanalabs line 26
Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10
-0+ + if (hdev->disabled) + return -ENODEV; + + value = hl_get_frequency(hdev, MME_PLL, false); + + if (value < 0) + return value; + + return snprintf(buf, PAGE_SIZE, "%lu\n", value);Meta-comment. You do this in all of your sysfs show functions, and I understand the "I want to be safe!" feeling here, but you should just use
sprintf(). The size of a sysfs buffer isPAGE_SIZE, and you should never even get close to it if you are only writing a single numeric value.
Quoted 1 line
So no need to do anything fancy, just use
sprintf()please.
Got it, fixed.
Quoted 11 lines
Documentation/ABI/testing/sysfs-driver-habanalabs line 32
Quoted 7 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +6
-0+static DEVICE_ATTR_RW(mme_clk); +static DEVICE_ATTR_RW(tpc_clk); +static DEVICE_ATTR_RW(ic_clk); +static DEVICE_ATTR_RO(mme_clk_curr); +static DEVICE_ATTR_RO(tpc_clk_curr); +static DEVICE_ATTR_RO(ic_clk_curr);Some people like to put the macro right under the show/store functions, to keep them all in one place. Makes it easier to change/add things later over time.
I really prefer it this way, you get a clear picture of all the ATTR you expose without the need to go over the entire file. If you don't mind I want to leave this as it is.
Quoted 51 lines
Quoted 33 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +32
-0+int goya_add_device_attr(struct hl_device *hdev) +{ + int rc; + + rc = device_create_file(hdev->dev, &dev_attr_mme_clk); + if (rc) { + dev_err(hdev->dev, "failed to create device file mme_clk\n"); + return rc; + } + + rc = device_create_file(hdev->dev, &dev_attr_tpc_clk); + if (rc) { + dev_err(hdev->dev, "failed to create device file tpc_clk\n"); + goto remove_mme_clk; + } + + rc = device_create_file(hdev->dev, &dev_attr_ic_clk); + if (rc) { + dev_err(hdev->dev, "failed to create device file ic_clk\n"); + goto remove_tpc_clk; + } + + rc = device_create_file(hdev->dev, &dev_attr_mme_clk_curr); + if (rc) { + dev_err(hdev->dev, + "failed to create device file mme_clk_curr\n"); + goto remove_ic_clk; + } + + rc = device_create_file(hdev->dev, &dev_attr_tpc_clk_curr); + if (rc) { + dev_err(hdev->dev,Documentation/ABI/testing/sysfs-driver-habanalabs line 74
Quoted 10 lines
Documentation/ABI/testing/sysfs-driver-habanalabs new file +10
-0+ "failed to create device file tpc_clk_curr\n"); + goto remove_mme_clk_curr; + } + + rc = device_create_file(hdev->dev, &dev_attr_ic_clk_curr); + if (rc) { + dev_err(hdev->dev, + "failed to create device file ic_clk_curr\n"); + goto remove_tpc_clk_curr; + }You do know about attribute groups, right? Please use them so you don't have to hand-roll the "add a bunch of files" logic here and elsewhere. The driver core will just automatically add and remove all of the files when the device is bound properly without you having to do anything. That saves you a lot of code and debugging logic.
Quoted 2 lines
Same for your other lists of attributes, they can all be handled automatically, no need for this type of logic.
ah... hmm... oops, didn't know about them. feelsbadman :( I don't think they are used in drm (or at least not in amd) which is where most of my kernel knowledge comes from.
Indeed, much more elegant. Fixed the code.
Quoted 4 lines
thanks,
greg k-h
PATCH v3 10/15 habanalabs: add device reset support
In reply to Oded GabbayThis patch adds support for doing various on-the-fly reset of Goya.
The driver supports two types of resets:
1. soft-reset
2. hard-reset
Soft-reset is done when the device detects a timeout of a command submission that was given to the device. The soft-reset process only resets the engines that are relevant for the submission of compute jobs, i.e. the DMA channels, the TPCs and the MME. The purpose is to bring the device as fast as possible to a working state.
Hard-reset is done in several cases:
1. After soft-reset is done but the device is not responding
2. When fatal errors occur inside the device, e.g. ECC error
3. When the driver is removed
Hard-reset performs a reset of the entire chip except for the PCI controller and the PLLs. It is a much longer process then soft-reset but it helps to recover the device without the need to reboot the Host.
After hard-reset, the driver will restore the max power attribute and in case of manual power management, the frequencies that were set.
This patch also adds two entries to the sysfs, which allows the root user to initiate a soft or hard reset.
---
Changes in v3:
- Remove use of three exclamation marks
- Add "init_done" field to hl_device to mark that initialization is done. This field is used in the reset function to avoid doing reset during initialization phase. This is instead of setting the in_reset atomic field to 1 during initialization, which is conceptually not accurate.
drivers/misc/habanalabs/command_buffer.c | 9 +- drivers/misc/habanalabs/device.c | 317 ++++++++++++++++++++- drivers/misc/habanalabs/goya/goya.c | 210 ++++++++++++++ drivers/misc/habanalabs/goya/goya_hwmgr.c | 18 +- drivers/misc/habanalabs/habanalabs.h | 38 +++ drivers/misc/habanalabs/habanalabs_drv.c | 9 +- drivers/misc/habanalabs/habanalabs_ioctl.c | 6 + drivers/misc/habanalabs/hwmon.c | 4 +- drivers/misc/habanalabs/irq.c | 31 ++ drivers/misc/habanalabs/sysfs.c | 120 +++++++- 10 files changed, 737 insertions(+), 25 deletions(-)
drivers/misc/habanalabs/command_buffer.c +7 -2
@@ -89,9 +89,14 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr,bool alloc_new_cb = true;int rc; -if (hdev->disabled) { +/* +* Can't use generic function to check this because of special case +* where we create a CB as part of the reset process +*/ +if ((hdev->disabled) || ((atomic_read (&hdev->in_reset)) && + (ctx_id !=HL_KERNEL_ASID_ID ))) {dev_warn_ratelimited (hdev->dev, -"Device is disabled. Can't create new CBs\n" ); +"Device is disabled or in reset. Can't create new CBs\n" ); rc = -EBUSY ; goto out_err; }
drivers/misc/habanalabs/device.c +316 -1
367 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya.c +210 -0
248 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya_hwmgr.c +9 -9
@@ -38,7 +38,7 @@ static ssize_t mme_clk_show(struct device *dev, struct device_attribute *attr,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,MME_PLL , false);
@@ -57,7 +57,7 @@ static ssize_t mme_clk_store(struct device *dev, struct device_attribute *attr,int rc;long value; -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) { count = -ENODEV ; goto fail; }
@@ -87,7 +87,7 @@ static ssize_t tpc_clk_show(struct device *dev, struct device_attribute *attr,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,TPC_PLL , false);
@@ -106,7 +106,7 @@ static ssize_t tpc_clk_store(struct device *dev, struct device_attribute *attr,int rc;long value; -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) { count = -ENODEV ; goto fail; }
@@ -136,7 +136,7 @@ static ssize_t ic_clk_show(struct device *dev, struct device_attribute *attr,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,IC_PLL , false);
@@ -155,7 +155,7 @@ static ssize_t ic_clk_store(struct device *dev, struct device_attribute *attr,int rc;long value; -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) { count = -ENODEV ; goto fail; }
@@ -185,7 +185,7 @@ static ssize_t mme_clk_curr_show(struct device *dev,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,MME_PLL , true);
@@ -202,7 +202,7 @@ static ssize_t tpc_clk_curr_show(struct device *dev,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,TPC_PLL , true);
@@ -219,7 +219,7 @@ static ssize_t ic_clk_curr_show(struct device *dev,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,IC_PLL , true);
drivers/misc/habanalabs/habanalabs.h +38 -0
@@ -26,8 +26,12 @@#define HL_MMAP_CB_MASK (0x8000000000000000ull >> PAGE_SHIFT) +#define HL_PENDING_RESET_PER_SEC 5 +#define HL_DEVICE_TIMEOUT_USEC 1000000/* 1 s */ +#define HL_HEARTBEAT_PER_USEC 5000000/* 5 s */ +#define HL_PLL_LOW_JOB_FREQ_USEC 5000000/* 5 s */ #define HL_MAX_QUEUES 128
@@ -367,8 +371,10 @@ enum hl_pll_frequency {* @handle_eqe: handle event queue entry (IRQ) from ArmCP. * @set_pll_profile: change PLL profile (manual/automatic). * @get_events_stat: retrieve event queue entries histogram. +* @send_heartbeat: send is-alive packet to ArmCP and verify response. * @enable_clock_gating: enable clock gating for reducing power consumption. * @disable_clock_gating: disable clock for accessing registers on HBW. +* @soft_reset_late_init: perform certain actions needed after soft reset. * @hw_queues_lock: acquire H/W queues lock. * @hw_queues_unlock: release H/W queues lock. * @get_eeprom_data: retrieve EEPROM data from F/W.
@@ -414,8 +420,10 @@ struct hl_asic_funcs {void (*set_pll_profile)(struct hl_device *hdev,enum hl_pll_frequency freq);void * (*get_events_stat)(struct hl_device *hdev,u32 *size); +int (*send_heartbeat)(struct hl_device *hdev);void (*enable_clock_gating)(struct hl_device *hdev);void (*disable_clock_gating)(struct hl_device *hdev); +int (*soft_reset_late_init)(struct hl_device *hdev);void (*hw_queues_lock)(struct hl_device *hdev);void (*hw_queues_unlock)(struct hl_device *hdev); int (*get_eeprom_data)(struct hl_device *hdev, void *data,
@@ -533,6 +541,16 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val); WREG32(mm##reg, (RREG32(mm##reg) & ~REG_FIELD_MASK(reg, field)) | \ (val) << REG_FIELD_SHIFT(reg, field)) +/** +* struct hl_device_reset_work - reset workqueue task wrapper. +* @reset_work: reset work to be done. +* @hdev: habanalabs device structure. +*/ +struct hl_device_reset_work { +struct work_struct reset_work; +struct hl_device *hdev; +}; +/** * struct hl_device - habanalabs device structure. * @pdev: pointer to PCI device, can be NULL in case of simulator device.
@@ -541,6 +559,7 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);* @cdev: related char device. * @dev: realted kernel basic device structure. * @work_freq: delayed work to lower device frequency if possible. +* @work_heartbeat: delayed work for ArmCP is-alive check. * @asic_name: ASIC specific nmae. * @asic_type: ASIC specific type. * @completion_queue: array of hl_cq.
@@ -572,6 +591,7 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);* @cb_pool: list of preallocated CBs. * @cb_pool_lock: protects the CB pool. * @user_ctx: current user context executing. +* @in_reset: is device in reset flow. * @curr_pll_profile: current PLL profile. * @fd_open_cnt: number of open user processes. * @max_power: the max power of the device, as configured by the sysadmin. This
@@ -579,10 +599,15 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);* value and update the F/W after the re-initialization * @major: habanalabs KMD major. * @high_pll: high PLL profile frequency. +* @soft_reset_cnt: number of soft reset since KMD loading. +* @hard_reset_cnt: number of hard reset since KMD loading. * @id: device minor. * @disabled: is device disabled. * @late_init_done: is late init stage was done during initialization. * @hwmon_initialized: is H/W monitor sensors was initialized. +* @hard_reset_pending: is there a hard reset work pending. +* @heartbeat: is heartbeat sanity check towards ArmCP enabled. +* @init_done: is the initialization of the device done. */ struct hl_device {struct pci_dev *pdev;
@@ -591,6 +616,7 @@ struct hl_device {struct cdev cdev;struct device *dev;struct delayed_work work_freq; +struct delayed_work work_heartbeat;char asic_name[16 ];enum hl_asic_type asic_type;struct hl_cq *completion_queue;
@@ -622,15 +648,21 @@ struct hl_device {/* TODO: remove user_ctx for multiple process support */ struct hl_ctx *user_ctx; +atomic_t in_reset;atomic_t curr_pll_profile;atomic_t fd_open_cnt;u64 max_power;u32 major;u32 high_pll; +u32 soft_reset_cnt; +u32 hard_reset_cnt;u16 id;u8 disabled;u8 late_init_done;u8 hwmon_initialized; +u8 hard_reset_pending; +u8 heartbeat; +u8 init_done;/* Parameters for bring-up */ u8 cpu_enable;
@@ -671,6 +703,7 @@ struct hl_ioctl_desc {*/ int hl_device_open (struct inode *inode,struct file *filp); +bool hl_device_disabled_or_in_reset (struct hl_device *hdev);int create_hdev (struct hl_device **dev,struct pci_dev *pdev,enum hl_asic_type asic_type,int minor);void destroy_hdev (struct hl_device *hdev);
@@ -684,6 +717,7 @@ int hl_hw_queue_send_cb_no_cmpl(struct hl_device *hdev, u32 hw_queue_id, u32 cb_size, u64 cb_ptr);u32 hl_hw_queue_add_ptr (u32 ptr,u16 val);void hl_hw_queue_inc_ci_kernel (struct hl_device *hdev,u32 hw_queue_id); +void hl_hw_queue_reset (struct hl_device *hdev,bool hard_reset);#define hl_queue_inc_ptr (p) hl_hw_queue_add_ptr(p, 1)#define hl_pi_2_offset (pi) ((pi) & (HL_QUEUE_LENGTH - 1))
@@ -692,6 +726,8 @@ int hl_cq_init(struct hl_device *hdev, struct hl_cq *q, u32 hw_queue_id);void hl_cq_fini (struct hl_device *hdev,struct hl_cq *q);int hl_eq_init (struct hl_device *hdev,struct hl_eq *q);void hl_eq_fini (struct hl_device *hdev,struct hl_eq *q); +void hl_cq_reset (struct hl_device *hdev,struct hl_cq *q); +void hl_eq_reset (struct hl_device *hdev,struct hl_eq *q);irqreturn_t hl_irq_handler_cq (int irq,void *arg);irqreturn_t hl_irq_handler_eq (int irq,void *arg);int hl_asid_init (struct hl_device *hdev);
@@ -709,6 +745,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass);void hl_device_fini (struct hl_device *hdev);int hl_device_suspend (struct hl_device *hdev);int hl_device_resume (struct hl_device *hdev); +int hl_device_reset (struct hl_device *hdev,bool hard_reset, +bool from_hard_reset_thread);void hl_hpriv_get (struct hl_fpriv *hpriv);void hl_hpriv_put (struct hl_fpriv *hpriv);int hl_device_set_frequency (struct hl_device *hdev,enum hl_pll_frequency freq);
drivers/misc/habanalabs/habanalabs_drv.c +7 -2
@@ -88,9 +88,9 @@ int hl_device_open(struct inode *inode, struct file *filp)mutex_lock (&hdev->fd_open_cnt_lock); -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) {dev_err_ratelimited (hdev->dev, -"Can't open %s because it is disabled\n" , +"Can't open %s because it is disabled or in reset\n" ,dev_name (hdev->dev));mutex_unlock (&hdev->fd_open_cnt_lock);return -EPERM ;
@@ -183,6 +183,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev, hdev->cpu_queues_enable =1 ; hdev->fw_loading =1 ; hdev->pldm =0 ; + hdev->heartbeat =1 ;/* If CPU is disabled, no point in loading FW */ if (!hdev->cpu_enable)
@@ -192,6 +193,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,if (!hdev->fw_loading) hdev->cpu_queues_enable =0 ; +/* If CPU queues not enabled, no way to do heartbeat */ +if (!hdev->cpu_queues_enable) + hdev->heartbeat =0 ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/habanalabs_ioctl.c +6 -0
@@ -33,6 +33,12 @@ long hl_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)unsigned int usize, asize;int retcode; +if (hdev->hard_reset_pending) { +dev_crit_ratelimited (hdev->dev, +"Device HARD reset pending! Please close FD\n" ); +return -ENODEV ; + } +if ((nr >=HL_COMMAND_START ) && (nr <HL_COMMAND_END )) {u32 hl_size;
drivers/misc/habanalabs/hwmon.c +2 -2
@@ -111,7 +111,7 @@ static int hl_read(struct device *dev, enum hwmon_sensor_types type, {struct hl_device *hdev =dev_get_drvdata (dev); -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ;switch (type) {
@@ -185,7 +185,7 @@ static int hl_write(struct device *dev, enum hwmon_sensor_types type, {struct hl_device *hdev =dev_get_drvdata (dev); -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ;switch (type) {
drivers/misc/habanalabs/irq.c +31 -0
@@ -250,6 +250,23 @@ void hl_cq_fini(struct hl_device *hdev, struct hl_cq *q) (void *) q->kernel_address, q->bus_address); } +void hl_cq_reset (struct hl_device *hdev,struct hl_cq *q) +{ + q->ci =0 ; + q->pi =0 ; + +atomic_set (&q->free_slots_cnt,HL_CQ_LENGTH ); + +/* +* It's not enough to just reset the PI/CI because the H/W may have +* written valid completion entries before it was halted and therefore +* we need to clean the actual queues so we won't process old entries +* when the device is operational again +*/ + +memset ((void *) q->kernel_address,0 ,HL_CQ_SIZE_IN_BYTES ); +} +/* * hl_eq_init - main initialization function for an event queue object *
@@ -292,3 +309,17 @@ void hl_eq_fini(struct hl_device *hdev, struct hl_eq *q) hdev->asic_funcs->dma_free_coherent (hdev,HL_EQ_SIZE_IN_BYTES , (void *) q->kernel_address, q->bus_address); } + +void hl_eq_reset (struct hl_device *hdev,struct hl_eq *q) +{ + q->ci =0 ; + +/* +* It's not enough to just reset the PI/CI because the H/W may have +* written valid completion entries before it was halted and therefore +* we need to clean the actual queues so we won't process old entries +* when the device is operational again +*/ + +memset ((void *) q->kernel_address,0 ,HL_EQ_SIZE_IN_BYTES ); +}
drivers/misc/habanalabs/sysfs.c +111 -9
228 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 11/15 habanalabs: add command submission module
In reply to Oded GabbayThis patch adds the main flow for the user to submit work to the device.
Each work is described by a command submission object (CS). The CS contains 3 arrays of command buffers: One for execution, and two for context-switch (store and restore).
For each CB, the user specifies on which queue to put that CB. In case of an internal queue, the entry doesn't contain a pointer to the CB but the address in the on-chip memory that the CB resides at.
The driver parses some of the CBs to enforce security restrictions.
The user receives a sequence number that represents the CS object. The user can then query the driver regarding the status of the CS, using that sequence number.
In case the CS doesn't finish before the timeout expires, the driver will perform a soft-reset of the device.
---
Changes in v3:
- Remove use of three exclamation marks
- Remove hdev->ifh field as it was used for debugging
drivers/misc/habanalabs/Makefile | 3 +- drivers/misc/habanalabs/command_submission.c | 770 ++++++++++++ drivers/misc/habanalabs/context.c | 52 +- drivers/misc/habanalabs/device.c | 16 + drivers/misc/habanalabs/goya/goya.c | 1122 ++++++++++++++++++ drivers/misc/habanalabs/habanalabs.h | 272 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 20 + drivers/misc/habanalabs/habanalabs_ioctl.c | 4 +- drivers/misc/habanalabs/hw_queue.c | 232 ++++ drivers/misc/habanalabs/memory.c | 200 ++++ include/uapi/misc/habanalabs.h | 158 ++- 11 files changed, 2842 insertions(+), 7 deletions(-) create mode 100644 drivers/misc/habanalabs/command_submission.c create mode 100644 drivers/misc/habanalabs/memory.c
drivers/misc/habanalabs/Makefile +2 -1
@@ -5,7 +5,8 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ - command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o + command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ + command_submission.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_submission.c new file +770 -0
770 lines not on this page · raw message
drivers/misc/habanalabs/context.c +48 -4
82 lines not on this page · raw message
drivers/misc/habanalabs/device.c +16 -0
@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)put_pid (hpriv->taskpid); +mutex_destroy (&hpriv->restore_phase_mutex); +kfree (hpriv);/* Now the FD is really closed */
@@ -201,6 +203,8 @@ static int device_early_init(struct hl_device *hdev)mutex_init (&hdev->fd_open_cnt_lock);mutex_init (&hdev->send_cpu_message_lock); +INIT_LIST_HEAD (&hdev->hw_queues_mirror_list); +spin_lock_init (&hdev->hw_queues_mirror_lock);atomic_set (&hdev->in_reset,0 );atomic_set (&hdev->fd_open_cnt,0 );
@@ -582,6 +586,9 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,*/ hdev->asic_funcs->halt_engines (hdev, hard_reset); +/* Go over all the queues, release all CS and their jobs */ +hl_cs_rollback_all (hdev); +if (hard_reset) {/* Release kernel context */ if (hl_ctx_put (hdev->kernel_ctx) !=1 ) {
@@ -605,6 +612,12 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,for (i =0 ; i < hdev->asic_prop.completion_queues_count ; i++)hl_cq_reset (hdev, &hdev->completion_queue[i]); +/* Make sure the setup phase for the user context will run again */ +if (hdev->user_ctx) { +atomic_set (&hdev->user_ctx->thread_restore_token,1 ); + hdev->user_ctx->thread_restore_wait_token =0 ; + } +/* Finished tear-down, starting to re-initialize */ if (hard_reset) {
@@ -930,6 +943,9 @@ void hl_device_fini(struct hl_device *hdev)*/ hdev->asic_funcs->halt_engines (hdev, true); +/* Go over all the queues, release all CS and their jobs */ +hl_cs_rollback_all (hdev); +hl_cb_pool_fini (hdev);/* Release kernel context */
drivers/misc/habanalabs/goya/goya.c +1122 -0
1,164 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +272 -0
454 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +20 -0
@@ -28,6 +28,17 @@ static struct class *hl_class;DEFINE_IDR (hl_devs_idr);DEFINE_MUTEX (hl_devs_idr_lock); +static int timeout_locked =5 ; +static int reset_on_lockup =1 ; + +module_param (timeout_locked, int,0444 ); +MODULE_PARM_DESC (timeout_locked, +"Device lockup timeout in seconds (0 = disabled, default 5s)" ); + +module_param (reset_on_lockup, int,0444 ); +MODULE_PARM_DESC (reset_on_lockup, +"Do device reset on lockup (0 = no, 1 = yes, default yes)" ); +#define PCI_VENDOR_ID_HABANALABS 0x1da3#define PCI_IDS_GOYA 0x0001
@@ -117,6 +128,7 @@ int hl_device_open(struct inode *inode, struct file *filp) hpriv->hdev = hdev; filp->private_data = hpriv; hpriv->filp = filp; +mutex_init (&hpriv->restore_phase_mutex);kref_init (&hpriv->refcount);nonseekable_open (inode, filp);
@@ -144,6 +156,7 @@ int hl_device_open(struct inode *inode, struct file *filp) filp->private_data =NULL ;hl_ctx_mgr_fini (hpriv->hdev, &hpriv->ctx_mgr);hl_cb_mgr_fini (hpriv->hdev, &hpriv->cb_mgr); +mutex_destroy (&hpriv->restore_phase_mutex);kfree (hpriv); close_device:
@@ -176,8 +189,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,return -ENOMEM ; hdev->major = hl_major; + hdev->reset_on_lockup = reset_on_lockup;/* Parameters for bring-up - set them to defaults */ + hdev->mmu_enable =0 ; hdev->cpu_enable =1 ; hdev->reset_pcilink =0 ; hdev->cpu_queues_enable =1 ;
@@ -197,6 +212,11 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,if (!hdev->cpu_queues_enable) hdev->heartbeat =0 ; +if (timeout_locked) + hdev->timeout_jiffies =msecs_to_jiffies (timeout_locked *1000 ); +else + hdev->timeout_jiffies =MAX_SCHEDULE_TIMEOUT ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/habanalabs_ioctl.c +3 -1
@@ -16,7 +16,9 @@ [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func}static const struct hl_ioctl_desc hl_ioctls[] = { -HL_IOCTL_DEF (HL_IOCTL_CB , hl_cb_ioctl) +HL_IOCTL_DEF (HL_IOCTL_CB , hl_cb_ioctl), +HL_IOCTL_DEF (HL_IOCTL_CS , hl_cs_ioctl), +HL_IOCTL_DEF (HL_IOCTL_WAIT_CS , hl_cs_wait_ioctl) };#define HL_CORE_IOCTL_COUNT ARRAY_SIZE(hl_ioctls)
drivers/misc/habanalabs/hw_queue.c +232 -0
250 lines not on this page · raw message
drivers/misc/habanalabs/memory.c new file +200 -0
163 lines not on this page · raw message
include/uapi/misc/habanalabs.h +157 -1
158 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
PATCH v3 12/15 habanalabs: add virtual memory and MMU modules
In reply to Oded GabbayFrom: Omer Shpigelman <[email protected]>
This patch adds the Virtual Memory and MMU modules.
Goya has an internal MMU which provides process isolation on the internal DDR. The internal MMU also performs translations for transactions that go from Goya to the Host.
The driver is responsible for allocating and freeing memory on the DDR upon user request. It also provides an interface to map and unmap DDR and Host memory to the device address space.
---
Changes in v3:
- Remove use of three exclamation marks
- Add optimization - support mapping for huge page host memory
- Minor bug fixes
drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/context.c | 19 +- drivers/misc/habanalabs/device.c | 20 +- drivers/misc/habanalabs/goya/goya.c | 393 +++++ drivers/misc/habanalabs/habanalabs.h | 188 +- drivers/misc/habanalabs/habanalabs_drv.c | 2 +- drivers/misc/habanalabs/habanalabs_ioctl.c | 3 +- .../include/hw_ip/mmu/mmu_general.h | 45 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/memory.c | 1514 +++++++++++++++++ drivers/misc/habanalabs/mmu.c | 604 +++++++ include/uapi/misc/habanalabs.h | 122 +- 12 files changed, 2919 insertions(+), 8 deletions(-) create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/mmu.c
drivers/misc/habanalabs/Makefile +1 -1
@@ -6,7 +6,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ - command_submission.o + command_submission.o mmu.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/context.c +18 -1
@@ -26,8 +26,10 @@ static void hl_ctx_fini(struct hl_ctx *ctx)for (i =0 ; i <HL_MAX_PENDING_CS ; i++)dma_fence_put (ctx->cs_pending[i]); -if (ctx->asid !=HL_KERNEL_ASID_ID ) +if (ctx->asid !=HL_KERNEL_ASID_ID ) { +hl_vm_ctx_fini (ctx);hl_asid_free (hdev, ctx->asid); + } }void hl_ctx_do_release (struct kref *ref)
@@ -97,6 +99,8 @@ void hl_ctx_free(struct hl_device *hdev, struct hl_ctx *ctx)int hl_ctx_init (struct hl_device *hdev,struct hl_ctx *ctx,bool is_kernel_ctx) { +int rc =0 ; + ctx->hdev = hdev;kref_init (&ctx->refcount);
@@ -114,9 +118,22 @@ int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx)dev_err (hdev->dev,"No free ASID, failed to create context\n" );return -ENOMEM ; } + + rc =hl_vm_ctx_init (ctx); +if (rc) { +dev_err (hdev->dev,"Failed to init mem ctx module\n" ); + rc = -ENOMEM ; + goto mem_ctx_err; + } }return 0 ; + +mem_ctx_err: +if (ctx->asid !=HL_KERNEL_ASID_ID ) +hl_asid_free (hdev, ctx->asid); + +return rc; }void hl_ctx_get (struct hl_device *hdev,struct hl_ctx *ctx)
drivers/misc/habanalabs/device.c +19 -1
@@ -604,8 +604,10 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,/* Reset the H/W. It will be in idle state after this returns */ hdev->asic_funcs->hw_fini (hdev, hard_reset); -if (hard_reset) +if (hard_reset) { +hl_vm_fini (hdev);hl_eq_reset (hdev, &hdev->event_queue); + }/* Re-initialize PI,CI to 0 in all queues (hw queue, cq) */ hl_hw_queue_reset (hdev, hard_reset);
@@ -666,6 +668,13 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset, goto out_err; } + rc =hl_vm_init (hdev); +if (rc) { +dev_err (hdev->dev, +"Failed to init memory module after hard reset\n" ); + goto out_err; + } +hl_set_max_power (hdev, hdev->max_power); hdev->hard_reset_pending = false;
@@ -844,6 +853,13 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) hdev->asic_name, hdev->asic_prop.dram_size / 1024 / 1024 / 1024); + rc =hl_vm_init (hdev); +if (rc) { +dev_err (hdev->dev,"Failed to initialize memory module\n" ); + rc =0 ; + goto out_disabled; + } + rc =hl_hwmon_init (hdev);if (rc) {dev_err (hdev->dev,"Failed to initialize hwmon\n" );
@@ -955,6 +971,8 @@ void hl_device_fini(struct hl_device *hdev)/* Reset the H/W. It will be in idle state after this returns */ hdev->asic_funcs->hw_fini (hdev, true); +hl_vm_fini (hdev); +hl_eq_fini (hdev, &hdev->event_queue);for (i =0 ; i < hdev->asic_prop.completion_queues_count ; i++)
drivers/misc/habanalabs/goya/goya.c +393 -0
466 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +186 -2
321 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +1 -1
@@ -192,7 +192,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev, hdev->reset_on_lockup = reset_on_lockup;/* Parameters for bring-up - set them to defaults */ - hdev->mmu_enable =0 ; + hdev->mmu_enable =1 ; hdev->cpu_enable =1 ; hdev->reset_pcilink =0 ; hdev->cpu_queues_enable =1 ;
drivers/misc/habanalabs/habanalabs_ioctl.c +2 -1
@@ -18,7 +18,8 @@static const struct hl_ioctl_desc hl_ioctls[] = {HL_IOCTL_DEF (HL_IOCTL_CB , hl_cb_ioctl),HL_IOCTL_DEF (HL_IOCTL_CS , hl_cs_ioctl), -HL_IOCTL_DEF (HL_IOCTL_WAIT_CS , hl_cs_wait_ioctl) +HL_IOCTL_DEF (HL_IOCTL_WAIT_CS , hl_cs_wait_ioctl), +HL_IOCTL_DEF (HL_IOCTL_MEMORY , hl_mem_ioctl) };#define HL_CORE_IOCTL_COUNT ARRAY_SIZE(hl_ioctls)
drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h new file +45 -0
45 lines not on this page · raw message
drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 +* +* Copyright 2016-2018 HabanaLabs, Ltd. +* All Rights Reserved. +* +*/ + +#ifndef INCLUDE_MMU_V1_0_H_ +#define INCLUDE_MMU_V1_0_H_ + +#define MMU_HOP0_PA43_12 0x490004 +#define MMU_HOP0_PA49_44 0x490008 +#define MMU_ASID_BUSY 0x490000 + +#endif /* INCLUDE_MMU_V1_0_H_ */
drivers/misc/habanalabs/memory.c +1514 -0
1,529 lines not on this page · raw message
drivers/misc/habanalabs/mmu.c new file +604 -0
513 lines not on this page · raw message
include/uapi/misc/habanalabs.h +121 -1
122 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.17.1
Part of the diff is not shown here. View the raw message.
Re: [PATCH v3 12/15] habanalabs: add virtual memory and MMU modules
Quoted 50 lines · Oded Gabbay
From: Omer Shpigelman <[email protected]>
This patch adds the Virtual Memory and MMU modules.
Goya has an internal MMU which provides process isolation on the internal DDR. The internal MMU also performs translations for transactions that go from Goya to the Host.
The driver is responsible for allocating and freeing memory on the DDR upon user request. It also provides an interface to map and unmap DDR and Host memory to the device address space.
Signed-off-by Omer Shpigelman <[email protected]>Signed-off-by Oded Gabbay <[email protected]>---
Changes in v3:
- Remove use of three exclamation marks
- Add optimization - support mapping for huge page host memory
- Minor bug fixesdrivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/context.c | 19 +- drivers/misc/habanalabs/device.c | 20 +- drivers/misc/habanalabs/goya/goya.c | 393 +++++ drivers/misc/habanalabs/habanalabs.h | 188 +- drivers/misc/habanalabs/habanalabs_drv.c | 2 +- drivers/misc/habanalabs/habanalabs_ioctl.c | 3 +- .../include/hw_ip/mmu/mmu_general.h | 45 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/memory.c | 1514 +++++++++++++++++ drivers/misc/habanalabs/mmu.c | 604 +++++++ include/uapi/misc/habanalabs.h | 122 +- 12 files changed, 2919 insertions(+), 8 deletions(-) create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/mmu.cdrivers/misc/habanalabs/Makefile +1
-1@@ -6,7 +6,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ - command_submission.o + command_submission.o mmu.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
[ ... ]
Quoted 18 lines
drivers/misc/habanalabs/memory.c +4
-0@@ -5,12 +5,1198 @@* All Rights Reserved. */ +#include <uapi/misc/habanalabs.h> #include "habanalabs.h" +#include "include/hw_ip/mmu/mmu_general.h" #include <linux/sched.h> #include <linux/uaccess.h> #include <linux/genalloc.h> +#define PGS_IN_HPAGE (HPAGE_SIZE >> PAGE_SHIFT) +#define HL_MMU_DEBUG 0
[ ... ]
Quoted 13 lines
drivers/misc/habanalabs/memory.c +13
-0+/* +* init_phys_pg_pack_from_userptr - initialize physical page pack from host +* memory +* +* @ctx : current context +* @userptr : userptr to initialize from +* @pphys_pg_pack : res pointer +* +* This function does the following: +* - Pin the physical pages related to the given virtual block +* - Create a physical page pack from the physical pages related to the given +* virtual block +*/
drivers/misc/habanalabs/memory.c line 653
Quoted 10 lines
drivers/misc/habanalabs/memory.c +10
-0+static int init_phys_pg_pack_from_userptr (struct hl_ctx *ctx, +struct hl_userptr *userptr, +struct hl_vm_phys_pg_pack **pphys_pg_pack) +{ +struct hl_vm_phys_pg_pack *phys_pg_pack; +struct scatterlist *sg; +dma_addr_t dma_addr; +u64 page_mask; +u32 npages, total_npages, page_size =PAGE_SIZE ; +bool first = true, is_2mb_opt = true;
Now you've mentioned 2M pages, I've started to wonder what would happen if PAGE_SIZE != 4k and HPAGE_SIZE != 2M?
There are architectures that may have different sizes for both...
Quoted 82 lines
drivers/misc/habanalabs/memory.c +82
-0+int rc, i, j; + + phys_pg_pack =kzalloc (sizeof (*phys_pg_pack),GFP_KERNEL ); +if (!phys_pg_pack) +return -ENOMEM ; + + phys_pg_pack->vm_type = userptr->vm_type; + phys_pg_pack->created_from_userptr = true; + phys_pg_pack->asid = ctx->asid; +atomic_set (&phys_pg_pack->mapping_cnt,1 ); + +/* Only if all dma_addrs are aligned to 2MB and their +* sizes is at least 2MB, we can use huge page mapping. +* We limit the 2MB optimization to this condition, +* since later on we acquire the related VA range as one +* consecutive block. +*/ + total_npages =0 ; +for_each_sg (userptr->sgt->sgl, sg, userptr->sgt->nents, i) { + npages =get_sg_info (sg, &dma_addr); + + total_npages += npages; + +if (first) { + first = false; + dma_addr &=HPAGE_MASK ; + } + +if ((npages %PGS_IN_HPAGE ) || (dma_addr & (HPAGE_SIZE -1 ))) + is_2mb_opt = false; + } + +if (is_2mb_opt) { + page_size =HPAGE_SIZE ; + total_npages /=PGS_IN_HPAGE ; + } + + page_mask = ~(((u64 ) page_size) -1 ); + + phys_pg_pack->pages =kcalloc (total_npages,sizeof (u64),GFP_KERNEL ); +if (!phys_pg_pack->pages) { + rc = -ENOMEM ; + goto page_pack_arr_mem_err; + } + + phys_pg_pack->npages = total_npages; + phys_pg_pack->page_size = page_size; + phys_pg_pack->total_size = total_npages * page_size; + + j =0 ; + first = true; +for_each_sg (userptr->sgt->sgl, sg, userptr->sgt->nents, i) { + npages =get_sg_info (sg, &dma_addr); + +/* align down to physical page size and save the offset */ +if (first) { + first = false; + phys_pg_pack->offset = dma_addr & (page_size -1 ); + dma_addr &= page_mask; + } + +while (npages) { + phys_pg_pack->pages[j++] = dma_addr; + dma_addr += page_size; + +if (is_2mb_opt) + npages -=PGS_IN_HPAGE ; +else + npages--; + } + } + + *pphys_pg_pack = phys_pg_pack; + +return 0 ; + +page_pack_arr_mem_err: +kfree (phys_pg_pack); + +return rc; +} +
[ ... ]
Quoted 81 lines
drivers/misc/habanalabs/mmu.c new file +75
-0@@ -0,0 +1,604 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* +* Copyright 2016-2018 HabanaLabs, Ltd. +* All Rights Reserved. +*/ + +#include "habanalabs.h" +#include "include/hw_ip/mmu/mmu_general.h" + +#include <linux/genalloc.h> + +#define HOP_NOT_FREED 0 +#define HOP_FREED 1 + +static struct pgt_info *get_pgt_info (struct hl_ctx *ctx,u64 addr) +{ +struct pgt_info *pgt_info =NULL ; + +hash_for_each_possible (ctx->mmu_hash, pgt_info, node, + (unsigned long ) addr) +if (addr == pgt_info->addr) +break ; + +return pgt_info; +} + +static void free_hop (struct hl_ctx *ctx,u64 hop_addr) +{ +struct pgt_info *pgt_info =get_pgt_info (ctx, hop_addr); + +gen_pool_free (pgt_info->ctx->hdev->mmu_pgt_pool, pgt_info->addr, + ctx->hdev->asic_prop.mmu_hop_table_size); +hash_del (&pgt_info->node); + +kfree (pgt_info); +} + +static u64 alloc_hop (struct hl_ctx *ctx) +{ +struct hl_device *hdev = ctx->hdev; +struct pgt_info *pgt_info; +u64 addr; + + pgt_info =kmalloc (sizeof (*pgt_info),GFP_KERNEL ); +if (!pgt_info) +return ULLONG_MAX ; + + addr = (u64 )gen_pool_alloc (hdev->mmu_pgt_pool, + hdev->asic_prop.mmu_hop_table_size); +if (!addr) { +dev_err (hdev->dev,"failed to allocate page\n" ); +kfree (pgt_info); +return ULLONG_MAX ; + } + + pgt_info->addr = addr; + pgt_info->ctx = ctx; + pgt_info->num_of_ptes =0 ; +hash_add (ctx->mmu_hash, &pgt_info->node, addr); + +return addr; +} + +static inline void clear_pte (struct hl_device *hdev,u64 pte_addr) +{ +/* clear the last and present bits */ + hdev->asic_funcs->write_pte (hdev, pte_addr,0 ); +} + +static inline void inc_num_of_ptes (struct hl_ctx *ctx,u64 hop_addr) +{ +get_pgt_info (ctx, hop_addr)->num_of_ptes++; +} +
drivers/misc/habanalabs/mmu.c line 85
Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10
-0+/* +* dec_num_of_ptes - decrement the num of ptes and free the hop if possible +* +* @ctx: pointer to the context structure +* @hop_addr: addr of the hop +* +* This function returns HOP_FREED if the hop was freed or HOP_NOT_FREED if the +* num of ptes was decremented without freeing the hop. +*/ +static inline int dec_num_of_ptes (struct hl_ctx *ctx,u64 hop_addr)
I think it's worth emphasizing that this function may free a pte. Maybe even use {get,put}_pte instead of {inc,dec}_num_of_ptes.
drivers/misc/habanalabs/mmu.c line 96
Quoted 11 lines
drivers/misc/habanalabs/mmu.c new file +11
-0+{ +struct pgt_info *pgt_info =get_pgt_info (ctx, hop_addr); + + pgt_info->num_of_ptes--; + +if (pgt_info->num_of_ptes ==0 ) { +free_hop (ctx, hop_addr); +return HOP_FREED ; + } + +return HOP_NOT_FREED ;
Why not simply return pgt_info->num_of_ptes?
Quoted 32 lines
drivers/misc/habanalabs/mmu.c new file +32
-0+} + +static inline u64 get_hop0_addr (struct hl_ctx *ctx) +{ +return ctx->hdev->asic_prop.mmu_pgt_addr + + (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size); +} + +static inline u64 get_hop0_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP0_MASK ) >>HOP0_SHIFT ); +} + +static inline u64 get_hop1_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP1_MASK ) >>HOP1_SHIFT ); +} + +static inline u64 get_hop2_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP2_MASK ) >>HOP2_SHIFT ); +} + +static inline u64 get_hop3_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{
drivers/misc/habanalabs/mmu.c line 138
Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10
-0+return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP3_MASK ) >>HOP3_SHIFT ); +} + +static inline u64 get_hop4_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP4_MASK ) >>HOP4_SHIFT ); +}
These are almost clones of each other. I'd factor out
static inline u64 get_hopN_pte(struct hl_ctx *ctx, u64 hop_addr,
u64 vaddr, u64 mask, u64 shift);
and alias it for each level.
Quoted 183 lines
drivers/misc/habanalabs/mmu.c new file +183
-0+static inline u64 get_next_hop_addr (u64 curr_pte) +{ +if (curr_pte &PAGE_PRESENT_MASK ) +return curr_pte &PHYS_ADDR_MASK ; +else +return ULLONG_MAX ; +} + +/* +* hl_mmu_init - init the mmu module +* +* @hdev: pointer to the habanalabs device structure +* +* This function does the following: +* - Allocate max_asid zeroed hop0 pgts so no mapping is available +* - Enable mmu in hw +* - Invalidate the mmu cache +* - Create a pool of pages for pgts +* - Returns 0 on success +* +* This function depends on DMA QMAN to be working! +*/ +int hl_mmu_init (struct hl_device *hdev) +{ +struct asic_fixed_properties *prop = &hdev->asic_prop; +int rc; + +if (!hdev->mmu_enable) +return 0 ; + +/* MMU HW init was already done in device hw_init() */ + +mutex_init (&hdev->mmu_cache_lock); + + hdev->mmu_pgt_pool = +gen_pool_create (__ffs (prop->mmu_hop_table_size),-1 ); + +if (!hdev->mmu_pgt_pool) { +dev_err (hdev->dev,"Failed to create page gen pool\n" ); + rc = -ENOMEM ; + goto err_pool_create; + } + + rc =gen_pool_add (hdev->mmu_pgt_pool, prop->mmu_pgt_addr + + prop->mmu_hop0_tables_total_size, + prop->mmu_pgt_size - prop->mmu_hop0_tables_total_size, +-1 ); +if (rc) { +dev_err (hdev->dev,"Failed to add memory to page gen pool\n" ); + goto err_pool_add; + } + +return 0 ; + +err_pool_add: +gen_pool_destroy (hdev->mmu_pgt_pool); +err_pool_create: +mutex_destroy (&hdev->mmu_cache_lock); + +return rc; +} + +/* +* hl_mmu_fini - release the mmu module. +* +* @hdev: pointer to the habanalabs device structure +* +* This function does the following: +* - Disable mmu in hw +* - free the pgts pool +* +* All ctxs should be freed before calling this func +*/ +void hl_mmu_fini (struct hl_device *hdev) +{ +if (!hdev->mmu_enable) +return ; + +gen_pool_destroy (hdev->mmu_pgt_pool); + +mutex_destroy (&hdev->mmu_cache_lock); + +/* MMU HW fini will be done in device hw_fini() */ +} + +/* +* hl_mmu_ctx_init - init a ctx for using the mmu module +* +* @ctx: pointer to the context structure +* +* This function does the following: +* - Init a mutex to protect the concurrent mapping flow +* - Init a hash to hold all pgts related to this ctx +*/ +void hl_mmu_ctx_init (struct hl_ctx *ctx) +{ +if (!ctx->hdev->mmu_enable) +return ; + +mutex_init (&ctx->mmu_lock); +hash_init (ctx->mmu_hash); +} + +/* +* hl_mmu_ctx_fini - disable a ctx from using the mmu module +* +* @ctx: pointer to the context structure +* +* This function does the following: +* - Free any pgts which were not freed yet +* - Free the mutex +*/ +void hl_mmu_ctx_fini (struct hl_ctx *ctx) +{ +struct pgt_info *pgt_info; +struct hlist_node *tmp; +int i; + +if (!ctx->hdev->mmu_enable) +return ; + +if (!hash_empty (ctx->mmu_hash)) +dev_err (ctx->hdev->dev, +"ctx is freed while it has pgts in use\n" ); + +hash_for_each_safe (ctx->mmu_hash, i, tmp, pgt_info, node) { +dev_err (ctx->hdev->dev, +"pgt_info of addr 0x%llx of asid %d was not destroyed, num_ptes: %d\n" , + pgt_info->addr, ctx->asid, pgt_info->num_of_ptes); +free_hop (ctx, pgt_info->addr); + } + +mutex_destroy (&ctx->mmu_lock); +} + +/* +* hl_mmu_map - maps a virtual addr to physical addr +* +* @ctx: pointer to the context structure +* @virt_addr: virt addr to map from +* @phys_addr: phys addr to map to +* @page_size: physical page size +* +* This function does the following: +* - Check that the virt addr is not mapped +* - Allocate pgts as necessary in order to map the virt addr to the phys +* - Returns 0 on success, -EINVAL if addr is already mapped, or -ENOMEM. +* +* Because this function changes the page tables in the device and because it +* changes the MMU hash, it must be protected by a lock. +* However, because it maps only a single page, the lock should be implemented +* in a higher level in order to protect the entire mapping of the memory area +*/ +int hl_mmu_map (struct hl_ctx *ctx,u64 virt_addr,u64 phys_addr,u32 page_size) +{ +struct hl_device *hdev = ctx->hdev; +u64 hop0_addr =0 , hop0_pte_addr =0 , + hop1_addr =0 , hop1_pte_addr =0 , + hop2_addr =0 , hop2_pte_addr =0 , + hop3_addr =0 , hop3_pte_addr =0 , + hop4_addr =0 , hop4_pte_addr =0 , + curr_pte =0 ; +int hop1_new =0 , hop2_new =0 , hop3_new =0 , hop4_new =0 , + rc = -ENOMEM ; +bool is_huge; + +if (!hdev->mmu_enable) +return 0 ; + +/* +* This mapping function can map a 4KB/2MB page. For 2MB page there are +* only 3 hops rather than 4. Currently the DRAM allocation uses 2MB +* pages only but user memory could have been allocated with one of the +* two page sizes. Since this is a common code for all the three cases, +* we need this hugs page check. +*/ + is_huge = page_size ==PAGE_SIZE_2MB ; + + hop0_addr =get_hop0_addr (ctx); + + hop0_pte_addr =get_hop0_pte_addr (ctx, hop0_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop0_pte_addr);
drivers/misc/habanalabs/mmu.c line 332
Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10
-0+ + hop1_addr =get_next_hop_addr (curr_pte); + +if (hop1_addr ==ULLONG_MAX ) { + hop1_addr =alloc_hop (ctx); +if (hop1_addr ==ULLONG_MAX ) + goto err; +else + hop1_new =1 ; + }
The allocation can be folded into get_alloc_next_hop_addr() along with get_next_hop_addr().
Quoted 107 lines
drivers/misc/habanalabs/mmu.c new file +107
-0+ + hop1_pte_addr =get_hop1_pte_addr (ctx, hop1_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop1_pte_addr); + + hop2_addr =get_next_hop_addr (curr_pte); + +if (hop2_addr ==ULLONG_MAX ) { + hop2_addr =alloc_hop (ctx); +if (hop2_addr ==ULLONG_MAX ) + goto err; +else + hop2_new =1 ; + } + + hop2_pte_addr =get_hop2_pte_addr (ctx, hop2_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop2_pte_addr); + + hop3_addr =get_next_hop_addr (curr_pte); + +if (hop3_addr ==ULLONG_MAX ) { + hop3_addr =alloc_hop (ctx); +if (hop3_addr ==ULLONG_MAX ) + goto err; +else + hop3_new =1 ; + } + + hop3_pte_addr =get_hop3_pte_addr (ctx, hop3_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop3_pte_addr); + +if (!is_huge) { + hop4_addr =get_next_hop_addr (curr_pte); + +if (hop4_addr ==ULLONG_MAX ) { + hop4_addr =alloc_hop (ctx); +if (hop4_addr ==ULLONG_MAX ) + goto err; +else + hop4_new =1 ; + } + + hop4_pte_addr =get_hop4_pte_addr (ctx, hop4_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop4_pte_addr); + } + +if (curr_pte &PAGE_PRESENT_MASK ) { +dev_err (hdev->dev, +"mapping already exists for virt_addr 0x%llx\n" , + virt_addr); + +dev_dbg (hdev->dev,"hop0 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop0_pte_addr), + hop0_pte_addr); +dev_dbg (hdev->dev,"hop1 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop1_pte_addr), + hop1_pte_addr); +dev_dbg (hdev->dev,"hop2 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop2_pte_addr), + hop2_pte_addr); +dev_dbg (hdev->dev,"hop3 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop3_pte_addr), + hop3_pte_addr); + +if (!is_huge) +dev_dbg (hdev->dev,"hop4 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, + hop4_pte_addr), + hop4_pte_addr); + + rc =EINVAL ; + goto err; + } + + curr_pte = (phys_addr &PTE_PHYS_ADDR_MASK ) |LAST_MASK + |PAGE_PRESENT_MASK ; + + hdev->asic_funcs->write_pte (hdev, + is_huge ? hop3_pte_addr : hop4_pte_addr, + curr_pte); + +if (hop1_new) { + curr_pte = (hop1_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, hop0_pte_addr, + curr_pte); + } +if (hop2_new) { + curr_pte = (hop2_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, hop1_pte_addr, + curr_pte); +inc_num_of_ptes (ctx, hop1_addr); + } +if (hop3_new) { + curr_pte = (hop3_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, hop2_pte_addr, + curr_pte); +inc_num_of_ptes (ctx, hop2_addr); + } + +if (!is_huge) { +if (hop4_new) {
drivers/misc/habanalabs/mmu.c line 449
Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10
-0+ curr_pte = (hop4_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, + hop3_pte_addr, curr_pte); +inc_num_of_ptes (ctx, hop3_addr); + } + +inc_num_of_ptes (ctx, hop4_addr); + }else +inc_num_of_ptes (ctx, hop3_addr);
Isn't hop3 would be incremented twice for 2M pages?
Quoted 108 lines
drivers/misc/habanalabs/mmu.c new file +108
-0+ +/* flush all writes from all cores to reach PCI */ +mb (); + + hdev->asic_funcs->read_pte (hdev, + is_huge ? hop3_pte_addr : hop4_pte_addr); + +return 0 ; + +err: +if (hop4_new) +free_hop (ctx, hop4_addr); +if (hop3_new) +free_hop (ctx, hop3_addr); +if (hop2_new) +free_hop (ctx, hop2_addr); +if (hop1_new) +free_hop (ctx, hop1_addr); + +return rc; +} + +/* +* hl_mmu_unmap - unmaps a virtual addr +* +* @ctx: pointer to the context structure +* @virt_addr: virt addr to map from +* +* This function does the following: +* - Check that the virt addr is mapped +* - Unmap the vurt addr and frees pgts if possible +* - Returns 0 on success, -EINVAL if the given addr is not mapped +* +* Because this function changes the page tables in the device and because it +* changes the MMU hash, it must be protected by a lock. +* However, because it maps only a single page, the lock should be implemented +* in a higher level in order to protect the entire mapping of the memory area +*/ +int hl_mmu_unmap (struct hl_ctx *ctx,u64 virt_addr) +{ +struct hl_device *hdev = ctx->hdev; +u64 hop0_addr =0 , hop0_pte_addr =0 , + hop1_addr =0 , hop1_pte_addr =0 , + hop2_addr =0 , hop2_pte_addr =0 , + hop3_addr =0 , hop3_pte_addr =0 , + hop4_addr =0 , hop4_pte_addr =0 , + curr_pte; +int clear_hop3 =1 ; + +if (!hdev->mmu_enable) +return 0 ; + + hop0_addr =get_hop0_addr (ctx); + + hop0_pte_addr =get_hop0_pte_addr (ctx, hop0_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop0_pte_addr); + + hop1_addr =get_next_hop_addr (curr_pte); + +if (hop1_addr ==ULLONG_MAX ) + goto not_mapped; + + hop1_pte_addr =get_hop1_pte_addr (ctx, hop1_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop1_pte_addr); + + hop2_addr =get_next_hop_addr (curr_pte); + +if (hop2_addr ==ULLONG_MAX ) + goto not_mapped; + + hop2_pte_addr =get_hop2_pte_addr (ctx, hop2_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop2_pte_addr); + + hop3_addr =get_next_hop_addr (curr_pte); + +if (hop3_addr ==ULLONG_MAX ) + goto not_mapped; + + hop3_pte_addr =get_hop3_pte_addr (ctx, hop3_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop3_pte_addr); + +if (!(curr_pte &LAST_MASK )) { + hop4_addr =get_next_hop_addr (curr_pte); + +if (hop4_addr ==ULLONG_MAX ) + goto not_mapped; + + hop4_pte_addr =get_hop4_pte_addr (ctx, hop4_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop4_pte_addr); + + clear_hop3 =0 ; + } + +if (!(curr_pte &PAGE_PRESENT_MASK )) + goto not_mapped; + +clear_pte (hdev, hop4_addr ? hop4_pte_addr : hop3_pte_addr); + +if (hop4_addr &&dec_num_of_ptes (ctx, hop4_addr) ==HOP_FREED ) + clear_hop3 =1 ; + +if (clear_hop3) { +clear_pte (hdev, hop3_pte_addr);
drivers/misc/habanalabs/mmu.c line 567
Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10
-0+if (dec_num_of_ptes (ctx, hop3_addr) ==HOP_FREED ) { +clear_pte (hdev, hop2_pte_addr); +if (dec_num_of_ptes (ctx, hop2_addr) ==HOP_FREED ) { +clear_pte (hdev, hop1_pte_addr); +if (dec_num_of_ptes (ctx, hop1_addr) == +HOP_FREED ) +clear_pte (hdev, hop0_pte_addr); + } + } + }
Consider inverting the logic and doing something like
if (dec_num_of_ptes() != HOP_FREED) goto flush; clear_pte(...) if (...) goto flush;
flush:
Quoted 15 lines
drivers/misc/habanalabs/mmu.c new file +15
-0+ +/* flush all writes from all cores to reach PCI */ +mb (); + + hdev->asic_funcs->read_pte (hdev, + hop4_addr ? hop4_pte_addr : hop3_pte_addr); + +return 0 ; + +not_mapped: +dev_err (hdev->dev,"virt addr 0x%llx is not mapped to phys addr\n" , + virt_addr); + +return -EINVAL ; +}
[ ... ]
Quoted 3 lines
Signature · 1 line
2.17.1
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 12/15] habanalabs: add virtual memory and MMU modules
Quoted 103 lines · Mike Rapoport
Quoted 50 lines · Oded Gabbay
From: Omer Shpigelman <[email protected]>
This patch adds the Virtual Memory and MMU modules.
Goya has an internal MMU which provides process isolation on the internal DDR. The internal MMU also performs translations for transactions that go from Goya to the Host.
The driver is responsible for allocating and freeing memory on the DDR upon user request. It also provides an interface to map and unmap DDR and Host memory to the device address space.
Signed-off-by Omer Shpigelman <[email protected]>Signed-off-by Oded Gabbay <[email protected]>---
Changes in v3:
- Remove use of three exclamation marks
- Add optimization - support mapping for huge page host memory
- Minor bug fixesdrivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/context.c | 19 +- drivers/misc/habanalabs/device.c | 20 +- drivers/misc/habanalabs/goya/goya.c | 393 +++++ drivers/misc/habanalabs/habanalabs.h | 188 +- drivers/misc/habanalabs/habanalabs_drv.c | 2 +- drivers/misc/habanalabs/habanalabs_ioctl.c | 3 +- .../include/hw_ip/mmu/mmu_general.h | 45 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/memory.c | 1514 +++++++++++++++++ drivers/misc/habanalabs/mmu.c | 604 +++++++ include/uapi/misc/habanalabs.h | 122 +- 12 files changed, 2919 insertions(+), 8 deletions(-) create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/mmu.cdrivers/misc/habanalabs/Makefile +1
-1@@ -6,7 +6,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ - command_submission.o + command_submission.o mmu.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)[ ... ]
Quoted 18 lines
drivers/misc/habanalabs/memory.c +4
-0@@ -5,12 +5,1198 @@* All Rights Reserved. */ +#include <uapi/misc/habanalabs.h> #include "habanalabs.h" +#include "include/hw_ip/mmu/mmu_general.h" #include <linux/sched.h> #include <linux/uaccess.h> #include <linux/genalloc.h> +#define PGS_IN_HPAGE (HPAGE_SIZE >> PAGE_SHIFT) +#define HL_MMU_DEBUG 0[ ... ]
Quoted 13 lines
drivers/misc/habanalabs/memory.c +13
-0+/* +* init_phys_pg_pack_from_userptr - initialize physical page pack from host +* memory +* +* @ctx : current context +* @userptr : userptr to initialize from +* @pphys_pg_pack : res pointer +* +* This function does the following: +* - Pin the physical pages related to the given virtual block +* - Create a physical page pack from the physical pages related to the given +* virtual block +*/ drivers/misc/habanalabs/memory.c line 653
Quoted 10 lines
drivers/misc/habanalabs/memory.c +10
-0+static int init_phys_pg_pack_from_userptr (struct hl_ctx *ctx, +struct hl_userptr *userptr, +struct hl_vm_phys_pg_pack **pphys_pg_pack) +{ +struct hl_vm_phys_pg_pack *phys_pg_pack; +struct scatterlist *sg; +dma_addr_t dma_addr; +u64 page_mask; +u32 npages, total_npages, page_size =PAGE_SIZE ; +bool first = true, is_2mb_opt = true;Now you've mentioned 2M pages, I've started to wonder what would happen if
PAGE_SIZE!= 4k andHPAGE_SIZE!= 2M?
Quoted 2 lines
There are architectures that may have different sizes for both...
Will be fixed but we currently don't have any H/W that we can check this on.
Quoted 171 lines
Quoted 82 lines
drivers/misc/habanalabs/memory.c +82
-0+int rc, i, j; + + phys_pg_pack =kzalloc (sizeof (*phys_pg_pack),GFP_KERNEL ); +if (!phys_pg_pack) +return -ENOMEM ; + + phys_pg_pack->vm_type = userptr->vm_type; + phys_pg_pack->created_from_userptr = true; + phys_pg_pack->asid = ctx->asid; +atomic_set (&phys_pg_pack->mapping_cnt,1 ); + +/* Only if all dma_addrs are aligned to 2MB and their +* sizes is at least 2MB, we can use huge page mapping. +* We limit the 2MB optimization to this condition, +* since later on we acquire the related VA range as one +* consecutive block. +*/ + total_npages =0 ; +for_each_sg (userptr->sgt->sgl, sg, userptr->sgt->nents, i) { + npages =get_sg_info (sg, &dma_addr); + + total_npages += npages; + +if (first) { + first = false; + dma_addr &=HPAGE_MASK ; + } + +if ((npages %PGS_IN_HPAGE ) || (dma_addr & (HPAGE_SIZE -1 ))) + is_2mb_opt = false; + } + +if (is_2mb_opt) { + page_size =HPAGE_SIZE ; + total_npages /=PGS_IN_HPAGE ; + } + + page_mask = ~(((u64 ) page_size) -1 ); + + phys_pg_pack->pages =kcalloc (total_npages,sizeof (u64),GFP_KERNEL ); +if (!phys_pg_pack->pages) { + rc = -ENOMEM ; + goto page_pack_arr_mem_err; + } + + phys_pg_pack->npages = total_npages; + phys_pg_pack->page_size = page_size; + phys_pg_pack->total_size = total_npages * page_size; + + j =0 ; + first = true; +for_each_sg (userptr->sgt->sgl, sg, userptr->sgt->nents, i) { + npages =get_sg_info (sg, &dma_addr); + +/* align down to physical page size and save the offset */ +if (first) { + first = false; + phys_pg_pack->offset = dma_addr & (page_size -1 ); + dma_addr &= page_mask; + } + +while (npages) { + phys_pg_pack->pages[j++] = dma_addr; + dma_addr += page_size; + +if (is_2mb_opt) + npages -=PGS_IN_HPAGE ; +else + npages--; + } + } + + *pphys_pg_pack = phys_pg_pack; + +return 0 ; + +page_pack_arr_mem_err: +kfree (phys_pg_pack); + +return rc; +} +[ ... ]
Quoted 86 lines
drivers/misc/habanalabs/mmu.c new file +80
-0@@ -0,0 +1,604 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* +* Copyright 2016-2018 HabanaLabs, Ltd. +* All Rights Reserved. +*/ + +#include "habanalabs.h" +#include "include/hw_ip/mmu/mmu_general.h" + +#include <linux/genalloc.h> + +#define HOP_NOT_FREED 0 +#define HOP_FREED 1 + +static struct pgt_info *get_pgt_info (struct hl_ctx *ctx,u64 addr) +{ +struct pgt_info *pgt_info =NULL ; + +hash_for_each_possible (ctx->mmu_hash, pgt_info, node, + (unsigned long ) addr) +if (addr == pgt_info->addr) +break ; + +return pgt_info; +} + +static void free_hop (struct hl_ctx *ctx,u64 hop_addr) +{ +struct pgt_info *pgt_info =get_pgt_info (ctx, hop_addr); + +gen_pool_free (pgt_info->ctx->hdev->mmu_pgt_pool, pgt_info->addr, + ctx->hdev->asic_prop.mmu_hop_table_size); +hash_del (&pgt_info->node); + +kfree (pgt_info); +} + +static u64 alloc_hop (struct hl_ctx *ctx) +{ +struct hl_device *hdev = ctx->hdev; +struct pgt_info *pgt_info; +u64 addr; + + pgt_info =kmalloc (sizeof (*pgt_info),GFP_KERNEL ); +if (!pgt_info) +return ULLONG_MAX ; + + addr = (u64 )gen_pool_alloc (hdev->mmu_pgt_pool, + hdev->asic_prop.mmu_hop_table_size); +if (!addr) { +dev_err (hdev->dev,"failed to allocate page\n" ); +kfree (pgt_info); +return ULLONG_MAX ; + } + + pgt_info->addr = addr; + pgt_info->ctx = ctx; + pgt_info->num_of_ptes =0 ; +hash_add (ctx->mmu_hash, &pgt_info->node, addr); + +return addr; +} + +static inline void clear_pte (struct hl_device *hdev,u64 pte_addr) +{ +/* clear the last and present bits */ + hdev->asic_funcs->write_pte (hdev, pte_addr,0 ); +} + +static inline void inc_num_of_ptes (struct hl_ctx *ctx,u64 hop_addr) +{ +get_pgt_info (ctx, hop_addr)->num_of_ptes++; +} + +/* +* dec_num_of_ptes - decrement the num of ptes and free the hop if possible +* +* @ctx: pointer to the context structure +* @hop_addr: addr of the hop
Quoted 9 lines
drivers/misc/habanalabs/mmu.c line 85
Quoted 5 lines
drivers/misc/habanalabs/mmu.c new file +5
-0+* +* This function returns HOP_FREED if the hop was freed or HOP_NOT_FREED if the +* num of ptes was decremented without freeing the hop. +*/ +static inline int dec_num_of_ptes (struct hl_ctx *ctx,u64 hop_addr)I think it's worth emphasizing that this function may free a pte. Maybe even use {get,put}_pte instead of {inc,dec}_num_of_ptes.
Agreed, will be fixed.
Quoted 5 lines
Quoted 5 lines
drivers/misc/habanalabs/mmu.c new file +5
-0+{ +struct pgt_info *pgt_info =get_pgt_info (ctx, hop_addr); + + pgt_info->num_of_ptes--; +
Quoted 8 lines
drivers/misc/habanalabs/mmu.c line 96
Quoted 6 lines
drivers/misc/habanalabs/mmu.c new file +6
-0+if (pgt_info->num_of_ptes ==0 ) { +free_hop (ctx, hop_addr); +return HOP_FREED ; + } + +return HOP_NOT_FREED ;Why not simply return
pgt_info->num_of_ptes?
Fixed
Quoted 46 lines
Quoted 33 lines
drivers/misc/habanalabs/mmu.c new file +32
-0+} + +static inline u64 get_hop0_addr (struct hl_ctx *ctx) +{ +return ctx->hdev->asic_prop.mmu_pgt_addr + + (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size); +} + +static inline u64 get_hop0_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP0_MASK ) >>HOP0_SHIFT ); +} + +static inline u64 get_hop1_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP1_MASK ) >>HOP1_SHIFT ); +} + +static inline u64 get_hop2_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP2_MASK ) >>HOP2_SHIFT ); +} + +static inline u64 get_hop3_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{drivers/misc/habanalabs/mmu.c line 138
Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10
-0+return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP3_MASK ) >>HOP3_SHIFT ); +} + +static inline u64 get_hop4_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP4_MASK ) >>HOP4_SHIFT ); +}These are almost clones of each other. I'd factor out
Quoted 5 lines
static inline u64 get_hopN_pte(struct
hl_ctx*ctx, u64hop_addr,u64 vaddr, u64 mask, u64 shift);and alias it for each level.
Fixed, but I'm aliasing them using inline functions, which I prefer to macros.
Quoted 188 lines
Quoted 188 lines
drivers/misc/habanalabs/mmu.c new file +188
-0+static inline u64 get_next_hop_addr (u64 curr_pte) +{ +if (curr_pte &PAGE_PRESENT_MASK ) +return curr_pte &PHYS_ADDR_MASK ; +else +return ULLONG_MAX ; +} + +/* +* hl_mmu_init - init the mmu module +* +* @hdev: pointer to the habanalabs device structure +* +* This function does the following: +* - Allocate max_asid zeroed hop0 pgts so no mapping is available +* - Enable mmu in hw +* - Invalidate the mmu cache +* - Create a pool of pages for pgts +* - Returns 0 on success +* +* This function depends on DMA QMAN to be working! +*/ +int hl_mmu_init (struct hl_device *hdev) +{ +struct asic_fixed_properties *prop = &hdev->asic_prop; +int rc; + +if (!hdev->mmu_enable) +return 0 ; + +/* MMU HW init was already done in device hw_init() */ + +mutex_init (&hdev->mmu_cache_lock); + + hdev->mmu_pgt_pool = +gen_pool_create (__ffs (prop->mmu_hop_table_size),-1 ); + +if (!hdev->mmu_pgt_pool) { +dev_err (hdev->dev,"Failed to create page gen pool\n" ); + rc = -ENOMEM ; + goto err_pool_create; + } + + rc =gen_pool_add (hdev->mmu_pgt_pool, prop->mmu_pgt_addr + + prop->mmu_hop0_tables_total_size, + prop->mmu_pgt_size - prop->mmu_hop0_tables_total_size, +-1 ); +if (rc) { +dev_err (hdev->dev,"Failed to add memory to page gen pool\n" ); + goto err_pool_add; + } + +return 0 ; + +err_pool_add: +gen_pool_destroy (hdev->mmu_pgt_pool); +err_pool_create: +mutex_destroy (&hdev->mmu_cache_lock); + +return rc; +} + +/* +* hl_mmu_fini - release the mmu module. +* +* @hdev: pointer to the habanalabs device structure +* +* This function does the following: +* - Disable mmu in hw +* - free the pgts pool +* +* All ctxs should be freed before calling this func +*/ +void hl_mmu_fini (struct hl_device *hdev) +{ +if (!hdev->mmu_enable) +return ; + +gen_pool_destroy (hdev->mmu_pgt_pool); + +mutex_destroy (&hdev->mmu_cache_lock); + +/* MMU HW fini will be done in device hw_fini() */ +} + +/* +* hl_mmu_ctx_init - init a ctx for using the mmu module +* +* @ctx: pointer to the context structure +* +* This function does the following: +* - Init a mutex to protect the concurrent mapping flow +* - Init a hash to hold all pgts related to this ctx +*/ +void hl_mmu_ctx_init (struct hl_ctx *ctx) +{ +if (!ctx->hdev->mmu_enable) +return ; + +mutex_init (&ctx->mmu_lock); +hash_init (ctx->mmu_hash); +} + +/* +* hl_mmu_ctx_fini - disable a ctx from using the mmu module +* +* @ctx: pointer to the context structure +* +* This function does the following: +* - Free any pgts which were not freed yet +* - Free the mutex +*/ +void hl_mmu_ctx_fini (struct hl_ctx *ctx) +{ +struct pgt_info *pgt_info; +struct hlist_node *tmp; +int i; + +if (!ctx->hdev->mmu_enable) +return ; + +if (!hash_empty (ctx->mmu_hash)) +dev_err (ctx->hdev->dev, +"ctx is freed while it has pgts in use\n" ); + +hash_for_each_safe (ctx->mmu_hash, i, tmp, pgt_info, node) { +dev_err (ctx->hdev->dev, +"pgt_info of addr 0x%llx of asid %d was not destroyed, num_ptes: %d\n" , + pgt_info->addr, ctx->asid, pgt_info->num_of_ptes); +free_hop (ctx, pgt_info->addr); + } + +mutex_destroy (&ctx->mmu_lock); +} + +/* +* hl_mmu_map - maps a virtual addr to physical addr +* +* @ctx: pointer to the context structure +* @virt_addr: virt addr to map from +* @phys_addr: phys addr to map to +* @page_size: physical page size +* +* This function does the following: +* - Check that the virt addr is not mapped +* - Allocate pgts as necessary in order to map the virt addr to the phys +* - Returns 0 on success, -EINVAL if addr is already mapped, or -ENOMEM. +* +* Because this function changes the page tables in the device and because it +* changes the MMU hash, it must be protected by a lock. +* However, because it maps only a single page, the lock should be implemented +* in a higher level in order to protect the entire mapping of the memory area +*/ +int hl_mmu_map (struct hl_ctx *ctx,u64 virt_addr,u64 phys_addr,u32 page_size) +{ +struct hl_device *hdev = ctx->hdev; +u64 hop0_addr =0 , hop0_pte_addr =0 , + hop1_addr =0 , hop1_pte_addr =0 , + hop2_addr =0 , hop2_pte_addr =0 , + hop3_addr =0 , hop3_pte_addr =0 , + hop4_addr =0 , hop4_pte_addr =0 , + curr_pte =0 ; +int hop1_new =0 , hop2_new =0 , hop3_new =0 , hop4_new =0 , + rc = -ENOMEM ; +bool is_huge; + +if (!hdev->mmu_enable) +return 0 ; + +/* +* This mapping function can map a 4KB/2MB page. For 2MB page there are +* only 3 hops rather than 4. Currently the DRAM allocation uses 2MB +* pages only but user memory could have been allocated with one of the +* two page sizes. Since this is a common code for all the three cases, +* we need this hugs page check. +*/ + is_huge = page_size ==PAGE_SIZE_2MB ; + + hop0_addr =get_hop0_addr (ctx); + + hop0_pte_addr =get_hop0_pte_addr (ctx, hop0_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop0_pte_addr); + + hop1_addr =get_next_hop_addr (curr_pte); + +if (hop1_addr ==ULLONG_MAX ) { + hop1_addr =alloc_hop (ctx);
Quoted 9 lines
drivers/misc/habanalabs/mmu.c line 332
Quoted 5 lines
drivers/misc/habanalabs/mmu.c new file +5
-0+if (hop1_addr ==ULLONG_MAX ) + goto err; +else + hop1_new =1 ; + }The allocation can be folded into
get_alloc_next_hop_addr()along withget_next_hop_addr().
Fixed
Quoted 111 lines
Quoted 111 lines
drivers/misc/habanalabs/mmu.c new file +111
-0+ + hop1_pte_addr =get_hop1_pte_addr (ctx, hop1_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop1_pte_addr); + + hop2_addr =get_next_hop_addr (curr_pte); + +if (hop2_addr ==ULLONG_MAX ) { + hop2_addr =alloc_hop (ctx); +if (hop2_addr ==ULLONG_MAX ) + goto err; +else + hop2_new =1 ; + } + + hop2_pte_addr =get_hop2_pte_addr (ctx, hop2_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop2_pte_addr); + + hop3_addr =get_next_hop_addr (curr_pte); + +if (hop3_addr ==ULLONG_MAX ) { + hop3_addr =alloc_hop (ctx); +if (hop3_addr ==ULLONG_MAX ) + goto err; +else + hop3_new =1 ; + } + + hop3_pte_addr =get_hop3_pte_addr (ctx, hop3_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop3_pte_addr); + +if (!is_huge) { + hop4_addr =get_next_hop_addr (curr_pte); + +if (hop4_addr ==ULLONG_MAX ) { + hop4_addr =alloc_hop (ctx); +if (hop4_addr ==ULLONG_MAX ) + goto err; +else + hop4_new =1 ; + } + + hop4_pte_addr =get_hop4_pte_addr (ctx, hop4_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop4_pte_addr); + } + +if (curr_pte &PAGE_PRESENT_MASK ) { +dev_err (hdev->dev, +"mapping already exists for virt_addr 0x%llx\n" , + virt_addr); + +dev_dbg (hdev->dev,"hop0 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop0_pte_addr), + hop0_pte_addr); +dev_dbg (hdev->dev,"hop1 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop1_pte_addr), + hop1_pte_addr); +dev_dbg (hdev->dev,"hop2 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop2_pte_addr), + hop2_pte_addr); +dev_dbg (hdev->dev,"hop3 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, hop3_pte_addr), + hop3_pte_addr); + +if (!is_huge) +dev_dbg (hdev->dev,"hop4 pte: 0x%llx (0x%llx)\n" , + hdev->asic_funcs->read_pte (hdev, + hop4_pte_addr), + hop4_pte_addr); + + rc =EINVAL ; + goto err; + } + + curr_pte = (phys_addr &PTE_PHYS_ADDR_MASK ) |LAST_MASK + |PAGE_PRESENT_MASK ; + + hdev->asic_funcs->write_pte (hdev, + is_huge ? hop3_pte_addr : hop4_pte_addr, + curr_pte); + +if (hop1_new) { + curr_pte = (hop1_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, hop0_pte_addr, + curr_pte); + } +if (hop2_new) { + curr_pte = (hop2_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, hop1_pte_addr, + curr_pte); +inc_num_of_ptes (ctx, hop1_addr); + } +if (hop3_new) { + curr_pte = (hop3_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, hop2_pte_addr, + curr_pte); +inc_num_of_ptes (ctx, hop2_addr); + } + +if (!is_huge) { +if (hop4_new) { + curr_pte = (hop4_addr &PTE_PHYS_ADDR_MASK ) | +PAGE_PRESENT_MASK ; + ctx->hdev->asic_funcs->write_pte (ctx->hdev, + hop3_pte_addr, curr_pte);
Quoted 9 lines
drivers/misc/habanalabs/mmu.c line 449
Quoted 6 lines
drivers/misc/habanalabs/mmu.c new file +6
-0+inc_num_of_ptes (ctx, hop3_addr); + } + +inc_num_of_ptes (ctx, hop4_addr); + }else +inc_num_of_ptes (ctx, hop3_addr);Isn't hop3 would be incremented twice for 2M pages?
Why ? 2M pages means is_huge, so you will go to the else part of the if above and inc the ptes of hop3 only once there.
The other inc happens if !is_huge, which is for 4K pages.
Quoted 121 lines
Quoted 108 lines
drivers/misc/habanalabs/mmu.c new file +108
-0+ +/* flush all writes from all cores to reach PCI */ +mb (); + + hdev->asic_funcs->read_pte (hdev, + is_huge ? hop3_pte_addr : hop4_pte_addr); + +return 0 ; + +err: +if (hop4_new) +free_hop (ctx, hop4_addr); +if (hop3_new) +free_hop (ctx, hop3_addr); +if (hop2_new) +free_hop (ctx, hop2_addr); +if (hop1_new) +free_hop (ctx, hop1_addr); + +return rc; +} + +/* +* hl_mmu_unmap - unmaps a virtual addr +* +* @ctx: pointer to the context structure +* @virt_addr: virt addr to map from +* +* This function does the following: +* - Check that the virt addr is mapped +* - Unmap the vurt addr and frees pgts if possible +* - Returns 0 on success, -EINVAL if the given addr is not mapped +* +* Because this function changes the page tables in the device and because it +* changes the MMU hash, it must be protected by a lock. +* However, because it maps only a single page, the lock should be implemented +* in a higher level in order to protect the entire mapping of the memory area +*/ +int hl_mmu_unmap (struct hl_ctx *ctx,u64 virt_addr) +{ +struct hl_device *hdev = ctx->hdev; +u64 hop0_addr =0 , hop0_pte_addr =0 , + hop1_addr =0 , hop1_pte_addr =0 , + hop2_addr =0 , hop2_pte_addr =0 , + hop3_addr =0 , hop3_pte_addr =0 , + hop4_addr =0 , hop4_pte_addr =0 , + curr_pte; +int clear_hop3 =1 ; + +if (!hdev->mmu_enable) +return 0 ; + + hop0_addr =get_hop0_addr (ctx); + + hop0_pte_addr =get_hop0_pte_addr (ctx, hop0_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop0_pte_addr); + + hop1_addr =get_next_hop_addr (curr_pte); + +if (hop1_addr ==ULLONG_MAX ) + goto not_mapped; + + hop1_pte_addr =get_hop1_pte_addr (ctx, hop1_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop1_pte_addr); + + hop2_addr =get_next_hop_addr (curr_pte); + +if (hop2_addr ==ULLONG_MAX ) + goto not_mapped; + + hop2_pte_addr =get_hop2_pte_addr (ctx, hop2_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop2_pte_addr); + + hop3_addr =get_next_hop_addr (curr_pte); + +if (hop3_addr ==ULLONG_MAX ) + goto not_mapped; + + hop3_pte_addr =get_hop3_pte_addr (ctx, hop3_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop3_pte_addr); + +if (!(curr_pte &LAST_MASK )) { + hop4_addr =get_next_hop_addr (curr_pte); + +if (hop4_addr ==ULLONG_MAX ) + goto not_mapped; + + hop4_pte_addr =get_hop4_pte_addr (ctx, hop4_addr, virt_addr); + + curr_pte = hdev->asic_funcs->read_pte (hdev, hop4_pte_addr); + + clear_hop3 =0 ; + } + +if (!(curr_pte &PAGE_PRESENT_MASK )) + goto not_mapped; + +clear_pte (hdev, hop4_addr ? hop4_pte_addr : hop3_pte_addr); + +if (hop4_addr &&dec_num_of_ptes (ctx, hop4_addr) ==HOP_FREED ) + clear_hop3 =1 ; + +if (clear_hop3) { +clear_pte (hdev, hop3_pte_addr);drivers/misc/habanalabs/mmu.c line 567
Quoted 10 lines
drivers/misc/habanalabs/mmu.c new file +10
-0+if (dec_num_of_ptes (ctx, hop3_addr) ==HOP_FREED ) { +clear_pte (hdev, hop2_pte_addr); +if (dec_num_of_ptes (ctx, hop2_addr) ==HOP_FREED ) { +clear_pte (hdev, hop1_pte_addr); +if (dec_num_of_ptes (ctx, hop1_addr) == +HOP_FREED ) +clear_pte (hdev, hop0_pte_addr); + } + } + }Consider inverting the logic and doing something like
Quoted 7 lines
if (dec_num_of_ptes() != HOP_FREED) goto flush; clear_pte(...) if (...) goto flush;flush:
Yes, it does look more neat that way. Fixed
Quoted 26 lines
Quoted 15 lines
drivers/misc/habanalabs/mmu.c new file +15
-0+ +/* flush all writes from all cores to reach PCI */ +mb (); + + hdev->asic_funcs->read_pte (hdev, + hop4_addr ? hop4_pte_addr : hop3_pte_addr); + +return 0 ; + +not_mapped: +dev_err (hdev->dev,"virt addr 0x%llx is not mapped to phys addr\n" , + virt_addr); + +return -EINVAL ; +}[ ... ]
Quoted 3 lines
--
2.17.1--
Sincerely yours,
Mike.
PATCH v3 13/15 habanalabs: implement INFO IOCTL
In reply to Oded GabbayThis patch implements the INFO IOCTL. That IOCTL is used by the user to query information that is relevant/needed by the user in order to submit deep learning jobs to Goya.
The information is divided into several categories, such as H/W IP, Events that happened, DDR usage and more.
---
Changes in v3:
- Remove use of three exclamation marks
drivers/misc/habanalabs/goya/goya.c | 6 + drivers/misc/habanalabs/habanalabs.h | 2 + drivers/misc/habanalabs/habanalabs_ioctl.c | 126 +++++++++++++++++++++ include/uapi/misc/habanalabs.h | 75 +++++++++++- 4 files changed, 208 insertions(+), 1 deletion(-)
drivers/misc/habanalabs/goya/goya.c +6 -0
@@ -5117,6 +5117,11 @@ static void goya_hw_queues_unlock(struct hl_device *hdev)spin_unlock (&goya->hw_queues_lock); } +static u32 goya_get_pci_id (struct hl_device *hdev) +{ +return hdev->pdev->device; +} +int goya_get_eeprom_data (struct hl_device *hdev,void *data,size_t max_size) {struct goya_device *goya = hdev->asic_specific;
@@ -5214,6 +5219,7 @@ static const struct hl_asic_funcs goya_funcs = { .soft_reset_late_init = goya_soft_reset_late_init, .hw_queues_lock = goya_hw_queues_lock, .hw_queues_unlock = goya_hw_queues_unlock, + .get_pci_id = goya_get_pci_id, .get_eeprom_data = goya_get_eeprom_data, .send_cpu_message = goya_send_cpu_message };
drivers/misc/habanalabs/habanalabs.h +2 -0
@@ -460,6 +460,7 @@ enum hl_pll_frequency {* @soft_reset_late_init: perform certain actions needed after soft reset. * @hw_queues_lock: acquire H/W queues lock. * @hw_queues_unlock: release H/W queues lock. +* @get_pci_id: retrieve PCI ID. * @get_eeprom_data: retrieve EEPROM data from F/W. * @send_cpu_message: send buffer to ArmCP. */
@@ -528,6 +529,7 @@ struct hl_asic_funcs {int (*soft_reset_late_init)(struct hl_device *hdev);void (*hw_queues_lock)(struct hl_device *hdev);void (*hw_queues_unlock)(struct hl_device *hdev); +u32 (*get_pci_id)(struct hl_device *hdev);int (*get_eeprom_data)(struct hl_device *hdev,void *data,size_t max_size); int (*send_cpu_message)(struct hl_device *hdev, u32 *msg,
drivers/misc/habanalabs/habanalabs_ioctl.c +126 -0
136 lines not on this page · raw message
include/uapi/misc/habanalabs.h +74 -1
93 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH v3 14/15 habanalabs: add debugfs support
In reply to Oded GabbayThis patch adds debugfs support to the driver. It allows the user-space to display information that is contained in the internal structures of the driver, such as:
- active command submissions
- active user virtual memory mappings
- number of allocated command buffers
It also enables the user to perform reads and writes through Goya's PCI bars.
---
Changes in v3:
- Adjust code to huge page optimization
.../ABI/testing/debugfs-driver-habanalabs | 127 ++ drivers/misc/habanalabs/Makefile | 2 + drivers/misc/habanalabs/command_buffer.c | 4 + drivers/misc/habanalabs/command_submission.c | 12 + drivers/misc/habanalabs/debugfs.c | 1071 +++++++++++++++++ drivers/misc/habanalabs/device.c | 6 + drivers/misc/habanalabs/goya/goya.c | 108 ++ drivers/misc/habanalabs/goya/goyaP.h | 5 + drivers/misc/habanalabs/habanalabs.h | 190 +++ drivers/misc/habanalabs/habanalabs_drv.c | 16 +- drivers/misc/habanalabs/memory.c | 8 + 11 files changed, 1547 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/debugfs.c
Documentation/ABI/testing/debugfs-driver-habanalabs new file +127 -0
@@ -0,0 +1,127 @@ +What: /sys/kernel/debug/habanalabs/hl<n>/addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the device address to be used for read or write through + PCI bar. The acceptable value is a string that starts with "0x" + +What: /sys/kernel/debug/habanalabs/hl<n>/command_buffers +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently allocated + command buffers + +What: /sys/kernel/debug/habanalabs/hl<n>/command_submission +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently active + command submissions + +What: /sys/kernel/debug/habanalabs/hl<n>/command_submission_jobs +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with detailed information about each JOB (CB) of + each active command submission + +What: /sys/kernel/debug/habanalabs/hl<n>/data32 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the root user to read or write directly through the + device's PCI bar. Writing to this file generates a write + transaction while reading from the file generates a read + transcation. This custom interface is needed (instead of using + the generic Linux user-space PCI mapping) because the DDR bar + is very small compared to the DDR memory and only the driver can + move the bar before and after the transaction + +What: /sys/kernel/debug/habanalabs/hl<n>/device +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Enables the root user to set the device to specific state. + Valid values are "disable", "enable", "suspend", "resume". + User can read this property to see the valid values + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C device address for I2C transaction that is generated + by the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_bus +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C bus address for I2C transaction that is generated by + the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_data +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Triggers an I2C transaction that is generated by the device's + CPU. Writing to this file generates a write transaction while + reading from the file generates a read transcation + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_reg +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C register id for I2C transaction that is generated by + the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/led0 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the first S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/led1 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the second S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/led2 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the third S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/mmu +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the hop values and physical address for a given ASID + and virtual address. The user should write the ASID and VA into + the file and then read the file to get the result. + e.g. to display info about VA 0x1000 for ASID 1 you need to do: + echo "1 0x1000" > /sys/kernel/debug/habanalabs/hl0/mmu + +What: /sys/kernel/debug/habanalabs/hl<n>/set_power_state +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the PCI power state. Valid values are "1" for D0 and "2" + for D3Hot + +What: /sys/kernel/debug/habanalabs/hl<n>/userptr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently user + pointers (user virtual addresses) that are pinned and mapped + to DMA addresses + +What: /sys/kernel/debug/habanalabs/hl<n>/vm +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about all the active virtual + address mappings per ASID +
drivers/misc/habanalabs/Makefile +2 -0
@@ -8,6 +8,8 @@ habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ command_submission.o mmu.o +habanalabs-$(CONFIG_DEBUG_FS) += debugfs.o + include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_buffer.c +4 -0
@@ -36,6 +36,8 @@ static void cb_release(struct kref *ref) cb = container_of(ref, struct hl_cb, refcount); hdev = cb->hdev; +hl_debugfs_remove_cb (cb); +cb_do_release (hdev, cb); }
@@ -153,6 +155,8 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, *handle = cb->id |HL_MMAP_CB_MASK ; *handle <<=PAGE_SHIFT ; +hl_debugfs_add_cb (cb); +return 0 ; release_cb:
drivers/misc/habanalabs/command_submission.c +12 -0
@@ -153,6 +153,8 @@ static void free_job(struct hl_device *hdev, struct hl_cs_job *job)list_del (&job->cs_node);spin_unlock (&cs->job_lock); +hl_debugfs_remove_job (hdev, job); +if (job->ext_queue)cs_put (cs);
@@ -216,6 +218,12 @@ static void cs_do_release(struct kref *ref) } } +/* +* Must be called before hl_ctx_put because inside we use ctx to get +* the device +*/ +hl_debugfs_remove_cs (cs); +hl_ctx_put (cs->ctx);if (cs->timedout)
@@ -484,6 +492,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks, *cs_seq = cs->sequence; +hl_debugfs_add_cs (cs); +/* Validate ALL the CS chunks before submitting the CS */ for (i =0 , parse_cnt =0 ; i < num_chunks ; i++, parse_cnt++) {struct hl_cs_chunk *chunk = &cs_chunk_array[i];
@@ -532,6 +542,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks,if (job->ext_queue)cs_get (cs); +hl_debugfs_add_job (hdev, job); + rc =cs_parser (hpriv, job);if (rc) { dev_err(hdev->dev,
drivers/misc/habanalabs/debugfs.c new file +1071 -0
1,071 lines not on this page · raw message
drivers/misc/habanalabs/device.c +6 -0
@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)put_pid (hpriv->taskpid); +hl_debugfs_remove_file (hpriv); +mutex_destroy (&hpriv->restore_phase_mutex);kfree (hpriv);
@@ -823,6 +825,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) goto free_cb_pool; } +hl_debugfs_add_device (hdev); + rc = hdev->asic_funcs->hw_init (hdev);if (rc) {dev_err (hdev->dev,"failed to initialize the H/W\n" );
@@ -950,6 +954,8 @@ void hl_device_fini(struct hl_device *hdev)device_late_fini (hdev); +hl_debugfs_remove_device (hdev); +hl_sysfs_fini (hdev);/*
drivers/misc/habanalabs/goya/goya.c +108 -0
144 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +5 -0
@@ -168,6 +168,10 @@ struct goya_device {u32 hw_cap_initialized; }; +int goya_debugfs_i2c_read (struct hl_device *hdev,u8 i2c_bus, +u8 i2c_addr,u8 i2c_reg,u32 *val); +int goya_debugfs_i2c_write (struct hl_device *hdev,u8 i2c_bus, +u8 i2c_addr,u8 i2c_reg,u32 val);int goya_test_cpu_queue (struct hl_device *hdev);int goya_send_cpu_message (struct hl_device *hdev,u32 *msg,u16 len,u32 timeout,long *result);
@@ -178,6 +182,7 @@ long goya_get_fan_speed(struct hl_device *hdev, int sensor_index, u32 attr);long goya_get_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr);void goya_set_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr,long value); +void goya_debugfs_led_set (struct hl_device *hdev,u8 led,u8 state);void goya_set_pll_profile (struct hl_device *hdev,enum hl_pll_frequency freq);int goya_add_device_attr (struct hl_device *hdev);void goya_remove_device_attr (struct hl_device *hdev);
drivers/misc/habanalabs/habanalabs.h +190 -0
285 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +14 -2
@@ -150,6 +150,8 @@ int hl_device_open(struct inode *inode, struct file *filp)*/ hl_device_set_frequency (hdev,PLL_HIGH ); +hl_debugfs_add_file (hpriv); +return 0 ; out_err:
@@ -417,17 +419,20 @@ static int __init hl_init(void) goto remove_major; } +hl_debugfs_init (); + rc =pci_register_driver (&hl_pci_driver);if (rc) {pr_err ("failed to register pci device\n" ); - goto remove_class; + goto remove_debugfs; }pr_debug ("driver loaded\n" );return 0 ; -remove_class: +remove_debugfs: +hl_debugfs_fini ();class_destroy (hl_class); remove_major:unregister_chrdev_region (MKDEV (hl_major,0 ),HL_MAX_MINORS );
@@ -441,6 +446,13 @@ static void __exit hl_exit(void) {pci_unregister_driver (&hl_pci_driver); +/* +* Removing debugfs must be after all devices or simulator devices +* have been removed because otherwise we get a bug in the +* debugfs module for referencing NULL objects +*/ +hl_debugfs_fini (); +class_destroy (hl_class);unregister_chrdev_region (MKDEV (hl_major,0 ),HL_MAX_MINORS );
drivers/misc/habanalabs/memory.c +8 -0
@@ -1289,6 +1289,8 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u32 size, goto free_sgt; } +hl_debugfs_add_userptr (hdev, userptr); +return 0 ; free_sgt:
@@ -1314,6 +1316,8 @@ int hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) {struct page **pages; +hl_debugfs_remove_userptr (hdev, userptr); +if (userptr->dma_mapped) hdev->asic_funcs->hl_dma_unmap_sg(hdev, userptr->sgt->sgl,
@@ -1475,6 +1479,8 @@ int hl_vm_ctx_init_with_ranges(struct hl_ctx *ctx, u64 host_range_start, goto dram_vm_err; } +hl_debugfs_add_ctx_mem_hash (hdev, ctx); +return 0 ; dram_vm_err:
@@ -1597,6 +1603,8 @@ void hl_vm_ctx_fini(struct hl_ctx *ctx)struct hlist_node *tmp_node;int i; +hl_debugfs_remove_ctx_mem_hash (hdev, ctx); +if (!hash_empty (ctx->mem_hash))dev_notice (hdev->dev,"ctx is freed while it has va in use\n" );
Signature · 1 line
2.17.1
Re: [PATCH v3 14/15] habanalabs: add debugfs support
Quoted 11 lines · Oded Gabbay
This patch adds debugfs support to the driver. It allows the user-space to display information that is contained in the internal structures of the driver, such as:
- active command submissions
- active user virtual memory mappings
- number of allocated command buffersIt also enables the user to perform reads and writes through Goya's PCI bars.
Signed-off-by Oded Gabbay <[email protected]>
Quoted 1,913 lines
---
Changes in v3:
- Adjust code to huge page optimization.../ABI/testing/debugfs-driver-habanalabs | 127 ++ drivers/misc/habanalabs/Makefile | 2 + drivers/misc/habanalabs/command_buffer.c | 4 + drivers/misc/habanalabs/command_submission.c | 12 + drivers/misc/habanalabs/debugfs.c | 1071 +++++++++++++++++ drivers/misc/habanalabs/device.c | 6 + drivers/misc/habanalabs/goya/goya.c | 108 ++ drivers/misc/habanalabs/goya/goyaP.h | 5 + drivers/misc/habanalabs/habanalabs.h | 190 +++ drivers/misc/habanalabs/habanalabs_drv.c | 16 +- drivers/misc/habanalabs/memory.c | 8 + 11 files changed, 1547 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/debugfs.cDocumentation/ABI/testing/debugfs-driver-habanalabs new file +127
-0@@ -0,0 +1,127 @@ +What: /sys/kernel/debug/habanalabs/hl<n>/addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the device address to be used for read or write through + PCI bar. The acceptable value is a string that starts with "0x" + +What: /sys/kernel/debug/habanalabs/hl<n>/command_buffers +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently allocated + command buffers + +What: /sys/kernel/debug/habanalabs/hl<n>/command_submission +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently active + command submissions + +What: /sys/kernel/debug/habanalabs/hl<n>/command_submission_jobs +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with detailed information about each JOB (CB) of + each active command submission + +What: /sys/kernel/debug/habanalabs/hl<n>/data32 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the root user to read or write directly through the + device's PCI bar. Writing to this file generates a write + transaction while reading from the file generates a read + transcation. This custom interface is needed (instead of using + the generic Linux user-space PCI mapping) because the DDR bar + is very small compared to the DDR memory and only the driver can + move the bar before and after the transaction + +What: /sys/kernel/debug/habanalabs/hl<n>/device +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Enables the root user to set the device to specific state. + Valid values are "disable", "enable", "suspend", "resume". + User can read this property to see the valid values + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C device address for I2C transaction that is generated + by the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_bus +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C bus address for I2C transaction that is generated by + the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_data +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Triggers an I2C transaction that is generated by the device's + CPU. Writing to this file generates a write transaction while + reading from the file generates a read transcation + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_reg +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C register id for I2C transaction that is generated by + the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/led0 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the first S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/led1 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the second S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/led2 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the third S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/mmu +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the hop values and physical address for a given ASID + and virtual address. The user should write the ASID and VA into + the file and then read the file to get the result. + e.g. to display info about VA 0x1000 for ASID 1 you need to do: + echo "1 0x1000" > /sys/kernel/debug/habanalabs/hl0/mmu + +What: /sys/kernel/debug/habanalabs/hl<n>/set_power_state +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the PCI power state. Valid values are "1" for D0 and "2" + for D3Hot + +What: /sys/kernel/debug/habanalabs/hl<n>/userptr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently user + pointers (user virtual addresses) that are pinned and mapped + to DMA addresses + +What: /sys/kernel/debug/habanalabs/hl<n>/vm +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about all the active virtual + address mappings per ASID +drivers/misc/habanalabs/Makefile +2
-0@@ -8,6 +8,8 @@ habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ command_submission.o mmu.o +habanalabs-$(CONFIG_DEBUG_FS) += debugfs.o + include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)drivers/misc/habanalabs/command_buffer.c +4
-0@@ -36,6 +36,8 @@ static void cb_release(struct kref *ref) cb = container_of(ref, struct hl_cb, refcount); hdev = cb->hdev; +hl_debugfs_remove_cb (cb); +cb_do_release (hdev, cb); }@@ -153,6 +155,8 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, *handle = cb->id |HL_MMAP_CB_MASK ; *handle <<=PAGE_SHIFT ; +hl_debugfs_add_cb (cb); +return 0 ; release_cb:drivers/misc/habanalabs/command_submission.c +12
-0@@ -153,6 +153,8 @@ static void free_job(struct hl_device *hdev, struct hl_cs_job *job)list_del (&job->cs_node);spin_unlock (&cs->job_lock); +hl_debugfs_remove_job (hdev, job); +if (job->ext_queue)cs_put (cs);@@ -216,6 +218,12 @@ static void cs_do_release(struct kref *ref) } } +/* +* Must be called before hl_ctx_put because inside we use ctx to get +* the device +*/ +hl_debugfs_remove_cs (cs); +hl_ctx_put (cs->ctx);if (cs->timedout)@@ -484,6 +492,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks, *cs_seq = cs->sequence; +hl_debugfs_add_cs (cs); +/* Validate ALL the CS chunks before submitting the CS */ for (i =0 , parse_cnt =0 ; i < num_chunks ; i++, parse_cnt++) {struct hl_cs_chunk *chunk = &cs_chunk_array[i];@@ -532,6 +542,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks,if (job->ext_queue)cs_get (cs); +hl_debugfs_add_job (hdev, job); + rc =cs_parser (hpriv, job);if (rc) { dev_err(hdev->dev,drivers/misc/habanalabs/debugfs.c new file +1071
-0@@ -0,0 +1,1071 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* +* Copyright 2016-2018 HabanaLabs, Ltd. +* All Rights Reserved. +*/ + +#include "habanalabs.h" +#include "include/hw_ip/mmu/mmu_general.h" + +#include <linux/debugfs.h> +#include <linux/uaccess.h> + +#define MMU_ADDR_BUF_SIZE 40 +#define MMU_ASID_BUF_SIZE 10 +#define MMU_KBUF_SIZE (MMU_ADDR_BUF_SIZE + MMU_ASID_BUF_SIZE) + +static struct dentry *hl_debug_root; + +static int hl_debugfs_i2c_read (struct hl_device *hdev,u8 i2c_bus,u8 i2c_addr, +u8 i2c_reg,u32 *val) +{ +struct armcp_packet pkt; +int rc; + +if (hl_device_disabled_or_in_reset (hdev)) +return 0 ; + +memset (&pkt,0 ,sizeof (pkt)); + + pkt.ctl =ARMCP_PACKET_I2C_RD <<ARMCP_PKT_CTL_OPCODE_SHIFT ; + pkt.i2c_bus = i2c_bus; + pkt.i2c_addr = i2c_addr; + pkt.i2c_reg = i2c_reg; + + rc = hdev->asic_funcs->send_cpu_message (hdev, (u32 *) &pkt,sizeof (pkt), +HL_DEVICE_TIMEOUT_USEC , (long *) val); + +if (rc) +dev_err (hdev->dev,"Failed to read from I2C, error %d\n" , rc); + +return rc; +} + +static int hl_debugfs_i2c_write (struct hl_device *hdev,u8 i2c_bus,u8 i2c_addr, +u8 i2c_reg,u32 val) +{ +struct armcp_packet pkt; +int rc; + +if (hl_device_disabled_or_in_reset (hdev)) +return 0 ; + +memset (&pkt,0 ,sizeof (pkt)); + + pkt.ctl =ARMCP_PACKET_I2C_WR <<ARMCP_PKT_CTL_OPCODE_SHIFT ; + pkt.i2c_bus = i2c_bus; + pkt.i2c_addr = i2c_addr; + pkt.i2c_reg = i2c_reg; + pkt.value = val; + + rc = hdev->asic_funcs->send_cpu_message (hdev, (u32 *) &pkt,sizeof (pkt), +HL_DEVICE_TIMEOUT_USEC ,NULL ); + +if (rc) +dev_err (hdev->dev,"Failed to write to I2C, error %d\n" , rc); + +return rc; +} + +static void hl_debugfs_led_set (struct hl_device *hdev,u8 led,u8 state) +{ +struct armcp_packet pkt; +int rc; + +if (hl_device_disabled_or_in_reset (hdev)) +return ; + +memset (&pkt,0 ,sizeof (pkt)); + + pkt.ctl =ARMCP_PACKET_LED_SET <<ARMCP_PKT_CTL_OPCODE_SHIFT ; + pkt.led_index = led; + pkt.value = state; + + rc = hdev->asic_funcs->send_cpu_message (hdev, (u32 *) &pkt,sizeof (pkt), +HL_DEVICE_TIMEOUT_USEC ,NULL ); + +if (rc) +dev_err (hdev->dev,"Failed to set LED %d, error %d\n" , led, rc); +} + +static int command_buffers_show (struct seq_file *s,void *data) +{ +struct hl_debugfs_entry *entry = s->private; +struct hl_dbg_device_entry *dev_entry = entry->dev_entry; +struct hl_cb *cb; +bool first = true; + +spin_lock (&dev_entry->cb_spinlock); + +list_for_each_entry (cb, &dev_entry->cb_list, debugfs_list) { +if (first) { + first = false; +seq_puts (s,"\n" ); +seq_puts (s," CB ID CTX ID CB size CB RefCnt mmap? CS counter\n" ); +seq_puts (s,"---------------------------------------------------------------\n" ); + } +seq_printf (s, +" %03d %d 0x%08x %d %d %d\n" , + cb->id, cb->ctx_id, cb->size, +kref_read (&cb->refcount), + cb->mmap, cb->cs_cnt); + } + +spin_unlock (&dev_entry->cb_spinlock); + +if (!first) +seq_puts (s,"\n" ); + +return 0 ; +} + +static int command_submission_show (struct seq_file *s,void *data) +{ +struct hl_debugfs_entry *entry = s->private; +struct hl_dbg_device_entry *dev_entry = entry->dev_entry; +struct hl_cs *cs; +bool first = true; + +spin_lock (&dev_entry->cs_spinlock); + +list_for_each_entry (cs, &dev_entry->cs_list, debugfs_list) { +if (first) { + first = false; +seq_puts (s,"\n" ); +seq_puts (s," CS ID CTX ASID CS RefCnt Submitted Completed\n" ); +seq_puts (s,"------------------------------------------------------\n" ); + } +seq_printf (s, +" %llu %d %d %d %d\n" , + cs->sequence, cs->ctx->asid, +kref_read (&cs->refcount), + cs->submitted, cs->completed); + } + +spin_unlock (&dev_entry->cs_spinlock); + +if (!first) +seq_puts (s,"\n" ); + +return 0 ; +} + +static int command_submission_jobs_show (struct seq_file *s,void *data) +{ +struct hl_debugfs_entry *entry = s->private; +struct hl_dbg_device_entry *dev_entry = entry->dev_entry; +struct hl_cs_job *job; +bool first = true; + +spin_lock (&dev_entry->cs_job_spinlock); + +list_for_each_entry (job, &dev_entry->cs_job_list, debugfs_list) { +if (first) { + first = false; +seq_puts (s,"\n" ); +seq_puts (s," JOB ID CS ID CTX ASID H/W Queue\n" ); +seq_puts (s,"---------------------------------------\n" ); + } +if (job->cs) +seq_printf (s, +" %02d %llu %d %d\n" , + job->id, job->cs->sequence, job->cs->ctx->asid, + job->hw_queue_id); +else +seq_printf (s, +" %02d 0 %d %d\n" , + job->id,HL_KERNEL_ASID_ID , job->hw_queue_id); + } + +spin_unlock (&dev_entry->cs_job_spinlock); + +if (!first) +seq_puts (s,"\n" ); + +return 0 ; +} + +static int userptr_show (struct seq_file *s,void *data) +{ +struct hl_debugfs_entry *entry = s->private; +struct hl_dbg_device_entry *dev_entry = entry->dev_entry; +struct hl_userptr *userptr; +char dma_dir[4 ][30 ] = {"DMA_BIDIRECTIONAL" ,"DMA_TO_DEVICE" , +"DMA_FROM_DEVICE" ,"DMA_NONE" }; +bool first = true; + +spin_lock (&dev_entry->userptr_spinlock); + +list_for_each_entry (userptr, &dev_entry->userptr_list, debugfs_list) { +if (first) { + first = false; +seq_puts (s,"\n" ); +seq_puts (s," user virtual address size dma dir\n" ); +seq_puts (s,"----------------------------------------------------------\n" ); + } +seq_printf (s, +" 0x%-14llx %-10u %-30s\n" , + userptr->addr, userptr->size, dma_dir[userptr->dir]); + } + +spin_unlock (&dev_entry->userptr_spinlock); + +if (!first) +seq_puts (s,"\n" ); + +return 0 ; +} + +static int vm_show (struct seq_file *s,void *data) +{ +struct hl_debugfs_entry *entry = s->private; +struct hl_dbg_device_entry *dev_entry = entry->dev_entry; +struct hl_ctx *ctx; +struct hl_vm *vm; +struct hl_vm_hash_node *hnode; +struct hl_userptr *userptr; +struct hl_vm_phys_pg_pack *phys_pg_pack =NULL ; +enum vm_type_t *vm_type; +bool once = true; +int i; + +if (!dev_entry->hdev->mmu_enable) +return 0 ; + +spin_lock (&dev_entry->ctx_mem_hash_spinlock); + +list_for_each_entry (ctx, &dev_entry->ctx_mem_hash_list, debugfs_list) { + once = false; +seq_puts (s,"\n\n----------------------------------------------------" ); +seq_puts (s,"\n----------------------------------------------------\n\n" ); +seq_printf (s,"ctx asid: %u\n" , ctx->asid); + +seq_puts (s,"\nmappings:\n\n" ); +seq_puts (s," virtual address size handle\n" ); +seq_puts (s,"----------------------------------------------------\n" ); +mutex_lock (&ctx->mem_hash_lock); +hash_for_each (ctx->mem_hash, i, hnode, node) { + vm_type = hnode->ptr; + +if (*vm_type ==VM_TYPE_USERPTR ) { + userptr = hnode->ptr; +seq_printf (s, +" 0x%-14llx %-10u\n" , + hnode->vaddr, userptr->size); + }else { + phys_pg_pack = hnode->ptr; +seq_printf (s, +" 0x%-14llx %-10u %-4u\n" , + hnode->vaddr, phys_pg_pack->total_size, + phys_pg_pack->handle); + } + } +mutex_unlock (&ctx->mem_hash_lock); + + vm = &ctx->hdev->vm; +spin_lock (&vm->idr_lock); + +if (!idr_is_empty (&vm->phys_pg_pack_handles)) +seq_puts (s,"\n\nallocations:\n" ); + +idr_for_each_entry (&vm->phys_pg_pack_handles, phys_pg_pack, i) { +if (phys_pg_pack->asid != ctx->asid) +continue ; + +seq_printf (s,"\nhandle: %u\n" , phys_pg_pack->handle); +seq_printf (s,"page size: %u\n\n" , + phys_pg_pack->page_size); +seq_puts (s," physical address\n" ); +seq_puts (s,"---------------------\n" ); +for (i =0 ; i < phys_pg_pack->npages ; i++) { +seq_printf (s," 0x%-14llx\n" , + phys_pg_pack->pages[i]); + } + } +spin_unlock (&vm->idr_lock); + + } + +spin_unlock (&dev_entry->ctx_mem_hash_spinlock); + +if (!once) +seq_puts (s,"\n" ); + +return 0 ; +} + +/* these inline functions are copied from mmu.c */ +static inline u64 get_hop0_addr (struct hl_ctx *ctx) +{ +return ctx->hdev->asic_prop.mmu_pgt_addr + + (ctx->asid * ctx->hdev->asic_prop.mmu_hop_table_size); +} + +static inline u64 get_hop0_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP0_MASK ) >>HOP0_SHIFT ); +} + +static inline u64 get_hop1_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP1_MASK ) >>HOP1_SHIFT ); +} + +static inline u64 get_hop2_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP2_MASK ) >>HOP2_SHIFT ); +} + +static inline u64 get_hop3_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP3_MASK ) >>HOP3_SHIFT ); +} + +static inline u64 get_hop4_pte_addr (struct hl_ctx *ctx,u64 hop_addr, +u64 virt_addr) +{ +return hop_addr + ctx->hdev->asic_prop.mmu_pte_size * + ((virt_addr &HOP4_MASK ) >>HOP4_SHIFT ); +} + +static inline u64 get_next_hop_addr (u64 curr_pte) +{ +if (curr_pte &PAGE_PRESENT_MASK ) +return curr_pte &PHYS_ADDR_MASK ; +else +return ULLONG_MAX ; +} + +static int mmu_show (struct seq_file *s,void *data) +{ +struct hl_debugfs_entry *entry = s->private; +struct hl_dbg_device_entry *dev_entry = entry->dev_entry; +struct hl_device *hdev = dev_entry->hdev; +struct hl_ctx *ctx = hdev->user_ctx; + +u64 hop0_addr =0 , hop0_pte_addr =0 , hop0_pte =0 , + hop1_addr =0 , hop1_pte_addr =0 , hop1_pte =0 , + hop2_addr =0 , hop2_pte_addr =0 , hop2_pte =0 , + hop3_addr =0 , hop3_pte_addr =0 , hop3_pte =0 , + hop4_addr =0 , hop4_pte_addr =0 , hop4_pte =0 , + virt_addr = dev_entry->mmu_addr; + +if (!hdev->mmu_enable) +return 0 ; + +if (!ctx) { +dev_err (hdev->dev,"no ctx available\n" ); +return 0 ; + } + +mutex_lock (&ctx->mmu_lock); + +/* the following lookup is copied from unmap() in mmu.c */ + + hop0_addr =get_hop0_addr (ctx); + hop0_pte_addr =get_hop0_pte_addr (ctx, hop0_addr, virt_addr); + hop0_pte = hdev->asic_funcs->read_pte (hdev, hop0_pte_addr); + hop1_addr =get_next_hop_addr (hop0_pte); + +if (hop1_addr ==ULLONG_MAX ) + goto not_mapped; + + hop1_pte_addr =get_hop1_pte_addr (ctx, hop1_addr, virt_addr); + hop1_pte = hdev->asic_funcs->read_pte (hdev, hop1_pte_addr); + hop2_addr =get_next_hop_addr (hop1_pte); + +if (hop2_addr ==ULLONG_MAX ) + goto not_mapped; + + hop2_pte_addr =get_hop2_pte_addr (ctx, hop2_addr, virt_addr); + hop2_pte = hdev->asic_funcs->read_pte (hdev, hop2_pte_addr); + hop3_addr =get_next_hop_addr (hop2_pte); + +if (hop3_addr ==ULLONG_MAX ) + goto not_mapped; + + hop3_pte_addr =get_hop3_pte_addr (ctx, hop3_addr, virt_addr); + hop3_pte = hdev->asic_funcs->read_pte (hdev, hop3_pte_addr); + +if (!(hop3_pte &LAST_MASK )) { + hop4_addr =get_next_hop_addr (hop3_pte); + +if (hop4_addr ==ULLONG_MAX ) + goto not_mapped; + + hop4_pte_addr =get_hop4_pte_addr (ctx, hop4_addr, virt_addr); + hop4_pte = hdev->asic_funcs->read_pte (hdev, hop4_pte_addr); +if (!(hop4_pte &PAGE_PRESENT_MASK )) + goto not_mapped; + }else { +if (!(hop3_pte &PAGE_PRESENT_MASK )) + goto not_mapped; + } + +seq_printf (s,"asid: %u, virt_addr: 0x%llx\n" , + dev_entry->mmu_asid, dev_entry->mmu_addr); + +seq_printf (s,"hop0_addr: 0x%llx\n" , hop0_addr); +seq_printf (s,"hop0_pte_addr: 0x%llx\n" , hop0_pte_addr); +seq_printf (s,"hop0_pte: 0x%llx\n" , hop0_pte); + +seq_printf (s,"hop1_addr: 0x%llx\n" , hop1_addr); +seq_printf (s,"hop1_pte_addr: 0x%llx\n" , hop1_pte_addr); +seq_printf (s,"hop1_pte: 0x%llx\n" , hop1_pte); + +seq_printf (s,"hop2_addr: 0x%llx\n" , hop2_addr); +seq_printf (s,"hop2_pte_addr: 0x%llx\n" , hop2_pte_addr); +seq_printf (s,"hop2_pte: 0x%llx\n" , hop2_pte); + +seq_printf (s,"hop3_addr: 0x%llx\n" , hop3_addr); +seq_printf (s,"hop3_pte_addr: 0x%llx\n" , hop3_pte_addr); +seq_printf (s,"hop3_pte: 0x%llx\n" , hop3_pte); + +if (!(hop3_pte &LAST_MASK )) { +seq_printf (s,"hop4_addr: 0x%llx\n" , hop4_addr); +seq_printf (s,"hop4_pte_addr: 0x%llx\n" , hop4_pte_addr); +seq_printf (s,"hop4_pte: 0x%llx\n" , hop4_pte); + } + + goto out; + +not_mapped: +dev_err (hdev->dev,"virt addr 0x%llx is not mapped to phys addr\n" , + virt_addr); +out: +mutex_unlock (&ctx->mmu_lock); + +return 0 ; +} + +static ssize_t mmu_write (struct file *file,const char __user *buf, +size_t count,loff_t *f_pos) +{ +struct seq_file *s = file->private_data; +struct hl_debugfs_entry *entry = s->private; +struct hl_dbg_device_entry *dev_entry = entry->dev_entry; +struct hl_device *hdev = dev_entry->hdev; +char kbuf[MMU_KBUF_SIZE ], asid_kbuf[MMU_ASID_BUF_SIZE ], + addr_kbuf[MMU_ADDR_BUF_SIZE ]; +char *c; +ssize_t rc; + +if (!hdev->mmu_enable) +return count; + +memset (kbuf,0 ,sizeof (kbuf)); +memset (asid_kbuf,0 ,sizeof (asid_kbuf)); +memset (addr_kbuf,0 ,sizeof (addr_kbuf)); + +if (copy_from_user (kbuf, buf, count)) + goto err; + + kbuf[MMU_KBUF_SIZE -1 ] =0 ; + + c =strchr (kbuf,' ' ); +if (!c) + goto err; + +memcpy (asid_kbuf, kbuf, c - kbuf); + + rc =kstrtouint (asid_kbuf,10 , &dev_entry->mmu_asid); +if (rc) + goto err; + + c =strstr (kbuf," 0x" ); +if (!c) + goto err; + + c +=3 ; +memcpy (addr_kbuf, c, (kbuf + count) - c); + + rc =kstrtoull (addr_kbuf,16 , &dev_entry->mmu_addr); +if (rc) + goto err; + +return count; + +err: +dev_err (hdev->dev,"usage: echo <asid> <0xaddr> > mmu\n" ); + +return -EINVAL ; +} + +static ssize_t hl_data_read32 (struct file *f,char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +char tmp_buf[32 ]; +u32 val; +ssize_t rc; + +if (*ppos) +return 0 ; + + rc = hdev->asic_funcs->debugfs_read32 (hdev, entry->addr, &val); +if (rc) { +dev_err (hdev->dev,"Failed to read from 0x%010llx\n" , + entry->addr); +return rc; + } + +sprintf (tmp_buf,"0x%08x\n" , val); + rc =simple_read_from_buffer (buf,strlen (tmp_buf) +1 , ppos, tmp_buf, +strlen (tmp_buf) +1 ); + +return rc; +} + +static ssize_t hl_data_write32 (struct file *f,const char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +u32 value; +ssize_t rc; + + rc =kstrtouint_from_user (buf, count,16 , &value); +if (rc) +return rc; + + rc = hdev->asic_funcs->debugfs_write32 (hdev, entry->addr, value); +if (rc) { +dev_err (hdev->dev,"Failed to write 0x%08x to 0x%010llx\n" , + value, entry->addr); +return rc; + } + +return count; +} + +static ssize_t hl_get_power_state (struct file *f,char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +char tmp_buf[200 ]; +ssize_t rc; +int i; + +if (*ppos) +return 0 ; + +if (hdev->pdev->current_state ==PCI_D0 ) + i =1 ; +else if (hdev->pdev->current_state == PCI_D3hot) + i =2 ; +else + i =3 ; + +sprintf (tmp_buf, +"current power state: %d\n1 - D0\n2 - D3hot\n3 - Unknown\n" , i); + rc =simple_read_from_buffer (buf,strlen (tmp_buf) +1 , ppos, tmp_buf, +strlen (tmp_buf) +1 ); + +return rc; +} + +static ssize_t hl_set_power_state (struct file *f,const char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +u32 value; +ssize_t rc; + + rc =kstrtouint_from_user (buf, count,10 , &value); +if (rc) +return rc; + +if (value ==1 ) { +pci_set_power_state (hdev->pdev,PCI_D0 ); +pci_restore_state (hdev->pdev); + rc =pci_enable_device (hdev->pdev); + }else if (value ==2 ) { +pci_save_state (hdev->pdev); +pci_disable_device (hdev->pdev); +pci_set_power_state (hdev->pdev, PCI_D3hot); + }else { +dev_dbg (hdev->dev,"invalid power state value %u\n" , value); +return -EINVAL ; + } + +return count; +} + +static ssize_t hl_i2c_data_read (struct file *f,char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +char tmp_buf[32 ]; +u32 val; +ssize_t rc; + +if (*ppos) +return 0 ; + + rc =hl_debugfs_i2c_read (hdev, entry->i2c_bus, entry->i2c_addr, + entry->i2c_reg, &val); +if (rc) { +dev_err (hdev->dev, +"Failed to read from I2C bus %d, addr %d, reg %d\n" , + entry->i2c_bus, entry->i2c_addr, entry->i2c_reg); +return rc; + } + +sprintf (tmp_buf,"0x%02x\n" , val); + rc =simple_read_from_buffer (buf,strlen (tmp_buf) +1 , ppos, tmp_buf, +strlen (tmp_buf) +1 ); + +return rc; +} + +static ssize_t hl_i2c_data_write (struct file *f,const char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +u32 value; +ssize_t rc; + + rc =kstrtouint_from_user (buf, count,16 , &value); +if (rc) +return rc; + + rc =hl_debugfs_i2c_write (hdev, entry->i2c_bus, entry->i2c_addr, + entry->i2c_reg, value); +if (rc) { +dev_err (hdev->dev, +"Failed to write 0x%02x to I2C bus %d, addr %d, reg %d\n" , + value, entry->i2c_bus, entry->i2c_addr, entry->i2c_reg); +return rc; + } + +return count; +} + +static ssize_t hl_led0_write (struct file *f,const char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +u32 value; +ssize_t rc; + + rc =kstrtouint_from_user (buf, count,10 , &value); +if (rc) +return rc; + + value = value ?1 :0 ; + +hl_debugfs_led_set (hdev,0 , value); + +return count; +} + +static ssize_t hl_led1_write (struct file *f,const char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +u32 value; +ssize_t rc; + + rc =kstrtouint_from_user (buf, count,10 , &value); +if (rc) +return rc; + + value = value ?1 :0 ; + +hl_debugfs_led_set (hdev,1 , value); + +return count; +} + +static ssize_t hl_led2_write (struct file *f,const char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; +u32 value; +ssize_t rc; + + rc =kstrtouint_from_user (buf, count,10 , &value); +if (rc) +return rc; + + value = value ?1 :0 ; + +hl_debugfs_led_set (hdev,2 , value); + +return count; +} + +static ssize_t hl_device_read (struct file *f,char __user *buf, +size_t count,loff_t *ppos) +{ +char tmp_buf[200 ]; +ssize_t rc; + +if (*ppos) +return 0 ; + +sprintf (tmp_buf, +"Valid values are: disable, enable, suspend, resume\n" ); + rc =simple_read_from_buffer (buf,strlen (tmp_buf) +1 , ppos, tmp_buf, +strlen (tmp_buf) +1 ); + +return rc; +} + +static ssize_t hl_device_write (struct file *f,const char __user *buf, +size_t count,loff_t *ppos) +{ +struct hl_dbg_device_entry *entry =file_inode (f)->i_private; +struct hl_device *hdev = entry->hdev; + +if (strncmp ("disable" , buf,strlen ("disable" )) ==0 ) { + hdev->disabled = true; + }else if (strncmp ("enable" , buf,strlen ("enable" )) ==0 ) { + hdev->disabled = false; + }else if (strncmp ("suspend" , buf,strlen ("suspend" )) ==0 ) { + hdev->asic_funcs->suspend (hdev); + }else if (strncmp ("resume" , buf,strlen ("resume" )) ==0 ) { + hdev->asic_funcs->resume (hdev); + }else { +dev_err (hdev->dev, +"Valid values are: disable, enable, suspend, resume\n" ); + count = -EINVAL ; + } + +return count; +} + +static const struct file_operations hl_data32b_fops = { + .owner =THIS_MODULE , + .read = hl_data_read32, + .write = hl_data_write32 +}; + +static const struct file_operations hl_i2c_data_fops = { + .owner =THIS_MODULE , + .read = hl_i2c_data_read, + .write = hl_i2c_data_write +}; + +static const struct file_operations hl_power_fops = { + .owner =THIS_MODULE , + .read = hl_get_power_state, + .write = hl_set_power_state +}; + +static const struct file_operations hl_led0_fops = { + .owner =THIS_MODULE , + .write = hl_led0_write +}; + +static const struct file_operations hl_led1_fops = { + .owner =THIS_MODULE , + .write = hl_led1_write +}; + +static const struct file_operations hl_led2_fops = { + .owner =THIS_MODULE , + .write = hl_led2_write +}; + +static const struct file_operations hl_device_fops = { + .owner =THIS_MODULE , + .read = hl_device_read, + .write = hl_device_write +}; + +static const struct hl_info_list hl_debugfs_list[] = { + {"command_buffers" , command_buffers_show,NULL }, + {"command_submission" , command_submission_show,NULL }, + {"command_submission_jobs" , command_submission_jobs_show,NULL }, + {"userptr" , userptr_show,NULL }, + {"vm" , vm_show,NULL }, + {"mmu" , mmu_show, mmu_write}, +}; + +static int hl_debugfs_open (struct inode *inode,struct file *file) +{ +struct hl_debugfs_entry *node = inode->i_private; + +return single_open (file, node->info_ent->show, node); +} + +static ssize_t hl_debugfs_write (struct file *file,const char __user *buf, +size_t count,loff_t *f_pos) +{ +struct hl_debugfs_entry *node = file->f_inode->i_private; + +if (node->info_ent->write) +return node->info_ent->write (file, buf, count, f_pos); +else +return -EINVAL ; + +} + +static const struct file_operations hl_debugfs_fops = { + .owner =THIS_MODULE , + .open = hl_debugfs_open, + .read = seq_read, + .write = hl_debugfs_write, + .llseek = seq_lseek, + .release = single_release, +}; + +void hl_debugfs_add_device (struct hl_device *hdev) +{ +struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; +int count =ARRAY_SIZE (hl_debugfs_list); +struct hl_debugfs_entry *entry; +struct dentry *ent; +int i; + + dev_entry->hdev = hdev; + dev_entry->entry_arr =kmalloc_array (count, +sizeof (struct hl_debugfs_entry ), +GFP_KERNEL ); +if (!dev_entry->entry_arr) +return ; + +INIT_LIST_HEAD (&dev_entry->file_list); +INIT_LIST_HEAD (&dev_entry->cb_list); +INIT_LIST_HEAD (&dev_entry->cs_list); +INIT_LIST_HEAD (&dev_entry->cs_job_list); +INIT_LIST_HEAD (&dev_entry->userptr_list); +INIT_LIST_HEAD (&dev_entry->ctx_mem_hash_list); +mutex_init (&dev_entry->file_mutex); +spin_lock_init (&dev_entry->cb_spinlock); +spin_lock_init (&dev_entry->cs_spinlock); +spin_lock_init (&dev_entry->cs_job_spinlock); +spin_lock_init (&dev_entry->userptr_spinlock); +spin_lock_init (&dev_entry->ctx_mem_hash_spinlock); + + dev_entry->root =debugfs_create_dir (dev_name (hdev->dev), + hl_debug_root); + +debugfs_create_x64 ("addr" , +0644 , + dev_entry->root, + &dev_entry->addr); + +debugfs_create_file ("data32" , +0644 , + dev_entry->root, + dev_entry, + &hl_data32b_fops); + +debugfs_create_file ("set_power_state" , +0200 , + dev_entry->root, + dev_entry, + &hl_power_fops); + +debugfs_create_u8 ("i2c_bus" , +0644 , + dev_entry->root, + &dev_entry->i2c_bus); + +debugfs_create_u8 ("i2c_addr" , +0644 , + dev_entry->root, + &dev_entry->i2c_addr); + +debugfs_create_u8 ("i2c_reg" , +0644 , + dev_entry->root, + &dev_entry->i2c_reg); + +debugfs_create_file ("i2c_data" , +0644 , + dev_entry->root, + dev_entry, + &hl_i2c_data_fops); + +debugfs_create_file ("led0" , +0200 , + dev_entry->root, + dev_entry, + &hl_led0_fops); + +debugfs_create_file ("led1" , +0200 , + dev_entry->root, + dev_entry, + &hl_led1_fops); + +debugfs_create_file ("led2" , +0200 , + dev_entry->root, + dev_entry, + &hl_led2_fops); + +debugfs_create_file ("device" , +0200 , + dev_entry->root, + dev_entry, + &hl_device_fops); + +for (i =0 , entry = dev_entry->entry_arr ; i < count ; i++, entry++) { + + ent =debugfs_create_file (hl_debugfs_list[i].name, +0444 , + dev_entry->root, + entry, + &hl_debugfs_fops); + entry->dent = ent; + entry->info_ent = &hl_debugfs_list[i]; + entry->dev_entry = dev_entry; + } +} + +void hl_debugfs_remove_device (struct hl_device *hdev) +{ +struct hl_dbg_device_entry *entry = &hdev->hl_debugfs; + +debugfs_remove_recursive (entry->root); + +mutex_destroy (&entry->file_mutex); +kfree (entry->entry_arr); +} + +void hl_debugfs_add_file (struct hl_fpriv *hpriv) +{ +struct hl_dbg_device_entry *dev_entry = &hpriv->hdev->hl_debugfs; + +mutex_lock (&dev_entry->file_mutex); +list_add (&hpriv->debugfs_list, &dev_entry->file_list); +mutex_unlock (&dev_entry->file_mutex); +} + +void hl_debugfs_remove_file (struct hl_fpriv *hpriv) +{ +struct hl_dbg_device_entry *dev_entry = &hpriv->hdev->hl_debugfs; + +mutex_lock (&dev_entry->file_mutex); +list_del (&hpriv->debugfs_list); +mutex_unlock (&dev_entry->file_mutex); +} + +void hl_debugfs_add_cb (struct hl_cb *cb) +{ +struct hl_dbg_device_entry *dev_entry = &cb->hdev->hl_debugfs; + +spin_lock (&dev_entry->cb_spinlock); +list_add (&cb->debugfs_list, &dev_entry->cb_list); +spin_unlock (&dev_entry->cb_spinlock); +} + +void hl_debugfs_remove_cb (struct hl_cb *cb) +{ +struct hl_dbg_device_entry *dev_entry = &cb->hdev->hl_debugfs; + +spin_lock (&dev_entry->cb_spinlock); +list_del (&cb->debugfs_list); +spin_unlock (&dev_entry->cb_spinlock); +} + +void hl_debugfs_add_cs (struct hl_cs *cs) +{ +struct hl_dbg_device_entry *dev_entry = &cs->ctx->hdev->hl_debugfs; + +spin_lock (&dev_entry->cs_spinlock); +list_add (&cs->debugfs_list, &dev_entry->cs_list); +spin_unlock (&dev_entry->cs_spinlock); +} + +void hl_debugfs_remove_cs (struct hl_cs *cs) +{ +struct hl_dbg_device_entry *dev_entry = &cs->ctx->hdev->hl_debugfs; + +spin_lock (&dev_entry->cs_spinlock); +list_del (&cs->debugfs_list); +spin_unlock (&dev_entry->cs_spinlock); +} + +void hl_debugfs_add_job (struct hl_device *hdev,struct hl_cs_job *job) +{ +struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; + +spin_lock (&dev_entry->cs_job_spinlock); +list_add (&job->debugfs_list, &dev_entry->cs_job_list); +spin_unlock (&dev_entry->cs_job_spinlock); +} + +void hl_debugfs_remove_job (struct hl_device *hdev,struct hl_cs_job *job) +{ +struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; + +spin_lock (&dev_entry->cs_job_spinlock); +list_del (&job->debugfs_list); +spin_unlock (&dev_entry->cs_job_spinlock); +} + +void hl_debugfs_add_userptr (struct hl_device *hdev,struct hl_userptr *userptr) +{ +struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; + +spin_lock (&dev_entry->userptr_spinlock); +list_add (&userptr->debugfs_list, &dev_entry->userptr_list); +spin_unlock (&dev_entry->userptr_spinlock); +} + +void hl_debugfs_remove_userptr (struct hl_device *hdev, +struct hl_userptr *userptr) +{ +struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; + +spin_lock (&dev_entry->userptr_spinlock); +list_del (&userptr->debugfs_list); +spin_unlock (&dev_entry->userptr_spinlock); +} + +void hl_debugfs_add_ctx_mem_hash (struct hl_device *hdev,struct hl_ctx *ctx) +{ +struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; + +spin_lock (&dev_entry->ctx_mem_hash_spinlock); +list_add (&ctx->debugfs_list, &dev_entry->ctx_mem_hash_list); +spin_unlock (&dev_entry->ctx_mem_hash_spinlock); +} + +void hl_debugfs_remove_ctx_mem_hash (struct hl_device *hdev,struct hl_ctx *ctx) +{ +struct hl_dbg_device_entry *dev_entry = &hdev->hl_debugfs; + +spin_lock (&dev_entry->ctx_mem_hash_spinlock); +list_del (&ctx->debugfs_list); +spin_unlock (&dev_entry->ctx_mem_hash_spinlock); +} + +int __init hl_debugfs_init (void ) +{ + hl_debug_root =debugfs_create_dir ("habanalabs" ,NULL ); +if (IS_ERR_OR_NULL (hl_debug_root)) { +pr_err ("can not create debugfs directory\n" ); + hl_debug_root =NULL ; +return -ENOMEM ; + } + +return 0 ; +} + +void hl_debugfs_fini (void ) +{ +debugfs_remove_recursive (hl_debug_root); +}drivers/misc/habanalabs/device.c +6
-0@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)put_pid (hpriv->taskpid); +hl_debugfs_remove_file (hpriv); +mutex_destroy (&hpriv->restore_phase_mutex);kfree (hpriv);@@ -823,6 +825,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) goto free_cb_pool; } +hl_debugfs_add_device (hdev); + rc = hdev->asic_funcs->hw_init (hdev);if (rc) {dev_err (hdev->dev,"failed to initialize the H/W\n" );@@ -950,6 +954,8 @@ void hl_device_fini(struct hl_device *hdev)device_late_fini (hdev); +hl_debugfs_remove_device (hdev); +hl_sysfs_fini (hdev);/* drivers/misc/habanalabs/goya/goya.c +108
-0@@ -4358,6 +4358,8 @@ int goya_context_switch(struct hl_device *hdev, u32 asid) job->user_cb_size = cb_size; job->hw_queue_id =GOYA_QUEUE_ID_DMA_0 ; +hl_debugfs_add_job (hdev, job); + parser.ctx_id =HL_KERNEL_ASID_ID ; parser.cs_sequence =0 ; parser.job_id = job->id;@@ -4390,6 +4392,7 @@ int goya_context_switch(struct hl_device *hdev, u32 asid) free_job:hl_userptr_delete_list (hdev, &job->userptr_list); +hl_debugfs_remove_job (hdev, job);kfree (job); cb->cs_cnt--;@@ -4420,6 +4423,106 @@ void goya_restore_phase_topology(struct hl_device *hdev) i =RREG32 (mmSYNC_MNGR_SOB_OBJ_0); } +/* +* goya_debugfs_read32 - read a 32bit value from a given device address +* +* @hdev: pointer to hl_device structure +* @addr: address in device +* @val: returned value +* +* In case of DDR address that is not mapped into the default aperture that +* the DDR bar exposes, the function will configure the iATU so that the DDR +* bar will be positioned at a base address that allows reading from the +* required address. Configuring the iATU during normal operation can +* lead to undefined behavior and therefore, should be done with extreme care +* +*/ +int goya_debugfs_read32 (struct hl_device *hdev,u64 addr,u32 *val) +{ +struct asic_fixed_properties *prop = &hdev->asic_prop; +int rc =0 ; + +if ((addr >=CFG_BASE ) && (addr <CFG_BASE +CFG_SIZE )) { + *val =RREG32 (addr -CFG_BASE ); + + }else if ((addr >=SRAM_BASE_ADDR ) && + (addr <SRAM_BASE_ADDR +SRAM_SIZE )) { + + *val =readl (hdev->pcie_bar[SRAM_CFG_BAR_ID ] + + (addr -SRAM_BASE_ADDR )); + + }else if ((addr >=DRAM_PHYS_BASE ) && + (addr <DRAM_PHYS_BASE + hdev->asic_prop.dram_size)) { + +u64 bar_base_addr =DRAM_PHYS_BASE + + (addr & ~(prop->dram_pci_bar_size -0x1ull )); + + rc =goya_set_ddr_bar_base (hdev, bar_base_addr); +if (!rc) { + *val =readl (hdev->pcie_bar[DDR_BAR_ID ] + + (addr - bar_base_addr)); + + rc =goya_set_ddr_bar_base (hdev,DRAM_PHYS_BASE + + (MMU_PAGE_TABLES_ADDR & + ~(prop->dram_pci_bar_size -0x1ull ))); + } + }else { + rc = -EFAULT ; + } + +return rc; +} + +/* +* goya_debugfs_write32 - write a 32bit value to a given device address +* +* @hdev: pointer to hl_device structure +* @addr: address in device +* @val: returned value +* +* In case of DDR address that is not mapped into the default aperture that +* the DDR bar exposes, the function will configure the iATU so that the DDR +* bar will be positioned at a base address that allows writing to the +* required address. Configuring the iATU during normal operation can +* lead to undefined behavior and therefore, should be done with extreme care +* +*/ +int goya_debugfs_write32 (struct hl_device *hdev,u64 addr,u32 val) +{ +struct asic_fixed_properties *prop = &hdev->asic_prop; +int rc =0 ; + +if ((addr >=CFG_BASE ) && (addr <CFG_BASE +CFG_SIZE )) { +WREG32 (addr -CFG_BASE , val); + + }else if ((addr >=SRAM_BASE_ADDR ) && + (addr <SRAM_BASE_ADDR +SRAM_SIZE )) { + +writel (val, hdev->pcie_bar[SRAM_CFG_BAR_ID ] + + (addr -SRAM_BASE_ADDR )); + + }else if ((addr >=DRAM_PHYS_BASE ) && + (addr <DRAM_PHYS_BASE + hdev->asic_prop.dram_size)) { + +u64 bar_base_addr =DRAM_PHYS_BASE + + (addr & ~(prop->dram_pci_bar_size -0x1ull )); + + rc =goya_set_ddr_bar_base (hdev, bar_base_addr); +if (!rc) { +writel (val, hdev->pcie_bar[DDR_BAR_ID ] + + (addr - bar_base_addr)); + + rc =goya_set_ddr_bar_base (hdev,DRAM_PHYS_BASE + + (MMU_PAGE_TABLES_ADDR & + ~(prop->dram_pci_bar_size -0x1ull ))); + } + }else { + rc = -EFAULT ; + } + +return rc; +} +static u64 goya_read_pte (struct hl_device *hdev,u64 addr) {struct goya_device *goya = hdev->asic_specific;@@ -4766,6 +4869,8 @@ static int goya_mmu_clear_pgt_range(struct hl_device *hdev) job->user_cb_size = cb_size; job->hw_queue_id =GOYA_QUEUE_ID_DMA_0 ; +hl_debugfs_add_job (hdev, job); + parser.ctx_id =HL_KERNEL_ASID_ID ; parser.cs_sequence =0 ; parser.job_id = job->id;@@ -4794,6 +4899,7 @@ static int goya_mmu_clear_pgt_range(struct hl_device *hdev) free_job:hl_userptr_delete_list (hdev, &job->userptr_list); +hl_debugfs_remove_job (hdev, job);kfree (job); cb->cs_cnt--;@@ -5203,6 +5309,8 @@ static const struct hl_asic_funcs goya_funcs = { .update_eq_ci = goya_update_eq_ci, .context_switch = goya_context_switch, .restore_phase_topology = goya_restore_phase_topology, + .debugfs_read32 = goya_debugfs_read32, + .debugfs_write32 = goya_debugfs_write32, .add_device_attr = goya_add_device_attr, .remove_device_attr = goya_remove_device_attr, .handle_eqe = goya_handle_eqe,drivers/misc/habanalabs/goya/goyaP.h +5
-0@@ -168,6 +168,10 @@ struct goya_device {u32 hw_cap_initialized; }; +int goya_debugfs_i2c_read (struct hl_device *hdev,u8 i2c_bus, +u8 i2c_addr,u8 i2c_reg,u32 *val); +int goya_debugfs_i2c_write (struct hl_device *hdev,u8 i2c_bus, +u8 i2c_addr,u8 i2c_reg,u32 val);int goya_test_cpu_queue (struct hl_device *hdev);int goya_send_cpu_message (struct hl_device *hdev,u32 *msg,u16 len,u32 timeout,long *result);@@ -178,6 +182,7 @@ long goya_get_fan_speed(struct hl_device *hdev, int sensor_index, u32 attr);long goya_get_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr);void goya_set_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr,long value); +void goya_debugfs_led_set (struct hl_device *hdev,u8 led,u8 state);void goya_set_pll_profile (struct hl_device *hdev,enum hl_pll_frequency freq);int goya_add_device_attr (struct hl_device *hdev);void goya_remove_device_attr (struct hl_device *hdev);drivers/misc/habanalabs/habanalabs.h +190
-0@@ -225,6 +225,7 @@ struct hl_cb_mgr {* @refcount: reference counter for usage of the CB. * @hdev: pointer to device this CB belongs to. * @lock: spinlock to protect mmap/cs flows. +* @debugfs_list: node in debugfs list of command buffers. * @pool_list: node in pool list of command buffers. * @kernel_address: Holds the CB's kernel virtual address. * @bus_address: Holds the CB's DMA address. @@ -241,6 +242,7 @@ struct hl_cb {struct kref refcount;struct hl_device *hdev;spinlock_t lock; +struct list_head debugfs_list;struct list_head pool_list;u64 kernel_address;dma_addr_t bus_address;@@ -442,6 +444,8 @@ enum hl_pll_frequency {* @update_eq_ci: update event queue CI. * @context_switch: called upon ASID context switch. * @restore_phase_topology: clear all SOBs amd MONs. +* @debugfs_read32: debug interface for reading u32 from DRAM/SRAM. +* @debugfs_write32: debug interface for writing u32 to DRAM/SRAM. * @add_device_attr: add ASIC specific device attributes. * @remove_device_attr: remove ASIC specific device attributes. * @handle_eqe: handle event queue entry (IRQ) from ArmCP. @@ -510,6 +514,8 @@ struct hl_asic_funcs {void (*update_eq_ci)(struct hl_device *hdev,u32 val);int (*context_switch)(struct hl_device *hdev,u32 asid);void (*restore_phase_topology)(struct hl_device *hdev); +int (*debugfs_read32)(struct hl_device *hdev,u64 addr,u32 *val); +int (*debugfs_write32)(struct hl_device *hdev,u64 addr,u32 val);int (*add_device_attr)(struct hl_device *hdev);void (*remove_device_attr)(struct hl_device *hdev); void (*handle_eqe)(struct hl_device *hdev,@@ -572,6 +578,7 @@ struct hl_va_range {* @mem_hash_lock: protects the mem_hash. * @mmu_lock: protects the MMU page tables. Any change to the PGT, modifing the * MMU hash or walking the PGT requires talking this lock +* @debugfs_list: node in debugfs list of contexts. * @cs_sequence: sequence number for CS. Value is assigned to a CS and passed * to user so user could inquire about CS. It is used as * index to cs_pending array. @@ -596,6 +603,7 @@ struct hl_ctx {struct hl_va_range dram_va_range;struct mutex mem_hash_lock;struct mutex mmu_lock; +struct list_head debugfs_list;u64 cs_sequence;spinlock_t cs_lock;atomic64_t dram_phys_mem;@@ -654,6 +662,7 @@ struct hl_userptr {* @fence: pointer to the fence object of this CS. * @work_tdr: delayed work node for TDR. * @mirror_node : node in device mirror list of command submissions. +* @debugfs_list: node in debugfs list of command submissions. * @sequence: the sequence number of this CS. * @submitted: true if CS was submitted to H/W. * @completed: true if CS was completed by device. @@ -671,6 +680,7 @@ struct hl_cs {struct dma_fence *fence;struct delayed_work work_tdr;struct list_head mirror_node; +struct list_head debugfs_list;u64 sequence;u8 submitted;u8 completed;@@ -689,6 +699,7 @@ struct hl_cs {* @finish_work: workqueue object to run when job is completed. * @userptr_list: linked-list of userptr mappings that belong to this job and * wait for completion. +* @debugfs_list: node in debugfs list of command submission jobs. * @id: the id of this job inside a CS. * @hw_queue_id: the id of the H/W queue this job is submitted to. * @user_cb_size: the actual size of the CB we got from the user. @@ -702,6 +713,7 @@ struct hl_cs_job {struct hl_cb *patched_cb;struct work_struct finish_work;struct list_head userptr_list; +struct list_head debugfs_list;u32 id;u32 hw_queue_id;u32 user_cb_size;@@ -832,6 +844,7 @@ struct hl_vm {* @ctx: current executing context. * @ctx_mgr: context manager to handle multiple context for this FD. * @cb_mgr: command buffer manager to handle multiple buffers for this FD. +* @debugfs_list: list of relevant ASIC debugfs. * @refcount: number of related contexts. * @restore_phase_mutex: lock for context switch and restore phase. */ @@ -842,11 +855,90 @@ struct hl_fpriv {struct hl_ctx *ctx;/* TODO: remove for multiple ctx */ struct hl_ctx_mgr ctx_mgr;struct hl_cb_mgr cb_mgr; +struct list_head debugfs_list;struct kref refcount;struct mutex restore_phase_mutex; }; +/* +* DebugFS +*/ + +/** +* struct hl_info_list - debugfs file ops. +* @name: file name. +* @show: function to output information. +* @write: function to write to the file. +*/ +struct hl_info_list { +const char *name; +int (*show)(struct seq_file *s,void *data); +ssize_t (*write)(struct file *file,const char __user *buf, +size_t count,loff_t *f_pos); +}; + +/** +* struct hl_debugfs_entry - debugfs dentry wrapper. +* @dent: base debugfs entry structure. +* @info_ent: dentry realted ops. +* @dev_entry: ASIC specific debugfs manager. +*/ +struct hl_debugfs_entry { +struct dentry *dent; +const struct hl_info_list *info_ent; +struct hl_dbg_device_entry *dev_entry; +}; + +/** +* struct hl_dbg_device_entry - ASIC specific debugfs manager. +* @root: root dentry. +* @hdev: habanalabs device structure. +* @entry_arr: array of available hl_debugfs_entry. +* @file_list: list of available debugfs files. +* @file_mutex: protects file_list. +* @cb_list: list of available CBs. +* @cb_spinlock: protects cb_list. +* @cs_list: list of available CSs. +* @cs_spinlock: protects cs_list. +* @cs_job_list: list of available CB jobs. +* @cs_job_spinlock: protects cs_job_list. +* @userptr_list: list of available userptrs (virtual memory chunk descriptor). +* @userptr_spinlock: protects userptr_list. +* @ctx_mem_hash_list: list of available contexts with MMU mappings. +* @ctx_mem_hash_spinlock: protects cb_list. +* @addr: next address to read/write from/to in read/write32. +* @mmu_addr: next virtual address to translate to physical address in mmu_show. +* @mmu_asid: ASID to use while translating in mmu_show. +* @i2c_bus: generic u8 debugfs file for bus value to use in i2c_data_read. +* @i2c_bus: generic u8 debugfs file for address value to use in i2c_data_read. +* @i2c_bus: generic u8 debugfs file for register value to use in i2c_data_read. +*/ +struct hl_dbg_device_entry { +struct dentry *root; +struct hl_device *hdev; +struct hl_debugfs_entry *entry_arr; +struct list_head file_list; +struct mutex file_mutex; +struct list_head cb_list; +spinlock_t cb_spinlock; +struct list_head cs_list; +spinlock_t cs_spinlock; +struct list_head cs_job_list; +spinlock_t cs_job_spinlock; +struct list_head userptr_list; +spinlock_t userptr_spinlock; +struct list_head ctx_mem_hash_list; +spinlock_t ctx_mem_hash_spinlock; +u64 addr; +u64 mmu_addr; +u32 mmu_asid; +u8 i2c_bus; +u8 i2c_addr; +u8 i2c_reg; +}; + +/* * DEVICES */ @@ -939,6 +1031,7 @@ struct hl_device_reset_work {* @hwmon_dev: H/W monitor device. * @pm_mng_profile: current power management profile. * @hl_chip_info: ASIC's sensors information. +* @hl_debugfs: device's debugfs manager. * @cb_pool: list of preallocated CBs. * @cb_pool_lock: protects the CB pool. * @user_ctx: current user context executing. @@ -1004,6 +1097,8 @@ struct hl_device {enum hl_pm_mng_profile pm_mng_profile;struct hwmon_chip_info hl_chip_info; +struct hl_dbg_device_entry hl_debugfs; +struct list_head cb_pool;spinlock_t cb_pool_lock;@@ -1241,6 +1336,101 @@ void hl_set_pwm_info(struct hl_device *hdev, int sensor_index, u32 attr,u64 hl_get_max_power (struct hl_device *hdev);void hl_set_max_power (struct hl_device *hdev,u64 value); +#ifdef CONFIG_DEBUG_FS + +int hl_debugfs_init (void ); +void hl_debugfs_fini (void ); +void hl_debugfs_add_device (struct hl_device *hdev); +void hl_debugfs_remove_device (struct hl_device *hdev); +void hl_debugfs_add_file (struct hl_fpriv *hpriv); +void hl_debugfs_remove_file (struct hl_fpriv *hpriv); +void hl_debugfs_add_cb (struct hl_cb *cb); +void hl_debugfs_remove_cb (struct hl_cb *cb); +void hl_debugfs_add_cs (struct hl_cs *cs); +void hl_debugfs_remove_cs (struct hl_cs *cs); +void hl_debugfs_add_job (struct hl_device *hdev,struct hl_cs_job *job); +void hl_debugfs_remove_job (struct hl_device *hdev,struct hl_cs_job *job); +void hl_debugfs_add_userptr (struct hl_device *hdev,struct hl_userptr *userptr); +void hl_debugfs_remove_userptr (struct hl_device *hdev, +struct hl_userptr *userptr); +void hl_debugfs_add_ctx_mem_hash (struct hl_device *hdev,struct hl_ctx *ctx); +void hl_debugfs_remove_ctx_mem_hash (struct hl_device *hdev,struct hl_ctx *ctx); + +#else + +static inline int __init hl_debugfs_init (void ) +{ +return 0 ; +} + +static inline void hl_debugfs_fini (void ) +{ +} + +static inline void hl_debugfs_add_device (struct hl_device *hdev) +{ +} + +static inline void hl_debugfs_remove_device (struct hl_device *hdev) +{ +} + +static inline void hl_debugfs_add_file (struct hl_fpriv *hpriv) +{ +} + +static inline void hl_debugfs_remove_file (struct hl_fpriv *hpriv) +{ +} + +static inline void hl_debugfs_add_cb (struct hl_cb *cb) +{ +} + +static inline void hl_debugfs_remove_cb (struct hl_cb *cb) +{ +} + +static inline void hl_debugfs_add_cs (struct hl_cs *cs) +{ +} + +static inline void hl_debugfs_remove_cs (struct hl_cs *cs) +{ +} + +static inline void hl_debugfs_add_job (struct hl_device *hdev, +struct hl_cs_job *job) +{ +} + +static inline void hl_debugfs_remove_job (struct hl_device *hdev, +struct hl_cs_job *job) +{ +} + +static inline void hl_debugfs_add_userptr (struct hl_device *hdev, +struct hl_userptr *userptr) +{ +} + +static inline void hl_debugfs_remove_userptr (struct hl_device *hdev, +struct hl_userptr *userptr) +{ +} + +static inline void hl_debugfs_add_ctx_mem_hash (struct hl_device *hdev, +struct hl_ctx *ctx) +{ +} + +static inline void hl_debugfs_remove_ctx_mem_hash (struct hl_device *hdev, +struct hl_ctx *ctx) +{ +} + +#endif +/* IOCTLs */ long hl_ioctl (struct file *filep,unsigned int cmd,unsigned long arg);int hl_cb_ioctl (struct hl_fpriv *hpriv,void *data);drivers/misc/habanalabs/habanalabs_drv.c +14
-2@@ -150,6 +150,8 @@ int hl_device_open(struct inode *inode, struct file *filp)*/ hl_device_set_frequency (hdev,PLL_HIGH ); +hl_debugfs_add_file (hpriv); +return 0 ; out_err:@@ -417,17 +419,20 @@ static int __init hl_init(void) goto remove_major; } +hl_debugfs_init (); + rc =pci_register_driver (&hl_pci_driver);if (rc) {pr_err ("failed to register pci device\n" ); - goto remove_class; + goto remove_debugfs; }pr_debug ("driver loaded\n" );return 0 ; -remove_class: +remove_debugfs: +hl_debugfs_fini ();class_destroy (hl_class); remove_major:unregister_chrdev_region (MKDEV (hl_major,0 ),HL_MAX_MINORS );@@ -441,6 +446,13 @@ static void __exit hl_exit(void) {pci_unregister_driver (&hl_pci_driver); +/* +* Removing debugfs must be after all devices or simulator devices +* have been removed because otherwise we get a bug in the +* debugfs module for referencing NULL objects +*/ +hl_debugfs_fini (); +class_destroy (hl_class);unregister_chrdev_region (MKDEV (hl_major,0 ),HL_MAX_MINORS );drivers/misc/habanalabs/memory.c +8
-0@@ -1289,6 +1289,8 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u32 size, goto free_sgt; } +hl_debugfs_add_userptr (hdev, userptr); +return 0 ; free_sgt:@@ -1314,6 +1316,8 @@ int hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) {struct page **pages; +hl_debugfs_remove_userptr (hdev, userptr); +if (userptr->dma_mapped) hdev->asic_funcs->hl_dma_unmap_sg(hdev, userptr->sgt->sgl,@@ -1475,6 +1479,8 @@ int hl_vm_ctx_init_with_ranges(struct hl_ctx *ctx, u64 host_range_start, goto dram_vm_err; } +hl_debugfs_add_ctx_mem_hash (hdev, ctx); +return 0 ; dram_vm_err:@@ -1597,6 +1603,8 @@ void hl_vm_ctx_fini(struct hl_ctx *ctx)struct hlist_node *tmp_node;int i; +hl_debugfs_remove_ctx_mem_hash (hdev, ctx); +if (!hash_empty (ctx->mem_hash))dev_notice (hdev->dev,"ctx is freed while it has va in use\n" );Signature · 1 line
2.17.1
Signature · 2 lines
Sincerely yours, Mike.
Greg KH
maintainerRe: [PATCH v3 14/15] habanalabs: add debugfs support
In reply to Oded Gabbaydrivers/misc/habanalabs/debugfs.c line 1066
Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/debugfs.c new file +11
-0+int __init hl_debugfs_init (void ) +{ + hl_debug_root =debugfs_create_dir ("habanalabs" ,NULL ); +if (IS_ERR_OR_NULL (hl_debug_root)) { +pr_err ("can not create debugfs directory\n" ); + hl_debug_root =NULL ; +return -ENOMEM ; + } + +return 0 ; +}
This function should just be one line:
void __init hl_debugfs_init(void)
{
hl_debug_root = debugfs_create_dir("habanalabs", NULL);
}No need to do anything else, or check anything.
thanks,
greg k-h
Re: [PATCH v3 14/15] habanalabs: add debugfs support
Quoted 16 lines · Greg KH
drivers/misc/habanalabs/debugfs.c line 1066
Quoted 11 lines · Oded Gabbay
drivers/misc/habanalabs/debugfs.c new file +11
-0+int __init hl_debugfs_init (void ) +{ + hl_debug_root =debugfs_create_dir ("habanalabs" ,NULL ); +if (IS_ERR_OR_NULL (hl_debug_root)) { +pr_err ("can not create debugfs directory\n" ); + hl_debug_root =NULL ; +return -ENOMEM ; + } + +return 0 ; +}This function should just be one line:
void__inithl_debugfs_init(void)
Quoted 6 lines
{ hl_debug_root = debugfs_create_dir("habanalabs", NULL); }No need to do anything else, or check anything.
Correct, fixed.
Quoted 3 lines
thanks,
greg k-h
PATCH v3 15/15 Update MAINTAINERS and CREDITS with habanalabs info
In reply to Oded GabbayThe habanalabs driver was written from scratch from the very first days of Habana and is maintained by Oded Gabbay.
--- CREDITS | 2 +- MAINTAINERS | 9 +++++++++ 2 files changed, 10 insertions(+), 1 deletion(-)
CREDITS +1 -1
@@ -1222,7 +1222,7 @@ S: Brazil N: Oded Gabbay E: [email protected] -D: AMD KFD maintainer +D: HabanaLabs and AMD KFD maintainer S: 12 Shraga Raphaeli S: Petah-Tikva, 4906418 S: Israel
MAINTAINERS +9 -0
@@ -6642,6 +6642,15 @@ F: drivers/clocksource/h8300_*.c F: drivers/clk/h8300/ F: drivers/irqchip/irq-renesas-h8*.c +HABANALABS PCI DRIVER +M: Oded Gabbay <[email protected]> +T: git https://github.com/HabanaAI/linux.git +S: Supported +F: drivers/misc/habanalabs/ +F: include/uapi/misc/habanalabs.h +F: Documentation/ABI/testing/sysfs-driver-habanalabs +F: Documentation/ABI/testing/debugfs-driver-habanalabs + HACKRF MEDIA DRIVER M: Antti Palosaari <[email protected]> L: [email protected]
Signature · 1 line
2.17.1
Quoted 34 lines · Oded Gabbay
Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.Link to v2 cover letter: https://lkml.org/lkml/2019/1/30/1003
Link to v1 cover letter: https://lwn.net/Articles/777342/
I would appricate any feedback, question and/or review.
Thanks,
Odedp.s. for those who prefer to clone the tree instead of looking at the emails, you can grab a copy from our company's page in GitHub:
https://github.com/HabanaAI/linux/releases/tag/hl_patchset_v3_20190204
Oded Gabbay (14):
habanalabs: add skeleton driver
habanalabs: add Goya registers header files
habanalabs: add basic Goya support
habanalabs: add context and ASID modules
habanalabs: add command buffer module
habanalabs: add basic Goya h/w initialization
habanalabs: add h/w queues module
habanalabs: add event queue and interrupts
habanalabs: add sysfs and hwmon support
habanalabs: add device reset support
habanalabs: add command submission module
habanalabs: implement INFO IOCTL
habanalabs: add debugfs support
UpdateMAINTAINERSand CREDITS with habanalabs info
Quoted 2 lines
Omer Shpigelman (1):
habanalabs: add virtual memory and MMU modules
Nit: git am is unhappy about black lines at EOF, e.g:
Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +
Aside from that, for patches 1, 3-11 and 13 feel free to add
As for the "habanalabs: add virtual memory and MMU modules" and the debugfs patch I'll need more time to review the changes related to 2M pages.
Quoted 231 lines
CREDITS | 2 +- .../ABI/testing/debugfs-driver-habanalabs | 127 + .../ABI/testing/sysfs-driver-habanalabs | 190 + MAINTAINERS | 9 + drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/habanalabs/Kconfig | 22 + drivers/misc/habanalabs/Makefile | 15 + drivers/misc/habanalabs/asid.c | 58 + drivers/misc/habanalabs/command_buffer.c | 432 ++ drivers/misc/habanalabs/command_submission.c | 782 +++ drivers/misc/habanalabs/context.c | 216 + drivers/misc/habanalabs/debugfs.c | 1071 ++++ drivers/misc/habanalabs/device.c | 1111 ++++ drivers/misc/habanalabs/goya/Makefile | 4 + drivers/misc/habanalabs/goya/goya.c | 5344 +++++++++++++++++ drivers/misc/habanalabs/goya/goyaP.h | 193 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 306 + drivers/misc/habanalabs/goya/goya_security.c | 2999 +++++++++ drivers/misc/habanalabs/habanalabs.h | 1441 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 465 ++ drivers/misc/habanalabs/habanalabs_ioctl.c | 234 + drivers/misc/habanalabs/hw_queue.c | 636 ++ drivers/misc/habanalabs/hwmon.c | 449 ++ drivers/misc/habanalabs/include/armcp_if.h | 335 ++ .../include/goya/asic_reg/cpu_ca53_cfg_regs.h | 213 + .../include/goya/asic_reg/cpu_if_regs.h | 110 + .../include/goya/asic_reg/cpu_pll_regs.h | 186 + .../include/goya/asic_reg/dma_ch_0_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_1_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_2_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_3_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_4_regs.h | 512 ++ .../include/goya/asic_reg/dma_macro_regs.h | 242 + .../include/goya/asic_reg/dma_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/dma_qm_0_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_1_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_2_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_3_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_4_regs.h | 543 ++ .../include/goya/asic_reg/goya_blocks.h | 1372 +++++ .../include/goya/asic_reg/goya_masks.h | 262 + .../include/goya/asic_reg/goya_regs.h | 99 + .../include/goya/asic_reg/ic_pll_regs.h | 186 + .../include/goya/asic_reg/mc_pll_regs.h | 186 + .../include/goya/asic_reg/mme1_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme2_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme3_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme4_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme5_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme6_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/mme_qm_regs.h | 543 ++ .../include/goya/asic_reg/mme_regs.h | 2422 ++++++++ .../include/goya/asic_reg/mmu_regs.h | 158 + .../include/goya/asic_reg/pci_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/pcie_aux_regs.h | 476 ++ .../goya/asic_reg/psoc_emmc_pll_regs.h | 186 + .../goya/asic_reg/psoc_global_conf_regs.h | 1119 ++++ .../include/goya/asic_reg/psoc_mme_pll_regs.h | 186 + .../include/goya/asic_reg/psoc_pci_pll_regs.h | 186 + .../include/goya/asic_reg/psoc_spi_regs.h | 427 ++ .../goya/asic_reg/sram_y0_x0_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x1_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x2_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x3_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x4_rtr_regs.h | 215 + .../include/goya/asic_reg/stlb_regs.h | 133 + .../include/goya/asic_reg/tpc0_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc0_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc0_eml_cfg_regs.h | 580 ++ .../include/goya/asic_reg/tpc0_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/tpc0_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc1_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc1_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc1_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc1_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc2_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc2_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc2_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc2_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc3_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc3_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc3_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc3_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc4_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc4_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc4_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc4_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc5_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc5_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc5_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc5_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc6_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc6_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc6_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc6_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc7_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc7_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc7_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/tpc7_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc_pll_regs.h | 186 + drivers/misc/habanalabs/include/goya/goya.h | 41 + .../include/goya/goya_async_events.h | 186 + .../misc/habanalabs/include/goya/goya_fw_if.h | 28 + .../habanalabs/include/goya/goya_packets.h | 129 + drivers/misc/habanalabs/include/hl_boot_if.h | 30 + .../include/hw_ip/mmu/mmu_general.h | 45 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/include/qman_if.h | 56 + drivers/misc/habanalabs/irq.c | 325 + drivers/misc/habanalabs/memory.c | 1722 ++++++ drivers/misc/habanalabs/mmu.c | 604 ++ drivers/misc/habanalabs/sysfs.c | 691 +++ include/uapi/misc/habanalabs.h | 444 ++ 115 files changed, 73533 insertions(+), 1 deletion(-) create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/Kconfig create mode 100644 drivers/misc/habanalabs/Makefile create mode 100644 drivers/misc/habanalabs/asid.c create mode 100644 drivers/misc/habanalabs/command_buffer.c create mode 100644 drivers/misc/habanalabs/command_submission.c create mode 100644 drivers/misc/habanalabs/context.c create mode 100644 drivers/misc/habanalabs/debugfs.c create mode 100644 drivers/misc/habanalabs/device.c create mode 100644 drivers/misc/habanalabs/goya/Makefile create mode 100644 drivers/misc/habanalabs/goya/goya.c create mode 100644 drivers/misc/habanalabs/goya/goyaP.h create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/goya/goya_security.c create mode 100644 drivers/misc/habanalabs/habanalabs.h create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c create mode 100644 drivers/misc/habanalabs/hw_queue.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/include/armcp_if.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/goya.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/include/qman_if.h create mode 100644 drivers/misc/habanalabs/irq.c create mode 100644 drivers/misc/habanalabs/memory.c create mode 100644 drivers/misc/habanalabs/mmu.c create mode 100644 drivers/misc/habanalabs/sysfs.c create mode 100644 include/uapi/misc/habanalabs.hSignature · 1 line
2.17.1
Signature · 2 lines
Sincerely yours, Mike.
Quoted 36 lines · Mike Rapoport
Quoted 34 lines · Oded Gabbay
Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.Link to v2 cover letter: https://lkml.org/lkml/2019/1/30/1003
Link to v1 cover letter: https://lwn.net/Articles/777342/
I would appricate any feedback, question and/or review.
Thanks,
Odedp.s. for those who prefer to clone the tree instead of looking at the emails, you can grab a copy from our company's page in GitHub:
https://github.com/HabanaAI/linux/releases/tag/hl_patchset_v3_20190204
Oded Gabbay (14):
habanalabs: add skeleton driver
habanalabs: add Goya registers header files
habanalabs: add basic Goya support
habanalabs: add context and ASID modules
habanalabs: add command buffer module
habanalabs: add basic Goya h/w initialization
habanalabs: add h/w queues module
habanalabs: add event queue and interrupts
habanalabs: add sysfs and hwmon support
habanalabs: add device reset support
habanalabs: add command submission module
habanalabs: implement INFO IOCTL
habanalabs: add debugfs support
UpdateMAINTAINERSand CREDITS with habanalabs info
Quoted 8 lines
Quoted 2 lines
Omer Shpigelman (1):
habanalabs: add virtual memory and MMU modulesNit: git am is unhappy about black lines at EOF, e.g:
Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +
Yes, but on the other hand, joe perches gave me a comment about missing newline at end of Makefile.
So either diff will complain or git am.
Quoted 5 lines
Aside from that, for patches 1, 3-11 and 13 feel free to add
Reviewed-by Mike Rapoport <[email protected]>
Thanks for the review!
Oded
Quoted 239 lines
As for the "habanalabs: add virtual memory and MMU modules" and the debugfs patch I'll need more time to review the changes related to 2M pages.
Quoted 228 lines
CREDITS | 2 +- .../ABI/testing/debugfs-driver-habanalabs | 127 + .../ABI/testing/sysfs-driver-habanalabs | 190 + MAINTAINERS | 9 + drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/habanalabs/Kconfig | 22 + drivers/misc/habanalabs/Makefile | 15 + drivers/misc/habanalabs/asid.c | 58 + drivers/misc/habanalabs/command_buffer.c | 432 ++ drivers/misc/habanalabs/command_submission.c | 782 +++ drivers/misc/habanalabs/context.c | 216 + drivers/misc/habanalabs/debugfs.c | 1071 ++++ drivers/misc/habanalabs/device.c | 1111 ++++ drivers/misc/habanalabs/goya/Makefile | 4 + drivers/misc/habanalabs/goya/goya.c | 5344 +++++++++++++++++ drivers/misc/habanalabs/goya/goyaP.h | 193 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 306 + drivers/misc/habanalabs/goya/goya_security.c | 2999 +++++++++ drivers/misc/habanalabs/habanalabs.h | 1441 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 465 ++ drivers/misc/habanalabs/habanalabs_ioctl.c | 234 + drivers/misc/habanalabs/hw_queue.c | 636 ++ drivers/misc/habanalabs/hwmon.c | 449 ++ drivers/misc/habanalabs/include/armcp_if.h | 335 ++ .../include/goya/asic_reg/cpu_ca53_cfg_regs.h | 213 + .../include/goya/asic_reg/cpu_if_regs.h | 110 + .../include/goya/asic_reg/cpu_pll_regs.h | 186 + .../include/goya/asic_reg/dma_ch_0_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_1_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_2_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_3_regs.h | 512 ++ .../include/goya/asic_reg/dma_ch_4_regs.h | 512 ++ .../include/goya/asic_reg/dma_macro_regs.h | 242 + .../include/goya/asic_reg/dma_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/dma_qm_0_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_1_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_2_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_3_regs.h | 543 ++ .../include/goya/asic_reg/dma_qm_4_regs.h | 543 ++ .../include/goya/asic_reg/goya_blocks.h | 1372 +++++ .../include/goya/asic_reg/goya_masks.h | 262 + .../include/goya/asic_reg/goya_regs.h | 99 + .../include/goya/asic_reg/ic_pll_regs.h | 186 + .../include/goya/asic_reg/mc_pll_regs.h | 186 + .../include/goya/asic_reg/mme1_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme2_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme3_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme4_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme5_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme6_rtr_regs.h | 876 +++ .../include/goya/asic_reg/mme_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/mme_qm_regs.h | 543 ++ .../include/goya/asic_reg/mme_regs.h | 2422 ++++++++ .../include/goya/asic_reg/mmu_regs.h | 158 + .../include/goya/asic_reg/pci_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/pcie_aux_regs.h | 476 ++ .../goya/asic_reg/psoc_emmc_pll_regs.h | 186 + .../goya/asic_reg/psoc_global_conf_regs.h | 1119 ++++ .../include/goya/asic_reg/psoc_mme_pll_regs.h | 186 + .../include/goya/asic_reg/psoc_pci_pll_regs.h | 186 + .../include/goya/asic_reg/psoc_spi_regs.h | 427 ++ .../goya/asic_reg/sram_y0_x0_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x1_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x2_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x3_rtr_regs.h | 215 + .../goya/asic_reg/sram_y0_x4_rtr_regs.h | 215 + .../include/goya/asic_reg/stlb_regs.h | 133 + .../include/goya/asic_reg/tpc0_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc0_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc0_eml_cfg_regs.h | 580 ++ .../include/goya/asic_reg/tpc0_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/tpc0_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc1_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc1_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc1_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc1_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc2_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc2_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc2_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc2_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc3_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc3_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc3_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc3_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc4_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc4_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc4_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc4_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc5_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc5_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc5_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc5_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc6_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc6_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc6_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc6_rtr_regs.h | 848 +++ .../include/goya/asic_reg/tpc7_cfg_regs.h | 2110 +++++++ .../include/goya/asic_reg/tpc7_cmdq_regs.h | 431 ++ .../include/goya/asic_reg/tpc7_nrtr_regs.h | 380 ++ .../include/goya/asic_reg/tpc7_qm_regs.h | 543 ++ .../include/goya/asic_reg/tpc_pll_regs.h | 186 + drivers/misc/habanalabs/include/goya/goya.h | 41 + .../include/goya/goya_async_events.h | 186 + .../misc/habanalabs/include/goya/goya_fw_if.h | 28 + .../habanalabs/include/goya/goya_packets.h | 129 + drivers/misc/habanalabs/include/hl_boot_if.h | 30 + .../include/hw_ip/mmu/mmu_general.h | 45 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/include/qman_if.h | 56 + drivers/misc/habanalabs/irq.c | 325 + drivers/misc/habanalabs/memory.c | 1722 ++++++ drivers/misc/habanalabs/mmu.c | 604 ++ drivers/misc/habanalabs/sysfs.c | 691 +++ include/uapi/misc/habanalabs.h | 444 ++ 115 files changed, 73533 insertions(+), 1 deletion(-) create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/Kconfig create mode 100644 drivers/misc/habanalabs/Makefile create mode 100644 drivers/misc/habanalabs/asid.c create mode 100644 drivers/misc/habanalabs/command_buffer.c create mode 100644 drivers/misc/habanalabs/command_submission.c create mode 100644 drivers/misc/habanalabs/context.c create mode 100644 drivers/misc/habanalabs/debugfs.c create mode 100644 drivers/misc/habanalabs/device.c create mode 100644 drivers/misc/habanalabs/goya/Makefile create mode 100644 drivers/misc/habanalabs/goya/goya.c create mode 100644 drivers/misc/habanalabs/goya/goyaP.h create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/goya/goya_security.c create mode 100644 drivers/misc/habanalabs/habanalabs.h create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c create mode 100644 drivers/misc/habanalabs/hw_queue.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/include/armcp_if.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/goya.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/include/qman_if.h create mode 100644 drivers/misc/habanalabs/irq.c create mode 100644 drivers/misc/habanalabs/memory.c create mode 100644 drivers/misc/habanalabs/mmu.c create mode 100644 drivers/misc/habanalabs/sysfs.c create mode 100644 include/uapi/misc/habanalabs.hQuoted 3 lines
--
2.17.1--
Sincerely yours,
Mike.
Quoted 5 lines · Oded Gabbay
Quoted 4 lines · Mike Rapoport
Quoted 3 lines · Oded Gabbay
Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.
[]
Quoted 8 lines
Quoted 5 lines
Nit: git am is unhappy about black lines at EOF, e.g:
Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +
Yes, but on the other hand, joe perches gave me a comment about missing newline at end of
Makefile.
So either diff will complain or git am.
Not really.
The desired modification was the last non-blank line of the Makefile should have a terminating newline, not that there was a blank line at EOF.
Quoted 17 lines · Joe Perches
Quoted 5 lines · Oded Gabbay
Quoted 4 lines · Mike Rapoport
Quoted 3 lines · Oded Gabbay
Hello,
This is v3 of the Habana Labs kernel driver patch-set. It contains minor fixes according to reviews done on v2. In addition, it is rebased on v5.0-rc5.[]
Quoted 8 lines
Quoted 5 lines
Nit: git am is unhappy about black lines at EOF, e.g:
Applying: habanalabs: add basic Goya support .git/rebase-apply/patch:29: new blank line at EOF. +
Yes, but on the other hand, joe perches gave me a comment about missing newline at end of
Makefile.
So either diff will complain or git am.
Quoted 7 lines
Not really.
The desired modification was the last non-blank line of the
Makefileshould have a terminating newline, not that there was a blank line at EOF.
ok, thanks for the clarification, I'll fix that.
Oded