Habana Labs kernel driver
Subsystems (2)
v6 cover Habana Labs kernel driver
Hello,
This is v6 of the Habana Labs kernel driver patch-set. This version fixes the compilation failure on arm 32-bit and removes many includes of headers which are not really needed by the code.
Link to v5 cover letter: https://lkml.org/lkml/2019/2/14/741
Link to v4 cover letter: https://lkml.org/lkml/2019/2/11/993
Link to v3 cover letter: https://lkml.org/lkml/2019/2/4/1033
Link to v2 cover letter: https://lkml.org/lkml/2019/1/30/1003
Link to v1 cover letter: https://lwn.net/Articles/777342/
I would appricate any feedback, question and/or review.
Thanks,
Oded
p.s. for those who prefer to clone the tree instead of looking at the emails, you can grab a copy from our company's page in GitHub:
https://github.com/HabanaAI/linux/releases/tag/hl_patchset_v6_20190216
Oded Gabbay (14):
habanalabs: add skeleton driver
habanalabs: add Goya registers header files
habanalabs: add basic Goya support
habanalabs: add context and ASID modules
habanalabs: add command buffer module
habanalabs: add basic Goya h/w initialization
habanalabs: add h/w queues module
habanalabs: add event queue and interrupts
habanalabs: add sysfs and hwmon support
habanalabs: add device reset support
habanalabs: add command submission module
habanalabs: implement INFO IOCTL
habanalabs: add debugfs support
Update MAINTAINERS and CREDITS with habanalabs info
Omer Shpigelman (1):
habanalabs: add virtual memory and MMU modules
CREDITS | 2 +- .../ABI/testing/debugfs-driver-habanalabs | 126 + .../ABI/testing/sysfs-driver-habanalabs | 190 + MAINTAINERS | 9 + drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/habanalabs/Kconfig | 22 + drivers/misc/habanalabs/Makefile | 14 + drivers/misc/habanalabs/asid.c | 57 + drivers/misc/habanalabs/command_buffer.c | 442 ++ drivers/misc/habanalabs/command_submission.c | 778 +++ drivers/misc/habanalabs/context.c | 215 + drivers/misc/habanalabs/debugfs.c | 1072 ++++ drivers/misc/habanalabs/device.c | 1138 ++++ drivers/misc/habanalabs/goya/Makefile | 3 + drivers/misc/habanalabs/goya/goya.c | 5363 +++++++++++++++++ drivers/misc/habanalabs/goya/goyaP.h | 190 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 254 + drivers/misc/habanalabs/goya/goya_security.c | 2999 +++++++++ drivers/misc/habanalabs/habanalabs.h | 1454 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 461 ++ drivers/misc/habanalabs/habanalabs_ioctl.c | 234 + drivers/misc/habanalabs/hw_queue.c | 632 ++ drivers/misc/habanalabs/hwmon.c | 452 ++ drivers/misc/habanalabs/include/armcp_if.h | 335 + .../goya/asic_reg/cpu_ca53_cfg_masks.h | 191 + .../include/goya/asic_reg/cpu_ca53_cfg_regs.h | 61 + .../include/goya/asic_reg/cpu_if_regs.h | 49 + .../include/goya/asic_reg/cpu_pll_regs.h | 105 + .../include/goya/asic_reg/dma_ch_0_regs.h | 209 + .../include/goya/asic_reg/dma_ch_1_regs.h | 209 + .../include/goya/asic_reg/dma_ch_2_regs.h | 209 + .../include/goya/asic_reg/dma_ch_3_regs.h | 209 + .../include/goya/asic_reg/dma_ch_4_regs.h | 209 + .../include/goya/asic_reg/dma_macro_masks.h | 105 + .../include/goya/asic_reg/dma_macro_regs.h | 181 + .../include/goya/asic_reg/dma_nrtr_masks.h | 209 + .../include/goya/asic_reg/dma_nrtr_regs.h | 227 + .../include/goya/asic_reg/dma_qm_0_masks.h | 465 ++ .../include/goya/asic_reg/dma_qm_0_regs.h | 179 + .../include/goya/asic_reg/dma_qm_1_regs.h | 179 + .../include/goya/asic_reg/dma_qm_2_regs.h | 179 + .../include/goya/asic_reg/dma_qm_3_regs.h | 179 + .../include/goya/asic_reg/dma_qm_4_regs.h | 179 + .../include/goya/asic_reg/goya_blocks.h | 1372 +++++ .../include/goya/asic_reg/goya_masks.h | 275 + .../include/goya/asic_reg/goya_regs.h | 117 + .../include/goya/asic_reg/ic_pll_regs.h | 105 + .../include/goya/asic_reg/mc_pll_regs.h | 105 + .../include/goya/asic_reg/mme1_rtr_masks.h | 653 ++ .../include/goya/asic_reg/mme1_rtr_regs.h | 331 + .../include/goya/asic_reg/mme2_rtr_regs.h | 331 + .../include/goya/asic_reg/mme3_rtr_regs.h | 331 + .../include/goya/asic_reg/mme4_rtr_regs.h | 331 + .../include/goya/asic_reg/mme5_rtr_regs.h | 331 + .../include/goya/asic_reg/mme6_rtr_regs.h | 331 + .../include/goya/asic_reg/mme_cmdq_masks.h | 373 ++ .../include/goya/asic_reg/mme_cmdq_regs.h | 139 + .../include/goya/asic_reg/mme_masks.h | 1537 +++++ .../include/goya/asic_reg/mme_qm_masks.h | 465 ++ .../include/goya/asic_reg/mme_qm_regs.h | 179 + .../include/goya/asic_reg/mme_regs.h | 1153 ++++ .../include/goya/asic_reg/mmu_masks.h | 143 + .../include/goya/asic_reg/mmu_regs.h | 53 + .../include/goya/asic_reg/pci_nrtr_masks.h | 209 + .../include/goya/asic_reg/pci_nrtr_regs.h | 227 + .../include/goya/asic_reg/pcie_aux_regs.h | 243 + .../goya/asic_reg/psoc_emmc_pll_regs.h | 105 + .../goya/asic_reg/psoc_global_conf_masks.h | 447 ++ .../goya/asic_reg/psoc_global_conf_regs.h | 745 +++ .../include/goya/asic_reg/psoc_mme_pll_regs.h | 105 + .../include/goya/asic_reg/psoc_pci_pll_regs.h | 105 + .../include/goya/asic_reg/psoc_spi_regs.h | 143 + .../goya/asic_reg/sram_y0_x0_rtr_regs.h | 83 + .../goya/asic_reg/sram_y0_x1_rtr_regs.h | 83 + .../goya/asic_reg/sram_y0_x2_rtr_regs.h | 83 + .../goya/asic_reg/sram_y0_x3_rtr_regs.h | 83 + .../goya/asic_reg/sram_y0_x4_rtr_regs.h | 83 + .../include/goya/asic_reg/stlb_masks.h | 117 + .../include/goya/asic_reg/stlb_regs.h | 55 + .../include/goya/asic_reg/tpc0_cfg_masks.h | 1607 +++++ .../include/goya/asic_reg/tpc0_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc0_cmdq_masks.h | 373 ++ .../include/goya/asic_reg/tpc0_cmdq_regs.h | 139 + .../goya/asic_reg/tpc0_eml_cfg_masks.h | 347 ++ .../include/goya/asic_reg/tpc0_eml_cfg_regs.h | 313 + .../include/goya/asic_reg/tpc0_nrtr_masks.h | 209 + .../include/goya/asic_reg/tpc0_nrtr_regs.h | 227 + .../include/goya/asic_reg/tpc0_qm_masks.h | 465 ++ .../include/goya/asic_reg/tpc0_qm_regs.h | 179 + .../include/goya/asic_reg/tpc1_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc1_cmdq_regs.h | 139 + .../include/goya/asic_reg/tpc1_qm_regs.h | 179 + .../include/goya/asic_reg/tpc1_rtr_regs.h | 323 + .../include/goya/asic_reg/tpc2_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc2_cmdq_regs.h | 139 + .../include/goya/asic_reg/tpc2_qm_regs.h | 179 + .../include/goya/asic_reg/tpc2_rtr_regs.h | 323 + .../include/goya/asic_reg/tpc3_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc3_cmdq_regs.h | 139 + .../include/goya/asic_reg/tpc3_qm_regs.h | 179 + .../include/goya/asic_reg/tpc3_rtr_regs.h | 323 + .../include/goya/asic_reg/tpc4_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc4_cmdq_regs.h | 139 + .../include/goya/asic_reg/tpc4_qm_regs.h | 179 + .../include/goya/asic_reg/tpc4_rtr_regs.h | 323 + .../include/goya/asic_reg/tpc5_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc5_cmdq_regs.h | 139 + .../include/goya/asic_reg/tpc5_qm_regs.h | 179 + .../include/goya/asic_reg/tpc5_rtr_regs.h | 323 + .../include/goya/asic_reg/tpc6_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc6_cmdq_regs.h | 139 + .../include/goya/asic_reg/tpc6_qm_regs.h | 179 + .../include/goya/asic_reg/tpc6_rtr_regs.h | 323 + .../include/goya/asic_reg/tpc7_cfg_regs.h | 887 +++ .../include/goya/asic_reg/tpc7_cmdq_regs.h | 139 + .../include/goya/asic_reg/tpc7_nrtr_regs.h | 227 + .../include/goya/asic_reg/tpc7_qm_regs.h | 179 + .../include/goya/asic_reg/tpc_pll_regs.h | 105 + drivers/misc/habanalabs/include/goya/goya.h | 45 + .../include/goya/goya_async_events.h | 186 + .../misc/habanalabs/include/goya/goya_fw_if.h | 28 + .../habanalabs/include/goya/goya_packets.h | 129 + drivers/misc/habanalabs/include/hl_boot_if.h | 30 + .../include/hw_ip/mmu/mmu_general.h | 46 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/include/qman_if.h | 56 + drivers/misc/habanalabs/irq.c | 325 + drivers/misc/habanalabs/memory.c | 1723 ++++++ drivers/misc/habanalabs/mmu.c | 691 +++ drivers/misc/habanalabs/sysfs.c | 536 ++ include/uapi/misc/habanalabs.h | 444 ++ 132 files changed, 51252 insertions(+), 1 deletion(-) create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/Kconfig create mode 100644 drivers/misc/habanalabs/Makefile create mode 100644 drivers/misc/habanalabs/asid.c create mode 100644 drivers/misc/habanalabs/command_buffer.c create mode 100644 drivers/misc/habanalabs/command_submission.c create mode 100644 drivers/misc/habanalabs/context.c create mode 100644 drivers/misc/habanalabs/debugfs.c create mode 100644 drivers/misc/habanalabs/device.c create mode 100644 drivers/misc/habanalabs/goya/Makefile create mode 100644 drivers/misc/habanalabs/goya/goya.c create mode 100644 drivers/misc/habanalabs/goya/goyaP.h create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/goya/goya_security.c create mode 100644 drivers/misc/habanalabs/habanalabs.h create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c create mode 100644 drivers/misc/habanalabs/hw_queue.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/include/armcp_if.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_ca53_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_if_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/cpu_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_ch_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_macro_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_0_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_1_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_2_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_3_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/dma_qm_4_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_blocks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/goya_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/ic_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mme_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/mmu_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pci_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/pcie_aux_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_emmc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_global_conf_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_mme_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_pci_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/psoc_spi_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x0_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/sram_y0_x4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/stlb_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_eml_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_masks.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc0_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc1_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc2_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc3_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc4_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc5_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc6_rtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cfg_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_cmdq_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_nrtr_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc7_qm_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/asic_reg/tpc_pll_regs.h create mode 100644 drivers/misc/habanalabs/include/goya/goya.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/include/qman_if.h create mode 100644 drivers/misc/habanalabs/irq.c create mode 100644 drivers/misc/habanalabs/memory.c create mode 100644 drivers/misc/habanalabs/mmu.c create mode 100644 drivers/misc/habanalabs/sysfs.c create mode 100644 include/uapi/misc/habanalabs.h
Signature · 1 line
2.18.0
PATCH v6 1/15 habanalabs: add skeleton driver
This patch adds the habanalabs skeleton driver. The driver does nothing at this stage except very basic operations. It contains the minimal code to insmod and rmmod the driver and to create a /dev/hlX file per PCI device.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
drivers/misc/Kconfig | 1 + drivers/misc/Makefile | 1 + drivers/misc/habanalabs/Kconfig | 22 ++ drivers/misc/habanalabs/Makefile | 7 + drivers/misc/habanalabs/device.c | 340 +++++++++++++++++++++ drivers/misc/habanalabs/habanalabs.h | 131 +++++++++ drivers/misc/habanalabs/habanalabs_drv.c | 358 +++++++++++++++++++++++ 7 files changed, 860 insertions(+) create mode 100644 drivers/misc/habanalabs/Kconfig create mode 100644 drivers/misc/habanalabs/Makefile create mode 100644 drivers/misc/habanalabs/device.c create mode 100644 drivers/misc/habanalabs/habanalabs.h create mode 100644 drivers/misc/habanalabs/habanalabs_drv.c
drivers/misc/Kconfig +1 -0
@@ -535,4 +535,5 @@ source "drivers/misc/echo/Kconfig" source "drivers/misc/cxl/Kconfig" source "drivers/misc/ocxl/Kconfig" source "drivers/misc/cardreader/Kconfig" +source "drivers/misc/habanalabs/Kconfig" endmenu
drivers/misc/Makefile +1 -0
@@ -59,3 +59,4 @@ obj-$(CONFIG_PCI_ENDPOINT_TEST) += pci_endpoint_test.o obj-$(CONFIG_OCXL) += ocxl/ obj-y += cardreader/ obj-$(CONFIG_PVPANIC) += pvpanic.o +obj-$(CONFIG_HABANA_AI) += habanalabs/
drivers/misc/habanalabs/Kconfig new file +22 -0
@@ -0,0 +1,22 @@ +# +# HabanaLabs AI accelerators driver +# + +config HABANA_AI + tristate "HabanaAI accelerators (habanalabs)" + depends on PCI + select FRAME_VECTOR + help + Enables PCIe card driver for Habana's AI Processors (AIP) that are + designed to accelerate Deep Learning inference and training workloads. + + The driver manages the PCIe devices and provides IOCTL interface for + the user to submit workloads to the devices. + + The user-space interface is described in + include/uapi/misc/habanalabs.h + + If unsure, say N. + + To compile this driver as a module, choose M here: the + module will be called habanalabs.
drivers/misc/habanalabs/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +# +# Makefile for HabanaLabs AI accelerators driver +# + +obj-m := habanalabs.o + +habanalabs-y := habanalabs_drv.o device.o
drivers/misc/habanalabs/device.c new file +340 -0
340 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h new file +131 -0
131 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c new file +358 -0
358 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v6 3/15 habanalabs: add basic Goya support
In reply to Oded GabbayThis patch adds a basic support for the Goya device. The code initializes the device's PCI controller and PCI bars. It also initializes various S/W structures and adds some basic helper functions.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
drivers/misc/habanalabs/Makefile | 3 + drivers/misc/habanalabs/device.c | 71 +++ drivers/misc/habanalabs/goya/Makefile | 3 + drivers/misc/habanalabs/goya/goya.c | 632 ++++++++++++++++++++ drivers/misc/habanalabs/goya/goyaP.h | 152 +++++ drivers/misc/habanalabs/habanalabs.h | 137 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 3 + drivers/misc/habanalabs/include/goya/goya.h | 45 ++ include/uapi/misc/habanalabs.h | 20 + 9 files changed, 1066 insertions(+) create mode 100644 drivers/misc/habanalabs/goya/Makefile create mode 100644 drivers/misc/habanalabs/goya/goya.c create mode 100644 drivers/misc/habanalabs/goya/goyaP.h create mode 100644 drivers/misc/habanalabs/include/goya/goya.h create mode 100644 include/uapi/misc/habanalabs.h
drivers/misc/habanalabs/Makefile +3 -0
@@ -5,3 +5,6 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o + +include $(src)/goya/Makefile +habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/device.c +71 -0
125 lines not on this page · raw message
drivers/misc/habanalabs/goya/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +subdir-ccflags-y += -I$(src) + +HL_GOYA_FILES := goya/goya.o
drivers/misc/habanalabs/goya/goya.c new file +632 -0
632 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h new file +152 -0
152 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +137 -0
182 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +3 -0
@@ -122,6 +122,9 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev, hdev->major = hl_major; +/* Parameters for bring-up - set them to defaults */ + hdev->reset_pcilink =0 ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/include/goya/goya.h new file +45 -0
45 lines not on this page · raw message
include/uapi/misc/habanalabs.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note +* +* Copyright 2016-2018 HabanaLabs, Ltd. +* All Rights Reserved. +* +*/ + +#ifndef HABANALABS_H_ +#define HABANALABS_H_ + +#include <linux/types.h> +#include <linux/ioctl.h> + +/* +* Defines that are asic-specific but constitutes as ABI between kernel driver +* and userspace +*/ +#define GOYA_KMD_SRAM_RESERVED_SIZE_FROM_START 0x8000/* 32KB */ + +#endif /* HABANALABS_H_ */
Signature · 1 line
2.18.0
PATCH v6 4/15 habanalabs: add context and ASID modules
In reply to Oded GabbayThis patch adds two modules - ASID and context.
Each user process that opens a device's file must have at least one context before it is able to "work" with the device. Each context has its own device address-space and contains information about its runtime state (its active command submissions).
To have address-space separation between contexts, each context is assigned a unique ASID, which stands for "address-space id". Goya supports up to 1024 ASIDs.
Currently, the driver doesn't support multiple contexts. Therefore, the user doesn't need to actively create a context. A "primary context" is created automatically when the user opens the device's file.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/asid.c | 57 +++++++++ drivers/misc/habanalabs/context.c | 154 +++++++++++++++++++++++ drivers/misc/habanalabs/device.c | 47 +++++++ drivers/misc/habanalabs/habanalabs.h | 71 +++++++++++ drivers/misc/habanalabs/habanalabs_drv.c | 46 ++++++- 6 files changed, 374 insertions(+), 3 deletions(-) create mode 100644 drivers/misc/habanalabs/asid.c create mode 100644 drivers/misc/habanalabs/context.c
drivers/misc/habanalabs/Makefile +1 -1
@@ -4,7 +4,7 @@ obj-m := habanalabs.o -habanalabs-y := habanalabs_drv.o device.o +habanalabs-y := habanalabs_drv.o device.o context.o asid.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/asid.c new file +57 -0
57 lines not on this page · raw message
drivers/misc/habanalabs/context.c new file +154 -0
154 lines not on this page · raw message
drivers/misc/habanalabs/device.c +47 -0
92 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +71 -0
131 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +44 -2
68 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v6 5/15 habanalabs: add command buffer module
In reply to Oded GabbayThis patch adds the command buffer (CB) module, which allows the user to create and destroy CBs and to map them to the user's process address-space.
A command buffer is a memory blocks that reside in DMA-able address-space and is physically contiguous so it can be accessed by the device without MMU translation. The command buffer memory is allocated using the coherent DMA API.
When creating a new CB, the IOCTL returns a handle of it, and the user-space process needs to use that handle to mmap the buffer to get a VA in the user's address-space.
Before destroying (freeing) a CB, the user must unmap the CB's VA using the CB handle.
Each CB has a reference counter, which tracks its usage in command submissions and also its mmaps (only a single mmap is allowed).
The driver maintains a pool of pre-allocated CBs in order to reduce latency during command submissions. In case the pool is empty, the driver will go to the slow-path of allocating a new CB, i.e. calling dma_alloc_coherent.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
drivers/misc/habanalabs/Makefile | 3 +- drivers/misc/habanalabs/command_buffer.c | 433 +++++++++++++++++++++ drivers/misc/habanalabs/device.c | 43 +- drivers/misc/habanalabs/goya/goya.c | 28 ++ drivers/misc/habanalabs/habanalabs.h | 86 ++++ drivers/misc/habanalabs/habanalabs_drv.c | 2 + drivers/misc/habanalabs/habanalabs_ioctl.c | 99 +++++ include/uapi/misc/habanalabs.h | 46 +++ 8 files changed, 738 insertions(+), 2 deletions(-) create mode 100644 drivers/misc/habanalabs/command_buffer.c create mode 100644 drivers/misc/habanalabs/habanalabs_ioctl.c
drivers/misc/habanalabs/Makefile +2 -1
@@ -4,7 +4,8 @@ obj-m := habanalabs.o -habanalabs-y := habanalabs_drv.o device.o context.o asid.o +habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ + command_buffer.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_buffer.c new file +433 -0
433 lines not on this page · raw message
drivers/misc/habanalabs/device.c +42 -1
87 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya.c +28 -0
@@ -82,6 +82,9 @@#define GOYA_MAX_INITIATORS 20 +#define GOYA_CB_POOL_CB_CNT 512 +#define GOYA_CB_POOL_CB_SIZE 0x20000/* 128KB */ +static void goya_get_fixed_properties (struct hl_device *hdev) {struct asic_fixed_properties *prop = &hdev->asic_prop;
@@ -109,6 +112,8 @@ static void goya_get_fixed_properties(struct hl_device *hdev) prop->tpc_enabled_mask =TPC_ENABLED_MASK ; prop->high_pll =PLL_HIGH_DEFAULT ; + prop->cb_pool_cb_cnt =GOYA_CB_POOL_CB_CNT ; + prop->cb_pool_cb_size =GOYA_CB_POOL_CB_SIZE ; }/*
@@ -597,6 +602,27 @@ int goya_resume(struct hl_device *hdev)return 0 ; } +int goya_mmap (struct hl_fpriv *hpriv,struct vm_area_struct *vma) +{ +return -EINVAL ; +} + +int goya_cb_mmap (struct hl_device *hdev,struct vm_area_struct *vma, +u64 kaddress,phys_addr_t paddress,u32 size) +{ +int rc; + + vma->vm_flags |=VM_IO |VM_PFNMAP |VM_DONTEXPAND |VM_DONTDUMP | +VM_DONTCOPY |VM_NORESERVE ; + + rc =remap_pfn_range (vma, vma->vm_start, paddress >>PAGE_SHIFT , + size, vma->vm_page_prot); +if (rc) +dev_err (hdev->dev,"remap_pfn_range error %d" , rc); + +return rc; +} +void *goya_dma_alloc_coherent (struct hl_device *hdev,size_t size,dma_addr_t *dma_handle,gfp_t flags) {
@@ -616,6 +642,8 @@ static const struct hl_asic_funcs goya_funcs = { .sw_fini = goya_sw_fini, .suspend = goya_suspend, .resume = goya_resume, + .mmap = goya_mmap, + .cb_mmap = goya_cb_mmap, .dma_alloc_coherent = goya_dma_alloc_coherent, .dma_free_coherent = goya_dma_free_coherent, };
drivers/misc/habanalabs/habanalabs.h +86 -0
166 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +2 -0
@@ -116,6 +116,7 @@ int hl_device_open(struct inode *inode, struct file *filp)kref_init (&hpriv->refcount);nonseekable_open (inode, filp); +hl_cb_mgr_init (&hpriv->cb_mgr);hl_ctx_mgr_init (&hpriv->ctx_mgr); rc =hl_ctx_create (hdev, hpriv);
@@ -131,6 +132,7 @@ int hl_device_open(struct inode *inode, struct file *filp) out_err: filp->private_data =NULL ;hl_ctx_mgr_fini (hpriv->hdev, &hpriv->ctx_mgr); +hl_cb_mgr_fini (hpriv->hdev, &hpriv->cb_mgr);kfree (hpriv); close_device:
drivers/misc/habanalabs/habanalabs_ioctl.c new file +99 -0
99 lines not on this page · raw message
include/uapi/misc/habanalabs.h +46 -0
50 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v6 6/15 habanalabs: add basic Goya h/w initialization
In reply to Oded GabbayThis patch adds the basic part of Goya's H/W initialization. It adds code that initializes Goya's internal CPU, various registers that are related to internal routing, scrambling, workarounds for H/W bugs, etc.
It also initializes Goya's security scheme that prevents the user from abusing Goya to steal data from the host, crash the host, change Goya's F/W, etc.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
drivers/misc/habanalabs/device.c | 12 + drivers/misc/habanalabs/goya/Makefile | 2 +- drivers/misc/habanalabs/goya/goya.c | 882 ++++- drivers/misc/habanalabs/goya/goyaP.h | 4 + drivers/misc/habanalabs/goya/goya_security.c | 2999 +++++++++++++++++ drivers/misc/habanalabs/habanalabs.h | 16 + drivers/misc/habanalabs/habanalabs_drv.c | 7 + drivers/misc/habanalabs/include/armcp_if.h | 19 + .../misc/habanalabs/include/goya/goya_fw_if.h | 28 + drivers/misc/habanalabs/include/hl_boot_if.h | 30 + 10 files changed, 3991 insertions(+), 8 deletions(-) create mode 100644 drivers/misc/habanalabs/goya/goya_security.c create mode 100644 drivers/misc/habanalabs/include/armcp_if.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_fw_if.h create mode 100644 drivers/misc/habanalabs/include/hl_boot_if.h
drivers/misc/habanalabs/device.c +12 -0
@@ -319,6 +319,15 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) goto release_ctx; } + rc = hdev->asic_funcs->hw_init (hdev); +if (rc) { +dev_err (hdev->dev,"failed to initialize the H/W\n" ); + rc =0 ; + goto out_disabled; + } + + hdev->disabled = false; +dev_notice (hdev->dev,"Successfully added device to habanalabs driver\n" );
@@ -370,6 +379,9 @@ void hl_device_fini(struct hl_device *hdev)if ((hdev->kernel_ctx) && (hl_ctx_put (hdev->kernel_ctx) !=1 ))dev_err (hdev->dev,"kernel ctx is still alive\n" ); +/* Reset the H/W. It will be in idle state after this returns */ + hdev->asic_funcs->hw_fini (hdev, true); +/* Call ASIC S/W finalize function */ hdev->asic_funcs->sw_fini (hdev);
drivers/misc/habanalabs/goya/Makefile +1 -1
@@ -1,3 +1,3 @@ subdir-ccflags-y += -I$(src) -HL_GOYA_FILES := goya/goya.o +HL_GOYA_FILES := goya/goya.o goya/goya_security.o
drivers/misc/habanalabs/goya/goya.c +875 -7
916 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +4 -0
@@ -10,7 +10,9 @@#include <uapi/misc/habanalabs.h> #include "habanalabs.h" +#include "include/hl_boot_if.h" #include "include/goya/goya.h" +#include "include/goya/goya_fw_if.h" #define NUMBER_OF_CMPLT_QUEUES 5#define NUMBER_OF_EXT_HW_QUEUES 5
@@ -149,4 +151,6 @@ struct goya_device { u32 hw_cap_initialized; }; +void goya_init_security (struct hl_device *hdev); +#endif /* GOYAP_H_ */
drivers/misc/habanalabs/goya/goya_security.c new file +2999 -0
2,041 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +7 -0
Lines of this file are not shown. View the raw message.
drivers/misc/habanalabs/include/armcp_if.h new file +19 -0
Lines of this file are not shown. View the raw message.
drivers/misc/habanalabs/include/goya/goya_fw_if.h new file +28 -0
Lines of this file are not shown. View the raw message.
drivers/misc/habanalabs/include/hl_boot_if.h new file +30 -0
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.18.0
Part of the diff is not shown here. View the raw message.
PATCH v6 7/15 habanalabs: add h/w queues module
In reply to Oded GabbayThis patch adds the H/W queues module and the code to initialize Goya's various compute and DMA engines and their queues.
Goya has 5 DMA channels, 8 TPC engines and a single MME engine. For each channel/engine, there is a H/W queue logic which is used to pass commands from the user to the H/W. That logic is called QMAN.
There are two types of QMANs: external and internal. The DMA QMANs are considered external while the TPC and MME QMANs are considered internal. For each external queue there is a completion queue, which is located on the Host memory.
The differences between external and internal QMANs are:
1. The location of the queue's memory. External QMANs are located on the Host memory while internal QMANs are located on the on-chip memory.
2. The external QMAN write an entry to a completion queue and sends an MSI-X interrupt upon completion of a command buffer that was given to it. The internal QMAN doesn't do that.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/device.c | 75 +- drivers/misc/habanalabs/goya/goya.c | 1527 +++++++++++++++-- drivers/misc/habanalabs/goya/goyaP.h | 7 + drivers/misc/habanalabs/habanalabs.h | 174 +- drivers/misc/habanalabs/habanalabs_drv.c | 5 + drivers/misc/habanalabs/hw_queue.c | 400 +++++ drivers/misc/habanalabs/include/armcp_if.h | 292 ++++ .../include/goya/goya_async_events.h | 186 ++ .../habanalabs/include/goya/goya_packets.h | 129 ++ drivers/misc/habanalabs/include/qman_if.h | 56 + drivers/misc/habanalabs/irq.c | 149 ++ include/uapi/misc/habanalabs.h | 29 + 13 files changed, 2915 insertions(+), 116 deletions(-) create mode 100644 drivers/misc/habanalabs/hw_queue.c create mode 100644 drivers/misc/habanalabs/include/goya/goya_async_events.h create mode 100644 drivers/misc/habanalabs/include/goya/goya_packets.h create mode 100644 drivers/misc/habanalabs/include/qman_if.h create mode 100644 drivers/misc/habanalabs/irq.c
drivers/misc/habanalabs/Makefile +1 -1
@@ -5,7 +5,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ - command_buffer.o + command_buffer.o hw_queue.o irq.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/device.c +73 -2
137 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya.c +1415 -112
1,641 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +7 -0
@@ -11,7 +11,9 @@#include <uapi/misc/habanalabs.h> #include "habanalabs.h" #include "include/hl_boot_if.h" +#include "include/goya/goya_packets.h" #include "include/goya/goya.h" +#include "include/goya/goya_async_events.h" #include "include/goya/goya_fw_if.h" #define NUMBER_OF_CMPLT_QUEUES 5
@@ -145,12 +147,17 @@ enum goya_fw_component { };struct goya_device { +int (*test_cpu_queue)(struct hl_device *hdev); +/* TODO: remove hw_queues_lock after moving to scheduler code */ spinlock_t hw_queues_lock;u64 ddr_bar_cur_addr;u32 hw_cap_initialized; }; +int goya_test_cpu_queue (struct hl_device *hdev); +int goya_send_cpu_message (struct hl_device *hdev,u32 *msg,u16 len, +u32 timeout,long *result);void goya_init_security (struct hl_device *hdev);#endif /* GOYAP_H_ */
drivers/misc/habanalabs/habanalabs.h +173 -1
268 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +5 -0
@@ -169,6 +169,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,/* Parameters for bring-up - set them to defaults */ hdev->cpu_enable =1 ; hdev->reset_pcilink =0 ; + hdev->cpu_queues_enable =1 ; hdev->fw_loading =1 ; hdev->pldm =0 ;
@@ -176,6 +177,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,if (!hdev->cpu_enable) hdev->fw_loading =0 ; +/* If we don't load FW, no need to initialize CPU queues */ +if (!hdev->fw_loading) + hdev->cpu_queues_enable =0 ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/hw_queue.c new file +400 -0
400 lines not on this page · raw message
drivers/misc/habanalabs/include/armcp_if.h +292 -0
302 lines not on this page · raw message
drivers/misc/habanalabs/include/goya/goya_async_events.h new file +186 -0
186 lines not on this page · raw message
drivers/misc/habanalabs/include/goya/goya_packets.h new file +129 -0
15 lines not on this page · raw message
drivers/misc/habanalabs/include/qman_if.h new file +56 -0
56 lines not on this page · raw message
drivers/misc/habanalabs/irq.c new file +149 -0
149 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.18.0
Part of the diff is not shown here. View the raw message.
PATCH v6 8/15 habanalabs: add event queue and interrupts
In reply to Oded GabbayThis patch adds support for receiving events from Goya's control CPU and for receiving MSI-X interrupts from Goya's DMA engines and CPU.
Goya's PCI controller supports up to 8 MSI-X interrupts, which only 6 of them are currently used. The first 5 interrupts are dedicated for Goya's DMA engine queues. The 6th interrupt is dedicated for Goya's control CPU.
The DMA queue will signal its MSI-X entry upon each completion of a command buffer that was placed on its primary queue. The driver will then mark that CB as completed and free the related resources. It will also update the command submission object which that CB belongs to.
There is a dedicated event queue (EQ) between the driver and Goya's control CPU. The EQ is located on the Host memory. The control CPU writes a new entry to the EQ for various reasons, such as ECC error, MMU page fault, Hot temperature. After writing the new entry to the EQ, the control CPU will trigger its dedicated MSI-X entry to signal the driver that there is a new entry in the EQ. The driver will then read the entry and act accordingly.
---
Changes in v6:
- Only include headers needed for compilation
drivers/misc/habanalabs/device.c | 34 +- drivers/misc/habanalabs/goya/goya.c | 461 ++++++++++++++++++++- drivers/misc/habanalabs/goya/goyaP.h | 1 + drivers/misc/habanalabs/habanalabs.h | 40 +- drivers/misc/habanalabs/include/armcp_if.h | 24 ++ drivers/misc/habanalabs/irq.c | 147 ++++++- 6 files changed, 694 insertions(+), 13 deletions(-)
drivers/misc/habanalabs/device.c +33 -1
@@ -181,6 +181,13 @@ static int device_early_init(struct hl_device *hdev) goto asid_fini; } + hdev->eq_wq =alloc_workqueue ("hl-events" ,WQ_UNBOUND ,0 ); +if (hdev->eq_wq ==NULL ) { +dev_err (hdev->dev,"Failed to allocate EQ workqueue\n" ); + rc = -ENOMEM ; + goto free_cq_wq; + } +hl_cb_mgr_init (&hdev->kernel_cb_mgr);mutex_init (&hdev->fd_open_cnt_lock);
@@ -189,6 +196,8 @@ static int device_early_init(struct hl_device *hdev)return 0 ; +free_cq_wq: +destroy_workqueue (hdev->cq_wq); asid_fini:hl_asid_fini (hdev); early_fini:
@@ -210,6 +219,7 @@ static void device_early_fini(struct hl_device *hdev)hl_cb_mgr_fini (hdev, &hdev->kernel_cb_mgr); +destroy_workqueue (hdev->eq_wq);destroy_workqueue (hdev->cq_wq);hl_asid_fini (hdev);
@@ -348,11 +358,22 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) } } +/* +* Initialize the event queue. Must be done before hw_init, +* because there the address of the event queue is being +* passed as argument to request_irq +*/ + rc =hl_eq_init (hdev, &hdev->event_queue); +if (rc) { +dev_err (hdev->dev,"failed to initialize event queue\n" ); + goto cq_fini; + } +/* Allocate the kernel context */ hdev->kernel_ctx =kzalloc (sizeof (*hdev->kernel_ctx),GFP_KERNEL );if (!hdev->kernel_ctx) { rc = -ENOMEM ; - goto cq_fini; + goto eq_fini; } hdev->user_ctx =NULL ;
@@ -397,6 +418,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) "kernel ctx is still alive on initialization failure\n"); free_ctx:kfree (hdev->kernel_ctx); +eq_fini: +hl_eq_fini (hdev, &hdev->event_queue); cq_fini:for (i =0 ; i < cq_ready_cnt ; i++)hl_cq_fini (hdev, &hdev->completion_queue[i]);
@@ -438,6 +461,13 @@ void hl_device_fini(struct hl_device *hdev)/* Mark device as disabled */ hdev->disabled = true; +/* +* Halt the engines and disable interrupts so we won't get any more +* completions from H/W and we won't have any accesses from the +* H/W to the host machine +*/ + hdev->asic_funcs->halt_engines (hdev, true); +hl_cb_pool_fini (hdev);/* Release kernel context */
@@ -447,6 +477,8 @@ void hl_device_fini(struct hl_device *hdev)/* Reset the H/W. It will be in idle state after this returns */ hdev->asic_funcs->hw_fini (hdev, true); +hl_eq_fini (hdev, &hdev->event_queue); +for (i =0 ; i < hdev->asic_prop.completion_queues_count ; i++)hl_cq_fini (hdev, &hdev->completion_queue[i]);kfree (hdev->completion_queue);
drivers/misc/habanalabs/goya/goya.c +452 -9
552 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +1 -0
@@ -152,6 +152,7 @@ struct goya_device {/* TODO: remove hw_queues_lock after moving to scheduler code */ spinlock_t hw_queues_lock;u64 ddr_bar_cur_addr; +u32 events_stat[GOYA_ASYNC_EVENT_ID_SIZE ];u32 hw_cap_initialized; };
drivers/misc/habanalabs/habanalabs.h +38 -2
@@ -15,6 +15,7 @@#include <linux/cdev.h> #include <linux/iopoll.h> +#include <linux/irqreturn.h> #define HL_NAME "habanalabs"
@@ -80,6 +81,7 @@ struct hw_queue_properties {* @cfg_size: configuration space size on SRAM. * @sram_size: total size of SRAM. * @max_asid: maximum number of open contexts (ASIDs). +* @num_of_events: number of possible internal H/W IRQs. * @completion_queues_count: number of completion queues. * @high_pll: high PLL frequency used by the device. * @cb_pool_cb_cnt: number of CBs in the CB pool.
@@ -106,6 +108,7 @@ struct asic_fixed_properties {u32 cfg_size;u32 sram_size;u32 max_asid; +u32 num_of_events;u32 high_pll;u32 cb_pool_cb_cnt;u32 cb_pool_cb_size;
@@ -198,6 +201,9 @@ struct hl_cs_job;#define HL_CQ_LENGTH HL_QUEUE_LENGTH#define HL_CQ_SIZE_IN_BYTES (HL_CQ_LENGTH * HL_CQ_ENTRY_SIZE) +/* Must be power of 2 (HL_PAGE_SIZE / HL_EQ_ENTRY_SIZE) */ +#define HL_EQ_LENGTH 64 +#define HL_EQ_SIZE_IN_BYTES (HL_EQ_LENGTH * HL_EQ_ENTRY_SIZE)/**
@@ -245,6 +251,20 @@ struct hl_cq {atomic_t free_slots_cnt; }; +/** +* struct hl_eq - describes the event queue (single one per device) +* @hdev: pointer to the device structure +* @kernel_address: holds the queue's kernel virtual address +* @bus_address: holds the queue's DMA address +* @ci: ci inside the queue +*/ +struct hl_eq { +struct hl_device *hdev; +u64 kernel_address; +dma_addr_t bus_address; +u32 ci; +}; +/* * ASICs
@@ -271,6 +291,9 @@ enum hl_asic_type {* @sw_fini: tears down driver state, does not configure H/W. * @hw_init: sets up the H/W state. * @hw_fini: tears down the H/W state. +* @halt_engines: halt engines, needed for reset sequence. This also disables +* interrupts from the device. Should be called before +* hw_fini and before CS rollback. * @suspend: handles IP specific H/W or SW changes for suspend. * @resume: handles IP specific H/W or SW changes for resume. * @mmap: mmap function, does nothing.
@@ -292,6 +315,9 @@ enum hl_asic_type {* @dma_pool_free: free small DMA allocation from pool. * @cpu_accessible_dma_pool_alloc: allocate CPU PQ packet from DMA pool. * @cpu_accessible_dma_pool_free: free CPU PQ packet from DMA pool. +* @update_eq_ci: update event queue CI. +* @handle_eqe: handle event queue entry (IRQ) from ArmCP. +* @get_events_stat: retrieve event queue entries histogram. * @hw_queues_lock: acquire H/W queues lock. * @hw_queues_unlock: release H/W queues lock. * @send_cpu_message: send buffer to ArmCP.
@@ -303,6 +329,7 @@ struct hl_asic_funcs {int (*sw_fini)(struct hl_device *hdev);int (*hw_init)(struct hl_device *hdev);void (*hw_fini)(struct hl_device *hdev,bool hard_reset); +void (*halt_engines)(struct hl_device *hdev,bool hard_reset);int (*suspend)(struct hl_device *hdev);int (*resume)(struct hl_device *hdev);int (*mmap)(struct hl_fpriv *hpriv,struct vm_area_struct *vma);
@@ -325,6 +352,10 @@ struct hl_asic_funcs { size_t size, dma_addr_t *dma_handle);void (*cpu_accessible_dma_pool_free)(struct hl_device *hdev,size_t size,void *vaddr); +void (*update_eq_ci)(struct hl_device *hdev,u32 val); +void (*handle_eqe)(struct hl_device *hdev, +struct hl_eq_entry *eq_entry); +void * (*get_events_stat)(struct hl_device *hdev,u32 *size);void (*hw_queues_lock)(struct hl_device *hdev);void (*hw_queues_unlock)(struct hl_device *hdev); int (*send_cpu_message)(struct hl_device *hdev, u32 *msg,
@@ -364,8 +395,6 @@ struct hl_ctx_mgr { }; - -/** * struct hl_cs_job - command submission job. * @finish_work: workqueue object to run when job is completed.
@@ -455,6 +484,7 @@ void hl_wreg(struct hl_device *hdev, u32 reg, u32 val);* @kernel_ctx: KMD context structure. * @kernel_queues: array of hl_hw_queue. * @kernel_cb_mgr: command buffer manager for creating/destroying/handling CGs. +* @event_queue: event queue for IRQ from ArmCP. * @dma_pool: DMA pool for small allocations. * @cpu_accessible_dma_mem: KMD <-> ArmCP shared memory CPU address. * @cpu_accessible_dma_address: KMD <-> ArmCP shared memory DMA address.
@@ -489,9 +519,11 @@ struct hl_device {enum hl_asic_type asic_type;struct hl_cq *completion_queue;struct workqueue_struct *cq_wq; +struct workqueue_struct *eq_wq;struct hl_ctx *kernel_ctx;struct hl_hw_queue *kernel_queues;struct hl_cb_mgr kernel_cb_mgr; +struct hl_eq event_queue;struct dma_pool *dma_pool;void *cpu_accessible_dma_mem;dma_addr_t cpu_accessible_dma_address;
@@ -573,6 +605,10 @@ void hl_hw_queue_inc_ci_kernel(struct hl_device *hdev, u32 hw_queue_id);int hl_cq_init (struct hl_device *hdev,struct hl_cq *q,u32 hw_queue_id);void hl_cq_fini (struct hl_device *hdev,struct hl_cq *q); +int hl_eq_init (struct hl_device *hdev,struct hl_eq *q); +void hl_eq_fini (struct hl_device *hdev,struct hl_eq *q); +irqreturn_t hl_irq_handler_cq (int irq,void *arg); +irqreturn_t hl_irq_handler_eq (int irq,void *arg);int hl_asid_init (struct hl_device *hdev);void hl_asid_fini (struct hl_device *hdev);unsigned long hl_asid_alloc (struct hl_device *hdev);
drivers/misc/habanalabs/include/armcp_if.h +24 -0
@@ -10,6 +10,30 @@#include <linux/types.h> +/* +* EVENT QUEUE +*/ + +struct hl_eq_header { +__le32 reserved; +__le32 ctl; +}; + +struct hl_eq_entry { +struct hl_eq_header hdr; +__le64 data[7 ]; +}; + +#define HL_EQ_ENTRY_SIZE sizeof(struct hl_eq_entry) + +#define EQ_CTL_READY_SHIFT 31 +#define EQ_CTL_READY_MASK 0x80000000 + +#define EQ_CTL_EVENT_TYPE_SHIFT 16 +#define EQ_CTL_EVENT_TYPE_MASK 0x03FF0000 + +#define EVENT_QUEUE_MSIX_IDX 5 +enum pq_init_status {PQ_INIT_STATUS_NA =0 ,PQ_INIT_STATUS_READY_FOR_CP ,
drivers/misc/habanalabs/irq.c +146 -1
168 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v6 9/15 habanalabs: add sysfs and hwmon support
In reply to Oded GabbayThis patch add the sysfs and hwmon entries that are exposed by the driver.
Goya has several sensors, from various categories such as temperature, voltage, current, etc. The driver exposes those sensors in the standard hwmon mechanism.
In addition, the driver exposes a couple of interfaces in sysfs, both for configuration and for providing status of the device or driver.
The configuration attributes is for Power Management:
- Automatic or manual
- Frequency value when moving to high frequency mode
- Maximum power the device is allowed to consume
The rest of the attributes are read-only and provide the following information:
- Versions of the various firmwares running on the device
- Contents of the device's EEPROM
- The device type (currently only Goya is supported)
- PCI address of the device (to allow user-space to connect between /dev/hlX to PCI address)
- Status of the device (operational, malfunction, in_reset)
- How many processes are open on the device's file
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
- hl_device holds a pointer to hwmon_chip_info struct instead of an instance
.../ABI/testing/sysfs-driver-habanalabs | 190 +++++++ drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/device.c | 163 ++++++ drivers/misc/habanalabs/goya/Makefile | 2 +- drivers/misc/habanalabs/goya/goya.c | 230 +++++++++ drivers/misc/habanalabs/goya/goyaP.h | 21 + drivers/misc/habanalabs/goya/goya_hwmgr.c | 254 ++++++++++ drivers/misc/habanalabs/habanalabs.h | 102 ++++ drivers/misc/habanalabs/habanalabs_drv.c | 7 + drivers/misc/habanalabs/hwmon.c | 452 +++++++++++++++++ drivers/misc/habanalabs/sysfs.c | 468 ++++++++++++++++++ 11 files changed, 1889 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/goya/goya_hwmgr.c create mode 100644 drivers/misc/habanalabs/hwmon.c create mode 100644 drivers/misc/habanalabs/sysfs.c
Documentation/ABI/testing/sysfs-driver-habanalabs new file +190 -0
@@ -0,0 +1,190 @@ +What: /sys/class/habanalabs/hl<n>/armcp_kernel_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Linux kernel running on the device's CPU + +What: /sys/class/habanalabs/hl<n>/armcp_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the application running on the device's CPU + +What: /sys/class/habanalabs/hl<n>/cpld_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Device's CPLD F/W + +What: /sys/class/habanalabs/hl<n>/device_type +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the code name of the device according to its type. + The supported values are: "GOYA" + +What: /sys/class/habanalabs/hl<n>/eeprom +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: A binary file attribute that contains the contents of the + on-board EEPROM + +What: /sys/class/habanalabs/hl<n>/fuse_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the device's version from the eFuse + +What: /sys/class/habanalabs/hl<n>/hard_reset +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Interface to trigger a hard-reset operation for the device. + Hard-reset will reset ALL internal components of the device + except for the PCI interface and the internal PLLs + +What: /sys/class/habanalabs/hl<n>/hard_reset_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays how many times the device have undergone a hard-reset + operation since the driver was loaded + +What: /sys/class/habanalabs/hl<n>/high_pll +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency for MME, TPC + and IC when the power management profile is set to "automatic". + +What: /sys/class/habanalabs/hl<n>/ic_clk +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency of the + Interconnect fabric. Writes to this parameter affect the device + only when the power management profile is set to "manual" mode. + The device IC clock might be set to lower value then the + maximum. The user should read the ic_clk_curr to see the actual + frequency value of the IC + +What: /sys/class/habanalabs/hl<n>/ic_clk_curr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the current clock frequency of the Interconnect fabric + +What: /sys/class/habanalabs/hl<n>/infineon_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Device's power supply F/W code + +What: /sys/class/habanalabs/hl<n>/max_power +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum power consumption of the + device in milliwatts. + +What: /sys/class/habanalabs/hl<n>/mme_clk +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency of the + MME compute engine. Writes to this parameter affect the device + only when the power management profile is set to "manual" mode. + The device MME clock might be set to lower value then the + maximum. The user should read the mme_clk_curr to see the actual + frequency value of the MME + +What: /sys/class/habanalabs/hl<n>/mme_clk_curr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the current clock frequency of the MME compute engine + +What: /sys/class/habanalabs/hl<n>/pci_addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the PCI address of the device. This is needed so the + user would be able to open a device based on its PCI address + +What: /sys/class/habanalabs/hl<n>/pm_mng_profile +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Power management profile. Values are "auto", "manual". In "auto" + mode, the driver will set the maximum clock frequency to a high + value when a user-space process opens the device's file (unless + it was already opened by another process). The driver will set + the max clock frequency to a low value when there are no user + processes that are opened on the device's file. In "manual" + mode, the user sets the maximum clock frequency by writing to + ic_clk, mme_clk and tpc_clk + + +What: /sys/class/habanalabs/hl<n>/preboot_btl_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the device's preboot F/W code + +What: /sys/class/habanalabs/hl<n>/soft_reset +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Interface to trigger a soft-reset operation for the device. + Soft-reset will reset only the compute and DMA engines of the + device + +What: /sys/class/habanalabs/hl<n>/soft_reset_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays how many times the device have undergone a soft-reset + operation since the driver was loaded + +What: /sys/class/habanalabs/hl<n>/status +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Status of the card: "Operational", "Malfunction", "In reset". + +What: /sys/class/habanalabs/hl<n>/thermal_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the Device's thermal daemon + +What: /sys/class/habanalabs/hl<n>/tpc_clk +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the user to set the maximum clock frequency of the + TPC compute engines. Writes to this parameter affect the device + only when the power management profile is set to "manual" mode. + The device TPC clock might be set to lower value then the + maximum. The user should read the tpc_clk_curr to see the actual + frequency value of the TPC + +What: /sys/class/habanalabs/hl<n>/tpc_clk_curr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the current clock frequency of the TPC compute engines + +What: /sys/class/habanalabs/hl<n>/uboot_ver +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Version of the u-boot running on the device's CPU + +What: /sys/class/habanalabs/hl<n>/write_open_cnt +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the total number of user processes that are currently + opened on the device's file
drivers/misc/habanalabs/Makefile +1 -1
@@ -5,7 +5,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ - command_buffer.o hw_queue.o irq.o + command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/device.c +163 -0
216 lines not on this page · raw message
drivers/misc/habanalabs/goya/Makefile +1 -1
@@ -1,3 +1,3 @@ subdir-ccflags-y += -I$(src) -HL_GOYA_FILES := goya/goya.o goya/goya_security.o +HL_GOYA_FILES := goya/goya.o goya/goya_security.o goya/goya_hwmgr.o
drivers/misc/habanalabs/goya/goya.c +230 -0
288 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +21 -0
@@ -47,7 +47,10 @@#define PLL_HIGH_DEFAULT 1575000000/* 1.575 GHz */ +#define MAX_POWER_DEFAULT 200000/* 200W */ +#define GOYA_ARMCP_INFO_TIMEOUT 10000000/* 10s */ +#define GOYA_ARMCP_EEPROM_TIMEOUT 10000000/* 10s */ #define DRAM_PHYS_DEFAULT_SIZE 0x100000000ull/* 4GB */
@@ -148,9 +151,15 @@ enum goya_fw_component {struct goya_device {int (*test_cpu_queue)(struct hl_device *hdev); +int (*armcp_info_get)(struct hl_device *hdev);/* TODO: remove hw_queues_lock after moving to scheduler code */ spinlock_t hw_queues_lock; + +u64 mme_clk; +u64 tpc_clk; +u64 ic_clk; +u64 ddr_bar_cur_addr;u32 events_stat[GOYA_ASYNC_EVENT_ID_SIZE ];u32 hw_cap_initialized;
@@ -159,6 +168,18 @@ struct goya_device {int goya_test_cpu_queue (struct hl_device *hdev);int goya_send_cpu_message (struct hl_device *hdev,u32 *msg,u16 len,u32 timeout,long *result); +long goya_get_temperature (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_voltage (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_current (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_fan_speed (struct hl_device *hdev,int sensor_index,u32 attr); +long goya_get_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr); +void goya_set_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr, +long value); +void goya_set_pll_profile (struct hl_device *hdev,enum hl_pll_frequency freq); +void goya_add_device_attr (struct hl_device *hdev, +struct attribute_group *dev_attr_grp);void goya_init_security (struct hl_device *hdev); +u64 goya_get_max_power (struct hl_device *hdev); +void goya_set_max_power (struct hl_device *hdev,u64 value);#endif /* GOYAP_H_ */
drivers/misc/habanalabs/goya/goya_hwmgr.c new file +254 -0
254 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +102 -0
246 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +7 -0
@@ -127,6 +127,13 @@ int hl_device_open(struct inode *inode, struct file *filp) hpriv->taskpid =find_get_pid (current->pid); +/* +* Device is IDLE at this point so it is legal to change PLLs. There +* is no need to check anything because if the PLL is already HIGH, the +* set function will return without doing anything +*/ +hl_device_set_frequency (hdev,PLL_HIGH ); +return 0 ; out_err:
drivers/misc/habanalabs/hwmon.c new file +452 -0
452 lines not on this page · raw message
drivers/misc/habanalabs/sysfs.c new file +468 -0
468 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v6 10/15 habanalabs: add device reset support
In reply to Oded GabbayThis patch adds support for doing various on-the-fly reset of Goya.
The driver supports two types of resets:
1. soft-reset
2. hard-reset
Soft-reset is done when the device detects a timeout of a command submission that was given to the device. The soft-reset process only resets the engines that are relevant for the submission of compute jobs, i.e. the DMA channels, the TPCs and the MME. The purpose is to bring the device as fast as possible to a working state.
Hard-reset is done in several cases:
1. After soft-reset is done but the device is not responding
2. When fatal errors occur inside the device, e.g. ECC error
3. When the driver is removed
Hard-reset performs a reset of the entire chip except for the PCI controller and the PLLs. It is a much longer process then soft-reset but it helps to recover the device without the need to reboot the Host.
After hard-reset, the driver will restore the max power attribute and in case of manual power management, the frequencies that were set.
This patch also adds two entries to the sysfs, which allows the root user to initiate a soft or hard reset.
---
Changes in v6:
- Only include headers needed for compilation
drivers/misc/habanalabs/command_buffer.c | 9 +- drivers/misc/habanalabs/device.c | 325 ++++++++++++++++++++- drivers/misc/habanalabs/goya/goya.c | 226 +++++++++++++- drivers/misc/habanalabs/goya/goya_hwmgr.c | 18 +- drivers/misc/habanalabs/habanalabs.h | 52 ++++ drivers/misc/habanalabs/habanalabs_drv.c | 9 +- drivers/misc/habanalabs/habanalabs_ioctl.c | 6 + drivers/misc/habanalabs/hwmon.c | 4 +- drivers/misc/habanalabs/irq.c | 31 ++ drivers/misc/habanalabs/sysfs.c | 82 +++++- 10 files changed, 737 insertions(+), 25 deletions(-)
drivers/misc/habanalabs/command_buffer.c +7 -2
@@ -91,9 +91,14 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr,bool alloc_new_cb = true;int rc; -if (hdev->disabled) { +/* +* Can't use generic function to check this because of special case +* where we create a CB as part of the reset process +*/ +if ((hdev->disabled) || ((atomic_read (&hdev->in_reset)) && + (ctx_id !=HL_KERNEL_ASID_ID ))) {dev_warn_ratelimited (hdev->dev, -"Device is disabled. Can't create new CBs\n" ); +"Device is disabled or in reset. Can't create new CBs\n" ); rc = -EBUSY ; goto out_err; }
drivers/misc/habanalabs/device.c +323 -2
383 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya.c +225 -1
279 lines not on this page · raw message
drivers/misc/habanalabs/goya/goya_hwmgr.c +9 -9
@@ -38,7 +38,7 @@ static ssize_t mme_clk_show(struct device *dev, struct device_attribute *attr,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,MME_PLL , false);
@@ -57,7 +57,7 @@ static ssize_t mme_clk_store(struct device *dev, struct device_attribute *attr,int rc;long value; -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) { count = -ENODEV ; goto fail; }
@@ -87,7 +87,7 @@ static ssize_t tpc_clk_show(struct device *dev, struct device_attribute *attr,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,TPC_PLL , false);
@@ -106,7 +106,7 @@ static ssize_t tpc_clk_store(struct device *dev, struct device_attribute *attr,int rc;long value; -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) { count = -ENODEV ; goto fail; }
@@ -136,7 +136,7 @@ static ssize_t ic_clk_show(struct device *dev, struct device_attribute *attr,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,IC_PLL , false);
@@ -155,7 +155,7 @@ static ssize_t ic_clk_store(struct device *dev, struct device_attribute *attr,int rc;long value; -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) { count = -ENODEV ; goto fail; }
@@ -185,7 +185,7 @@ static ssize_t mme_clk_curr_show(struct device *dev,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,MME_PLL , true);
@@ -202,7 +202,7 @@ static ssize_t tpc_clk_curr_show(struct device *dev,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,TPC_PLL , true);
@@ -219,7 +219,7 @@ static ssize_t ic_clk_curr_show(struct device *dev,struct hl_device *hdev =dev_get_drvdata (dev);long value; -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ; value =hl_get_frequency (hdev,IC_PLL , true);
drivers/misc/habanalabs/habanalabs.h +52 -0
165 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +7 -2
@@ -84,9 +84,9 @@ int hl_device_open(struct inode *inode, struct file *filp)mutex_lock (&hdev->fd_open_cnt_lock); -if (hdev->disabled) { +if (hl_device_disabled_or_in_reset (hdev)) {dev_err_ratelimited (hdev->dev, -"Can't open %s because it is disabled\n" , +"Can't open %s because it is disabled or in reset\n" ,dev_name (hdev->dev));mutex_unlock (&hdev->fd_open_cnt_lock);return -EPERM ;
@@ -179,6 +179,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev, hdev->cpu_queues_enable =1 ; hdev->fw_loading =1 ; hdev->pldm =0 ; + hdev->heartbeat =1 ;/* If CPU is disabled, no point in loading FW */ if (!hdev->cpu_enable)
@@ -188,6 +189,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,if (!hdev->fw_loading) hdev->cpu_queues_enable =0 ; +/* If CPU queues not enabled, no way to do heartbeat */ +if (!hdev->cpu_queues_enable) + hdev->heartbeat =0 ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/habanalabs_ioctl.c +6 -0
@@ -33,6 +33,12 @@ long hl_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)unsigned int usize, asize;int retcode; +if (hdev->hard_reset_pending) { +dev_crit_ratelimited (hdev->dev, +"Device HARD reset pending! Please close FD\n" ); +return -ENODEV ; + } +if ((nr >=HL_COMMAND_START ) && (nr <HL_COMMAND_END )) {u32 hl_size;
drivers/misc/habanalabs/hwmon.c +2 -2
@@ -114,7 +114,7 @@ static int hl_read(struct device *dev, enum hwmon_sensor_types type, {struct hl_device *hdev =dev_get_drvdata (dev); -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ;switch (type) {
@@ -188,7 +188,7 @@ static int hl_write(struct device *dev, enum hwmon_sensor_types type, {struct hl_device *hdev =dev_get_drvdata (dev); -if (hdev->disabled) +if (hl_device_disabled_or_in_reset (hdev))return -ENODEV ;switch (type) {
drivers/misc/habanalabs/irq.c +31 -0
@@ -250,6 +250,23 @@ void hl_cq_fini(struct hl_device *hdev, struct hl_cq *q) (void *) (uintptr_t) q->kernel_address, q->bus_address); } +void hl_cq_reset (struct hl_device *hdev,struct hl_cq *q) +{ + q->ci =0 ; + q->pi =0 ; + +atomic_set (&q->free_slots_cnt,HL_CQ_LENGTH ); + +/* +* It's not enough to just reset the PI/CI because the H/W may have +* written valid completion entries before it was halted and therefore +* we need to clean the actual queues so we won't process old entries +* when the device is operational again +*/ + +memset ((void *) (uintptr_t ) q->kernel_address,0 ,HL_CQ_SIZE_IN_BYTES ); +} +/* * hl_eq_init - main initialization function for an event queue object *
@@ -292,3 +309,17 @@ void hl_eq_fini(struct hl_device *hdev, struct hl_eq *q) hdev->asic_funcs->dma_free_coherent (hdev,HL_EQ_SIZE_IN_BYTES , (void *) (uintptr_t ) q->kernel_address, q->bus_address); } + +void hl_eq_reset (struct hl_device *hdev,struct hl_eq *q) +{ + q->ci =0 ; + +/* +* It's not enough to just reset the PI/CI because the H/W may have +* written valid completion entries before it was halted and therefore +* we need to clean the actual queues so we won't process old entries +* when the device is operational again +*/ + +memset ((void *) (uintptr_t ) q->kernel_address,0 ,HL_EQ_SIZE_IN_BYTES ); +}
drivers/misc/habanalabs/sysfs.c +75 -7
160 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v6 11/15 habanalabs: add command submission module
In reply to Oded GabbayThis patch adds the main flow for the user to submit work to the device.
Each work is described by a command submission object (CS). The CS contains 3 arrays of command buffers: One for execution, and two for context-switch (store and restore).
For each CB, the user specifies on which queue to put that CB. In case of an internal queue, the entry doesn't contain a pointer to the CB but the address in the on-chip memory that the CB resides at.
The driver parses some of the CBs to enforce security restrictions.
The user receives a sequence number that represents the CS object. The user can then query the driver regarding the status of the CS, using that sequence number.
In case the CS doesn't finish before the timeout expires, the driver will perform a soft-reset of the device.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
drivers/misc/habanalabs/Makefile | 3 +- drivers/misc/habanalabs/command_submission.c | 766 ++++++++++++ drivers/misc/habanalabs/context.c | 52 +- drivers/misc/habanalabs/device.c | 16 + drivers/misc/habanalabs/goya/goya.c | 1126 ++++++++++++++++++ drivers/misc/habanalabs/habanalabs.h | 275 +++++ drivers/misc/habanalabs/habanalabs_drv.c | 20 + drivers/misc/habanalabs/habanalabs_ioctl.c | 4 +- drivers/misc/habanalabs/hw_queue.c | 232 ++++ drivers/misc/habanalabs/memory.c | 198 +++ include/uapi/misc/habanalabs.h | 158 ++- 11 files changed, 2843 insertions(+), 7 deletions(-) create mode 100644 drivers/misc/habanalabs/command_submission.c create mode 100644 drivers/misc/habanalabs/memory.c
drivers/misc/habanalabs/Makefile +2 -1
@@ -5,7 +5,8 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ - command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o + command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ + command_submission.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_submission.c new file +766 -0
766 lines not on this page · raw message
drivers/misc/habanalabs/context.c +48 -4
82 lines not on this page · raw message
drivers/misc/habanalabs/device.c +16 -0
@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)put_pid (hpriv->taskpid); +mutex_destroy (&hpriv->restore_phase_mutex); +kfree (hpriv);/* Now the FD is really closed */
@@ -208,6 +210,8 @@ static int device_early_init(struct hl_device *hdev)mutex_init (&hdev->fd_open_cnt_lock);mutex_init (&hdev->send_cpu_message_lock); +INIT_LIST_HEAD (&hdev->hw_queues_mirror_list); +spin_lock_init (&hdev->hw_queues_mirror_lock);atomic_set (&hdev->in_reset,0 );atomic_set (&hdev->fd_open_cnt,0 );
@@ -593,6 +597,9 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,*/ hdev->asic_funcs->halt_engines (hdev, hard_reset); +/* Go over all the queues, release all CS and their jobs */ +hl_cs_rollback_all (hdev); +if (hard_reset) {/* Release kernel context */ if (hl_ctx_put (hdev->kernel_ctx) !=1 ) {
@@ -616,6 +623,12 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,for (i =0 ; i < hdev->asic_prop.completion_queues_count ; i++)hl_cq_reset (hdev, &hdev->completion_queue[i]); +/* Make sure the setup phase for the user context will run again */ +if (hdev->user_ctx) { +atomic_set (&hdev->user_ctx->thread_restore_token,1 ); + hdev->user_ctx->thread_restore_wait_token =0 ; + } +/* Finished tear-down, starting to re-initialize */ if (hard_reset) {
@@ -952,6 +965,9 @@ void hl_device_fini(struct hl_device *hdev)*/ hdev->asic_funcs->halt_engines (hdev, true); +/* Go over all the queues, release all CS and their jobs */ +hl_cs_rollback_all (hdev); +hl_cb_pool_fini (hdev);/* Release kernel context */
drivers/misc/habanalabs/goya/goya.c +1126 -0
1,168 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +275 -0
463 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +20 -0
@@ -24,6 +24,17 @@ static struct class *hl_class;DEFINE_IDR (hl_devs_idr);DEFINE_MUTEX (hl_devs_idr_lock); +static int timeout_locked =5 ; +static int reset_on_lockup =1 ; + +module_param (timeout_locked, int,0444 ); +MODULE_PARM_DESC (timeout_locked, +"Device lockup timeout in seconds (0 = disabled, default 5s)" ); + +module_param (reset_on_lockup, int,0444 ); +MODULE_PARM_DESC (reset_on_lockup, +"Do device reset on lockup (0 = no, 1 = yes, default yes)" ); +#define PCI_VENDOR_ID_HABANALABS 0x1da3#define PCI_IDS_GOYA 0x0001
@@ -113,6 +124,7 @@ int hl_device_open(struct inode *inode, struct file *filp) hpriv->hdev = hdev; filp->private_data = hpriv; hpriv->filp = filp; +mutex_init (&hpriv->restore_phase_mutex);kref_init (&hpriv->refcount);nonseekable_open (inode, filp);
@@ -140,6 +152,7 @@ int hl_device_open(struct inode *inode, struct file *filp) filp->private_data =NULL ;hl_ctx_mgr_fini (hpriv->hdev, &hpriv->ctx_mgr);hl_cb_mgr_fini (hpriv->hdev, &hpriv->cb_mgr); +mutex_destroy (&hpriv->restore_phase_mutex);kfree (hpriv); close_device:
@@ -172,8 +185,10 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,return -ENOMEM ; hdev->major = hl_major; + hdev->reset_on_lockup = reset_on_lockup;/* Parameters for bring-up - set them to defaults */ + hdev->mmu_enable =0 ; hdev->cpu_enable =1 ; hdev->reset_pcilink =0 ; hdev->cpu_queues_enable =1 ;
@@ -193,6 +208,11 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev,if (!hdev->cpu_queues_enable) hdev->heartbeat =0 ; +if (timeout_locked) + hdev->timeout_jiffies =msecs_to_jiffies (timeout_locked *1000 ); +else + hdev->timeout_jiffies =MAX_SCHEDULE_TIMEOUT ; + hdev->disabled = true; hdev->pdev = pdev;/* can be NULL in case of simulator device */
drivers/misc/habanalabs/habanalabs_ioctl.c +3 -1
@@ -16,7 +16,9 @@ [_IOC_NR(ioctl)] = {.cmd = ioctl, .func = _func}static const struct hl_ioctl_desc hl_ioctls[] = { -HL_IOCTL_DEF (HL_IOCTL_CB , hl_cb_ioctl) +HL_IOCTL_DEF (HL_IOCTL_CB , hl_cb_ioctl), +HL_IOCTL_DEF (HL_IOCTL_CS , hl_cs_ioctl), +HL_IOCTL_DEF (HL_IOCTL_WAIT_CS , hl_cs_wait_ioctl) };#define HL_CORE_IOCTL_COUNT ARRAY_SIZE(hl_ioctls)
drivers/misc/habanalabs/hw_queue.c +232 -0
250 lines not on this page · raw message
drivers/misc/habanalabs/memory.c new file +198 -0
154 lines not on this page · raw message
include/uapi/misc/habanalabs.h +157 -1
158 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.18.0
Part of the diff is not shown here. View the raw message.
PATCH v6 12/15 habanalabs: add virtual memory and MMU modules
In reply to Oded GabbayFrom: Omer Shpigelman <[email protected]>
This patch adds the Virtual Memory and MMU modules.
Goya has an internal MMU which provides process isolation on the internal DDR. The internal MMU also performs translations for transactions that go from Goya to the Host.
The driver is responsible for allocating and freeing memory on the DDR upon user request. It also provides an interface to map and unmap DDR and Host memory to the device address space.
The MMU in Goya supports 3-level and 4-level page tables. With 3-level, the size of each page is 2MB, while with 4-level the size of each page is 4KB.
In the DDR, the physical pages are always 2MB.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
- Use 2MB PAGE defines instead of the architecture huge page defines, because our MMU can map 2MB pages and not any arbitrary huge page size
- Added some text to the commit message
drivers/misc/habanalabs/Makefile | 2 +- drivers/misc/habanalabs/context.c | 19 +- drivers/misc/habanalabs/device.c | 20 +- drivers/misc/habanalabs/goya/goya.c | 395 +++++ drivers/misc/habanalabs/habanalabs.h | 189 +- drivers/misc/habanalabs/habanalabs_drv.c | 2 +- drivers/misc/habanalabs/habanalabs_ioctl.c | 3 +- .../include/hw_ip/mmu/mmu_general.h | 46 + .../habanalabs/include/hw_ip/mmu/mmu_v1_0.h | 15 + drivers/misc/habanalabs/memory.c | 1517 +++++++++++++++++ drivers/misc/habanalabs/mmu.c | 691 ++++++++ include/uapi/misc/habanalabs.h | 122 +- 12 files changed, 3013 insertions(+), 8 deletions(-) create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h create mode 100644 drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h create mode 100644 drivers/misc/habanalabs/mmu.c
drivers/misc/habanalabs/Makefile +1 -1
@@ -6,7 +6,7 @@ obj-m := habanalabs.o habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ - command_submission.o + command_submission.o mmu.o include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/context.c +18 -1
@@ -25,8 +25,10 @@ static void hl_ctx_fini(struct hl_ctx *ctx)for (i =0 ; i <HL_MAX_PENDING_CS ; i++)dma_fence_put (ctx->cs_pending[i]); -if (ctx->asid !=HL_KERNEL_ASID_ID ) +if (ctx->asid !=HL_KERNEL_ASID_ID ) { +hl_vm_ctx_fini (ctx);hl_asid_free (hdev, ctx->asid); + } }void hl_ctx_do_release (struct kref *ref)
@@ -96,6 +98,8 @@ void hl_ctx_free(struct hl_device *hdev, struct hl_ctx *ctx)int hl_ctx_init (struct hl_device *hdev,struct hl_ctx *ctx,bool is_kernel_ctx) { +int rc =0 ; + ctx->hdev = hdev;kref_init (&ctx->refcount);
@@ -113,9 +117,22 @@ int hl_ctx_init(struct hl_device *hdev, struct hl_ctx *ctx, bool is_kernel_ctx)dev_err (hdev->dev,"No free ASID, failed to create context\n" );return -ENOMEM ; } + + rc =hl_vm_ctx_init (ctx); +if (rc) { +dev_err (hdev->dev,"Failed to init mem ctx module\n" ); + rc = -ENOMEM ; + goto mem_ctx_err; + } }return 0 ; + +mem_ctx_err: +if (ctx->asid !=HL_KERNEL_ASID_ID ) +hl_asid_free (hdev, ctx->asid); + +return rc; }void hl_ctx_get (struct hl_device *hdev,struct hl_ctx *ctx)
drivers/misc/habanalabs/device.c +19 -1
@@ -615,8 +615,10 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset,/* Reset the H/W. It will be in idle state after this returns */ hdev->asic_funcs->hw_fini (hdev, hard_reset); -if (hard_reset) +if (hard_reset) { +hl_vm_fini (hdev);hl_eq_reset (hdev, &hdev->event_queue); + }/* Re-initialize PI,CI to 0 in all queues (hw queue, cq) */ hl_hw_queue_reset (hdev, hard_reset);
@@ -677,6 +679,13 @@ int hl_device_reset(struct hl_device *hdev, bool hard_reset, goto out_err; } + rc =hl_vm_init (hdev); +if (rc) { +dev_err (hdev->dev, +"Failed to init memory module after hard reset\n" ); + goto out_err; + } +hl_set_max_power (hdev, hdev->max_power); hdev->hard_reset_pending = false;
@@ -861,6 +870,13 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) hdev->asic_name, hdev->asic_prop.dram_size / 1024 / 1024 / 1024); + rc =hl_vm_init (hdev); +if (rc) { +dev_err (hdev->dev,"Failed to initialize memory module\n" ); + rc =0 ; + goto out_disabled; + } +/* * hl_hwmon_init must be called after device_late_init, because only * there we get the information from the device about which
@@ -977,6 +993,8 @@ void hl_device_fini(struct hl_device *hdev)/* Reset the H/W. It will be in idle state after this returns */ hdev->asic_funcs->hw_fini (hdev, true); +hl_vm_fini (hdev); +hl_eq_fini (hdev, &hdev->event_queue);for (i =0 ; i < hdev->asic_prop.completion_queues_count ; i++)
drivers/misc/habanalabs/goya/goya.c +395 -0
468 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs.h +187 -2
328 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +1 -1
@@ -188,7 +188,7 @@ int create_hdev(struct hl_device **dev, struct pci_dev *pdev, hdev->reset_on_lockup = reset_on_lockup;/* Parameters for bring-up - set them to defaults */ - hdev->mmu_enable =0 ; + hdev->mmu_enable =1 ; hdev->cpu_enable =1 ; hdev->reset_pcilink =0 ; hdev->cpu_queues_enable =1 ;
drivers/misc/habanalabs/habanalabs_ioctl.c +2 -1
@@ -18,7 +18,8 @@static const struct hl_ioctl_desc hl_ioctls[] = {HL_IOCTL_DEF (HL_IOCTL_CB , hl_cb_ioctl),HL_IOCTL_DEF (HL_IOCTL_CS , hl_cs_ioctl), -HL_IOCTL_DEF (HL_IOCTL_WAIT_CS , hl_cs_wait_ioctl) +HL_IOCTL_DEF (HL_IOCTL_WAIT_CS , hl_cs_wait_ioctl), +HL_IOCTL_DEF (HL_IOCTL_MEMORY , hl_mem_ioctl) };#define HL_CORE_IOCTL_COUNT ARRAY_SIZE(hl_ioctls)
drivers/misc/habanalabs/include/hw_ip/mmu/mmu_general.h new file +46 -0
46 lines not on this page · raw message
drivers/misc/habanalabs/include/hw_ip/mmu/mmu_v1_0.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 +* +* Copyright 2016-2018 HabanaLabs, Ltd. +* All Rights Reserved. +* +*/ + +#ifndef INCLUDE_MMU_V1_0_H_ +#define INCLUDE_MMU_V1_0_H_ + +#define MMU_HOP0_PA43_12 0x490004 +#define MMU_HOP0_PA49_44 0x490008 +#define MMU_ASID_BUSY 0x490000 + +#endif /* INCLUDE_MMU_V1_0_H_ */
drivers/misc/habanalabs/memory.c +1517 -0
1,530 lines not on this page · raw message
drivers/misc/habanalabs/mmu.c new file +691 -0
502 lines not on this page · raw message
include/uapi/misc/habanalabs.h +121 -1
122 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.18.0
Part of the diff is not shown here. View the raw message.
PATCH v6 13/15 habanalabs: implement INFO IOCTL
In reply to Oded GabbayThis patch implements the INFO IOCTL. That IOCTL is used by the user to query information that is relevant/needed by the user in order to submit deep learning jobs to Goya.
The information is divided into several categories, such as H/W IP, Events that happened, DDR usage and more.
--- drivers/misc/habanalabs/goya/goya.c | 6 + drivers/misc/habanalabs/habanalabs.h | 2 + drivers/misc/habanalabs/habanalabs_ioctl.c | 126 +++++++++++++++++++++ include/uapi/misc/habanalabs.h | 75 +++++++++++- 4 files changed, 208 insertions(+), 1 deletion(-)
drivers/misc/habanalabs/goya/goya.c +6 -0
@@ -5131,6 +5131,11 @@ static void goya_hw_queues_unlock(struct hl_device *hdev)spin_unlock (&goya->hw_queues_lock); } +static u32 goya_get_pci_id (struct hl_device *hdev) +{ +return hdev->pdev->device; +} +int goya_get_eeprom_data (struct hl_device *hdev,void *data,size_t max_size) {struct goya_device *goya = hdev->asic_specific;
@@ -5232,6 +5237,7 @@ static const struct hl_asic_funcs goya_funcs = { .soft_reset_late_init = goya_soft_reset_late_init, .hw_queues_lock = goya_hw_queues_lock, .hw_queues_unlock = goya_hw_queues_unlock, + .get_pci_id = goya_get_pci_id, .get_eeprom_data = goya_get_eeprom_data, .send_cpu_message = goya_send_cpu_message, .get_hw_state = goya_get_hw_state
drivers/misc/habanalabs/habanalabs.h +2 -0
@@ -470,6 +470,7 @@ enum hl_pll_frequency {* @soft_reset_late_init: perform certain actions needed after soft reset. * @hw_queues_lock: acquire H/W queues lock. * @hw_queues_unlock: release H/W queues lock. +* @get_pci_id: retrieve PCI ID. * @get_eeprom_data: retrieve EEPROM data from F/W. * @send_cpu_message: send buffer to ArmCP. * @get_hw_state: retrieve the H/W state
@@ -539,6 +540,7 @@ struct hl_asic_funcs {int (*soft_reset_late_init)(struct hl_device *hdev);void (*hw_queues_lock)(struct hl_device *hdev);void (*hw_queues_unlock)(struct hl_device *hdev); +u32 (*get_pci_id)(struct hl_device *hdev);int (*get_eeprom_data)(struct hl_device *hdev,void *data,size_t max_size); int (*send_cpu_message)(struct hl_device *hdev, u32 *msg,
drivers/misc/habanalabs/habanalabs_ioctl.c +126 -0
136 lines not on this page · raw message
include/uapi/misc/habanalabs.h +74 -1
93 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v6 14/15 habanalabs: add debugfs support
In reply to Oded GabbayThis patch adds debugfs support to the driver. It allows the user-space to display information that is contained in the internal structures of the driver, such as:
- active command submissions
- active user virtual memory mappings
- number of allocated command buffers
It also enables the user to perform reads and writes through Goya's PCI bars.
---
Changes in v6:
- Only include headers needed for compilation
- Update copyright years
.../ABI/testing/debugfs-driver-habanalabs | 126 ++ drivers/misc/habanalabs/Makefile | 2 + drivers/misc/habanalabs/command_buffer.c | 4 + drivers/misc/habanalabs/command_submission.c | 12 + drivers/misc/habanalabs/debugfs.c | 1072 +++++++++++++++++ drivers/misc/habanalabs/device.c | 6 + drivers/misc/habanalabs/goya/goya.c | 108 ++ drivers/misc/habanalabs/goya/goyaP.h | 5 + drivers/misc/habanalabs/habanalabs.h | 189 +++ drivers/misc/habanalabs/habanalabs_drv.c | 16 +- drivers/misc/habanalabs/memory.c | 8 + 11 files changed, 1546 insertions(+), 2 deletions(-) create mode 100644 Documentation/ABI/testing/debugfs-driver-habanalabs create mode 100644 drivers/misc/habanalabs/debugfs.c
Documentation/ABI/testing/debugfs-driver-habanalabs new file +126 -0
@@ -0,0 +1,126 @@ +What: /sys/kernel/debug/habanalabs/hl<n>/addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the device address to be used for read or write through + PCI bar. The acceptable value is a string that starts with "0x" + +What: /sys/kernel/debug/habanalabs/hl<n>/command_buffers +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently allocated + command buffers + +What: /sys/kernel/debug/habanalabs/hl<n>/command_submission +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently active + command submissions + +What: /sys/kernel/debug/habanalabs/hl<n>/command_submission_jobs +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with detailed information about each JOB (CB) of + each active command submission + +What: /sys/kernel/debug/habanalabs/hl<n>/data32 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Allows the root user to read or write directly through the + device's PCI bar. Writing to this file generates a write + transaction while reading from the file generates a read + transcation. This custom interface is needed (instead of using + the generic Linux user-space PCI mapping) because the DDR bar + is very small compared to the DDR memory and only the driver can + move the bar before and after the transaction + +What: /sys/kernel/debug/habanalabs/hl<n>/device +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Enables the root user to set the device to specific state. + Valid values are "disable", "enable", "suspend", "resume". + User can read this property to see the valid values + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_addr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C device address for I2C transaction that is generated + by the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_bus +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C bus address for I2C transaction that is generated by + the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_data +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Triggers an I2C transaction that is generated by the device's + CPU. Writing to this file generates a write transaction while + reading from the file generates a read transcation + +What: /sys/kernel/debug/habanalabs/hl<n>/i2c_reg +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets I2C register id for I2C transaction that is generated by + the device's CPU + +What: /sys/kernel/debug/habanalabs/hl<n>/led0 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the first S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/led1 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the second S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/led2 +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the state of the third S/W led on the device + +What: /sys/kernel/debug/habanalabs/hl<n>/mmu +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays the hop values and physical address for a given ASID + and virtual address. The user should write the ASID and VA into + the file and then read the file to get the result. + e.g. to display info about VA 0x1000 for ASID 1 you need to do: + echo "1 0x1000" > /sys/kernel/debug/habanalabs/hl0/mmu + +What: /sys/kernel/debug/habanalabs/hl<n>/set_power_state +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Sets the PCI power state. Valid values are "1" for D0 and "2" + for D3Hot + +What: /sys/kernel/debug/habanalabs/hl<n>/userptr +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about the currently user + pointers (user virtual addresses) that are pinned and mapped + to DMA addresses + +What: /sys/kernel/debug/habanalabs/hl<n>/vm +Date: Jan 2019 +KernelVersion: 5.1 +Contact: [email protected] +Description: Displays a list with information about all the active virtual + address mappings per ASID
drivers/misc/habanalabs/Makefile +2 -0
@@ -8,5 +8,7 @@ habanalabs-y := habanalabs_drv.o device.o context.o asid.o habanalabs_ioctl.o \ command_buffer.o hw_queue.o irq.o sysfs.o hwmon.o memory.o \ command_submission.o mmu.o +habanalabs-$(CONFIG_DEBUG_FS) += debugfs.o + include $(src)/goya/Makefile habanalabs-y += $(HL_GOYA_FILES)
drivers/misc/habanalabs/command_buffer.c +4 -0
@@ -38,6 +38,8 @@ static void cb_release(struct kref *ref) cb = container_of(ref, struct hl_cb, refcount); hdev = cb->hdev; +hl_debugfs_remove_cb (cb); +cb_do_release (hdev, cb); }
@@ -163,6 +165,8 @@ int hl_cb_create(struct hl_device *hdev, struct hl_cb_mgr *mgr, *handle = cb->id |HL_MMAP_CB_MASK ; *handle <<=PAGE_SHIFT ; +hl_debugfs_add_cb (cb); +return 0 ; release_cb:
drivers/misc/habanalabs/command_submission.c +12 -0
@@ -149,6 +149,8 @@ static void free_job(struct hl_device *hdev, struct hl_cs_job *job)list_del (&job->cs_node);spin_unlock (&cs->job_lock); +hl_debugfs_remove_job (hdev, job); +if (job->ext_queue)cs_put (cs);
@@ -212,6 +214,12 @@ static void cs_do_release(struct kref *ref) } } +/* +* Must be called before hl_ctx_put because inside we use ctx to get +* the device +*/ +hl_debugfs_remove_cs (cs); +hl_ctx_put (cs->ctx);if (cs->timedout)
@@ -480,6 +488,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks, *cs_seq = cs->sequence; +hl_debugfs_add_cs (cs); +/* Validate ALL the CS chunks before submitting the CS */ for (i =0 , parse_cnt =0 ; i < num_chunks ; i++, parse_cnt++) {struct hl_cs_chunk *chunk = &cs_chunk_array[i];
@@ -528,6 +538,8 @@ static int _hl_cs_ioctl(struct hl_fpriv *hpriv, void __user *chunks,if (job->ext_queue)cs_get (cs); +hl_debugfs_add_job (hdev, job); + rc =cs_parser (hpriv, job);if (rc) { dev_err(hdev->dev,
drivers/misc/habanalabs/debugfs.c new file +1072 -0
1,072 lines not on this page · raw message
drivers/misc/habanalabs/device.c +6 -0
@@ -30,6 +30,8 @@ static void hpriv_release(struct kref *ref)put_pid (hpriv->taskpid); +hl_debugfs_remove_file (hpriv); +mutex_destroy (&hpriv->restore_phase_mutex);kfree (hpriv);
@@ -834,6 +836,8 @@ int hl_device_init(struct hl_device *hdev, struct class *hclass) goto free_cb_pool; } +hl_debugfs_add_device (hdev); +if (hdev->asic_funcs->get_hw_state (hdev) ==HL_DEVICE_HW_STATE_DIRTY ) {dev_info (hdev->dev,"H/W state is dirty, must reset before initializing\n" );
@@ -972,6 +976,8 @@ void hl_device_fini(struct hl_device *hdev)device_late_fini (hdev); +hl_debugfs_remove_device (hdev); +hl_sysfs_fini (hdev);/*
drivers/misc/habanalabs/goya/goya.c +108 -0
144 lines not on this page · raw message
drivers/misc/habanalabs/goya/goyaP.h +5 -0
@@ -165,6 +165,10 @@ struct goya_device {u32 hw_cap_initialized; }; +int goya_debugfs_i2c_read (struct hl_device *hdev,u8 i2c_bus, +u8 i2c_addr,u8 i2c_reg,u32 *val); +int goya_debugfs_i2c_write (struct hl_device *hdev,u8 i2c_bus, +u8 i2c_addr,u8 i2c_reg,u32 val);int goya_test_cpu_queue (struct hl_device *hdev);int goya_send_cpu_message (struct hl_device *hdev,u32 *msg,u16 len,u32 timeout,long *result);
@@ -175,6 +179,7 @@ long goya_get_fan_speed(struct hl_device *hdev, int sensor_index, u32 attr);long goya_get_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr);void goya_set_pwm_info (struct hl_device *hdev,int sensor_index,u32 attr,long value); +void goya_debugfs_led_set (struct hl_device *hdev,u8 led,u8 state);void goya_set_pll_profile (struct hl_device *hdev,enum hl_pll_frequency freq);void goya_add_device_attr (struct hl_device *hdev,struct attribute_group *dev_attr_grp);
drivers/misc/habanalabs/habanalabs.h +189 -0
284 lines not on this page · raw message
drivers/misc/habanalabs/habanalabs_drv.c +14 -2
@@ -146,6 +146,8 @@ int hl_device_open(struct inode *inode, struct file *filp)*/ hl_device_set_frequency (hdev,PLL_HIGH ); +hl_debugfs_add_file (hpriv); +return 0 ; out_err:
@@ -413,17 +415,20 @@ static int __init hl_init(void) goto remove_major; } +hl_debugfs_init (); + rc =pci_register_driver (&hl_pci_driver);if (rc) {pr_err ("failed to register pci device\n" ); - goto remove_class; + goto remove_debugfs; }pr_debug ("driver loaded\n" );return 0 ; -remove_class: +remove_debugfs: +hl_debugfs_fini ();class_destroy (hl_class); remove_major:unregister_chrdev_region (MKDEV (hl_major,0 ),HL_MAX_MINORS );
@@ -437,6 +442,13 @@ static void __exit hl_exit(void) {pci_unregister_driver (&hl_pci_driver); +/* +* Removing debugfs must be after all devices or simulator devices +* have been removed because otherwise we get a bug in the +* debugfs module for referencing NULL objects +*/ +hl_debugfs_fini (); +class_destroy (hl_class);unregister_chrdev_region (MKDEV (hl_major,0 ),HL_MAX_MINORS );
drivers/misc/habanalabs/memory.c +8 -0
@@ -1290,6 +1290,8 @@ int hl_pin_host_memory(struct hl_device *hdev, u64 addr, u32 size, goto free_sgt; } +hl_debugfs_add_userptr (hdev, userptr); +return 0 ; free_sgt:
@@ -1315,6 +1317,8 @@ int hl_unpin_host_memory(struct hl_device *hdev, struct hl_userptr *userptr) {struct page **pages; +hl_debugfs_remove_userptr (hdev, userptr); +if (userptr->dma_mapped) hdev->asic_funcs->hl_dma_unmap_sg(hdev, userptr->sgt->sgl,
@@ -1476,6 +1480,8 @@ int hl_vm_ctx_init_with_ranges(struct hl_ctx *ctx, u64 host_range_start, goto dram_vm_err; } +hl_debugfs_add_ctx_mem_hash (hdev, ctx); +return 0 ; dram_vm_err:
@@ -1598,6 +1604,8 @@ void hl_vm_ctx_fini(struct hl_ctx *ctx)struct hlist_node *tmp_node;int i; +hl_debugfs_remove_ctx_mem_hash (hdev, ctx); +if (!hash_empty (ctx->mem_hash))dev_notice (hdev->dev,"ctx is freed while it has va in use\n" );
Signature · 1 line
2.18.0
PATCH v6 15/15 Update MAINTAINERS and CREDITS with habanalabs info
In reply to Oded GabbayThe habanalabs driver was written from scratch from the very first days of Habana and is maintained by Oded Gabbay.
--- CREDITS | 2 +- MAINTAINERS | 9 +++++++++ 2 files changed, 10 insertions(+), 1 deletion(-)
CREDITS +1 -1
@@ -1222,7 +1222,7 @@ S: Brazil N: Oded Gabbay E: [email protected] -D: AMD KFD maintainer +D: HabanaLabs and AMD KFD maintainer S: 12 Shraga Raphaeli S: Petah-Tikva, 4906418 S: Israel
MAINTAINERS +9 -0
@@ -6647,6 +6647,15 @@ F: drivers/clocksource/h8300_*.c F: drivers/clk/h8300/ F: drivers/irqchip/irq-renesas-h8*.c +HABANALABS PCI DRIVER +M: Oded Gabbay <[email protected]> +T: git https://github.com/HabanaAI/linux.git +S: Supported +F: drivers/misc/habanalabs/ +F: include/uapi/misc/habanalabs.h +F: Documentation/ABI/testing/sysfs-driver-habanalabs +F: Documentation/ABI/testing/debugfs-driver-habanalabs + HACKRF MEDIA DRIVER M: Antti Palosaari <[email protected]> L: [email protected]
Signature · 1 line
2.18.0