Skip to contentSkip to conversation

Apple M1 NVMe storage support

PATCH Superseded Mark Kettenis 3 participants 33 messages Jan 14, 2022 – Jan 22, 2022

Subsystems (6)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Apple M1 NVMe storage support

This adds support for the (rather quirky) NVMe storage controller integrated on Apple SoCs. This makes it possible to boot from the NVMe storage that is present on all the M1 machines that Apple has released so far.

The series has been designed to have as little impact on the existing NVMe support as possible. It splits out the PCIe-specific bits in its own file/driver and adds a platform driver for the Apple controller. This platform driver handles all the quirkiness through driver ops. The existing logic is retained when the driver ops aren't set. I've tested this on a firefly-rk3399 board with:

nvme0 at pci1 dev 0 function 0 "Silicon Motion SM2260 NVMe" rev 0x03: msix, NVMe 1.2
nvme0: ADATA SX8000NP, firmware C2.2.1, serial 2H1220011170

(for those not familliar with OpenBSD, that's an ADATA SX8000NP NVMe M.2 SSD with a Silicon Motion controller)

This patch series depends on the Apple M1 power management controller support series:

https://patchwork.ozlabs.org/project/uboot/list/?series=280359

Mark Kettenis (8):
nvme: Split out PCI support
mailbox: apple: Add driver for Apple IOP mailbox
arm: apple: Add RTKit support
nvme: Introduce driver ops
nvme: Add shutdown function
power: domain: apple: Add reset support
nvme: apple: Add driver for Apple NVMe storage controller
configs: apple: Add NVMe boot target

 arch/arm/Kconfig                              |   2 +
 arch/arm/include/asm/arch-apple/rtkit.h       |  11 +
 arch/arm/mach-apple/Makefile                  |   1 +
 arch/arm/mach-apple/rtkit.c                   | 231 +++++++++++++++++
 configs/apple_m1_defconfig                    |   1 +
 configs/clearfog_gt_8k_defconfig              |   2 +-
 configs/firefly-rk3399_defconfig              |   2 +-
 configs/khadas-vim3_android_ab_defconfig      |   2 +-
 configs/khadas-vim3_android_defconfig         |   2 +-
 configs/khadas-vim3_defconfig                 |   2 +-
 configs/khadas-vim3l_android_ab_defconfig     |   2 +-
 configs/khadas-vim3l_android_defconfig        |   2 +-
 configs/khadas-vim3l_defconfig                |   2 +-
 configs/kontron_sl28_defconfig                |   2 +-
 configs/ls1012afrdm_qspi_defconfig            |   2 +-
 configs/ls1012afrdm_tfa_defconfig             |   2 +-
 .../ls1012afrwy_qspi_SECURE_BOOT_defconfig    |   2 +-
 configs/ls1012afrwy_qspi_defconfig            |   2 +-
 configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig |   2 +-
 configs/ls1012afrwy_tfa_defconfig             |   2 +-
 configs/ls1012aqds_qspi_defconfig             |   2 +-
 configs/ls1012aqds_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1012aqds_tfa_defconfig              |   2 +-
 configs/ls1012ardb_qspi_SECURE_BOOT_defconfig |   2 +-
 configs/ls1012ardb_qspi_defconfig             |   2 +-
 configs/ls1012ardb_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1012ardb_tfa_defconfig              |   2 +-
 configs/ls1021aiot_qspi_defconfig             |   2 +-
 configs/ls1021aiot_sdcard_defconfig           |   2 +-
 configs/ls1021aqds_ddr4_nor_defconfig         |   2 +-
 configs/ls1021aqds_ddr4_nor_lpuart_defconfig  |   2 +-
 configs/ls1021aqds_nand_defconfig             |   2 +-
 configs/ls1021aqds_nor_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1021aqds_nor_defconfig              |   2 +-
 configs/ls1021aqds_nor_lpuart_defconfig       |   2 +-
 configs/ls1021aqds_qspi_defconfig             |   2 +-
 configs/ls1021aqds_sdcard_ifc_defconfig       |   2 +-
 configs/ls1021aqds_sdcard_qspi_defconfig      |   2 +-
 configs/ls1021atsn_qspi_defconfig             |   2 +-
 configs/ls1021atsn_sdcard_defconfig           |   2 +-
 configs/ls1021atwr_nor_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1021atwr_nor_defconfig              |   2 +-
 configs/ls1021atwr_nor_lpuart_defconfig       |   2 +-
 configs/ls1021atwr_qspi_defconfig             |   2 +-
 configs/ls1021atwr_sdcard_ifc_defconfig       |   2 +-
 configs/ls1021atwr_sdcard_qspi_defconfig      |   2 +-
 configs/ls1028aqds_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1028aqds_tfa_defconfig              |   2 +-
 configs/ls1028aqds_tfa_lpuart_defconfig       |   2 +-
 configs/ls1028ardb_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1028ardb_tfa_defconfig              |   2 +-
 configs/ls1043aqds_defconfig                  |   2 +-
 configs/ls1043aqds_lpuart_defconfig           |   2 +-
 configs/ls1043aqds_nand_defconfig             |   2 +-
 configs/ls1043aqds_nor_ddr3_defconfig         |   2 +-
 configs/ls1043aqds_qspi_defconfig             |   2 +-
 configs/ls1043aqds_sdcard_ifc_defconfig       |   2 +-
 configs/ls1043aqds_sdcard_qspi_defconfig      |   2 +-
 configs/ls1043aqds_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1043aqds_tfa_defconfig              |   2 +-
 configs/ls1043ardb_SECURE_BOOT_defconfig      |   2 +-
 configs/ls1043ardb_defconfig                  |   2 +-
 configs/ls1043ardb_nand_SECURE_BOOT_defconfig |   2 +-
 configs/ls1043ardb_nand_defconfig             |   2 +-
 configs/ls1043ardb_sdcard_defconfig           |   2 +-
 configs/ls1043ardb_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1043ardb_tfa_defconfig              |   2 +-
 configs/ls1046afrwy_tfa_defconfig             |   2 +-
 configs/ls1046aqds_SECURE_BOOT_defconfig      |   2 +-
 configs/ls1046aqds_defconfig                  |   2 +-
 configs/ls1046aqds_lpuart_defconfig           |   2 +-
 configs/ls1046aqds_nand_defconfig             |   2 +-
 configs/ls1046aqds_qspi_defconfig             |   2 +-
 configs/ls1046aqds_sdcard_ifc_defconfig       |   2 +-
 configs/ls1046aqds_sdcard_qspi_defconfig      |   2 +-
 configs/ls1046aqds_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1046aqds_tfa_defconfig              |   2 +-
 configs/ls1046ardb_emmc_defconfig             |   2 +-
 configs/ls1046ardb_qspi_SECURE_BOOT_defconfig |   2 +-
 configs/ls1046ardb_qspi_defconfig             |   2 +-
 configs/ls1046ardb_qspi_spl_defconfig         |   2 +-
 configs/ls1046ardb_sdcard_defconfig           |   2 +-
 configs/ls1046ardb_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1046ardb_tfa_defconfig              |   2 +-
 configs/ls1088aqds_defconfig                  |   2 +-
 configs/ls1088aqds_qspi_SECURE_BOOT_defconfig |   2 +-
 configs/ls1088aqds_qspi_defconfig             |   2 +-
 configs/ls1088aqds_sdcard_ifc_defconfig       |   2 +-
 configs/ls1088aqds_sdcard_qspi_defconfig      |   2 +-
 configs/ls1088aqds_tfa_defconfig              |   2 +-
 configs/ls1088ardb_qspi_SECURE_BOOT_defconfig |   2 +-
 configs/ls1088ardb_qspi_defconfig             |   2 +-
 configs/ls1088ardb_sdcard_qspi_defconfig      |   2 +-
 configs/ls1088ardb_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls1088ardb_tfa_defconfig              |   2 +-
 configs/ls2080aqds_SECURE_BOOT_defconfig      |   2 +-
 configs/ls2080aqds_defconfig                  |   2 +-
 configs/ls2080aqds_nand_defconfig             |   2 +-
 configs/ls2080aqds_qspi_defconfig             |   2 +-
 configs/ls2080aqds_sdcard_defconfig           |   2 +-
 configs/ls2080ardb_SECURE_BOOT_defconfig      |   2 +-
 configs/ls2080ardb_defconfig                  |   2 +-
 configs/ls2080ardb_nand_defconfig             |   2 +-
 configs/ls2081ardb_defconfig                  |   2 +-
 configs/ls2088aqds_tfa_defconfig              |   2 +-
 configs/ls2088ardb_qspi_SECURE_BOOT_defconfig |   2 +-
 configs/ls2088ardb_qspi_defconfig             |   2 +-
 configs/ls2088ardb_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/ls2088ardb_tfa_defconfig              |   2 +-
 configs/lx2160aqds_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/lx2160aqds_tfa_defconfig              |   2 +-
 configs/lx2160ardb_tfa_SECURE_BOOT_defconfig  |   2 +-
 configs/lx2160ardb_tfa_defconfig              |   2 +-
 configs/lx2160ardb_tfa_stmm_defconfig         |   2 +-
 configs/mvebu_crb_cn9130_defconfig            |   2 +-
 configs/mvebu_db_armada8k_defconfig           |   2 +-
 configs/mvebu_db_cn9130_defconfig             |   2 +-
 configs/mvebu_espressobin-88f3720_defconfig   |   2 +-
 configs/mvebu_mcbin-88f8040_defconfig         |   2 +-
 configs/mvebu_puzzle-m801-88f8040_defconfig   |   2 +-
 configs/nanopc-t4-rk3399_defconfig            |   2 +-
 configs/octeontx2_96xx_defconfig              |   2 +-
 configs/octeontx_81xx_defconfig               |   2 +-
 configs/octeontx_83xx_defconfig               |   2 +-
 configs/p3450-0000_defconfig                  |   2 +-
 configs/pinebook-pro-rk3399_defconfig         |   2 +-
 configs/qemu-x86_64_defconfig                 |   2 +-
 configs/qemu-x86_defconfig                    |   2 +-
 configs/qemu_arm64_defconfig                  |   2 +-
 configs/qemu_arm_defconfig                    |   2 +-
 configs/rcar3_salvator-x_defconfig            |   2 +-
 configs/roc-pc-mezzanine-rk3399_defconfig     |   2 +-
 configs/rock-pi-4-rk3399_defconfig            |   2 +-
 configs/rock-pi-4c-rk3399_defconfig           |   2 +-
 configs/rock-pi-n10-rk3399pro_defconfig       |   2 +-
 configs/rock960-rk3399_defconfig              |   2 +-
 configs/rockpro64-rk3399_defconfig            |   2 +-
 configs/sandbox64_defconfig                   |   2 +-
 configs/sandbox_defconfig                     |   2 +-
 configs/sandbox_flattree_defconfig            |   2 +-
 configs/sandbox_noinst_defconfig              |   2 +-
 configs/sandbox_spl_defconfig                 |   2 +-
 configs/sifive_unmatched_defconfig            |   2 +-
 configs/synquacer_developerbox_defconfig      |   2 +-
 configs/turris_mox_defconfig                  |   2 +-
 configs/turris_omnia_defconfig                |   2 +-
 doc/develop/driver-model/nvme.rst             |   1 +
 drivers/mailbox/Kconfig                       |  11 +
 drivers/mailbox/Makefile                      |   1 +
 drivers/mailbox/apple-mbox.c                  |  92 +++++++
 drivers/nvme/Kconfig                          |  21 +-
 drivers/nvme/Makefile                         |   2 +
 drivers/nvme/nvme.c                           |  86 ++-----
 drivers/nvme/nvme.h                           |  37 +++
 drivers/nvme/nvme_apple.c                     | 233 ++++++++++++++++++
 drivers/nvme/nvme_pci.c                       |  49 ++++
 drivers/power/domain/apple-pmgr.c             |  73 +++++-
 include/configs/apple.h                       |   7 +
 include/linux/apple-mailbox.h                 |  19 ++
 159 files changed, 957 insertions(+), 203 deletions(-)
 create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
 create mode 100644 arch/arm/mach-apple/rtkit.c
 create mode 100644 drivers/mailbox/apple-mbox.c
 create mode 100644 drivers/nvme/nvme_apple.c
 create mode 100644 drivers/nvme/nvme_pci.c
 create mode 100644 include/linux/apple-mailbox.h
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/8 nvme: Split out PCI support

Apple SoCs have an integrated NVMe controller that isn't connected over a PCIe bus. In preparation for adding support for this NVMe controller, split out the PCI support into its own file. This file is selected through a new CONFIG_NVME_PCI Kconfig option, so do a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.

---
 configs/clearfog_gt_8k_defconfig              |  2 +-
 configs/firefly-rk3399_defconfig              |  2 +-
 configs/khadas-vim3_android_ab_defconfig      |  2 +-
 configs/khadas-vim3_android_defconfig         |  2 +-
 configs/khadas-vim3_defconfig                 |  2 +-
 configs/khadas-vim3l_android_ab_defconfig     |  2 +-
 configs/khadas-vim3l_android_defconfig        |  2 +-
 configs/khadas-vim3l_defconfig                |  2 +-
 configs/kontron_sl28_defconfig                |  2 +-
 configs/ls1012afrdm_qspi_defconfig            |  2 +-
 configs/ls1012afrdm_tfa_defconfig             |  2 +-
 .../ls1012afrwy_qspi_SECURE_BOOT_defconfig    |  2 +-
 configs/ls1012afrwy_qspi_defconfig            |  2 +-
 configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012afrwy_tfa_defconfig             |  2 +-
 configs/ls1012aqds_qspi_defconfig             |  2 +-
 configs/ls1012aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012aqds_tfa_defconfig              |  2 +-
 configs/ls1012ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012ardb_qspi_defconfig             |  2 +-
 configs/ls1012ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012ardb_tfa_defconfig              |  2 +-
 configs/ls1021aiot_qspi_defconfig             |  2 +-
 configs/ls1021aiot_sdcard_defconfig           |  2 +-
 configs/ls1021aqds_ddr4_nor_defconfig         |  2 +-
 configs/ls1021aqds_ddr4_nor_lpuart_defconfig  |  2 +-
 configs/ls1021aqds_nand_defconfig             |  2 +-
 configs/ls1021aqds_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021aqds_nor_defconfig              |  2 +-
 configs/ls1021aqds_nor_lpuart_defconfig       |  2 +-
 configs/ls1021aqds_qspi_defconfig             |  2 +-
 configs/ls1021aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1021atsn_qspi_defconfig             |  2 +-
 configs/ls1021atsn_sdcard_defconfig           |  2 +-
 configs/ls1021atwr_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021atwr_nor_defconfig              |  2 +-
 configs/ls1021atwr_nor_lpuart_defconfig       |  2 +-
 configs/ls1021atwr_qspi_defconfig             |  2 +-
 configs/ls1021atwr_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021atwr_sdcard_qspi_defconfig      |  2 +-
 configs/ls1028aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028aqds_tfa_defconfig              |  2 +-
 configs/ls1028aqds_tfa_lpuart_defconfig       |  2 +-
 configs/ls1028ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028ardb_tfa_defconfig              |  2 +-
 configs/ls1043aqds_defconfig                  |  2 +-
 configs/ls1043aqds_lpuart_defconfig           |  2 +-
 configs/ls1043aqds_nand_defconfig             |  2 +-
 configs/ls1043aqds_nor_ddr3_defconfig         |  2 +-
 configs/ls1043aqds_qspi_defconfig             |  2 +-
 configs/ls1043aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1043aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1043aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043aqds_tfa_defconfig              |  2 +-
 configs/ls1043ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1043ardb_defconfig                  |  2 +-
 configs/ls1043ardb_nand_SECURE_BOOT_defconfig |  2 +-
 configs/ls1043ardb_nand_defconfig             |  2 +-
 configs/ls1043ardb_sdcard_defconfig           |  2 +-
 configs/ls1043ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043ardb_tfa_defconfig              |  2 +-
 configs/ls1046afrwy_tfa_defconfig             |  2 +-
 configs/ls1046aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1046aqds_defconfig                  |  2 +-
 configs/ls1046aqds_lpuart_defconfig           |  2 +-
 configs/ls1046aqds_nand_defconfig             |  2 +-
 configs/ls1046aqds_qspi_defconfig             |  2 +-
 configs/ls1046aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1046aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1046aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046aqds_tfa_defconfig              |  2 +-
 configs/ls1046ardb_emmc_defconfig             |  2 +-
 configs/ls1046ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1046ardb_qspi_defconfig             |  2 +-
 configs/ls1046ardb_qspi_spl_defconfig         |  2 +-
 configs/ls1046ardb_sdcard_defconfig           |  2 +-
 configs/ls1046ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046ardb_tfa_defconfig              |  2 +-
 configs/ls1088aqds_defconfig                  |  2 +-
 configs/ls1088aqds_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088aqds_qspi_defconfig             |  2 +-
 configs/ls1088aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1088aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088aqds_tfa_defconfig              |  2 +-
 configs/ls1088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088ardb_qspi_defconfig             |  2 +-
 configs/ls1088ardb_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1088ardb_tfa_defconfig              |  2 +-
 configs/ls2080aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080aqds_defconfig                  |  2 +-
 configs/ls2080aqds_nand_defconfig             |  2 +-
 configs/ls2080aqds_qspi_defconfig             |  2 +-
 configs/ls2080aqds_sdcard_defconfig           |  2 +-
 configs/ls2080ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080ardb_defconfig                  |  2 +-
 configs/ls2080ardb_nand_defconfig             |  2 +-
 configs/ls2081ardb_defconfig                  |  2 +-
 configs/ls2088aqds_tfa_defconfig              |  2 +-
 configs/ls2088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls2088ardb_qspi_defconfig             |  2 +-
 configs/ls2088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls2088ardb_tfa_defconfig              |  2 +-
 configs/lx2160aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160aqds_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160ardb_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_stmm_defconfig         |  2 +-
 configs/mvebu_crb_cn9130_defconfig            |  2 +-
 configs/mvebu_db_armada8k_defconfig           |  2 +-
 configs/mvebu_db_cn9130_defconfig             |  2 +-
 configs/mvebu_espressobin-88f3720_defconfig   |  2 +-
 configs/mvebu_mcbin-88f8040_defconfig         |  2 +-
 configs/mvebu_puzzle-m801-88f8040_defconfig   |  2 +-
 configs/nanopc-t4-rk3399_defconfig            |  2 +-
 configs/octeontx2_96xx_defconfig              |  2 +-
 configs/octeontx_81xx_defconfig               |  2 +-
 configs/octeontx_83xx_defconfig               |  2 +-
 configs/p3450-0000_defconfig                  |  2 +-
 configs/pinebook-pro-rk3399_defconfig         |  2 +-
 configs/qemu-x86_64_defconfig                 |  2 +-
 configs/qemu-x86_defconfig                    |  2 +-
 configs/qemu_arm64_defconfig                  |  2 +-
 configs/qemu_arm_defconfig                    |  2 +-
 configs/rcar3_salvator-x_defconfig            |  2 +-
 configs/roc-pc-mezzanine-rk3399_defconfig     |  2 +-
 configs/rock-pi-4-rk3399_defconfig            |  2 +-
 configs/rock-pi-4c-rk3399_defconfig           |  2 +-
 configs/rock-pi-n10-rk3399pro_defconfig       |  2 +-
 configs/rock960-rk3399_defconfig              |  2 +-
 configs/rockpro64-rk3399_defconfig            |  2 +-
 configs/sandbox64_defconfig                   |  2 +-
 configs/sandbox_defconfig                     |  2 +-
 configs/sandbox_flattree_defconfig            |  2 +-
 configs/sandbox_noinst_defconfig              |  2 +-
 configs/sandbox_spl_defconfig                 |  2 +-
 configs/sifive_unmatched_defconfig            |  2 +-
 configs/synquacer_developerbox_defconfig      |  2 +-
 configs/turris_mox_defconfig                  |  2 +-
 configs/turris_omnia_defconfig                |  2 +-
 doc/develop/driver-model/nvme.rst             |  1 +
 drivers/nvme/Kconfig                          | 10 +++-
 drivers/nvme/Makefile                         |  1 +
 drivers/nvme/nvme.c                           | 38 ++------------
 drivers/nvme/nvme.h                           |  3 ++
 drivers/nvme/nvme_pci.c                       | 49 +++++++++++++++++++
 147 files changed, 207 insertions(+), 177 deletions(-)
 create mode 100644 drivers/nvme/nvme_pci.c
configs/clearfog_gt_8k_defconfig +1 -1
index 6e344c9ce3..6dcd70fe12 100644
@@ -56,7 +56,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MVEBU=y
 CONFIG_PHY=y
configs/firefly-rk3399_defconfig +1 -1
index d576b5c38d..fe1c019f1d 100644
@@ -41,7 +41,7 @@ CONFIG_SF_DEFAULT_SPEED=20000000
 CONFIG_DM_ETH=y
 CONFIG_ETH_DESIGNWARE=y
 CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PMIC_RK8XX=y
 CONFIG_REGULATOR_PWM=y
configs/khadas-vim3_android_ab_defconfig +1 -1
index 88325c37db..7d3e2b8fdd 100644
@@ -62,7 +62,7 @@ CONFIG_DM_MDIO=y
 CONFIG_DM_MDIO_MUX=y
 CONFIG_ETH_DESIGNWARE_MESON8B=y
 CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MESON=y
 CONFIG_MESON_G12A_USB_PHY=y
configs/khadas-vim3_android_defconfig +1 -1
index 9305a54b1b..3f5992d5b6 100644
@@ -60,7 +60,7 @@ CONFIG_DM_MDIO=y
 CONFIG_DM_MDIO_MUX=y
 CONFIG_ETH_DESIGNWARE_MESON8B=y
 CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MESON=y
 CONFIG_MESON_G12A_USB_PHY=y
configs/khadas-vim3_defconfig +1 -1
index 65050efd3f..5e868ce2df 100644
@@ -47,7 +47,7 @@ CONFIG_DM_MDIO=y
 CONFIG_DM_MDIO_MUX=y
 CONFIG_ETH_DESIGNWARE_MESON8B=y
 CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MESON=y
 CONFIG_MESON_G12A_USB_PHY=y
configs/khadas-vim3l_android_ab_defconfig +1 -1
index 608b54dca2..4f4c55c8ea 100644
@@ -62,7 +62,7 @@ CONFIG_DM_MDIO=y
 CONFIG_DM_MDIO_MUX=y
 CONFIG_ETH_DESIGNWARE_MESON8B=y
 CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MESON=y
 CONFIG_MESON_G12A_USB_PHY=y
configs/khadas-vim3l_android_defconfig +1 -1
index 5eed79b263..fdee62d801 100644
@@ -60,7 +60,7 @@ CONFIG_DM_MDIO=y
 CONFIG_DM_MDIO_MUX=y
 CONFIG_ETH_DESIGNWARE_MESON8B=y
 CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MESON=y
 CONFIG_MESON_G12A_USB_PHY=y
configs/khadas-vim3l_defconfig +1 -1
index b9162c9f1e..2b391646c5 100644
@@ -47,7 +47,7 @@ CONFIG_DM_MDIO=y
 CONFIG_DM_MDIO_MUX=y
 CONFIG_ETH_DESIGNWARE_MESON8B=y
 CONFIG_MDIO_MUX_MESON_G12A=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MESON=y
 CONFIG_MESON_G12A_USB_PHY=y
configs/kontron_sl28_defconfig +1 -1
index 31a1083b0a..087dcb1d0e 100644
@@ -83,7 +83,7 @@ CONFIG_DM_DSA=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MSCC_FELIX_SWITCH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_RTC_RV8803=y
configs/ls1012afrdm_qspi_defconfig +1 -1
index ad3b7bff42..0b3964e9d8 100644
@@ -50,7 +50,7 @@ CONFIG_SPI_FLASH_SPANSION=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012afrdm_tfa_defconfig +1 -1
index 3317a0592b..93514082f2 100644
@@ -50,7 +50,7 @@ CONFIG_SPI_FLASH_SPANSION=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012afrwy_qspi_SECURE_BOOT_defconfig +1 -1
index cb8b288c39..8ecef720b0 100644
@@ -49,7 +49,7 @@ CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012afrwy_qspi_defconfig +1 -1
index 610f32cd99..097349e497 100644
@@ -53,7 +53,7 @@ CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig +1 -1
index b3cb6f706d..48d12802f1 100644
@@ -49,7 +49,7 @@ CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012afrwy_tfa_defconfig +1 -1
index afec07cf8e..5137b52fd8 100644
@@ -53,7 +53,7 @@ CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012aqds_qspi_defconfig +1 -1
index 93e76d95f1..d10a4f558f 100644
@@ -71,7 +71,7 @@ CONFIG_SPI_FLASH_SST=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012aqds_tfa_SECURE_BOOT_defconfig +1 -1
index 476ebd0415..14eeade305 100644
@@ -63,7 +63,7 @@ CONFIG_SPI_FLASH_SST=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012aqds_tfa_defconfig +1 -1
index d4d9b8f76b..08850e5981 100644
@@ -72,7 +72,7 @@ CONFIG_SPI_FLASH_SST=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012ardb_qspi_SECURE_BOOT_defconfig +1 -1
index 8767c87cc6..f495367808 100644
@@ -51,7 +51,7 @@ CONFIG_DM_SPI_FLASH=y
 CONFIG_SPI_FLASH_SPANSION=y
 # CONFIG_SPI_FLASH_USE_4K_SECTORS is not set
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1012ardb_qspi_defconfig +1 -1
index fccbef70ea..262ef9bd03 100644
@@ -56,7 +56,7 @@ CONFIG_SPI_FLASH_SPANSION=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012ardb_tfa_SECURE_BOOT_defconfig +1 -1
index bf35286848..1be4478bbf 100644
@@ -53,7 +53,7 @@ CONFIG_SPI_FLASH_SPANSION=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1012ardb_tfa_defconfig +1 -1
index 7a175d3c74..12e77de638 100644
@@ -55,7 +55,7 @@ CONFIG_SPI_FLASH_SPANSION=y
 CONFIG_FSL_PFE=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1021aiot_qspi_defconfig +1 -1
index 4705e34e1d..fca79db20a 100644
@@ -53,7 +53,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aiot_sdcard_defconfig +1 -1
index e387264d89..c099a3ccfa 100644
@@ -70,7 +70,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_ddr4_nor_defconfig +1 -1
index 79ccc41a43..797d114245 100644
@@ -74,7 +74,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_ddr4_nor_lpuart_defconfig +1 -1
index 0a2a076332..b0c542c266 100644
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_nand_defconfig +1 -1
index 0b12f100d4..648182ae4c 100644
@@ -97,7 +97,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_nor_SECURE_BOOT_defconfig +1 -1
index 63930e4a13..e02b573e17 100644
@@ -73,7 +73,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_nor_defconfig +1 -1
index f4e2274fb8..a8da00a42b 100644
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_nor_lpuart_defconfig +1 -1
index 0eacaa3354..8bea5fbfb9 100644
@@ -76,7 +76,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_qspi_defconfig +1 -1
index f967619ef9..3a1b33dd70 100644
@@ -67,7 +67,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_sdcard_ifc_defconfig +1 -1
index fa190a7502..b5f3d9d170 100644
@@ -93,7 +93,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021aqds_sdcard_qspi_defconfig +1 -1
index 625dd086ab..bd3ae1a188 100644
@@ -84,7 +84,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1021atsn_qspi_defconfig +1 -1
index 5c8cee3ea9..aaea7156c1 100644
@@ -59,7 +59,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_MII=y
 CONFIG_SJA1105=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_NS16550=y
configs/ls1021atsn_sdcard_defconfig +1 -1
index a3b8f248c7..870fdb05bf 100644
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_MII=y
 CONFIG_SJA1105=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_NS16550=y
configs/ls1021atwr_nor_SECURE_BOOT_defconfig +1 -1
index f61d29ce89..27b28cc1c9 100644
@@ -62,7 +62,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x60940000
configs/ls1021atwr_nor_defconfig +1 -1
index 02c25cda7d..1242f9c5aa 100644
@@ -64,7 +64,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x60940000
configs/ls1021atwr_nor_lpuart_defconfig +1 -1
index 49d6dda6a5..2562f9077f 100644
@@ -65,7 +65,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x60940000
configs/ls1021atwr_qspi_defconfig +1 -1
index 36a8aee0b1..602ce2e9e3 100644
@@ -63,7 +63,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls1021atwr_sdcard_ifc_defconfig +1 -1
index ede4c95dd1..cb454837bb 100644
@@ -82,7 +82,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls1021atwr_sdcard_qspi_defconfig +1 -1
index 1315043094..195ce83198 100644
@@ -80,7 +80,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_TSEC_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls1028aqds_tfa_SECURE_BOOT_defconfig +1 -1
index 2fb90735ab..64bd2bbc24 100644
@@ -71,7 +71,7 @@ CONFIG_DM_DSA=y
 CONFIG_E1000=y
 CONFIG_MSCC_FELIX_SWITCH=y
 CONFIG_MDIO_MUX_I2CREG=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1028aqds_tfa_defconfig +1 -1
index 0bdca833f1..ca827ac0b4 100644
@@ -77,7 +77,7 @@ CONFIG_DM_DSA=y
 CONFIG_E1000=y
 CONFIG_MSCC_FELIX_SWITCH=y
 CONFIG_MDIO_MUX_I2CREG=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1028aqds_tfa_lpuart_defconfig +1 -1
index dbf6af99f5..f7933e383d 100644
@@ -76,7 +76,7 @@ CONFIG_DM_MDIO_MUX=y
 CONFIG_E1000=y
 CONFIG_FSL_ENETC=y
 CONFIG_MDIO_MUX_I2CREG=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1028ardb_tfa_SECURE_BOOT_defconfig +1 -1
index 5cb55d046d..f37cca2ad6 100644
@@ -66,7 +66,7 @@ CONFIG_DM_DSA=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MSCC_FELIX_SWITCH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1028ardb_tfa_defconfig +1 -1
index b58254dc65..b753824e9d 100644
@@ -72,7 +72,7 @@ CONFIG_DM_DSA=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MSCC_FELIX_SWITCH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_defconfig +1 -1
index 1a484ff483..c30731e3df 100644
@@ -79,7 +79,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_lpuart_defconfig +1 -1
index dfd7fb881b..229f13cc40 100644
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_nand_defconfig +1 -1
index 9e87f0fd88..e6ea996d3c 100644
@@ -100,7 +100,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_nor_ddr3_defconfig +1 -1
index bffe105220..4d7d14ece6 100644
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_qspi_defconfig +1 -1
index e01324ccef..bbd5330c7d 100644
@@ -74,7 +74,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_sdcard_ifc_defconfig +1 -1
index b487b370b5..4ccbfb78db 100644
@@ -98,7 +98,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_sdcard_qspi_defconfig +1 -1
index 084d104ea4..a8cfbf09e8 100644
@@ -91,7 +91,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_tfa_SECURE_BOOT_defconfig +1 -1
index 3e303c5311..f495098481 100644
@@ -81,7 +81,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043aqds_tfa_defconfig +1 -1
index 8d02bed8e1..39368732a3 100644
@@ -90,7 +90,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1043ardb_SECURE_BOOT_defconfig +1 -1
index 2d85f671cc..bba6e2eb46 100644
@@ -64,7 +64,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x60940000
configs/ls1043ardb_defconfig +1 -1
index ca6958fdb4..e361c96049 100644
@@ -67,7 +67,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x60940000
configs/ls1043ardb_nand_SECURE_BOOT_defconfig +1 -1
index bbc775a4d9..6d421d5d28 100644
@@ -79,7 +79,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FMAN_FW_IN_NAND=y
configs/ls1043ardb_nand_defconfig +1 -1
index e8986db42b..7e0da803b2 100644
@@ -87,7 +87,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FMAN_FW_IN_NAND=y
configs/ls1043ardb_sdcard_defconfig +1 -1
index c399ed0891..933de4ed2a 100644
@@ -85,7 +85,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x940000
configs/ls1043ardb_tfa_SECURE_BOOT_defconfig +1 -1
index 8ac9a06c72..14100c8429 100644
@@ -65,7 +65,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x940000
configs/ls1043ardb_tfa_defconfig +1 -1
index 99c125731f..6281f841a4 100644
@@ -71,7 +71,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_SYS_QE_FW_ADDR=0x940000
configs/ls1046afrwy_tfa_defconfig +1 -1
index af07b1cd46..d587826d44 100644
@@ -58,7 +58,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_SECURE_BOOT_defconfig +1 -1
index 55426320d8..19c483ed12 100644
@@ -76,7 +76,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_defconfig +1 -1
index f136aeafae..c98d2d5995 100644
@@ -79,7 +79,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_lpuart_defconfig +1 -1
index 6d448905ef..07d8474e5a 100644
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x60900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_nand_defconfig +1 -1
index e4cd4a2f60..9b1eb76b20 100644
@@ -99,7 +99,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_qspi_defconfig +1 -1
index fe295c51e4..6ce6abac58 100644
@@ -75,7 +75,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_sdcard_ifc_defconfig +1 -1
index fe1fe6f162..2ebe445ac6 100644
@@ -99,7 +99,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_sdcard_qspi_defconfig +1 -1
index 18f560e947..f85af10f73 100644
@@ -93,7 +93,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_tfa_SECURE_BOOT_defconfig +1 -1
index b84f8772c6..3f0613d3d7 100644
@@ -80,7 +80,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046aqds_tfa_defconfig +1 -1
index f706dd6179..ae78bf7ea0 100644
@@ -90,7 +90,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1046ardb_emmc_defconfig +1 -1
index 46301fb183..5c288a9e83 100644
@@ -83,7 +83,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_EP=y
configs/ls1046ardb_qspi_SECURE_BOOT_defconfig +1 -1
index a4696f1a16..5b6d9aa24b 100644
@@ -66,7 +66,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_EP=y
configs/ls1046ardb_qspi_defconfig +1 -1
index 1837833a8d..fd3ccc7063 100644
@@ -70,7 +70,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_EP=y
configs/ls1046ardb_qspi_spl_defconfig +1 -1
index 777b7678a3..08e1fb2ca0 100644
@@ -88,7 +88,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x40900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_EP=y
configs/ls1046ardb_sdcard_defconfig +1 -1
index ac8f2bcb1b..6e3694c42f 100644
@@ -82,7 +82,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_EP=y
configs/ls1046ardb_tfa_SECURE_BOOT_defconfig +1 -1
index bd1dc05c58..eccd109b9a 100644
@@ -62,7 +62,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_EP=y
configs/ls1046ardb_tfa_defconfig +1 -1
index 149091af4a..ebcf6777be 100644
@@ -68,7 +68,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_FMAN_ENET=y
 CONFIG_SYS_FMAN_FW_ADDR=0x900000
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_EP=y
configs/ls1088aqds_defconfig +1 -1
index a2b2a34e3f..8bc243e36b 100644
@@ -78,7 +78,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1088aqds_qspi_SECURE_BOOT_defconfig +1 -1
index dbcd6e2ea2..bc9243567c 100644
@@ -74,7 +74,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1088aqds_qspi_defconfig +1 -1
index 7033c92897..05747b0952 100644
@@ -77,7 +77,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1088aqds_sdcard_ifc_defconfig +1 -1
index e5372317c3..07dfbae2d8 100644
@@ -88,7 +88,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1088aqds_sdcard_qspi_defconfig +1 -1
index 9df9845296..dcf50ff803 100644
@@ -87,7 +87,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls1088aqds_tfa_defconfig +1 -1
index aac8486bc3..4c2e9420e1 100644
@@ -99,7 +99,7 @@ CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_MDIO_MUX_I2CREG=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1088ardb_qspi_SECURE_BOOT_defconfig +1 -1
index ed0943b6ec..c4c1a39790 100644
@@ -75,7 +75,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls1088ardb_qspi_defconfig +1 -1
index 486a20dbc5..2f74b87597 100644
@@ -78,7 +78,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls1088ardb_sdcard_qspi_defconfig +1 -1
index d54ff504a8..89e2bcf138 100644
@@ -88,7 +88,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls1088ardb_tfa_SECURE_BOOT_defconfig +1 -1
index dd547f797b..e2b5116c4f 100644
@@ -76,7 +76,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls1088ardb_tfa_defconfig +1 -1
index fc2d1b475c..8f54aa8a12 100644
@@ -82,7 +82,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls2080aqds_SECURE_BOOT_defconfig +1 -1
index 0ad25aca76..13bbfad1e5 100644
@@ -66,7 +66,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2080aqds_defconfig +1 -1
index 30cdad6fbe..0586bd337a 100644
@@ -69,7 +69,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2080aqds_nand_defconfig +1 -1
index 47f09b6288..cc74c5a686 100644
@@ -77,7 +77,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2080aqds_qspi_defconfig +1 -1
index bbc9f3d113..8df52491a7 100644
@@ -66,7 +66,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2080aqds_sdcard_defconfig +1 -1
index c7a6e2a851..65a0b81f15 100644
@@ -72,7 +72,7 @@ CONFIG_PHY_VITESSE=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2080ardb_SECURE_BOOT_defconfig +1 -1
index 15dec62dd4..33a129b20b 100644
@@ -66,7 +66,7 @@ CONFIG_PHY_CORTINA=y
 CONFIG_CORTINA_FW_ADDR=0x580980000
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2080ardb_defconfig +1 -1
index 909c10b631..7461713d03 100644
@@ -69,7 +69,7 @@ CONFIG_PHY_CORTINA=y
 CONFIG_CORTINA_FW_ADDR=0x580980000
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2080ardb_nand_defconfig +1 -1
index f40ecb7e68..ad301b8810 100644
@@ -78,7 +78,7 @@ CONFIG_PHY_CORTINA=y
 CONFIG_CORTINA_FW_ADDR=0x980000
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2081ardb_defconfig +1 -1
index ed0b1b7622..e0c7e7b882 100644
@@ -62,7 +62,7 @@ CONFIG_PHY_CORTINA=y
 CONFIG_CORTINA_FW_ADDR=0x980000
 CONFIG_E1000=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_DM_PCI_COMPAT=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
configs/ls2088aqds_tfa_defconfig +1 -1
index 61e0fdf113..69cb3d7c40 100644
@@ -87,7 +87,7 @@ CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_MDIO_MUX_I2CREG=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls2088ardb_qspi_SECURE_BOOT_defconfig +1 -1
index c49d163346..037ee97a33 100644
@@ -65,7 +65,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls2088ardb_qspi_defconfig +1 -1
index b8c7c78b8e..b08bcd44e8 100644
@@ -72,7 +72,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_SCSI=y
configs/ls2088ardb_tfa_SECURE_BOOT_defconfig +1 -1
index 3cc9168290..590f30828f 100644
@@ -77,7 +77,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/ls2088ardb_tfa_defconfig +1 -1
index a28b45b129..dc1abed910 100644
@@ -85,7 +85,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_DM_RTC=y
configs/lx2160aqds_tfa_SECURE_BOOT_defconfig +1 -1
index a9a0a27888..d6f4275b9c 100644
@@ -80,7 +80,7 @@ CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_MDIO_MUX_I2CREG=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_GEN4=y
configs/lx2160aqds_tfa_defconfig +1 -1
index de8dc553d1..920dee7916 100644
@@ -87,7 +87,7 @@ CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_MDIO_MUX_I2CREG=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_GEN4=y
configs/lx2160ardb_tfa_SECURE_BOOT_defconfig +1 -1
index ac42c2508c..a82372d567 100644
@@ -72,7 +72,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_GEN4=y
configs/lx2160ardb_tfa_defconfig +1 -1
index 59955eebbe..31920f4430 100644
@@ -81,7 +81,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_GEN4=y
configs/lx2160ardb_tfa_stmm_defconfig +1 -1
index 149e82bed3..35bd6a26da 100644
@@ -81,7 +81,7 @@ CONFIG_DM_MDIO=y
 CONFIG_E1000=y
 CONFIG_MII=y
 CONFIG_FSL_LS_MDIO=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_LAYERSCAPE_RC=y
 CONFIG_PCIE_LAYERSCAPE_GEN4=y
configs/mvebu_crb_cn9130_defconfig +1 -1
index 039fd8b3dc..f1215fa359 100644
@@ -62,7 +62,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MVEBU=y
 CONFIG_PHY=y
configs/mvebu_db_armada8k_defconfig +1 -1
index e6168a76a6..622d687e5d 100644
@@ -51,7 +51,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MVEBU=y
 CONFIG_PHY=y
configs/mvebu_db_cn9130_defconfig +1 -1
index 42d7038c5a..1789838691 100644
@@ -67,7 +67,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MVEBU=y
 CONFIG_PHY=y
configs/mvebu_espressobin-88f3720_defconfig +1 -1
index 3a69954fcd..cb3ddcf2a9 100644
@@ -73,7 +73,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MVNETA=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_AARDVARK=y
 CONFIG_PHY=y
configs/mvebu_mcbin-88f8040_defconfig +1 -1
index 2aa06f80a0..f45ce91f6f 100644
@@ -56,7 +56,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MVEBU=y
 CONFIG_PHY=y
configs/mvebu_puzzle-m801-88f8040_defconfig +1 -1
index 053b2f4c96..af01e6176a 100644
@@ -60,7 +60,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_E1000=y
 CONFIG_MVPP2=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_MVEBU=y
 CONFIG_PHY=y
configs/nanopc-t4-rk3399_defconfig +1 -1
index f31668c5c2..3b3da3870b 100644
@@ -37,7 +37,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
 CONFIG_DM_ETH=y
 CONFIG_ETH_DESIGNWARE=y
 CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PMIC_RK8XX=y
 CONFIG_REGULATOR_PWM=y
configs/octeontx2_96xx_defconfig +1 -1
index 1298bfe309..1ce892d963 100644
@@ -99,7 +99,7 @@ CONFIG_E1000_SPI=y
 CONFIG_CMD_E1000=y
 CONFIG_NET_OCTEONTX2=y
 CONFIG_OCTEONTX_SMI=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_REGION_MULTI_ENTRY=y
 CONFIG_PCI_SRIOV=y
configs/octeontx_81xx_defconfig +1 -1
index ba8cc97ab8..ddb9007bf1 100644
@@ -99,7 +99,7 @@ CONFIG_E1000_SPI=y
 CONFIG_CMD_E1000=y
 CONFIG_NET_OCTEONTX=y
 CONFIG_OCTEONTX_SMI=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_REGION_MULTI_ENTRY=y
 CONFIG_PCI_SRIOV=y
configs/octeontx_83xx_defconfig +1 -1
index 26759341c5..b8ebc2812e 100644
@@ -96,7 +96,7 @@ CONFIG_E1000_SPI=y
 CONFIG_CMD_E1000=y
 CONFIG_NET_OCTEONTX=y
 CONFIG_OCTEONTX_SMI=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_REGION_MULTI_ENTRY=y
 CONFIG_PCI_SRIOV=y
configs/p3450-0000_defconfig +1 -1
index e4265d6321..46f4cd0110 100644
@@ -43,7 +43,7 @@ CONFIG_SYS_I2C_TEGRA=y
 CONFIG_SF_DEFAULT_SPEED=24000000
 CONFIG_SPI_FLASH_MACRONIX=y
 CONFIG_RTL8169=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_TEGRA=y
 CONFIG_SYS_NS16550=y
configs/pinebook-pro-rk3399_defconfig +1 -1
index 81aedb28e3..d7378f5eb3 100644
@@ -56,7 +56,7 @@ CONFIG_SF_DEFAULT_SPEED=20000000
 CONFIG_SPI_FLASH_GIGADEVICE=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PHY_ROCKCHIP_INNO_USB2=y
 CONFIG_PHY_ROCKCHIP_TYPEC=y
configs/qemu-x86_64_defconfig +1 -1
index 2dfb48b383..0d9071850a 100644
@@ -56,7 +56,7 @@ CONFIG_TFTP_TSIZE=y
 CONFIG_REGMAP=y
 CONFIG_SYSCON=y
 CONFIG_CPU=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_SPL_DM_RTC=y
 CONFIG_SPI=y
 CONFIG_USB_KEYBOARD=y
configs/qemu-x86_defconfig +1 -1
index 6be7ce0c6e..ea86d044a7 100644
@@ -39,7 +39,7 @@ CONFIG_TFTP_TSIZE=y
 CONFIG_REGMAP=y
 CONFIG_SYSCON=y
 CONFIG_CPU=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_SPI=y
 CONFIG_USB_KEYBOARD=y
 CONFIG_FRAMEBUFFER_SET_VESA_MODE=y
configs/qemu_arm64_defconfig +1 -1
index 5e41ea980f..ca5534cd3a 100644
@@ -46,7 +46,7 @@ CONFIG_FLASH_CFI_MTD=y
 CONFIG_SYS_FLASH_CFI=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_SCSI=y
configs/qemu_arm_defconfig +1 -1
index 4197cb0f92..42f85c8176 100644
@@ -48,7 +48,7 @@ CONFIG_FLASH_CFI_MTD=y
 CONFIG_SYS_FLASH_CFI=y
 CONFIG_DM_ETH=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_ECAM_GENERIC=y
 CONFIG_SCSI=y
configs/rcar3_salvator-x_defconfig +1 -1
index 57a2c01932..e0bd1de73e 100644
@@ -79,7 +79,7 @@ CONFIG_PHY_MICREL=y
 CONFIG_PHY_MICREL_KSZ90X1=y
 CONFIG_DM_ETH=y
 CONFIG_RENESAS_RAVB=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_REGION_MULTI_ENTRY=y
 CONFIG_PCI_RCAR_GEN3=y
configs/roc-pc-mezzanine-rk3399_defconfig +1 -1
index ca2fb9e13f..b79200fd3b 100644
@@ -49,7 +49,7 @@ CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
 CONFIG_ETH_DESIGNWARE=y
 CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PHY_ROCKCHIP_INNO_USB2=y
 CONFIG_PHY_ROCKCHIP_TYPEC=y
configs/rock-pi-4-rk3399_defconfig +1 -1
index 032b908669..73f2824f0a 100644
@@ -41,7 +41,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
 CONFIG_DM_ETH=y
 CONFIG_ETH_DESIGNWARE=y
 CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PHY_ROCKCHIP_INNO_USB2=y
 CONFIG_PHY_ROCKCHIP_TYPEC=y
configs/rock-pi-4c-rk3399_defconfig +1 -1
index 6f5e8666b0..ce4d7761a2 100644
@@ -41,7 +41,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
 CONFIG_DM_ETH=y
 CONFIG_ETH_DESIGNWARE=y
 CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PHY_ROCKCHIP_INNO_USB2=y
 CONFIG_PHY_ROCKCHIP_TYPEC=y
configs/rock-pi-n10-rk3399pro_defconfig +1 -1
index bd8b1201ef..c066d9160a 100644
@@ -42,7 +42,7 @@ CONFIG_MMC_SDHCI_ROCKCHIP=y
 CONFIG_DM_ETH=y
 CONFIG_ETH_DESIGNWARE=y
 CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PHY_ROCKCHIP_INNO_USB2=y
 CONFIG_PHY_ROCKCHIP_TYPEC=y
configs/rock960-rk3399_defconfig +1 -1
index e46f07e74d..d95da51891 100644
@@ -42,7 +42,7 @@ CONFIG_MMC_SDHCI=y
 CONFIG_MMC_SDHCI_SDMA=y
 CONFIG_MMC_SDHCI_ROCKCHIP=y
 CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PHY_ROCKCHIP_INNO_USB2=y
 CONFIG_PHY_ROCKCHIP_TYPEC=y
configs/rockpro64-rk3399_defconfig +1 -1
index 637c5c2466..d5e98a4f73 100644
@@ -48,7 +48,7 @@ CONFIG_SPI_FLASH_GIGADEVICE=y
 CONFIG_DM_ETH=y
 CONFIG_ETH_DESIGNWARE=y
 CONFIG_GMAC_ROCKCHIP=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PHY_ROCKCHIP_INNO_USB2=y
 CONFIG_PHY_ROCKCHIP_TYPEC=y
configs/sandbox64_defconfig +1 -1
index d849989cf6..46f30b55e4 100644
@@ -158,7 +158,7 @@ CONFIG_SPI_FLASH_STMICRO=y
 CONFIG_SPI_FLASH_SST=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_SANDBOX=y
 CONFIG_PHY=y
configs/sandbox_defconfig +1 -1
index 4f413582fb..7ab4e42802 100644
@@ -200,7 +200,7 @@ CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_MULTIPLEXER=y
 CONFIG_MUX_MMIO=y
 CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_REGION_MULTI_ENTRY=y
 CONFIG_PCI_SANDBOX=y
configs/sandbox_flattree_defconfig +1 -1
index 4d5a73fce0..325e36203f 100644
@@ -134,7 +134,7 @@ CONFIG_SPI_FLASH_STMICRO=y
 CONFIG_SPI_FLASH_SST=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_REGION_MULTI_ENTRY=y
 CONFIG_PCI_SANDBOX=y
configs/sandbox_noinst_defconfig +1 -1
index 490368e768..ff107e823e 100644
@@ -154,7 +154,7 @@ CONFIG_SPI_FLASH_STMICRO=y
 CONFIG_SPI_FLASH_SST=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_SANDBOX=y
 CONFIG_PHY=y
configs/sandbox_spl_defconfig +1 -1
index f1a54ace9b..5d60da0edf 100644
@@ -156,7 +156,7 @@ CONFIG_SPI_FLASH_STMICRO=y
 CONFIG_SPI_FLASH_SST=y
 CONFIG_SPI_FLASH_WINBOND=y
 CONFIG_DM_ETH=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_SANDBOX=y
 CONFIG_PHY=y
configs/sifive_unmatched_defconfig +1 -1
index 299580894c..86f8c7856e 100644
@@ -44,7 +44,7 @@ CONFIG_SPL_CLK=y
 CONFIG_SYS_I2C_EEPROM_ADDR=0x54
 CONFIG_SPI_FLASH_ISSI=y
 CONFIG_E1000=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCIE_DW_SIFIVE=y
 CONFIG_DM_RESET=y
configs/synquacer_developerbox_defconfig +1 -1
index da57dc288f..fe12c74374 100644
@@ -69,7 +69,7 @@ CONFIG_PHY_GIGE=y
 CONFIG_RGMII=y
 CONFIG_MII=y
 CONFIG_SNI_NETSEC=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_DM_RTC=y
 CONFIG_RTC_PCF8563=y
 CONFIG_SCSI=y
configs/turris_mox_defconfig +1 -1
index 3cae32f69b..d5ab77b273 100644
@@ -79,7 +79,7 @@ CONFIG_SPI_FLASH_MTD=y
 CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_MVNETA=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_AARDVARK=y
 CONFIG_PHY=y
configs/turris_omnia_defconfig +1 -1
index 9d121b7982..cf3c5c3248 100644
@@ -77,7 +77,7 @@ CONFIG_PHY_MARVELL=y
 CONFIG_PHY_GIGE=y
 CONFIG_MVNETA=y
 CONFIG_MII=y
-CONFIG_NVME=y
+CONFIG_NVME_PCI=y
 CONFIG_PCI=y
 CONFIG_PCI_MVEBU=y
 CONFIG_DM_RTC=y
doc/develop/driver-model/nvme.rst +1 -0
index 736c0a063d..fd0c0f00d2 100644
@@ -40,6 +40,7 @@ It only support basic block read/write functions in the NVMe driver.
 Config options
 --------------
 CONFIG_NVME	Enable NVMe device support
+CONFIG_NVME_PCI	Enable PCIe NVMe device support
 CONFIG_CMD_NVME	Enable basic NVMe commands
 
 Usage in U-Boot
drivers/nvme/Kconfig +9 -1
index 1f6d1f5648..78da444c8b 100644
@@ -4,8 +4,16 @@
 
 config NVME
 	bool "NVM Express device support"
-	depends on BLK && PCI
+	depends on BLK
 	select HAVE_BLOCK_DEVICE
 	help
 	  This option enables support for NVM Express devices.
 	  It supports basic functions of NVMe (read/write).
+
+config NVME_PCI
+	bool "NVM Express PCI device support"
+	depends on PCI
+	select NVME
+	help
+	  This option enables support for NVM Express PCI
+	  devices.
drivers/nvme/Makefile +1 -0
index 64f102b208..fad9724e17 100644
@@ -3,3 +3,4 @@
 # Copyright (C) 2017, Bin Meng <[email protected]>
 
 obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_PCI) += nvme_pci.o
drivers/nvme/nvme.c +3 -35
index 3c529a2fce..be518ec20b 100644
@@ -12,7 +12,6 @@
 #include <log.h>
 #include <malloc.h>
 #include <memalign.h>
-#include <pci.h>
 #include <time.h>
 #include <dm/device-internal.h>
 #include <linux/compat.h>
@@ -698,7 +697,6 @@ static int nvme_blk_probe(struct udevice *udev)
 	struct blk_desc *desc = dev_get_uclass_plat(udev);
 	struct nvme_ns *ns = dev_get_priv(udev);
 	u8 flbas;
-	struct pci_child_plat *pplat;
 	struct nvme_id_ns *id;
 
 	id = memalign(ndev->page_size, sizeof(struct nvme_id_ns));
@@ -723,8 +721,7 @@ static int nvme_blk_probe(struct udevice *udev)
 	desc->log2blksz = ns->lba_shift;
 	desc->blksz = 1 << ns->lba_shift;
 	desc->bdev = udev;
-	pplat = dev_get_parent_plat(udev->parent);
-	sprintf(desc->vendor, "0x%.4x", pplat->vendor);
+	memcpy(desc->vendor, ndev->vendor, sizeof(ndev->vendor));
 	memcpy(desc->product, ndev->serial, sizeof(ndev->serial));
 	memcpy(desc->revision, ndev->firmware_rev, sizeof(ndev->firmware_rev));
 
@@ -818,27 +815,13 @@ U_BOOT_DRIVER(nvme_blk) = {
 	.priv_auto	= sizeof(struct nvme_ns),
 };
 
-static int nvme_bind(struct udevice *udev)
+int nvme_init(struct udevice *udev)
 {
-	static int ndev_num;
-	char name[20];
-
-	sprintf(name, "nvme#%d", ndev_num++);
-
-	return device_set_name(udev, name);
-}
-
-static int nvme_probe(struct udevice *udev)
-{
-	int ret;
 	struct nvme_dev *ndev = dev_get_priv(udev);
 	struct nvme_id_ns *id;
-
-	ndev->instance = trailing_strtol(udev->name);
+	int ret;
 
 	INIT_LIST_HEAD(&ndev->namespaces);
-	ndev->bar = dm_pci_map_bar(udev, PCI_BASE_ADDRESS_0,
-			PCI_REGION_MEM);
 	if (readl(&ndev->bar->csts) == -1) {
 		ret = -ENODEV;
 		printf("Error: %s: Out of memory!\n", udev->name);
@@ -922,18 +905,3 @@ free_queue:
 free_nvme:
 	return ret;
 }
-
-U_BOOT_DRIVER(nvme) = {
-	.name	= "nvme",
-	.id	= UCLASS_NVME,
-	.bind	= nvme_bind,
-	.probe	= nvme_probe,
-	.priv_auto	= sizeof(struct nvme_dev),
-};
-
-struct pci_device_id nvme_supported[] = {
-	{ PCI_DEVICE_CLASS(PCI_CLASS_STORAGE_EXPRESS, ~0) },
-	{}
-};
-
-U_BOOT_PCI_DEVICE(nvme, nvme_supported);
drivers/nvme/nvme.h +3 -0
index c6aae4da5d..8e9ae3c7f6 100644
@@ -608,6 +608,7 @@ struct nvme_dev {
 	u32 ctrl_config;
 	struct nvme_bar __iomem *bar;
 	struct list_head namespaces;
+	char vendor[8];
 	char serial[20];
 	char model[40];
 	char firmware_rev[8];
@@ -635,4 +636,6 @@ struct nvme_ns {
 	u8 flbas;
 };
 
+int nvme_init(struct udevice *udev);
+
 #endif /* __DRIVER_NVME_H__ */
drivers/nvme/nvme_pci.c new file +49 -0

49 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by Tested-by on v1 1/8
#2 lore raw

Re: [PATCH 1/8] nvme: Split out PCI support

Hi Mark,

Quoted 150 lines · Mark Kettenis

Apple SoCs have an integrated NVMe controller that isn't connected over a PCIe bus. In preparation for adding support for this NVMe controller, split out the PCI support into its own file. This file is selected through a new CONFIG_NVME_PCI Kconfig option, so do a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.

---
 configs/clearfog_gt_8k_defconfig              |  2 +-
 configs/firefly-rk3399_defconfig              |  2 +-
 configs/khadas-vim3_android_ab_defconfig      |  2 +-
 configs/khadas-vim3_android_defconfig         |  2 +-
 configs/khadas-vim3_defconfig                 |  2 +-
 configs/khadas-vim3l_android_ab_defconfig     |  2 +-
 configs/khadas-vim3l_android_defconfig        |  2 +-
 configs/khadas-vim3l_defconfig                |  2 +-
 configs/kontron_sl28_defconfig                |  2 +-
 configs/ls1012afrdm_qspi_defconfig            |  2 +-
 configs/ls1012afrdm_tfa_defconfig             |  2 +-
 .../ls1012afrwy_qspi_SECURE_BOOT_defconfig    |  2 +-
 configs/ls1012afrwy_qspi_defconfig            |  2 +-
 configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012afrwy_tfa_defconfig             |  2 +-
 configs/ls1012aqds_qspi_defconfig             |  2 +-
 configs/ls1012aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012aqds_tfa_defconfig              |  2 +-
 configs/ls1012ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012ardb_qspi_defconfig             |  2 +-
 configs/ls1012ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012ardb_tfa_defconfig              |  2 +-
 configs/ls1021aiot_qspi_defconfig             |  2 +-
 configs/ls1021aiot_sdcard_defconfig           |  2 +-
 configs/ls1021aqds_ddr4_nor_defconfig         |  2 +-
 configs/ls1021aqds_ddr4_nor_lpuart_defconfig  |  2 +-
 configs/ls1021aqds_nand_defconfig             |  2 +-
 configs/ls1021aqds_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021aqds_nor_defconfig              |  2 +-
 configs/ls1021aqds_nor_lpuart_defconfig       |  2 +-
 configs/ls1021aqds_qspi_defconfig             |  2 +-
 configs/ls1021aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1021atsn_qspi_defconfig             |  2 +-
 configs/ls1021atsn_sdcard_defconfig           |  2 +-
 configs/ls1021atwr_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021atwr_nor_defconfig              |  2 +-
 configs/ls1021atwr_nor_lpuart_defconfig       |  2 +-
 configs/ls1021atwr_qspi_defconfig             |  2 +-
 configs/ls1021atwr_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021atwr_sdcard_qspi_defconfig      |  2 +-
 configs/ls1028aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028aqds_tfa_defconfig              |  2 +-
 configs/ls1028aqds_tfa_lpuart_defconfig       |  2 +-
 configs/ls1028ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028ardb_tfa_defconfig              |  2 +-
 configs/ls1043aqds_defconfig                  |  2 +-
 configs/ls1043aqds_lpuart_defconfig           |  2 +-
 configs/ls1043aqds_nand_defconfig             |  2 +-
 configs/ls1043aqds_nor_ddr3_defconfig         |  2 +-
 configs/ls1043aqds_qspi_defconfig             |  2 +-
 configs/ls1043aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1043aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1043aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043aqds_tfa_defconfig              |  2 +-
 configs/ls1043ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1043ardb_defconfig                  |  2 +-
 configs/ls1043ardb_nand_SECURE_BOOT_defconfig |  2 +-
 configs/ls1043ardb_nand_defconfig             |  2 +-
 configs/ls1043ardb_sdcard_defconfig           |  2 +-
 configs/ls1043ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043ardb_tfa_defconfig              |  2 +-
 configs/ls1046afrwy_tfa_defconfig             |  2 +-
 configs/ls1046aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1046aqds_defconfig                  |  2 +-
 configs/ls1046aqds_lpuart_defconfig           |  2 +-
 configs/ls1046aqds_nand_defconfig             |  2 +-
 configs/ls1046aqds_qspi_defconfig             |  2 +-
 configs/ls1046aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1046aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1046aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046aqds_tfa_defconfig              |  2 +-
 configs/ls1046ardb_emmc_defconfig             |  2 +-
 configs/ls1046ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1046ardb_qspi_defconfig             |  2 +-
 configs/ls1046ardb_qspi_spl_defconfig         |  2 +-
 configs/ls1046ardb_sdcard_defconfig           |  2 +-
 configs/ls1046ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046ardb_tfa_defconfig              |  2 +-
 configs/ls1088aqds_defconfig                  |  2 +-
 configs/ls1088aqds_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088aqds_qspi_defconfig             |  2 +-
 configs/ls1088aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1088aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088aqds_tfa_defconfig              |  2 +-
 configs/ls1088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088ardb_qspi_defconfig             |  2 +-
 configs/ls1088ardb_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1088ardb_tfa_defconfig              |  2 +-
 configs/ls2080aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080aqds_defconfig                  |  2 +-
 configs/ls2080aqds_nand_defconfig             |  2 +-
 configs/ls2080aqds_qspi_defconfig             |  2 +-
 configs/ls2080aqds_sdcard_defconfig           |  2 +-
 configs/ls2080ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080ardb_defconfig                  |  2 +-
 configs/ls2080ardb_nand_defconfig             |  2 +-
 configs/ls2081ardb_defconfig                  |  2 +-
 configs/ls2088aqds_tfa_defconfig              |  2 +-
 configs/ls2088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls2088ardb_qspi_defconfig             |  2 +-
 configs/ls2088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls2088ardb_tfa_defconfig              |  2 +-
 configs/lx2160aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160aqds_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160ardb_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_stmm_defconfig         |  2 +-
 configs/mvebu_crb_cn9130_defconfig            |  2 +-
 configs/mvebu_db_armada8k_defconfig           |  2 +-
 configs/mvebu_db_cn9130_defconfig             |  2 +-
 configs/mvebu_espressobin-88f3720_defconfig   |  2 +-
 configs/mvebu_mcbin-88f8040_defconfig         |  2 +-
 configs/mvebu_puzzle-m801-88f8040_defconfig   |  2 +-
 configs/nanopc-t4-rk3399_defconfig            |  2 +-
 configs/octeontx2_96xx_defconfig              |  2 +-
 configs/octeontx_81xx_defconfig               |  2 +-
 configs/octeontx_83xx_defconfig               |  2 +-
 configs/p3450-0000_defconfig                  |  2 +-
 configs/pinebook-pro-rk3399_defconfig         |  2 +-
 configs/qemu-x86_64_defconfig                 |  2 +-
 configs/qemu-x86_defconfig                    |  2 +-
 configs/qemu_arm64_defconfig                  |  2 +-
 configs/qemu_arm_defconfig                    |  2 +-
 configs/rcar3_salvator-x_defconfig            |  2 +-
 configs/roc-pc-mezzanine-rk3399_defconfig     |  2 +-
 configs/rock-pi-4-rk3399_defconfig            |  2 +-
 configs/rock-pi-4c-rk3399_defconfig           |  2 +-
 configs/rock-pi-n10-rk3399pro_defconfig       |  2 +-
 configs/rock960-rk3399_defconfig              |  2 +-
 configs/rockpro64-rk3399_defconfig            |  2 +-
 configs/sandbox64_defconfig                   |  2 +-
 configs/sandbox_defconfig                     |  2 +-
 configs/sandbox_flattree_defconfig            |  2 +-
 configs/sandbox_noinst_defconfig              |  2 +-
 configs/sandbox_spl_defconfig                 |  2 +-
 configs/sifive_unmatched_defconfig            |  2 +-
 configs/synquacer_developerbox_defconfig      |  2 +-
 configs/turris_mox_defconfig                  |  2 +-
 configs/turris_omnia_defconfig                |  2 +-
Quoted 9 lines
 doc/develop/driver-model/nvme.rst             |  1 +
 drivers/nvme/Kconfig                          | 10 +++-
 drivers/nvme/Makefile                         |  1 +
 drivers/nvme/nvme.c                           | 38 ++------------
 drivers/nvme/nvme.h                           |  3 ++
 drivers/nvme/nvme_pci.c                       | 49 +++++++++++++++++++
 147 files changed, 207 insertions(+), 177 deletions(-)
 create mode 100644 drivers/nvme/nvme_pci.c

Shouldn't the vendor addition go in another patch?

Otherwise:

Tested on: Macbook Air M1

#3 lore raw

Re: [PATCH 1/8] nvme: Split out PCI support

Quoted 166 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:11 -0700

Hi Mark,

Quoted 150 lines · Mark Kettenis

Apple SoCs have an integrated NVMe controller that isn't connected over a PCIe bus. In preparation for adding support for this NVMe controller, split out the PCI support into its own file. This file is selected through a new CONFIG_NVME_PCI Kconfig option, so do a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.

---
 configs/clearfog_gt_8k_defconfig              |  2 +-
 configs/firefly-rk3399_defconfig              |  2 +-
 configs/khadas-vim3_android_ab_defconfig      |  2 +-
 configs/khadas-vim3_android_defconfig         |  2 +-
 configs/khadas-vim3_defconfig                 |  2 +-
 configs/khadas-vim3l_android_ab_defconfig     |  2 +-
 configs/khadas-vim3l_android_defconfig        |  2 +-
 configs/khadas-vim3l_defconfig                |  2 +-
 configs/kontron_sl28_defconfig                |  2 +-
 configs/ls1012afrdm_qspi_defconfig            |  2 +-
 configs/ls1012afrdm_tfa_defconfig             |  2 +-
 .../ls1012afrwy_qspi_SECURE_BOOT_defconfig    |  2 +-
 configs/ls1012afrwy_qspi_defconfig            |  2 +-
 configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012afrwy_tfa_defconfig             |  2 +-
 configs/ls1012aqds_qspi_defconfig             |  2 +-
 configs/ls1012aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012aqds_tfa_defconfig              |  2 +-
 configs/ls1012ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012ardb_qspi_defconfig             |  2 +-
 configs/ls1012ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012ardb_tfa_defconfig              |  2 +-
 configs/ls1021aiot_qspi_defconfig             |  2 +-
 configs/ls1021aiot_sdcard_defconfig           |  2 +-
 configs/ls1021aqds_ddr4_nor_defconfig         |  2 +-
 configs/ls1021aqds_ddr4_nor_lpuart_defconfig  |  2 +-
 configs/ls1021aqds_nand_defconfig             |  2 +-
 configs/ls1021aqds_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021aqds_nor_defconfig              |  2 +-
 configs/ls1021aqds_nor_lpuart_defconfig       |  2 +-
 configs/ls1021aqds_qspi_defconfig             |  2 +-
 configs/ls1021aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1021atsn_qspi_defconfig             |  2 +-
 configs/ls1021atsn_sdcard_defconfig           |  2 +-
 configs/ls1021atwr_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021atwr_nor_defconfig              |  2 +-
 configs/ls1021atwr_nor_lpuart_defconfig       |  2 +-
 configs/ls1021atwr_qspi_defconfig             |  2 +-
 configs/ls1021atwr_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021atwr_sdcard_qspi_defconfig      |  2 +-
 configs/ls1028aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028aqds_tfa_defconfig              |  2 +-
 configs/ls1028aqds_tfa_lpuart_defconfig       |  2 +-
 configs/ls1028ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028ardb_tfa_defconfig              |  2 +-
 configs/ls1043aqds_defconfig                  |  2 +-
 configs/ls1043aqds_lpuart_defconfig           |  2 +-
 configs/ls1043aqds_nand_defconfig             |  2 +-
 configs/ls1043aqds_nor_ddr3_defconfig         |  2 +-
 configs/ls1043aqds_qspi_defconfig             |  2 +-
 configs/ls1043aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1043aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1043aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043aqds_tfa_defconfig              |  2 +-
 configs/ls1043ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1043ardb_defconfig                  |  2 +-
 configs/ls1043ardb_nand_SECURE_BOOT_defconfig |  2 +-
 configs/ls1043ardb_nand_defconfig             |  2 +-
 configs/ls1043ardb_sdcard_defconfig           |  2 +-
 configs/ls1043ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043ardb_tfa_defconfig              |  2 +-
 configs/ls1046afrwy_tfa_defconfig             |  2 +-
 configs/ls1046aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1046aqds_defconfig                  |  2 +-
 configs/ls1046aqds_lpuart_defconfig           |  2 +-
 configs/ls1046aqds_nand_defconfig             |  2 +-
 configs/ls1046aqds_qspi_defconfig             |  2 +-
 configs/ls1046aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1046aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1046aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046aqds_tfa_defconfig              |  2 +-
 configs/ls1046ardb_emmc_defconfig             |  2 +-
 configs/ls1046ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1046ardb_qspi_defconfig             |  2 +-
 configs/ls1046ardb_qspi_spl_defconfig         |  2 +-
 configs/ls1046ardb_sdcard_defconfig           |  2 +-
 configs/ls1046ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046ardb_tfa_defconfig              |  2 +-
 configs/ls1088aqds_defconfig                  |  2 +-
 configs/ls1088aqds_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088aqds_qspi_defconfig             |  2 +-
 configs/ls1088aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1088aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088aqds_tfa_defconfig              |  2 +-
 configs/ls1088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088ardb_qspi_defconfig             |  2 +-
 configs/ls1088ardb_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1088ardb_tfa_defconfig              |  2 +-
 configs/ls2080aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080aqds_defconfig                  |  2 +-
 configs/ls2080aqds_nand_defconfig             |  2 +-
 configs/ls2080aqds_qspi_defconfig             |  2 +-
 configs/ls2080aqds_sdcard_defconfig           |  2 +-
 configs/ls2080ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080ardb_defconfig                  |  2 +-
 configs/ls2080ardb_nand_defconfig             |  2 +-
 configs/ls2081ardb_defconfig                  |  2 +-
 configs/ls2088aqds_tfa_defconfig              |  2 +-
 configs/ls2088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls2088ardb_qspi_defconfig             |  2 +-
 configs/ls2088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls2088ardb_tfa_defconfig              |  2 +-
 configs/lx2160aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160aqds_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160ardb_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_stmm_defconfig         |  2 +-
 configs/mvebu_crb_cn9130_defconfig            |  2 +-
 configs/mvebu_db_armada8k_defconfig           |  2 +-
 configs/mvebu_db_cn9130_defconfig             |  2 +-
 configs/mvebu_espressobin-88f3720_defconfig   |  2 +-
 configs/mvebu_mcbin-88f8040_defconfig         |  2 +-
 configs/mvebu_puzzle-m801-88f8040_defconfig   |  2 +-
 configs/nanopc-t4-rk3399_defconfig            |  2 +-
 configs/octeontx2_96xx_defconfig              |  2 +-
 configs/octeontx_81xx_defconfig               |  2 +-
 configs/octeontx_83xx_defconfig               |  2 +-
 configs/p3450-0000_defconfig                  |  2 +-
 configs/pinebook-pro-rk3399_defconfig         |  2 +-
 configs/qemu-x86_64_defconfig                 |  2 +-
 configs/qemu-x86_defconfig                    |  2 +-
 configs/qemu_arm64_defconfig                  |  2 +-
 configs/qemu_arm_defconfig                    |  2 +-
 configs/rcar3_salvator-x_defconfig            |  2 +-
 configs/roc-pc-mezzanine-rk3399_defconfig     |  2 +-
 configs/rock-pi-4-rk3399_defconfig            |  2 +-
 configs/rock-pi-4c-rk3399_defconfig           |  2 +-
 configs/rock-pi-n10-rk3399pro_defconfig       |  2 +-
 configs/rock960-rk3399_defconfig              |  2 +-
 configs/rockpro64-rk3399_defconfig            |  2 +-
 configs/sandbox64_defconfig                   |  2 +-
 configs/sandbox_defconfig                     |  2 +-
 configs/sandbox_flattree_defconfig            |  2 +-
 configs/sandbox_noinst_defconfig              |  2 +-
 configs/sandbox_spl_defconfig                 |  2 +-
 configs/sifive_unmatched_defconfig            |  2 +-
 configs/synquacer_developerbox_defconfig      |  2 +-
 configs/turris_mox_defconfig                  |  2 +-
 configs/turris_omnia_defconfig                |  2 +-
Quoted 9 lines
 doc/develop/driver-model/nvme.rst             |  1 +
 drivers/nvme/Kconfig                          | 10 +++-
 drivers/nvme/Makefile                         |  1 +
 drivers/nvme/nvme.c                           | 38 ++------------
 drivers/nvme/nvme.h                           |  3 ++
 drivers/nvme/nvme_pci.c                       | 49 +++++++++++++++++++
 147 files changed, 207 insertions(+), 177 deletions(-)
 create mode 100644 drivers/nvme/nvme_pci.c
Quoted 1 line

Shouldn't the vendor addition go in another patch?

I don't think that would make a ton of sense. It is intricately linked to splitting out the PCI support, as the current code grovels into pci_child_plat struct in a place that needs to be bus-agnostic.

Quoted 4 lines

Otherwise:

Tested on: Macbook Air M1

#4 lore raw

Re: [PATCH 1/8] nvme: Split out PCI support

Hi Mark,

Quoted 167 lines · Mark Kettenis
Quoted 167 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:11 -0700

Hi Mark,

Quoted 159 lines · Mark Kettenis

Apple SoCs have an integrated NVMe controller that isn't connected over a PCIe bus. In preparation for adding support for this NVMe controller, split out the PCI support into its own file. This file is selected through a new CONFIG_NVME_PCI Kconfig option, so do a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.

---
 configs/clearfog_gt_8k_defconfig              |  2 +-
 configs/firefly-rk3399_defconfig              |  2 +-
 configs/khadas-vim3_android_ab_defconfig      |  2 +-
 configs/khadas-vim3_android_defconfig         |  2 +-
 configs/khadas-vim3_defconfig                 |  2 +-
 configs/khadas-vim3l_android_ab_defconfig     |  2 +-
 configs/khadas-vim3l_android_defconfig        |  2 +-
 configs/khadas-vim3l_defconfig                |  2 +-
 configs/kontron_sl28_defconfig                |  2 +-
 configs/ls1012afrdm_qspi_defconfig            |  2 +-
 configs/ls1012afrdm_tfa_defconfig             |  2 +-
 .../ls1012afrwy_qspi_SECURE_BOOT_defconfig    |  2 +-
 configs/ls1012afrwy_qspi_defconfig            |  2 +-
 configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012afrwy_tfa_defconfig             |  2 +-
 configs/ls1012aqds_qspi_defconfig             |  2 +-
 configs/ls1012aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012aqds_tfa_defconfig              |  2 +-
 configs/ls1012ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012ardb_qspi_defconfig             |  2 +-
 configs/ls1012ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012ardb_tfa_defconfig              |  2 +-
 configs/ls1021aiot_qspi_defconfig             |  2 +-
 configs/ls1021aiot_sdcard_defconfig           |  2 +-
 configs/ls1021aqds_ddr4_nor_defconfig         |  2 +-
 configs/ls1021aqds_ddr4_nor_lpuart_defconfig  |  2 +-
 configs/ls1021aqds_nand_defconfig             |  2 +-
 configs/ls1021aqds_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021aqds_nor_defconfig              |  2 +-
 configs/ls1021aqds_nor_lpuart_defconfig       |  2 +-
 configs/ls1021aqds_qspi_defconfig             |  2 +-
 configs/ls1021aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1021atsn_qspi_defconfig             |  2 +-
 configs/ls1021atsn_sdcard_defconfig           |  2 +-
 configs/ls1021atwr_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021atwr_nor_defconfig              |  2 +-
 configs/ls1021atwr_nor_lpuart_defconfig       |  2 +-
 configs/ls1021atwr_qspi_defconfig             |  2 +-
 configs/ls1021atwr_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021atwr_sdcard_qspi_defconfig      |  2 +-
 configs/ls1028aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028aqds_tfa_defconfig              |  2 +-
 configs/ls1028aqds_tfa_lpuart_defconfig       |  2 +-
 configs/ls1028ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028ardb_tfa_defconfig              |  2 +-
 configs/ls1043aqds_defconfig                  |  2 +-
 configs/ls1043aqds_lpuart_defconfig           |  2 +-
 configs/ls1043aqds_nand_defconfig             |  2 +-
 configs/ls1043aqds_nor_ddr3_defconfig         |  2 +-
 configs/ls1043aqds_qspi_defconfig             |  2 +-
 configs/ls1043aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1043aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1043aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043aqds_tfa_defconfig              |  2 +-
 configs/ls1043ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1043ardb_defconfig                  |  2 +-
 configs/ls1043ardb_nand_SECURE_BOOT_defconfig |  2 +-
 configs/ls1043ardb_nand_defconfig             |  2 +-
 configs/ls1043ardb_sdcard_defconfig           |  2 +-
 configs/ls1043ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043ardb_tfa_defconfig              |  2 +-
 configs/ls1046afrwy_tfa_defconfig             |  2 +-
 configs/ls1046aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1046aqds_defconfig                  |  2 +-
 configs/ls1046aqds_lpuart_defconfig           |  2 +-
 configs/ls1046aqds_nand_defconfig             |  2 +-
 configs/ls1046aqds_qspi_defconfig             |  2 +-
 configs/ls1046aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1046aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1046aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046aqds_tfa_defconfig              |  2 +-
 configs/ls1046ardb_emmc_defconfig             |  2 +-
 configs/ls1046ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1046ardb_qspi_defconfig             |  2 +-
 configs/ls1046ardb_qspi_spl_defconfig         |  2 +-
 configs/ls1046ardb_sdcard_defconfig           |  2 +-
 configs/ls1046ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046ardb_tfa_defconfig              |  2 +-
 configs/ls1088aqds_defconfig                  |  2 +-
 configs/ls1088aqds_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088aqds_qspi_defconfig             |  2 +-
 configs/ls1088aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1088aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088aqds_tfa_defconfig              |  2 +-
 configs/ls1088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088ardb_qspi_defconfig             |  2 +-
 configs/ls1088ardb_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1088ardb_tfa_defconfig              |  2 +-
 configs/ls2080aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080aqds_defconfig                  |  2 +-
 configs/ls2080aqds_nand_defconfig             |  2 +-
 configs/ls2080aqds_qspi_defconfig             |  2 +-
 configs/ls2080aqds_sdcard_defconfig           |  2 +-
 configs/ls2080ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080ardb_defconfig                  |  2 +-
 configs/ls2080ardb_nand_defconfig             |  2 +-
 configs/ls2081ardb_defconfig                  |  2 +-
 configs/ls2088aqds_tfa_defconfig              |  2 +-
 configs/ls2088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls2088ardb_qspi_defconfig             |  2 +-
 configs/ls2088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls2088ardb_tfa_defconfig              |  2 +-
 configs/lx2160aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160aqds_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160ardb_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_stmm_defconfig         |  2 +-
 configs/mvebu_crb_cn9130_defconfig            |  2 +-
 configs/mvebu_db_armada8k_defconfig           |  2 +-
 configs/mvebu_db_cn9130_defconfig             |  2 +-
 configs/mvebu_espressobin-88f3720_defconfig   |  2 +-
 configs/mvebu_mcbin-88f8040_defconfig         |  2 +-
 configs/mvebu_puzzle-m801-88f8040_defconfig   |  2 +-
 configs/nanopc-t4-rk3399_defconfig            |  2 +-
 configs/octeontx2_96xx_defconfig              |  2 +-
 configs/octeontx_81xx_defconfig               |  2 +-
 configs/octeontx_83xx_defconfig               |  2 +-
 configs/p3450-0000_defconfig                  |  2 +-
 configs/pinebook-pro-rk3399_defconfig         |  2 +-
 configs/qemu-x86_64_defconfig                 |  2 +-
 configs/qemu-x86_defconfig                    |  2 +-
 configs/qemu_arm64_defconfig                  |  2 +-
 configs/qemu_arm_defconfig                    |  2 +-
 configs/rcar3_salvator-x_defconfig            |  2 +-
 configs/roc-pc-mezzanine-rk3399_defconfig     |  2 +-
 configs/rock-pi-4-rk3399_defconfig            |  2 +-
 configs/rock-pi-4c-rk3399_defconfig           |  2 +-
 configs/rock-pi-n10-rk3399pro_defconfig       |  2 +-
 configs/rock960-rk3399_defconfig              |  2 +-
 configs/rockpro64-rk3399_defconfig            |  2 +-
 configs/sandbox64_defconfig                   |  2 +-
 configs/sandbox_defconfig                     |  2 +-
 configs/sandbox_flattree_defconfig            |  2 +-
 configs/sandbox_noinst_defconfig              |  2 +-
 configs/sandbox_spl_defconfig                 |  2 +-
 configs/sifive_unmatched_defconfig            |  2 +-
 configs/synquacer_developerbox_defconfig      |  2 +-
 configs/turris_mox_defconfig                  |  2 +-
 configs/turris_omnia_defconfig                |  2 +-
 doc/develop/driver-model/nvme.rst             |  1 +
 drivers/nvme/Kconfig                          | 10 +++-
 drivers/nvme/Makefile                         |  1 +
 drivers/nvme/nvme.c                           | 38 ++------------
 drivers/nvme/nvme.h                           |  3 ++
 drivers/nvme/nvme_pci.c                       | 49 +++++++++++++++++++
 147 files changed, 207 insertions(+), 177 deletions(-)
 create mode 100644 drivers/nvme/nvme_pci.c
Quoted 5 lines
Quoted 1 line

Shouldn't the vendor addition go in another patch?

I don't think that would make a ton of sense. It is intricately linked to splitting out the PCI support, as the current code grovels into pci_child_plat struct in a place that needs to be bus-agnostic.

OK.

BTW I noticed that the U-Boot logo has the wrong colours on the Macbook Air, with these patches applied to mainline.

Quoted 5 lines
Quoted 5 lines

Otherwise:

Tested on: Macbook Air M1

Regards,
Simon

#5 lore raw

Re: [PATCH 1/8] nvme: Split out PCI support

Quoted 175 lines

From: Simon Glass <[email protected]>
Date: Sat, 22 Jan 2022 06:18:18 -0700

Hi Mark,

Quoted 169 lines · Mark Kettenis
Quoted 168 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:11 -0700

Hi Mark,

Quoted 150 lines · Mark Kettenis

Apple SoCs have an integrated NVMe controller that isn't connected over a PCIe bus. In preparation for adding support for this NVMe controller, split out the PCI support into its own file. This file is selected through a new CONFIG_NVME_PCI Kconfig option, so do a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.

---
 configs/clearfog_gt_8k_defconfig              |  2 +-
 configs/firefly-rk3399_defconfig              |  2 +-
 configs/khadas-vim3_android_ab_defconfig      |  2 +-
 configs/khadas-vim3_android_defconfig         |  2 +-
 configs/khadas-vim3_defconfig                 |  2 +-
 configs/khadas-vim3l_android_ab_defconfig     |  2 +-
 configs/khadas-vim3l_android_defconfig        |  2 +-
 configs/khadas-vim3l_defconfig                |  2 +-
 configs/kontron_sl28_defconfig                |  2 +-
 configs/ls1012afrdm_qspi_defconfig            |  2 +-
 configs/ls1012afrdm_tfa_defconfig             |  2 +-
 .../ls1012afrwy_qspi_SECURE_BOOT_defconfig    |  2 +-
 configs/ls1012afrwy_qspi_defconfig            |  2 +-
 configs/ls1012afrwy_tfa_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012afrwy_tfa_defconfig             |  2 +-
 configs/ls1012aqds_qspi_defconfig             |  2 +-
 configs/ls1012aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012aqds_tfa_defconfig              |  2 +-
 configs/ls1012ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1012ardb_qspi_defconfig             |  2 +-
 configs/ls1012ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1012ardb_tfa_defconfig              |  2 +-
 configs/ls1021aiot_qspi_defconfig             |  2 +-
 configs/ls1021aiot_sdcard_defconfig           |  2 +-
 configs/ls1021aqds_ddr4_nor_defconfig         |  2 +-
 configs/ls1021aqds_ddr4_nor_lpuart_defconfig  |  2 +-
 configs/ls1021aqds_nand_defconfig             |  2 +-
 configs/ls1021aqds_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021aqds_nor_defconfig              |  2 +-
 configs/ls1021aqds_nor_lpuart_defconfig       |  2 +-
 configs/ls1021aqds_qspi_defconfig             |  2 +-
 configs/ls1021aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1021atsn_qspi_defconfig             |  2 +-
 configs/ls1021atsn_sdcard_defconfig           |  2 +-
 configs/ls1021atwr_nor_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1021atwr_nor_defconfig              |  2 +-
 configs/ls1021atwr_nor_lpuart_defconfig       |  2 +-
 configs/ls1021atwr_qspi_defconfig             |  2 +-
 configs/ls1021atwr_sdcard_ifc_defconfig       |  2 +-
 configs/ls1021atwr_sdcard_qspi_defconfig      |  2 +-
 configs/ls1028aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028aqds_tfa_defconfig              |  2 +-
 configs/ls1028aqds_tfa_lpuart_defconfig       |  2 +-
 configs/ls1028ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1028ardb_tfa_defconfig              |  2 +-
 configs/ls1043aqds_defconfig                  |  2 +-
 configs/ls1043aqds_lpuart_defconfig           |  2 +-
 configs/ls1043aqds_nand_defconfig             |  2 +-
 configs/ls1043aqds_nor_ddr3_defconfig         |  2 +-
 configs/ls1043aqds_qspi_defconfig             |  2 +-
 configs/ls1043aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1043aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1043aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043aqds_tfa_defconfig              |  2 +-
 configs/ls1043ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1043ardb_defconfig                  |  2 +-
 configs/ls1043ardb_nand_SECURE_BOOT_defconfig |  2 +-
 configs/ls1043ardb_nand_defconfig             |  2 +-
 configs/ls1043ardb_sdcard_defconfig           |  2 +-
 configs/ls1043ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1043ardb_tfa_defconfig              |  2 +-
 configs/ls1046afrwy_tfa_defconfig             |  2 +-
 configs/ls1046aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls1046aqds_defconfig                  |  2 +-
 configs/ls1046aqds_lpuart_defconfig           |  2 +-
 configs/ls1046aqds_nand_defconfig             |  2 +-
 configs/ls1046aqds_qspi_defconfig             |  2 +-
 configs/ls1046aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1046aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1046aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046aqds_tfa_defconfig              |  2 +-
 configs/ls1046ardb_emmc_defconfig             |  2 +-
 configs/ls1046ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1046ardb_qspi_defconfig             |  2 +-
 configs/ls1046ardb_qspi_spl_defconfig         |  2 +-
 configs/ls1046ardb_sdcard_defconfig           |  2 +-
 configs/ls1046ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1046ardb_tfa_defconfig              |  2 +-
 configs/ls1088aqds_defconfig                  |  2 +-
 configs/ls1088aqds_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088aqds_qspi_defconfig             |  2 +-
 configs/ls1088aqds_sdcard_ifc_defconfig       |  2 +-
 configs/ls1088aqds_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088aqds_tfa_defconfig              |  2 +-
 configs/ls1088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls1088ardb_qspi_defconfig             |  2 +-
 configs/ls1088ardb_sdcard_qspi_defconfig      |  2 +-
 configs/ls1088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls1088ardb_tfa_defconfig              |  2 +-
 configs/ls2080aqds_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080aqds_defconfig                  |  2 +-
 configs/ls2080aqds_nand_defconfig             |  2 +-
 configs/ls2080aqds_qspi_defconfig             |  2 +-
 configs/ls2080aqds_sdcard_defconfig           |  2 +-
 configs/ls2080ardb_SECURE_BOOT_defconfig      |  2 +-
 configs/ls2080ardb_defconfig                  |  2 +-
 configs/ls2080ardb_nand_defconfig             |  2 +-
 configs/ls2081ardb_defconfig                  |  2 +-
 configs/ls2088aqds_tfa_defconfig              |  2 +-
 configs/ls2088ardb_qspi_SECURE_BOOT_defconfig |  2 +-
 configs/ls2088ardb_qspi_defconfig             |  2 +-
 configs/ls2088ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/ls2088ardb_tfa_defconfig              |  2 +-
 configs/lx2160aqds_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160aqds_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_SECURE_BOOT_defconfig  |  2 +-
 configs/lx2160ardb_tfa_defconfig              |  2 +-
 configs/lx2160ardb_tfa_stmm_defconfig         |  2 +-
 configs/mvebu_crb_cn9130_defconfig            |  2 +-
 configs/mvebu_db_armada8k_defconfig           |  2 +-
 configs/mvebu_db_cn9130_defconfig             |  2 +-
 configs/mvebu_espressobin-88f3720_defconfig   |  2 +-
 configs/mvebu_mcbin-88f8040_defconfig         |  2 +-
 configs/mvebu_puzzle-m801-88f8040_defconfig   |  2 +-
 configs/nanopc-t4-rk3399_defconfig            |  2 +-
 configs/octeontx2_96xx_defconfig              |  2 +-
 configs/octeontx_81xx_defconfig               |  2 +-
 configs/octeontx_83xx_defconfig               |  2 +-
 configs/p3450-0000_defconfig                  |  2 +-
 configs/pinebook-pro-rk3399_defconfig         |  2 +-
 configs/qemu-x86_64_defconfig                 |  2 +-
 configs/qemu-x86_defconfig                    |  2 +-
 configs/qemu_arm64_defconfig                  |  2 +-
 configs/qemu_arm_defconfig                    |  2 +-
 configs/rcar3_salvator-x_defconfig            |  2 +-
 configs/roc-pc-mezzanine-rk3399_defconfig     |  2 +-
 configs/rock-pi-4-rk3399_defconfig            |  2 +-
 configs/rock-pi-4c-rk3399_defconfig           |  2 +-
 configs/rock-pi-n10-rk3399pro_defconfig       |  2 +-
 configs/rock960-rk3399_defconfig              |  2 +-
 configs/rockpro64-rk3399_defconfig            |  2 +-
 configs/sandbox64_defconfig                   |  2 +-
 configs/sandbox_defconfig                     |  2 +-
 configs/sandbox_flattree_defconfig            |  2 +-
 configs/sandbox_noinst_defconfig              |  2 +-
 configs/sandbox_spl_defconfig                 |  2 +-
 configs/sifive_unmatched_defconfig            |  2 +-
 configs/synquacer_developerbox_defconfig      |  2 +-
 configs/turris_mox_defconfig                  |  2 +-
 configs/turris_omnia_defconfig                |  2 +-
Quoted 9 lines
 doc/develop/driver-model/nvme.rst             |  1 +
 drivers/nvme/Kconfig                          | 10 +++-
 drivers/nvme/Makefile                         |  1 +
 drivers/nvme/nvme.c                           | 38 ++------------
 drivers/nvme/nvme.h                           |  3 ++
 drivers/nvme/nvme_pci.c                       | 49 +++++++++++++++++++
 147 files changed, 207 insertions(+), 177 deletions(-)
 create mode 100644 drivers/nvme/nvme_pci.c

Shouldn't the vendor addition go in another patch?

Quoted 8 lines
Quoted 3 lines

I don't think that would make a ton of sense. It is intricately linked to splitting out the PCI support, as the current code grovels into pci_child_plat struct in a place that needs to be bus-agnostic.

OK.

BTW I noticed that the U-Boot logo has the wrong colours on the Macbook Air, with these patches applied to mainline.

Yes. The code that turns on the logo was merged and I suspect that the relevant code doesn't do the proper pixel format conversions. I'll see if I can figure this out.

Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

Otherwise:

Tested on: Macbook Air M1

Regards,
Simon

#6 lore raw

PATCH 2/8 mailbox: apple: Add driver for Apple IOP mailbox

In reply to Mark Kettenis

This mailbox driver provides a communication channel with the Apple IOP controllers found on Apple SoCs. These IOP controllers are used to implement various functions such as the System Manegement Controller (SMC) and NVMe storage. It allows sending and receiving a 96-bit message over a single channel.

The header file with the struct used for mailbox messages is taken straight from Linux.

---
 arch/arm/Kconfig              |  1 +
 drivers/mailbox/Kconfig       | 11 +++++
 drivers/mailbox/Makefile      |  1 +
 drivers/mailbox/apple-mbox.c  | 92 +++++++++++++++++++++++++++++++++++
 include/linux/apple-mailbox.h | 19 ++++++++
 5 files changed, 124 insertions(+)
 create mode 100644 drivers/mailbox/apple-mbox.c
 create mode 100644 include/linux/apple-mailbox.h
arch/arm/Kconfig +1 -0
index 44954977b6..79eec3aa06 100644
@@ -934,6 +934,7 @@ config ARCH_APPLE
 	select DM
 	select DM_GPIO
 	select DM_KEYBOARD
+	select DM_MAILBOX
 	select DM_SERIAL
 	select DM_USB
 	select DM_VIDEO
drivers/mailbox/Kconfig +11 -0
index dd4b0ac0c3..73db2af0b8 100644
@@ -10,6 +10,17 @@ config DM_MAILBOX
 	  the basis of a variety of inter-process/inter-CPU communication
 	  protocols.
 
+config APPLE_MBOX
+	bool "Enable Apple IOP controller support"
+	depends on DM_MAILBOX && ARCH_APPLE
+	default y
+	help
+	  Enable support for the mailboxes that provide a comminucation
+	  channel with Apple IOP controllers integrated on Apple SoCs.
+	  These IOP controllers are used to implement various functions
+	  such as the System Management Controller (SMC) and NVMe and this
+	  driver is required to get that functionality up and running.
+
 config SANDBOX_MBOX
 	bool "Enable the sandbox mailbox test driver"
 	depends on DM_MAILBOX && SANDBOX
drivers/mailbox/Makefile +1 -0
index d2ace8cd21..59e8d0de93 100644
@@ -4,6 +4,7 @@
 #
 
 obj-$(CONFIG_$(SPL_)DM_MAILBOX) += mailbox-uclass.o
+obj-$(CONFIG_APPLE_MBOX) += apple-mbox.o
 obj-$(CONFIG_SANDBOX_MBOX) += sandbox-mbox.o
 obj-$(CONFIG_SANDBOX_MBOX) += sandbox-mbox-test.o
 obj-$(CONFIG_STM32_IPCC) += stm32-ipcc.o
drivers/mailbox/apple-mbox.c new file +92 -0

92 lines not on this page · raw message

include/linux/apple-mailbox.h new file +19 -0
new file mode 100644
index 0000000000..720fbb7029
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
+/*
+ * Apple mailbox message format
+ *
+ * Copyright (C) 2021 The Asahi Linux Contributors
+ */
+
+#ifndef _LINUX_APPLE_MAILBOX_H_
+#define _LINUX_APPLE_MAILBOX_H_
+
+#include <linux/types.h>
+
+/* encodes a single 96bit message sent over the single channel */
+struct apple_mbox_msg {
+	u64 msg0;
+	u32 msg1;
+};
+
+#endif
Signature · 1 line
2.34.1
Reviewed-by Tested-by on v1 2/8
#7 lore raw

Re: [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox

Quoted 12 lines · Mark Kettenis

This mailbox driver provides a communication channel with the Apple IOP controllers found on Apple SoCs. These IOP controllers are used to implement various functions such as the System Manegement Controller (SMC) and NVMe storage. It allows sending and receiving a 96-bit message over a single channel.

The header file with the struct used for mailbox messages is taken straight from Linux.

Quoted 9 lines
---
 arch/arm/Kconfig              |  1 +
 drivers/mailbox/Kconfig       | 11 +++++
 drivers/mailbox/Makefile      |  1 +
 drivers/mailbox/apple-mbox.c  | 92 +++++++++++++++++++++++++++++++++++
 include/linux/apple-mailbox.h | 19 ++++++++
 5 files changed, 124 insertions(+)
 create mode 100644 drivers/mailbox/apple-mbox.c
 create mode 100644 include/linux/apple-mailbox.h

Tested on: Macbook Air M1

with nit below

[..]

Quoted 6 lines
include/linux/apple-mailbox.h new file +0 -0
new file mode 100644
index 0000000000..720fbb7029

include/linux/apple-mailbox.h line 9

Quoted 9 lines
include/linux/apple-mailbox.h new file +9 -0
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
+/*
+ * Apple mailbox message format
+ *
+ * Copyright (C) 2021 The Asahi Linux Contributors
+ */
+
+#ifndef _LINUX_APPLE_MAILBOX_H_
+#define _LINUX_APPLE_MAILBOX_H_

Drop the _LINUX

Quoted 11 lines
include/linux/apple-mailbox.h new file +10 -1
+
+#include <linux/types.h>
+
+/* encodes a single 96bit message sent over the single channel */
+struct apple_mbox_msg {
+       u64 msg0;
+       u32 msg1;
+};
+
+#endif
--
Quoted 2 lines

2.34.1

Regards,
Simon

#8 lore raw

Re: [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox

Quoted 43 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:12 -0700

Quoted 12 lines · Mark Kettenis

This mailbox driver provides a communication channel with the Apple IOP controllers found on Apple SoCs. These IOP controllers are used to implement various functions such as the System Manegement Controller (SMC) and NVMe storage. It allows sending and receiving a 96-bit message over a single channel.

The header file with the struct used for mailbox messages is taken straight from Linux.

Quoted 9 lines
---
 arch/arm/Kconfig              |  1 +
 drivers/mailbox/Kconfig       | 11 +++++
 drivers/mailbox/Makefile      |  1 +
 drivers/mailbox/apple-mbox.c  | 92 +++++++++++++++++++++++++++++++++++
 include/linux/apple-mailbox.h | 19 ++++++++
 5 files changed, 124 insertions(+)
 create mode 100644 drivers/mailbox/apple-mbox.c
 create mode 100644 include/linux/apple-mailbox.h

Tested on: Macbook Air M1

with nit below

[..]

Quoted 9 lines
include/linux/apple-mailbox.h new file +3 -0
new file mode 100644
index 0000000000..720fbb7029
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
+/*
+ * Apple mailbox message format
Quoted 8 lines

include/linux/apple-mailbox.h line 9

Quoted 6 lines
include/linux/apple-mailbox.h new file +6 -0
+ *
+ * Copyright (C) 2021 The Asahi Linux Contributors
+ */
+
+#ifndef _LINUX_APPLE_MAILBOX_H_
+#define _LINUX_APPLE_MAILBOX_H_

Drop the _LINUX

Hmm, this is a straight copy of the Linux header file. Isn't it better to keep it unmodified? Most (all?) the other files in this directory return the _LINUX bit...

Quoted 17 lines
Quoted 11 lines
include/linux/apple-mailbox.h new file +10 -1
+
+#include <linux/types.h>
+
+/* encodes a single 96bit message sent over the single channel */
+struct apple_mbox_msg {
+       u64 msg0;
+       u32 msg1;
+};
+
+#endif
--
Quoted 2 lines

2.34.1

Regards,
Simon

#9 lore raw

Re: [PATCH 2/8] mailbox: apple: Add driver for Apple IOP mailbox

Hi Mark,

Quoted 53 lines · Mark Kettenis
Quoted 44 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:12 -0700

Quoted 12 lines · Mark Kettenis

This mailbox driver provides a communication channel with the Apple IOP controllers found on Apple SoCs. These IOP controllers are used to implement various functions such as the System Manegement Controller (SMC) and NVMe storage. It allows sending and receiving a 96-bit message over a single channel.

The header file with the struct used for mailbox messages is taken straight from Linux.

Quoted 9 lines
---
 arch/arm/Kconfig              |  1 +
 drivers/mailbox/Kconfig       | 11 +++++
 drivers/mailbox/Makefile      |  1 +
 drivers/mailbox/apple-mbox.c  | 92 +++++++++++++++++++++++++++++++++++
 include/linux/apple-mailbox.h | 19 ++++++++
 5 files changed, 124 insertions(+)
 create mode 100644 drivers/mailbox/apple-mbox.c
 create mode 100644 include/linux/apple-mailbox.h

Tested on: Macbook Air M1

with nit below

[..]

Quoted 9 lines
include/linux/apple-mailbox.h new file +3 -0
new file mode 100644
index 0000000000..720fbb7029
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0-only OR MIT */
+/*
+ * Apple mailbox message format
Quoted 8 lines

include/linux/apple-mailbox.h line 9

Quoted 6 lines
include/linux/apple-mailbox.h new file +6 -0
+ *
+ * Copyright (C) 2021 The Asahi Linux Contributors
+ */
+
+#ifndef _LINUX_APPLE_MAILBOX_H_
+#define _LINUX_APPLE_MAILBOX_H_

Drop the _LINUX

Quoted 3 lines

Hmm, this is a straight copy of the Linux header file. Isn't it better to keep it unmodified? Most (all?) the other files in this directory return the _LINUX bit...

Oh I missed that this was supposed to be in this dir. Yes seems OK.

Quoted 14 lines
Quoted 14 lines
Quoted 12 lines
include/linux/apple-mailbox.h new file +10 -1
+
+#include <linux/types.h>
+
+/* encodes a single 96bit message sent over the single channel */
+struct apple_mbox_msg {
+       u64 msg0;
+       u32 msg1;
+};
+
+#endif
--
Quoted 2 lines

2.34.1

Regards,
Simon

#10 lore raw

PATCH 3/8 arm: apple: Add RTKit support

In reply to Mark Kettenis

Most Apple IOPs run a firmware that is based on what Apple calls RTKit. RTKit implements a common mailbox protocol. This code provides an implementation of the AP side of this protocol, providing a function to initialize RTKit-based firmwares as well as a function to do a clean shutdown of this firmware.

---
 arch/arm/include/asm/arch-apple/rtkit.h |  11 ++
 arch/arm/mach-apple/Makefile            |   1 +
 arch/arm/mach-apple/rtkit.c             | 231 ++++++++++++++++++++++++
 3 files changed, 243 insertions(+)
 create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
 create mode 100644 arch/arm/mach-apple/rtkit.c
arch/arm/include/asm/arch-apple/rtkit.h new file +11 -0
new file mode 100644
index 0000000000..51f77f298c
@@ -0,0 +1,11 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <[email protected]>
+ */
+
+#define APPLE_RTKIT_PWR_STATE_SLEEP	0x01
+#define APPLE_RTKIT_PWR_STATE_QUIESCED	0x10
+#define APPLE_RTKIT_PWR_STATE_ON	0x20
+
+int apple_rtkit_init(struct mbox_chan *);
+int apple_rtkit_shutdown(struct mbox_chan *, int);
arch/arm/mach-apple/Makefile +1 -0
index e74a8c9df1..52f30a777b 100644
@@ -2,3 +2,4 @@
 
 obj-y += board.o
 obj-y += lowlevel_init.o
+obj-y += rtkit.o
arch/arm/mach-apple/rtkit.c new file +231 -0

231 lines not on this page · raw message

Signature · 1 line
2.34.1
#11 lore raw

Re: [PATCH 3/8] arm: apple: Add RTKit support

Hi Mark,

Quoted 7 lines · Mark Kettenis

Most Apple IOPs run a firmware that is based on what Apple calls RTKit. RTKit implements a common mailbox protocol. This code provides an implementation of the AP side of this protocol, providing a function to initialize RTKit-based firmwares as well as a function to do a clean shutdown of this firmware.

Quoted 8 lines
---
 arch/arm/include/asm/arch-apple/rtkit.h |  11 ++
 arch/arm/mach-apple/Makefile            |   1 +
 arch/arm/mach-apple/rtkit.c             | 231 ++++++++++++++++++++++++
 3 files changed, 243 insertions(+)
 create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
 create mode 100644 arch/arm/mach-apple/rtkit.c

This should be a driver.

Regards,
Simon

#12 lore raw

Re: [PATCH 3/8] arm: apple: Add RTKit support

Quoted 22 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:14 -0700

Hi Mark,

Quoted 7 lines · Mark Kettenis

Most Apple IOPs run a firmware that is based on what Apple calls RTKit. RTKit implements a common mailbox protocol. This code provides an implementation of the AP side of this protocol, providing a function to initialize RTKit-based firmwares as well as a function to do a clean shutdown of this firmware.

Quoted 8 lines
---
 arch/arm/include/asm/arch-apple/rtkit.h |  11 ++
 arch/arm/mach-apple/Makefile            |   1 +
 arch/arm/mach-apple/rtkit.c             | 231 ++++++++++++++++++++++++
 3 files changed, 243 insertions(+)
 create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
 create mode 100644 arch/arm/mach-apple/rtkit.c
Quoted 1 line

This should be a driver.

This isn't a driver but just a bit of helper code that implements the RTKit protocol. It will be used by the Apple NVMe driver and a future SMC driver. The same approach is being taken in Linux. Since it is intended to be shared, I didn't put it in the NVMe driver itself.

Reviewed-by Tested-by
#13 lore raw

Re: [PATCH 3/8] arm: apple: Add RTKit support

Hi Mark,

Quoted 23 lines · Mark Kettenis
Quoted 23 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:14 -0700

Hi Mark,

Quoted 15 lines · Mark Kettenis

Most Apple IOPs run a firmware that is based on what Apple calls RTKit. RTKit implements a common mailbox protocol. This code provides an implementation of the AP side of this protocol, providing a function to initialize RTKit-based firmwares as well as a function to do a clean shutdown of this firmware.

---
 arch/arm/include/asm/arch-apple/rtkit.h |  11 ++
 arch/arm/mach-apple/Makefile            |   1 +
 arch/arm/mach-apple/rtkit.c             | 231 ++++++++++++++++++++++++
 3 files changed, 243 insertions(+)
 create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
 create mode 100644 arch/arm/mach-apple/rtkit.c
Quoted 6 lines
Quoted 1 line

This should be a driver.

This isn't a driver but just a bit of helper code that implements the RTKit protocol. It will be used by the Apple NVMe driver and a future SMC driver. The same approach is being taken in Linux. Since it is intended to be shared, I didn't put it in the NVMe driver itself.

It just seems to set up and close down the peripheral, so this could be a driver. What happens when someone needs to change it for a new generation? Normally we would use the compatible string for that, but here you have code that might get duplicated or parameterised in another way.

Anyway if you really think this is the right way, it is OK. We do it elsewhere, iwc

Tested on: Macbook Air M1

Regards,
Simon

#14 lore raw

Re: [PATCH 3/8] arm: apple: Add RTKit support

Quoted 36 lines

From: Simon Glass <[email protected]>
Date: Sat, 22 Jan 2022 10:17:14 -0700

Hi Mark,

Quoted 23 lines · Mark Kettenis
Quoted 23 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:14 -0700

Hi Mark,

Quoted 15 lines · Mark Kettenis

Most Apple IOPs run a firmware that is based on what Apple calls RTKit. RTKit implements a common mailbox protocol. This code provides an implementation of the AP side of this protocol, providing a function to initialize RTKit-based firmwares as well as a function to do a clean shutdown of this firmware.

---
 arch/arm/include/asm/arch-apple/rtkit.h |  11 ++
 arch/arm/mach-apple/Makefile            |   1 +
 arch/arm/mach-apple/rtkit.c             | 231 ++++++++++++++++++++++++
 3 files changed, 243 insertions(+)
 create mode 100644 arch/arm/include/asm/arch-apple/rtkit.h
 create mode 100644 arch/arm/mach-apple/rtkit.c
Quoted 6 lines
Quoted 1 line

This should be a driver.

This isn't a driver but just a bit of helper code that implements the RTKit protocol. It will be used by the Apple NVMe driver and a future SMC driver. The same approach is being taken in Linux. Since it is intended to be shared, I didn't put it in the NVMe driver itself.

Quoted 5 lines

It just seems to set up and close down the peripheral, so this could be a driver. What happens when someone needs to change it for a new generation? Normally we would use the compatible string for that, but here you have code that might get duplicated or parameterised in another way.

The protocol is versioned, and apple_rtkit_init() does the appropriate version negotation. Currently we support protocol versions 11 and 12 (which seem to correspond to the firmware distributed with macOS 11 and macOS 12). But we haven't noticed any real differences between those versions yet.

Quoted 10 lines

Anyway if you really think this is the right way, it is OK. We do it elsewhere, iwc

Tested on: Macbook Air M1

Regards,
Simon

#15 lore raw

PATCH 4/8 nvme: Introduce driver ops

In reply to Mark Kettenis

The NVMe storage controller integrated on Apple SoCs deviates from the NVMe standard in two aspects. It uses a "linear" submission queue and it integrates an NVMMU that needs to be programmed for each NVMe command. Introduce driver ops such that we can set up the linear submission queue and program the NVMMU in the driver for this strange beast.

---
 drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
 drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 27 deletions(-)
drivers/nvme/nvme.c +18 -27
index be518ec20b..e2d0f9c668 100644
@@ -27,33 +27,6 @@
 #define IO_TIMEOUT		30
 #define MAX_PRP_POOL		512
 
-enum nvme_queue_id {
-	NVME_ADMIN_Q,
-	NVME_IO_Q,
-	NVME_Q_NUM,
-};
-
-/*
- * An NVM Express queue. Each device has at least two (one for admin
- * commands and one for I/O commands).
- */
-struct nvme_queue {
-	struct nvme_dev *dev;
-	struct nvme_command *sq_cmds;
-	struct nvme_completion *cqes;
-	wait_queue_head_t sq_full;
-	u32 __iomem *q_db;
-	u16 q_depth;
-	s16 cq_vector;
-	u16 sq_head;
-	u16 sq_tail;
-	u16 cq_head;
-	u16 qid;
-	u8 cq_phase;
-	u8 cqe_seen;
-	unsigned long cmdid_data[];
-};
-
 static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
 {
 	u32 bit = enabled ? NVME_CSTS_RDY : 0;
@@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
  */
 static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
 {
+	struct nvme_ops *ops;
 	u16 tail = nvmeq->sq_tail;
 
 	memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
 	flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
 			   (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
 
+	ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+	if (ops && ops->submit_cmd) {
+		ops->submit_cmd(nvmeq, cmd);
+		return;
+	}
+
 	if (++tail == nvmeq->q_depth)
 		tail = 0;
 	writel(tail, nvmeq->q_db);
@@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
 				struct nvme_command *cmd,
 				u32 *result, unsigned timeout)
 {
+	struct nvme_ops *ops;
 	u16 head = nvmeq->cq_head;
 	u16 phase = nvmeq->cq_phase;
 	u16 status;
@@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
 			return -ETIMEDOUT;
 	}
 
+	ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+	if (ops && ops->complete_cmd)
+		ops->complete_cmd(nvmeq, cmd);
+
 	status >>= 1;
 	if (status) {
 		printf("ERROR: status = %x, phase = %d, head = %d\n",
@@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
 					   int qid, int depth)
 {
+	struct nvme_ops *ops;
 	struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
 	if (!nvmeq)
 		return NULL;
@@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
 	dev->queue_count++;
 	dev->queues[qid] = nvmeq;
 
+	ops = (struct nvme_ops *)dev->udev->driver->ops;
+	if (ops && ops->alloc_queue)
+		ops->alloc_queue(nvmeq);
+
 	return nvmeq;
 
  free_queue:
@@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
 	struct nvme_id_ns *id;
 	int ret;
 
+	ndev->udev = udev;
 	INIT_LIST_HEAD(&ndev->namespaces);
 	if (readl(&ndev->bar->csts) == -1) {
 		ret = -ENODEV;
drivers/nvme/nvme.h +33 -0
index 8e9ae3c7f6..57803b43fd 100644
@@ -596,6 +596,7 @@ enum {
 
 /* Represents an NVM Express device. Each nvme_dev is a PCI function. */
 struct nvme_dev {
+	struct udevice *udev;
 	struct list_head node;
 	struct nvme_queue **queues;
 	u32 __iomem *dbs;
@@ -622,6 +623,32 @@ struct nvme_dev {
 	u32 nn;
 };
 
+enum nvme_queue_id {
+	NVME_ADMIN_Q,
+	NVME_IO_Q,
+	NVME_Q_NUM,
+};
+
+/*
+ * An NVM Express queue. Each device has at least two (one for admin
+ * commands and one for I/O commands).
+ */
+struct nvme_queue {
+	struct nvme_dev *dev;
+	struct nvme_command *sq_cmds;
+	struct nvme_completion *cqes;
+	u32 __iomem *q_db;
+	u16 q_depth;
+	s16 cq_vector;
+	u16 sq_head;
+	u16 sq_tail;
+	u16 cq_head;
+	u16 qid;
+	u8 cq_phase;
+	u8 cqe_seen;
+	unsigned long cmdid_data[];
+};
+
 /*
  * An NVM Express namespace is equivalent to a SCSI LUN.
  * Each namespace is operated as an independent "device".
@@ -636,6 +663,12 @@ struct nvme_ns {
 	u8 flbas;
 };
 
+struct nvme_ops {
+	int (*alloc_queue)(struct nvme_queue *);
+	void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
+	void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);
+};
+
 int nvme_init(struct udevice *udev);
 
 #endif /* __DRIVER_NVME_H__ */
Signature · 1 line
2.34.1
Reviewed-by Tested-by on v1 4/8
#16 lore raw

Re: [PATCH 4/8] nvme: Introduce driver ops

Hi Mark,

Quoted 8 lines · Mark Kettenis

The NVMe storage controller integrated on Apple SoCs deviates from the NVMe standard in two aspects. It uses a "linear" submission queue and it integrates an NVMMU that needs to be programmed for each NVMe command. Introduce driver ops such that we can set up the linear submission queue and program the NVMMU in the driver for this strange beast.

Quoted 5 lines
---
 drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
 drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 27 deletions(-)

Tested on: Macbook Air M1

But please see below

Quoted 118 lines
drivers/nvme/nvme.c +18 -27
index be518ec20b..e2d0f9c668 100644
@@ -27,33 +27,6 @@
 #define IO_TIMEOUT             30
 #define MAX_PRP_POOL           512
 
-enum nvme_queue_id {
-       NVME_ADMIN_Q,
-       NVME_IO_Q,
-       NVME_Q_NUM,
-};
-
-/*
- * An NVM Express queue. Each device has at least two (one for admin
- * commands and one for I/O commands).
- */
-struct nvme_queue {
-       struct nvme_dev *dev;
-       struct nvme_command *sq_cmds;
-       struct nvme_completion *cqes;
-       wait_queue_head_t sq_full;
-       u32 __iomem *q_db;
-       u16 q_depth;
-       s16 cq_vector;
-       u16 sq_head;
-       u16 sq_tail;
-       u16 cq_head;
-       u16 qid;
-       u8 cq_phase;
-       u8 cqe_seen;
-       unsigned long cmdid_data[];
-};
-
 static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
 {
        u32 bit = enabled ? NVME_CSTS_RDY : 0;
@@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
  */
 static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
 {
+       struct nvme_ops *ops;
        u16 tail = nvmeq->sq_tail;
 
        memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
        flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
                           (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
 
+       ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+       if (ops && ops->submit_cmd) {
+               ops->submit_cmd(nvmeq, cmd);
+               return;
+       }
+
        if (++tail == nvmeq->q_depth)
                tail = 0;
        writel(tail, nvmeq->q_db);
@@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
                                struct nvme_command *cmd,
                                u32 *result, unsigned timeout)
 {
+       struct nvme_ops *ops;
        u16 head = nvmeq->cq_head;
        u16 phase = nvmeq->cq_phase;
        u16 status;
@@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
                        return -ETIMEDOUT;
        }
 
+       ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+       if (ops && ops->complete_cmd)
+               ops->complete_cmd(nvmeq, cmd);
+
        status >>= 1;
        if (status) {
                printf("ERROR: status = %x, phase = %d, head = %d\n",
@@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
                                           int qid, int depth)
 {
+       struct nvme_ops *ops;
        struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
        if (!nvmeq)
                return NULL;
@@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
        dev->queue_count++;
        dev->queues[qid] = nvmeq;
 
+       ops = (struct nvme_ops *)dev->udev->driver->ops;
+       if (ops && ops->alloc_queue)
+               ops->alloc_queue(nvmeq);
+
        return nvmeq;
 
  free_queue:
@@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
        struct nvme_id_ns *id;
        int ret;
 
+       ndev->udev = udev;
        INIT_LIST_HEAD(&ndev->namespaces);
        if (readl(&ndev->bar->csts) == -1) {
                ret = -ENODEV;
drivers/nvme/nvme.h +1 -0
index 8e9ae3c7f6..57803b43fd 100644
@@ -596,6 +596,7 @@ enum {
 
 /* Represents an NVM Express device. Each nvme_dev is a PCI function. */
 struct nvme_dev {
+       struct udevice *udev;
        struct list_head node;
        struct nvme_queue **queues;
        u32 __iomem *dbs;

drivers/nvme/nvme.h struct nvme_dev { · line 626

Quoted 4 lines
drivers/nvme/nvme.h +1 -0
@@ -622,6 +623,32 @@ struct nvme_dev {
        u32 nn;
 };
 
+enum nvme_queue_id {

comment

Quoted 29 lines
drivers/nvme/nvme.h +25 -0
+       NVME_ADMIN_Q,
+       NVME_IO_Q,
+       NVME_Q_NUM,
+};
+
+/*
+ * An NVM Express queue. Each device has at least two (one for admin
+ * commands and one for I/O commands).
+ */
+struct nvme_queue {
+       struct nvme_dev *dev;
+       struct nvme_command *sq_cmds;
+       struct nvme_completion *cqes;
+       u32 __iomem *q_db;
+       u16 q_depth;
+       s16 cq_vector;
+       u16 sq_head;
+       u16 sq_tail;
+       u16 cq_head;
+       u16 qid;
+       u8 cq_phase;
+       u8 cqe_seen;
+       unsigned long cmdid_data[];
+};
+
 /*
  * An NVM Express namespace is equivalent to a SCSI LUN.
  * Each namespace is operated as an independent "device".

drivers/nvme/nvme.h struct nvme_ns { · line 669

Quoted 7 lines
drivers/nvme/nvme.h +4 -0
@@ -636,6 +663,12 @@ struct nvme_ns {
        u8 flbas;
 };
 
+struct nvme_ops {
+       int (*alloc_queue)(struct nvme_queue *);
+       void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
+       void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);

each of these needs a comment

drivers/nvme/nvme.h struct nvme_ns { · line 672

Quoted 3 lines
drivers/nvme/nvme.h +2 -0
+};
+
 int nvme_init(struct udevice *udev);

(for later, this should be dev, not udev)

Quoted 5 lines
drivers/nvme/nvme.h +0 -1
 #endif /* __DRIVER_NVME_H__ */
--

2.34.1

Regards,
Simon

#17 lore raw

Re: [PATCH 4/8] nvme: Introduce driver ops

Quoted 20 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:19 -0700

Hi Mark,

Quoted 8 lines · Mark Kettenis

The NVMe storage controller integrated on Apple SoCs deviates from the NVMe standard in two aspects. It uses a "linear" submission queue and it integrates an NVMMU that needs to be programmed for each NVMe command. Introduce driver ops such that we can set up the linear submission queue and program the NVMMU in the driver for this strange beast.

Quoted 5 lines
---
 drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
 drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 27 deletions(-)
Quoted 3 lines

Tested on: Macbook Air M1

Thanks...

Quoted 123 lines

But please see below

Quoted 121 lines
drivers/nvme/nvme.c +18 -27
index be518ec20b..e2d0f9c668 100644
@@ -27,33 +27,6 @@
 #define IO_TIMEOUT             30
 #define MAX_PRP_POOL           512
 
-enum nvme_queue_id {
-       NVME_ADMIN_Q,
-       NVME_IO_Q,
-       NVME_Q_NUM,
-};
-
-/*
- * An NVM Express queue. Each device has at least two (one for admin
- * commands and one for I/O commands).
- */
-struct nvme_queue {
-       struct nvme_dev *dev;
-       struct nvme_command *sq_cmds;
-       struct nvme_completion *cqes;
-       wait_queue_head_t sq_full;
-       u32 __iomem *q_db;
-       u16 q_depth;
-       s16 cq_vector;
-       u16 sq_head;
-       u16 sq_tail;
-       u16 cq_head;
-       u16 qid;
-       u8 cq_phase;
-       u8 cqe_seen;
-       unsigned long cmdid_data[];
-};
-
 static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
 {
        u32 bit = enabled ? NVME_CSTS_RDY : 0;
@@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
  */
 static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
 {
+       struct nvme_ops *ops;
        u16 tail = nvmeq->sq_tail;
 
        memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
        flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
                           (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
 
+       ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+       if (ops && ops->submit_cmd) {
+               ops->submit_cmd(nvmeq, cmd);
+               return;
+       }
+
        if (++tail == nvmeq->q_depth)
                tail = 0;
        writel(tail, nvmeq->q_db);
@@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
                                struct nvme_command *cmd,
                                u32 *result, unsigned timeout)
 {
+       struct nvme_ops *ops;
        u16 head = nvmeq->cq_head;
        u16 phase = nvmeq->cq_phase;
        u16 status;
@@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
                        return -ETIMEDOUT;
        }
 
+       ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+       if (ops && ops->complete_cmd)
+               ops->complete_cmd(nvmeq, cmd);
+
        status >>= 1;
        if (status) {
                printf("ERROR: status = %x, phase = %d, head = %d\n",
@@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
                                           int qid, int depth)
 {
+       struct nvme_ops *ops;
        struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
        if (!nvmeq)
                return NULL;
@@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
        dev->queue_count++;
        dev->queues[qid] = nvmeq;
 
+       ops = (struct nvme_ops *)dev->udev->driver->ops;
+       if (ops && ops->alloc_queue)
+               ops->alloc_queue(nvmeq);
+
        return nvmeq;
 
  free_queue:
@@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
        struct nvme_id_ns *id;
        int ret;
 
+       ndev->udev = udev;
        INIT_LIST_HEAD(&ndev->namespaces);
        if (readl(&ndev->bar->csts) == -1) {
                ret = -ENODEV;
drivers/nvme/nvme.h +1 -0
index 8e9ae3c7f6..57803b43fd 100644
@@ -596,6 +596,7 @@ enum {
 
 /* Represents an NVM Express device. Each nvme_dev is a PCI function. */
 struct nvme_dev {
+       struct udevice *udev;
        struct list_head node;
        struct nvme_queue **queues;
        u32 __iomem *dbs;
@@ -622,6 +623,32 @@ struct nvme_dev {
        u32 nn;
 };
 
Quoted 3 lines

drivers/nvme/nvme.h struct nvme_dev { · line 626

Quoted 1 line
drivers/nvme/nvme.h +1 -0
+enum nvme_queue_id {

comment

Hmm, I merely moved this definition. Feels a bit awkward to come up with a comment when I didn't write the bit of code in question. I can come up with something though if you insist.

Quoted 33 lines
Quoted 33 lines
drivers/nvme/nvme.h +25 -0
+       NVME_ADMIN_Q,
+       NVME_IO_Q,
+       NVME_Q_NUM,
+};
+
+/*
+ * An NVM Express queue. Each device has at least two (one for admin
+ * commands and one for I/O commands).
+ */
+struct nvme_queue {
+       struct nvme_dev *dev;
+       struct nvme_command *sq_cmds;
+       struct nvme_completion *cqes;
+       u32 __iomem *q_db;
+       u16 q_depth;
+       s16 cq_vector;
+       u16 sq_head;
+       u16 sq_tail;
+       u16 cq_head;
+       u16 qid;
+       u8 cq_phase;
+       u8 cqe_seen;
+       unsigned long cmdid_data[];
+};
+
 /*
  * An NVM Express namespace is equivalent to a SCSI LUN.
  * Each namespace is operated as an independent "device".
@@ -636,6 +663,12 @@ struct nvme_ns {
        u8 flbas;
 };
 
Quoted 6 lines

drivers/nvme/nvme.h struct nvme_ns { · line 669

Quoted 4 lines
drivers/nvme/nvme.h +4 -0
+struct nvme_ops {
+       int (*alloc_queue)(struct nvme_queue *);
+       void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
+       void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);

each of these needs a comment

Indeed. While writing the comment, I realized that setup_queue would be a better name than alloc_queue, so I changed that. Is that a small-enough change to retain your tags?

Quoted 6 lines

drivers/nvme/nvme.h struct nvme_ns { · line 672

Quoted 4 lines
drivers/nvme/nvme.h +2 -0
+};
+
 int nvme_init(struct udevice *udev);

(for later, this should be dev, not udev)

As in, this is fine for now but will need to be changed at some point in the future?

Quoted 10 lines
Quoted 6 lines
drivers/nvme/nvme.h +0 -1
 #endif /* __DRIVER_NVME_H__ */
--

2.34.1

Regards,
Simon

#18 lore raw

Re: [PATCH 4/8] nvme: Introduce driver ops

Hi Mark,

Quoted 150 lines · Mark Kettenis
Quoted 21 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:19 -0700

Hi Mark,

Quoted 8 lines · Mark Kettenis

The NVMe storage controller integrated on Apple SoCs deviates from the NVMe standard in two aspects. It uses a "linear" submission queue and it integrates an NVMMU that needs to be programmed for each NVMe command. Introduce driver ops such that we can set up the linear submission queue and program the NVMMU in the driver for this strange beast.

Quoted 5 lines
---
 drivers/nvme/nvme.c | 45 ++++++++++++++++++---------------------------
 drivers/nvme/nvme.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 27 deletions(-)
Quoted 3 lines

Tested on: Macbook Air M1

Thanks...

Quoted 123 lines

But please see below

Quoted 121 lines
drivers/nvme/nvme.c +18 -27
index be518ec20b..e2d0f9c668 100644
@@ -27,33 +27,6 @@
 #define IO_TIMEOUT             30
 #define MAX_PRP_POOL           512
 
-enum nvme_queue_id {
-       NVME_ADMIN_Q,
-       NVME_IO_Q,
-       NVME_Q_NUM,
-};
-
-/*
- * An NVM Express queue. Each device has at least two (one for admin
- * commands and one for I/O commands).
- */
-struct nvme_queue {
-       struct nvme_dev *dev;
-       struct nvme_command *sq_cmds;
-       struct nvme_completion *cqes;
-       wait_queue_head_t sq_full;
-       u32 __iomem *q_db;
-       u16 q_depth;
-       s16 cq_vector;
-       u16 sq_head;
-       u16 sq_tail;
-       u16 cq_head;
-       u16 qid;
-       u8 cq_phase;
-       u8 cqe_seen;
-       unsigned long cmdid_data[];
-};
-
 static int nvme_wait_ready(struct nvme_dev *dev, bool enabled)
 {
        u32 bit = enabled ? NVME_CSTS_RDY : 0;
@@ -167,12 +140,19 @@ static u16 nvme_read_completion_status(struct nvme_queue *nvmeq, u16 index)
  */
 static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd)
 {
+       struct nvme_ops *ops;
        u16 tail = nvmeq->sq_tail;
 
        memcpy(&nvmeq->sq_cmds[tail], cmd, sizeof(*cmd));
        flush_dcache_range((ulong)&nvmeq->sq_cmds[tail],
                           (ulong)&nvmeq->sq_cmds[tail] + sizeof(*cmd));
 
+       ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+       if (ops && ops->submit_cmd) {
+               ops->submit_cmd(nvmeq, cmd);
+               return;
+       }
+
        if (++tail == nvmeq->q_depth)
                tail = 0;
        writel(tail, nvmeq->q_db);
@@ -183,6 +163,7 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
                                struct nvme_command *cmd,
                                u32 *result, unsigned timeout)
 {
+       struct nvme_ops *ops;
        u16 head = nvmeq->cq_head;
        u16 phase = nvmeq->cq_phase;
        u16 status;
@@ -203,6 +184,10 @@ static int nvme_submit_sync_cmd(struct nvme_queue *nvmeq,
                        return -ETIMEDOUT;
        }
 
+       ops = (struct nvme_ops *)nvmeq->dev->udev->driver->ops;
+       if (ops && ops->complete_cmd)
+               ops->complete_cmd(nvmeq, cmd);
+
        status >>= 1;
        if (status) {
                printf("ERROR: status = %x, phase = %d, head = %d\n",
@@ -243,6 +228,7 @@ static int nvme_submit_admin_cmd(struct nvme_dev *dev, struct nvme_command *cmd,
 static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
                                           int qid, int depth)
 {
+       struct nvme_ops *ops;
        struct nvme_queue *nvmeq = malloc(sizeof(*nvmeq));
        if (!nvmeq)
                return NULL;
@@ -268,6 +254,10 @@ static struct nvme_queue *nvme_alloc_queue(struct nvme_dev *dev,
        dev->queue_count++;
        dev->queues[qid] = nvmeq;
 
+       ops = (struct nvme_ops *)dev->udev->driver->ops;
+       if (ops && ops->alloc_queue)
+               ops->alloc_queue(nvmeq);
+
        return nvmeq;
 
  free_queue:
@@ -821,6 +811,7 @@ int nvme_init(struct udevice *udev)
        struct nvme_id_ns *id;
        int ret;
 
+       ndev->udev = udev;
        INIT_LIST_HEAD(&ndev->namespaces);
        if (readl(&ndev->bar->csts) == -1) {
                ret = -ENODEV;
drivers/nvme/nvme.h +1 -0
index 8e9ae3c7f6..57803b43fd 100644
@@ -596,6 +596,7 @@ enum {
 
 /* Represents an NVM Express device. Each nvme_dev is a PCI function. */
 struct nvme_dev {
+       struct udevice *udev;
        struct list_head node;
        struct nvme_queue **queues;
        u32 __iomem *dbs;
@@ -622,6 +623,32 @@ struct nvme_dev {
        u32 nn;
 };
 
Quoted 7 lines
Quoted 3 lines

drivers/nvme/nvme.h struct nvme_dev { · line 626

Quoted 1 line
drivers/nvme/nvme.h +1 -0
+enum nvme_queue_id {

comment

Hmm, I merely moved this definition. Feels a bit awkward to come up with a comment when I didn't write the bit of code in question. I can come up with something though if you insist.

Oh. Well, if you can...

Quoted 41 lines
Quoted 34 lines
Quoted 34 lines
drivers/nvme/nvme.h +25 -0
+       NVME_ADMIN_Q,
+       NVME_IO_Q,
+       NVME_Q_NUM,
+};
+
+/*
+ * An NVM Express queue. Each device has at least two (one for admin
+ * commands and one for I/O commands).
+ */
+struct nvme_queue {
+       struct nvme_dev *dev;
+       struct nvme_command *sq_cmds;
+       struct nvme_completion *cqes;
+       u32 __iomem *q_db;
+       u16 q_depth;
+       s16 cq_vector;
+       u16 sq_head;
+       u16 sq_tail;
+       u16 cq_head;
+       u16 qid;
+       u8 cq_phase;
+       u8 cqe_seen;
+       unsigned long cmdid_data[];
+};
+
 /*
  * An NVM Express namespace is equivalent to a SCSI LUN.
  * Each namespace is operated as an independent "device".
@@ -636,6 +663,12 @@ struct nvme_ns {
        u8 flbas;
 };
 
Quoted 6 lines

drivers/nvme/nvme.h struct nvme_ns { · line 669

Quoted 4 lines
drivers/nvme/nvme.h +4 -0
+struct nvme_ops {
+       int (*alloc_queue)(struct nvme_queue *);
+       void (*submit_cmd)(struct nvme_queue *, struct nvme_command *);
+       void (*complete_cmd)(struct nvme_queue *, struct nvme_command *);

each of these needs a comment

Quoted 3 lines

Indeed. While writing the comment, I realized that setup_queue would be a better name than alloc_queue, so I changed that. Is that a small-enough change to retain your tags?

Yes

Quoted 10 lines
Quoted 7 lines

drivers/nvme/nvme.h struct nvme_ns { · line 672

Quoted 5 lines
drivers/nvme/nvme.h +2 -0
+};
+
 int nvme_init(struct udevice *udev);

(for later, this should be dev, not udev)

As in, this is fine for now but will need to be changed at some point in the future?

Yes, it's just a style thing.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/nvme/nvme.h +0 -1
 #endif /* __DRIVER_NVME_H__ */
--

2.34.1

Regards,
Simon

#19 lore raw

PATCH 5/8 nvme: Add shutdown function

In reply to Mark Kettenis

Add a function to disable the NVMe controller. This will be used to let the driver for the NVMe storage integrated on Apple SoCs shutdown the NVMe controller such we can shutdown the NVMe IOP controller in a clean way afterwards before handing control to the OS.

---
 drivers/nvme/nvme.c | 7 +++++++
 drivers/nvme/nvme.h | 1 +
 2 files changed, 8 insertions(+)
drivers/nvme/nvme.c +7 -0
index e2d0f9c668..3b1a5c718f 100644
@@ -896,3 +896,10 @@ free_queue:
 free_nvme:
 	return ret;
 }
+
+int nvme_shutdown(struct udevice *udev)
+{
+	struct nvme_dev *ndev = dev_get_priv(udev);
+
+	return nvme_disable_ctrl(ndev);
+}
drivers/nvme/nvme.h +1 -0
index 57803b43fd..6e39af8831 100644
@@ -670,5 +670,6 @@ struct nvme_ops {
 };
 
 int nvme_init(struct udevice *udev);
+int nvme_shutdown(struct udevice *udev);
 
 #endif /* __DRIVER_NVME_H__ */
Signature · 1 line
2.34.1
Reviewed-by Tested-by on v1 5/8
#20 lore raw

Re: [PATCH 5/8] nvme: Add shutdown function

Hi Mark,

Quoted 7 lines · Mark Kettenis

Add a function to disable the NVMe controller. This will be used to let the driver for the NVMe storage integrated on Apple SoCs shutdown the NVMe controller such we can shutdown the NVMe IOP controller in a clean way afterwards before handing control to the OS.

Quoted 5 lines
---
 drivers/nvme/nvme.c | 7 +++++++
 drivers/nvme/nvme.h | 1 +
 2 files changed, 8 insertions(+)

Tested on: Macbook Air M1

with change below

Quoted 21 lines
drivers/nvme/nvme.c +7 -0
index e2d0f9c668..3b1a5c718f 100644
@@ -896,3 +896,10 @@ free_queue:
 free_nvme:
        return ret;
 }
+
+int nvme_shutdown(struct udevice *udev)
+{
+       struct nvme_dev *ndev = dev_get_priv(udev);
+
+       return nvme_disable_ctrl(ndev);
+}
drivers/nvme/nvme.h +0 -0
index 57803b43fd..6e39af8831 100644

drivers/nvme/nvme.h struct nvme_ops { · line 673

Quoted 4 lines
drivers/nvme/nvme.h +1 -0
@@ -670,5 +670,6 @@ struct nvme_ops {
 };
 
 int nvme_init(struct udevice *udev);
+int nvme_shutdown(struct udevice *udev);

Please add a comment

Quoted 5 lines
drivers/nvme/nvme.h +0 -1
 #endif /* __DRIVER_NVME_H__ */
--

2.34.1

#21 lore raw

PATCH 6/8 power: domain: apple: Add reset support

In reply to Mark Kettenis

The power management controller found on Apple SoCs als provides a way to reset all devices within a power domain. This is needed to cleanly shutdown the NVMe controller before we hand over control to the OS.

---
 arch/arm/Kconfig                  |  1 +
 drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
 2 files changed, 73 insertions(+), 1 deletion(-)
arch/arm/Kconfig +1 -0
index 79eec3aa06..04b4a20211 100644
@@ -935,6 +935,7 @@ config ARCH_APPLE
 	select DM_GPIO
 	select DM_KEYBOARD
 	select DM_MAILBOX
+	select DM_RESET
 	select DM_SERIAL
 	select DM_USB
 	select DM_VIDEO
drivers/power/domain/apple-pmgr.c +72 -1

104 lines not on this page · raw message

Signature · 1 line
2.34.1
#22 lore raw

Re: [PATCH 6/8] power: domain: apple: Add reset support

Hi Mark,

Quoted 11 lines · Mark Kettenis

The power management controller found on Apple SoCs als provides a way to reset all devices within a power domain. This is needed to cleanly shutdown the NVMe controller before we hand over control to the OS.

---
 arch/arm/Kconfig                  |  1 +
 drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
 2 files changed, 73 insertions(+), 1 deletion(-)

This should use devicetree instead of device_bind() and be a reset driver in drivers/reset

Regards,
Simon

#23 lore raw

Re: [PATCH 6/8] power: domain: apple: Add reset support

Quoted 12 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:23 -0700

Hi Mark,

Quoted 6 lines · Mark Kettenis

The power management controller found on Apple SoCs als provides a way to reset all devices within a power domain. This is needed to cleanly shutdown the NVMe controller before we hand over control to the OS.

Quoted 8 lines
Quoted 5 lines
---
 arch/arm/Kconfig                  |  1 +
 drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
 2 files changed, 73 insertions(+), 1 deletion(-)

This should use devicetree instead of device_bind() and be a reset driver in drivers/reset

Not sure what you mean with "this should use devicetree". The reset and power domain functionality is integrated in the same hardware register and there is a single node that decsribes the device.

The Linux driver implements the power domain and reset functionality in a single driver as well. I suppose I could move the reset code into a file of its own in drivers/reset, but I don't think it would make the code easier to understand. And I'd still need to bind the reset driver explicitly in apple_pmgr_probe() as it isn't possible to automatically bind two drivers to a single device tree node as far as I can tell.

Reviewed-by Tested-by
#24 lore raw

Re: [PATCH 6/8] power: domain: apple: Add reset support

Hi Mark,

Quoted 26 lines · Mark Kettenis
Quoted 13 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:23 -0700

Hi Mark,

Quoted 6 lines · Mark Kettenis

The power management controller found on Apple SoCs als provides a way to reset all devices within a power domain. This is needed to cleanly shutdown the NVMe controller before we hand over control to the OS.

Quoted 8 lines
Quoted 5 lines
---
 arch/arm/Kconfig                  |  1 +
 drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
 2 files changed, 73 insertions(+), 1 deletion(-)

This should use devicetree instead of device_bind() and be a reset driver in drivers/reset

Not sure what you mean with "this should use devicetree". The reset and power domain functionality is integrated in the same hardware register and there is a single node that decsribes the device.

Quoted 7 lines

The Linux driver implements the power domain and reset functionality in a single driver as well. I suppose I could move the reset code into a file of its own in drivers/reset, but I don't think it would make the code easier to understand. And I'd still need to bind the reset driver explicitly in apple_pmgr_probe() as it isn't possible to automatically bind two drivers to a single device tree node as far as I can tell.

It seems odd that Linux does this sort of thing. The normal U-Boot approach would be to create a parent MFD driver with children for each uclass. Does linux do that these days?

Driver model cannot create two drivers from one device tree node, although you can create one manually later as you have done.

It seems OK to follow along with Linux, if it makes it easier to maintain. Otherwise, drivers/reset would be better for the reset driver.

Either way:

Tested on: Macbook Air M1

Regards,
Simon

#25 lore raw

Re: [PATCH 6/8] power: domain: apple: Add reset support

Quoted 2 lines

From: Simon Glass <[email protected]>
Date: Sat, 22 Jan 2022 10:17:10 -0700

Hi Simon,

Quoted 37 lines

Hi Mark,

Quoted 26 lines · Mark Kettenis
Quoted 13 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:23 -0700

Hi Mark,

Quoted 6 lines · Mark Kettenis

The power management controller found on Apple SoCs als provides a way to reset all devices within a power domain. This is needed to cleanly shutdown the NVMe controller before we hand over control to the OS.

Quoted 8 lines
Quoted 5 lines
---
 arch/arm/Kconfig                  |  1 +
 drivers/power/domain/apple-pmgr.c | 73 ++++++++++++++++++++++++++++++-
 2 files changed, 73 insertions(+), 1 deletion(-)

This should use devicetree instead of device_bind() and be a reset driver in drivers/reset

Not sure what you mean with "this should use devicetree". The reset and power domain functionality is integrated in the same hardware register and there is a single node that decsribes the device.

Quoted 7 lines

The Linux driver implements the power domain and reset functionality in a single driver as well. I suppose I could move the reset code into a file of its own in drivers/reset, but I don't think it would make the code easier to understand. And I'd still need to bind the reset driver explicitly in apple_pmgr_probe() as it isn't possible to automatically bind two drivers to a single device tree node as far as I can tell.

Quoted 3 lines

It seems odd that Linux does this sort of thing. The normal U-Boot approach would be to create a parent MFD driver with children for each uclass. Does linux do that these days?

It seems to be common for pinctrl/gpio and clk/reset drivers to be combined in Linux. Not sure if there was a pd/reset combo before the M1 support was accepted, but it is there.

MFD is typically used for higher-level functions, i.e. PMIC IC's that include both voltage regulators and an RTC for example.

Quoted 7 lines

Driver model cannot create two drivers from one device tree node, although you can create one manually later as you have done.

It seems OK to follow along with Linux, if it makes it easier to maintain. Otherwise, drivers/reset would be better for the reset driver.

Quoted 5 lines

Either way:

Tested on: Macbook Air M1

I'll keep it as-is then.

Thanks,

Mark

#26 lore raw

PATCH 7/8 nvme: apple: Add driver for Apple NVMe storage controller

In reply to Mark Kettenis

Add a driver for the NVMe storage controller integrated on Apple SoCs. This NVMe controller isn't PCI based and deviates from the NVMe standard in its implementation of the command submission queue and the integration of an NVMMU that needs to be managed. This commit tweaks the core NVMe code to support the linear command submission queue implemented by this controller. But setting up the submission queue and managing the NVMMU controller is handled by implementing the driver ops that were added in an earlier commit.

Tested-on: firefly-rk3399

---
 configs/apple_m1_defconfig |   1 +
 drivers/nvme/Kconfig       |  11 ++
 drivers/nvme/Makefile      |   1 +
 drivers/nvme/nvme_apple.c  | 233 +++++++++++++++++++++++++++++++++++++
 4 files changed, 246 insertions(+)
 create mode 100644 drivers/nvme/nvme_apple.c
configs/apple_m1_defconfig +1 -0
index cb235e4e7d..1528217b17 100644
@@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
 # CONFIG_NET is not set
 # CONFIG_MMC is not set
 CONFIG_DEBUG_UART_ANNOUNCE=y
+CONFIG_NVME_APPLE=y
 CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_XHCI_DWC3=y
 CONFIG_USB_KEYBOARD=y
drivers/nvme/Kconfig +11 -0
index 78da444c8b..0cb465160b 100644
@@ -10,6 +10,17 @@ config NVME
 	  This option enables support for NVM Express devices.
 	  It supports basic functions of NVMe (read/write).
 
+config NVME_APPLE
+	bool "Apple NVMe controller support"
+	select NVME
+	help
+	  This option enables support for the NVMe storage
+	  controller integrated on Apple SoCs.  This controller
+	  isn't PCI-based based and deviates from the NVMe
+	  standard implementation in its implementation of
+	  the command submission queue and the integration
+	  of an NVMMU that needs to be managed.
+
 config NVME_PCI
 	bool "NVM Express PCI device support"
 	depends on PCI
drivers/nvme/Makefile +1 -0
index fad9724e17..fa7b619446 100644
@@ -3,4 +3,5 @@
 # Copyright (C) 2017, Bin Meng <[email protected]>
 
 obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
 obj-$(CONFIG_NVME_PCI) += nvme_pci.o
drivers/nvme/nvme_apple.c new file +233 -0

233 lines not on this page · raw message

Signature · 1 line
2.34.1
Tested-by on v1 7/8
#27 lore raw

Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller

Hi Mark,

Quoted 13 lines · Mark Kettenis

Add a driver for the NVMe storage controller integrated on Apple SoCs. This NVMe controller isn't PCI based and deviates from the NVMe standard in its implementation of the command submission queue and the integration of an NVMMU that needs to be managed. This commit tweaks the core NVMe code to support the linear command submission queue implemented by this controller. But setting up the submission queue and managing the NVMMU controller is handled by implementing the driver ops that were added in an earlier commit.

Tested-on: firefly-rk3399

Quoted 8 lines
---
 configs/apple_m1_defconfig |   1 +
 drivers/nvme/Kconfig       |  11 ++
 drivers/nvme/Makefile      |   1 +
 drivers/nvme/nvme_apple.c  | 233 +++++++++++++++++++++++++++++++++++++
 4 files changed, 246 insertions(+)
 create mode 100644 drivers/nvme/nvme_apple.c

Tested on: Macbook Air M1

Quoted 55 lines
configs/apple_m1_defconfig +1 -0
index cb235e4e7d..1528217b17 100644
@@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
 # CONFIG_NET is not set
 # CONFIG_MMC is not set
 CONFIG_DEBUG_UART_ANNOUNCE=y
+CONFIG_NVME_APPLE=y
 CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_XHCI_DWC3=y
 CONFIG_USB_KEYBOARD=y
drivers/nvme/Kconfig +11 -0
index 78da444c8b..0cb465160b 100644
@@ -10,6 +10,17 @@ config NVME
          This option enables support for NVM Express devices.
          It supports basic functions of NVMe (read/write).
 
+config NVME_APPLE
+       bool "Apple NVMe controller support"
+       select NVME
+       help
+         This option enables support for the NVMe storage
+         controller integrated on Apple SoCs.  This controller
+         isn't PCI-based based and deviates from the NVMe
+         standard implementation in its implementation of
+         the command submission queue and the integration
+         of an NVMMU that needs to be managed.
+
 config NVME_PCI
        bool "NVM Express PCI device support"
        depends on PCI
drivers/nvme/Makefile +1 -0
index fad9724e17..fa7b619446 100644
@@ -3,4 +3,5 @@
 # Copyright (C) 2017, Bin Meng <[email protected]>
 
 obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
 obj-$(CONFIG_NVME_PCI) += nvme_pci.o
drivers/nvme/nvme_apple.c new file +4 -0
new file mode 100644
index 0000000000..b0dc8492f0
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <[email protected]>
+ */

drivers/nvme/nvme_apple.c line 14

Quoted 10 lines
drivers/nvme/nvme_apple.c new file +10 -0
+
+#include <common.h>
+#include <dm.h>
+#include <mailbox.h>
+#include <mapmem.h>
+#include "nvme.h"
+#include <reset.h>
+
+#include <asm/io.h>
+#include <asm/arch-apple/rtkit.h>

asm/arch/ should work

drivers/nvme/nvme_apple.c line 18

Quoted 4 lines
drivers/nvme/nvme_apple.c new file +4 -0
+#include <linux/iopoll.h>
+
+#undef readl_poll_timeout
+#define readl_poll_timeout readl_poll_sleep_timeout

Do we need this?

Quoted 23 lines
drivers/nvme/nvme_apple.c new file +23 -0
+
+#define REG_CPU_CTRL           0x0044
+#define  REG_CPU_CTRL_RUN      BIT(4)
+
+#define ANS_MAX_PEND_CMDS_CTRL 0x01210
+#define  ANS_MAX_QUEUE_DEPTH   64
+#define ANS_BOOT_STATUS                0x01300
+#define  ANS_BOOT_STATUS_OK    0xde71ce55
+#define ANS_MODESEL            0x01304
+#define ANS_UNKNOWN_CTRL       0x24008
+#define  ANS_PRP_NULL_CHECK    (1 << 11)
+#define ANS_LINEAR_SQ_CTRL     0x24908
+#define  ANS_LINEAR_SQ_CTRL_EN (1 << 0)
+#define ANS_ASQ_DB             0x2490c
+#define ANS_IOSQ_DB            0x24910
+#define ANS_NVMMU_NUM          0x28100
+#define ANS_NVMMU_BASE_ASQ     0x28108
+#define ANS_NVMMU_BASE_IOSQ    0x28110
+#define ANS_NVMMU_TCB_INVAL    0x28118
+#define ANS_NVMMU_TCB_STAT     0x28120
+
+#define ANS_NVMMU_TCB_SIZE     0x4000
+#define ANS_NVMMU_TCB_PITCH    0x80

drivers/nvme/nvme_apple.c line 51

Quoted 10 lines
drivers/nvme/nvme_apple.c new file +10 -0
+
+struct ans_nvmmu_tcb {
+       u8 opcode;
+       u8 flags;
+       u8 command_id;
+       u8 pad0;
+       u32 prpl_len;
+       u8 pad1[16];
+       u64 prp1;
+       u64 prp2;

Needs comments

drivers/nvme/nvme_apple.c line 64

Quoted 13 lines
drivers/nvme/nvme_apple.c new file +13 -0
+};
+
+#define ANS_NVMMU_TCB_WRITE    BIT(0)
+#define ANS_NVMMU_TCB_READ     BIT(1)
+
+struct apple_nvme_priv {
+       struct nvme_dev ndev;
+       void *base;
+       void *asc;
+       struct reset_ctl_bulk resets;
+       struct mbox_chan chan;
+       struct ans_nvmmu_tcb *tcbs[NVME_Q_NUM];
+       u32 __iomem *q_db[NVME_Q_NUM];

Needs comments

Quoted 86 lines
drivers/nvme/nvme_apple.c new file +86 -0
+};
+
+static int apple_nvme_alloc_queue(struct nvme_queue *nvmeq)
+{
+       struct apple_nvme_priv *priv =
+               container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+       struct nvme_dev *dev = nvmeq->dev;
+
+       switch (nvmeq->qid) {
+       case NVME_ADMIN_Q:
+       case NVME_IO_Q:
+               break;
+       default:
+               return -EINVAL;
+       }
+
+       priv->tcbs[nvmeq->qid] = (void *)memalign(4096, ANS_NVMMU_TCB_SIZE);
+       memset((void *)priv->tcbs[nvmeq->qid], 0, ANS_NVMMU_TCB_SIZE);
+
+       switch (nvmeq->qid) {
+       case NVME_ADMIN_Q:
+               priv->q_db[nvmeq->qid] =
+                       ((void __iomem *)dev->bar) + ANS_ASQ_DB;
+               nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
+                           ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_ASQ);
+               break;
+       case NVME_IO_Q:
+               priv->q_db[nvmeq->qid] =
+                       ((void __iomem *)dev->bar) + ANS_IOSQ_DB;
+               nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
+                           ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_IOSQ);
+               break;
+       }
+
+       return 0;
+}
+
+static void apple_nvme_submit_cmd(struct nvme_queue *nvmeq,
+                                 struct nvme_command *cmd)
+{
+       struct apple_nvme_priv *priv =
+               container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+       struct ans_nvmmu_tcb *tcb;
+       u16 tail = nvmeq->sq_tail;
+
+       tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
+       memset(tcb, 0, sizeof(*tcb));
+       tcb->opcode = cmd->common.opcode;
+       tcb->flags = ANS_NVMMU_TCB_WRITE | ANS_NVMMU_TCB_READ;
+       tcb->command_id = tail;
+       tcb->prpl_len = cmd->rw.length;
+       tcb->prp1 = cmd->common.prp1;
+       tcb->prp2 = cmd->common.prp2;
+
+       writel(tail, priv->q_db[nvmeq->qid]);
+}
+
+static void apple_nvme_complete_cmd(struct nvme_queue *nvmeq,
+                                   struct nvme_command *cmd)
+{
+       struct apple_nvme_priv *priv =
+               container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+       struct ans_nvmmu_tcb *tcb;
+       u16 tail = nvmeq->sq_tail;
+
+       tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
+       memset(tcb, 0, sizeof(*tcb));
+       writel(tail, ((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_INVAL);
+       readl(((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_STAT);
+
+       if (++tail == nvmeq->q_depth)
+               tail = 0;
+       nvmeq->sq_tail = tail;
+}
+
+static int apple_nvme_probe(struct udevice *dev)
+{
+       struct apple_nvme_priv *priv = dev_get_priv(dev);
+       fdt_addr_t addr;
+       u32 ctrl, stat;
+       int ret;
+
+       priv->base = dev_read_addr_ptr(dev);
+       if (!priv->base)
+               return -EINVAL;
+

drivers/nvme/nvme_apple.c line 160

Quoted 10 lines
drivers/nvme/nvme_apple.c new file +10 -0
+       addr = dev_read_addr_index(dev, 1);
+       if (addr == FDT_ADDR_T_NONE)
+               return -EINVAL;
+       priv->asc = map_sysmem(addr, 0);
+
+       ret = reset_get_bulk(dev, &priv->resets);
+       if (ret < 0)
+               return ret;
+
+       ret = mbox_get_by_index(dev, 0, &priv->chan);

Is the mailbox not mentioned in the device tre?

drivers/nvme/nvme_apple.c line 167

Quoted 7 lines
drivers/nvme/nvme_apple.c new file +7 -0
+       if (ret < 0)
+               return ret;
+
+       ctrl = readl(priv->asc + REG_CPU_CTRL);
+       writel(ctrl | REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
+
+       ret = apple_rtkit_init(&priv->chan);

Should probe a driver here

Quoted 67 lines
drivers/nvme/nvme_apple.c new file +66 -1
+       if (ret < 0)
+               return ret;
+
+       ret = readl_poll_timeout(priv->base + ANS_BOOT_STATUS, stat,
+                                (stat == ANS_BOOT_STATUS_OK), 100, 500000);
+       if (ret < 0) {
+               printf("%s: NVMe firmware didn't boot\n", __func__);
+               return -ETIMEDOUT;
+       }
+
+       writel(ANS_LINEAR_SQ_CTRL_EN, priv->base + ANS_LINEAR_SQ_CTRL);
+       writel(((ANS_MAX_QUEUE_DEPTH << 16) | ANS_MAX_QUEUE_DEPTH),
+              priv->base + ANS_MAX_PEND_CMDS_CTRL);
+
+       writel(readl(priv->base + ANS_UNKNOWN_CTRL) & ~ANS_PRP_NULL_CHECK,
+              priv->base + ANS_UNKNOWN_CTRL);
+
+       strcpy(priv->ndev.vendor, "Apple");
+
+       writel((ANS_NVMMU_TCB_SIZE / ANS_NVMMU_TCB_PITCH) - 1,
+              priv->base + ANS_NVMMU_NUM);
+       writel(0, priv->base + ANS_MODESEL);
+
+       priv->ndev.bar = priv->base;
+       return nvme_init(dev);
+}
+
+static int apple_nvme_remove(struct udevice *dev)
+{
+       struct apple_nvme_priv *priv = dev_get_priv(dev);
+       u32 ctrl;
+
+       nvme_shutdown(dev);
+
+       apple_rtkit_shutdown(&priv->chan, APPLE_RTKIT_PWR_STATE_SLEEP);
+
+       ctrl = readl(priv->asc + REG_CPU_CTRL);
+       writel(ctrl & ~REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
+
+       reset_assert_bulk(&priv->resets);
+       reset_deassert_bulk(&priv->resets);
+
+       return 0;
+}
+
+static const struct nvme_ops apple_nvme_ops = {
+       .alloc_queue = apple_nvme_alloc_queue,
+       .submit_cmd = apple_nvme_submit_cmd,
+       .complete_cmd = apple_nvme_complete_cmd,
+};
+
+static const struct udevice_id apple_nvme_ids[] = {
+       { .compatible = "apple,nvme-ans2" },
+       { /* sentinel */ }
+};
+
+U_BOOT_DRIVER(apple_nvme) = {
+       .name = "apple_nvme",
+       .id = UCLASS_NVME,
+       .of_match = apple_nvme_ids,
+       .priv_auto = sizeof(struct apple_nvme_priv),
+       .probe = apple_nvme_probe,
+       .remove = apple_nvme_remove,
+       .ops = &apple_nvme_ops,
+       .flags = DM_FLAG_OS_PREPARE,
+};
--
Quoted 2 lines

2.34.1

Regards,
Simon

#28 lore raw

Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller

Quoted 90 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:24 -0700

Hi Mark,

Quoted 13 lines · Mark Kettenis

Add a driver for the NVMe storage controller integrated on Apple SoCs. This NVMe controller isn't PCI based and deviates from the NVMe standard in its implementation of the command submission queue and the integration of an NVMMU that needs to be managed. This commit tweaks the core NVMe code to support the linear command submission queue implemented by this controller. But setting up the submission queue and managing the NVMMU controller is handled by implementing the driver ops that were added in an earlier commit.

Tested-on: firefly-rk3399

Quoted 8 lines
---
 configs/apple_m1_defconfig |   1 +
 drivers/nvme/Kconfig       |  11 ++
 drivers/nvme/Makefile      |   1 +
 drivers/nvme/nvme_apple.c  | 233 +++++++++++++++++++++++++++++++++++++
 4 files changed, 246 insertions(+)
 create mode 100644 drivers/nvme/nvme_apple.c

Tested on: Macbook Air M1

Quoted 59 lines
configs/apple_m1_defconfig +1 -0
index cb235e4e7d..1528217b17 100644
@@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
 # CONFIG_NET is not set
 # CONFIG_MMC is not set
 CONFIG_DEBUG_UART_ANNOUNCE=y
+CONFIG_NVME_APPLE=y
 CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_XHCI_DWC3=y
 CONFIG_USB_KEYBOARD=y
drivers/nvme/Kconfig +11 -0
index 78da444c8b..0cb465160b 100644
@@ -10,6 +10,17 @@ config NVME
          This option enables support for NVM Express devices.
          It supports basic functions of NVMe (read/write).
 
+config NVME_APPLE
+       bool "Apple NVMe controller support"
+       select NVME
+       help
+         This option enables support for the NVMe storage
+         controller integrated on Apple SoCs.  This controller
+         isn't PCI-based based and deviates from the NVMe
+         standard implementation in its implementation of
+         the command submission queue and the integration
+         of an NVMMU that needs to be managed.
+
 config NVME_PCI
        bool "NVM Express PCI device support"
        depends on PCI
drivers/nvme/Makefile +1 -0
index fad9724e17..fa7b619446 100644
@@ -3,4 +3,5 @@
 # Copyright (C) 2017, Bin Meng <[email protected]>
 
 obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
 obj-$(CONFIG_NVME_PCI) += nvme_pci.o
drivers/nvme/nvme_apple.c new file +8 -0
new file mode 100644
index 0000000000..b0dc8492f0
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <[email protected]>
+ */
+
+#include <common.h>
+#include <dm.h>
+#include <mailbox.h>
Quoted 8 lines

drivers/nvme/nvme_apple.c line 14

Quoted 6 lines
drivers/nvme/nvme_apple.c new file +6 -0
+#include <mapmem.h>
+#include "nvme.h"
+#include <reset.h>
+
+#include <asm/io.h>
+#include <asm/arch-apple/rtkit.h>

asm/arch/ should work

It doesn't. For some reason I end up with

arch/arm/include/asm/arch -> arch-m1

But this rtkit stuff is expected to be generic and apply to all Apple SoCs, not just the M1.

Quoted 7 lines

drivers/nvme/nvme_apple.c line 18

Quoted 5 lines
drivers/nvme/nvme_apple.c new file +4 -0
+#include <linux/iopoll.h>
+
+#undef readl_poll_timeout
+#define readl_poll_timeout readl_poll_sleep_timeout

Do we need this?

No we don't. Fixed.

Quoted 27 lines
Quoted 27 lines
drivers/nvme/nvme_apple.c new file +27 -0
+
+#define REG_CPU_CTRL           0x0044
+#define  REG_CPU_CTRL_RUN      BIT(4)
+
+#define ANS_MAX_PEND_CMDS_CTRL 0x01210
+#define  ANS_MAX_QUEUE_DEPTH   64
+#define ANS_BOOT_STATUS                0x01300
+#define  ANS_BOOT_STATUS_OK    0xde71ce55
+#define ANS_MODESEL            0x01304
+#define ANS_UNKNOWN_CTRL       0x24008
+#define  ANS_PRP_NULL_CHECK    (1 << 11)
+#define ANS_LINEAR_SQ_CTRL     0x24908
+#define  ANS_LINEAR_SQ_CTRL_EN (1 << 0)
+#define ANS_ASQ_DB             0x2490c
+#define ANS_IOSQ_DB            0x24910
+#define ANS_NVMMU_NUM          0x28100
+#define ANS_NVMMU_BASE_ASQ     0x28108
+#define ANS_NVMMU_BASE_IOSQ    0x28110
+#define ANS_NVMMU_TCB_INVAL    0x28118
+#define ANS_NVMMU_TCB_STAT     0x28120
+
+#define ANS_NVMMU_TCB_SIZE     0x4000
+#define ANS_NVMMU_TCB_PITCH    0x80
+
+struct ans_nvmmu_tcb {
+       u8 opcode;
+       u8 flags;
Quoted 8 lines

drivers/nvme/nvme_apple.c line 51

Quoted 6 lines
drivers/nvme/nvme_apple.c new file +6 -0
+       u8 command_id;
+       u8 pad0;
+       u32 prpl_len;
+       u8 pad1[16];
+       u64 prp1;
+       u64 prp2;

Needs comments

I put a comment at the top explaining my limited understanding of what the TCBs are used for.

Quoted 16 lines
Quoted 4 lines
drivers/nvme/nvme_apple.c new file +3 -0
+};
+
+#define ANS_NVMMU_TCB_WRITE    BIT(0)

drivers/nvme/nvme_apple.c line 64

Quoted 10 lines
drivers/nvme/nvme_apple.c new file +10 -0
+#define ANS_NVMMU_TCB_READ     BIT(1)
+
+struct apple_nvme_priv {
+       struct nvme_dev ndev;
+       void *base;
+       void *asc;
+       struct reset_ctl_bulk resets;
+       struct mbox_chan chan;
+       struct ans_nvmmu_tcb *tcbs[NVME_Q_NUM];
+       u32 __iomem *q_db[NVME_Q_NUM];

Needs comments

Done, although they give me a captain obvious vibe...

Quoted 90 lines
Quoted 90 lines
drivers/nvme/nvme_apple.c new file +90 -0
+};
+
+static int apple_nvme_alloc_queue(struct nvme_queue *nvmeq)
+{
+       struct apple_nvme_priv *priv =
+               container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+       struct nvme_dev *dev = nvmeq->dev;
+
+       switch (nvmeq->qid) {
+       case NVME_ADMIN_Q:
+       case NVME_IO_Q:
+               break;
+       default:
+               return -EINVAL;
+       }
+
+       priv->tcbs[nvmeq->qid] = (void *)memalign(4096, ANS_NVMMU_TCB_SIZE);
+       memset((void *)priv->tcbs[nvmeq->qid], 0, ANS_NVMMU_TCB_SIZE);
+
+       switch (nvmeq->qid) {
+       case NVME_ADMIN_Q:
+               priv->q_db[nvmeq->qid] =
+                       ((void __iomem *)dev->bar) + ANS_ASQ_DB;
+               nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
+                           ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_ASQ);
+               break;
+       case NVME_IO_Q:
+               priv->q_db[nvmeq->qid] =
+                       ((void __iomem *)dev->bar) + ANS_IOSQ_DB;
+               nvme_writeq((ulong)priv->tcbs[nvmeq->qid],
+                           ((void __iomem *)dev->bar) + ANS_NVMMU_BASE_IOSQ);
+               break;
+       }
+
+       return 0;
+}
+
+static void apple_nvme_submit_cmd(struct nvme_queue *nvmeq,
+                                 struct nvme_command *cmd)
+{
+       struct apple_nvme_priv *priv =
+               container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+       struct ans_nvmmu_tcb *tcb;
+       u16 tail = nvmeq->sq_tail;
+
+       tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
+       memset(tcb, 0, sizeof(*tcb));
+       tcb->opcode = cmd->common.opcode;
+       tcb->flags = ANS_NVMMU_TCB_WRITE | ANS_NVMMU_TCB_READ;
+       tcb->command_id = tail;
+       tcb->prpl_len = cmd->rw.length;
+       tcb->prp1 = cmd->common.prp1;
+       tcb->prp2 = cmd->common.prp2;
+
+       writel(tail, priv->q_db[nvmeq->qid]);
+}
+
+static void apple_nvme_complete_cmd(struct nvme_queue *nvmeq,
+                                   struct nvme_command *cmd)
+{
+       struct apple_nvme_priv *priv =
+               container_of(nvmeq->dev, struct apple_nvme_priv, ndev);
+       struct ans_nvmmu_tcb *tcb;
+       u16 tail = nvmeq->sq_tail;
+
+       tcb = ((void *)priv->tcbs[nvmeq->qid]) + tail * ANS_NVMMU_TCB_PITCH;
+       memset(tcb, 0, sizeof(*tcb));
+       writel(tail, ((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_INVAL);
+       readl(((void __iomem *)nvmeq->dev->bar) + ANS_NVMMU_TCB_STAT);
+
+       if (++tail == nvmeq->q_depth)
+               tail = 0;
+       nvmeq->sq_tail = tail;
+}
+
+static int apple_nvme_probe(struct udevice *dev)
+{
+       struct apple_nvme_priv *priv = dev_get_priv(dev);
+       fdt_addr_t addr;
+       u32 ctrl, stat;
+       int ret;
+
+       priv->base = dev_read_addr_ptr(dev);
+       if (!priv->base)
+               return -EINVAL;
+
+       addr = dev_read_addr_index(dev, 1);
+       if (addr == FDT_ADDR_T_NONE)
+               return -EINVAL;
+       priv->asc = map_sysmem(addr, 0);
Quoted 8 lines

drivers/nvme/nvme_apple.c line 160

Quoted 6 lines
drivers/nvme/nvme_apple.c new file +6 -0
+
+       ret = reset_get_bulk(dev, &priv->resets);
+       if (ret < 0)
+               return ret;
+
+       ret = mbox_get_by_index(dev, 0, &priv->chan);

Is the mailbox not mentioned in the device tre?

There is an "mboxes" property, which is what mbox_get_by_index() uses. But there is only one mbox so it isn't named.

Quoted 10 lines

drivers/nvme/nvme_apple.c line 167

Quoted 8 lines
drivers/nvme/nvme_apple.c new file +7 -0
+       if (ret < 0)
+               return ret;
+
+       ctrl = readl(priv->asc + REG_CPU_CTRL);
+       writel(ctrl | REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
+
+       ret = apple_rtkit_init(&priv->chan);

Should probe a driver here

See my reply to PATCH 3/8 in this series.

Quoted 73 lines
Quoted 67 lines
drivers/nvme/nvme_apple.c new file +66 -1
+       if (ret < 0)
+               return ret;
+
+       ret = readl_poll_timeout(priv->base + ANS_BOOT_STATUS, stat,
+                                (stat == ANS_BOOT_STATUS_OK), 100, 500000);
+       if (ret < 0) {
+               printf("%s: NVMe firmware didn't boot\n", __func__);
+               return -ETIMEDOUT;
+       }
+
+       writel(ANS_LINEAR_SQ_CTRL_EN, priv->base + ANS_LINEAR_SQ_CTRL);
+       writel(((ANS_MAX_QUEUE_DEPTH << 16) | ANS_MAX_QUEUE_DEPTH),
+              priv->base + ANS_MAX_PEND_CMDS_CTRL);
+
+       writel(readl(priv->base + ANS_UNKNOWN_CTRL) & ~ANS_PRP_NULL_CHECK,
+              priv->base + ANS_UNKNOWN_CTRL);
+
+       strcpy(priv->ndev.vendor, "Apple");
+
+       writel((ANS_NVMMU_TCB_SIZE / ANS_NVMMU_TCB_PITCH) - 1,
+              priv->base + ANS_NVMMU_NUM);
+       writel(0, priv->base + ANS_MODESEL);
+
+       priv->ndev.bar = priv->base;
+       return nvme_init(dev);
+}
+
+static int apple_nvme_remove(struct udevice *dev)
+{
+       struct apple_nvme_priv *priv = dev_get_priv(dev);
+       u32 ctrl;
+
+       nvme_shutdown(dev);
+
+       apple_rtkit_shutdown(&priv->chan, APPLE_RTKIT_PWR_STATE_SLEEP);
+
+       ctrl = readl(priv->asc + REG_CPU_CTRL);
+       writel(ctrl & ~REG_CPU_CTRL_RUN, priv->asc + REG_CPU_CTRL);
+
+       reset_assert_bulk(&priv->resets);
+       reset_deassert_bulk(&priv->resets);
+
+       return 0;
+}
+
+static const struct nvme_ops apple_nvme_ops = {
+       .alloc_queue = apple_nvme_alloc_queue,
+       .submit_cmd = apple_nvme_submit_cmd,
+       .complete_cmd = apple_nvme_complete_cmd,
+};
+
+static const struct udevice_id apple_nvme_ids[] = {
+       { .compatible = "apple,nvme-ans2" },
+       { /* sentinel */ }
+};
+
+U_BOOT_DRIVER(apple_nvme) = {
+       .name = "apple_nvme",
+       .id = UCLASS_NVME,
+       .of_match = apple_nvme_ids,
+       .priv_auto = sizeof(struct apple_nvme_priv),
+       .probe = apple_nvme_probe,
+       .remove = apple_nvme_remove,
+       .ops = &apple_nvme_ops,
+       .flags = DM_FLAG_OS_PREPARE,
+};
--
Quoted 2 lines

2.34.1

Regards,
Simon

#29 lore raw

Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller

Hi Mark,

Quoted 100 lines · Mark Kettenis
Quoted 91 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:24 -0700

Hi Mark,

Quoted 13 lines · Mark Kettenis

Add a driver for the NVMe storage controller integrated on Apple SoCs. This NVMe controller isn't PCI based and deviates from the NVMe standard in its implementation of the command submission queue and the integration of an NVMMU that needs to be managed. This commit tweaks the core NVMe code to support the linear command submission queue implemented by this controller. But setting up the submission queue and managing the NVMMU controller is handled by implementing the driver ops that were added in an earlier commit.

Tested-on: firefly-rk3399

Quoted 8 lines
---
 configs/apple_m1_defconfig |   1 +
 drivers/nvme/Kconfig       |  11 ++
 drivers/nvme/Makefile      |   1 +
 drivers/nvme/nvme_apple.c  | 233 +++++++++++++++++++++++++++++++++++++
 4 files changed, 246 insertions(+)
 create mode 100644 drivers/nvme/nvme_apple.c

Tested on: Macbook Air M1

Quoted 59 lines
configs/apple_m1_defconfig +1 -0
index cb235e4e7d..1528217b17 100644
@@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
 # CONFIG_NET is not set
 # CONFIG_MMC is not set
 CONFIG_DEBUG_UART_ANNOUNCE=y
+CONFIG_NVME_APPLE=y
 CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_XHCI_DWC3=y
 CONFIG_USB_KEYBOARD=y
drivers/nvme/Kconfig +11 -0
index 78da444c8b..0cb465160b 100644
@@ -10,6 +10,17 @@ config NVME
          This option enables support for NVM Express devices.
          It supports basic functions of NVMe (read/write).
 
+config NVME_APPLE
+       bool "Apple NVMe controller support"
+       select NVME
+       help
+         This option enables support for the NVMe storage
+         controller integrated on Apple SoCs.  This controller
+         isn't PCI-based based and deviates from the NVMe
+         standard implementation in its implementation of
+         the command submission queue and the integration
+         of an NVMMU that needs to be managed.
+
 config NVME_PCI
        bool "NVM Express PCI device support"
        depends on PCI
drivers/nvme/Makefile +1 -0
index fad9724e17..fa7b619446 100644
@@ -3,4 +3,5 @@
 # Copyright (C) 2017, Bin Meng <[email protected]>
 
 obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
 obj-$(CONFIG_NVME_PCI) += nvme_pci.o
drivers/nvme/nvme_apple.c new file +8 -0
new file mode 100644
index 0000000000..b0dc8492f0
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <[email protected]>
+ */
+
+#include <common.h>
+#include <dm.h>
+#include <mailbox.h>
Quoted 8 lines

drivers/nvme/nvme_apple.c line 14

Quoted 6 lines
drivers/nvme/nvme_apple.c new file +6 -0
+#include <mapmem.h>
+#include "nvme.h"
+#include <reset.h>
+
+#include <asm/io.h>
+#include <asm/arch-apple/rtkit.h>

asm/arch/ should work

Quoted 6 lines

It doesn't. For some reason I end up with

arch/arm/include/asm/arch -> arch-m1

But this rtkit stuff is expected to be generic and apply to all Apple SoCs, not just the M1.

Hmm so you plan to have an arch/m2 as well? Can we not put everything in arch/apple then? We don't normally have arch directories for different generations unless they are completely different.

[..]

Regards,
Simon

#30 lore raw

Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller

Quoted 108 lines

From: Simon Glass <[email protected]>
Date: Sat, 22 Jan 2022 10:17:08 -0700

Hi Mark,

Quoted 102 lines · Mark Kettenis
Quoted 91 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:24 -0700

Hi Mark,

Quoted 13 lines · Mark Kettenis

Add a driver for the NVMe storage controller integrated on Apple SoCs. This NVMe controller isn't PCI based and deviates from the NVMe standard in its implementation of the command submission queue and the integration of an NVMMU that needs to be managed. This commit tweaks the core NVMe code to support the linear command submission queue implemented by this controller. But setting up the submission queue and managing the NVMMU controller is handled by implementing the driver ops that were added in an earlier commit.

Tested-on: firefly-rk3399

Quoted 8 lines
---
 configs/apple_m1_defconfig |   1 +
 drivers/nvme/Kconfig       |  11 ++
 drivers/nvme/Makefile      |   1 +
 drivers/nvme/nvme_apple.c  | 233 +++++++++++++++++++++++++++++++++++++
 4 files changed, 246 insertions(+)
 create mode 100644 drivers/nvme/nvme_apple.c

Tested on: Macbook Air M1

Quoted 59 lines
configs/apple_m1_defconfig +1 -0
index cb235e4e7d..1528217b17 100644
@@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
 # CONFIG_NET is not set
 # CONFIG_MMC is not set
 CONFIG_DEBUG_UART_ANNOUNCE=y
+CONFIG_NVME_APPLE=y
 CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_XHCI_DWC3=y
 CONFIG_USB_KEYBOARD=y
drivers/nvme/Kconfig +11 -0
index 78da444c8b..0cb465160b 100644
@@ -10,6 +10,17 @@ config NVME
          This option enables support for NVM Express devices.
          It supports basic functions of NVMe (read/write).
 
+config NVME_APPLE
+       bool "Apple NVMe controller support"
+       select NVME
+       help
+         This option enables support for the NVMe storage
+         controller integrated on Apple SoCs.  This controller
+         isn't PCI-based based and deviates from the NVMe
+         standard implementation in its implementation of
+         the command submission queue and the integration
+         of an NVMMU that needs to be managed.
+
 config NVME_PCI
        bool "NVM Express PCI device support"
        depends on PCI
drivers/nvme/Makefile +1 -0
index fad9724e17..fa7b619446 100644
@@ -3,4 +3,5 @@
 # Copyright (C) 2017, Bin Meng <[email protected]>
 
 obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
 obj-$(CONFIG_NVME_PCI) += nvme_pci.o
drivers/nvme/nvme_apple.c new file +8 -0
new file mode 100644
index 0000000000..b0dc8492f0
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <[email protected]>
+ */
+
+#include <common.h>
+#include <dm.h>
+#include <mailbox.h>
Quoted 8 lines

drivers/nvme/nvme_apple.c line 14

Quoted 6 lines
drivers/nvme/nvme_apple.c new file +6 -0
+#include <mapmem.h>
+#include "nvme.h"
+#include <reset.h>
+
+#include <asm/io.h>
+#include <asm/arch-apple/rtkit.h>

asm/arch/ should work

It doesn't. For some reason I end up with

Quoted 8 lines
Quoted 4 lines

arch/arm/include/asm/arch -> arch-m1

But this rtkit stuff is expected to be generic and apply to all Apple SoCs, not just the M1.

Hmm so you plan to have an arch/m2 as well? Can we not put everything in arch/apple then? We don't normally have arch directories for different generations unless they are completely different.

No, I think a single arch/arch-apple is better. That isolates us from whatever silly marketing names Apple comes up with for their SoCs. The goal is to have a single U-Boot binary that supports multiple generations of the SoC.

That means I have to set SYS_SOC to "apple" instead of "m1" and move the existing uart.h file into arch/arm/include/arch/arch-apple/ isn't it?

I can add a diff that does this to the series.

#31 lore raw

Re: [PATCH 7/8] nvme: apple: Add driver for Apple NVMe storage controller

Hi Mark,

Quoted 114 lines · Mark Kettenis
Quoted 114 lines

From: Simon Glass <[email protected]>
Date: Sat, 22 Jan 2022 10:17:08 -0700

Hi Mark,

Quoted 106 lines · Mark Kettenis
Quoted 91 lines

From: Simon Glass <[email protected]>
Date: Fri, 21 Jan 2022 18:40:24 -0700

Hi Mark,

Quoted 13 lines · Mark Kettenis

Add a driver for the NVMe storage controller integrated on Apple SoCs. This NVMe controller isn't PCI based and deviates from the NVMe standard in its implementation of the command submission queue and the integration of an NVMMU that needs to be managed. This commit tweaks the core NVMe code to support the linear command submission queue implemented by this controller. But setting up the submission queue and managing the NVMMU controller is handled by implementing the driver ops that were added in an earlier commit.

Tested-on: firefly-rk3399

Quoted 8 lines
---
 configs/apple_m1_defconfig |   1 +
 drivers/nvme/Kconfig       |  11 ++
 drivers/nvme/Makefile      |   1 +
 drivers/nvme/nvme_apple.c  | 233 +++++++++++++++++++++++++++++++++++++
 4 files changed, 246 insertions(+)
 create mode 100644 drivers/nvme/nvme_apple.c

Tested on: Macbook Air M1

Quoted 59 lines
configs/apple_m1_defconfig +1 -0
index cb235e4e7d..1528217b17 100644
@@ -11,6 +11,7 @@ CONFIG_DISPLAY_BOARDINFO_LATE=y
 # CONFIG_NET is not set
 # CONFIG_MMC is not set
 CONFIG_DEBUG_UART_ANNOUNCE=y
+CONFIG_NVME_APPLE=y
 CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_XHCI_DWC3=y
 CONFIG_USB_KEYBOARD=y
drivers/nvme/Kconfig +11 -0
index 78da444c8b..0cb465160b 100644
@@ -10,6 +10,17 @@ config NVME
          This option enables support for NVM Express devices.
          It supports basic functions of NVMe (read/write).
 
+config NVME_APPLE
+       bool "Apple NVMe controller support"
+       select NVME
+       help
+         This option enables support for the NVMe storage
+         controller integrated on Apple SoCs.  This controller
+         isn't PCI-based based and deviates from the NVMe
+         standard implementation in its implementation of
+         the command submission queue and the integration
+         of an NVMMU that needs to be managed.
+
 config NVME_PCI
        bool "NVM Express PCI device support"
        depends on PCI
drivers/nvme/Makefile +1 -0
index fad9724e17..fa7b619446 100644
@@ -3,4 +3,5 @@
 # Copyright (C) 2017, Bin Meng <[email protected]>
 
 obj-y += nvme-uclass.o nvme.o nvme_show.o
+obj-$(CONFIG_NVME_APPLE) += nvme_apple.o
 obj-$(CONFIG_NVME_PCI) += nvme_pci.o
drivers/nvme/nvme_apple.c new file +8 -0
new file mode 100644
index 0000000000..b0dc8492f0
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * (C) Copyright 2021 Mark Kettenis <[email protected]>
+ */
+
+#include <common.h>
+#include <dm.h>
+#include <mailbox.h>
Quoted 8 lines

drivers/nvme/nvme_apple.c line 14

Quoted 6 lines
drivers/nvme/nvme_apple.c new file +6 -0
+#include <mapmem.h>
+#include "nvme.h"
+#include <reset.h>
+
+#include <asm/io.h>
+#include <asm/arch-apple/rtkit.h>

asm/arch/ should work

It doesn't. For some reason I end up with

arch/arm/include/asm/arch -> arch-m1

But this rtkit stuff is expected to be generic and apply to all Apple SoCs, not just the M1.

Quoted 8 lines
Quoted 3 lines

Hmm so you plan to have an arch/m2 as well? Can we not put everything in arch/apple then? We don't normally have arch directories for different generations unless they are completely different.

No, I think a single arch/arch-apple is better. That isolates us from whatever silly marketing names Apple comes up with for their SoCs. The goal is to have a single U-Boot binary that supports multiple generations of the SoC.

Sounds good

Quoted 4 lines

That means I have to set SYS_SOC to "apple" instead of "m1" and move the existing uart.h file into arch/arm/include/arch/arch-apple/ isn't it?

Yes that seems right.

Quoted 2 lines

I can add a diff that does this to the series.

Regards,
Simon

#32 lore raw

PATCH 8/8 configs: apple: Add NVMe boot target

In reply to Mark Kettenis

Add a boot target for NVMe such that we can boot from NVMe.

---
 include/configs/apple.h | 7 +++++++
 1 file changed, 7 insertions(+)
include/configs/apple.h +7 -0
index 3e5fb495f1..47faad8150 100644
@@ -13,6 +13,12 @@
 	"fdt_addr_r=0x960100000\0" \
 	"kernel_addr_r=0x960200000\0"
 
+#if CONFIG_IS_ENABLED(CMD_NVME)
+	#define BOOT_TARGET_NVME(func) func(NVME, nvme, 0)
+#else
+	#define BOOT_TARGET_NVME(func)
+#endif
+
 #if CONFIG_IS_ENABLED(CMD_USB)
 	#define BOOT_TARGET_USB(func) func(USB, usb, 0)
 #else
@@ -20,6 +26,7 @@
 #endif
 
 #define BOOT_TARGET_DEVICES(func) \
+	BOOT_TARGET_NVME(func) \
 	BOOT_TARGET_USB(func)
 
 #include <config_distro_bootcmd.h>
Signature · 1 line
2.34.1