Skip to contentSkip to conversation

Add PMEM support for RISC-V

PATCH Applied Anup Patel 4 participants 10 messages Nov 14, 2022 – Dec 9, 2022 RISC-V

lore
Share & tools
lore

Anup Patel

author
#0 lore raw

v6 cover Add PMEM support for RISC-V

The Linux NVDIMM PEM drivers require arch support to map and access the persistent memory device. This series adds RISC-V PMEM support using recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required PMEM support for Linux RISC-V.

These patches can also be found in riscv_pmem_v6 branch at:
https://github.com/avpatel/linux.git

Changes since v5:
- Rebased on Linux-6.1-rc5
- Directly add pmem.o in obj-y of PATCH3
- Dropped PATCH1 since this is already merged.

Changes since v4:
- Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()

Changes since v3:
- Pickup correct version of Drew's patch as PATCH1

Changes since v2:
- Rebased on Linux-6.1-rc1
- Replaced PATCH1 with the patch proposed by Drew

Changes since v1:
- Fix error reported by test bot
https://lore.kernel.org/all/[email protected]/

Anup Patel (3):
RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
RISC-V: Implement arch specific PMEM APIs
RISC-V: Enable PMEM drivers

 arch/riscv/Kconfig           |  1 +
 arch/riscv/configs/defconfig |  1 +
 arch/riscv/include/asm/io.h  |  5 +++++
 arch/riscv/mm/Makefile       |  1 +
 arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
 5 files changed, 29 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH v6 1/3 RISC-V: Fix MEMREMAP_WB for systems with Svpbmt

Currently, the memremap() called with MEMREMAP_WB maps memory using the generic ioremap() function which breaks on system with Svpbmt because memory mapped using _PAGE_IOREMAP page attributes is treated as strongly-ordered non-cacheable IO memory.

To address this, we implement RISC-V specific arch_memremap_wb() which maps memory using _PAGE_KERNEL page attributes resulting in write-back cacheable mapping on systems with Svpbmt.

Fixes ff689fd21cb1 ("riscv: add RISC-V Svpbmt extension support")
---
 arch/riscv/include/asm/io.h | 5 +++++
 1 file changed, 5 insertions(+)
arch/riscv/include/asm/io.h +5 -0
index 92080a227937..42497d487a17 100644
@@ -135,4 +135,9 @@ __io_writes_outs(outs, u64, q, __io_pbr(), __io_paw())
 
 #include <asm-generic/io.h>
 
+#ifdef CONFIG_MMU
+#define arch_memremap_wb(addr, size)	\
+	((__force void *)ioremap_prot((addr), (size), _PAGE_KERNEL))
+#endif
+
 #endif /* _ASM_RISCV_IO_H */
Signature · 1 line
2.34.1
author
#2 lore raw

PATCH v6 2/3 RISC-V: Implement arch specific PMEM APIs

In reply to Anup Patel

The NVDIMM PMEM driver expects arch specific APIs for cache maintenance and if arch does not provide these APIs then NVDIMM PMEM driver will always use MEMREMAP_WT to map persistent memory which in-turn maps as UC memory type defined by the RISC-V Svpbmt specification.

Now that the Svpbmt and Zicbom support is available in RISC-V kernel, we implement PMEM APIs using ALT_CMO_OP() macros so that the NVDIMM PMEM driver can use MEMREMAP_WB to map persistent memory.

---
 arch/riscv/Kconfig     |  1 +
 arch/riscv/mm/Makefile |  1 +
 arch/riscv/mm/pmem.c   | 21 +++++++++++++++++++++
 3 files changed, 23 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c
arch/riscv/Kconfig +1 -0
index fa78595a6089..c0e22648bd16 100644
@@ -25,6 +25,7 @@ config RISCV
 	select ARCH_HAS_GIGANTIC_PAGE
 	select ARCH_HAS_KCOV
 	select ARCH_HAS_MMIOWB
+	select ARCH_HAS_PMEM_API
 	select ARCH_HAS_PTE_SPECIAL
 	select ARCH_HAS_SET_DIRECT_MAP if MMU
 	select ARCH_HAS_SET_MEMORY if MMU
arch/riscv/mm/Makefile +1 -0
index d76aabf4b94d..b4f35da889bf 100644
@@ -13,6 +13,7 @@ obj-y += extable.o
 obj-$(CONFIG_MMU) += fault.o pageattr.o
 obj-y += cacheflush.o
 obj-y += context.o
+obj-y += pmem.o
 
 ifeq ($(CONFIG_MMU),y)
 obj-$(CONFIG_SMP) += tlbflush.o
arch/riscv/mm/pmem.c new file +21 -0
new file mode 100644
index 000000000000..089df92ae876
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2022 Ventana Micro Systems Inc.
+ */
+
+#include <linux/export.h>
+#include <linux/libnvdimm.h>
+
+#include <asm/cacheflush.h>
+
+void arch_wb_cache_pmem(void *addr, size_t size)
+{
+	ALT_CMO_OP(clean, addr, size, riscv_cbom_block_size);
+}
+EXPORT_SYMBOL_GPL(arch_wb_cache_pmem);
+
+void arch_invalidate_pmem(void *addr, size_t size)
+{
+	ALT_CMO_OP(inval, addr, size, riscv_cbom_block_size);
+}
+EXPORT_SYMBOL_GPL(arch_invalidate_pmem);
Signature · 1 line
2.34.1
author
#3 lore raw

PATCH v6 3/3 RISC-V: Enable PMEM drivers

In reply to Anup Patel

We now have PMEM arch support available in RISC-V kernel so let us enable relevant drivers in defconfig.

---
 arch/riscv/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)
arch/riscv/configs/defconfig +1 -0
index 05fd5fcf24f9..462da9f7410d 100644
@@ -159,6 +159,7 @@ CONFIG_VIRTIO_MMIO=y
 CONFIG_RPMSG_CHAR=y
 CONFIG_RPMSG_CTRL=y
 CONFIG_RPMSG_VIRTIO=y
+CONFIG_LIBNVDIMM=y
 CONFIG_EXT4_FS=y
 CONFIG_EXT4_FS_POSIX_ACL=y
 CONFIG_EXT4_FS_SECURITY=y
Signature · 1 line
2.34.1
#4 lore raw
In reply to Anup Patel

Hey Anup,

Looks like patches 2 and 3 are failing for the nommu defconfigs:
https://patchwork.kernel.org/project/linux-riscv/list/?series=694995
Unfortunately, if the build fails NIPA doesn't appear to save a log.

In case it's relevant, should be gcc-11 & bintutils 2.37.

Thanks,
Conor.

Quoted 34 lines · Anup Patel

The Linux NVDIMM PEM drivers require arch support to map and access the persistent memory device. This series adds RISC-V PMEM support using recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required PMEM support for Linux RISC-V.

These patches can also be found in riscv_pmem_v6 branch at:
https://github.com/avpatel/linux.git

Changes since v5:
- Rebased on Linux-6.1-rc5
- Directly add pmem.o in obj-y of PATCH3
- Dropped PATCH1 since this is already merged.

Changes since v4:
- Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()

Changes since v3:
- Pickup correct version of Drew's patch as PATCH1

Changes since v2:
- Rebased on Linux-6.1-rc1
- Replaced PATCH1 with the patch proposed by Drew

Changes since v1:
- Fix error reported by test bot
https://lore.kernel.org/all/[email protected]/

Anup Patel (3):
RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
RISC-V: Implement arch specific PMEM APIs
RISC-V: Enable PMEM drivers

Quoted 11 lines
 arch/riscv/Kconfig           |  1 +
 arch/riscv/configs/defconfig |  1 +
 arch/riscv/include/asm/io.h  |  5 +++++
 arch/riscv/mm/Makefile       |  1 +
 arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
 5 files changed, 29 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c
Signature · 1 line
2.34.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

author
#5 lore raw
Quoted 8 lines · Conor Dooley

Hey Anup,

Looks like patches 2 and 3 are failing for the nommu defconfigs:
https://patchwork.kernel.org/project/linux-riscv/list/?series=694995
Unfortunately, if the build fails NIPA doesn't appear to save a log.

In case it's relevant, should be gcc-11 & bintutils 2.37.

I tried nommu defconfigs with gcc-12.1.0 and binutils-2.39 and I did not see any failure.

Regards,
Anup

Quoted 50 lines

Thanks,
Conor.

Quoted 45 lines · Anup Patel

The Linux NVDIMM PEM drivers require arch support to map and access the persistent memory device. This series adds RISC-V PMEM support using recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required PMEM support for Linux RISC-V.

These patches can also be found in riscv_pmem_v6 branch at:
https://github.com/avpatel/linux.git

Changes since v5:
- Rebased on Linux-6.1-rc5
- Directly add pmem.o in obj-y of PATCH3
- Dropped PATCH1 since this is already merged.

Changes since v4:
- Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()

Changes since v3:
- Pickup correct version of Drew's patch as PATCH1

Changes since v2:
- Rebased on Linux-6.1-rc1
- Replaced PATCH1 with the patch proposed by Drew

Changes since v1:
- Fix error reported by test bot
https://lore.kernel.org/all/[email protected]/

Anup Patel (3):
RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
RISC-V: Implement arch specific PMEM APIs
RISC-V: Enable PMEM drivers

 arch/riscv/Kconfig           |  1 +
 arch/riscv/configs/defconfig |  1 +
 arch/riscv/include/asm/io.h  |  5 +++++
 arch/riscv/mm/Makefile       |  1 +
 arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
 5 files changed, 29 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c

--
2.34.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Acked-by
#6 lore raw
Quoted 4 lines · Anup Patel
Quoted 3 lines · Conor Dooley

Hey Anup,

Quoted 8 lines
Quoted 5 lines

Looks like patches 2 and 3 are failing for the nommu defconfigs:
https://patchwork.kernel.org/project/linux-riscv/list/?series=694995
Unfortunately, if the build fails NIPA doesn't appear to save a log.

In case it's relevant, should be gcc-11 & bintutils 2.37.

I tried nommu defconfigs with gcc-12.1.0 and binutils-2.39 and I did not see any failure.

In the interim, I got my new machine set up properly and have gcc-11 locally now (11.1.0) with binutils 2.37. I just gave the series a try and I cannot see the failure either but I am not using tuxmake.

Apologies for the noise Anup!
FWIW:

Quoted 54 lines

Regards,
Anup

Quoted 50 lines

Thanks,
Conor.

Quoted 45 lines · Anup Patel

The Linux NVDIMM PEM drivers require arch support to map and access the persistent memory device. This series adds RISC-V PMEM support using recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required PMEM support for Linux RISC-V.

These patches can also be found in riscv_pmem_v6 branch at:
https://github.com/avpatel/linux.git

Changes since v5:
- Rebased on Linux-6.1-rc5
- Directly add pmem.o in obj-y of PATCH3
- Dropped PATCH1 since this is already merged.

Changes since v4:
- Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()

Changes since v3:
- Pickup correct version of Drew's patch as PATCH1

Changes since v2:
- Rebased on Linux-6.1-rc1
- Replaced PATCH1 with the patch proposed by Drew

Changes since v1:
- Fix error reported by test bot
https://lore.kernel.org/all/[email protected]/

Anup Patel (3):
RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
RISC-V: Implement arch specific PMEM APIs
RISC-V: Enable PMEM drivers

 arch/riscv/Kconfig           |  1 +
 arch/riscv/configs/defconfig |  1 +
 arch/riscv/include/asm/io.h  |  5 +++++
 arch/riscv/mm/Makefile       |  1 +
 arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
 5 files changed, 29 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c

--
2.34.1

author
#7 lore raw
In reply to Anup Patel

Hi Palmer,

Quoted 26 lines · Anup Patel

The Linux NVDIMM PEM drivers require arch support to map and access the persistent memory device. This series adds RISC-V PMEM support using recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required PMEM support for Linux RISC-V.

These patches can also be found in riscv_pmem_v6 branch at:
https://github.com/avpatel/linux.git

Changes since v5:
- Rebased on Linux-6.1-rc5
- Directly add pmem.o in obj-y of PATCH3
- Dropped PATCH1 since this is already merged.

Changes since v4:
- Simplify PATCH2 by implementing RISC-V specific arch_memremap_wb()

Changes since v3:
- Pickup correct version of Drew's patch as PATCH1

Changes since v2:
- Rebased on Linux-6.1-rc1
- Replaced PATCH1 with the patch proposed by Drew

Quoted 8 lines

Changes since v1:
- Fix error reported by test bot
https://lore.kernel.org/all/[email protected]/

Anup Patel (3):
RISC-V: Fix MEMREMAP_WB for systems with Svpbmt
RISC-V: Implement arch specific PMEM APIs
RISC-V: Enable PMEM drivers

Can this series be considered for Linux-6.2 ?

Regards,
Anup

Quoted 12 lines
 arch/riscv/Kconfig           |  1 +
 arch/riscv/configs/defconfig |  1 +
 arch/riscv/include/asm/io.h  |  5 +++++
 arch/riscv/mm/Makefile       |  1 +
 arch/riscv/mm/pmem.c         | 21 +++++++++++++++++++++
 5 files changed, 29 insertions(+)
 create mode 100644 arch/riscv/mm/pmem.c

--
2.34.1

#8 lore raw
In reply to Anup Patel
Quoted 8 lines · Anup Patel

The Linux NVDIMM PEM drivers require arch support to map and access the persistent memory device. This series adds RISC-V PMEM support using recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required PMEM support for Linux RISC-V.

[...]

Applied, thanks!

[1/3] RISC-V: Fix MEMREMAP_WB for systems with Svpbmt

      https://git.kernel.org/palmer/c/b91676fc16cd
[2/3] RISC-V: Implement arch specific PMEM APIs
      https://git.kernel.org/palmer/c/a49ab905a1fc
[3/3] RISC-V: Enable PMEM drivers
      https://git.kernel.org/palmer/c/497bcbe3ce04

Best regards,

Signature · 6 lines
Palmer Dabbelt <[email protected]>

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv
#9 lore raw
In reply to Anup Patel
This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:

Hello:

This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <[email protected]>:

Quoted 8 lines · On Mon, 14 Nov 2022 14:35:33 +0530 you wrote:

The Linux NVDIMM PEM drivers require arch support to map and access the persistent memory device. This series adds RISC-V PMEM support using recently added Svpbmt and Zicbom support.

First two patches are fixes and remaining two patches add the required PMEM support for Linux RISC-V.

[...]

Here is the summary with links:
- [v6,1/3] RISC-V: Fix MEMREMAP_WB for systems with Svpbmt

    https://git.kernel.org/riscv/c/b91676fc16cd
  - [v6,2/3] RISC-V: Implement arch specific PMEM APIs
    https://git.kernel.org/riscv/c/a49ab905a1fc
  - [v6,3/3] RISC-V: Enable PMEM drivers
    https://git.kernel.org/riscv/c/497bcbe3ce04

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html