riscv: errata: Add ERRATA_THEAD_WRITE_ONCE fixup

From: "Guo Ren (Alibaba DAMO Academy)" <[email protected]>

The early version of XuanTie C910 core has a store merge buffer delay problem. The store merge buffer could improve the store queue performance by merging multi-store requests, but when there are not continued store requests, the prior single store request would be waiting in the store queue for a long time. That would cause significant problems for communication between multi-cores. This problem was found on sg2042 & th1520 platforms with the qspinlock lock torture test.

So appending a fence w.o could immediately flush the store merge buffer and let other cores see the write result.

This will apply the WRITE_ONCE errata to handle the non-standard behavior via appending a fence w.o instruction for WRITE_ONCE().

This problem is only observed on the sg2042 hardware platform by running the lock_torture test program for half an hour. The problem was not found in the user space application, because interrupt can break the livelock.

Acked-by Arnd Bergmann <[email protected]>

Reviewed-by Alexandre Ghiti <[email protected]>

Reviewed-by Leonardo Bras <[email protected]>

Reviewed-by Inochi Amaoto <[email protected]>

Tested-by Han Gao <[email protected]>

Tested-by Yao Zi <[email protected]>

Cc Chen Wang <[email protected]>

Cc Xiaoguang Xing <[email protected]>

Cc Paul Walmsley <[email protected]>

Signed-off-by Guo Ren (Alibaba DAMO Academy) <[email protected]>

---
Changelog

v5:
- Add Acked-by: Arnd Bergmann <[email protected]> for asm/generic
- Add Reviewed-by: Inochi Amaoto <[email protected]>
- Rebase on v7.0

Quoted 32 lines · On Tue, Apr 21, 2026 at 10:31:40AM -0400, [email protected] wrote:

From: "Guo Ren (Alibaba DAMO Academy)" <[email protected]>

The early version of XuanTie C910 core has a store merge buffer delay problem. The store merge buffer could improve the store queue performance by merging multi-store requests, but when there are not continued store requests, the prior single store request would be waiting in the store queue for a long time. That would cause significant problems for communication between multi-cores. This problem was found on sg2042 & th1520 platforms with the qspinlock lock torture test.

So appending a fence w.o could immediately flush the store merge buffer and let other cores see the write result.

This will apply the WRITE_ONCE errata to handle the non-standard behavior via appending a fence w.o instruction for WRITE_ONCE().

This problem is only observed on the sg2042 hardware platform by running the lock_torture test program for half an hour. The problem was not found in the user space application, because interrupt can break the livelock.

Acked-by Arnd Bergmann <[email protected]>

Reviewed-by Alexandre Ghiti <[email protected]>

Reviewed-by Leonardo Bras <[email protected]>

Reviewed-by Inochi Amaoto <[email protected]>

Tested-by Han Gao <[email protected]>

Tested-by Yao Zi <[email protected]>

Cc Chen Wang <[email protected]>

Cc Xiaoguang Xing <[email protected]>

Cc Paul Walmsley <[email protected]>

Signed-off-by Guo Ren (Alibaba DAMO Academy) <[email protected]>

Quoted 7 lines

---
Changelog

v5:
- Add Acked-by: Arnd Bergmann <[email protected]> for asm/generic
- Add Reviewed-by: Inochi Amaoto <[email protected]>
- Rebase on v7.0

Hi Paul,

Han Gao (revy) let me know that there is now an errata entry for 'Writes might stick in C910/C920v1 store buffer for too long' [1].

Does that address the concern you had about documenting the errata?

Thanks,
Drew

[1] https://github.com/revyos/xuantie-c900-bugs#writes-might-stick-in-c910c920v1-store-buffer-for-too-long

From: "Guo Ren (Alibaba DAMO Academy)" <[email protected]>

The early version of XuanTie C910 core has a store merge buffer delay problem. The store merge buffer could improve the store queue performance by merging multi-store requests, but when there are not continued store requests, the prior single store request would be waiting in the store queue for a long time. That would cause significant problems for communication between multi-cores. This problem was found on sg2042 & th1520 platforms with the qspinlock lock torture test.

So appending a fence w.o could immediately flush the store merge buffer and let other cores see the write result.

This will apply the WRITE_ONCE errata to handle the non-standard behavior via appending a fence w.o instruction for WRITE_ONCE().

This problem is only observed on the sg2042 hardware platform by running the lock_torture test program for half an hour. The problem was not found in the user space application, because interrupt can break the livelock.

Acked-by Arnd Bergmann <[email protected]>

Reviewed-by Alexandre Ghiti <[email protected]>

Reviewed-by Leonardo Bras <[email protected]>

Reviewed-by Inochi Amaoto <[email protected]>

Tested-by Han Gao <[email protected]>

Tested-by Yao Zi <[email protected]>

Cc Chen Wang <[email protected]>

Cc Xiaoguang Xing <[email protected]>

Cc Paul Walmsley <[email protected]>

Signed-off-by Guo Ren (Alibaba DAMO Academy) <[email protected]>

---
Changelog

v5:
- Add Acked-by: Arnd Bergmann <[email protected]> for asm/generic
- Add Reviewed-by: Inochi Amaoto <[email protected]>
- Rebase on v7.0

v4:
https://lore.kernel.org/linux-riscv/[email protected]/
- Add Reviewed-by: Alexandre Ghiti <[email protected]>
(https://lore.kernel.org/linux-riscv/[email protected]/)
- Add Tested-by: Yao Zi <[email protected]>
- Rebase on v7.0-rc7

v3:
https://lore.kernel.org/linux-riscv/[email protected]/
- Rebase on 6.19-rc6.
- Remove errata_list_vendors patch which has been merged.
- Add Tested-by tag.

v2:
https://lore.kernel.org/linux-riscv/[email protected]/
- Add new header file for errata_list_vendors.
- Rebase newest kernel version.

v1:
https://lore.kernel.org/all/[email protected]/

---
 arch/riscv/Kconfig.errata                    | 17 ++++++++++
 arch/riscv/errata/thead/errata.c             | 20 ++++++++++++
 arch/riscv/include/asm/errata_list_vendors.h |  3 +-
 arch/riscv/include/asm/rwonce.h              | 34 ++++++++++++++++++++
 include/asm-generic/rwonce.h                 |  2 ++
 5 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100644 arch/riscv/include/asm/rwonce.h

arch/riscv/Kconfig.errata +17 -0

index 3c945d086c7d..4d3f13522da6 100644
@@ -154,4 +154,21 @@ config ERRATA_THEAD_GHOSTWRITE
 
 	  If you don't know what to do here, say "Y".
 
+config ERRATA_THEAD_WRITE_ONCE
+	bool "Apply T-Head WRITE_ONCE errata"
+	depends on ERRATA_THEAD
+	default y
+	help
+	  The early version of T-Head C9xx cores of sg2042 & th1520 have a store
+	  merge buffer delay problem. The store merge buffer could improve the
+	  store queue performance by merging multi-store requests, but when there
+	  are no continued store requests, the prior single store request would be
+	  waiting in the store queue for a long time. That would cause signifi-
+	  cant problems for communication between multi-cores. Appending a
+	  fence w.o could immediately flush the store merge buffer and let other
+	  cores see the write result.
+
+	  This will apply the WRITE_ONCE errata to handle the non-standard beh-
+	  avior via appending a fence w.o instruction for WRITE_ONCE().
+
 endmenu # "CPU errata selection"

arch/riscv/errata/thead/errata.c +20 -0

index 0b942183f708..fbe46f2fa8fb 100644
@@ -168,6 +168,23 @@ static bool errata_probe_ghostwrite(unsigned int stage,
 	return true;
 }
 
+static bool errata_probe_write_once(unsigned int stage,
+				    unsigned long arch_id, unsigned long impid)
+{
+	if (!IS_ENABLED(CONFIG_ERRATA_THEAD_WRITE_ONCE))
+		return false;
+
+	/* target-c9xx cores report arch_id and impid as 0 */
+	if (arch_id != 0 || impid != 0)
+		return false;
+
+	if (stage == RISCV_ALTERNATIVES_BOOT ||
+	    stage == RISCV_ALTERNATIVES_MODULE)
+		return true;
+
+	return false;
+}
+
 static u32 thead_errata_probe(unsigned int stage,
 			      unsigned long archid, unsigned long impid)
 {

@@ -183,6 +200,9 @@ static u32 thead_errata_probe(unsigned int stage,
 
 	errata_probe_ghostwrite(stage, archid, impid);
 
+	if (errata_probe_write_once(stage, archid, impid))
+		cpu_req_errata |= BIT(ERRATA_THEAD_WRITE_ONCE);
+
 	return cpu_req_errata;
 }
 

arch/riscv/include/asm/errata_list_vendors.h +2 -1

index ec7eba373437..8fd7c36307e2 100644
@@ -18,7 +18,8 @@
 #define	ERRATA_THEAD_MAE 0
 #define	ERRATA_THEAD_PMU 1
 #define	ERRATA_THEAD_GHOSTWRITE 2
-#define	ERRATA_THEAD_NUMBER 3
+#define	ERRATA_THEAD_WRITE_ONCE 3
+#define	ERRATA_THEAD_NUMBER 4
 #endif
 
 #ifdef CONFIG_ERRATA_MIPS

arch/riscv/include/asm/rwonce.h new file +34 -0

new file mode 100644
index 000000000000..081793d4d772
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#ifndef __ASM_RWONCE_H
+#define __ASM_RWONCE_H
+
+#include <linux/compiler_types.h>
+#include <asm/alternative-macros.h>
+#include <asm/vendorid_list.h>
+#include <asm/errata_list_vendors.h>
+
+#if defined(CONFIG_ERRATA_THEAD_WRITE_ONCE) && !defined(NO_ALTERNATIVE)
+
+#define write_once_fence()				\
+do {							\
+	asm volatile(ALTERNATIVE(			\
+		"nop",					\
+		"fence w, o",				\
+		THEAD_VENDOR_ID,			\
+		ERRATA_THEAD_WRITE_ONCE,		\
+		CONFIG_ERRATA_THEAD_WRITE_ONCE)		\
+		: : : "memory");			\
+} while (0)
+
+#define __WRITE_ONCE(x, val)				\
+do {							\
+	*(volatile typeof(x) *)&(x) = (val);		\
+	write_once_fence();				\
+} while (0)
+
+#endif /* defined(CONFIG_ERRATA_THEAD_WRITE_ONCE) && !defined(NO_ALTERNATIVE) */
+
+#include <asm-generic/rwonce.h>
+
+#endif	/* __ASM_RWONCE_H */

include/asm-generic/rwonce.h +2 -0

index 52b969c7cef9..4e2d941f15a1 100644
@@ -50,10 +50,12 @@
 	__READ_ONCE(x);							\
 })
 
+#ifndef __WRITE_ONCE
 #define __WRITE_ONCE(x, val)						\
 do {									\
 	*(volatile typeof(x) *)&(x) = (val);				\
 } while (0)
+#endif
 
 #define WRITE_ONCE(x, val)						\
 do {									\

Signature · 1 line

2.40.1