Skip to contentSkip to conversation

RISC-V Hibernation Support

PATCH Superseded Sia Jee Heng 2 participants 12 messages Mar 8, 2023 – Mar 10, 2023

Subsystems (2)

v5 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v5 cover RISC-V Hibernation Support

This series adds RISC-V Hibernation/suspend to disk support. Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid, and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

At high-level, this series includes the following changes:
1) Change suspend_save_csrs() and suspend_restore_csrs()
to public function as these functions are common to suspend/hibernation. (patch 1)
2) Refactor the common code in the __cpu_resume_enter() function and __hibernate_cpu_resume() function. The common code are used by hibernation and suspend. (patch 2)
3) Enhance kernel_page_present() function to support huge page. (patch 3)
4) Add arch/riscv low level functions to support hibernation/suspend to disk. (patch 4)

The above patches are based on kernel v6.3-rc1 and are tested on StarFive VF2 SBC board and Qemu.
ACPI platform mode is not supported in this series.

Changes since v4:
- Rebased to kernel v6.3-rc1
- Resolved typo(s)
- Removed unnecessary helper function
- Removed unnecessary "addr" local variable
- Removed typecast of 'int'
- Used def_bool HIBERNATION
- Used "mv a0, zero" instead of "add a0, zero, zero"
- Make linear region as executable and writable when restoring the image

Changes since v3:
- Rebased to kernel v6.2
- Temporary page table code refactoring by reference to ARM64
- Resolved typo(s) and grammars
- Resolved documentation errors
- Resolved clang build issue
- Removed unnecessary comments
- Used kzalloc instead of kcalloc

Changes since v2:
- Rebased to kernel v6.2-rc5
- Refactor the common code used by hibernation and suspend
- Create copy_page macro
- Solved other comments from Andrew and Conor

Changes since v1:
- Rebased to kernel v6.2-rc3
- Fixed bot's compilation error

Sia Jee Heng (4):
RISC-V: Change suspend_save_csrs and suspend_restore_csrs to public
function
RISC-V: Factor out common code of __cpu_resume_enter()
RISC-V: mm: Enable huge page support to kernel_page_present() function
RISC-V: Add arch functions to support hibernation/suspend-to-disk

 arch/riscv/Kconfig                 |   6 +
 arch/riscv/include/asm/assembler.h |  82 ++++++
 arch/riscv/include/asm/suspend.h   |  22 ++
 arch/riscv/kernel/Makefile         |   1 +
 arch/riscv/kernel/asm-offsets.c    |   5 +
 arch/riscv/kernel/hibernate-asm.S  |  77 ++++++
 arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++
 arch/riscv/kernel/suspend.c        |   4 +-
 arch/riscv/kernel/suspend_entry.S  |  34 +--
 arch/riscv/mm/pageattr.c           |   8 +
 10 files changed, 630 insertions(+), 33 deletions(-)
 create mode 100644 arch/riscv/include/asm/assembler.h
 create mode 100644 arch/riscv/kernel/hibernate-asm.S
 create mode 100644 arch/riscv/kernel/hibernate.c

base-commit: 63355b9884b3d1677de6bd1517cd2b8a9bf53978

Signature · 1 line
2.34.1
#1 lore raw

PATCH v5 1/4 RISC-V: Change suspend_save_csrs and suspend_restore_csrs to public function

Currently suspend_save_csrs() and suspend_restore_csrs() functions are statically defined in the suspend.c. Change the function's attribute to public so that the functions can be used by hibernation as well.

---
 arch/riscv/include/asm/suspend.h | 3 +++
 arch/riscv/kernel/suspend.c      | 4 ++--
 2 files changed, 5 insertions(+), 2 deletions(-)
arch/riscv/include/asm/suspend.h +3 -0
index 8be391c2aecb..67e047445662 100644
@@ -33,4 +33,7 @@ int cpu_suspend(unsigned long arg,
 /* Low-level CPU resume entry function */
 int __cpu_resume_enter(unsigned long hartid, unsigned long context);
 
+/* Used to save and restore the CSRs */
+void suspend_save_csrs(struct suspend_context *context);
+void suspend_restore_csrs(struct suspend_context *context);
 #endif
arch/riscv/kernel/suspend.c +2 -2
index 9ba24fb8cc93..3c89b8ec69c4 100644
@@ -8,7 +8,7 @@
 #include <asm/csr.h>
 #include <asm/suspend.h>
 
-static void suspend_save_csrs(struct suspend_context *context)
+void suspend_save_csrs(struct suspend_context *context)
 {
 	context->scratch = csr_read(CSR_SCRATCH);
 	context->tvec = csr_read(CSR_TVEC);
@@ -29,7 +29,7 @@ static void suspend_save_csrs(struct suspend_context *context)
 #endif
 }
 
-static void suspend_restore_csrs(struct suspend_context *context)
+void suspend_restore_csrs(struct suspend_context *context)
 {
 	csr_write(CSR_SCRATCH, context->scratch);
 	csr_write(CSR_TVEC, context->tvec);
Signature · 1 line
2.34.1
#2 lore raw

PATCH v5 2/4 RISC-V: Factor out common code of __cpu_resume_enter()

In reply to Sia Jee Heng

The cpu_resume() function is very similar for the suspend to disk and suspend to ram cases. Factor out the common code into suspend_restore_csrs macro and suspend_restore_regs macro.

---
 arch/riscv/include/asm/assembler.h | 62 ++++++++++++++++++++++++++++++
 arch/riscv/kernel/suspend_entry.S  | 34 ++--------------
 2 files changed, 65 insertions(+), 31 deletions(-)
 create mode 100644 arch/riscv/include/asm/assembler.h
arch/riscv/include/asm/assembler.h new file +62 -0
new file mode 100644
index 000000000000..ba59d38f8937
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#ifndef __ASSEMBLY__
+#error "Only include this from assembly code"
+#endif
+
+#ifndef __ASM_ASSEMBLER_H
+#define __ASM_ASSEMBLER_H
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/csr.h>
+
+/*
+ * suspend_restore_csrs - restore CSRs
+ */
+	.macro suspend_restore_csrs
+		REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_EPC)(a0)
+		csrw	CSR_EPC, t0
+		REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_STATUS)(a0)
+		csrw	CSR_STATUS, t0
+		REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_BADADDR)(a0)
+		csrw	CSR_TVAL, t0
+		REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_CAUSE)(a0)
+		csrw	CSR_CAUSE, t0
+	.endm
+
+/*
+ * suspend_restore_regs - Restore registers (except A0 and T0-T6)
+ */
+	.macro suspend_restore_regs
+		REG_L	ra, (SUSPEND_CONTEXT_REGS + PT_RA)(a0)
+		REG_L	sp, (SUSPEND_CONTEXT_REGS + PT_SP)(a0)
+		REG_L	gp, (SUSPEND_CONTEXT_REGS + PT_GP)(a0)
+		REG_L	tp, (SUSPEND_CONTEXT_REGS + PT_TP)(a0)
+		REG_L	s0, (SUSPEND_CONTEXT_REGS + PT_S0)(a0)
+		REG_L	s1, (SUSPEND_CONTEXT_REGS + PT_S1)(a0)
+		REG_L	a1, (SUSPEND_CONTEXT_REGS + PT_A1)(a0)
+		REG_L	a2, (SUSPEND_CONTEXT_REGS + PT_A2)(a0)
+		REG_L	a3, (SUSPEND_CONTEXT_REGS + PT_A3)(a0)
+		REG_L	a4, (SUSPEND_CONTEXT_REGS + PT_A4)(a0)
+		REG_L	a5, (SUSPEND_CONTEXT_REGS + PT_A5)(a0)
+		REG_L	a6, (SUSPEND_CONTEXT_REGS + PT_A6)(a0)
+		REG_L	a7, (SUSPEND_CONTEXT_REGS + PT_A7)(a0)
+		REG_L	s2, (SUSPEND_CONTEXT_REGS + PT_S2)(a0)
+		REG_L	s3, (SUSPEND_CONTEXT_REGS + PT_S3)(a0)
+		REG_L	s4, (SUSPEND_CONTEXT_REGS + PT_S4)(a0)
+		REG_L	s5, (SUSPEND_CONTEXT_REGS + PT_S5)(a0)
+		REG_L	s6, (SUSPEND_CONTEXT_REGS + PT_S6)(a0)
+		REG_L	s7, (SUSPEND_CONTEXT_REGS + PT_S7)(a0)
+		REG_L	s8, (SUSPEND_CONTEXT_REGS + PT_S8)(a0)
+		REG_L	s9, (SUSPEND_CONTEXT_REGS + PT_S9)(a0)
+		REG_L	s10, (SUSPEND_CONTEXT_REGS + PT_S10)(a0)
+		REG_L	s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
+	.endm
+
+#endif	/* __ASM_ASSEMBLER_H */
arch/riscv/kernel/suspend_entry.S +3 -31
index aafcca58c19d..12b52afe09a4 100644
@@ -7,6 +7,7 @@
 #include <linux/linkage.h>
 #include <asm/asm.h>
 #include <asm/asm-offsets.h>
+#include <asm/assembler.h>
 #include <asm/csr.h>
 #include <asm/xip_fixup.h>
 
@@ -83,39 +84,10 @@ ENTRY(__cpu_resume_enter)
 	add	a0, a1, zero
 
 	/* Restore CSRs */
-	REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_EPC)(a0)
-	csrw	CSR_EPC, t0
-	REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_STATUS)(a0)
-	csrw	CSR_STATUS, t0
-	REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_BADADDR)(a0)
-	csrw	CSR_TVAL, t0
-	REG_L	t0, (SUSPEND_CONTEXT_REGS + PT_CAUSE)(a0)
-	csrw	CSR_CAUSE, t0
+	suspend_restore_csrs
 
 	/* Restore registers (except A0 and T0-T6) */
-	REG_L	ra, (SUSPEND_CONTEXT_REGS + PT_RA)(a0)
-	REG_L	sp, (SUSPEND_CONTEXT_REGS + PT_SP)(a0)
-	REG_L	gp, (SUSPEND_CONTEXT_REGS + PT_GP)(a0)
-	REG_L	tp, (SUSPEND_CONTEXT_REGS + PT_TP)(a0)
-	REG_L	s0, (SUSPEND_CONTEXT_REGS + PT_S0)(a0)
-	REG_L	s1, (SUSPEND_CONTEXT_REGS + PT_S1)(a0)
-	REG_L	a1, (SUSPEND_CONTEXT_REGS + PT_A1)(a0)
-	REG_L	a2, (SUSPEND_CONTEXT_REGS + PT_A2)(a0)
-	REG_L	a3, (SUSPEND_CONTEXT_REGS + PT_A3)(a0)
-	REG_L	a4, (SUSPEND_CONTEXT_REGS + PT_A4)(a0)
-	REG_L	a5, (SUSPEND_CONTEXT_REGS + PT_A5)(a0)
-	REG_L	a6, (SUSPEND_CONTEXT_REGS + PT_A6)(a0)
-	REG_L	a7, (SUSPEND_CONTEXT_REGS + PT_A7)(a0)
-	REG_L	s2, (SUSPEND_CONTEXT_REGS + PT_S2)(a0)
-	REG_L	s3, (SUSPEND_CONTEXT_REGS + PT_S3)(a0)
-	REG_L	s4, (SUSPEND_CONTEXT_REGS + PT_S4)(a0)
-	REG_L	s5, (SUSPEND_CONTEXT_REGS + PT_S5)(a0)
-	REG_L	s6, (SUSPEND_CONTEXT_REGS + PT_S6)(a0)
-	REG_L	s7, (SUSPEND_CONTEXT_REGS + PT_S7)(a0)
-	REG_L	s8, (SUSPEND_CONTEXT_REGS + PT_S8)(a0)
-	REG_L	s9, (SUSPEND_CONTEXT_REGS + PT_S9)(a0)
-	REG_L	s10, (SUSPEND_CONTEXT_REGS + PT_S10)(a0)
-	REG_L	s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
+	suspend_restore_regs
 
 	/* Return zero value */
 	add	a0, zero, zero
Signature · 1 line
2.34.1
#3 lore raw

PATCH v5 3/4 RISC-V: mm: Enable huge page support to kernel_page_present() function

In reply to Sia Jee Heng

Currently kernel_page_present() function doesn't support huge page detection causes the function to mistakenly return false to the hibernation core.

Add huge page detection to the function to solve the problem.

Fixes 9e953cda5cdf ("riscv: Introduce huge page support for 32/64bit kernel")
---
 arch/riscv/mm/pageattr.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/riscv/mm/pageattr.c +8 -0
index 86c56616e5de..ea3d61de065b 100644
@@ -217,18 +217,26 @@ bool kernel_page_present(struct page *page)
 	pgd = pgd_offset_k(addr);
 	if (!pgd_present(*pgd))
 		return false;
+	if (pgd_leaf(*pgd))
+		return true;
 
 	p4d = p4d_offset(pgd, addr);
 	if (!p4d_present(*p4d))
 		return false;
+	if (p4d_leaf(*p4d))
+		return true;
 
 	pud = pud_offset(p4d, addr);
 	if (!pud_present(*pud))
 		return false;
+	if (pud_leaf(*pud))
+		return true;
 
 	pmd = pmd_offset(pud, addr);
 	if (!pmd_present(*pmd))
 		return false;
+	if (pmd_leaf(*pmd))
+		return true;
 
 	pte = pte_offset_kernel(pmd, addr);
 	return pte_present(*pte);
Signature · 1 line
2.34.1
#4 lore raw

PATCH v5 4/4 RISC-V: Add arch functions to support hibernation/suspend-to-disk

In reply to Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid, and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
 arch/riscv/Kconfig                 |   6 +
 arch/riscv/include/asm/assembler.h |  20 ++
 arch/riscv/include/asm/suspend.h   |  19 ++
 arch/riscv/kernel/Makefile         |   1 +
 arch/riscv/kernel/asm-offsets.c    |   5 +
 arch/riscv/kernel/hibernate-asm.S  |  77 ++++++
 arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++
 7 files changed, 552 insertions(+)
 create mode 100644 arch/riscv/kernel/hibernate-asm.S
 create mode 100644 arch/riscv/kernel/hibernate.c
arch/riscv/Kconfig +6 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"
 
 source "kernel/power/Kconfig"
 
+config ARCH_HIBERNATION_POSSIBLE
+	def_bool y
+
+config ARCH_HIBERNATION_HEADER
+	def_bool HIBERNATION
+
 endmenu # "Power management options"
 
 menu "CPU Power Management"
arch/riscv/include/asm/assembler.h +20 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
 		REG_L	s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
 	.endm
 
+/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+	.macro	copy_page a0, a1
+		lui	a2, 0x1
+		add	a2, a2, a0
+1 :
+		REG_L	t0, 0(a1)
+		REG_L	t1, SZREG(a1)
+
+		REG_S	t0, 0(a0)
+		REG_S	t1, SZREG(a0)
+
+		addi	a0, a0, 2 * SZREG
+		addi	a1, a1, 2 * SZREG
+		bne	a2, a0, 1b
+	.endm
+
 #endif	/* __ASM_ASSEMBLER_H */
arch/riscv/include/asm/suspend.h +19 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
 #endif
 };
 
+/*
+ * Used by hibernation core and cleared during resume sequence
+ */
+extern int in_suspend;
+
 /* Low-level CPU suspend entry function */
 int __cpu_suspend_enter(struct suspend_context *context);
 
@@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid, unsigned long context);
 /* Used to save and restore the CSRs */
 void suspend_save_csrs(struct suspend_context *context);
 void suspend_restore_csrs(struct suspend_context *context);
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp, unsigned long satp_temp,
+					unsigned long cpu_resume);
+asmlinkage int hibernate_core_restore_code(void);
 #endif
arch/riscv/kernel/Makefile +1 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)		+= module.o
 obj-$(CONFIG_MODULE_SECTIONS)	+= module-sections.o
 
 obj-$(CONFIG_CPU_PM)		+= suspend_entry.o suspend.o
+obj-$(CONFIG_HIBERNATION)	+= hibernate.o hibernate-asm.o
 
 obj-$(CONFIG_FUNCTION_TRACER)	+= mcount.o ftrace.o
 obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
arch/riscv/kernel/asm-offsets.c +5 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
 #include <linux/kbuild.h>
 #include <linux/mm.h>
 #include <linux/sched.h>
+#include <linux/suspend.h>
 #include <asm/kvm_host.h>
 #include <asm/thread_info.h>
 #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
 
 	OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
 
+	OFFSET(HIBERN_PBE_ADDR, pbe, address);
+	OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+	OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
 	OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
 	OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
 	OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +77 -0

77 lines not on this page · raw message

arch/riscv/kernel/hibernate.c new file +424 -0

424 lines not on this page · raw message

Signature · 1 line
2.34.1
#5 lore raw

Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 607 lines · Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid, and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
  arch/riscv/Kconfig                 |   6 +
  arch/riscv/include/asm/assembler.h |  20 ++
  arch/riscv/include/asm/suspend.h   |  19 ++
  arch/riscv/kernel/Makefile         |   1 +
  arch/riscv/kernel/asm-offsets.c    |   5 +
  arch/riscv/kernel/hibernate-asm.S  |  77 ++++++
  arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++
  7 files changed, 552 insertions(+)

create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c

arch/riscv/Kconfig +6 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"
 
  source "kernel/power/Kconfig"
 
+config ARCH_HIBERNATION_POSSIBLE
+	def_bool y
+
+config ARCH_HIBERNATION_HEADER
+	def_bool HIBERNATION
+
  endmenu # "Power management options"
 
  menu "CPU Power Management"
arch/riscv/include/asm/assembler.h +20 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
  		REG_L	s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
  	.endm
 
+/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+	.macro	copy_page a0, a1
+		lui	a2, 0x1
+		add	a2, a2, a0
+1 :
+		REG_L	t0, 0(a1)
+		REG_L	t1, SZREG(a1)
+
+		REG_S	t0, 0(a0)
+		REG_S	t1, SZREG(a0)
+
+		addi	a0, a0, 2 * SZREG
+		addi	a1, a1, 2 * SZREG
+		bne	a2, a0, 1b
+	.endm
+
  #endif	/* __ASM_ASSEMBLER_H */
arch/riscv/include/asm/suspend.h +19 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
  #endif
  };
 
+/*
+ * Used by hibernation core and cleared during resume sequence
+ */
+extern int in_suspend;
+
  /* Low-level CPU suspend entry function */
  int __cpu_suspend_enter(struct suspend_context *context);
 
@@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid, unsigned long context);
  /* Used to save and restore the CSRs */
  void suspend_save_csrs(struct suspend_context *context);
  void suspend_restore_csrs(struct suspend_context *context);
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp, unsigned long satp_temp,
+					unsigned long cpu_resume);
+asmlinkage int hibernate_core_restore_code(void);
  #endif
arch/riscv/kernel/Makefile +1 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)		+= module.o
  obj-$(CONFIG_MODULE_SECTIONS)	+= module-sections.o
 
  obj-$(CONFIG_CPU_PM)		+= suspend_entry.o suspend.o
+obj-$(CONFIG_HIBERNATION)	+= hibernate.o hibernate-asm.o
 
  obj-$(CONFIG_FUNCTION_TRACER)	+= mcount.o ftrace.o
  obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
arch/riscv/kernel/asm-offsets.c +5 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
  #include <linux/kbuild.h>
  #include <linux/mm.h>
  #include <linux/sched.h>
+#include <linux/suspend.h>
  #include <asm/kvm_host.h>
  #include <asm/thread_info.h>
  #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
 
  	OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
 
+	OFFSET(HIBERN_PBE_ADDR, pbe, address);
+	OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+	OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
  	OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
  	OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
  	OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +77 -0
new file mode 100644
index 000000000000..effaf5ca5da0
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hibernation low level support for RISCV.
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/csr.h>
+
+#include <linux/linkage.h>
+
+/*
+ * int __hibernate_cpu_resume(void)
+ * Switch back to the hibernated image's page table prior to restoring the CPU
+ * context.
+ *
+ * Always returns 0
+ */
+ENTRY(__hibernate_cpu_resume)
+	/* switch to hibernated image's page table. */
+	csrw CSR_SATP, s0
+	sfence.vma
+
+	REG_L	a0, hibernate_cpu_context
+
+	suspend_restore_csrs
+	suspend_restore_regs
+
+	/* Return zero value. */
+	mv	a0, zero
+
+	ret
+END(__hibernate_cpu_resume)
+
+/*
+ * Prepare to restore the image.
+ * a0: satp of saved page tables.
+ * a1: satp of temporary page tables.
+ * a2: cpu_resume.
+ */
+ENTRY(hibernate_restore_image)
+	mv	s0, a0
+	mv	s1, a1
+	mv	s2, a2
+	REG_L	s4, restore_pblist
+	REG_L	a1, relocated_restore_code
+
+	jalr	a1
+END(hibernate_restore_image)
+
+/*
+ * The below code will be executed from a 'safe' page.
+ * It first switches to the temporary page table, then starts to copy the pages
+ * back to the original memory location. Finally, it jumps to __hibernate_cpu_resume()
+ * to restore the CPU context.
+ */
+ENTRY(hibernate_core_restore_code)
+	/* switch to temp page table. */
+	csrw satp, s1
+	sfence.vma
+.Lcopy:
+	/* The below code will restore the hibernated image. */
+	REG_L	a1, HIBERN_PBE_ADDR(s4)
+	REG_L	a0, HIBERN_PBE_ORIG(s4)
+
+	copy_page a0, a1
+
+	REG_L	s4, HIBERN_PBE_NEXT(s4)
+	bnez	s4, .Lcopy
+
+	jalr	s2
+END(hibernate_core_restore_code)
arch/riscv/kernel/hibernate.c new file +363 -0
new file mode 100644
index 000000000000..8fab50122e54
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hibernation support for RISCV
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/barrier.h>
+#include <asm/cacheflush.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/sections.h>
+#include <asm/set_memory.h>
+#include <asm/smp.h>
+#include <asm/suspend.h>
+
+#include <linux/cpu.h>
+#include <linux/memblock.h>
+#include <linux/pm.h>
+#include <linux/sched.h>
+#include <linux/suspend.h>
+#include <linux/utsname.h>
+
+/* The logical cpu number we should resume on, initialised to a non-cpu number. */
+static int sleep_cpu = -EINVAL;
+
+/* Pointer to the temporary resume page table. */
+static pgd_t *resume_pg_dir;
+
+/* CPU context to be saved. */
+struct suspend_context *hibernate_cpu_context;
+EXPORT_SYMBOL_GPL(hibernate_cpu_context);
+
+unsigned long relocated_restore_code;
+EXPORT_SYMBOL_GPL(relocated_restore_code);
+
+/**
+ * struct arch_hibernate_hdr_invariants - container to store kernel build version.
+ * @uts_version: to save the build number and date so that the we do not resume with
+ *		a different kernel.
+ */
+struct arch_hibernate_hdr_invariants {
+	char		uts_version[__NEW_UTS_LEN + 1];
+};
+
+/**
+ * struct arch_hibernate_hdr - helper parameters that help us to restore the image.
+ * @invariants: container to store kernel build version.
+ * @hartid: to make sure same boot_cpu executes the hibernate/restore code.
+ * @saved_satp: original page table used by the hibernated image.
+ * @restore_cpu_addr: the kernel's image address to restore the CPU context.
+ */
+static struct arch_hibernate_hdr {
+	struct arch_hibernate_hdr_invariants invariants;
+	unsigned long	hartid;
+	unsigned long	saved_satp;
+	unsigned long	restore_cpu_addr;
+} resume_hdr;
+
+static void arch_hdr_invariants(struct arch_hibernate_hdr_invariants *i)
+{
+	memset(i, 0, sizeof(*i));
+	memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version));
+}
+
+/*
+ * Check if the given pfn is in the 'nosave' section.
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+	unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin);
+	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
+
+	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn));
+}
+
+void notrace save_processor_state(void)
+{
+	WARN_ON(num_online_cpus() != 1);
+}
+
+void notrace restore_processor_state(void)
+{
+}
+
+/*
+ * Helper parameters need to be saved to the hibernation image header.
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+	struct arch_hibernate_hdr *hdr = addr;
+
+	if (max_size < sizeof(*hdr))
+		return -EOVERFLOW;
+
+	arch_hdr_invariants(&hdr->invariants);
+
+	hdr->hartid = cpuid_to_hartid_map(sleep_cpu);
+	hdr->saved_satp = csr_read(CSR_SATP);
+	hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_save);
+
+/*
+ * Retrieve the helper parameters from the hibernation image header.
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+	struct arch_hibernate_hdr_invariants invariants;
+	struct arch_hibernate_hdr *hdr = addr;
+	int ret = 0;
+
+	arch_hdr_invariants(&invariants);
+
+	if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) {
+		pr_crit("Hibernate image not generated by this kernel!\n");
+		return -EINVAL;
+	}
+
+	sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid);
+	if (sleep_cpu < 0) {
+		pr_crit("Hibernated on a CPU not known to this kernel!\n");
+		sleep_cpu = -EINVAL;
+		return -EINVAL;
+	}
+
+#ifdef CONFIG_SMP
+	ret = bringup_hibernate_cpu(sleep_cpu);
+	if (ret) {
+		sleep_cpu = -EINVAL;
+		return ret;
+	}
+#endif
+	resume_hdr = *hdr;
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_restore);
+
+int swsusp_arch_suspend(void)
+{
+	int ret = 0;
+
+	if (__cpu_suspend_enter(hibernate_cpu_context)) {
+		sleep_cpu = smp_processor_id();
+		suspend_save_csrs(hibernate_cpu_context);
+		ret = swsusp_save();
+	} else {
+		suspend_restore_csrs(hibernate_cpu_context);
+		flush_tlb_all();
+		flush_icache_all();
+
+		/*
+		 * Tell the hibernation core that we've just restored the memory.
+		 */
+		in_suspend = 0;
+		sleep_cpu = -EINVAL;
+	}
+
+	return ret;
+}
+
+static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start,
+				unsigned long end, pgprot_t prot)
+{
+	pte_t *src_ptep;
+	pte_t *dst_ptep;
+
+	if (pmd_none(READ_ONCE(*dst_pmdp))) {
+		dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC);
+		if (!dst_ptep)
+			return -ENOMEM;
+
+		pmd_populate_kernel(NULL, dst_pmdp, dst_ptep);
+	}
+
+	dst_ptep = pte_offset_kernel(dst_pmdp, start);
+	src_ptep = pte_offset_kernel(src_pmdp, start);
+
+	do {
+		pte_t pte = READ_ONCE(*src_ptep);
+
+		if (pte_present(pte))
+			set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot)));
+	} while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end);
+
+	return 0;
+}
+
+static int temp_pgtable_map_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start,
+				unsigned long end, pgprot_t prot)
+{
+	unsigned long next;
+	unsigned long ret;
+	pmd_t *src_pmdp;
+	pmd_t *dst_pmdp;
+
+	if (pud_none(READ_ONCE(*dst_pudp))) {
+		dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC);
+		if (!dst_pmdp)
+			return -ENOMEM;
+
+		pud_populate(NULL, dst_pudp, dst_pmdp);
+	}
+
+	dst_pmdp = pmd_offset(dst_pudp, start);
+	src_pmdp = pmd_offset(src_pudp, start);
+
+	do {
+		pmd_t pmd = READ_ONCE(*src_pmdp);
+
+		next = pmd_addr_end(start, end);
+
+		if (pmd_none(pmd))
+			continue;
+
+		if (pmd_leaf(pmd)) {
+			set_pmd(dst_pmdp, __pmd(pmd_val(pmd) | pgprot_val(prot)));
+		} else {
+			ret = temp_pgtable_map_pte(dst_pmdp, src_pmdp, start, next, prot);
+			if (ret)
+				return -ENOMEM;
+		}
+	} while (dst_pmdp++, src_pmdp++, start = next, start != end);
+
+	return 0;
+}
+
+static int temp_pgtable_map_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, unsigned long start,
+				unsigned long end, pgprot_t prot)
+{
+	unsigned long next;
+	unsigned long ret;
+	pud_t *dst_pudp;
+	pud_t *src_pudp;
+
+	if (p4d_none(READ_ONCE(*dst_p4dp))) {
+		dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC);
+		if (!dst_pudp)
+			return -ENOMEM;
+
+		p4d_populate(NULL, dst_p4dp, dst_pudp);
+	}
+
+	dst_pudp = pud_offset(dst_p4dp, start);
+	src_pudp = pud_offset(src_p4dp, start);
+
+	do {
+		pud_t pud = READ_ONCE(*src_pudp);
+
+		next = pud_addr_end(start, end);
+
+		if (pud_none(pud))
+			continue;
+
+		if (pud_leaf(pud)) {
+			set_pud(dst_pudp, __pud(pud_val(pud) | pgprot_val(prot)));
+		} else {
+			ret = temp_pgtable_map_pmd(dst_pudp, src_pudp, start, next, prot);
+			if (ret)
+				return -ENOMEM;
+		}
+	} while (dst_pudp++, src_pudp++, start = next, start != end);
+
+	return 0;
+}
+
+static int temp_pgtable_map_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start,
+				unsigned long end, pgprot_t prot)
+{
+	unsigned long next;
+	unsigned long ret;
+	p4d_t *dst_p4dp;
+	p4d_t *src_p4dp;
+
+	if (pgd_none(READ_ONCE(*dst_pgdp))) {
+		dst_p4dp = (p4d_t *)get_safe_page(GFP_ATOMIC);
+		if (!dst_p4dp)
+			return -ENOMEM;
+
+		pgd_populate(NULL, dst_pgdp, dst_p4dp);
+	}
+
+	dst_p4dp = p4d_offset(dst_pgdp, start);
+	src_p4dp = p4d_offset(src_pgdp, start);
+
+	do {
+		p4d_t p4d = READ_ONCE(*src_p4dp);
+
+		next = p4d_addr_end(start, end);
+
+		if (p4d_none(p4d))
+			continue;
+
+		if (p4d_leaf(p4d)) {
+			set_p4d(dst_p4dp, __p4d(p4d_val(p4d) | pgprot_val(prot)));
+		} else {
+			ret = temp_pgtable_map_pud(dst_p4dp, src_p4dp, start, next, prot);
+			if (ret)
+				return -ENOMEM;
+		}
+	} while (dst_p4dp++, src_p4dp++, start = next, start != end);
+
+	return 0;
+}
+
+static int temp_pgtable_mapping(pgd_t *pgdp, unsigned long start, unsigned long end, pgprot_t prot)
+{
+	pgd_t *dst_pgdp = pgd_offset_pgd(pgdp, start);
+	pgd_t *src_pgdp = pgd_offset_k(start);
+	unsigned long next;
+	unsigned long ret;
+
+	do {
+		pgd_t pgd = READ_ONCE(*src_pgdp);
+
+		next = pgd_addr_end(start, end);
+
+		if (pgd_none(pgd))
+			continue;
+
+		if (pgd_leaf(pgd)) {
+			set_pgd(dst_pgdp, __pgd(pgd_val(pgd) | pgprot_val(prot)));
+		} else {
+			ret = temp_pgtable_map_p4d(dst_pgdp, src_pgdp, start, next, prot);
+			if (ret)
+				return -ENOMEM;
+		}
+	} while (dst_pgdp++, src_pgdp++, start = next, start != end);
+
+	return 0;
+}
+
+static unsigned long relocate_restore_code(void)
+{
+	void *page = (void *)get_safe_page(GFP_ATOMIC);
+
+	if (!page)
+		return -ENOMEM;
+
+	copy_page(page, hibernate_core_restore_code);
+
+	/* Make the page containing the relocated code executable. */
+	set_memory_x((unsigned long)page, 1);
+
+	return (unsigned long)page;
+}
+
+int swsusp_arch_resume(void)
+{
+	unsigned long end = (unsigned long)pfn_to_virt(max_low_pfn);
+	unsigned long start = PAGE_OFFSET;
+	int ret;
+
+	/*
+	 * Memory allocated by get_safe_page() will be dealt with by the hibernation core,
+	 * we don't need to free it here.

arch/riscv/kernel/hibernate.c line 373

Quoted 10 lines
arch/riscv/kernel/hibernate.c new file +10 -0
+	 */
+	resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC);
+	if (!resume_pg_dir)
+		return -ENOMEM;
+
+	/*
+	 * Create a temporary page table and map the whole linear region as executable and
+	 * writable.
+	 */
+	ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));

Hmm weird protections, PAGE_PRESENT and PAGE_READ are not even set. Shouldn't you use PAGE_KERNEL_EXEC instead?

Quoted 11 lines
arch/riscv/kernel/hibernate.c new file +11 -0
+	if (ret)
+		return ret;
+
+	/* Move the restore code to a new page so that it doesn't get overwritten by itself. */
+	relocated_restore_code = relocate_restore_code();
+	if (relocated_restore_code == -ENOMEM)
+		return -ENOMEM;
+
+	/*
+	 * Map the __hibernate_cpu_resume() address to the temporary page table so that the
+	 * restore code can jumps to it after finished restore the image. The next execution

arch/riscv/kernel/hibernate.c line 394

Quoted 10 lines
arch/riscv/kernel/hibernate.c new file +10 -0
+	 * code doesn't find itself in a different address space after switching over to the
+	 * original page table used by the hibernated image.
+	 *
+	 */
+	start = (unsigned long)resume_hdr.restore_cpu_addr;
+	end = start + PAGE_SIZE;
+
+	ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));
+	if (ret)
+		return ret;

You set the linear mapping as executable so no need for this.

Quoted 20 lines
arch/riscv/kernel/hibernate.c new file +20 -0
+
+	hibernate_restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode),
+				resume_hdr.restore_cpu_addr);
+
+	return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP_SMP
+int hibernate_resume_nonboot_cpu_disable(void)
+{
+	if (sleep_cpu < 0) {
+		pr_err("Failing to resume from hibernate on an unknown CPU\n");
+		return -ENODEV;
+	}
+
+	return freeze_secondary_cpus(sleep_cpu);
+}
+#endif
+
+static int __init riscv_hibernate_init(void)

arch/riscv/kernel/hibernate.c line 424

Quoted 10 lines
arch/riscv/kernel/hibernate.c new file +10 -0
+{
+	hibernate_cpu_context = kzalloc(sizeof(*hibernate_cpu_context), GFP_KERNEL);
+
+	if (WARN_ON(!hibernate_cpu_context))
+		return -ENOMEM;
+
+	return 0;
+}
+
+early_initcall(riscv_hibernate_init);

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#6 lore raw

Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 629 lines · Alexandre Ghiti
Quoted 627 lines · Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid, and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
  arch/riscv/Kconfig                 |   6 + arch/riscv/include/asm/assembler.h |  20 ++ arch/riscv/include/asm/suspend.h   |  19 ++ arch/riscv/kernel/Makefile         |   1 + arch/riscv/kernel/asm-offsets.c    |   5 + arch/riscv/kernel/hibernate-asm.S  |  77 ++++++ arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++

  7 files changed, 552 insertions(+)

  create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c

arch/riscv/Kconfig +0 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"

    source "kernel/power/Kconfig"
  +config ARCH_HIBERNATION_POSSIBLE
+    def_bool y

arch/riscv/Kconfig +4 -0
+
+config ARCH_HIBERNATION_HEADER
+    def_bool HIBERNATION
+

  endmenu # "Power management options"
    menu "CPU Power Management"

arch/riscv/include/asm/assembler.h +0 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
           REG_L    s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)

      .endm
  +/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+    .macro    copy_page a0, a1
+        lui    a2, 0x1
+        add    a2, a2, a0
+1 :
+        REG_L    t0, 0(a1)
+        REG_L    t1, SZREG(a1)

arch/riscv/include/asm/assembler.h +9 -0
+
+        REG_S    t0, 0(a0)
+        REG_S    t1, SZREG(a0)
+
+        addi    a0, a0, 2 * SZREG
+        addi    a1, a1, 2 * SZREG
+        bne    a2, a0, 1b
+    .endm
+

  #endif    /* __ASM_ASSEMBLER_H */

arch/riscv/include/asm/suspend.h +0 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
   #endif
   };

  +/*
+ * Used by hibernation core and cleared during resume sequence
+ */

arch/riscv/include/asm/suspend.h +2 -0
+extern int in_suspend;
+

  /* Low-level CPU suspend entry function */
  int __cpu_suspend_enter(struct suspend_context *context);
  @@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid, unsigned long context); /* Used to save and restore the CSRs */
  void suspend_save_csrs(struct suspend_context *context);
  void suspend_restore_csrs(struct suspend_context *context);

arch/riscv/include/asm/suspend.h +12 -0
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp, 

unsigned long satp_temp,
+                    unsigned long cpu_resume); +asmlinkage int hibernate_core_restore_code(void); #endif

arch/riscv/kernel/Makefile +0 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)        += module.o

  obj-$(CONFIG_MODULE_SECTIONS)    += module-sections.o obj-$(CONFIG_CPU_PM)        += suspend_entry.o suspend.o +obj-$(CONFIG_HIBERNATION)    += hibernate.o hibernate-asm.o obj-$(CONFIG_FUNCTION_TRACER)    += mcount.o ftrace.o obj-$(CONFIG_DYNAMIC_FTRACE)    += mcount-dyn.o

arch/riscv/kernel/asm-offsets.c +4 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
   #include <linux/kbuild.h>
   #include <linux/mm.h>
   #include <linux/sched.h>
+#include <linux/suspend.h>
   #include <asm/kvm_host.h>
   #include <asm/thread_info.h>
   #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
         OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
   +    OFFSET(HIBERN_PBE_ADDR, pbe, address);
+    OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+    OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
       OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
       OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
       OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +19 -0
new file mode 100644
index 000000000000..effaf5ca5da0
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hibernation low level support for RISCV.
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/csr.h>
+
+#include <linux/linkage.h>
+
+/*
+ * int __hibernate_cpu_resume(void)
+ * Switch back to the hibernated image's page table prior to 

restoring the CPU
+ * context.
+ *
+ * Always returns 0
+ */

arch/riscv/kernel/hibernate-asm.S new file +35 -0
+ENTRY(__hibernate_cpu_resume)
+    /* switch to hibernated image's page table. */
+    csrw CSR_SATP, s0
+    sfence.vma
+
+    REG_L    a0, hibernate_cpu_context
+
+    suspend_restore_csrs
+    suspend_restore_regs
+
+    /* Return zero value. */
+    mv    a0, zero
+
+    ret
+END(__hibernate_cpu_resume)
+
+/*
+ * Prepare to restore the image.
+ * a0: satp of saved page tables.
+ * a1: satp of temporary page tables.
+ * a2: cpu_resume.
+ */
+ENTRY(hibernate_restore_image)
+    mv    s0, a0
+    mv    s1, a1
+    mv    s2, a2
+    REG_L    s4, restore_pblist
+    REG_L    a1, relocated_restore_code
+
+    jalr    a1
+END(hibernate_restore_image)
+
+/*
+ * The below code will be executed from a 'safe' page.
+ * It first switches to the temporary page table, then starts to 

copy the pages
+ * back to the original memory location. Finally, it jumps to __hibernate_cpu_resume()
+ * to restore the CPU context.
+ */

arch/riscv/kernel/hibernate-asm.S new file +16 -0
+ENTRY(hibernate_core_restore_code)
+    /* switch to temp page table. */
+    csrw satp, s1
+    sfence.vma
+.Lcopy:
+    /* The below code will restore the hibernated image. */
+    REG_L    a1, HIBERN_PBE_ADDR(s4)
+    REG_L    a0, HIBERN_PBE_ORIG(s4)
+
+    copy_page a0, a1
+
+    REG_L    s4, HIBERN_PBE_NEXT(s4)
+    bnez    s4, .Lcopy
+
+    jalr    s2
+END(hibernate_core_restore_code)
arch/riscv/kernel/hibernate.c new file +28 -0
new file mode 100644
index 000000000000..8fab50122e54
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hibernation support for RISCV
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/barrier.h>
+#include <asm/cacheflush.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/sections.h>
+#include <asm/set_memory.h>
+#include <asm/smp.h>
+#include <asm/suspend.h>
+
+#include <linux/cpu.h>
+#include <linux/memblock.h>
+#include <linux/pm.h>
+#include <linux/sched.h>
+#include <linux/suspend.h>
+#include <linux/utsname.h>
+
+/* The logical cpu number we should resume on, initialised to a 

non-cpu number. */

arch/riscv/kernel/hibernate.c new file +14 -0
+static int sleep_cpu = -EINVAL;
+
+/* Pointer to the temporary resume page table. */
+static pgd_t *resume_pg_dir;
+
+/* CPU context to be saved. */
+struct suspend_context *hibernate_cpu_context;
+EXPORT_SYMBOL_GPL(hibernate_cpu_context);
+
+unsigned long relocated_restore_code;
+EXPORT_SYMBOL_GPL(relocated_restore_code);
+
+/**
+ * struct arch_hibernate_hdr_invariants - container to store kernel 

build version.
+ * @uts_version: to save the build number and date so that the we do not resume with
+ *        a different kernel.
+ */

arch/riscv/kernel/hibernate.c new file +6 -0
+struct arch_hibernate_hdr_invariants {
+    char        uts_version[__NEW_UTS_LEN + 1];
+};
+
+/**
+ * struct arch_hibernate_hdr - helper parameters that help us to 

restore the image.
+ * @invariants: container to store kernel build version.
+ * @hartid: to make sure same boot_cpu executes the hibernate/restore code.
+ * @saved_satp: original page table used by the hibernated image.
+ * @restore_cpu_addr: the kernel's image address to restore the CPU context.
+ */

arch/riscv/kernel/hibernate.c new file +8 -0
+static struct arch_hibernate_hdr {
+    struct arch_hibernate_hdr_invariants invariants;
+    unsigned long    hartid;
+    unsigned long    saved_satp;
+    unsigned long    restore_cpu_addr;
+} resume_hdr;
+
+static void arch_hdr_invariants(struct arch_hibernate_hdr_invariants 

*i)
+{
+    memset(i, 0, sizeof(*i));
+    memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version));

arch/riscv/kernel/hibernate.c new file +93 -0
+}
+
+/*
+ * Check if the given pfn is in the 'nosave' section.
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+    unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin);
+    unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
+
+    return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn));
+}
+
+void notrace save_processor_state(void)
+{
+    WARN_ON(num_online_cpus() != 1);
+}
+
+void notrace restore_processor_state(void)
+{
+}
+
+/*
+ * Helper parameters need to be saved to the hibernation image header.
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+    struct arch_hibernate_hdr *hdr = addr;
+
+    if (max_size < sizeof(*hdr))
+        return -EOVERFLOW;
+
+    arch_hdr_invariants(&hdr->invariants);
+
+    hdr->hartid = cpuid_to_hartid_map(sleep_cpu);
+    hdr->saved_satp = csr_read(CSR_SATP);
+    hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume;
+
+    return 0;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_save);
+
+/*
+ * Retrieve the helper parameters from the hibernation image header.
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+    struct arch_hibernate_hdr_invariants invariants;
+    struct arch_hibernate_hdr *hdr = addr;
+    int ret = 0;
+
+    arch_hdr_invariants(&invariants);
+
+    if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) {
+        pr_crit("Hibernate image not generated by this kernel!\n");
+        return -EINVAL;
+    }
+
+    sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid);
+    if (sleep_cpu < 0) {
+        pr_crit("Hibernated on a CPU not known to this kernel!\n");
+        sleep_cpu = -EINVAL;
+        return -EINVAL;
+    }
+
+#ifdef CONFIG_SMP
+    ret = bringup_hibernate_cpu(sleep_cpu);
+    if (ret) {
+        sleep_cpu = -EINVAL;
+        return ret;
+    }
+#endif
+    resume_hdr = *hdr;
+
+    return ret;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_restore);
+
+int swsusp_arch_suspend(void)
+{
+    int ret = 0;
+
+    if (__cpu_suspend_enter(hibernate_cpu_context)) {
+        sleep_cpu = smp_processor_id();
+        suspend_save_csrs(hibernate_cpu_context);
+        ret = swsusp_save();
+    } else {
+        suspend_restore_csrs(hibernate_cpu_context);
+        flush_tlb_all();
+        flush_icache_all();
+
+        /*
+         * Tell the hibernation core that we've just restored the 

memory.
+         */
+        in_suspend = 0;
+        sleep_cpu = -EINVAL;
+    }

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return ret;
+}
+
+static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, 

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +26 -0
+{
+    pte_t *src_ptep;
+    pte_t *dst_ptep;
+
+    if (pmd_none(READ_ONCE(*dst_pmdp))) {
+        dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_ptep)
+            return -ENOMEM;
+
+        pmd_populate_kernel(NULL, dst_pmdp, dst_ptep);
+    }
+
+    dst_ptep = pte_offset_kernel(dst_pmdp, start);
+    src_ptep = pte_offset_kernel(src_pmdp, start);
+
+    do {
+        pte_t pte = READ_ONCE(*src_ptep);
+
+        if (pte_present(pte))
+            set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot)));
+    } while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end);
+
+    return 0;
+}
+
+static int temp_pgtable_map_pmd(pud_t *dst_pudp, pud_t *src_pudp, 

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pmd_t *src_pmdp;
+    pmd_t *dst_pmdp;
+
+    if (pud_none(READ_ONCE(*dst_pudp))) {
+        dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pmdp)
+            return -ENOMEM;
+
+        pud_populate(NULL, dst_pudp, dst_pmdp);
+    }
+
+    dst_pmdp = pmd_offset(dst_pudp, start);
+    src_pmdp = pmd_offset(src_pudp, start);
+
+    do {
+        pmd_t pmd = READ_ONCE(*src_pmdp);
+
+        next = pmd_addr_end(start, end);
+
+        if (pmd_none(pmd))
+            continue;
+
+        if (pmd_leaf(pmd)) {
+            set_pmd(dst_pmdp, __pmd(pmd_val(pmd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pte(dst_pmdp, src_pmdp, start, 

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pmdp++, src_pmdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, 

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pud_t *dst_pudp;
+    pud_t *src_pudp;
+
+    if (p4d_none(READ_ONCE(*dst_p4dp))) {
+        dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pudp)
+            return -ENOMEM;
+
+        p4d_populate(NULL, dst_p4dp, dst_pudp);
+    }
+
+    dst_pudp = pud_offset(dst_p4dp, start);
+    src_pudp = pud_offset(src_p4dp, start);
+
+    do {
+        pud_t pud = READ_ONCE(*src_pudp);
+
+        next = pud_addr_end(start, end);
+
+        if (pud_none(pud))
+            continue;
+
+        if (pud_leaf(pud)) {
+            set_pud(dst_pudp, __pud(pud_val(pud) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pmd(dst_pudp, src_pudp, start, 

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pudp++, src_pudp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp, 

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    p4d_t *dst_p4dp;
+    p4d_t *src_p4dp;
+
+    if (pgd_none(READ_ONCE(*dst_pgdp))) {
+        dst_p4dp = (p4d_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_p4dp)
+            return -ENOMEM;
+
+        pgd_populate(NULL, dst_pgdp, dst_p4dp);
+    }
+
+    dst_p4dp = p4d_offset(dst_pgdp, start);
+    src_p4dp = p4d_offset(src_pgdp, start);
+
+    do {
+        p4d_t p4d = READ_ONCE(*src_p4dp);
+
+        next = p4d_addr_end(start, end);
+
+        if (p4d_none(p4d))
+            continue;
+
+        if (p4d_leaf(p4d)) {
+            set_p4d(dst_p4dp, __p4d(p4d_val(p4d) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pud(dst_p4dp, src_p4dp, start, 

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_p4dp++, src_p4dp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_mapping(pgd_t *pgdp, unsigned long start, 

unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +18 -0
+{
+    pgd_t *dst_pgdp = pgd_offset_pgd(pgdp, start);
+    pgd_t *src_pgdp = pgd_offset_k(start);
+    unsigned long next;
+    unsigned long ret;
+
+    do {
+        pgd_t pgd = READ_ONCE(*src_pgdp);
+
+        next = pgd_addr_end(start, end);
+
+        if (pgd_none(pgd))
+            continue;
+
+        if (pgd_leaf(pgd)) {
+            set_pgd(dst_pgdp, __pgd(pgd_val(pgd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_p4d(dst_pgdp, src_pgdp, start, 

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pgdp++, src_pgdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +27 -0
+
+    return 0;
+}
+
+static unsigned long relocate_restore_code(void)
+{
+    void *page = (void *)get_safe_page(GFP_ATOMIC);
+
+    if (!page)
+        return -ENOMEM;
+
+    copy_page(page, hibernate_core_restore_code);
+
+    /* Make the page containing the relocated code executable. */
+    set_memory_x((unsigned long)page, 1);
+
+    return (unsigned long)page;
+}
+
+int swsusp_arch_resume(void)
+{
+    unsigned long end = (unsigned long)pfn_to_virt(max_low_pfn);
+    unsigned long start = PAGE_OFFSET;
+    int ret;
+
+    /*
+     * Memory allocated by get_safe_page() will be dealt with by the 

hibernation core,
+     * we don't need to free it here.
+     */
+    resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC);
+    if (!resume_pg_dir)
+        return -ENOMEM;
+
+    /*
+     * Create a temporary page table and map the whole linear region

Quoted 8 lines
Quoted 5 lines

as executable and
+     * writable.
+     */
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));

Hmm weird protections, PAGE_PRESENT and PAGE_READ are not even set. Shouldn't you use PAGE_KERNEL_EXEC instead?

Forget this one, that's the protections you add, so that's good :)

Quoted 70 lines
Quoted 20 lines

+    if (ret)
+        return ret;
+
+    /* Move the restore code to a new page so that it doesn't get overwritten by itself. */
+    relocated_restore_code = relocate_restore_code();
+    if (relocated_restore_code == -ENOMEM)
+        return -ENOMEM;
+
+    /*
+     * Map the __hibernate_cpu_resume() address to the temporary page table so that the
+     * restore code can jumps to it after finished restore the image. The next execution
+     * code doesn't find itself in a different address space after switching over to the
+     * original page table used by the hibernated image. +     *

Quoted 8 lines

+     */
+    start = (unsigned long)resume_hdr.restore_cpu_addr; +    end = start + PAGE_SIZE;
+
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));
+    if (ret)
+        return ret;

You set the linear mapping as executable so no need for this.

Quoted 24 lines

+
+    hibernate_restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), +                resume_hdr.restore_cpu_addr);

arch/riscv/kernel/hibernate.c new file +19 -0
+
+    return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP_SMP
+int hibernate_resume_nonboot_cpu_disable(void)
+{
+    if (sleep_cpu < 0) {
+        pr_err("Failing to resume from hibernate on an unknown CPU\n");
+        return -ENODEV;
+    }
+
+    return freeze_secondary_cpus(sleep_cpu);
+}
+#endif
+
+static int __init riscv_hibernate_init(void)
+{
+    hibernate_cpu_context = kzalloc(sizeof(*hibernate_cpu_context), 

GFP_KERNEL);

arch/riscv/kernel/hibernate.c line 424

Quoted 8 lines
arch/riscv/kernel/hibernate.c new file +8 -0
+
+    if (WARN_ON(!hibernate_cpu_context))
+        return -ENOMEM;
+
+    return 0;
+}
+
+early_initcall(riscv_hibernate_init);

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#7 lore raw

RE: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 659 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Wednesday, March 8, 2023 5:28 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 637 lines · Alexandre Ghiti
Quoted 635 lines · Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid,
and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
  arch/riscv/Kconfig                 |   6 + arch/riscv/include/asm/assembler.h |  20 ++ arch/riscv/include/asm/suspend.h   |  19 ++ arch/riscv/kernel/Makefile         |   1 + arch/riscv/kernel/asm-offsets.c    |   5 + arch/riscv/kernel/hibernate-asm.S  |  77 ++++++ arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++

  7 files changed, 552 insertions(+)

  create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c

arch/riscv/Kconfig +0 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"

    source "kernel/power/Kconfig"
  +config ARCH_HIBERNATION_POSSIBLE
+    def_bool y

arch/riscv/Kconfig +4 -0
+
+config ARCH_HIBERNATION_HEADER
+    def_bool HIBERNATION
+

  endmenu # "Power management options"
    menu "CPU Power Management"

arch/riscv/include/asm/assembler.h +0 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
           REG_L    s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)

      .endm
  +/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+    .macro    copy_page a0, a1
+        lui    a2, 0x1
+        add    a2, a2, a0
+1 :
+        REG_L    t0, 0(a1)
+        REG_L    t1, SZREG(a1)

arch/riscv/include/asm/assembler.h +9 -0
+
+        REG_S    t0, 0(a0)
+        REG_S    t1, SZREG(a0)
+
+        addi    a0, a0, 2 * SZREG
+        addi    a1, a1, 2 * SZREG
+        bne    a2, a0, 1b
+    .endm
+

  #endif    /* __ASM_ASSEMBLER_H */

arch/riscv/include/asm/suspend.h +0 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
   #endif
   };

  +/*
+ * Used by hibernation core and cleared during resume sequence
+ */

arch/riscv/include/asm/suspend.h +2 -0
+extern int in_suspend;
+

  /* Low-level CPU suspend entry function */
  int __cpu_suspend_enter(struct suspend_context *context); @@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid, unsigned long context);
  /* Used to save and restore the CSRs */
  void suspend_save_csrs(struct suspend_context *context); void suspend_restore_csrs(struct suspend_context *context);

arch/riscv/include/asm/suspend.h +12 -0
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp,

unsigned long satp_temp,
+                    unsigned long cpu_resume); +asmlinkage int hibernate_core_restore_code(void); #endif

arch/riscv/kernel/Makefile +0 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)        += module.o

  obj-$(CONFIG_MODULE_SECTIONS)    += module-sections.o obj-$(CONFIG_CPU_PM)        += suspend_entry.o suspend.o +obj-$(CONFIG_HIBERNATION)    += hibernate.o hibernate-asm.o obj-$(CONFIG_FUNCTION_TRACER)    += mcount.o ftrace.o obj-$(CONFIG_DYNAMIC_FTRACE)    += mcount-dyn.o

arch/riscv/kernel/asm-offsets.c +4 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
   #include <linux/kbuild.h>
   #include <linux/mm.h>
   #include <linux/sched.h>
+#include <linux/suspend.h>
   #include <asm/kvm_host.h>
   #include <asm/thread_info.h>
   #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
         OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
   +    OFFSET(HIBERN_PBE_ADDR, pbe, address);
+    OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+    OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
       OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
       OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
       OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +19 -0
new file mode 100644
index 000000000000..effaf5ca5da0
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hibernation low level support for RISCV.
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/csr.h>
+
+#include <linux/linkage.h>
+
+/*
+ * int __hibernate_cpu_resume(void)
+ * Switch back to the hibernated image's page table prior to

restoring the CPU
+ * context.
+ *
+ * Always returns 0
+ */

arch/riscv/kernel/hibernate-asm.S new file +35 -0
+ENTRY(__hibernate_cpu_resume)
+    /* switch to hibernated image's page table. */
+    csrw CSR_SATP, s0
+    sfence.vma
+
+    REG_L    a0, hibernate_cpu_context
+
+    suspend_restore_csrs
+    suspend_restore_regs
+
+    /* Return zero value. */
+    mv    a0, zero
+
+    ret
+END(__hibernate_cpu_resume)
+
+/*
+ * Prepare to restore the image.
+ * a0: satp of saved page tables.
+ * a1: satp of temporary page tables.
+ * a2: cpu_resume.
+ */
+ENTRY(hibernate_restore_image)
+    mv    s0, a0
+    mv    s1, a1
+    mv    s2, a2
+    REG_L    s4, restore_pblist
+    REG_L    a1, relocated_restore_code
+
+    jalr    a1
+END(hibernate_restore_image)
+
+/*
+ * The below code will be executed from a 'safe' page.
+ * It first switches to the temporary page table, then starts to

copy the pages
+ * back to the original memory location. Finally, it jumps to __hibernate_cpu_resume()
+ * to restore the CPU context.
+ */

arch/riscv/kernel/hibernate-asm.S new file +16 -0
+ENTRY(hibernate_core_restore_code)
+    /* switch to temp page table. */
+    csrw satp, s1
+    sfence.vma
+.Lcopy:
+    /* The below code will restore the hibernated image. */
+    REG_L    a1, HIBERN_PBE_ADDR(s4)
+    REG_L    a0, HIBERN_PBE_ORIG(s4)
+
+    copy_page a0, a1
+
+    REG_L    s4, HIBERN_PBE_NEXT(s4)
+    bnez    s4, .Lcopy
+
+    jalr    s2
+END(hibernate_core_restore_code)
arch/riscv/kernel/hibernate.c new file +28 -0
new file mode 100644
index 000000000000..8fab50122e54
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hibernation support for RISCV
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/barrier.h>
+#include <asm/cacheflush.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/sections.h>
+#include <asm/set_memory.h>
+#include <asm/smp.h>
+#include <asm/suspend.h>
+
+#include <linux/cpu.h>
+#include <linux/memblock.h>
+#include <linux/pm.h>
+#include <linux/sched.h>
+#include <linux/suspend.h>
+#include <linux/utsname.h>
+
+/* The logical cpu number we should resume on, initialised to a

non-cpu number. */

arch/riscv/kernel/hibernate.c new file +14 -0
+static int sleep_cpu = -EINVAL;
+
+/* Pointer to the temporary resume page table. */
+static pgd_t *resume_pg_dir;
+
+/* CPU context to be saved. */
+struct suspend_context *hibernate_cpu_context;
+EXPORT_SYMBOL_GPL(hibernate_cpu_context);
+
+unsigned long relocated_restore_code;
+EXPORT_SYMBOL_GPL(relocated_restore_code);
+
+/**
+ * struct arch_hibernate_hdr_invariants - container to store kernel

build version.
+ * @uts_version: to save the build number and date so that the we do not resume with
+ *        a different kernel.
+ */

arch/riscv/kernel/hibernate.c new file +6 -0
+struct arch_hibernate_hdr_invariants {
+    char        uts_version[__NEW_UTS_LEN + 1];
+};
+
+/**
+ * struct arch_hibernate_hdr - helper parameters that help us to

restore the image.
+ * @invariants: container to store kernel build version.
+ * @hartid: to make sure same boot_cpu executes the hibernate/restore code.
+ * @saved_satp: original page table used by the hibernated image.
+ * @restore_cpu_addr: the kernel's image address to restore the CPU context.
+ */

arch/riscv/kernel/hibernate.c new file +8 -0
+static struct arch_hibernate_hdr {
+    struct arch_hibernate_hdr_invariants invariants;
+    unsigned long    hartid;
+    unsigned long    saved_satp;
+    unsigned long    restore_cpu_addr;
+} resume_hdr;
+
+static void arch_hdr_invariants(struct arch_hibernate_hdr_invariants

*i)
+{
+    memset(i, 0, sizeof(*i));
+    memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version));

arch/riscv/kernel/hibernate.c new file +93 -0
+}
+
+/*
+ * Check if the given pfn is in the 'nosave' section.
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+    unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin);
+    unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
+
+    return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn));
+}
+
+void notrace save_processor_state(void)
+{
+    WARN_ON(num_online_cpus() != 1);
+}
+
+void notrace restore_processor_state(void)
+{
+}
+
+/*
+ * Helper parameters need to be saved to the hibernation image header.
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+    struct arch_hibernate_hdr *hdr = addr;
+
+    if (max_size < sizeof(*hdr))
+        return -EOVERFLOW;
+
+    arch_hdr_invariants(&hdr->invariants);
+
+    hdr->hartid = cpuid_to_hartid_map(sleep_cpu);
+    hdr->saved_satp = csr_read(CSR_SATP);
+    hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume;
+
+    return 0;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_save);
+
+/*
+ * Retrieve the helper parameters from the hibernation image header.
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+    struct arch_hibernate_hdr_invariants invariants;
+    struct arch_hibernate_hdr *hdr = addr;
+    int ret = 0;
+
+    arch_hdr_invariants(&invariants);
+
+    if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) {
+        pr_crit("Hibernate image not generated by this kernel!\n");
+        return -EINVAL;
+    }
+
+    sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid);
+    if (sleep_cpu < 0) {
+        pr_crit("Hibernated on a CPU not known to this kernel!\n");
+        sleep_cpu = -EINVAL;
+        return -EINVAL;
+    }
+
+#ifdef CONFIG_SMP
+    ret = bringup_hibernate_cpu(sleep_cpu);
+    if (ret) {
+        sleep_cpu = -EINVAL;
+        return ret;
+    }
+#endif
+    resume_hdr = *hdr;
+
+    return ret;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_restore);
+
+int swsusp_arch_suspend(void)
+{
+    int ret = 0;
+
+    if (__cpu_suspend_enter(hibernate_cpu_context)) {
+        sleep_cpu = smp_processor_id();
+        suspend_save_csrs(hibernate_cpu_context);
+        ret = swsusp_save();
+    } else {
+        suspend_restore_csrs(hibernate_cpu_context);
+        flush_tlb_all();
+        flush_icache_all();
+
+        /*
+         * Tell the hibernation core that we've just restored the

memory.
+         */
+        in_suspend = 0;
+        sleep_cpu = -EINVAL;
+    }

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return ret;
+}
+
+static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +26 -0
+{
+    pte_t *src_ptep;
+    pte_t *dst_ptep;
+
+    if (pmd_none(READ_ONCE(*dst_pmdp))) {
+        dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_ptep)
+            return -ENOMEM;
+
+        pmd_populate_kernel(NULL, dst_pmdp, dst_ptep);
+    }
+
+    dst_ptep = pte_offset_kernel(dst_pmdp, start);
+    src_ptep = pte_offset_kernel(src_pmdp, start);
+
+    do {
+        pte_t pte = READ_ONCE(*src_ptep);
+
+        if (pte_present(pte))
+            set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot)));
+    } while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end);
+
+    return 0;
+}
+
+static int temp_pgtable_map_pmd(pud_t *dst_pudp, pud_t *src_pudp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pmd_t *src_pmdp;
+    pmd_t *dst_pmdp;
+
+    if (pud_none(READ_ONCE(*dst_pudp))) {
+        dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pmdp)
+            return -ENOMEM;
+
+        pud_populate(NULL, dst_pudp, dst_pmdp);
+    }
+
+    dst_pmdp = pmd_offset(dst_pudp, start);
+    src_pmdp = pmd_offset(src_pudp, start);
+
+    do {
+        pmd_t pmd = READ_ONCE(*src_pmdp);
+
+        next = pmd_addr_end(start, end);
+
+        if (pmd_none(pmd))
+            continue;
+
+        if (pmd_leaf(pmd)) {
+            set_pmd(dst_pmdp, __pmd(pmd_val(pmd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pte(dst_pmdp, src_pmdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pmdp++, src_pmdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pud_t *dst_pudp;
+    pud_t *src_pudp;
+
+    if (p4d_none(READ_ONCE(*dst_p4dp))) {
+        dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pudp)
+            return -ENOMEM;
+
+        p4d_populate(NULL, dst_p4dp, dst_pudp);
+    }
+
+    dst_pudp = pud_offset(dst_p4dp, start);
+    src_pudp = pud_offset(src_p4dp, start);
+
+    do {
+        pud_t pud = READ_ONCE(*src_pudp);
+
+        next = pud_addr_end(start, end);
+
+        if (pud_none(pud))
+            continue;
+
+        if (pud_leaf(pud)) {
+            set_pud(dst_pudp, __pud(pud_val(pud) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pmd(dst_pudp, src_pudp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pudp++, src_pudp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    p4d_t *dst_p4dp;
+    p4d_t *src_p4dp;
+
+    if (pgd_none(READ_ONCE(*dst_pgdp))) {
+        dst_p4dp = (p4d_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_p4dp)
+            return -ENOMEM;
+
+        pgd_populate(NULL, dst_pgdp, dst_p4dp);
+    }
+
+    dst_p4dp = p4d_offset(dst_pgdp, start);
+    src_p4dp = p4d_offset(src_pgdp, start);
+
+    do {
+        p4d_t p4d = READ_ONCE(*src_p4dp);
+
+        next = p4d_addr_end(start, end);
+
+        if (p4d_none(p4d))
+            continue;
+
+        if (p4d_leaf(p4d)) {
+            set_p4d(dst_p4dp, __p4d(p4d_val(p4d) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pud(dst_p4dp, src_p4dp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_p4dp++, src_p4dp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_mapping(pgd_t *pgdp, unsigned long start,

unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +18 -0
+{
+    pgd_t *dst_pgdp = pgd_offset_pgd(pgdp, start);
+    pgd_t *src_pgdp = pgd_offset_k(start);
+    unsigned long next;
+    unsigned long ret;
+
+    do {
+        pgd_t pgd = READ_ONCE(*src_pgdp);
+
+        next = pgd_addr_end(start, end);
+
+        if (pgd_none(pgd))
+            continue;
+
+        if (pgd_leaf(pgd)) {
+            set_pgd(dst_pgdp, __pgd(pgd_val(pgd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_p4d(dst_pgdp, src_pgdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pgdp++, src_pgdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +27 -0
+
+    return 0;
+}
+
+static unsigned long relocate_restore_code(void)
+{
+    void *page = (void *)get_safe_page(GFP_ATOMIC);
+
+    if (!page)
+        return -ENOMEM;
+
+    copy_page(page, hibernate_core_restore_code);
+
+    /* Make the page containing the relocated code executable. */
+    set_memory_x((unsigned long)page, 1);
+
+    return (unsigned long)page;
+}
+
+int swsusp_arch_resume(void)
+{
+    unsigned long end = (unsigned long)pfn_to_virt(max_low_pfn);
+    unsigned long start = PAGE_OFFSET;
+    int ret;
+
+    /*
+     * Memory allocated by get_safe_page() will be dealt with by the

hibernation core,
+     * we don't need to free it here.
+     */
+    resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC);
+    if (!resume_pg_dir)
+        return -ENOMEM;
+
+    /*
+     * Create a temporary page table and map the whole linear region as executable and

Quoted 8 lines
Quoted 4 lines

+     * writable.
+     */
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));

Hmm weird protections, PAGE_PRESENT and PAGE_READ are not even set. Shouldn't you use PAGE_KERNEL_EXEC instead?

Quoted 1 line

Forget this one, that's the protections you add, so that's good :)

OK.

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

+    if (ret)
+        return ret;
+
+    /* Move the restore code to a new page so that it doesn't get overwritten by itself. */
+    relocated_restore_code = relocate_restore_code();
+    if (relocated_restore_code == -ENOMEM)
+        return -ENOMEM;
+
+    /*
+     * Map the __hibernate_cpu_resume() address to the temporary page table so that the
+     * restore code can jumps to it after finished restore the image. The next execution
+     * code doesn't find itself in a different address space after switching over to the
+     * original page table used by the hibernated image. +     *
+     */
+    start = (unsigned long)resume_hdr.restore_cpu_addr; +    end = start + PAGE_SIZE;

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

+
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));
+    if (ret)
+        return ret;

You set the linear mapping as executable so no need for this.

We need this mapping because the kernel addresses and the linear addresses in riscv-64bit are in different boundary. For example:
Linear address start with 0xff60000000000000
Kernel address start with 0xffffffff80000000

Quoted 39 lines
Quoted 39 lines
Quoted 26 lines

+
+    hibernate_restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), +                resume_hdr.restore_cpu_addr);

arch/riscv/kernel/hibernate.c new file +19 -0
+
+    return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP_SMP
+int hibernate_resume_nonboot_cpu_disable(void)
+{
+    if (sleep_cpu < 0) {
+        pr_err("Failing to resume from hibernate on an unknown CPU\n");
+        return -ENODEV;
+    }
+
+    return freeze_secondary_cpus(sleep_cpu);
+}
+#endif
+
+static int __init riscv_hibernate_init(void)
+{
+    hibernate_cpu_context = kzalloc(sizeof(*hibernate_cpu_context),

GFP_KERNEL);

arch/riscv/kernel/hibernate.c line 424

Quoted 8 lines
arch/riscv/kernel/hibernate.c new file +8 -0
+
+    if (WARN_ON(!hibernate_cpu_context))
+        return -ENOMEM;
+
+    return 0;
+}
+
+early_initcall(riscv_hibernate_init);

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#8 lore raw

Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Hi JeeHeng,

Quoted 687 lines · JeeHeng Sia
Quoted 655 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Wednesday, March 8, 2023 5:28 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 635 lines · Alexandre Ghiti
Quoted 634 lines · Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid,
and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
   arch/riscv/Kconfig                 |   6 +
   arch/riscv/include/asm/assembler.h |  20 ++
   arch/riscv/include/asm/suspend.h   |  19 ++
   arch/riscv/kernel/Makefile         |   1 +
   arch/riscv/kernel/asm-offsets.c    |   5 +
   arch/riscv/kernel/hibernate-asm.S  |  77 ++++++
   arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++
   7 files changed, 552 insertions(+)

  create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c

arch/riscv/Kconfig +5 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"
     source "kernel/power/Kconfig"
   +config ARCH_HIBERNATION_POSSIBLE
+    def_bool y
+
+config ARCH_HIBERNATION_HEADER
+    def_bool HIBERNATION
+
   endmenu # "Power management options"
     menu "CPU Power Management"
arch/riscv/include/asm/assembler.h +19 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
           REG_L    s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
       .endm
   +/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+    .macro    copy_page a0, a1
+        lui    a2, 0x1
+        add    a2, a2, a0
+1 :
+        REG_L    t0, 0(a1)
+        REG_L    t1, SZREG(a1)
+
+        REG_S    t0, 0(a0)
+        REG_S    t1, SZREG(a0)
+
+        addi    a0, a0, 2 * SZREG
+        addi    a1, a1, 2 * SZREG
+        bne    a2, a0, 1b
+    .endm
+
   #endif    /* __ASM_ASSEMBLER_H */
arch/riscv/include/asm/suspend.h +4 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
   #endif
   };
   +/*
+ * Used by hibernation core and cleared during resume sequence
+ */
+extern int in_suspend;
+
   /* Low-level CPU suspend entry function */
   int __cpu_suspend_enter(struct suspend_context *context);
   @@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid,

unsigned long context);
  /* Used to save and restore the CSRs */
  void suspend_save_csrs(struct suspend_context *context); void suspend_restore_csrs(struct suspend_context *context);

arch/riscv/include/asm/suspend.h +12 -0
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp,

unsigned long satp_temp,
+                    unsigned long cpu_resume);

arch/riscv/include/asm/suspend.h +1 -0
+asmlinkage int hibernate_core_restore_code(void);
   #endif
arch/riscv/kernel/Makefile +1 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)        += module.o
   obj-$(CONFIG_MODULE_SECTIONS)    += module-sections.o
     obj-$(CONFIG_CPU_PM)        += suspend_entry.o suspend.o
+obj-$(CONFIG_HIBERNATION)    += hibernate.o hibernate-asm.o
     obj-$(CONFIG_FUNCTION_TRACER)    += mcount.o ftrace.o
   obj-$(CONFIG_DYNAMIC_FTRACE)    += mcount-dyn.o
arch/riscv/kernel/asm-offsets.c +4 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
   #include <linux/kbuild.h>
   #include <linux/mm.h>
   #include <linux/sched.h>
+#include <linux/suspend.h>
   #include <asm/kvm_host.h>
   #include <asm/thread_info.h>
   #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
         OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
   +    OFFSET(HIBERN_PBE_ADDR, pbe, address);
+    OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+    OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
       OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
       OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
       OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +19 -0
new file mode 100644
index 000000000000..effaf5ca5da0
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hibernation low level support for RISCV.
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/csr.h>
+
+#include <linux/linkage.h>
+
+/*
+ * int __hibernate_cpu_resume(void)
+ * Switch back to the hibernated image's page table prior to

restoring the CPU
+ * context.
+ *
+ * Always returns 0
+ */

arch/riscv/kernel/hibernate-asm.S new file +35 -0
+ENTRY(__hibernate_cpu_resume)
+    /* switch to hibernated image's page table. */
+    csrw CSR_SATP, s0
+    sfence.vma
+
+    REG_L    a0, hibernate_cpu_context
+
+    suspend_restore_csrs
+    suspend_restore_regs
+
+    /* Return zero value. */
+    mv    a0, zero
+
+    ret
+END(__hibernate_cpu_resume)
+
+/*
+ * Prepare to restore the image.
+ * a0: satp of saved page tables.
+ * a1: satp of temporary page tables.
+ * a2: cpu_resume.
+ */
+ENTRY(hibernate_restore_image)
+    mv    s0, a0
+    mv    s1, a1
+    mv    s2, a2
+    REG_L    s4, restore_pblist
+    REG_L    a1, relocated_restore_code
+
+    jalr    a1
+END(hibernate_restore_image)
+
+/*
+ * The below code will be executed from a 'safe' page.
+ * It first switches to the temporary page table, then starts to

copy the pages
+ * back to the original memory location. Finally, it jumps to __hibernate_cpu_resume()
+ * to restore the CPU context.
+ */

arch/riscv/kernel/hibernate-asm.S new file +16 -0
+ENTRY(hibernate_core_restore_code)
+    /* switch to temp page table. */
+    csrw satp, s1
+    sfence.vma
+.Lcopy:
+    /* The below code will restore the hibernated image. */
+    REG_L    a1, HIBERN_PBE_ADDR(s4)
+    REG_L    a0, HIBERN_PBE_ORIG(s4)
+
+    copy_page a0, a1
+
+    REG_L    s4, HIBERN_PBE_NEXT(s4)
+    bnez    s4, .Lcopy
+
+    jalr    s2
+END(hibernate_core_restore_code)
arch/riscv/kernel/hibernate.c new file +28 -0
new file mode 100644
index 000000000000..8fab50122e54
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hibernation support for RISCV
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/barrier.h>
+#include <asm/cacheflush.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/sections.h>
+#include <asm/set_memory.h>
+#include <asm/smp.h>
+#include <asm/suspend.h>
+
+#include <linux/cpu.h>
+#include <linux/memblock.h>
+#include <linux/pm.h>
+#include <linux/sched.h>
+#include <linux/suspend.h>
+#include <linux/utsname.h>
+
+/* The logical cpu number we should resume on, initialised to a

non-cpu number. */

arch/riscv/kernel/hibernate.c new file +14 -0
+static int sleep_cpu = -EINVAL;
+
+/* Pointer to the temporary resume page table. */
+static pgd_t *resume_pg_dir;
+
+/* CPU context to be saved. */
+struct suspend_context *hibernate_cpu_context;
+EXPORT_SYMBOL_GPL(hibernate_cpu_context);
+
+unsigned long relocated_restore_code;
+EXPORT_SYMBOL_GPL(relocated_restore_code);
+
+/**
+ * struct arch_hibernate_hdr_invariants - container to store kernel

build version.
+ * @uts_version: to save the build number and date so that the we do not resume with
+ *        a different kernel.
+ */

arch/riscv/kernel/hibernate.c new file +6 -0
+struct arch_hibernate_hdr_invariants {
+    char        uts_version[__NEW_UTS_LEN + 1];
+};
+
+/**
+ * struct arch_hibernate_hdr - helper parameters that help us to

restore the image.
+ * @invariants: container to store kernel build version.
+ * @hartid: to make sure same boot_cpu executes the hibernate/restore code.
+ * @saved_satp: original page table used by the hibernated image.
+ * @restore_cpu_addr: the kernel's image address to restore the CPU context.
+ */

arch/riscv/kernel/hibernate.c new file +8 -0
+static struct arch_hibernate_hdr {
+    struct arch_hibernate_hdr_invariants invariants;
+    unsigned long    hartid;
+    unsigned long    saved_satp;
+    unsigned long    restore_cpu_addr;
+} resume_hdr;
+
+static void arch_hdr_invariants(struct arch_hibernate_hdr_invariants

*i)
+{
+    memset(i, 0, sizeof(*i));
+    memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version));

arch/riscv/kernel/hibernate.c new file +93 -0
+}
+
+/*
+ * Check if the given pfn is in the 'nosave' section.
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+    unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin);
+    unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
+
+    return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn));
+}
+
+void notrace save_processor_state(void)
+{
+    WARN_ON(num_online_cpus() != 1);
+}
+
+void notrace restore_processor_state(void)
+{
+}
+
+/*
+ * Helper parameters need to be saved to the hibernation image header.
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+    struct arch_hibernate_hdr *hdr = addr;
+
+    if (max_size < sizeof(*hdr))
+        return -EOVERFLOW;
+
+    arch_hdr_invariants(&hdr->invariants);
+
+    hdr->hartid = cpuid_to_hartid_map(sleep_cpu);
+    hdr->saved_satp = csr_read(CSR_SATP);
+    hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume;
+
+    return 0;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_save);
+
+/*
+ * Retrieve the helper parameters from the hibernation image header.
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+    struct arch_hibernate_hdr_invariants invariants;
+    struct arch_hibernate_hdr *hdr = addr;
+    int ret = 0;
+
+    arch_hdr_invariants(&invariants);
+
+    if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) {
+        pr_crit("Hibernate image not generated by this kernel!\n");
+        return -EINVAL;
+    }
+
+    sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid);
+    if (sleep_cpu < 0) {
+        pr_crit("Hibernated on a CPU not known to this kernel!\n");
+        sleep_cpu = -EINVAL;
+        return -EINVAL;
+    }
+
+#ifdef CONFIG_SMP
+    ret = bringup_hibernate_cpu(sleep_cpu);
+    if (ret) {
+        sleep_cpu = -EINVAL;
+        return ret;
+    }
+#endif
+    resume_hdr = *hdr;
+
+    return ret;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_restore);
+
+int swsusp_arch_suspend(void)
+{
+    int ret = 0;
+
+    if (__cpu_suspend_enter(hibernate_cpu_context)) {
+        sleep_cpu = smp_processor_id();
+        suspend_save_csrs(hibernate_cpu_context);
+        ret = swsusp_save();
+    } else {
+        suspend_restore_csrs(hibernate_cpu_context);
+        flush_tlb_all();
+        flush_icache_all();
+
+        /*
+         * Tell the hibernation core that we've just restored the

memory.
+         */
+        in_suspend = 0;
+        sleep_cpu = -EINVAL;
+    }

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return ret;
+}
+
+static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +26 -0
+{
+    pte_t *src_ptep;
+    pte_t *dst_ptep;
+
+    if (pmd_none(READ_ONCE(*dst_pmdp))) {
+        dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_ptep)
+            return -ENOMEM;
+
+        pmd_populate_kernel(NULL, dst_pmdp, dst_ptep);
+    }
+
+    dst_ptep = pte_offset_kernel(dst_pmdp, start);
+    src_ptep = pte_offset_kernel(src_pmdp, start);
+
+    do {
+        pte_t pte = READ_ONCE(*src_ptep);
+
+        if (pte_present(pte))
+            set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot)));
+    } while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end);
+
+    return 0;
+}
+
+static int temp_pgtable_map_pmd(pud_t *dst_pudp, pud_t *src_pudp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pmd_t *src_pmdp;
+    pmd_t *dst_pmdp;
+
+    if (pud_none(READ_ONCE(*dst_pudp))) {
+        dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pmdp)
+            return -ENOMEM;
+
+        pud_populate(NULL, dst_pudp, dst_pmdp);
+    }
+
+    dst_pmdp = pmd_offset(dst_pudp, start);
+    src_pmdp = pmd_offset(src_pudp, start);
+
+    do {
+        pmd_t pmd = READ_ONCE(*src_pmdp);
+
+        next = pmd_addr_end(start, end);
+
+        if (pmd_none(pmd))
+            continue;
+
+        if (pmd_leaf(pmd)) {
+            set_pmd(dst_pmdp, __pmd(pmd_val(pmd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pte(dst_pmdp, src_pmdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pmdp++, src_pmdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pud_t *dst_pudp;
+    pud_t *src_pudp;
+
+    if (p4d_none(READ_ONCE(*dst_p4dp))) {
+        dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pudp)
+            return -ENOMEM;
+
+        p4d_populate(NULL, dst_p4dp, dst_pudp);
+    }
+
+    dst_pudp = pud_offset(dst_p4dp, start);
+    src_pudp = pud_offset(src_p4dp, start);
+
+    do {
+        pud_t pud = READ_ONCE(*src_pudp);
+
+        next = pud_addr_end(start, end);
+
+        if (pud_none(pud))
+            continue;
+
+        if (pud_leaf(pud)) {
+            set_pud(dst_pudp, __pud(pud_val(pud) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pmd(dst_pudp, src_pudp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pudp++, src_pudp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    p4d_t *dst_p4dp;
+    p4d_t *src_p4dp;
+
+    if (pgd_none(READ_ONCE(*dst_pgdp))) {
+        dst_p4dp = (p4d_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_p4dp)
+            return -ENOMEM;
+
+        pgd_populate(NULL, dst_pgdp, dst_p4dp);
+    }
+
+    dst_p4dp = p4d_offset(dst_pgdp, start);
+    src_p4dp = p4d_offset(src_pgdp, start);
+
+    do {
+        p4d_t p4d = READ_ONCE(*src_p4dp);
+
+        next = p4d_addr_end(start, end);
+
+        if (p4d_none(p4d))
+            continue;
+
+        if (p4d_leaf(p4d)) {
+            set_p4d(dst_p4dp, __p4d(p4d_val(p4d) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pud(dst_p4dp, src_p4dp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_p4dp++, src_p4dp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_mapping(pgd_t *pgdp, unsigned long start,

unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +18 -0
+{
+    pgd_t *dst_pgdp = pgd_offset_pgd(pgdp, start);
+    pgd_t *src_pgdp = pgd_offset_k(start);
+    unsigned long next;
+    unsigned long ret;
+
+    do {
+        pgd_t pgd = READ_ONCE(*src_pgdp);
+
+        next = pgd_addr_end(start, end);
+
+        if (pgd_none(pgd))
+            continue;
+
+        if (pgd_leaf(pgd)) {
+            set_pgd(dst_pgdp, __pgd(pgd_val(pgd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_p4d(dst_pgdp, src_pgdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pgdp++, src_pgdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +27 -0
+
+    return 0;
+}
+
+static unsigned long relocate_restore_code(void)
+{
+    void *page = (void *)get_safe_page(GFP_ATOMIC);
+
+    if (!page)
+        return -ENOMEM;
+
+    copy_page(page, hibernate_core_restore_code);
+
+    /* Make the page containing the relocated code executable. */
+    set_memory_x((unsigned long)page, 1);
+
+    return (unsigned long)page;
+}
+
+int swsusp_arch_resume(void)
+{
+    unsigned long end = (unsigned long)pfn_to_virt(max_low_pfn);
+    unsigned long start = PAGE_OFFSET;
+    int ret;
+
+    /*
+     * Memory allocated by get_safe_page() will be dealt with by the

hibernation core,
+     * we don't need to free it here.
+     */
+    resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC);
+    if (!resume_pg_dir)
+        return -ENOMEM;
+
+    /*
+     * Create a temporary page table and map the whole linear region

Quoted 8 lines
Quoted 5 lines

as executable and
+     * writable.
+     */
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));

Hmm weird protections, PAGE_PRESENT and PAGE_READ are not even set. Shouldn't you use PAGE_KERNEL_EXEC instead?

Quoted 1 line

Forget this one, that's the protections you add, so that's good :)

OK.

Quoted 22 lines
Quoted 22 lines
Quoted 22 lines

+    if (ret)
+        return ret;
+
+    /* Move the restore code to a new page so that it doesn't get overwritten by itself. */
+    relocated_restore_code = relocate_restore_code();
+    if (relocated_restore_code == -ENOMEM)
+        return -ENOMEM;
+
+    /*
+     * Map the __hibernate_cpu_resume() address to the temporary page table so that the
+     * restore code can jumps to it after finished restore the image. The next execution
+     * code doesn't find itself in a different address space after switching over to the
+     * original page table used by the hibernated image. +     *
+     */
+    start = (unsigned long)resume_hdr.restore_cpu_addr;

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

+    end = start + PAGE_SIZE;
+
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));
+    if (ret)
+        return ret;

You set the linear mapping as executable so no need for this.

Quoted 3 lines

We need this mapping because the kernel addresses and the linear addresses in riscv-64bit are in different boundary. For example:
Linear address start with 0xff60000000000000
Kernel address start with 0xffffffff80000000

Ok I understand now that you need to map this into the temporary page table: one nit though, since you copy the kernel mapping, no need to use PAGE_WRITE | PAGE_EXEC, the protections are already correct.

Thanks,

Alex

Quoted 41 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines

+
+    hibernate_restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), +                resume_hdr.restore_cpu_addr);

arch/riscv/kernel/hibernate.c new file +19 -0
+
+    return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP_SMP
+int hibernate_resume_nonboot_cpu_disable(void)
+{
+    if (sleep_cpu < 0) {
+        pr_err("Failing to resume from hibernate on an unknown CPU\n");
+        return -ENODEV;
+    }
+
+    return freeze_secondary_cpus(sleep_cpu);
+}
+#endif
+
+static int __init riscv_hibernate_init(void)
+{
+    hibernate_cpu_context = kzalloc(sizeof(*hibernate_cpu_context),

GFP_KERNEL);

arch/riscv/kernel/hibernate.c new file +4 -0
+
+    if (WARN_ON(!hibernate_cpu_context))
+        return -ENOMEM;
+
Quoted 8 lines
Quoted 8 lines

arch/riscv/kernel/hibernate.c line 424

Quoted 4 lines

+    return 0;

arch/riscv/kernel/hibernate.c new file +3 -0
+}
+
+early_initcall(riscv_hibernate_init);

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#9 lore raw

RE: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Hi Alex,

Quoted 698 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Friday, March 10, 2023 5:59 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Hi JeeHeng,

Quoted 687 lines · JeeHeng Sia
Quoted 655 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Wednesday, March 8, 2023 5:28 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 635 lines · Alexandre Ghiti
Quoted 634 lines · Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid,
and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
   arch/riscv/Kconfig                 |   6 +
   arch/riscv/include/asm/assembler.h |  20 ++
   arch/riscv/include/asm/suspend.h   |  19 ++
   arch/riscv/kernel/Makefile         |   1 +
   arch/riscv/kernel/asm-offsets.c    |   5 +
   arch/riscv/kernel/hibernate-asm.S  |  77 ++++++
   arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++
   7 files changed, 552 insertions(+)

  create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c

arch/riscv/Kconfig +5 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"
     source "kernel/power/Kconfig"
   +config ARCH_HIBERNATION_POSSIBLE
+    def_bool y
+
+config ARCH_HIBERNATION_HEADER
+    def_bool HIBERNATION
+
   endmenu # "Power management options"
     menu "CPU Power Management"
arch/riscv/include/asm/assembler.h +19 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
           REG_L    s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
       .endm
   +/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+    .macro    copy_page a0, a1
+        lui    a2, 0x1
+        add    a2, a2, a0
+1 :
+        REG_L    t0, 0(a1)
+        REG_L    t1, SZREG(a1)
+
+        REG_S    t0, 0(a0)
+        REG_S    t1, SZREG(a0)
+
+        addi    a0, a0, 2 * SZREG
+        addi    a1, a1, 2 * SZREG
+        bne    a2, a0, 1b
+    .endm
+
   #endif    /* __ASM_ASSEMBLER_H */
arch/riscv/include/asm/suspend.h +4 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
   #endif
   };
   +/*
+ * Used by hibernation core and cleared during resume sequence
+ */
+extern int in_suspend;
+
   /* Low-level CPU suspend entry function */
   int __cpu_suspend_enter(struct suspend_context *context);
   @@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid,

unsigned long context);
  /* Used to save and restore the CSRs */
  void suspend_save_csrs(struct suspend_context *context); void suspend_restore_csrs(struct suspend_context *context);

arch/riscv/include/asm/suspend.h +12 -0
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp,

unsigned long satp_temp,
+                    unsigned long cpu_resume);

arch/riscv/include/asm/suspend.h +1 -0
+asmlinkage int hibernate_core_restore_code(void);
   #endif
arch/riscv/kernel/Makefile +1 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)        += module.o
   obj-$(CONFIG_MODULE_SECTIONS)    += module-sections.o
     obj-$(CONFIG_CPU_PM)        += suspend_entry.o suspend.o
+obj-$(CONFIG_HIBERNATION)    += hibernate.o hibernate-asm.o
     obj-$(CONFIG_FUNCTION_TRACER)    += mcount.o ftrace.o
   obj-$(CONFIG_DYNAMIC_FTRACE)    += mcount-dyn.o
arch/riscv/kernel/asm-offsets.c +4 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
   #include <linux/kbuild.h>
   #include <linux/mm.h>
   #include <linux/sched.h>
+#include <linux/suspend.h>
   #include <asm/kvm_host.h>
   #include <asm/thread_info.h>
   #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
         OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
   +    OFFSET(HIBERN_PBE_ADDR, pbe, address);
+    OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+    OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
       OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
       OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
       OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +19 -0
new file mode 100644
index 000000000000..effaf5ca5da0
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hibernation low level support for RISCV.
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/csr.h>
+
+#include <linux/linkage.h>
+
+/*
+ * int __hibernate_cpu_resume(void)
+ * Switch back to the hibernated image's page table prior to

restoring the CPU
+ * context.
+ *
+ * Always returns 0
+ */

arch/riscv/kernel/hibernate-asm.S new file +35 -0
+ENTRY(__hibernate_cpu_resume)
+    /* switch to hibernated image's page table. */
+    csrw CSR_SATP, s0
+    sfence.vma
+
+    REG_L    a0, hibernate_cpu_context
+
+    suspend_restore_csrs
+    suspend_restore_regs
+
+    /* Return zero value. */
+    mv    a0, zero
+
+    ret
+END(__hibernate_cpu_resume)
+
+/*
+ * Prepare to restore the image.
+ * a0: satp of saved page tables.
+ * a1: satp of temporary page tables.
+ * a2: cpu_resume.
+ */
+ENTRY(hibernate_restore_image)
+    mv    s0, a0
+    mv    s1, a1
+    mv    s2, a2
+    REG_L    s4, restore_pblist
+    REG_L    a1, relocated_restore_code
+
+    jalr    a1
+END(hibernate_restore_image)
+
+/*
+ * The below code will be executed from a 'safe' page.
+ * It first switches to the temporary page table, then starts to

copy the pages
+ * back to the original memory location. Finally, it jumps to __hibernate_cpu_resume()
+ * to restore the CPU context.
+ */

arch/riscv/kernel/hibernate-asm.S new file +16 -0
+ENTRY(hibernate_core_restore_code)
+    /* switch to temp page table. */
+    csrw satp, s1
+    sfence.vma
+.Lcopy:
+    /* The below code will restore the hibernated image. */
+    REG_L    a1, HIBERN_PBE_ADDR(s4)
+    REG_L    a0, HIBERN_PBE_ORIG(s4)
+
+    copy_page a0, a1
+
+    REG_L    s4, HIBERN_PBE_NEXT(s4)
+    bnez    s4, .Lcopy
+
+    jalr    s2
+END(hibernate_core_restore_code)
arch/riscv/kernel/hibernate.c new file +28 -0
new file mode 100644
index 000000000000..8fab50122e54
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hibernation support for RISCV
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/barrier.h>
+#include <asm/cacheflush.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/sections.h>
+#include <asm/set_memory.h>
+#include <asm/smp.h>
+#include <asm/suspend.h>
+
+#include <linux/cpu.h>
+#include <linux/memblock.h>
+#include <linux/pm.h>
+#include <linux/sched.h>
+#include <linux/suspend.h>
+#include <linux/utsname.h>
+
+/* The logical cpu number we should resume on, initialised to a

non-cpu number. */

arch/riscv/kernel/hibernate.c new file +14 -0
+static int sleep_cpu = -EINVAL;
+
+/* Pointer to the temporary resume page table. */
+static pgd_t *resume_pg_dir;
+
+/* CPU context to be saved. */
+struct suspend_context *hibernate_cpu_context;
+EXPORT_SYMBOL_GPL(hibernate_cpu_context);
+
+unsigned long relocated_restore_code;
+EXPORT_SYMBOL_GPL(relocated_restore_code);
+
+/**
+ * struct arch_hibernate_hdr_invariants - container to store kernel

build version.
+ * @uts_version: to save the build number and date so that the we do not resume with
+ *        a different kernel.
+ */

arch/riscv/kernel/hibernate.c new file +6 -0
+struct arch_hibernate_hdr_invariants {
+    char        uts_version[__NEW_UTS_LEN + 1];
+};
+
+/**
+ * struct arch_hibernate_hdr - helper parameters that help us to

restore the image.
+ * @invariants: container to store kernel build version.
+ * @hartid: to make sure same boot_cpu executes the hibernate/restore code.
+ * @saved_satp: original page table used by the hibernated image.
+ * @restore_cpu_addr: the kernel's image address to restore the CPU context.
+ */

arch/riscv/kernel/hibernate.c new file +8 -0
+static struct arch_hibernate_hdr {
+    struct arch_hibernate_hdr_invariants invariants;
+    unsigned long    hartid;
+    unsigned long    saved_satp;
+    unsigned long    restore_cpu_addr;
+} resume_hdr;
+
+static void arch_hdr_invariants(struct arch_hibernate_hdr_invariants

*i)
+{
+    memset(i, 0, sizeof(*i));
+    memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version));

arch/riscv/kernel/hibernate.c new file +93 -0
+}
+
+/*
+ * Check if the given pfn is in the 'nosave' section.
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+    unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin);
+    unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
+
+    return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn));
+}
+
+void notrace save_processor_state(void)
+{
+    WARN_ON(num_online_cpus() != 1);
+}
+
+void notrace restore_processor_state(void)
+{
+}
+
+/*
+ * Helper parameters need to be saved to the hibernation image header.
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+    struct arch_hibernate_hdr *hdr = addr;
+
+    if (max_size < sizeof(*hdr))
+        return -EOVERFLOW;
+
+    arch_hdr_invariants(&hdr->invariants);
+
+    hdr->hartid = cpuid_to_hartid_map(sleep_cpu);
+    hdr->saved_satp = csr_read(CSR_SATP);
+    hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume;
+
+    return 0;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_save);
+
+/*
+ * Retrieve the helper parameters from the hibernation image header.
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+    struct arch_hibernate_hdr_invariants invariants;
+    struct arch_hibernate_hdr *hdr = addr;
+    int ret = 0;
+
+    arch_hdr_invariants(&invariants);
+
+    if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) {
+        pr_crit("Hibernate image not generated by this kernel!\n");
+        return -EINVAL;
+    }
+
+    sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid);
+    if (sleep_cpu < 0) {
+        pr_crit("Hibernated on a CPU not known to this kernel!\n");
+        sleep_cpu = -EINVAL;
+        return -EINVAL;
+    }
+
+#ifdef CONFIG_SMP
+    ret = bringup_hibernate_cpu(sleep_cpu);
+    if (ret) {
+        sleep_cpu = -EINVAL;
+        return ret;
+    }
+#endif
+    resume_hdr = *hdr;
+
+    return ret;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_restore);
+
+int swsusp_arch_suspend(void)
+{
+    int ret = 0;
+
+    if (__cpu_suspend_enter(hibernate_cpu_context)) {
+        sleep_cpu = smp_processor_id();
+        suspend_save_csrs(hibernate_cpu_context);
+        ret = swsusp_save();
+    } else {
+        suspend_restore_csrs(hibernate_cpu_context);
+        flush_tlb_all();
+        flush_icache_all();
+
+        /*
+         * Tell the hibernation core that we've just restored the

memory.
+         */
+        in_suspend = 0;
+        sleep_cpu = -EINVAL;
+    }

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return ret;
+}
+
+static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +26 -0
+{
+    pte_t *src_ptep;
+    pte_t *dst_ptep;
+
+    if (pmd_none(READ_ONCE(*dst_pmdp))) {
+        dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_ptep)
+            return -ENOMEM;
+
+        pmd_populate_kernel(NULL, dst_pmdp, dst_ptep);
+    }
+
+    dst_ptep = pte_offset_kernel(dst_pmdp, start);
+    src_ptep = pte_offset_kernel(src_pmdp, start);
+
+    do {
+        pte_t pte = READ_ONCE(*src_ptep);
+
+        if (pte_present(pte))
+            set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot)));
+    } while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end);
+
+    return 0;
+}
+
+static int temp_pgtable_map_pmd(pud_t *dst_pudp, pud_t *src_pudp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pmd_t *src_pmdp;
+    pmd_t *dst_pmdp;
+
+    if (pud_none(READ_ONCE(*dst_pudp))) {
+        dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pmdp)
+            return -ENOMEM;
+
+        pud_populate(NULL, dst_pudp, dst_pmdp);
+    }
+
+    dst_pmdp = pmd_offset(dst_pudp, start);
+    src_pmdp = pmd_offset(src_pudp, start);
+
+    do {
+        pmd_t pmd = READ_ONCE(*src_pmdp);
+
+        next = pmd_addr_end(start, end);
+
+        if (pmd_none(pmd))
+            continue;
+
+        if (pmd_leaf(pmd)) {
+            set_pmd(dst_pmdp, __pmd(pmd_val(pmd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pte(dst_pmdp, src_pmdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pmdp++, src_pmdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pud_t *dst_pudp;
+    pud_t *src_pudp;
+
+    if (p4d_none(READ_ONCE(*dst_p4dp))) {
+        dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pudp)
+            return -ENOMEM;
+
+        p4d_populate(NULL, dst_p4dp, dst_pudp);
+    }
+
+    dst_pudp = pud_offset(dst_p4dp, start);
+    src_pudp = pud_offset(src_p4dp, start);
+
+    do {
+        pud_t pud = READ_ONCE(*src_pudp);
+
+        next = pud_addr_end(start, end);
+
+        if (pud_none(pud))
+            continue;
+
+        if (pud_leaf(pud)) {
+            set_pud(dst_pudp, __pud(pud_val(pud) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pmd(dst_pudp, src_pudp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pudp++, src_pudp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    p4d_t *dst_p4dp;
+    p4d_t *src_p4dp;
+
+    if (pgd_none(READ_ONCE(*dst_pgdp))) {
+        dst_p4dp = (p4d_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_p4dp)
+            return -ENOMEM;
+
+        pgd_populate(NULL, dst_pgdp, dst_p4dp);
+    }
+
+    dst_p4dp = p4d_offset(dst_pgdp, start);
+    src_p4dp = p4d_offset(src_pgdp, start);
+
+    do {
+        p4d_t p4d = READ_ONCE(*src_p4dp);
+
+        next = p4d_addr_end(start, end);
+
+        if (p4d_none(p4d))
+            continue;
+
+        if (p4d_leaf(p4d)) {
+            set_p4d(dst_p4dp, __p4d(p4d_val(p4d) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pud(dst_p4dp, src_p4dp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_p4dp++, src_p4dp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_mapping(pgd_t *pgdp, unsigned long start,

unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +18 -0
+{
+    pgd_t *dst_pgdp = pgd_offset_pgd(pgdp, start);
+    pgd_t *src_pgdp = pgd_offset_k(start);
+    unsigned long next;
+    unsigned long ret;
+
+    do {
+        pgd_t pgd = READ_ONCE(*src_pgdp);
+
+        next = pgd_addr_end(start, end);
+
+        if (pgd_none(pgd))
+            continue;
+
+        if (pgd_leaf(pgd)) {
+            set_pgd(dst_pgdp, __pgd(pgd_val(pgd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_p4d(dst_pgdp, src_pgdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pgdp++, src_pgdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +27 -0
+
+    return 0;
+}
+
+static unsigned long relocate_restore_code(void)
+{
+    void *page = (void *)get_safe_page(GFP_ATOMIC);
+
+    if (!page)
+        return -ENOMEM;
+
+    copy_page(page, hibernate_core_restore_code);
+
+    /* Make the page containing the relocated code executable. */
+    set_memory_x((unsigned long)page, 1);
+
+    return (unsigned long)page;
+}
+
+int swsusp_arch_resume(void)
+{
+    unsigned long end = (unsigned long)pfn_to_virt(max_low_pfn);
+    unsigned long start = PAGE_OFFSET;
+    int ret;
+
+    /*
+     * Memory allocated by get_safe_page() will be dealt with by the

hibernation core,
+     * we don't need to free it here.
+     */
+    resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC);
+    if (!resume_pg_dir)
+        return -ENOMEM;
+
+    /*
+     * Create a temporary page table and map the whole linear region

Quoted 8 lines
Quoted 5 lines

as executable and
+     * writable.
+     */
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));

Hmm weird protections, PAGE_PRESENT and PAGE_READ are not even set. Shouldn't you use PAGE_KERNEL_EXEC instead?

Quoted 1 line

Forget this one, that's the protections you add, so that's good :)

OK.

Quoted 30 lines
Quoted 30 lines
Quoted 20 lines

+    if (ret)
+        return ret;
+
+    /* Move the restore code to a new page so that it doesn't get overwritten by itself. */
+    relocated_restore_code = relocate_restore_code();
+    if (relocated_restore_code == -ENOMEM)
+        return -ENOMEM;
+
+    /*
+     * Map the __hibernate_cpu_resume() address to the temporary page table so that the
+     * restore code can jumps to it after finished restore the image. The next execution
+     * code doesn't find itself in a different address space after switching over to the
+     * original page table used by the hibernated image. +     *

Quoted 8 lines

+     */
+    start = (unsigned long)resume_hdr.restore_cpu_addr; +    end = start + PAGE_SIZE;
+
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));
+    if (ret)
+        return ret;

You set the linear mapping as executable so no need for this.

Quoted 8 lines
Quoted 3 lines

We need this mapping because the kernel addresses and the linear addresses in riscv-64bit are in different boundary. For example:
Linear address start with 0xff60000000000000
Kernel address start with 0xffffffff80000000

Ok I understand now that you need to map this into the temporary page
table: one nit though, since you copy the kernel mapping, no need to use PAGE_WRITE | PAGE_EXEC, the protections are already correct.

Here is the thing. For riscv_64bit, I am agreed with you that we don’t need to use PAGE_WRITE | PAGE_EXEC.
However, we need to use PAGE_WRITE | PAGE_EXEC for riscv_32bit because the linear addresses and kernel addresses are similar. The attribute becomes not writable if following the "original" page table. I am thinking to use #ifdef config_32bit to split the path for 64bit and 32bit but I just afraid the linear addresses and kernel addresses might change again in the future. To be on the safe side, I let both riscv_32bit & riscv_64bit to use the PAGE_WRITE | PAGE_EXEC. For example:
Linear address and kernel address for 32bit started with 0xc0000000
Linear address for 64bit start with 0xff60000000000000
Kernel address for 64bit start with 0xffffffff80000000

Please let me know your thoughts.

Quoted 47 lines

Thanks,

Alex

Quoted 41 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines

+
+    hibernate_restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), +                resume_hdr.restore_cpu_addr);

arch/riscv/kernel/hibernate.c new file +19 -0
+
+    return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP_SMP
+int hibernate_resume_nonboot_cpu_disable(void)
+{
+    if (sleep_cpu < 0) {
+        pr_err("Failing to resume from hibernate on an unknown CPU\n");
+        return -ENODEV;
+    }
+
+    return freeze_secondary_cpus(sleep_cpu);
+}
+#endif
+
+static int __init riscv_hibernate_init(void)
+{
+    hibernate_cpu_context = kzalloc(sizeof(*hibernate_cpu_context),

GFP_KERNEL);

arch/riscv/kernel/hibernate.c new file +4 -0
+
+    if (WARN_ON(!hibernate_cpu_context))
+        return -ENOMEM;
+
Quoted 8 lines
Quoted 8 lines

arch/riscv/kernel/hibernate.c line 424

Quoted 4 lines

+    return 0;

arch/riscv/kernel/hibernate.c new file +3 -0
+}
+
+early_initcall(riscv_hibernate_init);

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#10 lore raw

Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 698 lines · JeeHeng Sia

Hi Alex,

Quoted 696 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Friday, March 10, 2023 5:59 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Hi JeeHeng,

Quoted 684 lines · JeeHeng Sia
Quoted 652 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Wednesday, March 8, 2023 5:28 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 634 lines · Alexandre Ghiti
Quoted 633 lines · Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid,
and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
    arch/riscv/Kconfig                 |   6 +
    arch/riscv/include/asm/assembler.h |  20 ++
    arch/riscv/include/asm/suspend.h   |  19 ++
    arch/riscv/kernel/Makefile         |   1 +
    arch/riscv/kernel/asm-offsets.c    |   5 +
    arch/riscv/kernel/hibernate-asm.S  |  77 ++++++
    arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++
    7 files changed, 552 insertions(+)

  create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c

arch/riscv/Kconfig +5 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"
      source "kernel/power/Kconfig"
    +config ARCH_HIBERNATION_POSSIBLE
+    def_bool y
+
+config ARCH_HIBERNATION_HEADER
+    def_bool HIBERNATION
+
    endmenu # "Power management options"
      menu "CPU Power Management"
arch/riscv/include/asm/assembler.h +19 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
            REG_L    s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
        .endm
    +/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+    .macro    copy_page a0, a1
+        lui    a2, 0x1
+        add    a2, a2, a0
+1 :
+        REG_L    t0, 0(a1)
+        REG_L    t1, SZREG(a1)
+
+        REG_S    t0, 0(a0)
+        REG_S    t1, SZREG(a0)
+
+        addi    a0, a0, 2 * SZREG
+        addi    a1, a1, 2 * SZREG
+        bne    a2, a0, 1b
+    .endm
+
    #endif    /* __ASM_ASSEMBLER_H */
arch/riscv/include/asm/suspend.h +4 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
    #endif
    };
    +/*
+ * Used by hibernation core and cleared during resume sequence
+ */
+extern int in_suspend;
+
    /* Low-level CPU suspend entry function */
    int __cpu_suspend_enter(struct suspend_context *context);
    @@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid,

unsigned long context);
  /* Used to save and restore the CSRs */
  void suspend_save_csrs(struct suspend_context *context); void suspend_restore_csrs(struct suspend_context *context);

arch/riscv/include/asm/suspend.h +12 -0
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp,

unsigned long satp_temp,
+                    unsigned long cpu_resume);

arch/riscv/include/asm/suspend.h +1 -0
+asmlinkage int hibernate_core_restore_code(void);
    #endif
arch/riscv/kernel/Makefile +1 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)        += module.o
    obj-$(CONFIG_MODULE_SECTIONS)    += module-sections.o
      obj-$(CONFIG_CPU_PM)        += suspend_entry.o suspend.o
+obj-$(CONFIG_HIBERNATION)    += hibernate.o hibernate-asm.o
      obj-$(CONFIG_FUNCTION_TRACER)    += mcount.o ftrace.o
    obj-$(CONFIG_DYNAMIC_FTRACE)    += mcount-dyn.o
arch/riscv/kernel/asm-offsets.c +4 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
    #include <linux/kbuild.h>
    #include <linux/mm.h>
    #include <linux/sched.h>
+#include <linux/suspend.h>
    #include <asm/kvm_host.h>
    #include <asm/thread_info.h>
    #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
          OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
    +    OFFSET(HIBERN_PBE_ADDR, pbe, address);
+    OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+    OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
        OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
        OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
        OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +19 -0
new file mode 100644
index 000000000000..effaf5ca5da0
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hibernation low level support for RISCV.
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/csr.h>
+
+#include <linux/linkage.h>
+
+/*
+ * int __hibernate_cpu_resume(void)
+ * Switch back to the hibernated image's page table prior to

restoring the CPU
+ * context.
+ *
+ * Always returns 0
+ */

arch/riscv/kernel/hibernate-asm.S new file +35 -0
+ENTRY(__hibernate_cpu_resume)
+    /* switch to hibernated image's page table. */
+    csrw CSR_SATP, s0
+    sfence.vma
+
+    REG_L    a0, hibernate_cpu_context
+
+    suspend_restore_csrs
+    suspend_restore_regs
+
+    /* Return zero value. */
+    mv    a0, zero
+
+    ret
+END(__hibernate_cpu_resume)
+
+/*
+ * Prepare to restore the image.
+ * a0: satp of saved page tables.
+ * a1: satp of temporary page tables.
+ * a2: cpu_resume.
+ */
+ENTRY(hibernate_restore_image)
+    mv    s0, a0
+    mv    s1, a1
+    mv    s2, a2
+    REG_L    s4, restore_pblist
+    REG_L    a1, relocated_restore_code
+
+    jalr    a1
+END(hibernate_restore_image)
+
+/*
+ * The below code will be executed from a 'safe' page.
+ * It first switches to the temporary page table, then starts to

copy the pages
+ * back to the original memory location. Finally, it jumps to __hibernate_cpu_resume()
+ * to restore the CPU context.
+ */

arch/riscv/kernel/hibernate-asm.S new file +16 -0
+ENTRY(hibernate_core_restore_code)
+    /* switch to temp page table. */
+    csrw satp, s1
+    sfence.vma
+.Lcopy:
+    /* The below code will restore the hibernated image. */
+    REG_L    a1, HIBERN_PBE_ADDR(s4)
+    REG_L    a0, HIBERN_PBE_ORIG(s4)
+
+    copy_page a0, a1
+
+    REG_L    s4, HIBERN_PBE_NEXT(s4)
+    bnez    s4, .Lcopy
+
+    jalr    s2
+END(hibernate_core_restore_code)
arch/riscv/kernel/hibernate.c new file +28 -0
new file mode 100644
index 000000000000..8fab50122e54
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hibernation support for RISCV
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/barrier.h>
+#include <asm/cacheflush.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/sections.h>
+#include <asm/set_memory.h>
+#include <asm/smp.h>
+#include <asm/suspend.h>
+
+#include <linux/cpu.h>
+#include <linux/memblock.h>
+#include <linux/pm.h>
+#include <linux/sched.h>
+#include <linux/suspend.h>
+#include <linux/utsname.h>
+
+/* The logical cpu number we should resume on, initialised to a

non-cpu number. */

arch/riscv/kernel/hibernate.c new file +14 -0
+static int sleep_cpu = -EINVAL;
+
+/* Pointer to the temporary resume page table. */
+static pgd_t *resume_pg_dir;
+
+/* CPU context to be saved. */
+struct suspend_context *hibernate_cpu_context;
+EXPORT_SYMBOL_GPL(hibernate_cpu_context);
+
+unsigned long relocated_restore_code;
+EXPORT_SYMBOL_GPL(relocated_restore_code);
+
+/**
+ * struct arch_hibernate_hdr_invariants - container to store kernel

build version.
+ * @uts_version: to save the build number and date so that the we do not resume with
+ *        a different kernel.
+ */

arch/riscv/kernel/hibernate.c new file +6 -0
+struct arch_hibernate_hdr_invariants {
+    char        uts_version[__NEW_UTS_LEN + 1];
+};
+
+/**
+ * struct arch_hibernate_hdr - helper parameters that help us to

restore the image.
+ * @invariants: container to store kernel build version.
+ * @hartid: to make sure same boot_cpu executes the hibernate/restore code.
+ * @saved_satp: original page table used by the hibernated image.
+ * @restore_cpu_addr: the kernel's image address to restore the CPU context.
+ */

arch/riscv/kernel/hibernate.c new file +8 -0
+static struct arch_hibernate_hdr {
+    struct arch_hibernate_hdr_invariants invariants;
+    unsigned long    hartid;
+    unsigned long    saved_satp;
+    unsigned long    restore_cpu_addr;
+} resume_hdr;
+
+static void arch_hdr_invariants(struct arch_hibernate_hdr_invariants

*i)
+{
+    memset(i, 0, sizeof(*i));
+    memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version));

arch/riscv/kernel/hibernate.c new file +93 -0
+}
+
+/*
+ * Check if the given pfn is in the 'nosave' section.
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+    unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin);
+    unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
+
+    return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn));
+}
+
+void notrace save_processor_state(void)
+{
+    WARN_ON(num_online_cpus() != 1);
+}
+
+void notrace restore_processor_state(void)
+{
+}
+
+/*
+ * Helper parameters need to be saved to the hibernation image header.
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+    struct arch_hibernate_hdr *hdr = addr;
+
+    if (max_size < sizeof(*hdr))
+        return -EOVERFLOW;
+
+    arch_hdr_invariants(&hdr->invariants);
+
+    hdr->hartid = cpuid_to_hartid_map(sleep_cpu);
+    hdr->saved_satp = csr_read(CSR_SATP);
+    hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume;
+
+    return 0;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_save);
+
+/*
+ * Retrieve the helper parameters from the hibernation image header.
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+    struct arch_hibernate_hdr_invariants invariants;
+    struct arch_hibernate_hdr *hdr = addr;
+    int ret = 0;
+
+    arch_hdr_invariants(&invariants);
+
+    if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) {
+        pr_crit("Hibernate image not generated by this kernel!\n");
+        return -EINVAL;
+    }
+
+    sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid);
+    if (sleep_cpu < 0) {
+        pr_crit("Hibernated on a CPU not known to this kernel!\n");
+        sleep_cpu = -EINVAL;
+        return -EINVAL;
+    }
+
+#ifdef CONFIG_SMP
+    ret = bringup_hibernate_cpu(sleep_cpu);
+    if (ret) {
+        sleep_cpu = -EINVAL;
+        return ret;
+    }
+#endif
+    resume_hdr = *hdr;
+
+    return ret;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_restore);
+
+int swsusp_arch_suspend(void)
+{
+    int ret = 0;
+
+    if (__cpu_suspend_enter(hibernate_cpu_context)) {
+        sleep_cpu = smp_processor_id();
+        suspend_save_csrs(hibernate_cpu_context);
+        ret = swsusp_save();
+    } else {
+        suspend_restore_csrs(hibernate_cpu_context);
+        flush_tlb_all();
+        flush_icache_all();
+
+        /*
+         * Tell the hibernation core that we've just restored the

memory.
+         */
+        in_suspend = 0;
+        sleep_cpu = -EINVAL;
+    }

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return ret;
+}
+
+static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +26 -0
+{
+    pte_t *src_ptep;
+    pte_t *dst_ptep;
+
+    if (pmd_none(READ_ONCE(*dst_pmdp))) {
+        dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_ptep)
+            return -ENOMEM;
+
+        pmd_populate_kernel(NULL, dst_pmdp, dst_ptep);
+    }
+
+    dst_ptep = pte_offset_kernel(dst_pmdp, start);
+    src_ptep = pte_offset_kernel(src_pmdp, start);
+
+    do {
+        pte_t pte = READ_ONCE(*src_ptep);
+
+        if (pte_present(pte))
+            set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot)));
+    } while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end);
+
+    return 0;
+}
+
+static int temp_pgtable_map_pmd(pud_t *dst_pudp, pud_t *src_pudp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pmd_t *src_pmdp;
+    pmd_t *dst_pmdp;
+
+    if (pud_none(READ_ONCE(*dst_pudp))) {
+        dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pmdp)
+            return -ENOMEM;
+
+        pud_populate(NULL, dst_pudp, dst_pmdp);
+    }
+
+    dst_pmdp = pmd_offset(dst_pudp, start);
+    src_pmdp = pmd_offset(src_pudp, start);
+
+    do {
+        pmd_t pmd = READ_ONCE(*src_pmdp);
+
+        next = pmd_addr_end(start, end);
+
+        if (pmd_none(pmd))
+            continue;
+
+        if (pmd_leaf(pmd)) {
+            set_pmd(dst_pmdp, __pmd(pmd_val(pmd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pte(dst_pmdp, src_pmdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pmdp++, src_pmdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pud_t *dst_pudp;
+    pud_t *src_pudp;
+
+    if (p4d_none(READ_ONCE(*dst_p4dp))) {
+        dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pudp)
+            return -ENOMEM;
+
+        p4d_populate(NULL, dst_p4dp, dst_pudp);
+    }
+
+    dst_pudp = pud_offset(dst_p4dp, start);
+    src_pudp = pud_offset(src_p4dp, start);
+
+    do {
+        pud_t pud = READ_ONCE(*src_pudp);
+
+        next = pud_addr_end(start, end);
+
+        if (pud_none(pud))
+            continue;
+
+        if (pud_leaf(pud)) {
+            set_pud(dst_pudp, __pud(pud_val(pud) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pmd(dst_pudp, src_pudp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pudp++, src_pudp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    p4d_t *dst_p4dp;
+    p4d_t *src_p4dp;
+
+    if (pgd_none(READ_ONCE(*dst_pgdp))) {
+        dst_p4dp = (p4d_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_p4dp)
+            return -ENOMEM;
+
+        pgd_populate(NULL, dst_pgdp, dst_p4dp);
+    }
+
+    dst_p4dp = p4d_offset(dst_pgdp, start);
+    src_p4dp = p4d_offset(src_pgdp, start);
+
+    do {
+        p4d_t p4d = READ_ONCE(*src_p4dp);
+
+        next = p4d_addr_end(start, end);
+
+        if (p4d_none(p4d))
+            continue;
+
+        if (p4d_leaf(p4d)) {
+            set_p4d(dst_p4dp, __p4d(p4d_val(p4d) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pud(dst_p4dp, src_p4dp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_p4dp++, src_p4dp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_mapping(pgd_t *pgdp, unsigned long start,

unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +18 -0
+{
+    pgd_t *dst_pgdp = pgd_offset_pgd(pgdp, start);
+    pgd_t *src_pgdp = pgd_offset_k(start);
+    unsigned long next;
+    unsigned long ret;
+
+    do {
+        pgd_t pgd = READ_ONCE(*src_pgdp);
+
+        next = pgd_addr_end(start, end);
+
+        if (pgd_none(pgd))
+            continue;
+
+        if (pgd_leaf(pgd)) {
+            set_pgd(dst_pgdp, __pgd(pgd_val(pgd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_p4d(dst_pgdp, src_pgdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pgdp++, src_pgdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +27 -0
+
+    return 0;
+}
+
+static unsigned long relocate_restore_code(void)
+{
+    void *page = (void *)get_safe_page(GFP_ATOMIC);
+
+    if (!page)
+        return -ENOMEM;
+
+    copy_page(page, hibernate_core_restore_code);
+
+    /* Make the page containing the relocated code executable. */
+    set_memory_x((unsigned long)page, 1);
+
+    return (unsigned long)page;
+}
+
+int swsusp_arch_resume(void)
+{
+    unsigned long end = (unsigned long)pfn_to_virt(max_low_pfn);
+    unsigned long start = PAGE_OFFSET;
+    int ret;
+
+    /*
+     * Memory allocated by get_safe_page() will be dealt with by the

hibernation core,
+     * we don't need to free it here.
+     */
+    resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC); +    if (!resume_pg_dir)
+        return -ENOMEM;
+
+    /*

Quoted 8 lines
Quoted 6 lines

+     * Create a temporary page table and map the whole linear region as executable and
+     * writable.
+     */
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));

Hmm weird protections, PAGE_PRESENT and PAGE_READ are not even set. Shouldn't you use PAGE_KERNEL_EXEC instead?

Quoted 1 line

Forget this one, that's the protections you add, so that's good :)

OK.

Quoted 19 lines
Quoted 19 lines
Quoted 19 lines

+    if (ret)
+        return ret;
+
+    /* Move the restore code to a new page so that it doesn't get overwritten by itself. */
+    relocated_restore_code = relocate_restore_code();
+    if (relocated_restore_code == -ENOMEM)
+        return -ENOMEM;
+
+    /*
+     * Map the __hibernate_cpu_resume() address to the temporary page table so that the
+     * restore code can jumps to it after finished restore the image. The next execution
+     * code doesn't find itself in a different address space after switching over to the
+     * original page table used by the hibernated image. +     *
+     */

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

+    start = (unsigned long)resume_hdr.restore_cpu_addr; +    end = start + PAGE_SIZE;
+
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));
+    if (ret)
+        return ret;

You set the linear mapping as executable so no need for this.

We need this mapping because the kernel addresses and the linear addresses in riscv-64bit are in different boundary. For example:
Linear address start with 0xff60000000000000
Kernel address start with 0xffffffff80000000

Quoted 5 lines
Quoted 3 lines

Ok I understand now that you need to map this into the temporary page
table: one nit though, since you copy the kernel mapping, no need to use PAGE_WRITE | PAGE_EXEC, the protections are already correct.

Here is the thing. For riscv_64bit, I am agreed with you that we don’t need to use PAGE_WRITE | PAGE_EXEC.
However, we need to use PAGE_WRITE | PAGE_EXEC for riscv_32bit because the linear addresses and kernel addresses are similar. The attribute becomes not writable if following the "original" page table. I am thinking to use #ifdef config_32bit to split the path for 64bit and 32bit but I just afraid the linear addresses and kernel addresses might change again in the future. To be on the safe side, I let both riscv_32bit & riscv_64bit to use the PAGE_WRITE | PAGE_EXEC.

Indeed, you really thought about everything, well done! I would add a comment that explains just that above, no need for other ifdefs IMO.

Quoted 52 lines

For example:
Linear address and kernel address for 32bit started with 0xc0000000 Linear address for 64bit start with 0xff60000000000000 Kernel address for 64bit start with 0xffffffff80000000

Please let me know your thoughts.

Quoted 45 lines

Thanks,

Alex

Quoted 40 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines

+
+    hibernate_restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), +                resume_hdr.restore_cpu_addr);

arch/riscv/kernel/hibernate.c new file +19 -0
+
+    return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP_SMP
+int hibernate_resume_nonboot_cpu_disable(void)
+{
+    if (sleep_cpu < 0) {
+        pr_err("Failing to resume from hibernate on an unknown CPU\n");
+        return -ENODEV;
+    }
+
+    return freeze_secondary_cpus(sleep_cpu);
+}
+#endif
+
+static int __init riscv_hibernate_init(void)
+{
+    hibernate_cpu_context = kzalloc(sizeof(*hibernate_cpu_context),

GFP_KERNEL);

arch/riscv/kernel/hibernate.c new file +4 -0
+
+    if (WARN_ON(!hibernate_cpu_context))
+        return -ENOMEM;
+
Quoted 8 lines
Quoted 8 lines

arch/riscv/kernel/hibernate.c line 424

Quoted 4 lines

+    return 0;

arch/riscv/kernel/hibernate.c new file +3 -0
+}
+
+early_initcall(riscv_hibernate_init);

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

#11 lore raw

RE: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 716 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Friday, March 10, 2023 6:50 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 695 lines · JeeHeng Sia

Hi Alex,

Quoted 693 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Friday, March 10, 2023 5:59 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Hi JeeHeng,

Quoted 682 lines · JeeHeng Sia
Quoted 652 lines

-----Original Message-----
From: Alexandre Ghiti <[email protected]>
Sent: Wednesday, March 8, 2023 5:28 PM
To: JeeHeng Sia <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; Leyfoon Tan <[email protected]>; Mason Huo <[email protected]>
Subject: Re: [PATCH v5 4/4] RISC-V: Add arch functions to support hibernation/suspend-to-disk

Quoted 634 lines · Alexandre Ghiti
Quoted 633 lines · Sia Jee Heng

Low level Arch functions were created to support hibernation. swsusp_arch_suspend() relies code from __cpu_suspend_enter() to write cpu state onto the stack, then calling swsusp_save() to save the memory image.

Arch specific hibernation header is implemented and is utilized by the arch_hibernation_header_restore() and arch_hibernation_header_save() functions. The arch specific hibernation header consists of satp, hartid,
and the cpu_resume address. The kernel built version is also need to be saved into the hibernation image header to making sure only the same kernel is restore when resume.

swsusp_arch_resume() creates a temporary page table that covering only the linear map. It copies the restore code to a 'safe' page, then start to restore the memory image. Once completed, it restores the original kernel's page table. It then calls into __hibernate_cpu_resume() to restore the CPU context. Finally, it follows the normal hibernation path back to the hibernation core.

To enable hibernation/suspend to disk into RISCV, the below config need to be enabled:
- CONFIG_ARCH_HIBERNATION_HEADER
- CONFIG_ARCH_HIBERNATION_POSSIBLE

---
    arch/riscv/Kconfig                 |   6 +
    arch/riscv/include/asm/assembler.h |  20 ++
    arch/riscv/include/asm/suspend.h   |  19 ++
    arch/riscv/kernel/Makefile         |   1 +
    arch/riscv/kernel/asm-offsets.c    |   5 +
    arch/riscv/kernel/hibernate-asm.S  |  77 ++++++
    arch/riscv/kernel/hibernate.c      | 424 +++++++++++++++++++++++++++++
    7 files changed, 552 insertions(+)

  create mode 100644 arch/riscv/kernel/hibernate-asm.S create mode 100644 arch/riscv/kernel/hibernate.c

arch/riscv/Kconfig +5 -0
index c5e42cc37604..473c2a1a6884 100644
@@ -707,6 +707,12 @@ menu "Power management options"
      source "kernel/power/Kconfig"
    +config ARCH_HIBERNATION_POSSIBLE
+    def_bool y
+
+config ARCH_HIBERNATION_HEADER
+    def_bool HIBERNATION
+
    endmenu # "Power management options"
      menu "CPU Power Management"
arch/riscv/include/asm/assembler.h +19 -0
index ba59d38f8937..44b1457d3e95 100644
@@ -59,4 +59,24 @@
            REG_L    s11, (SUSPEND_CONTEXT_REGS + PT_S11)(a0)
        .endm
    +/*
+ * copy_page - copy 1 page (4KB) of data from source to destination
+ * @a0 - destination
+ * @a1 - source
+ */
+    .macro    copy_page a0, a1
+        lui    a2, 0x1
+        add    a2, a2, a0
+1 :
+        REG_L    t0, 0(a1)
+        REG_L    t1, SZREG(a1)
+
+        REG_S    t0, 0(a0)
+        REG_S    t1, SZREG(a0)
+
+        addi    a0, a0, 2 * SZREG
+        addi    a1, a1, 2 * SZREG
+        bne    a2, a0, 1b
+    .endm
+
    #endif    /* __ASM_ASSEMBLER_H */
arch/riscv/include/asm/suspend.h +4 -0
index 67e047445662..02f87867389a 100644
@@ -21,6 +21,11 @@ struct suspend_context {
    #endif
    };
    +/*
+ * Used by hibernation core and cleared during resume sequence
+ */
+extern int in_suspend;
+
    /* Low-level CPU suspend entry function */
    int __cpu_suspend_enter(struct suspend_context *context);
    @@ -36,4 +41,18 @@ int __cpu_resume_enter(unsigned long hartid,

unsigned long context);
  /* Used to save and restore the CSRs */
  void suspend_save_csrs(struct suspend_context *context); void suspend_restore_csrs(struct suspend_context *context);

arch/riscv/include/asm/suspend.h +12 -0
+
+/* Low-level API to support hibernation */
+int swsusp_arch_suspend(void);
+int swsusp_arch_resume(void);
+int arch_hibernation_header_save(void *addr, unsigned int max_size);
+int arch_hibernation_header_restore(void *addr);
+int __hibernate_cpu_resume(void);
+
+/* Used to resume on the CPU we hibernated on */
+int hibernate_resume_nonboot_cpu_disable(void);
+
+asmlinkage void hibernate_restore_image(unsigned long resume_satp,

unsigned long satp_temp,
+                    unsigned long cpu_resume);

arch/riscv/include/asm/suspend.h +1 -0
+asmlinkage int hibernate_core_restore_code(void);
    #endif
arch/riscv/kernel/Makefile +1 -0
index 4cf303a779ab..daab341d55e4 100644
@@ -64,6 +64,7 @@ obj-$(CONFIG_MODULES)        += module.o
    obj-$(CONFIG_MODULE_SECTIONS)    += module-sections.o
      obj-$(CONFIG_CPU_PM)        += suspend_entry.o suspend.o
+obj-$(CONFIG_HIBERNATION)    += hibernate.o hibernate-asm.o
      obj-$(CONFIG_FUNCTION_TRACER)    += mcount.o ftrace.o
    obj-$(CONFIG_DYNAMIC_FTRACE)    += mcount-dyn.o
arch/riscv/kernel/asm-offsets.c +4 -0
index df9444397908..d6a75aac1d27 100644
@@ -9,6 +9,7 @@
    #include <linux/kbuild.h>
    #include <linux/mm.h>
    #include <linux/sched.h>
+#include <linux/suspend.h>
    #include <asm/kvm_host.h>
    #include <asm/thread_info.h>
    #include <asm/ptrace.h>
@@ -116,6 +117,10 @@ void asm_offsets(void)
          OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
    +    OFFSET(HIBERN_PBE_ADDR, pbe, address);
+    OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
+    OFFSET(HIBERN_PBE_NEXT, pbe, next);
+
        OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
        OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
        OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
arch/riscv/kernel/hibernate-asm.S new file +19 -0
new file mode 100644
index 000000000000..effaf5ca5da0
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Hibernation low level support for RISCV.
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/csr.h>
+
+#include <linux/linkage.h>
+
+/*
+ * int __hibernate_cpu_resume(void)
+ * Switch back to the hibernated image's page table prior to

restoring the CPU
+ * context.
+ *
+ * Always returns 0
+ */

arch/riscv/kernel/hibernate-asm.S new file +35 -0
+ENTRY(__hibernate_cpu_resume)
+    /* switch to hibernated image's page table. */
+    csrw CSR_SATP, s0
+    sfence.vma
+
+    REG_L    a0, hibernate_cpu_context
+
+    suspend_restore_csrs
+    suspend_restore_regs
+
+    /* Return zero value. */
+    mv    a0, zero
+
+    ret
+END(__hibernate_cpu_resume)
+
+/*
+ * Prepare to restore the image.
+ * a0: satp of saved page tables.
+ * a1: satp of temporary page tables.
+ * a2: cpu_resume.
+ */
+ENTRY(hibernate_restore_image)
+    mv    s0, a0
+    mv    s1, a1
+    mv    s2, a2
+    REG_L    s4, restore_pblist
+    REG_L    a1, relocated_restore_code
+
+    jalr    a1
+END(hibernate_restore_image)
+
+/*
+ * The below code will be executed from a 'safe' page.
+ * It first switches to the temporary page table, then starts to

copy the pages
+ * back to the original memory location. Finally, it jumps to __hibernate_cpu_resume()
+ * to restore the CPU context.
+ */

arch/riscv/kernel/hibernate-asm.S new file +16 -0
+ENTRY(hibernate_core_restore_code)
+    /* switch to temp page table. */
+    csrw satp, s1
+    sfence.vma
+.Lcopy:
+    /* The below code will restore the hibernated image. */
+    REG_L    a1, HIBERN_PBE_ADDR(s4)
+    REG_L    a0, HIBERN_PBE_ORIG(s4)
+
+    copy_page a0, a1
+
+    REG_L    s4, HIBERN_PBE_NEXT(s4)
+    bnez    s4, .Lcopy
+
+    jalr    s2
+END(hibernate_core_restore_code)
arch/riscv/kernel/hibernate.c new file +28 -0
new file mode 100644
index 000000000000..8fab50122e54
@@ -0,0 +1,424 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Hibernation support for RISCV
+ *
+ * Copyright (C) 2023 StarFive Technology Co., Ltd.
+ *
+ * Author: Jee Heng Sia <[email protected]>
+ */
+
+#include <asm/barrier.h>
+#include <asm/cacheflush.h>
+#include <asm/mmu_context.h>
+#include <asm/page.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+#include <asm/sections.h>
+#include <asm/set_memory.h>
+#include <asm/smp.h>
+#include <asm/suspend.h>
+
+#include <linux/cpu.h>
+#include <linux/memblock.h>
+#include <linux/pm.h>
+#include <linux/sched.h>
+#include <linux/suspend.h>
+#include <linux/utsname.h>
+
+/* The logical cpu number we should resume on, initialised to a

non-cpu number. */

arch/riscv/kernel/hibernate.c new file +14 -0
+static int sleep_cpu = -EINVAL;
+
+/* Pointer to the temporary resume page table. */
+static pgd_t *resume_pg_dir;
+
+/* CPU context to be saved. */
+struct suspend_context *hibernate_cpu_context;
+EXPORT_SYMBOL_GPL(hibernate_cpu_context);
+
+unsigned long relocated_restore_code;
+EXPORT_SYMBOL_GPL(relocated_restore_code);
+
+/**
+ * struct arch_hibernate_hdr_invariants - container to store kernel

build version.
+ * @uts_version: to save the build number and date so that the we do not resume with
+ *        a different kernel.
+ */

arch/riscv/kernel/hibernate.c new file +6 -0
+struct arch_hibernate_hdr_invariants {
+    char        uts_version[__NEW_UTS_LEN + 1];
+};
+
+/**
+ * struct arch_hibernate_hdr - helper parameters that help us to

restore the image.
+ * @invariants: container to store kernel build version.
+ * @hartid: to make sure same boot_cpu executes the hibernate/restore code.
+ * @saved_satp: original page table used by the hibernated image.
+ * @restore_cpu_addr: the kernel's image address to restore the CPU context.
+ */

arch/riscv/kernel/hibernate.c new file +8 -0
+static struct arch_hibernate_hdr {
+    struct arch_hibernate_hdr_invariants invariants;
+    unsigned long    hartid;
+    unsigned long    saved_satp;
+    unsigned long    restore_cpu_addr;
+} resume_hdr;
+
+static void arch_hdr_invariants(struct arch_hibernate_hdr_invariants

*i)
+{
+    memset(i, 0, sizeof(*i));
+    memcpy(i->uts_version, init_utsname()->version, sizeof(i->uts_version));

arch/riscv/kernel/hibernate.c new file +93 -0
+}
+
+/*
+ * Check if the given pfn is in the 'nosave' section.
+ */
+int pfn_is_nosave(unsigned long pfn)
+{
+    unsigned long nosave_begin_pfn = sym_to_pfn(&__nosave_begin);
+    unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
+
+    return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn));
+}
+
+void notrace save_processor_state(void)
+{
+    WARN_ON(num_online_cpus() != 1);
+}
+
+void notrace restore_processor_state(void)
+{
+}
+
+/*
+ * Helper parameters need to be saved to the hibernation image header.
+ */
+int arch_hibernation_header_save(void *addr, unsigned int max_size)
+{
+    struct arch_hibernate_hdr *hdr = addr;
+
+    if (max_size < sizeof(*hdr))
+        return -EOVERFLOW;
+
+    arch_hdr_invariants(&hdr->invariants);
+
+    hdr->hartid = cpuid_to_hartid_map(sleep_cpu);
+    hdr->saved_satp = csr_read(CSR_SATP);
+    hdr->restore_cpu_addr = (unsigned long)__hibernate_cpu_resume;
+
+    return 0;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_save);
+
+/*
+ * Retrieve the helper parameters from the hibernation image header.
+ */
+int arch_hibernation_header_restore(void *addr)
+{
+    struct arch_hibernate_hdr_invariants invariants;
+    struct arch_hibernate_hdr *hdr = addr;
+    int ret = 0;
+
+    arch_hdr_invariants(&invariants);
+
+    if (memcmp(&hdr->invariants, &invariants, sizeof(invariants))) {
+        pr_crit("Hibernate image not generated by this kernel!\n");
+        return -EINVAL;
+    }
+
+    sleep_cpu = riscv_hartid_to_cpuid(hdr->hartid);
+    if (sleep_cpu < 0) {
+        pr_crit("Hibernated on a CPU not known to this kernel!\n");
+        sleep_cpu = -EINVAL;
+        return -EINVAL;
+    }
+
+#ifdef CONFIG_SMP
+    ret = bringup_hibernate_cpu(sleep_cpu);
+    if (ret) {
+        sleep_cpu = -EINVAL;
+        return ret;
+    }
+#endif
+    resume_hdr = *hdr;
+
+    return ret;
+}
+EXPORT_SYMBOL_GPL(arch_hibernation_header_restore);
+
+int swsusp_arch_suspend(void)
+{
+    int ret = 0;
+
+    if (__cpu_suspend_enter(hibernate_cpu_context)) {
+        sleep_cpu = smp_processor_id();
+        suspend_save_csrs(hibernate_cpu_context);
+        ret = swsusp_save();
+    } else {
+        suspend_restore_csrs(hibernate_cpu_context);
+        flush_tlb_all();
+        flush_icache_all();
+
+        /*
+         * Tell the hibernation core that we've just restored the

memory.
+         */
+        in_suspend = 0;
+        sleep_cpu = -EINVAL;
+    }

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return ret;
+}
+
+static int temp_pgtable_map_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +26 -0
+{
+    pte_t *src_ptep;
+    pte_t *dst_ptep;
+
+    if (pmd_none(READ_ONCE(*dst_pmdp))) {
+        dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_ptep)
+            return -ENOMEM;
+
+        pmd_populate_kernel(NULL, dst_pmdp, dst_ptep);
+    }
+
+    dst_ptep = pte_offset_kernel(dst_pmdp, start);
+    src_ptep = pte_offset_kernel(src_pmdp, start);
+
+    do {
+        pte_t pte = READ_ONCE(*src_ptep);
+
+        if (pte_present(pte))
+            set_pte(dst_ptep, __pte(pte_val(pte) | pgprot_val(prot)));
+    } while (dst_ptep++, src_ptep++, start += PAGE_SIZE, start < end);
+
+    return 0;
+}
+
+static int temp_pgtable_map_pmd(pud_t *dst_pudp, pud_t *src_pudp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pmd_t *src_pmdp;
+    pmd_t *dst_pmdp;
+
+    if (pud_none(READ_ONCE(*dst_pudp))) {
+        dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pmdp)
+            return -ENOMEM;
+
+        pud_populate(NULL, dst_pudp, dst_pmdp);
+    }
+
+    dst_pmdp = pmd_offset(dst_pudp, start);
+    src_pmdp = pmd_offset(src_pudp, start);
+
+    do {
+        pmd_t pmd = READ_ONCE(*src_pmdp);
+
+        next = pmd_addr_end(start, end);
+
+        if (pmd_none(pmd))
+            continue;
+
+        if (pmd_leaf(pmd)) {
+            set_pmd(dst_pmdp, __pmd(pmd_val(pmd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pte(dst_pmdp, src_pmdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pmdp++, src_pmdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    pud_t *dst_pudp;
+    pud_t *src_pudp;
+
+    if (p4d_none(READ_ONCE(*dst_p4dp))) {
+        dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_pudp)
+            return -ENOMEM;
+
+        p4d_populate(NULL, dst_p4dp, dst_pudp);
+    }
+
+    dst_pudp = pud_offset(dst_p4dp, start);
+    src_pudp = pud_offset(src_p4dp, start);
+
+    do {
+        pud_t pud = READ_ONCE(*src_pudp);
+
+        next = pud_addr_end(start, end);
+
+        if (pud_none(pud))
+            continue;
+
+        if (pud_leaf(pud)) {
+            set_pud(dst_pudp, __pud(pud_val(pud) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pmd(dst_pudp, src_pudp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pudp++, src_pudp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_map_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp,

unsigned long start,
+                unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +29 -0
+{
+    unsigned long next;
+    unsigned long ret;
+    p4d_t *dst_p4dp;
+    p4d_t *src_p4dp;
+
+    if (pgd_none(READ_ONCE(*dst_pgdp))) {
+        dst_p4dp = (p4d_t *)get_safe_page(GFP_ATOMIC);
+        if (!dst_p4dp)
+            return -ENOMEM;
+
+        pgd_populate(NULL, dst_pgdp, dst_p4dp);
+    }
+
+    dst_p4dp = p4d_offset(dst_pgdp, start);
+    src_p4dp = p4d_offset(src_pgdp, start);
+
+    do {
+        p4d_t p4d = READ_ONCE(*src_p4dp);
+
+        next = p4d_addr_end(start, end);
+
+        if (p4d_none(p4d))
+            continue;
+
+        if (p4d_leaf(p4d)) {
+            set_p4d(dst_p4dp, __p4d(p4d_val(p4d) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_pud(dst_p4dp, src_p4dp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_p4dp++, src_p4dp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +5 -0
+
+    return 0;
+}
+
+static int temp_pgtable_mapping(pgd_t *pgdp, unsigned long start,

unsigned long end, pgprot_t prot)

arch/riscv/kernel/hibernate.c new file +18 -0
+{
+    pgd_t *dst_pgdp = pgd_offset_pgd(pgdp, start);
+    pgd_t *src_pgdp = pgd_offset_k(start);
+    unsigned long next;
+    unsigned long ret;
+
+    do {
+        pgd_t pgd = READ_ONCE(*src_pgdp);
+
+        next = pgd_addr_end(start, end);
+
+        if (pgd_none(pgd))
+            continue;
+
+        if (pgd_leaf(pgd)) {
+            set_pgd(dst_pgdp, __pgd(pgd_val(pgd) | pgprot_val(prot)));
+        } else {
+            ret = temp_pgtable_map_p4d(dst_pgdp, src_pgdp, start,

next, prot);
+            if (ret)
+                return -ENOMEM;
+        }
+    } while (dst_pgdp++, src_pgdp++, start = next, start != end);

arch/riscv/kernel/hibernate.c new file +27 -0
+
+    return 0;
+}
+
+static unsigned long relocate_restore_code(void)
+{
+    void *page = (void *)get_safe_page(GFP_ATOMIC);
+
+    if (!page)
+        return -ENOMEM;
+
+    copy_page(page, hibernate_core_restore_code);
+
+    /* Make the page containing the relocated code executable. */
+    set_memory_x((unsigned long)page, 1);
+
+    return (unsigned long)page;
+}
+
+int swsusp_arch_resume(void)
+{
+    unsigned long end = (unsigned long)pfn_to_virt(max_low_pfn);
+    unsigned long start = PAGE_OFFSET;
+    int ret;
+
+    /*
+     * Memory allocated by get_safe_page() will be dealt with by the

hibernation core,
+     * we don't need to free it here.
+     */
+    resume_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC); +    if (!resume_pg_dir)
+        return -ENOMEM;
+
+    /*

Quoted 8 lines
Quoted 6 lines

+     * Create a temporary page table and map the whole linear region as executable and
+     * writable.
+     */
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));

Hmm weird protections, PAGE_PRESENT and PAGE_READ are not even set. Shouldn't you use PAGE_KERNEL_EXEC instead?

Quoted 1 line

Forget this one, that's the protections you add, so that's good :)

OK.

Quoted 19 lines
Quoted 19 lines
Quoted 19 lines

+    if (ret)
+        return ret;
+
+    /* Move the restore code to a new page so that it doesn't get overwritten by itself. */
+    relocated_restore_code = relocate_restore_code();
+    if (relocated_restore_code == -ENOMEM)
+        return -ENOMEM;
+
+    /*
+     * Map the __hibernate_cpu_resume() address to the temporary page table so that the
+     * restore code can jumps to it after finished restore the image. The next execution
+     * code doesn't find itself in a different address space after switching over to the
+     * original page table used by the hibernated image. +     *
+     */

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

+    start = (unsigned long)resume_hdr.restore_cpu_addr; +    end = start + PAGE_SIZE;
+
+    ret = temp_pgtable_mapping(resume_pg_dir, start, end, __pgprot(_PAGE_WRITE | _PAGE_EXEC));
+    if (ret)
+        return ret;

You set the linear mapping as executable so no need for this.

We need this mapping because the kernel addresses and the linear addresses in riscv-64bit are in different boundary. For

example:

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Linear address start with 0xff60000000000000
Kernel address start with 0xffffffff80000000

Ok I understand now that you need to map this into the temporary page
table: one nit though, since you copy the kernel mapping, no need to use PAGE_WRITE | PAGE_EXEC, the protections are already correct.

Here is the thing. For riscv_64bit, I am agreed with you that we don’t need to use PAGE_WRITE | PAGE_EXEC. However, we need to use PAGE_WRITE | PAGE_EXEC for riscv_32bit because the linear addresses and kernel addresses are similar.

Quoted 7 lines

The attribute becomes not writable if following the "original" page table. I am thinking to use #ifdef config_32bit to split the path for 64bit and 32bit but I just afraid the linear addresses and kernel addresses might change again in the future. To be on the safe side, I let both riscv_32bit & riscv_64bit to use the PAGE_WRITE | PAGE_EXEC.

Indeed, you really thought about everything, well done! I would add a comment that explains just that above, no need for other ifdefs IMO.

Sure, can add comment to explain the above situation.

Quoted 50 lines
Quoted 50 lines

For example:
Linear address and kernel address for 32bit started with 0xc0000000 Linear address for 64bit start with 0xff60000000000000 Kernel address for 64bit start with 0xffffffff80000000

Please let me know your thoughts.

Quoted 41 lines

Thanks,

Alex

Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
Quoted 24 lines

+
+    hibernate_restore_image(resume_hdr.saved_satp, (PFN_DOWN(__pa(resume_pg_dir)) | satp_mode), +                resume_hdr.restore_cpu_addr);

arch/riscv/kernel/hibernate.c new file +19 -0
+
+    return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP_SMP
+int hibernate_resume_nonboot_cpu_disable(void)
+{
+    if (sleep_cpu < 0) {
+        pr_err("Failing to resume from hibernate on an unknown CPU\n");
+        return -ENODEV;
+    }
+
+    return freeze_secondary_cpus(sleep_cpu);
+}
+#endif
+
+static int __init riscv_hibernate_init(void)
+{
+    hibernate_cpu_context = kzalloc(sizeof(*hibernate_cpu_context),

GFP_KERNEL);

arch/riscv/kernel/hibernate.c

Quoted 8 lines
arch/riscv/kernel/hibernate.c new file +8 -0
+
+    if (WARN_ON(!hibernate_cpu_context))
+        return -ENOMEM;
+
+    return 0;
+}
+
+early_initcall(riscv_hibernate_init);

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv