Skip to contentSkip to conversation

kexec-tools: arm64: Enable D-cache in purgatory

PATCH Superseded Pratyush Anand 3 participants 11 messages Dec 19, 2016 – Mar 15, 2017

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover kexec-tools: arm64: Enable D-cache in purgatory

It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image is around 13MB and initramfs is around 30MB. It takes more than 20 second even when we have -O2 optimization enabled. However, if dcache is enabled during purgatory execution then, it takes just a second in SHA verification.

Therefore, these patches adds support for dcache enabling facility during purgatory execution.

Although I have simplified the logic a bit now, however I understand that there are reservations for introducing this complexity for gaining few seconding of execution time during kexec or crash reboot. But, I believe if d-cache enabling code is stable enough then there should not be any hindrances to accept it. So, please give it a try with your platform and let me know if you see any issue or it does not work. I am still open to improve it further if needed.

Changes since V1:
- Moved page table creation logic from purgatory to kexec code.
- Only 4K page table is supported, with 48 bit VA and 2M block size
- if platform supports a 4K page size, then D-cache is always enabled now.

Pratyush Anand (2):
kexec: arm64: create identity page table to be used in purgatory
arm64: enable d-cache support during purgatory sha verification

 kexec/arch/arm64/kexec-arm64.c         | 152 ++++++++++++++++++
 purgatory/arch/arm64/Makefile          |   1 +
 purgatory/arch/arm64/cache.S           | 281 +++++++++++++++++++++++++++++++++
 purgatory/arch/arm64/purgatory-arm64.c |   5 +
 4 files changed, 439 insertions(+)
 create mode 100644 purgatory/arch/arm64/cache.S
Signature · 1 line
2.7.4
#1 lore raw

PATCH v2 1/2 kexec: arm64: create identity page table to be used in purgatory

Purgatory sha verification is very slow when D-cache is not enabled there. We need to enable MMU as well to enable D-Cache.Therefore,we need to an identity mapped page table in purgatory.

Since debugging is very difficult in purgatory therefore we prefer to do as much work as possible in kexec.

This patch prepares page table for purgatory in advance. We support only 4K page table,because it will be available on most of the platform. This page table is passed to the purgatory as a new segment.

VA bit is fixed as 48, page table level is 3 where 3rd level page table contains 2M block entries.

---
 kexec/arch/arm64/kexec-arm64.c | 152 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 152 insertions(+)
kexec/arch/arm64/kexec-arm64.c +152 -0
index 04fd3968bb52..c2c8ff1b6940 100644
@@ -24,6 +24,45 @@
 #include "kexec-syscall.h"
 #include "arch/options.h"
 
+/*
+ * kexec creates identity page table to be used in purgatory so that
+ * dcache verification becomes faster.
+ *
+ * These are the definitions to be used by page table creation routine.
+ *
+ * Only 4K page table, 3 level, 2M block mapping, 48bit VA is supported
+ */
+#define PGDIR_SHIFT		39
+#define PUD_SHIFT		30
+#define PMD_SHIFT		21
+#define PTRS_PER_PGD		0x1FF
+#define PTRS_PER_PUD		0x1FF
+#define PTRS_PER_PMD		0x1FF
+#define PMD_TYPE_TABLE		(3UL << 0)
+#define PMD_TYPE_SECT		(1UL << 0)
+#define PMD_SECT_AF		(1UL << 10)
+#define PMD_ATTRINDX(t)		((unsigned long)(t) << 2)
+#define MT_NORMAL		4
+#define PMD_FLAGS_NORMAL	(PMD_TYPE_SECT | PMD_SECT_AF)
+#define MMU_FLAGS_NORMAL	(PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS_NORMAL)
+#define SECTION_SIZE		(2 * 1024 * 1024)
+#define PAGE_SIZE		(4 * 1024)
+/* Since we are using 3 level of page tables, therefore minimum number of
+ * table will be 3. Most likely we will never need more than 3. Each entry
+ * in level 3 page table can map 2MB memory area. Thus a level 3 page table
+ * indexed by bit 29:21 can map a total of 1G memory area. Therefore, if
+ * any segment crosses 1G boundary, then we will need one more level 3
+ * table. Similarly, level 2 page table indexed by bit 38:30 can map a
+ * total of 512G memory area. If segment addresses are more than 512G apart
+ * then we will need two more table for each such block. We do not expect
+ * any memory segment to cross 512G boundary, however if we will ever wish
+ * to support uart debugging in purgatory then that might cross the
+ * boundary and therefore additional 2 more table space. Thus we will need
+ * maximum of 6 table space.
+ */
+#define MAX_PGTBLE_SZ	(6 * 4096)
+static int next_tbl_cnt = 1;
+
 /* Global varables the core kexec routines expect. */
 
 unsigned char reuse_initrd;
@@ -316,6 +355,117 @@ unsigned long arm64_locate_kernel_segment(struct kexec_info *info)
 	return hole;
 }
 
+static unsigned long *create_table_entry(unsigned long *pgtbl_buf,
+		unsigned long pgtbl_mem, unsigned long *tbl,
+		unsigned long virt, int shift,
+		unsigned long ptrs)
+{
+	unsigned long index, desc, offset;
+
+	index = (virt >> shift) & ptrs;
+	/* check if we have allocated a table already for this index */
+	if (tbl[index]) {
+		/*
+		 * table index will have entry as per purgatory page table
+		 * memory. Find out corresponding buffer address of table.
+		 */
+		desc = tbl[index] & ~3UL;
+		offset = desc - pgtbl_mem;
+		return &pgtbl_buf[offset >> 3];
+	}
+
+	/*
+	 * Always write page table content as per page table memory allocated
+	 * for purgaory area, but return corresponding buffer area alloced
+	 * in kexec
+	 */
+	if (next_tbl_cnt > 5)
+		die("%s: No more memory for page table\n", __func__);
+
+	tbl[index] = (pgtbl_mem + PAGE_SIZE * next_tbl_cnt) | PMD_TYPE_TABLE;
+
+	return &pgtbl_buf[(next_tbl_cnt++ * PAGE_SIZE) >> 3];
+}
+
+static void craete_block_entry(unsigned long *tbl, unsigned long flags,
+		unsigned long phys, unsigned long virt)
+{
+	unsigned long index;
+	unsigned long desc;
+
+	index = (virt >> PMD_SHIFT) & PTRS_PER_PMD;
+	desc = (phys >> PMD_SHIFT) << PMD_SHIFT;
+	desc |= flags;
+	tbl[index] = desc;
+}
+
+static void create_identity_entry(unsigned long *pgtbl_buf,
+		unsigned long pgtbl_mem, unsigned long virt,
+		unsigned long flags)
+{
+	unsigned long *tbl = pgtbl_buf;
+
+	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+			PGDIR_SHIFT, PTRS_PER_PGD);
+	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+			PUD_SHIFT, PTRS_PER_PUD);
+	craete_block_entry(tbl, flags, virt, virt);
+}
+
+/**
+ * arm64_create_pgtbl_segment - Create page table segments to be used by
+ * purgatory. Page table will have entries to access memory area of all
+ * those segments which becomes part of sha verification in purgatory.
+ * Additionaly, we also create page table for purgatory segment as well.
+ */
+
+static int arm64_create_pgtbl_segment(struct kexec_info *info,
+		unsigned long hole_min, unsigned long hole_max)
+{
+	unsigned long *pgtbl_buf;
+	int i;
+	unsigned long mstart, mend, pgtbl_mem;
+	unsigned long purgatory_base, purgatory_len;
+
+	pgtbl_buf = xmalloc(MAX_PGTBLE_SZ);
+	memset(pgtbl_buf, 0, MAX_PGTBLE_SZ);
+	pgtbl_mem = add_buffer_phys_virt(info, pgtbl_buf, MAX_PGTBLE_SZ,
+			MAX_PGTBLE_SZ, PAGE_SIZE, hole_min, hole_max, 1, 0);
+	for (i = 0; i < info->nr_segments; i++) {
+		if (info->segment[i].mem == (void *)info->rhdr.rel_addr) {
+			purgatory_base = (unsigned long)info->segment[i].mem;
+			purgatory_len = info->segment[i].memsz;
+		}
+		mstart = (unsigned long)info->segment[i].mem;
+		mend = mstart + info->segment[i].memsz;
+		mstart &= ~(SECTION_SIZE - 1);
+		while (mstart < mend) {
+			create_identity_entry(pgtbl_buf, pgtbl_mem,
+					mstart, MMU_FLAGS_NORMAL);
+			mstart += SECTION_SIZE;
+		}
+	}
+
+	/* we will need pgtble_base in purgatory for enabling d-cache */
+	elf_rel_set_symbol(&info->rhdr, "pgtble_base", &pgtbl_mem,
+		sizeof(pgtbl_mem));
+	/*
+	 * We need to disable d-cache before we exit from purgatory.
+	 * Since, only dcache flush by VAs is recomeneded, therefore we
+	 * will also need memory location of all those area which will be
+	 * accessed in purgatory with enabled d-cache. sha256_regions
+	 * already have start and length for all the segments except
+	 * purgatory. Therefore, we will need to pass start and length of
+	 * purgatory additionaly.
+	 */
+	elf_rel_set_symbol(&info->rhdr, "purgatory_base", &purgatory_base,
+		sizeof(purgatory_base));
+	elf_rel_set_symbol(&info->rhdr, "purgatory_len", &purgatory_len,
+		sizeof(purgatory_len));
+
+	return 0;
+}
+
 /**
  * arm64_load_other_segments - Prepare the dtb, initrd and purgatory segments.
  */
@@ -423,6 +573,8 @@ int arm64_load_other_segments(struct kexec_info *info,
 	elf_rel_set_symbol(&info->rhdr, "arm64_dtb_addr", &dtb_base,
 		sizeof(dtb_base));
 
+	arm64_create_pgtbl_segment(info, hole_min, hole_max);
+
 	return 0;
 }
 
Signature · 1 line
2.7.4
#7 lore raw

PATCH v2 2/2 arm64: enable d-cache support during purgatory sha verification

In reply to Pratyush Anand

If a platform supports 4K page table then enable D-cache in purgatory before SHA verification. Disable it before switching to kernel.

---
 purgatory/arch/arm64/Makefile          |   1 +
 purgatory/arch/arm64/cache.S           | 281 +++++++++++++++++++++++++++++++++
 purgatory/arch/arm64/purgatory-arm64.c |   5 +
 3 files changed, 287 insertions(+)
 create mode 100644 purgatory/arch/arm64/cache.S
purgatory/arch/arm64/Makefile +1 -0
index 636abeab17b2..db28a0de6891 100644
@@ -11,6 +11,7 @@ arm64_PURGATORY_EXTRA_CFLAGS = \
 
 arm64_PURGATORY_SRCS += \
 	purgatory/arch/arm64/entry.S \
+	purgatory/arch/arm64/cache.S \
 	purgatory/arch/arm64/purgatory-arm64.c
 
 dist += \
purgatory/arch/arm64/cache.S new file +281 -0

281 lines not on this page · raw message

purgatory/arch/arm64/purgatory-arm64.c +5 -0
index fe50fcf8ebc3..638fb11d9843 100644
@@ -5,6 +5,9 @@
 #include <stdint.h>
 #include <purgatory.h>
 
+void enable_dcache(void);
+void disable_dcache(void);
+
 void putchar(int ch)
 {
 	/* Nothing for now */
@@ -12,8 +15,10 @@ void putchar(int ch)
 
 void post_verification_setup_arch(void)
 {
+	disable_dcache();
 }
 
 void setup_arch(void)
 {
+	enable_dcache();
 }
Signature · 1 line
2.7.4
#2 lore raw

Re: [PATCH V2 1/2] kexec: arm64: create identity page table to be used in purgatory

In reply to Pratyush Anand

2016-12-19 10:13 GMT+03:00 Pratyush Anand <[email protected]>:

Quoted 96 lines

Purgatory sha verification is very slow when D-cache is not enabled there. We need to enable MMU as well to enable D-Cache.Therefore,we need to an identity mapped page table in purgatory.

Since debugging is very difficult in purgatory therefore we prefer to do as much work as possible in kexec.

This patch prepares page table for purgatory in advance. We support only 4K page table,because it will be available on most of the platform. This page table is passed to the purgatory as a new segment.

VA bit is fixed as 48, page table level is 3 where 3rd level page table contains 2M block entries.

---
 kexec/arch/arm64/kexec-arm64.c | 152 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 152 insertions(+)
kexec/arch/arm64/kexec-arm64.c +62 -0
index 04fd3968bb52..c2c8ff1b6940 100644
@@ -24,6 +24,45 @@
 #include "kexec-syscall.h"
 #include "arch/options.h"
 
+/*
+ * kexec creates identity page table to be used in purgatory so that
+ * dcache verification becomes faster.
+ *
+ * These are the definitions to be used by page table creation routine.
+ *
+ * Only 4K page table, 3 level, 2M block mapping, 48bit VA is supported
+ */
+#define PGDIR_SHIFT            39
+#define PUD_SHIFT              30
+#define PMD_SHIFT              21
+#define PTRS_PER_PGD           0x1FF
+#define PTRS_PER_PUD           0x1FF
+#define PTRS_PER_PMD           0x1FF
+#define PMD_TYPE_TABLE         (3UL << 0)
+#define PMD_TYPE_SECT          (1UL << 0)
+#define PMD_SECT_AF            (1UL << 10)
+#define PMD_ATTRINDX(t)                ((unsigned long)(t) << 2)
+#define MT_NORMAL              4
+#define PMD_FLAGS_NORMAL       (PMD_TYPE_SECT | PMD_SECT_AF)
+#define MMU_FLAGS_NORMAL       (PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS_NORMAL)
+#define SECTION_SIZE           (2 * 1024 * 1024)
+#define PAGE_SIZE              (4 * 1024)
+/* Since we are using 3 level of page tables, therefore minimum number of
+ * table will be 3. Most likely we will never need more than 3. Each entry
+ * in level 3 page table can map 2MB memory area. Thus a level 3 page table
+ * indexed by bit 29:21 can map a total of 1G memory area. Therefore, if
+ * any segment crosses 1G boundary, then we will need one more level 3
+ * table. Similarly, level 2 page table indexed by bit 38:30 can map a
+ * total of 512G memory area. If segment addresses are more than 512G apart
+ * then we will need two more table for each such block. We do not expect
+ * any memory segment to cross 512G boundary, however if we will ever wish
+ * to support uart debugging in purgatory then that might cross the
+ * boundary and therefore additional 2 more table space. Thus we will need
+ * maximum of 6 table space.
+ */
+#define MAX_PGTBLE_SZ  (6 * 4096)
+static int next_tbl_cnt = 1;
+
 /* Global varables the core kexec routines expect. */
 
 unsigned char reuse_initrd;
@@ -316,6 +355,117 @@ unsigned long arm64_locate_kernel_segment(struct kexec_info *info)
        return hole;
 }
 
+static unsigned long *create_table_entry(unsigned long *pgtbl_buf,
+               unsigned long pgtbl_mem, unsigned long *tbl,
+               unsigned long virt, int shift,
+               unsigned long ptrs)
+{
+       unsigned long index, desc, offset;
+
+       index = (virt >> shift) & ptrs;
+       /* check if we have allocated a table already for this index */
+       if (tbl[index]) {
+               /*
+                * table index will have entry as per purgatory page table
+                * memory. Find out corresponding buffer address of table.
+                */
+               desc = tbl[index] & ~3UL;
+               offset = desc - pgtbl_mem;
+               return &pgtbl_buf[offset >> 3];
+       }
+
+       /*
+        * Always write page table content as per page table memory allocated
+        * for purgaory area, but return corresponding buffer area alloced
+        * in kexec

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 390

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+        */
+       if (next_tbl_cnt > 5)
+               die("%s: No more memory for page table\n", __func__);
+
+       tbl[index] = (pgtbl_mem + PAGE_SIZE * next_tbl_cnt) | PMD_TYPE_TABLE;
+
+       return &pgtbl_buf[(next_tbl_cnt++ * PAGE_SIZE) >> 3];
+}
+
+static void craete_block_entry(unsigned long *tbl, unsigned long flags,

typo in name

Quoted 98 lines
kexec/arch/arm64/kexec-arm64.c +80 -1
+               unsigned long phys, unsigned long virt)
+{
+       unsigned long index;
+       unsigned long desc;
+
+       index = (virt >> PMD_SHIFT) & PTRS_PER_PMD;
+       desc = (phys >> PMD_SHIFT) << PMD_SHIFT;
+       desc |= flags;
+       tbl[index] = desc;
+}
+
+static void create_identity_entry(unsigned long *pgtbl_buf,
+               unsigned long pgtbl_mem, unsigned long virt,
+               unsigned long flags)
+{
+       unsigned long *tbl = pgtbl_buf;
+
+       tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+                       PGDIR_SHIFT, PTRS_PER_PGD);
+       tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+                       PUD_SHIFT, PTRS_PER_PUD);
+       craete_block_entry(tbl, flags, virt, virt);
+}
+
+/**
+ * arm64_create_pgtbl_segment - Create page table segments to be used by
+ * purgatory. Page table will have entries to access memory area of all
+ * those segments which becomes part of sha verification in purgatory.
+ * Additionaly, we also create page table for purgatory segment as well.
+ */
+
+static int arm64_create_pgtbl_segment(struct kexec_info *info,
+               unsigned long hole_min, unsigned long hole_max)
+{
+       unsigned long *pgtbl_buf;
+       int i;
+       unsigned long mstart, mend, pgtbl_mem;
+       unsigned long purgatory_base, purgatory_len;
+
+       pgtbl_buf = xmalloc(MAX_PGTBLE_SZ);
+       memset(pgtbl_buf, 0, MAX_PGTBLE_SZ);
+       pgtbl_mem = add_buffer_phys_virt(info, pgtbl_buf, MAX_PGTBLE_SZ,
+                       MAX_PGTBLE_SZ, PAGE_SIZE, hole_min, hole_max, 1, 0);
+       for (i = 0; i < info->nr_segments; i++) {
+               if (info->segment[i].mem == (void *)info->rhdr.rel_addr) {
+                       purgatory_base = (unsigned long)info->segment[i].mem;
+                       purgatory_len = info->segment[i].memsz;
+               }
+               mstart = (unsigned long)info->segment[i].mem;
+               mend = mstart + info->segment[i].memsz;
+               mstart &= ~(SECTION_SIZE - 1);
+               while (mstart < mend) {
+                       create_identity_entry(pgtbl_buf, pgtbl_mem,
+                                       mstart, MMU_FLAGS_NORMAL);
+                       mstart += SECTION_SIZE;
+               }
+       }
+
+       /* we will need pgtble_base in purgatory for enabling d-cache */
+       elf_rel_set_symbol(&info->rhdr, "pgtble_base", &pgtbl_mem,
+               sizeof(pgtbl_mem));
+       /*
+        * We need to disable d-cache before we exit from purgatory.
+        * Since, only dcache flush by VAs is recomeneded, therefore we
+        * will also need memory location of all those area which will be
+        * accessed in purgatory with enabled d-cache. sha256_regions
+        * already have start and length for all the segments except
+        * purgatory. Therefore, we will need to pass start and length of
+        * purgatory additionaly.
+        */
+       elf_rel_set_symbol(&info->rhdr, "purgatory_base", &purgatory_base,
+               sizeof(purgatory_base));
+       elf_rel_set_symbol(&info->rhdr, "purgatory_len", &purgatory_len,
+               sizeof(purgatory_len));
+
+       return 0;
+}
+
 /**
  * arm64_load_other_segments - Prepare the dtb, initrd and purgatory segments.
  */
@@ -423,6 +573,8 @@ int arm64_load_other_segments(struct kexec_info *info,
        elf_rel_set_symbol(&info->rhdr, "arm64_dtb_addr", &dtb_base,
                sizeof(dtb_base));
 
+       arm64_create_pgtbl_segment(info, hole_min, hole_max);
+
        return 0;
 }
 
--

2.7.4

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Signature · 7 lines
Best regards,
Maxim Uvarov

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#3 lore raw

Re: [PATCH V2 1/2] kexec: arm64: create identity page table to be used in purgatory

Quoted 4 lines · Maxim Uvarov
Quoted 2 lines

+

Quoted 1 line

+static void craete_block_entry(unsigned long *tbl, unsigned long flags,

typo in name

Thanks.
Will fix.

~Pratyush

#8 lore raw
In reply to Pratyush Anand

Hi James and All,

Any feedback/review comment on it?

~Pratyush

Quoted 34 lines · Pratyush Anand

It takes more that 2 minutes to verify SHA in purgatory when vmlinuz image is around 13MB and initramfs is around 30MB. It takes more than 20 second even when we have -O2 optimization enabled. However, if dcache is enabled during purgatory execution then, it takes just a second in SHA verification.

Therefore, these patches adds support for dcache enabling facility during purgatory execution.

Although I have simplified the logic a bit now, however I understand that there are reservations for introducing this complexity for gaining few seconding of execution time during kexec or crash reboot. But, I believe if d-cache enabling code is stable enough then there should not be any hindrances to accept it. So, please give it a try with your platform and let me know if you see any issue or it does not work. I am still open to improve it further if needed.

Changes since V1:
- Moved page table creation logic from purgatory to kexec code.
- Only 4K page table is supported, with 48 bit VA and 2M block size
- if platform supports a 4K page size, then D-cache is always enabled now.

Pratyush Anand (2):
kexec: arm64: create identity page table to be used in purgatory
arm64: enable d-cache support during purgatory sha verification

 kexec/arch/arm64/kexec-arm64.c         | 152 ++++++++++++++++++
 purgatory/arch/arm64/Makefile          |   1 +
 purgatory/arch/arm64/cache.S           | 281 +++++++++++++++++++++++++++++++++
 purgatory/arch/arm64/purgatory-arm64.c |   5 +
 4 files changed, 439 insertions(+)
 create mode 100644 purgatory/arch/arm64/cache.S
#4 lore raw

Re: [PATCH V2 1/2] kexec: arm64: create identity page table to be used in purgatory

In reply to Pratyush Anand

Hi Pratyush,

Quoted 26 lines · Pratyush Anand

Purgatory sha verification is very slow when D-cache is not enabled there. We need to enable MMU as well to enable D-Cache.Therefore,we need to an identity mapped page table in purgatory.

Since debugging is very difficult in purgatory therefore we prefer to do as much work as possible in kexec.

This patch prepares page table for purgatory in advance. We support only 4K page table,because it will be available on most of the platform. This page table is passed to the purgatory as a new segment.

VA bit is fixed as 48, page table level is 3 where 3rd level page table contains 2M block entries.

kexec/arch/arm64/kexec-arm64.c +4 -0
index 04fd3968bb52..c2c8ff1b6940 100644
@@ -24,6 +24,45 @@
 #include "kexec-syscall.h"
 #include "arch/options.h"
 
+/*
+ * kexec creates identity page table to be used in purgatory so that
+ * dcache verification becomes faster.
+ *

kexec/arch/arm64/kexec-arm64.c line 40

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+ * These are the definitions to be used by page table creation routine.
+ *
+ * Only 4K page table, 3 level, 2M block mapping, 48bit VA is supported
+ */
+#define PGDIR_SHIFT		39
+#define PUD_SHIFT		30
+#define PMD_SHIFT		21
+#define PTRS_PER_PGD		0x1FF
+#define PTRS_PER_PUD		0x1FF
+#define PTRS_PER_PMD		0x1FF

Aren't these 0x200 for 4K pages in the kernel? It looks like you use them as a mask instead.

kexec/arch/arm64/kexec-arm64.c

Quoted 6 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+#define PMD_TYPE_TABLE		(3UL << 0)
+#define PMD_TYPE_SECT		(1UL << 0)
+#define PMD_SECT_AF		(1UL << 10)
+#define PMD_ATTRINDX(t)		((unsigned long)(t) << 2)
 
+#define MT_NORMAL		4

This needs to correspond to the part of MAIR that describes the memory type for 'normal'. You define MT_NORMAL again in the next patch, it would be better to share the definition so they can't be different. (I don't see why you can't use 0!)

kexec/arch/arm64/kexec-arm64.c line 47

Quoted 2 lines
kexec/arch/arm64/kexec-arm64.c +2 -0
+#define PMD_FLAGS_NORMAL	(PMD_TYPE_SECT | PMD_SECT_AF)
+#define MMU_FLAGS_NORMAL	(PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS_NORMAL)

The SH and AP bits are left as zero, which means non-shareable, read/writeable, (which I think is fine).

Quoted 4 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+#define SECTION_SIZE		(2 * 1024 * 1024)
+#define PAGE_SIZE		(4 * 1024)
+/* Since we are using 3 level of page tables, therefore minimum number of
+ * table will be 3. Most likely we will never need more than 3. Each entry

kexec/arch/arm64/kexec-arm64.c line 61

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+ * in level 3 page table can map 2MB memory area. Thus a level 3 page table
+ * indexed by bit 29:21 can map a total of 1G memory area. Therefore, if
+ * any segment crosses 1G boundary, then we will need one more level 3
+ * table. Similarly, level 2 page table indexed by bit 38:30 can map a
+ * total of 512G memory area. If segment addresses are more than 512G apart
+ * then we will need two more table for each such block. We do not expect
+ * any memory segment to cross 512G boundary, however if we will ever wish
+ * to support uart debugging in purgatory then that might cross the
+ * boundary and therefore additional 2 more table space. Thus we will need
+ * maximum of 6 table space.

Surely we only need 6 page_size table entries if we support uart debugging, which your 'if we ever' suggests we don't. So surely we only need 3, and a comment that this needs expanding to 6 if we need to map two distinct areas.

Quoted 17 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+ */
+#define MAX_PGTBLE_SZ	(6 * 4096)
+static int next_tbl_cnt = 1;
+
 /* Global varables the core kexec routines expect. */
 

unsigned char reuse_initrd;

kexec/arch/arm64/kexec-arm64.c +6 -0
@@ -316,6 +355,117 @@ unsigned long arm64_locate_kernel_segment(struct kexec_info *info)
 	return hole;
 }
 
+static unsigned long *create_table_entry(unsigned long *pgtbl_buf,
+		unsigned long pgtbl_mem, unsigned long *tbl,
+		unsigned long virt, int shift,
+		unsigned long ptrs)
+{
+	unsigned long index, desc, offset;

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 373

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+
+	index = (virt >> shift) & ptrs;
+	/* check if we have allocated a table already for this index */
+	if (tbl[index]) {
+		/*
+		 * table index will have entry as per purgatory page table
+		 * memory. Find out corresponding buffer address of table.
+		 */
+		desc = tbl[index] & ~3UL;
+		offset = desc - pgtbl_mem;

wait .. no .. pgtbl_mem is also a guest physical address, so this is fine... its not obvious at first glance! You could do with a typedef to make it clear which addresses are the guests (pgtable_mem) and which are the hosts (pgtable_buf). Something like phys_addr_t?

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 379

Quoted 6 lines
kexec/arch/arm64/kexec-arm64.c +6 -0
+		return &pgtbl_buf[offset >> 3];
+	}
+
+	/*
+	 * Always write page table content as per page table memory allocated
+	 * for purgaory area, but return corresponding buffer area alloced

Nit: purgatory, allocated,

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 383

Quoted 4 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+	 * in kexec
+	 */
+	if (next_tbl_cnt > 5)
+		die("%s: No more memory for page table\n", __func__);

die()? With a bit of juggling can't we return an error so we never try to enable the MMU+dcache instead?

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 391

Quoted 8 lines
kexec/arch/arm64/kexec-arm64.c +8 -0
+
+	tbl[index] = (pgtbl_mem + PAGE_SIZE * next_tbl_cnt) | PMD_TYPE_TABLE;
+
+	return &pgtbl_buf[(next_tbl_cnt++ * PAGE_SIZE) >> 3];
+}
+
+static void craete_block_entry(unsigned long *tbl, unsigned long flags,
+		unsigned long phys, unsigned long virt)

Why have separate phys and virt parameters if all this ever does is idmap?

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 396

Quoted 5 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+{
+	unsigned long index;
+	unsigned long desc;
+
+	index = (virt >> PMD_SHIFT) & PTRS_PER_PMD;

Copying the pmd_index() macro would make this clearer.

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 397

Quoted 1 line
kexec/arch/arm64/kexec-arm64.c +1 -0
+	desc = (phys >> PMD_SHIFT) << PMD_SHIFT;

Looks like you wanted a PMD_MASK.

Quoted 5 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+	desc |= flags;
+	tbl[index] = desc;
+}
+
+static void create_identity_entry(unsigned long *pgtbl_buf,

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 412

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+		unsigned long pgtbl_mem, unsigned long virt,
+		unsigned long flags)
+{
+	unsigned long *tbl = pgtbl_buf;
+
+	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+			PGDIR_SHIFT, PTRS_PER_PGD);
+	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+			PUD_SHIFT, PTRS_PER_PUD);
+	craete_block_entry(tbl, flags, virt, virt);

Nit: create

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 419

Quoted 7 lines
kexec/arch/arm64/kexec-arm64.c +7 -0
+}
+
+/**
+ * arm64_create_pgtbl_segment - Create page table segments to be used by
+ * purgatory. Page table will have entries to access memory area of all
+ * those segments which becomes part of sha verification in purgatory.
+ * Additionaly, we also create page table for purgatory segment as well.

Nit: additionally,

Quoted 5 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+ */
+
+static int arm64_create_pgtbl_segment(struct kexec_info *info,
+		unsigned long hole_min, unsigned long hole_max)
+{

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 433

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +9 -0
+	unsigned long *pgtbl_buf;
+	int i;
+	unsigned long mstart, mend, pgtbl_mem;
+	unsigned long purgatory_base, purgatory_len;
+
+	pgtbl_buf = xmalloc(MAX_PGTBLE_SZ);
+	memset(pgtbl_buf, 0, MAX_PGTBLE_SZ);
 
+	pgtbl_mem = add_buffer_phys_virt(info, pgtbl_buf, MAX_PGTBLE_SZ,
+			MAX_PGTBLE_SZ, PAGE_SIZE, hole_min, hole_max, 1, 0);

I want to check what this does, but this all looks like it is doing the right thing.

Quoted 4 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+	for (i = 0; i < info->nr_segments; i++) {
+		if (info->segment[i].mem == (void *)info->rhdr.rel_addr) {
+			purgatory_base = (unsigned long)info->segment[i].mem;
+			purgatory_len = info->segment[i].memsz;

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 447

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+		}
+		mstart = (unsigned long)info->segment[i].mem;
+		mend = mstart + info->segment[i].memsz;
+		mstart &= ~(SECTION_SIZE - 1);
+		while (mstart < mend) {
+			create_identity_entry(pgtbl_buf, pgtbl_mem,
+					mstart, MMU_FLAGS_NORMAL);
+			mstart += SECTION_SIZE;
+		}
+	}

Thanks,

James

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#5 lore raw

Re: [PATCH V2 1/2] kexec: arm64: create identity page table to be used in purgatory

Hi James,

Thanks for your review.

Quoted 34 lines · James Morse

Hi Pratyush,

Quoted 31 lines · Pratyush Anand

Purgatory sha verification is very slow when D-cache is not enabled there. We need to enable MMU as well to enable D-Cache.Therefore,we need to an identity mapped page table in purgatory.

Since debugging is very difficult in purgatory therefore we prefer to do as much work as possible in kexec.

This patch prepares page table for purgatory in advance. We support only 4K page table,because it will be available on most of the platform. This page table is passed to the purgatory as a new segment.

VA bit is fixed as 48, page table level is 3 where 3rd level page table contains 2M block entries.

kexec/arch/arm64/kexec-arm64.c +9 -0
index 04fd3968bb52..c2c8ff1b6940 100644
@@ -24,6 +24,45 @@
 #include "kexec-syscall.h"
 #include "arch/options.h"
 
+/*
+ * kexec creates identity page table to be used in purgatory so that
+ * dcache verification becomes faster.
+ *
+ * These are the definitions to be used by page table creation routine.
+ *
+ * Only 4K page table, 3 level, 2M block mapping, 48bit VA is supported
+ */
+#define PGDIR_SHIFT		39
Quoted 9 lines

kexec/arch/arm64/kexec-arm64.c line 40

Quoted 5 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+#define PUD_SHIFT		30
+#define PMD_SHIFT		21
+#define PTRS_PER_PGD		0x1FF
+#define PTRS_PER_PUD		0x1FF
+#define PTRS_PER_PMD		0x1FF

Aren't these 0x200 for 4K pages in the kernel? It looks like you use them as a mask instead.

will use definitions as in kernel.

Quoted 11 lines

kexec/arch/arm64/kexec-arm64.c

Quoted 7 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+#define PMD_TYPE_TABLE		(3UL << 0)
+#define PMD_TYPE_SECT		(1UL << 0)
+#define PMD_SECT_AF		(1UL << 10)
+#define PMD_ATTRINDX(t)		((unsigned long)(t) << 2)
 
+#define MT_NORMAL		4

This needs to correspond to the part of MAIR that describes the memory type for 'normal'. You define MT_NORMAL again in the next patch, it would be better to share the definition so they can't be different. (I don't see why you can't use 0!)

Hummm...Had thought to share the definition, but inclusion of header file into purgatory code looked like ugly. But I see purgatory/purgatory.c is including "../kexec/kexec-sha256.h". So, probably I can do that similarly.

Yes, can use 0 as well, as there is no other memory type now.

Quoted 19 lines

kexec/arch/arm64/kexec-arm64.c line 47

Quoted 4 lines
kexec/arch/arm64/kexec-arm64.c +2 -0
+#define PMD_FLAGS_NORMAL	(PMD_TYPE_SECT | PMD_SECT_AF)
+#define MMU_FLAGS_NORMAL	(PMD_ATTRINDX(MT_NORMAL) | PMD_FLAGS_NORMAL)

The SH and AP bits are left as zero, which means non-shareable, read/writeable, (which I think is fine).

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+#define SECTION_SIZE		(2 * 1024 * 1024)
+#define PAGE_SIZE		(4 * 1024)
+/* Since we are using 3 level of page tables, therefore minimum number of
+ * table will be 3. Most likely we will never need more than 3. Each entry
+ * in level 3 page table can map 2MB memory area. Thus a level 3 page table
+ * indexed by bit 29:21 can map a total of 1G memory area. Therefore, if
+ * any segment crosses 1G boundary, then we will need one more level 3
+ * table. Similarly, level 2 page table indexed by bit 38:30 can map a
+ * total of 512G memory area. If segment addresses are more than 512G apart
+ * then we will need two more table for each such block. We do not expect
Quoted 9 lines

kexec/arch/arm64/kexec-arm64.c line 61

Quoted 4 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+ * any memory segment to cross 512G boundary, however if we will ever wish
+ * to support uart debugging in purgatory then that might cross the
+ * boundary and therefore additional 2 more table space. Thus we will need
+ * maximum of 6 table space.

Surely we only need 6 page_size table entries if we support uart debugging, which your 'if we ever' suggests we don't. So surely we only need 3, and a comment that this needs expanding to 6 if we need to map two distinct areas.

OK, will use 3 now and write comments accordingly.

Quoted 25 lines
Quoted 25 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+ */
+#define MAX_PGTBLE_SZ	(6 * 4096)
+static int next_tbl_cnt = 1;
+
 /* Global varables the core kexec routines expect. */
 

unsigned char reuse_initrd;

kexec/arch/arm64/kexec-arm64.c +13 -0
@@ -316,6 +355,117 @@ unsigned long arm64_locate_kernel_segment(struct kexec_info *info)
 	return hole;
 }
 
+static unsigned long *create_table_entry(unsigned long *pgtbl_buf,
+		unsigned long pgtbl_mem, unsigned long *tbl,
+		unsigned long virt, int shift,
+		unsigned long ptrs)
+{
+	unsigned long index, desc, offset;
+
+	index = (virt >> shift) & ptrs;
+	/* check if we have allocated a table already for this index */
+	if (tbl[index]) {
+		/*
+		 * table index will have entry as per purgatory page table
+		 * memory. Find out corresponding buffer address of table.
Quoted 8 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 373

Quoted 3 lines
kexec/arch/arm64/kexec-arm64.c +3 -0
+		 */
+		desc = tbl[index] & ~3UL;
+		offset = desc - pgtbl_mem;

wait .. no .. pgtbl_mem is also a guest physical address, so this is fine... its not obvious at first glance! You could do with a typedef to make it clear which addresses are the guests (pgtable_mem) and which are the hosts (pgtable_buf). Something like phys_addr_t?

so may be host_addr_t and guest_addr_t??

Quoted 10 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 379

Quoted 8 lines
kexec/arch/arm64/kexec-arm64.c +6 -0
+		return &pgtbl_buf[offset >> 3];
+	}
+
+	/*
+	 * Always write page table content as per page table memory allocated
+	 * for purgaory area, but return corresponding buffer area alloced

Nit: purgatory, allocated,

ok.

Quoted 9 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 383

Quoted 5 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+	 * in kexec
+	 */
+	if (next_tbl_cnt > 5)
+		die("%s: No more memory for page table\n", __func__);

die()? With a bit of juggling can't we return an error so we never try to enable the MMU+dcache instead?

OK. can do that.

Quoted 11 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 391

Quoted 9 lines
kexec/arch/arm64/kexec-arm64.c +8 -0
+
+	tbl[index] = (pgtbl_mem + PAGE_SIZE * next_tbl_cnt) | PMD_TYPE_TABLE;
+
+	return &pgtbl_buf[(next_tbl_cnt++ * PAGE_SIZE) >> 3];
+}
+
+static void craete_block_entry(unsigned long *tbl, unsigned long flags,
+		unsigned long phys, unsigned long virt)

Why have separate phys and virt parameters if all this ever does is idmap?

Agreed.

Quoted 9 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 396

Quoted 7 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+{
+	unsigned long index;
+	unsigned long desc;
+
+	index = (virt >> PMD_SHIFT) & PTRS_PER_PMD;

Copying the pmd_index() macro would make this clearer.

Right.

Quoted 5 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 397

Quoted 3 lines
kexec/arch/arm64/kexec-arm64.c +1 -0
+	desc = (phys >> PMD_SHIFT) << PMD_SHIFT;

Looks like you wanted a PMD_MASK.

Right.

Quoted 11 lines
Quoted 11 lines
kexec/arch/arm64/kexec-arm64.c +9 -0
+	desc |= flags;
+	tbl[index] = desc;
+}
+
+static void create_identity_entry(unsigned long *pgtbl_buf,
+		unsigned long pgtbl_mem, unsigned long virt,
+		unsigned long flags)
+{
+	unsigned long *tbl = pgtbl_buf;
Quoted 8 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 412

Quoted 6 lines
kexec/arch/arm64/kexec-arm64.c +6 -0
+
+	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+			PGDIR_SHIFT, PTRS_PER_PGD);
+	tbl = create_table_entry(pgtbl_buf, pgtbl_mem, tbl, virt,
+			PUD_SHIFT, PTRS_PER_PUD);
+	craete_block_entry(tbl, flags, virt, virt);

Nit: create

Ok.

Quoted 10 lines

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 419

Quoted 8 lines
kexec/arch/arm64/kexec-arm64.c +7 -0
+}
+
+/**
+ * arm64_create_pgtbl_segment - Create page table segments to be used by
+ * purgatory. Page table will have entries to access memory area of all
+ * those segments which becomes part of sha verification in purgatory.
+ * Additionaly, we also create page table for purgatory segment as well.

Nit: additionally,

Ok.

Quoted 20 lines
Quoted 6 lines
kexec/arch/arm64/kexec-arm64.c +5 -0
+ */
+
+static int arm64_create_pgtbl_segment(struct kexec_info *info,
+		unsigned long hole_min, unsigned long hole_max)
+{

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 433

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +9 -0
+	unsigned long *pgtbl_buf;
+	int i;
+	unsigned long mstart, mend, pgtbl_mem;
+	unsigned long purgatory_base, purgatory_len;
+
+	pgtbl_buf = xmalloc(MAX_PGTBLE_SZ);
+	memset(pgtbl_buf, 0, MAX_PGTBLE_SZ);
 
+	pgtbl_mem = add_buffer_phys_virt(info, pgtbl_buf, MAX_PGTBLE_SZ,
+			MAX_PGTBLE_SZ, PAGE_SIZE, hole_min, hole_max, 1, 0);

I want to check what this does, but this all looks like it is doing the right thing.

Quoted 15 lines
Quoted 4 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+	for (i = 0; i < info->nr_segments; i++) {
+		if (info->segment[i].mem == (void *)info->rhdr.rel_addr) {
+			purgatory_base = (unsigned long)info->segment[i].mem;
+			purgatory_len = info->segment[i].memsz;

kexec/arch/arm64/kexec-arm64.c unsigned long arm64_locate_kernel_segment(struct kexec_info *info) · line 447

Quoted 10 lines
kexec/arch/arm64/kexec-arm64.c +10 -0
+		}
+		mstart = (unsigned long)info->segment[i].mem;
+		mend = mstart + info->segment[i].memsz;
+		mstart &= ~(SECTION_SIZE - 1);
+		while (mstart < mend) {
+			create_identity_entry(pgtbl_buf, pgtbl_mem,
+					mstart, MMU_FLAGS_NORMAL);
+			mstart += SECTION_SIZE;
+		}
+	}

~Pratyush

#10 lore raw
In reply to James Morse

Hi James and all,

Any review comment for patch 2/2?

If no, then I will address comment for 1/2 and will send v3.

~Pratyush

Quoted 14 lines · James Morse

Hi Pratyush,

Quoted 1 line · Pratyush Anand

Any feedback/review comment on it?

I started going through this last week, I hope to get back to it later this week. (I also need to learn/remember how some of the kexec-tools stuff works).

Thanks,

James

#6 lore raw

Re: [PATCH V2 1/2] kexec: arm64: create identity page table to be used in purgatory

In reply to Pratyush Anand

Hi James,

Quoted 7 lines · Pratyush Anand
Quoted 7 lines
Quoted 6 lines
kexec/arch/arm64/kexec-arm64.c +4 -0
+     * in kexec
+     */
+    if (next_tbl_cnt > 5)
+        die("%s: No more memory for page table\n", __func__);
Quoted 6 lines
Quoted 4 lines

die()? With a bit of juggling can't we return an error so we never try to enable
the MMU+dcache instead?

OK. can do that.

It does not seem easy to do that. We will have to revert job of add_buffer_phys_virt() in case of failure, and it will need lot of new code. So, IMHO it would be better to keep die() as of now. What do you say?

~Pratyush