Skip to contentSkip to conversation

xen/arm: Unbreak ACPI

PATCH Superseded Julien Grall 8 participants 55 messages Sep 26, 2020 – Jan 20, 2021

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover xen/arm: Unbreak ACPI

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I have only build tested the x86 side so far.

Cheers,

*** BLURB HERE ***

Julien Grall (4):
xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory() xen/arm: acpi: The fixmap area should always be cleared during
failure/unmap
xen/arm: Check if the platform is not using ACPI before initializing
Dom0less
xen/arm: Introduce fw_unreserved_regions() and use it

 xen/arch/arm/acpi/lib.c     | 79 ++++++++++++++++++++++++++++++-------
 xen/arch/arm/kernel.c       |  2 +-
 xen/arch/arm/setup.c        | 25 +++++++++---
 xen/arch/x86/acpi/lib.c     | 18 +++++++++
 xen/drivers/acpi/osl.c      | 34 ++++++++--------
 xen/include/asm-arm/setup.h |  2 +-
 xen/include/xen/acpi.h      |  1 +
 7 files changed, 123 insertions(+), 38 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/4 xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

From: Julien Grall <[email protected]>

The functions acpi_os_{un,}map_memory() are meant to be arch-agnostic while the __acpi_os_{un,}map_memory() are meant to be arch-specific.

Currently, the former are still containing x86 specific code.

To avoid this rather strange split, the generic helpers are reworked so they are arch-agnostic. This requires the introduction of a new helper __acpi_os_unmap_memory() that will undo any mapping done by __acpi_os_map_memory().

Currently, the arch-helper for unmap is basically a no-op so it only returns whether the mapping was arch specific. But this will change in the future.

Note that the x86 version of acpi_os_map_memory() was already able to able the 1MB region. Hence why there is no addition of new code.

---
 xen/arch/arm/acpi/lib.c | 10 ++++++++++
 xen/arch/x86/acpi/lib.c | 18 ++++++++++++++++++
 xen/drivers/acpi/osl.c  | 34 ++++++++++++++++++----------------
 xen/include/xen/acpi.h  |  1 +
 4 files changed, 47 insertions(+), 16 deletions(-)
xen/arch/arm/acpi/lib.c +10 -0
index 4fc6e17322c1..2192a5519171 100644
@@ -30,6 +30,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
     unsigned long base, offset, mapped_size;
     int idx;
 
+    /* No arch specific implementation after early boot */
+    if ( system_state >= SYS_STATE_boot )
+        return NULL;
+
     offset = phys & (PAGE_SIZE - 1);
     mapped_size = PAGE_SIZE - offset;
     set_fixmap(FIXMAP_ACPI_BEGIN, maddr_to_mfn(phys), PAGE_HYPERVISOR);
@@ -49,6 +53,12 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
     return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
+             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+}
+
 /* True to indicate PSCI 0.2+ is implemented */
 bool __init acpi_psci_present(void)
 {
xen/arch/x86/acpi/lib.c +18 -0
index 265b9ad81905..77803f4d4c63 100644
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	if ((phys + size) <= (1 * 1024 * 1024))
 		return __va(phys);
 
+	/* No arch specific implementation after early boot */
+	if (system_state >= SYS_STATE_boot)
+		return NULL;
+
 	offset = phys & (PAGE_SIZE - 1);
 	mapped_size = PAGE_SIZE - offset;
 	set_fixmap(FIX_ACPI_END, phys);
@@ -66,6 +70,20 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+	unsigned long vaddr = (unsigned long)ptr;
+
+	if (vaddr >= DIRECTMAP_VIRT_START &&
+	    vaddr < DIRECTMAP_VIRT_END) {
+		ASSERT(!((__pa(ptr) + size - 1) >> 20));
+		return true;
+	}
+
+	return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+		(vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));
+}
+
 unsigned int acpi_get_processor_id(unsigned int cpu)
 {
 	unsigned int acpiid, apicid;
xen/drivers/acpi/osl.c +18 -16
index 4c8bb7839eda..100eee72def2 100644
@@ -92,27 +92,29 @@ acpi_physical_address __init acpi_os_get_root_pointer(void)
 void __iomem *
 acpi_os_map_memory(acpi_physical_address phys, acpi_size size)
 {
-	if (system_state >= SYS_STATE_boot) {
-		mfn_t mfn = _mfn(PFN_DOWN(phys));
-		unsigned int offs = phys & (PAGE_SIZE - 1);
-
-		/* The low first Mb is always mapped on x86. */
-		if (IS_ENABLED(CONFIG_X86) && !((phys + size - 1) >> 20))
-			return __va(phys);
-		return __vmap(&mfn, PFN_UP(offs + size), 1, 1,
-			      ACPI_MAP_MEM_ATTR, VMAP_DEFAULT) + offs;
-	}
-	return __acpi_map_table(phys, size);
+	void *ptr;
+	mfn_t mfn = _mfn(PFN_DOWN(phys));
+	unsigned int offs = phys & (PAGE_SIZE - 1);
+
+	/* Try the arch specific implementation first */
+	ptr = __acpi_map_table(phys, size);
+	if (ptr)
+		return ptr;
+
+	/* No common implementation for early boot map */
+	if (unlikely(system_state < SYS_STATE_boot))
+	     return NULL;
+
+	ptr = __vmap(&mfn, PFN_UP(offs + size), 1, 1,
+		     ACPI_MAP_MEM_ATTR, VMAP_DEFAULT);
+
+	return !ptr ? NULL : (ptr + offs);
 }
 
 void acpi_os_unmap_memory(void __iomem * virt, acpi_size size)
 {
-	if (IS_ENABLED(CONFIG_X86) &&
-	    (unsigned long)virt >= DIRECTMAP_VIRT_START &&
-	    (unsigned long)virt < DIRECTMAP_VIRT_END) {
-		ASSERT(!((__pa(virt) + size - 1) >> 20));
+	if (__acpi_unmap_table(virt, size))
 		return;
-	}
 
 	if (system_state >= SYS_STATE_boot)
 		vunmap((void *)((unsigned long)virt & PAGE_MASK));
xen/include/xen/acpi.h +1 -0
index c945ab05c864..5a84a4bf54e0 100644
@@ -68,6 +68,7 @@ typedef int (*acpi_table_entry_handler) (struct acpi_subtable_header *header, co
 
 unsigned int acpi_get_processor_id (unsigned int cpu);
 char * __acpi_map_table (paddr_t phys_addr, unsigned long size);
+bool __acpi_unmap_table(void *ptr, unsigned long size);
 int acpi_boot_init (void);
 int acpi_boot_table_init (void);
 int acpi_numa_init (void);
Signature · 1 line
2.17.1
#2 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

xen/arch/arm/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 59

Quoted 8 lines · Julien Grall
xen/arch/arm/acpi/lib.c +4 -0
@@ -49,6 +53,12 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
     return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
+             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );

Nit: If this wasn't Arm code, I'd ask for the parentheses to be dropped altogether, but at least the stray blanks should go away imo.

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 51

Quoted 9 lines
xen/arch/x86/acpi/lib.c +3 -0
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	if ((phys + size) <= (1 * 1024 * 1024))
 		return __va(phys);
 
+	/* No arch specific implementation after early boot */
+	if (system_state >= SYS_STATE_boot)
+		return NULL;

Considering the code in context above, the comment isn't entirely correct.

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 73

Quoted 5 lines
xen/arch/x86/acpi/lib.c +1 -0
@@ -66,6 +70,20 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)

Is there anything standing in the way of making ptr "const void *"?

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 84

Quoted 11 lines
xen/arch/x86/acpi/lib.c +11 -0
+{
+	unsigned long vaddr = (unsigned long)ptr;
+
+	if (vaddr >= DIRECTMAP_VIRT_START &&
+	    vaddr < DIRECTMAP_VIRT_END) {
+		ASSERT(!((__pa(ptr) + size - 1) >> 20));
+		return true;
+	}
+
+	return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+		(vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));

Indentation is slightly wrong here. Also please consider reducing the number of parentheses here; at the very least the return and the earlier if() want to be consistent in when ones are(n't) used.

Quoted 10 lines
xen/drivers/acpi/osl.c +0 -4
@@ -92,27 +92,29 @@ acpi_physical_address __init acpi_os_get_root_pointer(void)
 void __iomem *
 acpi_os_map_memory(acpi_physical_address phys, acpi_size size)
 {
-	if (system_state >= SYS_STATE_boot) {
-		mfn_t mfn = _mfn(PFN_DOWN(phys));
-		unsigned int offs = phys & (PAGE_SIZE - 1);
-

xen/drivers/acpi/osl.c acpi_physical_address __init acpi_os_get_root_pointer(void) · line 97

Quoted 10 lines
xen/drivers/acpi/osl.c +3 -7
-		/* The low first Mb is always mapped on x86. */
-		if (IS_ENABLED(CONFIG_X86) && !((phys + size - 1) >> 20))
-			return __va(phys);
-		return __vmap(&mfn, PFN_UP(offs + size), 1, 1,
-			      ACPI_MAP_MEM_ATTR, VMAP_DEFAULT) + offs;
-	}
-	return __acpi_map_table(phys, size);
+	void *ptr;
+	mfn_t mfn = _mfn(PFN_DOWN(phys));
+	unsigned int offs = phys & (PAGE_SIZE - 1);

Open-coding PAGE_OFFSET()?

xen/drivers/acpi/osl.c acpi_physical_address __init acpi_os_get_root_pointer(void) · line 106

Quoted 8 lines
xen/drivers/acpi/osl.c +8 -0
+	/* Try the arch specific implementation first */
+	ptr = __acpi_map_table(phys, size);
+	if (ptr)
+		return ptr;
+
+	/* No common implementation for early boot map */
+	if (unlikely(system_state < SYS_STATE_boot))
+	     return NULL;

Consistently hard tabs for indentation here, please.

xen/drivers/acpi/osl.c acpi_physical_address __init acpi_os_get_root_pointer(void) · line 111

Quoted 4 lines
xen/drivers/acpi/osl.c +4 -0
+	ptr = __vmap(&mfn, PFN_UP(offs + size), 1, 1,
+		     ACPI_MAP_MEM_ATTR, VMAP_DEFAULT);
+
+	return !ptr ? NULL : (ptr + offs);

Slightly odd that you don't let the success case go first, the more that it's minimally shorter:

	return ptr ? ptr + offs : NULL;

Jan

#3 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

Hi Jan,

Quoted 5 lines · Jan Beulich
Quoted 4 lines · Julien Grall
xen/arch/arm/acpi/lib.c +0 -0
@@ -49,6 +53,12 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
      return ((char *) base + offset);
  }
 
Quoted 8 lines

xen/arch/arm/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 59

Quoted 4 lines
xen/arch/arm/acpi/lib.c +4 -0
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
+             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );

Nit: If this wasn't Arm code, I'd ask for the parentheses to be dropped altogether, but at least the stray blanks should go away imo.

I will drop the stray blanks.

Quoted 7 lines
Quoted 7 lines
xen/arch/x86/acpi/lib.c +0 -0
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
  	if ((phys + size) <= (1 * 1024 * 1024))
  		return __va(phys);
 
Quoted 6 lines

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 51

Quoted 3 lines
xen/arch/x86/acpi/lib.c +3 -0
+	/* No arch specific implementation after early boot */
+	if (system_state >= SYS_STATE_boot)
+		return NULL;

Considering the code in context above, the comment isn't entirely correct.

How about "No arch specific implementation after early boot but for the first 1MB"?

Quoted 8 lines

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 73

Quoted 6 lines
xen/arch/x86/acpi/lib.c +1 -0
@@ -66,6 +70,20 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
  	return ((char *) base + offset);
  }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)

Is there anything standing in the way of making ptr "const void *"?

It might be possible, I will have a look.

Quoted 5 lines
Quoted 5 lines
xen/arch/x86/acpi/lib.c +5 -0
+{
+	unsigned long vaddr = (unsigned long)ptr;
+
+	if (vaddr >= DIRECTMAP_VIRT_START &&
+	    vaddr < DIRECTMAP_VIRT_END) {
Quoted 8 lines

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 84

Quoted 6 lines
xen/arch/x86/acpi/lib.c +6 -0
+		ASSERT(!((__pa(ptr) + size - 1) >> 20));
+		return true;
+	}
+
+	return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+		(vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));

Indentation is slightly wrong here.

This is Linux coding style and therefore is using hard tab. Care to explain the problem?

Quoted 3 lines

Also please consider reducing
the number of parentheses here; at the very least the return and the earlier if() want to be consistent in when ones are(n't) used.

I will add extra parentheses in the if.

Quoted 15 lines
Quoted 15 lines
xen/drivers/acpi/osl.c +0 -8
@@ -92,27 +92,29 @@ acpi_physical_address __init acpi_os_get_root_pointer(void)
  void __iomem *
  acpi_os_map_memory(acpi_physical_address phys, acpi_size size)
  {
-	if (system_state >= SYS_STATE_boot) {
-		mfn_t mfn = _mfn(PFN_DOWN(phys));
-		unsigned int offs = phys & (PAGE_SIZE - 1);
-
-		/* The low first Mb is always mapped on x86. */
-		if (IS_ENABLED(CONFIG_X86) && !((phys + size - 1) >> 20))
-			return __va(phys);
-		return __vmap(&mfn, PFN_UP(offs + size), 1, 1,
Quoted 8 lines

xen/drivers/acpi/osl.c acpi_physical_address __init acpi_os_get_root_pointer(void) · line 97

Quoted 6 lines
xen/drivers/acpi/osl.c +3 -3
-			      ACPI_MAP_MEM_ATTR, VMAP_DEFAULT) + offs;
-	}
-	return __acpi_map_table(phys, size);
+	void *ptr;
+	mfn_t mfn = _mfn(PFN_DOWN(phys));
+	unsigned int offs = phys & (PAGE_SIZE - 1);

Open-coding PAGE_OFFSET()?

I was looking for a macro to avoid open-coding but I couldn't find it. I will use it in the next version.

Quoted 10 lines

xen/drivers/acpi/osl.c acpi_physical_address __init acpi_os_get_root_pointer(void) · line 106

Quoted 8 lines
xen/drivers/acpi/osl.c +8 -0
+	/* Try the arch specific implementation first */
+	ptr = __acpi_map_table(phys, size);
+	if (ptr)
+		return ptr;
+
+	/* No common implementation for early boot map */
+	if (unlikely(system_state < SYS_STATE_boot))
+	     return NULL;

Consistently hard tabs for indentation here, please.

Will do.

Quoted 6 lines

xen/drivers/acpi/osl.c acpi_physical_address __init acpi_os_get_root_pointer(void) · line 111

Quoted 4 lines
xen/drivers/acpi/osl.c +4 -0
+	ptr = __vmap(&mfn, PFN_UP(offs + size), 1, 1,
+		     ACPI_MAP_MEM_ATTR, VMAP_DEFAULT);
+
+	return !ptr ? NULL : (ptr + offs);

Slightly odd that you don't let the success case go first,

I don't really see the problem. Are you nitpicking?

Quoted 3 lines

the more that it's minimally shorter:

	return ptr ? ptr + offs : NULL;

Cheers,

Signature · 1 line
Julien Grall
#4 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

Quoted 15 lines · Julien Grall
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Julien Grall
xen/arch/x86/acpi/lib.c +0 -0
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
  	if ((phys + size) <= (1 * 1024 * 1024))
  		return __va(phys);
 
Quoted 6 lines

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 51

Quoted 3 lines
xen/arch/x86/acpi/lib.c +3 -0
+	/* No arch specific implementation after early boot */
+	if (system_state >= SYS_STATE_boot)
+		return NULL;

Considering the code in context above, the comment isn't entirely correct.

Quoted 2 lines

How about "No arch specific implementation after early boot but for the first 1MB"?

That or simply "No further ...".

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
xen/arch/x86/acpi/lib.c +5 -0
+{
+	unsigned long vaddr = (unsigned long)ptr;
+
+	if (vaddr >= DIRECTMAP_VIRT_START &&
+	    vaddr < DIRECTMAP_VIRT_END) {
Quoted 8 lines

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 84

Quoted 6 lines
xen/arch/x86/acpi/lib.c +6 -0
+		ASSERT(!((__pa(ptr) + size - 1) >> 20));
+		return true;
+	}
+
+	return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+		(vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));

Indentation is slightly wrong here.

Quoted 2 lines

This is Linux coding style and therefore is using hard tab. Care to explain the problem?

The two opening parentheses should align with one another, shouldn't they?

Quoted 7 lines
Quoted 6 lines

xen/arch/x86/acpi/lib.c

Quoted 4 lines
xen/arch/x86/acpi/lib.c +4 -0
+	ptr = __vmap(&mfn, PFN_UP(offs + size), 1, 1,
+		     ACPI_MAP_MEM_ATTR, VMAP_DEFAULT);
+
+	return !ptr ? NULL : (ptr + offs);

Slightly odd that you don't let the success case go first,

Quoted 5 lines

I don't really see the problem. Are you nitpicking?

Quoted 3 lines

the more that it's minimally shorter:

	return ptr ? ptr + offs : NULL;

Well, I said "slightly odd" as sort of a replacement of "Nit:". But I really think it would be more logical the other way around, not so much for how it looks like, but to aid the not uncommon compiler heuristics of assuming the "true" path is the common one.

Jan

#5 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

Quoted 16 lines · Jan Beulich
Quoted 15 lines · Julien Grall
Quoted 13 lines · Jan Beulich

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 51

Quoted 9 lines · Julien Grall
xen/arch/x86/acpi/lib.c +3 -0
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
   	if ((phys + size) <= (1 * 1024 * 1024))
   		return __va(phys);
 
+	/* No arch specific implementation after early boot */
+	if (system_state >= SYS_STATE_boot)
+		return NULL;

Considering the code in context above, the comment isn't entirely correct.

Quoted 4 lines
Quoted 2 lines

How about "No arch specific implementation after early boot but for the first 1MB"?

That or simply "No further ...".

I will do that.

Quoted 14 lines
Quoted 14 lines
Quoted 13 lines

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 84

Quoted 11 lines
xen/arch/x86/acpi/lib.c +11 -0
+{
+	unsigned long vaddr = (unsigned long)ptr;
+
+	if (vaddr >= DIRECTMAP_VIRT_START &&
+	    vaddr < DIRECTMAP_VIRT_END) {
+		ASSERT(!((__pa(ptr) + size - 1) >> 20));
+		return true;
+	}
+
+	return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+		(vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));

Indentation is slightly wrong here.

Quoted 5 lines
Quoted 2 lines

This is Linux coding style and therefore is using hard tab. Care to explain the problem?

The two opening parentheses should align with one another, shouldn't they?

Hmmm... somehow vim wants to indent this way. I am not entirely sure why...

I will force the indentation manually.

Quoted 12 lines
Quoted 12 lines
Quoted 7 lines

xen/arch/x86/acpi/lib.c

Quoted 5 lines
xen/arch/x86/acpi/lib.c +4 -0
+	ptr = __vmap(&mfn, PFN_UP(offs + size), 1, 1,
+		     ACPI_MAP_MEM_ATTR, VMAP_DEFAULT);
+
+	return !ptr ? NULL : (ptr + offs);

Slightly odd that you don't let the success case go first,

I don't really see the problem. Are you nitpicking?

Quoted 2 lines

the more that it's minimally shorter:

Quoted 7 lines
Quoted 1 line
Quoted 1 line
	return ptr ? ptr + offs : NULL;

Well, I said "slightly odd" as sort of a replacement of "Nit:". But I really think it would be more logical the other way around, not so much for how it looks like, but to aid the not uncommon compiler heuristics of assuming the "true" path is the common one.

Ah, you are trying to outsmart the compilers again...

Cheers,

Signature · 1 line
Julien Grall
#6 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

Hi,

Quoted 40 lines · Julien Grall
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Julien Grall
Quoted 13 lines · Jan Beulich

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned · line 51

Quoted 9 lines · Julien Grall
xen/arch/x86/acpi/lib.c +3 -0
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned 
 long size)
        if ((phys + size) <= (1 * 1024 * 1024))
            return __va(phys);
+    /* No arch specific implementation after early boot */
+    if (system_state >= SYS_STATE_boot)
+        return NULL;

Considering the code in context above, the comment isn't entirely correct.

Quoted 4 lines
Quoted 2 lines

How about "No arch specific implementation after early boot but for the first 1MB"?

That or simply "No further ...".

I will do that.

Quoted 14 lines
Quoted 14 lines
Quoted 13 lines

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 84

Quoted 11 lines
xen/arch/x86/acpi/lib.c +11 -0
+{
+    unsigned long vaddr = (unsigned long)ptr;
+
+    if (vaddr >= DIRECTMAP_VIRT_START &&
+        vaddr < DIRECTMAP_VIRT_END) {
+        ASSERT(!((__pa(ptr) + size - 1) >> 20));
+        return true;
+    }
+
+    return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+        (vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));

Indentation is slightly wrong here.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

This is Linux coding style and therefore is using hard tab. Care to explain the problem?

The two opening parentheses should align with one another, shouldn't they?

Hmmm... somehow vim wants to indent this way. I am not entirely sure why...

Looking at the Linux codebase this is the expected indentation. This is because the first ( on the first line is not closed and until the last ) on the second line.

So I will stick with this code.

Cheers,

Signature · 1 line
Julien Grall
#7 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

Quoted 46 lines · Julien Grall

Hi,

Quoted 43 lines · Julien Grall
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Julien Grall
Quoted 13 lines · Jan Beulich

xen/arch/x86/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned · line 51

Quoted 9 lines · Julien Grall
xen/arch/x86/acpi/lib.c +3 -0
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned 
 long size)
        if ((phys + size) <= (1 * 1024 * 1024))
            return __va(phys);
+    /* No arch specific implementation after early boot */
+    if (system_state >= SYS_STATE_boot)
+        return NULL;

Considering the code in context above, the comment isn't entirely correct.

Quoted 4 lines
Quoted 2 lines

How about "No arch specific implementation after early boot but for the first 1MB"?

That or simply "No further ...".

I will do that.

Quoted 17 lines
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
xen/arch/x86/acpi/lib.c +5 -0
+{
+    unsigned long vaddr = (unsigned long)ptr;
+
+    if (vaddr >= DIRECTMAP_VIRT_START &&
+        vaddr < DIRECTMAP_VIRT_END) {
Quoted 8 lines
Quoted 6 lines

+        ASSERT(!((__pa(ptr) + size - 1) >> 20));
+        return true;
+    }
+
+    return (vaddr >= __fix_to_virt(FIX_ACPI_END)) && +        (vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));

Indentation is slightly wrong here.

This is Linux coding style and therefore is using hard tab. Care to explain the problem?

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

The two opening parentheses should align with one another, shouldn't they?

Hmmm... somehow vim wants to indent this way. I am not entirely sure why...

Looking at the Linux codebase this is the expected indentation. This is because the first ( on the first line is not closed and until the last ) on the second line.

Hrmm... I obviously misread the code I wrote... Sorry for the noise.

Quoted 5 lines

So I will stick with this code.

Cheers,

Signature · 1 line
Julien Grall
Reviewed-by Tested-by on v1 1/4
#8 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

In reply to Julien Grall

Hello,

Quoted 14 lines · Julien Grall

From: Julien Grall <[email protected]>

The functions acpi_os_{un,}map_memory() are meant to be arch-agnostic while the __acpi_os_{un,}map_memory() are meant to be arch-specific.

Currently, the former are still containing x86 specific code.

To avoid this rather strange split, the generic helpers are reworked so they are arch-agnostic. This requires the introduction of a new helper __acpi_os_unmap_memory() that will undo any mapping done by __acpi_os_map_memory().

Quoted 8 lines

Currently, the arch-helper for unmap is basically a no-op so it only returns whether the mapping was arch specific. But this will change in the future.

Note that the x86 version of acpi_os_map_memory() was already able to able the 1MB region. Hence why there is no addition of new code.

Quoted 137 lines

---

xen/arch/arm/acpi/lib.c | 10 ++++++++++
xen/arch/x86/acpi/lib.c | 18 ++++++++++++++++++
xen/drivers/acpi/osl.c  | 34 ++++++++++++++++++----------------
xen/include/xen/acpi.h  |  1 +
4 files changed, 47 insertions(+), 16 deletions(-)
xen/arch/arm/acpi/lib.c +10 -0
index 4fc6e17322c1..2192a5519171 100644
@@ -30,6 +30,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
    unsigned long base, offset, mapped_size;
    int idx;
 
+    /* No arch specific implementation after early boot */
+    if ( system_state >= SYS_STATE_boot )
+        return NULL;
+
    offset = phys & (PAGE_SIZE - 1);
    mapped_size = PAGE_SIZE - offset;
    set_fixmap(FIXMAP_ACPI_BEGIN, maddr_to_mfn(phys), PAGE_HYPERVISOR);
@@ -49,6 +53,12 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
    return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
+             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+}
+
 /* True to indicate PSCI 0.2+ is implemented */
 bool __init acpi_psci_present(void)
 {
xen/arch/x86/acpi/lib.c +18 -0
index 265b9ad81905..77803f4d4c63 100644
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	if ((phys + size) <= (1 * 1024 * 1024))
 		return __va(phys);
 
+	/* No arch specific implementation after early boot */
+	if (system_state >= SYS_STATE_boot)
+		return NULL;
+
 	offset = phys & (PAGE_SIZE - 1);
 	mapped_size = PAGE_SIZE - offset;
 	set_fixmap(FIX_ACPI_END, phys);
@@ -66,6 +70,20 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+	unsigned long vaddr = (unsigned long)ptr;
+
+	if (vaddr >= DIRECTMAP_VIRT_START &&
+	    vaddr < DIRECTMAP_VIRT_END) {
+		ASSERT(!((__pa(ptr) + size - 1) >> 20));
+		return true;
+	}
+
+	return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+		(vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));
+}
+
 unsigned int acpi_get_processor_id(unsigned int cpu)
 {
 	unsigned int acpiid, apicid;
xen/drivers/acpi/osl.c +0 -0
index 4c8bb7839eda..100eee72def2 100644
@@ -92,27 +92,29 @@ acpi_physical_address __init acpi_os_get_root_pointer(void)

void __iomem * acpi_os_map_memory(acpi_physical_address phys, acpi_size size) {

xen/drivers/acpi/osl.c +17 -11
-	if (system_state >= SYS_STATE_boot) {
-		mfn_t mfn = _mfn(PFN_DOWN(phys));
-		unsigned int offs = phys & (PAGE_SIZE - 1);
-
-		/* The low first Mb is always mapped on x86. */
-		if (IS_ENABLED(CONFIG_X86) && !((phys + size - 1) >> 20))
-			return __va(phys);
-		return __vmap(&mfn, PFN_UP(offs + size), 1, 1,
-			      ACPI_MAP_MEM_ATTR, VMAP_DEFAULT) + offs;
-	}
-	return __acpi_map_table(phys, size);
+	void *ptr;
+	mfn_t mfn = _mfn(PFN_DOWN(phys));
+	unsigned int offs = phys & (PAGE_SIZE - 1);
+
+	/* Try the arch specific implementation first */
+	ptr = __acpi_map_table(phys, size);
+	if (ptr)
+		return ptr;
+
+	/* No common implementation for early boot map */
+	if (unlikely(system_state < SYS_STATE_boot))
+	     return NULL;
+
+	ptr = __vmap(&mfn, PFN_UP(offs + size), 1, 1,
+		     ACPI_MAP_MEM_ATTR, VMAP_DEFAULT);
+
+	return !ptr ? NULL : (ptr + offs);

}

void acpi_os_unmap_memory(void __iomem * virt, acpi_size size) {

xen/drivers/acpi/osl.c +1 -4
-	if (IS_ENABLED(CONFIG_X86) &&
-	    (unsigned long)virt >= DIRECTMAP_VIRT_START &&
-	    (unsigned long)virt < DIRECTMAP_VIRT_END) {
-		ASSERT(!((__pa(virt) + size - 1) >> 20));
+	if (__acpi_unmap_table(virt, size))
		return;
xen/drivers/acpi/osl.c +0 -1
-	}
	if (system_state >= SYS_STATE_boot)
		vunmap((void *)((unsigned long)virt & PAGE_MASK));
xen/include/xen/acpi.h +1 -0
index c945ab05c864..5a84a4bf54e0 100644
@@ -68,6 +68,7 @@ typedef int (*acpi_table_entry_handler) (struct acpi_subtable_header *header, co
 
 unsigned int acpi_get_processor_id (unsigned int cpu);
 char * __acpi_map_table (paddr_t phys_addr, unsigned long size);
+bool __acpi_unmap_table(void *ptr, unsigned long size);
 int acpi_boot_init (void);
 int acpi_boot_table_init (void);
 int acpi_numa_init (void);
Signature · 1 line
2.17.1

Regards,
Rahul

#9 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

In reply to Julien Grall
Quoted 43 lines · Julien Grall

From: Julien Grall <[email protected]>

The functions acpi_os_{un,}map_memory() are meant to be arch-agnostic while the __acpi_os_{un,}map_memory() are meant to be arch-specific.

Currently, the former are still containing x86 specific code.

To avoid this rather strange split, the generic helpers are reworked so they are arch-agnostic. This requires the introduction of a new helper __acpi_os_unmap_memory() that will undo any mapping done by __acpi_os_map_memory().

Currently, the arch-helper for unmap is basically a no-op so it only returns whether the mapping was arch specific. But this will change in the future.

Note that the x86 version of acpi_os_map_memory() was already able to able the 1MB region. Hence why there is no addition of new code.

---
 xen/arch/arm/acpi/lib.c | 10 ++++++++++
 xen/arch/x86/acpi/lib.c | 18 ++++++++++++++++++
 xen/drivers/acpi/osl.c  | 34 ++++++++++++++++++----------------
 xen/include/xen/acpi.h  |  1 +
 4 files changed, 47 insertions(+), 16 deletions(-)
xen/arch/arm/acpi/lib.c +4 -0
index 4fc6e17322c1..2192a5519171 100644
@@ -30,6 +30,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
     unsigned long base, offset, mapped_size;
     int idx;
 
+    /* No arch specific implementation after early boot */
+    if ( system_state >= SYS_STATE_boot )
+        return NULL;
+
     offset = phys & (PAGE_SIZE - 1);
     mapped_size = PAGE_SIZE - offset;
     set_fixmap(FIXMAP_ACPI_BEGIN, maddr_to_mfn(phys), PAGE_HYPERVISOR);

xen/arch/arm/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 60

Quoted 8 lines
xen/arch/arm/acpi/lib.c +5 -0
@@ -49,6 +53,12 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
     return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
+             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+}

vaddr or ptr? :-)

lib.c: In function '__acpi_unmap_table':
lib.c:58:14: error: 'vaddr' undeclared (first use in this function)

     return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
              ^

lib.c:58:14: note: each undeclared identifier is reported only once for each function it appears in lib.c:60:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
cc1: all warnings being treated as errors

Quoted 103 lines
xen/arch/arm/acpi/lib.c +0 -0
 /* True to indicate PSCI 0.2+ is implemented */
 bool __init acpi_psci_present(void)
 {
xen/arch/x86/acpi/lib.c +18 -0
index 265b9ad81905..77803f4d4c63 100644
@@ -46,6 +46,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	if ((phys + size) <= (1 * 1024 * 1024))
 		return __va(phys);
 
+	/* No arch specific implementation after early boot */
+	if (system_state >= SYS_STATE_boot)
+		return NULL;
+
 	offset = phys & (PAGE_SIZE - 1);
 	mapped_size = PAGE_SIZE - offset;
 	set_fixmap(FIX_ACPI_END, phys);
@@ -66,6 +70,20 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
 	return ((char *) base + offset);
 }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+	unsigned long vaddr = (unsigned long)ptr;
+
+	if (vaddr >= DIRECTMAP_VIRT_START &&
+	    vaddr < DIRECTMAP_VIRT_END) {
+		ASSERT(!((__pa(ptr) + size - 1) >> 20));
+		return true;
+	}
+
+	return (vaddr >= __fix_to_virt(FIX_ACPI_END)) &&
+		(vaddr < (__fix_to_virt(FIX_ACPI_BEGIN) + PAGE_SIZE));
+}
+
 unsigned int acpi_get_processor_id(unsigned int cpu)
 {
 	unsigned int acpiid, apicid;
xen/drivers/acpi/osl.c +18 -15
index 4c8bb7839eda..100eee72def2 100644
@@ -92,27 +92,29 @@ acpi_physical_address __init acpi_os_get_root_pointer(void)
 void __iomem *
 acpi_os_map_memory(acpi_physical_address phys, acpi_size size)
 {
-	if (system_state >= SYS_STATE_boot) {
-		mfn_t mfn = _mfn(PFN_DOWN(phys));
-		unsigned int offs = phys & (PAGE_SIZE - 1);
-		/* The low first Mb is always mapped on x86. */
-		if (IS_ENABLED(CONFIG_X86) && !((phys + size - 1) >> 20))
-			return __va(phys);
-		return __vmap(&mfn, PFN_UP(offs + size), 1, 1,
-			      ACPI_MAP_MEM_ATTR, VMAP_DEFAULT) + offs;
-	}
-	return __acpi_map_table(phys, size);
+	void *ptr;
+	mfn_t mfn = _mfn(PFN_DOWN(phys));
+	unsigned int offs = phys & (PAGE_SIZE - 1);
+
+	/* Try the arch specific implementation first */
+	ptr = __acpi_map_table(phys, size);
+	if (ptr)
+		return ptr;
+
+	/* No common implementation for early boot map */
+	if (unlikely(system_state < SYS_STATE_boot))
+	     return NULL;
+
+	ptr = __vmap(&mfn, PFN_UP(offs + size), 1, 1,
+		     ACPI_MAP_MEM_ATTR, VMAP_DEFAULT);
+
+	return !ptr ? NULL : (ptr + offs);
 }
 
 void acpi_os_unmap_memory(void __iomem * virt, acpi_size size)
 {
-	if (IS_ENABLED(CONFIG_X86) &&
-	    (unsigned long)virt >= DIRECTMAP_VIRT_START &&
-	    (unsigned long)virt < DIRECTMAP_VIRT_END) {
-		ASSERT(!((__pa(virt) + size - 1) >> 20));
+	if (__acpi_unmap_table(virt, size))
 		return;
-	}
 
 	if (system_state >= SYS_STATE_boot)
 		vunmap((void *)((unsigned long)virt & PAGE_MASK));
xen/include/xen/acpi.h +1 -0
index c945ab05c864..5a84a4bf54e0 100644
@@ -68,6 +68,7 @@ typedef int (*acpi_table_entry_handler) (struct acpi_subtable_header *header, co
 
 unsigned int acpi_get_processor_id (unsigned int cpu);
 char * __acpi_map_table (paddr_t phys_addr, unsigned long size);
+bool __acpi_unmap_table(void *ptr, unsigned long size);
 int acpi_boot_init (void);
 int acpi_boot_table_init (void);
 int acpi_numa_init (void);
Signature · 1 line
2.17.1
#10 lore raw

Re: [PATCH 1/4] xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()

Hi Stefano,

Quoted 57 lines · Stefano Stabellini
Quoted 43 lines · Julien Grall

From: Julien Grall <[email protected]>

The functions acpi_os_{un,}map_memory() are meant to be arch-agnostic while the __acpi_os_{un,}map_memory() are meant to be arch-specific.

Currently, the former are still containing x86 specific code.

To avoid this rather strange split, the generic helpers are reworked so they are arch-agnostic. This requires the introduction of a new helper __acpi_os_unmap_memory() that will undo any mapping done by __acpi_os_map_memory().

Currently, the arch-helper for unmap is basically a no-op so it only returns whether the mapping was arch specific. But this will change in the future.

Note that the x86 version of acpi_os_map_memory() was already able to able the 1MB region. Hence why there is no addition of new code.

---
  xen/arch/arm/acpi/lib.c | 10 ++++++++++
  xen/arch/x86/acpi/lib.c | 18 ++++++++++++++++++
  xen/drivers/acpi/osl.c  | 34 ++++++++++++++++++----------------
  xen/include/xen/acpi.h  |  1 +
  4 files changed, 47 insertions(+), 16 deletions(-)
xen/arch/arm/acpi/lib.c +4 -0
index 4fc6e17322c1..2192a5519171 100644
@@ -30,6 +30,10 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
      unsigned long base, offset, mapped_size;
      int idx;
 
+    /* No arch specific implementation after early boot */
+    if ( system_state >= SYS_STATE_boot )
+        return NULL;
+
      offset = phys & (PAGE_SIZE - 1);
      mapped_size = PAGE_SIZE - offset;
      set_fixmap(FIXMAP_ACPI_BEGIN, maddr_to_mfn(phys), PAGE_HYPERVISOR);

xen/arch/arm/acpi/lib.c char *__acpi_map_table(paddr_t phys, unsigned long size) · line 60

Quoted 8 lines
xen/arch/arm/acpi/lib.c +5 -0
@@ -49,6 +53,12 @@ char *__acpi_map_table(paddr_t phys, unsigned long size)
      return ((char *) base + offset);
  }
 
+bool __acpi_unmap_table(void *ptr, unsigned long size)
+{
+    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
+             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+}

vaddr or ptr? :-)

lib.c: In function '__acpi_unmap_table':
lib.c:58:14: error: 'vaddr' undeclared (first use in this function)

Quoted 7 lines
      return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
               ^

lib.c:58:14: note: each undeclared identifier is reported only once for each function it appears in lib.c:60:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
cc1: all warnings being treated as errors

The whole series builds because 'vaddr' is added in the next patch. But I forgot to build patch by patch :(.

I will fix it in the next version.

Cheers,

Signature · 1 line
Julien Grall
#11 lore raw

PATCH 2/4 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap

In reply to Julien Grall

From: Julien Grall <[email protected]>

Commit 022387ee1ad3 "xen/arm: mm: Don't open-code Xen PT update in {set, clear}_fixmap()" enforced that each set_fixmap() should be paired with a clear_fixmap(). Any failure to follow the model would result to a platform crash.

Unfortunately, the use of fixmap in the ACPI code was overlooked as it is calling set_fixmap() but not clear_fixmap().

The function __acpi_os_map_table() is reworked so:
- We know before the mapping whether the fixmap region is big enough for the mapping.
- It will fail if the fixmap is always inuse.

The function __acpi_os_unmap_table() will now call clear_fixmap().

---

The discussion on the original thread [1] suggested to also zap it on x86. This is technically not necessary today, so it is left alone for now.

I looked at making the fixmap code common but the index are inverted between Arm and x86.

[1] https://lore.kernel.org/xen-devel/[email protected]/

---
 xen/arch/arm/acpi/lib.c | 75 +++++++++++++++++++++++++++++++----------
 1 file changed, 58 insertions(+), 17 deletions(-)
xen/arch/arm/acpi/lib.c +58 -17
index 2192a5519171..eebaca695562 100644
@@ -25,38 +25,79 @@
 #include <xen/init.h>
 #include <xen/mm.h>
 
+static bool fixmap_inuse;
+
 char *__acpi_map_table(paddr_t phys, unsigned long size)
 {
-    unsigned long base, offset, mapped_size;
-    int idx;
+    unsigned long base, offset;
+    mfn_t mfn;
+    unsigned int idx;
 
     /* No arch specific implementation after early boot */
     if ( system_state >= SYS_STATE_boot )
         return NULL;
 
     offset = phys & (PAGE_SIZE - 1);
-    mapped_size = PAGE_SIZE - offset;
-    set_fixmap(FIXMAP_ACPI_BEGIN, maddr_to_mfn(phys), PAGE_HYPERVISOR);
-    base = FIXMAP_ADDR(FIXMAP_ACPI_BEGIN);
+    base = FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) + offset;
+
+    /* Check the fixmap is big enough to map the region */
+    if ( (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - base) < size )
+        return NULL;
+
+    /* With the fixmap, we can only map one region at the time */
+    if ( fixmap_inuse )
+        return NULL;
 
-    /* Most cases can be covered by the below. */
+    fixmap_inuse = true;
+
+    size += offset;
+    mfn = maddr_to_mfn(phys);
     idx = FIXMAP_ACPI_BEGIN;
-    while ( mapped_size < size )
-    {
-        if ( ++idx > FIXMAP_ACPI_END )
-            return NULL;    /* cannot handle this */
-        phys += PAGE_SIZE;
-        set_fixmap(idx, maddr_to_mfn(phys), PAGE_HYPERVISOR);
-        mapped_size += PAGE_SIZE;
-    }
 
-    return ((char *) base + offset);
+    do {
+        set_fixmap(idx, mfn, PAGE_HYPERVISOR);
+        size -= min(size, (unsigned long)PAGE_SIZE);
+        mfn = mfn_add(mfn, 1);
+        idx++;
+    } while ( size > 0 );
+
+    return (char *)base;
 }
 
 bool __acpi_unmap_table(void *ptr, unsigned long size)
 {
-    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
-             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+    vaddr_t vaddr = (vaddr_t)ptr;
+    unsigned int idx;
+
+    /* We are only handling fixmap address in the arch code */
+    if ( vaddr < FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) ||
+         vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_END) )
+        return false;
+
+    /*
+     * __acpi_map_table() will always return a pointer in the first page
+     * for the ACPI fixmap region. The caller is expected to free with
+     * the same address.
+     */
+    ASSERT((vaddr & PAGE_MASK) == FIXMAP_ADDR(FIXMAP_ACPI_BEGIN));
+
+    /* The region allocated fit in the ACPI fixmap region. */
+    ASSERT(size < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - vaddr));
+    ASSERT(fixmap_inuse);
+
+    fixmap_inuse = false;
+
+    size += FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) - vaddr;
+    idx = FIXMAP_ACPI_BEGIN;
+
+    do
+    {
+        clear_fixmap(idx);
+        size -= min(size, (unsigned long)PAGE_SIZE);
+        idx++;
+    } while ( size > 0 );
+
+    return true;
 }
 
 /* True to indicate PSCI 0.2+ is implemented */
Signature · 1 line
2.17.1
Reviewed-by Tested-by on v1 2/4
#12 lore raw

Re: [PATCH 2/4] xen/arm: acpi: The fixmap area should always be cleared during failure/unmap

Hello,

Quoted 17 lines · Julien Grall

From: Julien Grall <[email protected]>

Commit 022387ee1ad3 "xen/arm: mm: Don't open-code Xen PT update in {set, clear}_fixmap()" enforced that each set_fixmap() should be paired with a clear_fixmap(). Any failure to follow the model would result to a platform crash.

Unfortunately, the use of fixmap in the ACPI code was overlooked as it is calling set_fixmap() but not clear_fixmap().

The function __acpi_os_map_table() is reworked so:
- We know before the mapping whether the fixmap region is big enough for the mapping.
- It will fail if the fixmap is always inuse.

Quoted 4 lines

The function __acpi_os_unmap_table() will now call clear_fixmap().

Quoted 120 lines

---

The discussion on the original thread [1] suggested to also zap it on x86. This is technically not necessary today, so it is left alone for now.

I looked at making the fixmap code common but the index are inverted between Arm and x86.

[1] https://lore.kernel.org/xen-devel/[email protected]/
---
xen/arch/arm/acpi/lib.c | 75 +++++++++++++++++++++++++++++++----------

1 file changed, 58 insertions(+), 17 deletions(-)
xen/arch/arm/acpi/lib.c +58 -17
index 2192a5519171..eebaca695562 100644
@@ -25,38 +25,79 @@
 #include <xen/init.h>
 #include <xen/mm.h>
 
+static bool fixmap_inuse;
+
 char *__acpi_map_table(paddr_t phys, unsigned long size)
 {
-    unsigned long base, offset, mapped_size;
-    int idx;
+    unsigned long base, offset;
+    mfn_t mfn;
+    unsigned int idx;
 
    /* No arch specific implementation after early boot */
    if ( system_state >= SYS_STATE_boot )
        return NULL;
 
    offset = phys & (PAGE_SIZE - 1);
-    mapped_size = PAGE_SIZE - offset;
-    set_fixmap(FIXMAP_ACPI_BEGIN, maddr_to_mfn(phys), PAGE_HYPERVISOR);
-    base = FIXMAP_ADDR(FIXMAP_ACPI_BEGIN);
+    base = FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) + offset;
+
+    /* Check the fixmap is big enough to map the region */
+    if ( (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - base) < size )
+        return NULL;
+
+    /* With the fixmap, we can only map one region at the time */
+    if ( fixmap_inuse )
+        return NULL;
 
-    /* Most cases can be covered by the below. */
+    fixmap_inuse = true;
+
+    size += offset;
+    mfn = maddr_to_mfn(phys);
    idx = FIXMAP_ACPI_BEGIN;
-    while ( mapped_size < size )
-    {
-        if ( ++idx > FIXMAP_ACPI_END )
-            return NULL;    /* cannot handle this */
-        phys += PAGE_SIZE;
-        set_fixmap(idx, maddr_to_mfn(phys), PAGE_HYPERVISOR);
-        mapped_size += PAGE_SIZE;
-    }
 
-    return ((char *) base + offset);
+    do {
+        set_fixmap(idx, mfn, PAGE_HYPERVISOR);
+        size -= min(size, (unsigned long)PAGE_SIZE);
+        mfn = mfn_add(mfn, 1);
+        idx++;
+    } while ( size > 0 );
+
+    return (char *)base;
 }
 
 bool __acpi_unmap_table(void *ptr, unsigned long size)
 {
-    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
-             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+    vaddr_t vaddr = (vaddr_t)ptr;
+    unsigned int idx;
+
+    /* We are only handling fixmap address in the arch code */
+    if ( vaddr < FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) ||
+         vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_END) )
+        return false;
+
+    /*
+     * __acpi_map_table() will always return a pointer in the first page
+     * for the ACPI fixmap region. The caller is expected to free with
+     * the same address.
+     */
+    ASSERT((vaddr & PAGE_MASK) == FIXMAP_ADDR(FIXMAP_ACPI_BEGIN));
+
+    /* The region allocated fit in the ACPI fixmap region. */
+    ASSERT(size < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - vaddr));
+    ASSERT(fixmap_inuse);
+
+    fixmap_inuse = false;
+
+    size += FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) - vaddr;
+    idx = FIXMAP_ACPI_BEGIN;
+
+    do
+    {
+        clear_fixmap(idx);
+        size -= min(size, (unsigned long)PAGE_SIZE);
+        idx++;
+    } while ( size > 0 );
+
+    return true;
 }
 
 /* True to indicate PSCI 0.2+ is implemented */
Signature · 1 line
2.17.1

Regards,
Rahul

#13 lore raw

Re: [PATCH 2/4] xen/arm: acpi: The fixmap area should always be cleared during failure/unmap

In reply to Julien Grall
Quoted 7 lines · Julien Grall

From: Julien Grall <[email protected]>

Commit 022387ee1ad3 "xen/arm: mm: Don't open-code Xen PT update in {set, clear}_fixmap()" enforced that each set_fixmap() should be paired with a clear_fixmap(). Any failure to follow the model would result to a platform crash.

Quoted 7 lines

Unfortunately, the use of fixmap in the ACPI code was overlooked as it is calling set_fixmap() but not clear_fixmap().

The function __acpi_os_map_table() is reworked so:
- We know before the mapping whether the fixmap region is big enough for the mapping.
- It will fail if the fixmap is always inuse.

I take you mean "it will fail if the fixmap is *already* in use"?

If so, can it be a problem? Or the expectation is that in practice __acpi_os_map_table() will only get called once before SYS_STATE_boot?

Looking at the code it would seem that even before this patch __acpi_os_map_table() wasn't able to handle multiple calls before SYS_STATE_boot.

Quoted 82 lines

The function __acpi_os_unmap_table() will now call clear_fixmap().

---

The discussion on the original thread [1] suggested to also zap it on x86. This is technically not necessary today, so it is left alone for now.

I looked at making the fixmap code common but the index are inverted between Arm and x86.

[1] https://lore.kernel.org/xen-devel/[email protected]/

---
 xen/arch/arm/acpi/lib.c | 75 +++++++++++++++++++++++++++++++----------
 1 file changed, 58 insertions(+), 17 deletions(-)
xen/arch/arm/acpi/lib.c +26 -15
index 2192a5519171..eebaca695562 100644
@@ -25,38 +25,79 @@
 #include <xen/init.h>
 #include <xen/mm.h>
 
+static bool fixmap_inuse;
+
 char *__acpi_map_table(paddr_t phys, unsigned long size)
 {
-    unsigned long base, offset, mapped_size;
-    int idx;
+    unsigned long base, offset;
+    mfn_t mfn;
+    unsigned int idx;
 
     /* No arch specific implementation after early boot */
     if ( system_state >= SYS_STATE_boot )
         return NULL;
 
     offset = phys & (PAGE_SIZE - 1);
-    mapped_size = PAGE_SIZE - offset;
-    set_fixmap(FIXMAP_ACPI_BEGIN, maddr_to_mfn(phys), PAGE_HYPERVISOR);
-    base = FIXMAP_ADDR(FIXMAP_ACPI_BEGIN);
+    base = FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) + offset;
+
+    /* Check the fixmap is big enough to map the region */
+    if ( (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - base) < size )
+        return NULL;
+
+    /* With the fixmap, we can only map one region at the time */
+    if ( fixmap_inuse )
+        return NULL;
 
-    /* Most cases can be covered by the below. */
+    fixmap_inuse = true;
+
+    size += offset;
+    mfn = maddr_to_mfn(phys);
     idx = FIXMAP_ACPI_BEGIN;
-    while ( mapped_size < size )
-    {
-        if ( ++idx > FIXMAP_ACPI_END )
-            return NULL;    /* cannot handle this */
-        phys += PAGE_SIZE;
-        set_fixmap(idx, maddr_to_mfn(phys), PAGE_HYPERVISOR);
-        mapped_size += PAGE_SIZE;
-    }
 
-    return ((char *) base + offset);
+    do {
+        set_fixmap(idx, mfn, PAGE_HYPERVISOR);
+        size -= min(size, (unsigned long)PAGE_SIZE);
+        mfn = mfn_add(mfn, 1);
+        idx++;
+    } while ( size > 0 );
+
+    return (char *)base;
 }
 

xen/arch/arm/acpi/lib.c line 74

Quoted 10 lines
xen/arch/arm/acpi/lib.c +6 -2
 bool __acpi_unmap_table(void *ptr, unsigned long size)
 {
-    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
-             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+    vaddr_t vaddr = (vaddr_t)ptr;
+    unsigned int idx;
+
+    /* We are only handling fixmap address in the arch code */
+    if ( vaddr < FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) ||
+         vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_END) )

The "+ PAGE_SIZE" got lost

Quoted 6 lines
xen/arch/arm/acpi/lib.c +6 -0
+        return false;
+
+    /*
+     * __acpi_map_table() will always return a pointer in the first page
+     * for the ACPI fixmap region. The caller is expected to free with
+     * the same address.

xen/arch/arm/acpi/lib.c line 90

Quoted 10 lines
xen/arch/arm/acpi/lib.c +10 -0
+     */
+    ASSERT((vaddr & PAGE_MASK) == FIXMAP_ADDR(FIXMAP_ACPI_BEGIN));
+
+    /* The region allocated fit in the ACPI fixmap region. */
+    ASSERT(size < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - vaddr));
+    ASSERT(fixmap_inuse);
+
+    fixmap_inuse = false;
+
+    size += FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) - vaddr;

Sorry I got confused.. Shouldn't this be:

  size += vaddr - FIXMAP_ADDR(FIXMAP_ACPI_BEGIN);

?

Quoted 16 lines
xen/arch/arm/acpi/lib.c +10 -0
+    idx = FIXMAP_ACPI_BEGIN;
+
+    do
+    {
+        clear_fixmap(idx);
+        size -= min(size, (unsigned long)PAGE_SIZE);
+        idx++;
+    } while ( size > 0 );
+
+    return true;
 }
 
 /* True to indicate PSCI 0.2+ is implemented */
Signature · 1 line
2.17.1
#14 lore raw

Re: [PATCH 2/4] xen/arm: acpi: The fixmap area should always be cleared during failure/unmap

Hi Stefano,

Quoted 11 lines · Stefano Stabellini
Quoted 10 lines · Julien Grall

From: Julien Grall <[email protected]>

Commit 022387ee1ad3 "xen/arm: mm: Don't open-code Xen PT update in {set, clear}_fixmap()" enforced that each set_fixmap() should be paired with a clear_fixmap(). Any failure to follow the model would result to a platform crash.

Unfortunately, the use of fixmap in the ACPI code was overlooked as it is calling set_fixmap() but not clear_fixmap().

Quoted 6 lines
Quoted 4 lines

The function __acpi_os_map_table() is reworked so:
- We know before the mapping whether the fixmap region is big enough for the mapping.
- It will fail if the fixmap is always inuse.

I take you mean "it will fail if the fixmap is *already* in use"?

Yes.

Quoted 7 lines

If so, can it be a problem? Or the expectation is that in practice __acpi_os_map_table() will only get called once before SYS_STATE_boot?

Looking at the code it would seem that even before this patch __acpi_os_map_table() wasn't able to handle multiple calls before SYS_STATE_boot.

Correct, I am not changing any expectation here. It is only making clearer because before commit 022387ee1ad3 we would just overwrite the existing mapping with no warning.

After commit 022387ee1ad3, we would just hit the BUG_ON() in set_fixmap().

I will clarify it in the commit message.

[...]

Quoted 11 lines

xen/arch/arm/acpi/lib.c line 74

Quoted 10 lines
  bool __acpi_unmap_table(void *ptr, unsigned long size)
  {
xen/arch/arm/acpi/lib.c +6 -2
-    return ( vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) &&
-             vaddr < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE) );
+    vaddr_t vaddr = (vaddr_t)ptr;
+    unsigned int idx;
+
+    /* We are only handling fixmap address in the arch code */
+    if ( vaddr < FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) ||
+         vaddr >= FIXMAP_ADDR(FIXMAP_ACPI_END) )
Quoted 1 line

The "+ PAGE_SIZE" got lost

Hmmm yes.

Quoted 21 lines
Quoted 8 lines
xen/arch/arm/acpi/lib.c +6 -0
+        return false;
+
+    /*
+     * __acpi_map_table() will always return a pointer in the first page
+     * for the ACPI fixmap region. The caller is expected to free with
+     * the same address.

xen/arch/arm/acpi/lib.c line 90

Quoted 10 lines
xen/arch/arm/acpi/lib.c +10 -0
+     */
+    ASSERT((vaddr & PAGE_MASK) == FIXMAP_ADDR(FIXMAP_ACPI_BEGIN));
+
+    /* The region allocated fit in the ACPI fixmap region. */
+    ASSERT(size < (FIXMAP_ADDR(FIXMAP_ACPI_END) + PAGE_SIZE - vaddr));
+    ASSERT(fixmap_inuse);
+
+    fixmap_inuse = false;
+
+    size += FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) - vaddr;

Sorry I got confused.. Shouldn't this be:

Quoted 3 lines
   size += vaddr - FIXMAP_ADDR(FIXMAP_ACPI_BEGIN);

?

It should be. :) I guess this was unoticed because vaddr == FIXMAP_ADDR(FIXMAP_ACPI_BEGIN) in my testing.

I will fix it.

Quoted 14 lines
Quoted 14 lines
xen/arch/arm/acpi/lib.c +10 -0
+    idx = FIXMAP_ACPI_BEGIN;
+
+    do
+    {
+        clear_fixmap(idx);
+        size -= min(size, (unsigned long)PAGE_SIZE);
+        idx++;
+    } while ( size > 0 );
+
+    return true;
  }
Quoted 4 lines
Quoted 4 lines

/* True to indicate PSCI 0.2+ is implemented */

Signature · 1 line
2.17.1

Cheers,

Signature · 1 line
Julien Grall
#15 lore raw

PATCH 3/4 xen/arm: Check if the platform is not using ACPI before initializing Dom0less

In reply to Julien Grall

From: Julien Grall <[email protected]>

Dom0less requires a device-tree. However, since commit 6e3e77120378 "xen/arm: setup: Relocate the Device-Tree later on in the boot", the device-tree will not get unflatten when using ACPI.

This will lead to a crash during boot.

Given the complexity to setup dom0less with ACPI (for instance how to assign device?), we should skip any code related to Dom0less when using ACPI.

---
 xen/arch/arm/setup.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/arm/setup.c +2 -1
index f16b33fa87a2..35e5bee04efa 100644
@@ -987,7 +987,8 @@ void __init start_xen(unsigned long boot_phys_offset,
 
     system_state = SYS_STATE_active;
 
-    create_domUs();
+    if ( acpi_disabled )
+        create_domUs();
 
     domain_unpause_by_systemcontroller(dom0);
 
Signature · 1 line
2.17.1
Reviewed-by Tested-by on v1 3/4
#16 lore raw

Re: [PATCH 3/4] xen/arm: Check if the platform is not using ACPI before initializing Dom0less

Hello

Quoted 8 lines · Julien Grall

From: Julien Grall <[email protected]>

Dom0less requires a device-tree. However, since commit 6e3e77120378 "xen/arm: setup: Relocate the Device-Tree later on in the boot", the device-tree will not get unflatten when using ACPI.

Quoted 7 lines

This will lead to a crash during boot.

Given the complexity to setup dom0less with ACPI (for instance how to assign device?), we should skip any code related to Dom0less when using ACPI.

Quoted 22 lines

---
xen/arch/arm/setup.c | 3 ++-

1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/arm/setup.c +2 -1
index f16b33fa87a2..35e5bee04efa 100644
@@ -987,7 +987,8 @@ void __init start_xen(unsigned long boot_phys_offset,
 
    system_state = SYS_STATE_active;
 
-    create_domUs();
+    if ( acpi_disabled )
+        create_domUs();
 
    domain_unpause_by_systemcontroller(dom0);
 
Signature · 1 line
2.17.1

Regards,
Rahul

Reviewed-by on v1 3/4
#17 lore raw

Re: [PATCH 3/4] xen/arm: Check if the platform is not using ACPI before initializing Dom0less

In reply to Julien Grall
Quoted 11 lines · Julien Grall

From: Julien Grall <[email protected]>

Dom0less requires a device-tree. However, since commit 6e3e77120378 "xen/arm: setup: Relocate the Device-Tree later on in the boot", the device-tree will not get unflatten when using ACPI.

This will lead to a crash during boot.

Given the complexity to setup dom0less with ACPI (for instance how to assign device?), we should skip any code related to Dom0less when using ACPI.

Yeah...

Quoted 22 lines
---
 xen/arch/arm/setup.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/arm/setup.c +2 -1
index f16b33fa87a2..35e5bee04efa 100644
@@ -987,7 +987,8 @@ void __init start_xen(unsigned long boot_phys_offset,
 
     system_state = SYS_STATE_active;
 
-    create_domUs();
+    if ( acpi_disabled )
+        create_domUs();
 
     domain_unpause_by_systemcontroller(dom0);
 
Signature · 1 line
2.17.1
#18 lore raw

PATCH 4/4 xen/arm: Introduce fw_unreserved_regions() and use it

In reply to Julien Grall

From: Julien Grall <[email protected]>

Since commit 6e3e77120378 "xen/arm: setup: Relocate the Device-Tree later on in the boot", the device-tree will not be kept mapped when using ACPI.

However, a few places are calling dt_unreserved_regions() which expects a valid DT. This will lead to a crash.

As the DT should not be used for ACPI (other than for detecting the modules), a new function fw_unreserved_regions() is introduced.

It will behave the same way on DT system. On ACPI system, it will unreserve the whole region.

---

Is there any region we should exclude on ACPI?

---
 xen/arch/arm/kernel.c       |  2 +-
 xen/arch/arm/setup.c        | 22 +++++++++++++++++-----
 xen/include/asm-arm/setup.h |  2 +-
 3 files changed, 19 insertions(+), 7 deletions(-)
xen/arch/arm/kernel.c +1 -1
index 032923853f2c..ab78689ed2a6 100644
@@ -307,7 +307,7 @@ static __init int kernel_decompress(struct bootmodule *mod)
      * Free the original kernel, update the pointers to the
      * decompressed kernel
      */
-    dt_unreserved_regions(addr, addr + size, init_domheap_pages, 0);
+    fw_unreserved_regions(addr, addr + size, init_domheap_pages, 0);
 
     return 0;
 }
xen/arch/arm/setup.c +17 -5
index 35e5bee04efa..7fcff9af2a7e 100644
@@ -196,8 +196,9 @@ static void __init processor_id(void)
     processor_setup();
 }
 
-void __init dt_unreserved_regions(paddr_t s, paddr_t e,
-                                  void (*cb)(paddr_t, paddr_t), int first)
+static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
+                                         void (*cb)(paddr_t, paddr_t),
+                                         int first)
 {
     int i, nr = fdt_num_mem_rsv(device_tree_flattened);
 
@@ -244,6 +245,17 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t e,
     cb(s, e);
 }
 
+void __init fw_unreserved_regions(paddr_t s, paddr_t e,
+                                  void (*cb)(paddr_t, paddr_t), int first)
+{
+    if ( acpi_disabled )
+        dt_unreserved_regions(s, e, cb, first);
+    else
+        cb(s, e);
+}
+
+
+
 struct bootmodule __init *add_boot_module(bootmodule_kind kind,
                                           paddr_t start, paddr_t size,
                                           bool domU)
@@ -405,7 +417,7 @@ void __init discard_initial_modules(void)
              !mfn_valid(maddr_to_mfn(e)) )
             continue;
 
-        dt_unreserved_regions(s, e, init_domheap_pages, 0);
+        fw_unreserved_regions(s, e, init_domheap_pages, 0);
     }
 
     mi->nr_mods = 0;
@@ -712,7 +724,7 @@ static void __init setup_mm(void)
                 n = mfn_to_maddr(mfn_add(xenheap_mfn_start, xenheap_pages));
             }
 
-            dt_unreserved_regions(s, e, init_boot_pages, 0);
+            fw_unreserved_regions(s, e, init_boot_pages, 0);
 
             s = n;
         }
@@ -765,7 +777,7 @@ static void __init setup_mm(void)
             if ( e > bank_end )
                 e = bank_end;
 
-            dt_unreserved_regions(s, e, init_boot_pages, 0);
+            fw_unreserved_regions(s, e, init_boot_pages, 0);
             s = n;
         }
     }
xen/include/asm-arm/setup.h +1 -1
index 2f8f24e286ed..34df23247b87 100644
@@ -96,7 +96,7 @@ int construct_dom0(struct domain *d);
 void create_domUs(void);
 
 void discard_initial_modules(void);
-void dt_unreserved_regions(paddr_t s, paddr_t e,
+void fw_unreserved_regions(paddr_t s, paddr_t e,
                            void (*cb)(paddr_t, paddr_t), int first);
 
 size_t boot_fdt_info(const void *fdt, paddr_t paddr);
Signature · 1 line
2.17.1
Reviewed-by on v1 4/4
#19 lore raw

Re: [PATCH 4/4] xen/arm: Introduce fw_unreserved_regions() and use it

Quoted 6 lines · Julien Grall

From: Julien Grall <[email protected]>

Since commit 6e3e77120378 "xen/arm: setup: Relocate the Device-Tree later on in the boot", the device-tree will not be kept mapped when using ACPI.

Quoted 8 lines

However, a few places are calling dt_unreserved_regions() which expects a valid DT. This will lead to a crash.

As the DT should not be used for ACPI (other than for detecting the modules), a new function fw_unreserved_regions() is introduced.

It will behave the same way on DT system. On ACPI system, it will unreserve the whole region.

The patch is good.

I have a small suggestion for improvement that could be done on commit:
given that bootinfo is actually used on EFI systems (granted, not bootinfo.reserved_mem but bootinfo.mem, see xen/arch/arm/efi/efi-boot.h:efi_process_memory_map_bootinfo) so technically bootinfo could be in-use with ACPI, maybe we could add a comment on top of xen/include/asm-arm/setup.h:bootinfo to say that reserved_mem is device tree only?

Quoted 101 lines

---

Is there any region we should exclude on ACPI?

---
 xen/arch/arm/kernel.c       |  2 +-
 xen/arch/arm/setup.c        | 22 +++++++++++++++++-----
 xen/include/asm-arm/setup.h |  2 +-
 3 files changed, 19 insertions(+), 7 deletions(-)
xen/arch/arm/kernel.c +1 -1
index 032923853f2c..ab78689ed2a6 100644
@@ -307,7 +307,7 @@ static __init int kernel_decompress(struct bootmodule *mod)
      * Free the original kernel, update the pointers to the
      * decompressed kernel
      */
-    dt_unreserved_regions(addr, addr + size, init_domheap_pages, 0);
+    fw_unreserved_regions(addr, addr + size, init_domheap_pages, 0);
 
     return 0;
 }
xen/arch/arm/setup.c +17 -5
index 35e5bee04efa..7fcff9af2a7e 100644
@@ -196,8 +196,9 @@ static void __init processor_id(void)
     processor_setup();
 }
 
-void __init dt_unreserved_regions(paddr_t s, paddr_t e,
-                                  void (*cb)(paddr_t, paddr_t), int first)
+static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
+                                         void (*cb)(paddr_t, paddr_t),
+                                         int first)
 {
     int i, nr = fdt_num_mem_rsv(device_tree_flattened);
 
@@ -244,6 +245,17 @@ void __init dt_unreserved_regions(paddr_t s, paddr_t e,
     cb(s, e);
 }
 
+void __init fw_unreserved_regions(paddr_t s, paddr_t e,
+                                  void (*cb)(paddr_t, paddr_t), int first)
+{
+    if ( acpi_disabled )
+        dt_unreserved_regions(s, e, cb, first);
+    else
+        cb(s, e);
+}
+
+
+
 struct bootmodule __init *add_boot_module(bootmodule_kind kind,
                                           paddr_t start, paddr_t size,
                                           bool domU)
@@ -405,7 +417,7 @@ void __init discard_initial_modules(void)
              !mfn_valid(maddr_to_mfn(e)) )
             continue;
 
-        dt_unreserved_regions(s, e, init_domheap_pages, 0);
+        fw_unreserved_regions(s, e, init_domheap_pages, 0);
     }
 
     mi->nr_mods = 0;
@@ -712,7 +724,7 @@ static void __init setup_mm(void)
                 n = mfn_to_maddr(mfn_add(xenheap_mfn_start, xenheap_pages));
             }
 
-            dt_unreserved_regions(s, e, init_boot_pages, 0);
+            fw_unreserved_regions(s, e, init_boot_pages, 0);
 
             s = n;
         }
@@ -765,7 +777,7 @@ static void __init setup_mm(void)
             if ( e > bank_end )
                 e = bank_end;
 
-            dt_unreserved_regions(s, e, init_boot_pages, 0);
+            fw_unreserved_regions(s, e, init_boot_pages, 0);
             s = n;
         }
     }
xen/include/asm-arm/setup.h +1 -1
index 2f8f24e286ed..34df23247b87 100644
@@ -96,7 +96,7 @@ int construct_dom0(struct domain *d);
 void create_domUs(void);
 
 void discard_initial_modules(void);
-void dt_unreserved_regions(paddr_t s, paddr_t e,
+void fw_unreserved_regions(paddr_t s, paddr_t e,
                            void (*cb)(paddr_t, paddr_t), int first);
 
 size_t boot_fdt_info(const void *fdt, paddr_t paddr);
Signature · 1 line
2.17.1
#20 lore raw

Re: [PATCH 4/4] xen/arm: Introduce fw_unreserved_regions() and use it

Hi Stefano,

Quoted 8 lines · Stefano Stabellini

On Sat, 26 Sep 2020, Julien Grall wrote:
I have a small suggestion for improvement that could be done on commit:
given that bootinfo is actually used on EFI systems (granted, not bootinfo.reserved_mem but bootinfo.mem, see xen/arch/arm/efi/efi-boot.h:efi_process_memory_map_bootinfo) so technically bootinfo could be in-use with ACPI, maybe we could add a comment on top of xen/include/asm-arm/setup.h:bootinfo to say that reserved_mem is device tree only?

That's fine with me. I will need to resend the rest of the series, so I will update it at the same time.

Cheers,

Signature · 1 line
Julien Grall
#21 lore raw
In reply to Julien Grall
Quoted 7 lines · Julien Grall

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I have only build tested the x86 side so far.

On the ARM device I'm trying to get operational the boot got distinctly further along so appears to be a distinct ACPI improvement here.

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#22 lore raw
Quoted 2 lines · Elliott Mitchell

On the ARM device I'm trying to get operational the boot got distinctly further along so appears to be a distinct ACPI improvement here.

Just in case it wasn't clear, that does amount to:

Now to look over a different set of shoulders and find out whether the next problem is what they've been running into, or a distinct one...

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#23 lore raw
In reply to Julien Grall

Hello,

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

Thank you,

2020年9月27日(日) 5:56 Julien Grall <[email protected]>:

Quoted 34 lines

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I have only build tested the x86 side so far.

Cheers,

*** BLURB HERE ***

Julien Grall (4):
xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory() xen/arm: acpi: The fixmap area should always be cleared during
failure/unmap
xen/arm: Check if the platform is not using ACPI before initializing
Dom0less
xen/arm: Introduce fw_unreserved_regions() and use it

 xen/arch/arm/acpi/lib.c     | 79 ++++++++++++++++++++++++++++++-------
 xen/arch/arm/kernel.c       |  2 +-
 xen/arch/arm/setup.c        | 25 +++++++++---
 xen/arch/x86/acpi/lib.c     | 18 +++++++++
 xen/drivers/acpi/osl.c      | 34 ++++++++--------
 xen/include/asm-arm/setup.h |  2 +-
 xen/include/xen/acpi.h      |  1 +
 7 files changed, 123 insertions(+), 38 deletions(-)
Quoted 3 lines

--
2.17.1

--
Masami Hiramatsu

#24 lore raw

Hi,

I've missed the explanation of the attached patch. This prototype patch was also needed for booting up the Xen on my box (for the system which has no SPCR).

Thank you,

2020年9月28日(月) 15:47 Masami Hiramatsu <[email protected]>:

Quoted 52 lines

Hello,

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

Thank you,

2020年9月27日(日) 5:56 Julien Grall <[email protected]>:

Quoted 34 lines

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I have only build tested the x86 side so far.

Cheers,

*** BLURB HERE ***

Julien Grall (4):
xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory() xen/arm: acpi: The fixmap area should always be cleared during
failure/unmap
xen/arm: Check if the platform is not using ACPI before initializing
Dom0less
xen/arm: Introduce fw_unreserved_regions() and use it

 xen/arch/arm/acpi/lib.c     | 79 ++++++++++++++++++++++++++++++-------
 xen/arch/arm/kernel.c       |  2 +-
 xen/arch/arm/setup.c        | 25 +++++++++---
 xen/arch/x86/acpi/lib.c     | 18 +++++++++
 xen/drivers/acpi/osl.c      | 34 ++++++++--------
 xen/include/asm-arm/setup.h |  2 +-
 xen/include/xen/acpi.h      |  1 +
 7 files changed, 123 insertions(+), 38 deletions(-)
Quoted 3 lines

--
2.17.1

--
Masami Hiramatsu

Signature · 1 line
Masami Hiramatsu
#25 lore raw

Xen-ARM EFI/ACPI problems (was: Re: [PATCH 0/4] xen/arm: Unbreak ACPI)

Quoted 3 lines · Masami Hiramatsu

I've missed the explanation of the attached patch. This prototype patch was also needed for booting up the Xen on my box (for the system which has no SPCR).

I'm pretty sure of this on the hardware I'm dealing with, but don't know about the hardware you're dealing with.

Does your device have a framebuffer? Have you ever tried/succeeded booting your device with the framebuffer disabled? (booted with a HDMI/DVI cable disconnected or the device on the other end *completely* powered down)

Based upon the back and forth both on xen-devel and some messages which were split off due to not being general. An observation I had made a while ago finally caused me to try recreating it.

On the device I'm on (Raspberry PI 4B with Tianocore -> GRUB -> Xen) I discovered a SPCR table shows up if I boot with the device the output is plugged into is powered down. I'm guessing this causes Tianocore to advise GRUB/Linux/Xen to boot with a serial console (presenting a Serial Port Console Redirect table), whereas if the display device is functioning the absense of SPCR is supposed to indicate console on framebuffer.

This means the ACPI_FAILURE case in acpi_iomem_deny_access() simply needs to be filled in similar to how it likely is on x86. Allocate a serial port for Xen's use as console, present it to domain 0 as hvc0, and hide it from domain 0.

Next issue for me will be getting the framebuffer operational. Apparently the Xen-ARM EFI implementation doesn't provide any EFI variables to domain 0? Jan Beulich, your name was mentioned as person likely to have ideas for getting Linux's efifb code operational Xen-ARM.

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#26 lore raw

Re: Xen-ARM EFI/ACPI problems (was: Re: [PATCH 0/4] xen/arm: Unbreak ACPI)

Quoted 8 lines · Elliott Mitchell

On the device I'm on (Raspberry PI 4B with Tianocore -> GRUB -> Xen) I discovered a SPCR table shows up if I boot with the device the output is plugged into is powered down. I'm guessing this causes Tianocore to advise GRUB/Linux/Xen to boot with a serial console (presenting a Serial Port Console Redirect table), whereas if the display device is functioning the absense of SPCR is supposed to indicate console on framebuffer.

Quoted 4 lines

This means the ACPI_FAILURE case in acpi_iomem_deny_access() simply needs to be filled in similar to how it likely is on x86. Allocate a serial port for Xen's use as console, present it to domain 0 as hvc0, and hide it from domain 0.

Looks like things are worse than I thought.

Upon examining some of my dmesg copies it looks like Linux interprets the ignore_uart field in STAO tables as applying strictly to the UART referenced in the SPCR table. As such, when booted with the framebuffer available, Linux thinks it can freely access the UART found in the hardware table. The specification for the STAO table is apparently garbage since it only allows a hypervisor to tell a VM to ignore a single UART. Instead it really needs to be possible to mask arbitrary devices. :-(

As such, for ARM devices which can include framebuffers, I'm guessing Xen will need to either pass a modified table to domain 0, or simulate the device sufficiently to prevent concurrent access. This could be as simple as simulating a MMIO page which discards all writes.

Quoted 4 lines

Next issue for me will be getting the framebuffer operational. Apparently the Xen-ARM EFI implementation doesn't provide any EFI variables to domain 0? Jan Beulich, your name was mentioned as person likely to have ideas for getting Linux's efifb code operational Xen-ARM.

There may be multiple pieces to this.

For the framebuffer this might be as simple as parsing the BGRT table and ensuring the addresses are directly mapped to domain 0. I just noticed in dmesg, "efi_bgrt: Ignoring BGRT: invalid image address". I'm guessing one thing got remapped, but a second didn't.

The other need for EFI variable access is for modifying EFI's boot process. While I suspect it may be feasible for most users to reboot to a kernel directly on hardware to update GRUB/other bootloader, adding an extra step increases the potential for a failure at the Wrong Time.

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#27 lore raw
In reply to Masami Hiramatsu
Quoted 2 lines · Masami Hiramatsu

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

Adding your patch on top of Julien Grall's patch appears to push the Xen boot of my target device (Raspberry PI 4B with Tianocore) further. Still yet to see any output attributable to the Domain 0 kernel though.

(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading d0 kernel from boot module @ 0000000032234000
(XEN) Loading ramdisk from boot module @ 0000000031747000
(XEN) Allocating 1:1 mappings totalling 2048MB for dom0:
(XEN) BANK[0] 0x00000020000000-0x00000030000000 (256MB)
(XEN) BANK[1] 0x00000040000000-0x000000b0000000 (1792MB)
(XEN) Grant table range: 0x000000315f3000-0x00000031633000
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading zImage from 0000000032234000 to 0000000020080000-0000000021359200 (XEN) Loading d0 initrd from 0000000031747000 to 0x0000000028200000-0x0000000028cebfe4 (XEN) Loading d0 DTB to 0x0000000028000000-0x0000000028000273
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) Freed 396kB init memory.

The line, "Loading d0 DTB to 0x0000000028000000-0x0000000028000273" seems rather suspicious as I would expect Domain 0 to see ACPI tables, not a device tree.

Your (Masami Hiramatsu) patch seems plausible, but things haven't progressed enough for me to endorse it. Looks like something closer to the core of ACPI still needs further work, Julien Grall?

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#28 lore raw

Hello,

Quoted 26 lines · Elliott Mitchell
Quoted 2 lines · Masami Hiramatsu

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

Adding your patch on top of Julien Grall's patch appears to push the Xen boot of my target device (Raspberry PI 4B with Tianocore) further. Still yet to see any output attributable to the Domain 0 kernel though.

(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading d0 kernel from boot module @ 0000000032234000
(XEN) Loading ramdisk from boot module @ 0000000031747000
(XEN) Allocating 1:1 mappings totalling 2048MB for dom0:
(XEN) BANK[0] 0x00000020000000-0x00000030000000 (256MB)
(XEN) BANK[1] 0x00000040000000-0x000000b0000000 (1792MB)
(XEN) Grant table range: 0x000000315f3000-0x00000031633000
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading zImage from 0000000032234000 to 0000000020080000-0000000021359200 (XEN) Loading d0 initrd from 0000000031747000 to 0x0000000028200000-0x0000000028cebfe4 (XEN) Loading d0 DTB to 0x0000000028000000-0x0000000028000273
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) Freed 396kB init memory.

Quoted 3 lines

The line, "Loading d0 DTB to 0x0000000028000000-0x0000000028000273" seems rather suspicious as I would expect Domain 0 to see ACPI tables, not a device tree.

This is normal, we are creating a small device-tree to pass some information to dom0 (such as the ACPI tables, command line, initrd).

Quoted 4 lines

Your (Masami Hiramatsu) patch seems plausible, but things haven't progressed enough for me to endorse it. Looks like something closer to the core of ACPI still needs further work, Julien Grall?

I didn't go very far during my testing because QEMU is providing ACPI 5.1 (Xen only supports 6.0+ so far).

For your log above, Xen finished to boot and now dom0 should start booting. The lack of console output may be because of a crash in Linux during earlyboot.

Do you have the early console enabled Linux? This can be done by adding earlycon=xenboot on the Linux command line.

Cheers,

Signature · 1 line
Julien Grall
#29 lore raw
Quoted 8 lines · Julien Grall
Quoted 3 lines · Elliott Mitchell

Your (Masami Hiramatsu) patch seems plausible, but things haven't progressed enough for me to endorse it. Looks like something closer to the core of ACPI still needs further work, Julien Grall?

I didn't go very far during my testing because QEMU is providing ACPI 5.1 (Xen only supports 6.0+ so far).

Quoted 6 lines

For your log above, Xen finished to boot and now dom0 should start booting. The lack of console output may be because of a crash in Linux during earlyboot.

Do you have the early console enabled Linux? This can be done by adding earlycon=xenboot on the Linux command line.

Finding all the command-line console settings can be a challenge. I had thought it was supposed to be "console=hvc0 earlycon=hvc0".

With that though I finally have some output which claims to come from the Linux kernel (yay! finally hit this point!). As we were both guessing, very early kernel panic:

[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: Can't find 'System Table' in device tree!
[    0.000000] cma: Failed to reserve 64 MiB
[    0.000000] Kernel panic - not syncing: Failed to allocate page table page

I don't know whether this is a problem with the mini-DT which was passed in versus ACPI tables. I note a complete lack of ACPI table information. The kernel is from a 5.6-based kernel tree. I'm unsure which portion to try updating next.

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#30 lore raw

Hi Elliott,

Quoted 19 lines · Elliott Mitchell
Quoted 8 lines · Julien Grall
Quoted 3 lines · Elliott Mitchell

Your (Masami Hiramatsu) patch seems plausible, but things haven't progressed enough for me to endorse it. Looks like something closer to the core of ACPI still needs further work, Julien Grall?

I didn't go very far during my testing because QEMU is providing ACPI 5.1 (Xen only supports 6.0+ so far).

Quoted 6 lines

For your log above, Xen finished to boot and now dom0 should start booting. The lack of console output may be because of a crash in Linux during earlyboot.

Do you have the early console enabled Linux? This can be done by adding earlycon=xenboot on the Linux command line.

Finding all the command-line console settings can be a challenge. I had thought it was supposed to be "console=hvc0 earlycon=hvc0".

Quoted 6 lines

With that though I finally have some output which claims to come from the Linux kernel (yay! finally hit this point!). As we were both guessing, very early kernel panic:

[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: Can't find 'System Table' in device tree!

Thank you for sending part of the log. Looking at Linux 5.6 code, the error message is printed by efi_get_fdt_params() (see drivers/firmware/efi.c) when one of the property is missing.

'System Table' suggests that Linux wasn't enable to find "linux,uefi-system-table" or "xen,uefi-system-table".

Xen will only create later. Would it be possible to add some code in __find_uefi_params() to confirm which property Linux thinks is missing?

Quoted 5 lines
[    0.000000] cma: Failed to reserve 64 MiB
[    0.000000] Kernel panic - not syncing: Failed to allocate page table page

I don't know whether this is a problem with the mini-DT which was passed in versus ACPI tables. I note a complete lack of ACPI table information.

I think this is normal because, IIRC, the ACPI root pointer will be found using the System Table.

Quoted 4 lines

The kernel is from a 5.6-based kernel tree. I'm unsure which portion to try updating next.

Cheers,

Signature · 1 line
Julien Grall
#31 lore raw
Quoted 15 lines · Julien Grall
Quoted 9 lines · Elliott Mitchell

Finding all the command-line console settings can be a challenge. I had thought it was supposed to be "console=hvc0 earlycon=hvc0".

With that though I finally have some output which claims to come from the Linux kernel (yay! finally hit this point!). As we were both guessing, very early kernel panic:

[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: Can't find 'System Table' in device tree!

Thank you for sending part of the log. Looking at Linux 5.6 code, the error message is printed by efi_get_fdt_params() (see drivers/firmware/efi.c) when one of the property is missing.

Quoted 5 lines

'System Table' suggests that Linux wasn't enable to find "linux,uefi-system-table" or "xen,uefi-system-table".

Xen will only create later. Would it be possible to add some code in __find_uefi_params() to confirm which property Linux thinks is missing?

Trying to rebuild a configuration long after the prior build can be entertaining. Finally identified one patch appears to be for 4.19, but breaks 5.x. With that and an appropriate adjustment:

[    0.000000] efi: __find_uefi_params(): Missing "linux,uefi-system-table"
[    0.000000] efi: __find_uefi_params(): Missing "linux,uefi-system-table"

Good news is I had been meaning to try a later kernel for a while and thus tried building a 5.8 kernel. Once built, the 5.8 kernel booted successfully. As such the next issue is the driver for the graphics chip doesn't function under Xen as of the 5.8 kernel and Xen 4.14.

I can now state I've gotten success with a combination of Julien Grall's patches and Masami Hiramatsu. Certainly qualifies for:

For all 5 patches.

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#32 lore raw
In reply to Masami Hiramatsu
Quoted 2 lines · Masami Hiramatsu

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

Now things have progressed a bit more and I can confirm the patch provides useful progress. I cannot say whether there is a better way since I'm not familiar enough with the code.

As such, for both Masami Hiramatsu's "Hide UART address only if STAO..." and Julien Grall's set of four:

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#33 lore raw
In reply to Masami Hiramatsu
Quoted 1 line · Masami Hiramatsu

Hello,

Hi Masami,

Quoted 3 lines

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

I have reviewed the patch attached and I have a couple of remarks about it.

The STAO table was originally created to allow an hypervisor to hide devices from a controller domain (such as Dom0). If this table is not present, then it means the OS/hypervisor can use any device listed in the ACPI table.

Additionally, the STAO table should never be present in the host ACPI table.

Therefore, I think the code should not try to find the STAO. Instead, it should check whether the SPCR table is present.

Cheers,

Signature · 1 line
Julien Grall
#34 lore raw
Quoted 11 lines · Julien Grall
Quoted 1 line · Masami Hiramatsu

Hello,

Hi Masami,

Quoted 3 lines

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

I have reviewed the patch attached and I have a couple of remarks about it.

Quoted 8 lines

The STAO table was originally created to allow an hypervisor to hide devices from a controller domain (such as Dom0). If this table is not present, then it means the OS/hypervisor can use any device listed in the ACPI table.

Additionally, the STAO table should never be present in the host ACPI table.

Therefore, I think the code should not try to find the STAO. Instead, it should check whether the SPCR table is present.

Yes, that makes sense, but that brings me to the next question.

SPCR seems to be required by SBBR, however, Masami wrote that he could boot on a system without SPCR, which gets me very confused for two reasons:

1) Why there is no SPCR? Isn't it supposed to be mandatory? Is it because there no UART on Masami's system?

2) If there is no SPCR, how did Masami manage to boot Xen? I take without any serial output? Just with the framebuffer?

#35 lore raw
Quoted 4 lines · Stefano Stabellini
Quoted 2 lines · Julien Grall

Therefore, I think the code should not try to find the STAO. Instead, it should check whether the SPCR table is present.

Quoted 8 lines

Yes, that makes sense, but that brings me to the next question.

SPCR seems to be required by SBBR, however, Masami wrote that he could boot on a system without SPCR, which gets me very confused for two reasons:

1) Why there is no SPCR? Isn't it supposed to be mandatory? Is it because there no UART on Masami's system?

I'm on different hardware, but some folks have setup Tianocore for it. According to Documentation/arm64/acpi_object_usage.rst, "Required: DSDT, FADT, GTDT, MADT, MCFG, RSDP, SPCR, XSDT". Yet when booting a Linux kernel directly on the hardware it lists APIC, BGRT, CSRT, DSDT, DBG2, FACP, GTDT, PPTT, RSDP, and XSDT.

I don't know whether Linux's ACPI code omits mention of some required tables and merely panics if they're absent. Yet I'm speculating the list of required tables has shrunk, SPCR is no longer required, and the documentation is out of date. Perhaps SPCR was required in early Linux ACPI implementations, but more recent ones removed that requirement?

Quoted 2 lines

2) If there is no SPCR, how did Masami manage to boot Xen? I take without any serial output? Just with the framebuffer?

On my board the provided tables are sufficient to let Linux identify ttyAMA0. Linux's efifb driver finds the framebuffer and apparently has it usable.

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#36 lore raw
Quoted 20 lines · Elliott Mitchell
Quoted 4 lines · Stefano Stabellini
Quoted 2 lines · Julien Grall

Therefore, I think the code should not try to find the STAO. Instead, it should check whether the SPCR table is present.

Quoted 8 lines

Yes, that makes sense, but that brings me to the next question.

SPCR seems to be required by SBBR, however, Masami wrote that he could boot on a system without SPCR, which gets me very confused for two reasons:

1) Why there is no SPCR? Isn't it supposed to be mandatory? Is it because there no UART on Masami's system?

I'm on different hardware, but some folks have setup Tianocore for it. According to Documentation/arm64/acpi_object_usage.rst, "Required: DSDT, FADT, GTDT, MADT, MCFG, RSDP, SPCR, XSDT". Yet when booting a Linux kernel directly on the hardware it lists APIC, BGRT, CSRT, DSDT, DBG2, FACP, GTDT, PPTT, RSDP, and XSDT.

Quoted 5 lines

I don't know whether Linux's ACPI code omits mention of some required tables and merely panics if they're absent. Yet I'm speculating the list of required tables has shrunk, SPCR is no longer required, and the documentation is out of date. Perhaps SPCR was required in early Linux ACPI implementations, but more recent ones removed that requirement?

I have just checked and SPCR is still a mandatory table in the latest SBBR specification. It is probably one of those cases where the firmware claims to be SBBR compliant, but it is not, and it happens to work with Linux.

#37 lore raw
Quoted 13 lines · Stefano Stabellini
Quoted 11 lines · Elliott Mitchell

I'm on different hardware, but some folks have setup Tianocore for it. According to Documentation/arm64/acpi_object_usage.rst, "Required: DSDT, FADT, GTDT, MADT, MCFG, RSDP, SPCR, XSDT". Yet when booting a Linux kernel directly on the hardware it lists APIC, BGRT, CSRT, DSDT, DBG2, FACP, GTDT, PPTT, RSDP, and XSDT.

I don't know whether Linux's ACPI code omits mention of some required tables and merely panics if they're absent. Yet I'm speculating the list of required tables has shrunk, SPCR is no longer required, and the documentation is out of date. Perhaps SPCR was required in early Linux ACPI implementations, but more recent ones removed that requirement?

Quoted 4 lines

I have just checked and SPCR is still a mandatory table in the latest SBBR specification. It is probably one of those cases where the firmware claims to be SBBR compliant, but it is not, and it happens to work with Linux.

Is meeting the SBBR specification supposed to be a requirement of running Xen-ARM?

I don't seen any mention of such.
find docs xen/arch/arm -type f -print0 | xargs -0 grep -eSBBR produces no output.

Perhaps you've been adding this as a presumptive requirement since previously the only hardware capable of running Xen due to an appropriately unlocked bootloader was SBBR compliant? If so, it seems time to either add this as an explicit requirement and document it, or else remove this implicit requirement and start acting as such.

The Raspberry PI 4B has a UEFI implementation available which is based on Tianocore. No statement has been made of it qualifying as SBBR. Yet it is clearly mostly able to boot Xen, just this is exposing issues.

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#38 lore raw

Hi Elliot,

Quoted 14 lines · Elliott Mitchell
Quoted 13 lines · Stefano Stabellini
Quoted 11 lines · Elliott Mitchell

I'm on different hardware, but some folks have setup Tianocore for it. According to Documentation/arm64/acpi_object_usage.rst, "Required: DSDT, FADT, GTDT, MADT, MCFG, RSDP, SPCR, XSDT". Yet when booting a Linux kernel directly on the hardware it lists APIC, BGRT, CSRT, DSDT, DBG2, FACP, GTDT, PPTT, RSDP, and XSDT.

I don't know whether Linux's ACPI code omits mention of some required tables and merely panics if they're absent. Yet I'm speculating the list of required tables has shrunk, SPCR is no longer required, and the documentation is out of date. Perhaps SPCR was required in early Linux ACPI implementations, but more recent ones removed that requirement?

Quoted 7 lines
Quoted 4 lines

I have just checked and SPCR is still a mandatory table in the latest SBBR specification. It is probably one of those cases where the firmware claims to be SBBR compliant, but it is not, and it happens to work with Linux.

Is meeting the SBBR specification supposed to be a requirement of running Xen-ARM?

This is not my goal. We should try to get Xen running everywhere as long as this doesn't require a lot of extra code. IOW, don't ask me to review/accept a port of Xen to RPI3 ;).

Cheers,

Signature · 1 line
Julien Grall
#39 lore raw
Quoted 23 lines · Julien Grall

Hi Elliot,

Quoted 20 lines · Elliott Mitchell
Quoted 18 lines · Stefano Stabellini
Quoted 6 lines · Elliott Mitchell

I'm on different hardware, but some folks have setup Tianocore for it. According to Documentation/arm64/acpi_object_usage.rst, "Required: DSDT, FADT, GTDT, MADT, MCFG, RSDP, SPCR, XSDT". Yet when booting a Linux kernel directly on the hardware it lists APIC, BGRT, CSRT, DSDT, DBG2, FACP, GTDT, PPTT, RSDP, and XSDT.

Quoted 6 lines

I don't know whether Linux's ACPI code omits mention of some required tables and merely panics if they're absent. Yet I'm speculating the list
of required tables has shrunk, SPCR is no longer required, and the documentation is out of date. Perhaps SPCR was required in early Linux ACPI implementations, but more recent ones removed that requirement?

I have just checked and SPCR is still a mandatory table in the latest SBBR specification. It is probably one of those cases where the firmware claims to be SBBR compliant, but it is not, and it happens to work with Linux.

Quoted 6 lines
Quoted 2 lines

Is meeting the SBBR specification supposed to be a requirement of running Xen-ARM?

This is not my goal. We should try to get Xen running everywhere as long as this doesn't require a lot of extra code. IOW, don't ask me to review/accept a port of Xen to RPI3 ;).

I agree with Julien's statement.

For context, my previous comment in regards to SBBR was because I am positive that Masami's platform is expected to be SBBR compliant.

#40 lore raw
In reply to Stefano Stabellini

Hi,

Quoted 21 lines · Stefano Stabellini
Quoted 11 lines · Julien Grall
Quoted 1 line · Masami Hiramatsu

Hello,

Hi Masami,

Quoted 3 lines

This made progress with my Xen boot on DeveloperBox (
https://www.96boards.org/product/developerbox/ ) with ACPI.

I have reviewed the patch attached and I have a couple of remarks about it.

Quoted 8 lines

The STAO table was originally created to allow an hypervisor to hide devices from a controller domain (such as Dom0). If this table is not present, then it means the OS/hypervisor can use any device listed in the ACPI table.

Additionally, the STAO table should never be present in the host ACPI table.

Therefore, I think the code should not try to find the STAO. Instead, it should check whether the SPCR table is present.

Quoted 8 lines

Yes, that makes sense, but that brings me to the next question.

SPCR seems to be required by SBBR, however, Masami wrote that he could boot on a system without SPCR, which gets me very confused for two reasons:

1) Why there is no SPCR? Isn't it supposed to be mandatory? Is it because there no UART on Masami's system?

I can't comment specically on Masami's system, but I can make two broad comments:
1) Not all the systems have to be compliant with the SBBR. But in
theory, only systems passing the SBBR test can be claimed to be complained. This brings to the second point...
2) "Mandatory" is what the specs aim to enforce. In my experience,
some of the vendors may bend the rule and still claim they are compliant.

Even Linux documentation says that the SPCR is mandatory. But in the reality, the implementation will just ignore it.

From my understanding of the code, Linux will only use it to discover the preferred console and enable earlycon. Without it, Linux will require the user to specify console=<...> or earlycon=<...>.

The UART will then be discovered via the DSDT.

Quoted 3 lines

2) If there is no SPCR, how did Masami manage to boot Xen? I take without any serial output? Just with the framebuffer?

After his patch, Xen can boot without the SPCR. You will just see no logs. But I belive, he enabled earlyprintk for his platform.

Cheers,

Signature · 1 line
Julien Grall
#41 lore raw

PATCH 1/1 xen: acpi: Hide UART address only if SPCR exists

In reply to Masami Hiramatsu

Hi,

With Julien's ACPI fix, I found my DeveloperBox (
https://www.96boards.org/product/developerbox/ ) still failed to boot bcause of SPCR issue. According to the UEFI EDK2 implementation, the SCPR table will not be made if user chooses graphical console instead of serial.
In such case, we should ignore the SPCR missing error.

Compared with Linux kernel implementation, this doesn't check the STAO (status override) table, becaue it seems STAO (or XENV) will be provided by Xen itself.
Or, should we check STAO too?

Thank you,

---

Masami Hiramatsu (1):
xen: acpi: Hide UART address only if SPCR exists

 xen/arch/arm/acpi/domain_build.c |   19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

--
Masami Hiramatsu (Linaro) <[email protected]>

#42 lore raw

Re: [PATCH] xen: acpi: Hide UART address only if SPCR exists

Quoted 6 lines · Masami Hiramatsu

With Julien's ACPI fix, I found my DeveloperBox (
https://www.96boards.org/product/developerbox/ ) still failed to boot bcause of SPCR issue. According to the UEFI EDK2 implementation, the SCPR table will not be made if user chooses graphical console instead of serial.
In such case, we should ignore the SPCR missing error.

You weren't the only person running into this. Since you didn't get back sooner I ended up submitting a pretty similar patch (I kept the printk() due to informational value).

861f0c110976fa8879b7bf63d9478b6be83d4ab6 on master (switching to "main" in the future?)

Quoted 4 lines

Compared with Linux kernel implementation, this doesn't check the STAO (status override) table, becaue it seems STAO (or XENV) will be provided by Xen itself.
Or, should we check STAO too?

Until we start seeing super-duper-visors Xen won't need to look for STAO. Linux needs to look for STAO since it expects to potentially run under a hypervisor, whereas Xen this isn't expected.

Graphics are a problem due to limits of ACPI table support. Due to graphics being a high priority for me, I'm temporarily using device-trees. :-(

Signature · 4 lines
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         [email protected]  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445
#43 lore raw

Re: [PATCH] xen: acpi: Hide UART address only if SPCR exists

Hi,

Quoted 11 lines · Elliott Mitchell
Quoted 6 lines · Masami Hiramatsu

With Julien's ACPI fix, I found my DeveloperBox (
https://www.96boards.org/product/developerbox/ ) still failed to boot bcause of SPCR issue. According to the UEFI EDK2 implementation, the SCPR table will not be made if user chooses graphical console instead of serial.
In such case, we should ignore the SPCR missing error.

You weren't the only person running into this. Since you didn't get back sooner I ended up submitting a pretty similar patch (I kept the printk() due to informational value).

Right, a cut-down version of the patch submitted has been merged.

@Masami, I believe all the ACPI patches for Arm went in. Would you mind to try the latest staging branch?

Quoted 3 lines

861f0c110976fa8879b7bf63d9478b6be83d4ab6 on master (switching to "main" in the future?)

There was a discussion about it a few months ago. The last input we were waiting on github to make the change and adopt the same name.

Feel free to give a nudge on the thread [1].

Cheers,

[1] <[email protected]>

Signature · 1 line
Julien Grall
#44 lore raw

PATCH 1/1 xen: acpi: Hide UART address only if SPCR exists

In reply to Masami Hiramatsu

Since there is a case that UEFI (EDK2) doesn't make the SPCR table when it is booted with graphic console, ignore it if SPCR doesn't exist.

---
 xen/arch/arm/acpi/domain_build.c |   19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)
xen/arch/arm/acpi/domain_build.c +8 -11
index 1b1cfabb00..a50adbb9ab 100644
@@ -40,20 +40,17 @@ static int __init acpi_iomem_deny_access(struct domain *d)
 
     /* TODO: Deny MMIO access for SMMU, GIC ITS */
     status = acpi_get_table(ACPI_SIG_SPCR, 0,
-                            (struct acpi_table_header **)&spcr);
-
-    if ( ACPI_FAILURE(status) )
+                        (struct acpi_table_header **)&spcr);
+    /* SPCR may not set when the UEFI uses graphical console. */
+    if ( ACPI_SUCCESS(status) )
     {
-        printk("Failed to get SPCR table\n");
-        return -EINVAL;
+        mfn = spcr->serial_port.address >> PAGE_SHIFT;
+        /* Deny MMIO access for UART */
+        rc = iomem_deny_access(d, mfn, mfn + 1);
+        if ( rc )
+            return rc;
     }
 
-    mfn = spcr->serial_port.address >> PAGE_SHIFT;
-    /* Deny MMIO access for UART */
-    rc = iomem_deny_access(d, mfn, mfn + 1);
-    if ( rc )
-        return rc;
-
     /* Deny MMIO access for GIC regions */
     return gic_iomem_deny_access(d);
 }
#45 lore raw
In reply to Julien Grall

Hello,

Quoted 8 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

We also observed the panic after enabling the ACPI for ARM N1SDP board earlier.

(XEN) Xen call trace:

(XEN)    [<0000000000271b24>] set_fixmap+0x28/0x2c (PC)
(XEN)    [<0000000000271b18>] set_fixmap+0x1c/0x2c (LR)
(XEN)    [<0000000000267d58>] __acpi_map_table+0x38/0x94
(XEN)    [<00000000002501a4>] acpi_os_map_memory+0x18/0x64

After applying this patch series panic is fixed and XEN is able to boot with ACPI enabled.

Quoted 31 lines

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I have only build tested the x86 side so far.

Cheers,

*** BLURB HERE ***

Julien Grall (4):
xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory() xen/arm: acpi: The fixmap area should always be cleared during
failure/unmap
xen/arm: Check if the platform is not using ACPI before initializing
Dom0less
xen/arm: Introduce fw_unreserved_regions() and use it

xen/arch/arm/acpi/lib.c     | 79 ++++++++++++++++++++++++++++++-------
xen/arch/arm/kernel.c       |  2 +-
xen/arch/arm/setup.c        | 25 +++++++++---
xen/arch/x86/acpi/lib.c     | 18 +++++++++
xen/drivers/acpi/osl.c      | 34 ++++++++--------
xen/include/asm-arm/setup.h |  2 +-
xen/include/xen/acpi.h      |  1 +
7 files changed, 123 insertions(+), 38 deletions(-)
Signature · 1 line
2.17.1
#46 lore raw
In reply to Julien Grall
Quoted 9 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

Loading Xen 4.15-unstable ...
Loading Linux 4.19.0-11-arm64 ...
Loading initial ramdisk ...
Using modules provided by bootloader in FDT
Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader ...silence...

I have a grub installed from testing on a buster base:

  dpkg --status grub-arm64-efi
  Version: 2.04-8

With:

GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_CMDLINE_LINUX="console=ttyAMA0" GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen" GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

Quoted 23 lines

I have only build tested the x86 side so far.

Cheers,

*** BLURB HERE ***

Julien Grall (4):
xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory() xen/arm: acpi: The fixmap area should always be cleared during
failure/unmap
xen/arm: Check if the platform is not using ACPI before initializing
Dom0less
xen/arm: Introduce fw_unreserved_regions() and use it

 xen/arch/arm/acpi/lib.c     | 79 ++++++++++++++++++++++++++++++-------
 xen/arch/arm/kernel.c       |  2 +-
 xen/arch/arm/setup.c        | 25 +++++++++---
 xen/arch/x86/acpi/lib.c     | 18 +++++++++
 xen/drivers/acpi/osl.c      | 34 ++++++++--------
 xen/include/asm-arm/setup.h |  2 +-
 xen/include/xen/acpi.h      |  1 +
 7 files changed, 123 insertions(+), 38 deletions(-)
Signature · 1 line
Alex Bennée
#47 lore raw

Hi Alex,

Quoted 35 lines · Alex Bennée
Quoted 9 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

   Loading Xen 4.15-unstable ...
   Loading Linux 4.19.0-11-arm64 ...
   Loading initial ramdisk ...
   Using modules provided by bootloader in FDT
   Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader
   ...silence...

I have a grub installed from testing on a buster base:

   dpkg --status grub-arm64-efi
   Version: 2.04-8

With:

   GRUB_CMDLINE_LINUX_DEFAULT=""
   GRUB_CMDLINE_LINUX="console=ttyAMA0"
   GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen"
   GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"
Quoted 6 lines

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

The error is hapenning before Xen setup the console. You can get early output on QEMU if you rebuild Xen with the following .config options:

CONFIG_DEBUG=y
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x09000000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_PRINTK_INC="debug-pl011.inc"

Cheers,

Signature · 1 line
Julien Grall
#48 lore raw
Quoted 48 lines · Julien Grall

Hi Alex,

Quoted 35 lines · Alex Bennée
Quoted 9 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

   Loading Xen 4.15-unstable ...
   Loading Linux 4.19.0-11-arm64 ...
   Loading initial ramdisk ...
   Using modules provided by bootloader in FDT
   Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader
   ...silence...

I have a grub installed from testing on a buster base:

   dpkg --status grub-arm64-efi
   Version: 2.04-8

With:

   GRUB_CMDLINE_LINUX_DEFAULT=""
   GRUB_CMDLINE_LINUX="console=ttyAMA0"
   GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen"
   GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"
Quoted 6 lines

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

The error is hapenning before Xen setup the console. You can get early output on QEMU if you rebuild Xen with the following .config options:

Quoted 7 lines

CONFIG_DEBUG=y
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x09000000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_PRINTK_INC="debug-pl011.inc"

OK I can see it fails on the ACPI and then tries to fall back to FDT and then fails to find the GIC:

(XEN) CMDLINE[00000000f7bbe000]:chosen placeholder root=UUID=cf00cd3a-066b-4146-bedf-f811d3343077 ro console=hvc0 earlyprintk=xen (XEN)
(XEN) Command line: placeholder loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg no-real-mode edd=off (XEN) parameter "placeholder" unknown!
(XEN) parameter "no-real-mode" unknown!
(XEN) parameter "edd" unknown!
(XEN) ACPI: RSDP 138560000, 0024 (r2 BOCHS )

  (XEN) ACPI: XSDT 138550000, 004C (r1 BOCHS  BXPCFACP        1       1000013)
  (XEN) ACPI: FACP 138510000, 010C (r5 BOCHS  BXPCFACP        1 BXPC        1)
  (XEN) ACPI: DSDT 138520000, 14A6 (r2 BOCHS  BXPCDSDT        1 BXPC        1)
  (XEN) ACPI: APIC 138500000, 018C (r3 BOCHS  BXPCAPIC        1 BXPC        1)
  (XEN) ACPI: GTDT 1384F0000, 0060 (r2 BOCHS  BXPCGTDT        1 BXPC        1)
  (XEN) ACPI: MCFG 1384E0000, 003C (r1 BOCHS  BXPCMCFG        1 BXPC        1)
  (XEN) ACPI: SPCR 1384D0000, 0050 (r2 BOCHS  BXPCSPCR        1 BXPC        1)

(XEN) Unsupported FADT revision 5.1, should be 6.0+, will disable ACPI (XEN) acpi_boot_table_init: FADT not found (-22)
(XEN) Domain heap initialised
(XEN) Booting using Device Tree
(XEN) Platform: Generic System
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Unable to find compatible GIC in the device tree (XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

Despite saying it is going to reboot it never manages to. Any idea how it is trying to reset the system?

Signature · 1 line
Alex Bennée
#49 lore raw

On 29/09/2020 22:11, Alex Bennée wrote:

Hi,

Quoted 10 lines
Quoted 8 lines · Julien Grall

Hi Alex,

Quoted 5 lines · Alex Bennée
Quoted 2 lines · Julien Grall

From: Julien Grall <[email protected]>

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1).

Does QEMU provide ACPI tables? Or is this actually EDK2 generating them?

Quoted 1 line

So I did only some light testing.

So about that v6.0 or later: it seems like the requirement comes from:
commit 1c9bd43019cd "arm/acpi: Parse FADT table and get PSCI flags":
"Since STAO table and the GIC version are introduced by ACPI 6.0, we will check the version and only parse FADT table with version >= 6.0. If firmware provides ACPI tables with ACPI version less than 6.0, OS will be messed up with those information, so disable ACPI if we get an FADT table with version less than 6.0."

STAO (and XENV) have indeed been introduced by ACPI v6.0, but those tables are supposed to be *generated* by Xen and handed down to Dom0, they will never be provided by firmware (or parsed) by the Xen hypervisor. Checking the Linux code it seems to actually not (yet?) support the STAO named list, and currently finds and uses the STAO (UART block bit) regardless of the FADT version number.

I can't find anything GIC related in the FADT, the whole GIC information is described in MADT.

Linux/arm64 seems to be happy with ACPI >= v5.1:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kernel/acpi.c#n148

So can we relax the v6.0 requirement, to be actually >= v5.1? That is among the first to support ARM anyway, IIRC.

I have only a smattering about ACPI, so happy to stand corrected.

Cheers,
Andre

Quoted 74 lines
Quoted 33 lines
Quoted 23 lines

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

   Loading Xen 4.15-unstable ...
   Loading Linux 4.19.0-11-arm64 ...
   Loading initial ramdisk ...
   Using modules provided by bootloader in FDT
   Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader
   ...silence...

I have a grub installed from testing on a buster base:

   dpkg --status grub-arm64-efi
   Version: 2.04-8

With:

   GRUB_CMDLINE_LINUX_DEFAULT=""
   GRUB_CMDLINE_LINUX="console=ttyAMA0"
   GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen"
   GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"
Quoted 6 lines

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

The error is hapenning before Xen setup the console. You can get early output on QEMU if you rebuild Xen with the following .config options:

Quoted 7 lines

CONFIG_DEBUG=y
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x09000000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_PRINTK_INC="debug-pl011.inc"

OK I can see it fails on the ACPI and then tries to fall back to FDT and then fails to find the GIC:

(XEN) CMDLINE[00000000f7bbe000]:chosen placeholder root=UUID=cf00cd3a-066b-4146-bedf-f811d3343077 ro console=hvc0 earlyprintk=xen (XEN)
(XEN) Command line: placeholder loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg no-real-mode edd=off (XEN) parameter "placeholder" unknown!
(XEN) parameter "no-real-mode" unknown!
(XEN) parameter "edd" unknown!
(XEN) ACPI: RSDP 138560000, 0024 (r2 BOCHS )

  (XEN) ACPI: XSDT 138550000, 004C (r1 BOCHS  BXPCFACP        1       1000013)
  (XEN) ACPI: FACP 138510000, 010C (r5 BOCHS  BXPCFACP        1 BXPC        1)
  (XEN) ACPI: DSDT 138520000, 14A6 (r2 BOCHS  BXPCDSDT        1 BXPC        1)
  (XEN) ACPI: APIC 138500000, 018C (r3 BOCHS  BXPCAPIC        1 BXPC        1)
  (XEN) ACPI: GTDT 1384F0000, 0060 (r2 BOCHS  BXPCGTDT        1 BXPC        1)
  (XEN) ACPI: MCFG 1384E0000, 003C (r1 BOCHS  BXPCMCFG        1 BXPC        1)
  (XEN) ACPI: SPCR 1384D0000, 0050 (r2 BOCHS  BXPCSPCR        1 BXPC        1)

(XEN) Unsupported FADT revision 5.1, should be 6.0+, will disable ACPI (XEN) acpi_boot_table_init: FADT not found (-22)
(XEN) Domain heap initialised
(XEN) Booting using Device Tree
(XEN) Platform: Generic System
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Unable to find compatible GIC in the device tree (XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

Despite saying it is going to reboot it never manages to. Any idea how it is trying to reset the system?

#50 lore raw
Quoted 16 lines · André Przywara

On 29/09/2020 22:11, Alex Bennée wrote:

Hi,

Quoted 12 lines
Quoted 10 lines · Julien Grall

Hi Alex,

Quoted 7 lines · Alex Bennée
Quoted 4 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1).

Does QEMU provide ACPI tables? Or is this actually EDK2 generating them?

It certainly does - whether EDK2 mangles them afterwards is another question. For the -M virt machine we currently generate:

  /* FADT */
  static void build_fadt_rev5(GArray *table_data, BIOSLinker *linker,
                              VirtMachineState *vms, unsigned dsdt_tbl_offset)
  {
      /* ACPI v5.1 */
      AcpiFadtData fadt = {
          .rev = 5,
          .minor_ver = 1,
          .flags = 1 << ACPI_FADT_F_HW_REDUCED_ACPI,
          .xdsdt_tbl_offset = &dsdt_tbl_offset,
      };
      switch (vms->psci_conduit) {
      case QEMU_PSCI_CONDUIT_DISABLED:
          fadt.arm_boot_arch = 0;
          break;
      case QEMU_PSCI_CONDUIT_HVC:
          fadt.arm_boot_arch = ACPI_FADT_ARM_PSCI_COMPLIANT |
                               ACPI_FADT_ARM_PSCI_USE_HVC;
          break;
      case QEMU_PSCI_CONDUIT_SMC:
          fadt.arm_boot_arch = ACPI_FADT_ARM_PSCI_COMPLIANT;
          break;
      default:
          g_assert_not_reached();
      }

      build_fadt(table_data, linker, &fadt, NULL, NULL);

}

Looking through the code it seems we stop around 5.0 - even the new fancy x86 microvm ACPI tables are described as /* ACPI 5.0: 4.1 Hardware-Reduced ACPI */.

Quoted 18 lines
Quoted 2 lines

So I did only some light testing.

So about that v6.0 or later: it seems like the requirement comes from:
commit 1c9bd43019cd "arm/acpi: Parse FADT table and get PSCI flags":
"Since STAO table and the GIC version are introduced by ACPI 6.0, we will check the version and only parse FADT table with version >= 6.0. If firmware provides ACPI tables with ACPI version less than 6.0, OS will be messed up with those information, so disable ACPI if we get an FADT table with version less than 6.0."

STAO (and XENV) have indeed been introduced by ACPI v6.0, but those tables are supposed to be *generated* by Xen and handed down to Dom0, they will never be provided by firmware (or parsed) by the Xen hypervisor. Checking the Linux code it seems to actually not (yet?) support the STAO named list, and currently finds and uses the STAO (UART block bit) regardless of the FADT version number.

Quoted 2 lines

I can't find anything GIC related in the FADT, the whole GIC information is described in MADT.

I think we are generating full data for both GIC2 and GIC3:

  /* MADT */
  static void
  build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
  {
      VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms);
      int madt_start = table_data->len;
      const MemMapEntry *memmap = vms->memmap;
      const int *irqmap = vms->irqmap;
      AcpiMadtGenericDistributor *gicd;
      AcpiMadtGenericMsiFrame *gic_msi;
      int i;
      acpi_data_push(table_data, sizeof(AcpiMultipleApicTable));

      gicd = acpi_data_push(table_data, sizeof *gicd);
      gicd->type = ACPI_APIC_GENERIC_DISTRIBUTOR;
      gicd->length = sizeof(*gicd);
      gicd->base_address = cpu_to_le64(memmap[VIRT_GIC_DIST].base);
      gicd->version = vms->gic_version;

      for (i = 0; i < vms->smp_cpus; i++) {
          AcpiMadtGenericCpuInterface *gicc = acpi_data_push(table_data,
                                                             sizeof(*gicc));
          ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(i));

          gicc->type = ACPI_APIC_GENERIC_CPU_INTERFACE;
          gicc->length = sizeof(*gicc);
          if (vms->gic_version == 2) {
              gicc->base_address = cpu_to_le64(memmap[VIRT_GIC_CPU].base);
              gicc->gich_base_address = cpu_to_le64(memmap[VIRT_GIC_HYP].base);
              gicc->gicv_base_address = cpu_to_le64(memmap[VIRT_GIC_VCPU].base);
          }
          gicc->cpu_interface_number = cpu_to_le32(i);
          gicc->arm_mpidr = cpu_to_le64(armcpu->mp_affinity);
          gicc->uid = cpu_to_le32(i);
          gicc->flags = cpu_to_le32(ACPI_MADT_GICC_ENABLED);

          if (arm_feature(&armcpu->env, ARM_FEATURE_PMU)) {
              gicc->performance_interrupt = cpu_to_le32(PPI(VIRTUAL_PMU_IRQ));
          }
          if (vms->virt) {
              gicc->vgic_interrupt = cpu_to_le32(PPI(ARCH_GIC_MAINT_IRQ));
          }
      }

      if (vms->gic_version == 3) {
          AcpiMadtGenericTranslator *gic_its;
          int nb_redist_regions = virt_gicv3_redist_region_count(vms);
          AcpiMadtGenericRedistributor *gicr = acpi_data_push(table_data,
                                                           sizeof *gicr);

          gicr->type = ACPI_APIC_GENERIC_REDISTRIBUTOR;
          gicr->length = sizeof(*gicr);
          gicr->base_address = cpu_to_le64(memmap[VIRT_GIC_REDIST].base);
          gicr->range_length = cpu_to_le32(memmap[VIRT_GIC_REDIST].size);

          if (nb_redist_regions == 2) {
              gicr = acpi_data_push(table_data, sizeof(*gicr));
              gicr->type = ACPI_APIC_GENERIC_REDISTRIBUTOR;
              gicr->length = sizeof(*gicr);
              gicr->base_address =
                  cpu_to_le64(memmap[VIRT_HIGH_GIC_REDIST2].base);
              gicr->range_length =
                  cpu_to_le32(memmap[VIRT_HIGH_GIC_REDIST2].size);
          }

          if (its_class_name() && !vmc->no_its) {
              gic_its = acpi_data_push(table_data, sizeof *gic_its);
              gic_its->type = ACPI_APIC_GENERIC_TRANSLATOR;
              gic_its->length = sizeof(*gic_its);
              gic_its->translation_id = 0;
              gic_its->base_address = cpu_to_le64(memmap[VIRT_GIC_ITS].base);
          }
      } else {
          gic_msi = acpi_data_push(table_data, sizeof *gic_msi);
          gic_msi->type = ACPI_APIC_GENERIC_MSI_FRAME;
          gic_msi->length = sizeof(*gic_msi);
          gic_msi->gic_msi_frame_id = 0;
          gic_msi->base_address = cpu_to_le64(memmap[VIRT_GIC_V2M].base);
          gic_msi->flags = cpu_to_le32(1);
          gic_msi->spi_count = cpu_to_le16(NUM_GICV2M_SPIS);
          gic_msi->spi_base = cpu_to_le16(irqmap[VIRT_GIC_V2M] + ARM_SPI_BASE);
      }

      build_header(linker, table_data,
                   (void *)(table_data->data + madt_start), "APIC",
                   table_data->len - madt_start, 3, NULL, NULL);

}

Quoted 8 lines

Linux/arm64 seems to be happy with ACPI >= v5.1:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kernel/acpi.c#n148

So can we relax the v6.0 requirement, to be actually >= v5.1? That is among the first to support ARM anyway, IIRC.

I have only a smattering about ACPI, so happy to stand corrected.

I'd certainly be happy with that as reduces the number of components that need to be uprevved to get support.

Quoted 78 lines

Cheers,
Andre

Quoted 74 lines
Quoted 33 lines
Quoted 23 lines

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

   Loading Xen 4.15-unstable ...
   Loading Linux 4.19.0-11-arm64 ...
   Loading initial ramdisk ...
   Using modules provided by bootloader in FDT
   Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader
   ...silence...

I have a grub installed from testing on a buster base:

   dpkg --status grub-arm64-efi
   Version: 2.04-8

With:

   GRUB_CMDLINE_LINUX_DEFAULT=""
   GRUB_CMDLINE_LINUX="console=ttyAMA0"
   GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen"
   GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"
Quoted 6 lines

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

The error is hapenning before Xen setup the console. You can get early output on QEMU if you rebuild Xen with the following .config options:

Quoted 7 lines

CONFIG_DEBUG=y
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x09000000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_PRINTK_INC="debug-pl011.inc"

OK I can see it fails on the ACPI and then tries to fall back to FDT and then fails to find the GIC:

(XEN) CMDLINE[00000000f7bbe000]:chosen placeholder root=UUID=cf00cd3a-066b-4146-bedf-f811d3343077 ro console=hvc0 earlyprintk=xen (XEN)
(XEN) Command line: placeholder loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg no-real-mode edd=off (XEN) parameter "placeholder" unknown!
(XEN) parameter "no-real-mode" unknown!
(XEN) parameter "edd" unknown!
(XEN) ACPI: RSDP 138560000, 0024 (r2 BOCHS )

  (XEN) ACPI: XSDT 138550000, 004C (r1 BOCHS  BXPCFACP        1       1000013)
  (XEN) ACPI: FACP 138510000, 010C (r5 BOCHS  BXPCFACP        1 BXPC        1)
  (XEN) ACPI: DSDT 138520000, 14A6 (r2 BOCHS  BXPCDSDT        1 BXPC        1)
  (XEN) ACPI: APIC 138500000, 018C (r3 BOCHS  BXPCAPIC        1 BXPC        1)
  (XEN) ACPI: GTDT 1384F0000, 0060 (r2 BOCHS  BXPCGTDT        1 BXPC        1)
  (XEN) ACPI: MCFG 1384E0000, 003C (r1 BOCHS  BXPCMCFG        1 BXPC        1)
  (XEN) ACPI: SPCR 1384D0000, 0050 (r2 BOCHS  BXPCSPCR        1 BXPC        1)

(XEN) Unsupported FADT revision 5.1, should be 6.0+, will disable ACPI (XEN) acpi_boot_table_init: FADT not found (-22)
(XEN) Domain heap initialised
(XEN) Booting using Device Tree
(XEN) Platform: Generic System
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Unable to find compatible GIC in the device tree (XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

Despite saying it is going to reboot it never manages to. Any idea how it is trying to reset the system?

Signature · 1 line
Alex Bennée
author Acked-by Sudeep Holla
#51 lore raw
In reply to André Przywara

(Removing the x86 folks from the CC list)

Hi André,

Quoted 34 lines · André Przywara

On 29/09/2020 22:11, Alex Bennée wrote:

Hi,

Quoted 10 lines
Quoted 8 lines · Julien Grall

Hi Alex,

Quoted 5 lines · Alex Bennée
Quoted 2 lines · Julien Grall

From: Julien Grall <[email protected]>

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1).

Does QEMU provide ACPI tables? Or is this actually EDK2 generating them?

Quoted 1 line

So I did only some light testing.

So about that v6.0 or later: it seems like the requirement comes from:
commit 1c9bd43019cd "arm/acpi: Parse FADT table and get PSCI flags":
"Since STAO table and the GIC version are introduced by ACPI 6.0, we will check the version and only parse FADT table with version >= 6.0. If firmware provides ACPI tables with ACPI version less than 6.0, OS will be messed up with those information, so disable ACPI if we get an FADT table with version less than 6.0."

Quoted 5 lines

STAO (and XENV) have indeed been introduced by ACPI v6.0, but those tables are supposed to be *generated* by Xen and handed down to Dom0, they will never be provided by firmware (or parsed) by the Xen hypervisor. Checking the Linux code it seems to actually not (yet?) support the STAO named list,

I may be because we don't yet use the named list in Xen. I am not sure if other hypervisor ever used the STAO.

Quoted 2 lines

and currently finds and uses the STAO (UART
block bit) regardless of the FADT version number.

IIRC, the concern at the time is an OS may decide to bail out if it finds a table that is not meant to exist.

Quoted 2 lines

I can't find anything GIC related in the FADT, the whole GIC information is described in MADT.

My memory is quite fuzzy... IIRC the problem is (was?) related to how Linux used to check the size of the MADT.

Before commit [1], Linux was checking that the MADT entry was matching the ACPI version. As Xen generates the MADT using the ACPI 6.0 layout, it wouldn't be possible to boot Linux.

Quoted 6 lines

Linux/arm64 seems to be happy with ACPI >= v5.1:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/kernel/acpi.c#n148

So can we relax the v6.0 requirement, to be actually >= v5.1? That is among the first to support ARM anyway, IIRC.

I remember trying to relax the requiring in the past. However, I can't remember why I didn't upstream it. There was possibly some issues I could not get around?

I think supporting ACPI 5.1 would be useful. So I would suggest to attempt it again and see what breaks.

Cheers,

[1]
commit 9eb1c92b47c73249465d388eaa394fe436a3b489

Author: Jeremy Linton <[email protected]>
Date:   Tue Nov 27 17:59:12 2018 +0000

arm64: acpi: Prepare for longer MADTs

The BAD_MADT_GICC_ENTRY check is a little too strict because it rejects MADT entries that don't match the currently known lengths. We should remove this restriction to avoid problems if the table length changes. Future code which might depend on additional fields should be written to validate those fields before using them, rather than trying to globally check known MADT version lengths.

Link:
https://lkml.kernel.org/r/[email protected]

[[email protected]: added MADT macro comments]
Cc Will Deacon <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Al Stone <[email protected]>
Cc "Rafael J. Wysocki" <[email protected]>
Signature · 1 line
Julien Grall
#52 lore raw
In reply to Alex Bennée

Hi Alex,

Quoted 89 lines · Alex Bennée
Quoted 48 lines · Julien Grall

Hi Alex,

Quoted 35 lines · Alex Bennée
Quoted 9 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

    Loading Xen 4.15-unstable ...
    Loading Linux 4.19.0-11-arm64 ...
    Loading initial ramdisk ...
    Using modules provided by bootloader in FDT
    Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader
    ...silence...

I have a grub installed from testing on a buster base:

    dpkg --status grub-arm64-efi
    Version: 2.04-8

With:

    GRUB_CMDLINE_LINUX_DEFAULT=""
    GRUB_CMDLINE_LINUX="console=ttyAMA0"
    GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen"
    GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"
Quoted 6 lines

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

The error is hapenning before Xen setup the console. You can get early output on QEMU if you rebuild Xen with the following .config options:

Quoted 7 lines

CONFIG_DEBUG=y
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x09000000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_PRINTK_INC="debug-pl011.inc"

OK I can see it fails on the ACPI and then tries to fall back to FDT and then fails to find the GIC:

   (XEN) CMDLINE[00000000f7bbe000]:chosen placeholder root=UUID=cf00cd3a-066b-4146-bedf-f811d3343077 ro console=hvc0 earlyprintk=xen
   (XEN)
   (XEN) Command line: placeholder loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg no-real-mode edd=off
   (XEN) parameter "placeholder" unknown!
   (XEN) parameter "no-real-mode" unknown!
   (XEN) parameter "edd" unknown!
   (XEN) ACPI: RSDP 138560000, 0024 (r2 BOCHS )
   (XEN) ACPI: XSDT 138550000, 004C (r1 BOCHS  BXPCFACP        1       1000013)
   (XEN) ACPI: FACP 138510000, 010C (r5 BOCHS  BXPCFACP        1 BXPC        1)
   (XEN) ACPI: DSDT 138520000, 14A6 (r2 BOCHS  BXPCDSDT        1 BXPC        1)
   (XEN) ACPI: APIC 138500000, 018C (r3 BOCHS  BXPCAPIC        1 BXPC        1)
   (XEN) ACPI: GTDT 1384F0000, 0060 (r2 BOCHS  BXPCGTDT        1 BXPC        1)
   (XEN) ACPI: MCFG 1384E0000, 003C (r1 BOCHS  BXPCMCFG        1 BXPC        1)
   (XEN) ACPI: SPCR 1384D0000, 0050 (r2 BOCHS  BXPCSPCR        1 BXPC        1)
   (XEN) Unsupported FADT revision 5.1, should be 6.0+, will disable ACPI
   (XEN) acpi_boot_table_init: FADT not found (-22)
   (XEN) Domain heap initialised
   (XEN) Booting using Device Tree
   (XEN) Platform: Generic System
   (XEN)
   (XEN) ****************************************
   (XEN) Panic on CPU 0:
   (XEN) Unable to find compatible GIC in the device tree
   (XEN) ****************************************
   (XEN)
   (XEN) Reboot in five seconds...
Quoted 2 lines

Despite saying it is going to reboot it never manages to. Any idea how it is trying to reset the system?

This is a bit of chicken and eggs problem. To know the reset method, you need to parse the ACPI tables. As we can't parse then we don't know the reset method. So, Xen will just do an infinite loop.

It would probably be good to be more forthcoming with the users and say it will not reboot.

Also, IIRC, the time subsystem is not yet initialized. So it might be possible to mdelay() doesn't work properly.

Cheers,

Signature · 1 line
Julien Grall
#53 lore raw
Quoted 92 lines · Julien Grall

Hi Alex,

Quoted 89 lines · Alex Bennée
Quoted 48 lines · Julien Grall

Hi Alex,

Quoted 35 lines · Alex Bennée
Quoted 9 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

    Loading Xen 4.15-unstable ...
    Loading Linux 4.19.0-11-arm64 ...
    Loading initial ramdisk ...
    Using modules provided by bootloader in FDT
    Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader
    ...silence...

I have a grub installed from testing on a buster base:

    dpkg --status grub-arm64-efi
    Version: 2.04-8

With:

    GRUB_CMDLINE_LINUX_DEFAULT=""
    GRUB_CMDLINE_LINUX="console=ttyAMA0"
    GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen"
    GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"
Quoted 6 lines

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

The error is hapenning before Xen setup the console. You can get early output on QEMU if you rebuild Xen with the following .config options:

Quoted 7 lines

CONFIG_DEBUG=y
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x09000000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_PRINTK_INC="debug-pl011.inc"

OK I can see it fails on the ACPI and then tries to fall back to FDT and then fails to find the GIC:

   (XEN) CMDLINE[00000000f7bbe000]:chosen placeholder root=UUID=cf00cd3a-066b-4146-bedf-f811d3343077 ro console=hvc0 earlyprintk=xen
   (XEN)
   (XEN) Command line: placeholder loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg no-real-mode edd=off
   (XEN) parameter "placeholder" unknown!
   (XEN) parameter "no-real-mode" unknown!
   (XEN) parameter "edd" unknown!
   (XEN) ACPI: RSDP 138560000, 0024 (r2 BOCHS )
   (XEN) ACPI: XSDT 138550000, 004C (r1 BOCHS  BXPCFACP        1       1000013)
   (XEN) ACPI: FACP 138510000, 010C (r5 BOCHS  BXPCFACP        1 BXPC        1)
   (XEN) ACPI: DSDT 138520000, 14A6 (r2 BOCHS  BXPCDSDT        1 BXPC        1)
   (XEN) ACPI: APIC 138500000, 018C (r3 BOCHS  BXPCAPIC        1 BXPC        1)
   (XEN) ACPI: GTDT 1384F0000, 0060 (r2 BOCHS  BXPCGTDT        1 BXPC        1)
   (XEN) ACPI: MCFG 1384E0000, 003C (r1 BOCHS  BXPCMCFG        1 BXPC        1)
   (XEN) ACPI: SPCR 1384D0000, 0050 (r2 BOCHS  BXPCSPCR        1 BXPC        1)
   (XEN) Unsupported FADT revision 5.1, should be 6.0+, will disable ACPI
   (XEN) acpi_boot_table_init: FADT not found (-22)
   (XEN) Domain heap initialised
   (XEN) Booting using Device Tree
   (XEN) Platform: Generic System
   (XEN)
   (XEN) ****************************************
   (XEN) Panic on CPU 0:
   (XEN) Unable to find compatible GIC in the device tree
   (XEN) ****************************************
   (XEN)
   (XEN) Reboot in five seconds...
Quoted 6 lines
Quoted 2 lines

Despite saying it is going to reboot it never manages to. Any idea how it is trying to reset the system?

This is a bit of chicken and eggs problem. To know the reset method, you need to parse the ACPI tables. As we can't parse then we don't know the reset method. So, Xen will just do an infinite loop.

Well you do get some ACPI tables - downgrading the minimum at least restores the reset method detection. I wonder if it would be worth defaulting to PSCI if you don't know rather than hang indefinitely?

FWIW the failure after that is failing to find the GIC - I'm just looking at the MADT table parsing now. Why am I getting a sense of DejaVu?

Quoted 5 lines

It would probably be good to be more forthcoming with the users and say it will not reboot.

Also, IIRC, the time subsystem is not yet initialized. So it might be possible to mdelay() doesn't work properly.

Surely that's an architectural subsystem so there is no reason that couldn't be up and running.

Quoted 2 lines

Cheers,

Signature · 1 line
Alex Bennée
#54 lore raw

Hi Alex,

Quoted 98 lines · Alex Bennée
Quoted 95 lines · Julien Grall

Hi Alex,

Quoted 91 lines · Alex Bennée
Quoted 48 lines · Julien Grall

Hi Alex,

Quoted 35 lines · Alex Bennée
Quoted 9 lines · Julien Grall

From: Julien Grall <[email protected]>

Hi all,

Xen on ARM has been broken for quite a while on ACPI systems. This series aims to fix it.

Unfortunately I don't have a system with ACPI v6.0 or later (QEMU seems to only support 5.1). So I did only some light testing.

I was hoping to get more diagnostics out to get it working under QEMU TCG so I think must of missed a step:

     Loading Xen 4.15-unstable ...
     Loading Linux 4.19.0-11-arm64 ...
     Loading initial ramdisk ...
     Using modules provided by bootloader in FDT
     Xen 4.15-unstable (c/s Sat Sep 26 21:55:42 2020 +0100 git:72f3d495d0) EFI loader
     ...silence...

I have a grub installed from testing on a buster base:

     dpkg --status grub-arm64-efi
     Version: 2.04-8

With:

     GRUB_CMDLINE_LINUX_DEFAULT=""
     GRUB_CMDLINE_LINUX="console=ttyAMA0"
     GRUB_CMDLINE_LINUX_XEN_REPLACE="console=hvc0 earlyprintk=xen"
     GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg"
Quoted 6 lines

And I built Xen with --enable-systemd and tweaked the hypervisor .config:

CONFIG_EXPERT=y
CONFIG_ACPI=y

So any pointers to make it more verbose would be helpful.

The error is hapenning before Xen setup the console. You can get early output on QEMU if you rebuild Xen with the following .config options:

Quoted 7 lines

CONFIG_DEBUG=y
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x09000000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_PRINTK_INC="debug-pl011.inc"

OK I can see it fails on the ACPI and then tries to fall back to FDT and then fails to find the GIC:

    (XEN) CMDLINE[00000000f7bbe000]:chosen placeholder root=UUID=cf00cd3a-066b-4146-bedf-f811d3343077 ro console=hvc0 earlyprintk=xen
    (XEN)
    (XEN) Command line: placeholder loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5console=com1,vg no-real-mode edd=off
    (XEN) parameter "placeholder" unknown!
    (XEN) parameter "no-real-mode" unknown!
    (XEN) parameter "edd" unknown!
    (XEN) ACPI: RSDP 138560000, 0024 (r2 BOCHS )
    (XEN) ACPI: XSDT 138550000, 004C (r1 BOCHS  BXPCFACP        1       1000013)
    (XEN) ACPI: FACP 138510000, 010C (r5 BOCHS  BXPCFACP        1 BXPC        1)
    (XEN) ACPI: DSDT 138520000, 14A6 (r2 BOCHS  BXPCDSDT        1 BXPC        1)
    (XEN) ACPI: APIC 138500000, 018C (r3 BOCHS  BXPCAPIC        1 BXPC        1)
    (XEN) ACPI: GTDT 1384F0000, 0060 (r2 BOCHS  BXPCGTDT        1 BXPC        1)
    (XEN) ACPI: MCFG 1384E0000, 003C (r1 BOCHS  BXPCMCFG        1 BXPC        1)
    (XEN) ACPI: SPCR 1384D0000, 0050 (r2 BOCHS  BXPCSPCR        1 BXPC        1)
    (XEN) Unsupported FADT revision 5.1, should be 6.0+, will disable ACPI
    (XEN) acpi_boot_table_init: FADT not found (-22)
    (XEN) Domain heap initialised
    (XEN) Booting using Device Tree
    (XEN) Platform: Generic System
    (XEN)
    (XEN) ****************************************
    (XEN) Panic on CPU 0:
    (XEN) Unable to find compatible GIC in the device tree
    (XEN) ****************************************
    (XEN)
    (XEN) Reboot in five seconds...

Despite saying it is going to reboot it never manages to. Any idea how it is trying to reset the system?

Quoted 7 lines
Quoted 3 lines

This is a bit of chicken and eggs problem. To know the reset method, you need to parse the ACPI tables. As we can't parse then we don't know the reset method. So, Xen will just do an infinite loop.

Well you do get some ACPI tables - downgrading the minimum at least restores the reset method detection. I wonder if it would be worth defaulting to PSCI if you don't know rather than hang indefinitely?

The risk is probably low enough to try to use PSCI even on platform not supporting it.

Although, it might be worth to check if EL3 is present to avoid panicking again and again on XGene.

Quoted 4 lines

FWIW the failure after that is failing to find the GIC - I'm just looking at the MADT table parsing now. Why am I getting a sense of DejaVu?

The ACPI code in Xen is based on the first ACPI implementation in Linux. So it is quite possible you encountered the bug there :).

Quoted 8 lines
Quoted 5 lines

It would probably be good to be more forthcoming with the users and say it will not reboot.

Also, IIRC, the time subsystem is not yet initialized. So it might be possible to mdelay() doesn't work properly.

Surely that's an architectural subsystem so there is no reason that couldn't be up and running.

In theory yes, but the code is also catering some interesting/weird platforms behavior:
1) There are (were?) platform where CNTFREQ was not set correctly
2) Some platforms, such as the one with Exynos 5, (used to?) require
specific code to enable the arch timer.

We are still using the Arndale for automated testing. So we would need to keep the hacks.

But it would be possible to rework the code and try to make the timer available earlier for well-behaved platforms.

Cheers,

Signature · 1 line
Julien Grall