Adjustment after introducing ASSERT_ALLOC_CONTEXT
cover Adjustment after introducing ASSERT_ALLOC_CONTEXT
With the enhanced ASSERT_ALLOC_CONTEXT, calling request_irq before local_irq_enable on secondary cores on Arm will lead to
(XEN) Xen call trace:
(XEN) [<000000000021d86c>] alloc_xenheap_pages+0x74/0x194 (PC) (XEN) [<000000000021d864>] alloc_xenheap_pages+0x6c/0x194 (LR) (XEN) [<0000000000229e90>] xmalloc_tlsf.c#xmalloc_pool_get+0x1c/0x28 (XEN) [<000000000022a270>] xmem_pool_alloc+0x21c/0x448 (XEN) [<000000000022a8dc>] _xmalloc+0x8c/0x290
(XEN) [<000000000026b57c>] request_irq+0x40/0xb8
(XEN) [<0000000000272780>] init_timer_interrupt+0x74/0xcc (XEN) [<000000000027212c>] start_secondary+0x1b4/0x238 (XEN) [<0000000084000200>] 0000000084000200
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 4:
(XEN) Assertion '!in_irq() && (local_irq_is_enabled() || num_online_cpus() <= 1)' failed at common/page_alloc.c:2212 (XEN) ****************************************
on systems without a big enough pool for xmalloc() to cater the requested size. To solve this issue, this series introduces two patches. The first one defers the calling of request_irq on secondary CPUs after local_irq_enable on Arm. The second one moves the definition of ASSERT_ALLOC_CONTEXT to header and uses the ASSERT_ALLOC_CONTEXT to replace the original assertion in xmalloc().
Henry Wang (2):
xen/arm: Defer request_irq on secondary CPUs after local_irq_enable xen/common: Use enhanced ASSERT_ALLOC_CONTEXT in xmalloc()
xen/arch/arm/smpboot.c | 7 ++++--- xen/common/page_alloc.c | 7 ------- xen/common/xmalloc_tlsf.c | 2 +- xen/include/xen/irq.h | 7 +++++++ 4 files changed, 12 insertions(+), 11 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/2 xen/arm: Defer request_irq on secondary CPUs after local_irq_enable
With the enhanced ASSERT_ALLOC_CONTEXT, calling request_irq before local_irq_enable on secondary cores will lead to
(XEN) Xen call trace:
(XEN) [<000000000021d86c>] alloc_xenheap_pages+0x74/0x194 (PC) (XEN) [<000000000021d864>] alloc_xenheap_pages+0x6c/0x194 (LR) (XEN) [<0000000000229e90>] xmalloc_tlsf.c#xmalloc_pool_get+0x1c/0x28 (XEN) [<000000000022a270>] xmem_pool_alloc+0x21c/0x448 (XEN) [<000000000022a8dc>] _xmalloc+0x8c/0x290
(XEN) [<000000000026b57c>] request_irq+0x40/0xb8
(XEN) [<0000000000272780>] init_timer_interrupt+0x74/0xcc (XEN) [<000000000027212c>] start_secondary+0x1b4/0x238 (XEN) [<0000000084000200>] 0000000084000200
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 4:
(XEN) Assertion '!in_irq() && (local_irq_is_enabled() || num_online_cpus() <= 1)' failed at common/page_alloc.c:2212 (XEN) ****************************************
on systems without a big enough pool for xmalloc() to cater the requested size.
--- xen/arch/arm/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
xen/arch/arm/smpboot.c +4 -3
@@ -361,9 +361,6 @@ void start_secondary(void)init_secondary_IRQ (); -init_maintenance_interrupt (); -init_timer_interrupt (); -set_current (idle_vcpu[cpuid]);setup_cpu_sibling_map (cpuid);
@@ -380,6 +377,10 @@ void start_secondary(void)cpumask_set_cpu (cpuid, &cpu_online_map);local_irq_enable (); + +init_maintenance_interrupt (); +init_timer_interrupt (); +local_abort_enable ();check_local_cpu_errata ();
Signature · 1 line
2.25.1
RE: [PATCH 1/2] xen/arm: Defer request_irq on secondary CPUs after local_irq_enable
Hi,
Quoted 25 lines
-----Original Message-----
Subject: [PATCH 1/2] xen/arm: Deferrequest_irqon secondary CPUs afterlocal_irq_enableWith the enhanced
ASSERT_ALLOC_CONTEXT, callingrequest_irqbeforelocal_irq_enableon secondary cores will lead to(XEN) Xen call trace:
(XEN) [<000000000021d86c>]alloc_xenheap_pages+0x74/0x194(PC) (XEN) [<000000000021d864>]alloc_xenheap_pages+0x6c/0x194(LR) (XEN) [<0000000000229e90>] xmalloc_tlsf.c#xmalloc_pool_get+0x1c/0x28 (XEN) [<000000000022a270>]xmem_pool_alloc+0x21c/0x448(XEN) [<000000000022a8dc>]_xmalloc+0x8c/0x290
(XEN) [<000000000026b57c>]request_irq+0x40/0xb8
(XEN) [<0000000000272780>]init_timer_interrupt+0x74/0xcc(XEN) [<000000000027212c>]start_secondary+0x1b4/0x238(XEN) [<0000000084000200>] 0000000084000200
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 4:
(XEN) Assertion '!in_irq() && (local_irq_is_enabled()||num_online_cpus()<= 1)' failed atcommon/page_alloc.c:2212(XEN) ****************************************
Quoted 7 lines
on systems without a big enough pool for
xmalloc()to cater the requested size.Reported-by Wei Chen <[email protected]>Suggested-by Julien Grall <[email protected]>Signed-off-by Henry Wang <[email protected]>Change-Id Iebdde81f52785b0c6e037c981ff68922db016d08
Sorry about the Change-Id, forgot to remove it before sending the mail to ML, will remove it in V2. Same for PATCH 2/2.
Kind regards,
Henry
Quoted 3 lines
--- xen/arch/arm/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
Re: [PATCH 1/2] xen/arm: Defer request_irq on secondary CPUs after local_irq_enable
In reply to Henry WangHi Henry,
The CC list doesn't seem to contain the other maintainers. Please use the scripts provided in the repo (such scripts/add_maintainers.pl) to generate the CC.
Quoted 21 lines · Henry Wang
With the enhanced
ASSERT_ALLOC_CONTEXT, callingrequest_irqbeforelocal_irq_enableon secondary cores will lead to(XEN) Xen call trace:
(XEN) [<000000000021d86c>]alloc_xenheap_pages+0x74/0x194(PC) (XEN) [<000000000021d864>]alloc_xenheap_pages+0x6c/0x194(LR) (XEN) [<0000000000229e90>] xmalloc_tlsf.c#xmalloc_pool_get+0x1c/0x28 (XEN) [<000000000022a270>]xmem_pool_alloc+0x21c/0x448(XEN) [<000000000022a8dc>]_xmalloc+0x8c/0x290
(XEN) [<000000000026b57c>]request_irq+0x40/0xb8
(XEN) [<0000000000272780>]init_timer_interrupt+0x74/0xcc(XEN) [<000000000027212c>]start_secondary+0x1b4/0x238(XEN) [<0000000084000200>] 0000000084000200
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 4:
(XEN) Assertion '!in_irq() && (local_irq_is_enabled()||num_online_cpus()<= 1)' failed atcommon/page_alloc.c:2212(XEN) ****************************************
Quoted 2 lines
on systems without a big enough pool for
xmalloc()to cater the requested size.
Can you explain in the commit message, why this is OK to move the code after enabling interrupts?
Quoted 24 lines
Reported-by Wei Chen <[email protected]>Suggested-by Julien Grall <[email protected]>Signed-off-by Henry Wang <[email protected]>Change-Id Iebdde81f52785b0c6e037c981ff68922db016d08--- xen/arch/arm/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)xen/arch/arm/smpboot.c +0
-3@@ -361,9 +361,6 @@ void start_secondary(void)init_secondary_IRQ (); -init_maintenance_interrupt (); -init_timer_interrupt (); -set_current (idle_vcpu[cpuid]);setup_cpu_sibling_map (cpuid);
xen/arch/arm/smpboot.c void start_secondary(void) · line 386
Quoted 10 lines
xen/arch/arm/smpboot.c +4
-0@@ -380,6 +377,10 @@ void start_secondary(void)cpumask_set_cpu (cpuid, &cpu_online_map);local_irq_enable (); + +init_maintenance_interrupt (); +init_timer_interrupt (); +local_abort_enable ();check_local_cpu_errata ();
Cheers,
Signature · 1 line
Julien Grall
RE: [PATCH 1/2] xen/arm: Defer request_irq on secondary CPUs after local_irq_enable
Hi Julien,
Quoted 30 lines
-----Original Message-----
From: Julien Grall <[email protected]>
Hi Henry,The CC list doesn't seem to contain the other maintainers. Please use the scripts provided in the repo (such
scripts/add_maintainers.pl) to generate the CC.Quoted 21 lines · Henry Wang
With the enhanced
ASSERT_ALLOC_CONTEXT, callingrequest_irqbeforelocal_irq_enableon secondary cores will lead to(XEN) Xen call trace:
(XEN) [<000000000021d86c>]alloc_xenheap_pages+0x74/0x194(PC) (XEN) [<000000000021d864>]alloc_xenheap_pages+0x6c/0x194(LR) (XEN) [<0000000000229e90>] xmalloc_tlsf.c#xmalloc_pool_get+0x1c/0x28 (XEN) [<000000000022a270>]xmem_pool_alloc+0x21c/0x448(XEN) [<000000000022a8dc>]_xmalloc+0x8c/0x290
(XEN) [<000000000026b57c>]request_irq+0x40/0xb8
(XEN) [<0000000000272780>]init_timer_interrupt+0x74/0xcc(XEN) [<000000000027212c>]start_secondary+0x1b4/0x238(XEN) [<0000000084000200>] 0000000084000200
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 4:
(XEN) Assertion '!in_irq() && (local_irq_is_enabled()||num_online_cpus()<= 1)' failed atcommon/page_alloc.c:2212(XEN) ****************************************
Quoted 5 lines
Quoted 2 lines
on systems without a big enough pool for
xmalloc()to cater the requested size.Can you explain in the commit message, why this is OK to move the code after enabling interrupts?
Sure, will correct both issues in v2.
Kind regards,
Henry
Quoted 40 lines
Quoted 25 lines
Reported-by Wei Chen <[email protected]>Suggested-by Julien Grall <[email protected]>Signed-off-by Henry Wang <[email protected]>Change-Id Iebdde81f52785b0c6e037c981ff68922db016d08--- xen/arch/arm/smpboot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)xen/arch/arm/smpboot.c +0
-3@@ -361,9 +361,6 @@ void start_secondary(void)init_secondary_IRQ (); -init_maintenance_interrupt (); -init_timer_interrupt (); -set_current (idle_vcpu[cpuid]);setup_cpu_sibling_map (cpuid);xen/arch/arm/smpboot.c void start_secondary(void) · line 386
Quoted 10 lines
xen/arch/arm/smpboot.c +4
-0@@ -380,6 +377,10 @@ void start_secondary(void)cpumask_set_cpu (cpuid, &cpu_online_map);local_irq_enable (); + +init_maintenance_interrupt (); +init_timer_interrupt (); +local_abort_enable ();check_local_cpu_errata ();Cheers,
--
Julien Grall
PATCH 2/2 xen/common: Use enhanced ASSERT_ALLOC_CONTEXT in xmalloc()
In reply to Henry Wangxmalloc() will use a pool for allocation smaller than a page. The pool is extended only when there are no more space. At which point, alloc_xenheap_pages() is called to add more memory.
xmalloc() must be protected by ASSERT_ALLOC_CONTEXT. It should not rely on pool expanding to trigger the ASSERT_ALLOC_CONTEXT in alloc_xenheap_pages(). Hence, this commit moves the definition of ASSERT_ALLOC_CONTEXT to header and uses the ASSERT_ALLOC_CONTEXT to replace the original assertion in xmalloc().
--- xen/common/page_alloc.c | 7 ------- xen/common/xmalloc_tlsf.c | 2 +- xen/include/xen/irq.h | 7 +++++++ 3 files changed, 8 insertions(+), 8 deletions(-)
xen/common/page_alloc.c +0 -7
@@ -162,13 +162,6 @@static char __initdata opt_badpage[100 ] ="" ;string_param ("badpage" , opt_badpage); -/* -* Heap allocations may need TLB flushes which may require IRQs to be -* enabled (except when only 1 PCPU is online). -*/ -#define ASSERT_ALLOC_CONTEXT () \ - ASSERT(!in_irq() && (local_irq_is_enabled() || num_online_cpus() <= 1)) -/* * no-bootscrub -> Free pages are not zeroed during boot. */
xen/common/xmalloc_tlsf.c +1 -1
@@ -594,7 +594,7 @@ void *_xmalloc(unsigned long size, unsigned long align) {void *p =NULL ; -ASSERT (!in_irq ()); +ASSERT_ALLOC_CONTEXT ();if ( !size )return ZERO_BLOCK_PTR ;
xen/include/xen/irq.h +7 -0
@@ -10,6 +10,13 @@#include <asm/hardirq.h> #include <public/event_channel.h> +/* +* Heap allocations may need TLB flushes which may require IRQs to be +* enabled (except when only 1 PCPU is online). +*/ +#define ASSERT_ALLOC_CONTEXT () \ + ASSERT(!in_irq() && (local_irq_is_enabled() || num_online_cpus() <= 1)) +struct irqaction {void (*handler)(int ,void *,struct cpu_user_regs *);const char *name;
Signature · 1 line
2.25.1
Re: [PATCH 2/2] xen/common: Use enhanced ASSERT_ALLOC_CONTEXT in xmalloc()
Quoted 10 lines · Henry Wang
xmalloc()will use a pool for allocation smaller than a page. The pool is extended only when there are no more space. At which point,alloc_xenheap_pages()is called to add more memory.
xmalloc()must be protected byASSERT_ALLOC_CONTEXT. It should not rely on pool expanding to trigger theASSERT_ALLOC_CONTEXTinalloc_xenheap_pages(). Hence, this commit moves the definition ofASSERT_ALLOC_CONTEXTto header and uses theASSERT_ALLOC_CONTEXTto replace the original assertion inxmalloc().
Quoted 4 lines
Reported-by Wei Chen <[email protected]>Suggested-by Julien Grall <[email protected]>Signed-off-by Henry Wang <[email protected]>Change-Id Ia463d2241e80e8a78d7dbb5b2318694d3ca5ed67
Just two formal remarks for starters: What's this last tag? And why am I on the To: list of this patch, when ...
Quoted 5 lines
--- xen/common/page_alloc.c | 7 ------- xen/common/xmalloc_tlsf.c | 2 +- xen/include/xen/irq.h | 7 +++++++ 3 files changed, 8 insertions(+), 8 deletions(-)
... with this diffstat you should instead have _Cc_-ed REST maintainers?
Jan
RE: [PATCH 2/2] xen/common: Use enhanced ASSERT_ALLOC_CONTEXT in xmalloc()
Hi Jan,
Quoted 6 lines
-----Original Message-----
From: Jan Beulich <[email protected]>Quoted 1 line
Change-Id: Ia463d2241e80e8a78d7dbb5b2318694d3ca5ed67
Just two formal remarks for starters: What's this last tag? And why am I on the To: list of this patch, when ...
I forgot to remove the Change-Id before sending the patch, sorry about that. This will be removed in v2.
Quoted 8 lines
Quoted 6 lines
--- xen/common/page_alloc.c | 7 ------- xen/common/xmalloc_tlsf.c | 2 +- xen/include/xen/irq.h | 7 +++++++ 3 files changed, 8 insertions(+), 8 deletions(-)... with this diffstat you should instead have _Cc_-ed REST maintainers?
I think this is because by switching to add-maintainers.pl I didn't change my git send-email command line where originally it has a --suppress-cc=all. Since you are the maintainer of common code I thought I should directly "To" you.
Kind regards,
Henry
Quoted 2 lines
Jan
Re: [PATCH 2/2] xen/common: Use enhanced ASSERT_ALLOC_CONTEXT in xmalloc()
Quoted 17 lines · Henry Wang
Quoted 6 lines
-----Original Message-----
From: Jan Beulich <[email protected]>Quoted 1 line
Change-Id: Ia463d2241e80e8a78d7dbb5b2318694d3ca5ed67
Just two formal remarks for starters: What's this last tag? And why am I on the To: list of this patch, when ...
I forgot to remove the Change-Id before sending the patch, sorry about that. This will be removed in v2.
Quoted 7 lines
Quoted 6 lines
--- xen/common/page_alloc.c | 7 ------- xen/common/xmalloc_tlsf.c | 2 +- xen/include/xen/irq.h | 7 +++++++ 3 files changed, 8 insertions(+), 8 deletions(-)
Quoted 6 lines
Quoted 1 line
... with this diffstat you should instead have _Cc_-ed REST maintainers?
I think this is because by switching to add-maintainers.pl I didn't change my git send-email command line where originally it has a --suppress-cc=all. Since you are the maintainer of common code I thought I should directly "To" you.
I'm sorry, but no: For one I am not "the" maintainer, but one of several. And then patches are generally only sent _To_ the list, with _Cc_ to maintainers.
Jan