xen/mm: limit in-place scrubbing
v4 cover xen/mm: limit in-place scrubbing
Hello,
In XenServer we have seen the watchdog occasionally triggering during domain creation if 1GB pages are scrubbed in-place during physmap population. The following series attempt to mitigate this by adding preemption to page scrubbing in populate_physmap(). Also a new limit and command line option to signal the maximum allocation order when doing in-place scrubbing. This is set by default to CONFIG_PTDOM_MAX_ORDER.
Thanks, Roger.
Roger Pau Monne (3):
xen/mm: enforce SCRUB_DEBUG checks for MEMF_no_scrub allocations xen/mm: allow deferred scrub of physmap populate allocated pages xen/mm: limit non-scrubbed allocations to a specific order
docs/misc/xen-command-line.pandoc | 13 ++++ xen/common/domain.c | 30 +++++++++ xen/common/memory.c | 104 ++++++++++++++++++++++++++++-- xen/common/page_alloc.c | 30 +++++++-- xen/include/xen/mm.h | 14 ++++ xen/include/xen/sched.h | 5 ++ 6 files changed, 187 insertions(+), 9 deletions(-)
Signature · 1 line
2.51.0
PATCH v4 1/3 xen/mm: enforce SCRUB_DEBUG checks for MEMF_no_scrub allocations
The logic in alloc_heap_pages() only checks for scrubbing pattern correctness when the caller doesn't pass MEMF_no_scrub in memflags. However already scrubbed pages can be checked for correctness, regardless of the caller having requested MEMF_no_scrub.
Relax the checking around the check_one_page() call, to allow for calls with MEMF_no_scrub to also check the correctness of pages marked as already scrubbed when allocated. This widens the checking of scrubbing correctness, so it would also check the scrubbing correctness of MEMF_no_scrub allocations.
---
After discussing with Jan I've deliberately omitted the tag:
The intended approach might have been to ensure the caller of alloc_heap_pages() gets properly scrubbed pages, rather than asserting the internal state of free pages is as expected.
--- xen/common/page_alloc.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
xen/common/page_alloc.c +2 -3
@@ -1105,8 +1105,7 @@ static struct page_info *alloc_heap_pages(spin_unlock (&heap_lock); -if ( first_dirty !=INVALID_DIRTY_IDX || - (scrub_debug && !(memflags & MEMF_no_scrub)) ) +if ( first_dirty !=INVALID_DIRTY_IDX || scrub_debug ) {bool cold = d && d != current->domain;
@@ -1119,7 +1118,7 @@ static struct page_info *alloc_heap_pages( dirty_cnt++; } -else if ( !(memflags & MEMF_no_scrub) ) +else check_one_page (&pg[i]); }
Signature · 1 line
2.51.0
PATCH v4 2/3 xen/mm: allow deferred scrub of physmap populate allocated pages
In reply to Roger Pau MonnePhysmap population has the need to use pages as big as possible to reduce p2m shattering. However that triggers issues when big enough pages are not yet scrubbed, and so scrubbing must be done at allocation time. On some scenarios with added contention the watchdog can trigger:
Watchdog timer detects that CPU55 is stuck! ----[ Xen-4.17.5-21 x86_64 debug=n Not tainted ]----
CPU: 55
RIP: e008:[<ffff82d040204c4a>] clear_page_sse2+0x1a/0x30
RFLAGS: 0000000000000202 CONTEXT: hypervisor (d0v12)
[...]
Xen call trace:
[<ffff82d040204c4a>] R clear_page_sse2+0x1a/0x30 [<ffff82d04022a121>] S clear_domain_page+0x11/0x20 [<ffff82d04022c170>] S common/page_alloc.c#alloc_heap_pages+0x400/0x5a0 [<ffff82d04022d4a7>] S alloc_domheap_pages+0x67/0x180 [<ffff82d040226f9f>] S common/memory.c#populate_physmap+0x22f/0x3b0 [<ffff82d040228ec8>] S do_memory_op+0x728/0x1970
Introduce a mechanism to preempt page scrubbing in populate_physmap(). It relies on stashing the dirty page in the domain struct temporarily to preempt to guest context, so the scrubbing can resume when the domain re-enters the hypercall. The added deferral mechanism will only be used for domain construction, and is designed to be used with a single threaded domain builder. If the toolstack makes concurrent calls to XENMEM_populate_physmap for the same target domain it will trash stashed pages, resulting in slow domain physmap population.
Note a similar issue is present in increase reservation. However that hypercall is likely to only be used once the domain is already running and the known implementations use 4K pages. It will be deal with in a separate patch using a different approach, that will also take care of the allocation in populate_physmap() once the domain is running.
---
Changes since v3:
- Introduce helper to free stashed pages.
- Attempt to free stashed pages from domain_unpause_by_systemcontroller() also.
- Free stashed page in get_stashed_allocation() if it doesn't match the requested parameters.
Changes since v2:
- Introduce FREE_DOMHEAP_PAGE{,S}().
- Remove j local counter.
- Free page pending scrub in domain_kill() also.
- Remove BUG_ON().
- Reorder get_stashed_allocation() flow.
- s/dirty/unscrubbed/ in a printk message.
Changes since v1:
- New in this version, different approach than v1.
--- xen/common/domain.c | 30 ++++++++++++ xen/common/memory.c | 101 +++++++++++++++++++++++++++++++++++++++- xen/common/page_alloc.c | 2 +- xen/include/xen/mm.h | 10 ++++ xen/include/xen/sched.h | 5 ++ 5 files changed, 146 insertions(+), 2 deletions(-)
xen/common/domain.c +30 -0
@@ -710,6 +710,23 @@ static int domain_teardown(struct domain *d)return 0 ; } +/* +* Called multiple times during domain destruction, to attempt to early free +* any stashed pages to be scrubbed. The call from _domain_destroy() is done +* when the toolstack can no longer stash any pages. +*/ +static void domain_free_pending_scrub (struct domain *d) +{ +rspin_lock (&d->page_alloc_lock); +if ( d->pending_scrub ) + { +FREE_DOMHEAP_PAGES (d->pending_scrub, d->pending_scrub_order); + d->pending_scrub_order =0 ; + d->pending_scrub_index =0 ; + } +rspin_unlock (&d->page_alloc_lock); +} +/* * Destroy a domain once all references to it have been dropped. Used either * from the RCU path, or from the domain_create() error path before the domain
@@ -722,6 +739,8 @@ static void _domain_destroy(struct domain *d)XVFREE (d->console); +domain_free_pending_scrub (d); +argo_destroy (d);rangeset_domain_destroy (d);
@@ -1286,6 +1305,8 @@ int domain_kill(struct domain *d)rspin_barrier (&d->domain_lock);argo_destroy (d);vnuma_destroy (d->vnuma); +domain_free_pending_scrub (d); +rspin_unlock (&d->page_alloc_lock);domain_set_outstanding_pages (d,0 );/* fallthrough */ case DOMDYING_dying:
@@ -1678,6 +1699,15 @@ int domain_unpause_by_systemcontroller(struct domain *d)*/ if ( new ==0 && !d->creation_finished ) { +if ( d->pending_scrub ) + { +printk (XENLOG_ERR +"%pd: cannot be started with pending unscrubbed pages, destroying\n" , + d); +domain_crash (d); +domain_free_pending_scrub (d); +return -EBUSY ; + } d->creation_finished = true;arch_domain_creation_finished (d); }
xen/common/memory.c +100 -1
119 lines not on this page · raw message
xen/common/page_alloc.c +1 -1
@@ -792,7 +792,7 @@ static void page_list_add_scrub(struct page_info *pg, unsigned int node,# define scrub_page_cold clear_page_cold#endif -static void scrub_one_page (const struct page_info *pg,bool cold) +void scrub_one_page (const struct page_info *pg,bool cold) {void *ptr;
xen/include/xen/mm.h +10 -0
@@ -145,6 +145,16 @@ unsigned long avail_node_heap_pages(unsigned int nodeid);#define alloc_domheap_page (d,f) (alloc_domheap_pages(d,0,f))#define free_domheap_page (p) (free_domheap_pages(p,0)) +/* Free an allocation, and zero the pointer to it. */ +#define FREE_DOMHEAP_PAGES (p, o) do { \ + void *_ptr_ = (p); \ + (p) = NULL; \ + free_domheap_pages(_ptr_, o); \ +} while ( false ) +#define FREE_DOMHEAP_PAGE (p) FREE_DOMHEAP_PAGES(p, 0) + +void scrub_one_page (const struct page_info *pg,bool cold); +int online_page (mfn_t mfn,uint32_t *status);int offline_page (mfn_t mfn,int broken,uint32_t *status);int query_page_offline (mfn_t mfn,uint32_t *status);
xen/include/xen/sched.h +5 -0
@@ -661,6 +661,11 @@ struct domain/* Pointer to console settings; NULL for system domains. */ struct domain_console *console; + +/* Pointer to allocated domheap page that possibly needs scrubbing. */ +struct page_info *pending_scrub; +unsigned int pending_scrub_order; +unsigned int pending_scrub_index; }__aligned(PAGE_SIZE) ; static inline struct page_list_head *page_to_list(
Signature · 1 line
2.51.0
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
Quoted 38 lines · Roger Pau Monne
xen/common/domain.c +19
-0@@ -710,6 +710,23 @@ static int domain_teardown(struct domain *d)return 0 ; } +/* +* Called multiple times during domain destruction, to attempt to early free +* any stashed pages to be scrubbed. The call from _domain_destroy() is done +* when the toolstack can no longer stash any pages. +*/ +static void domain_free_pending_scrub (struct domain *d) +{ +rspin_lock (&d->page_alloc_lock); +if ( d->pending_scrub ) + { +FREE_DOMHEAP_PAGES (d->pending_scrub, d->pending_scrub_order); + d->pending_scrub_order =0 ; + d->pending_scrub_index =0 ; + } +rspin_unlock (&d->page_alloc_lock); +} +/* * Destroy a domain once all references to it have been dropped. Used either * from the RCU path, or from the domain_create() error path before the domain @@ -722,6 +739,8 @@ static void _domain_destroy(struct domain *d)XVFREE (d->console); +domain_free_pending_scrub (d); +argo_destroy (d);rangeset_domain_destroy (d);
xen/common/domain.c int domain_kill(struct domain *d) · line 1309
Quoted 5 lines
xen/common/domain.c +2
-0@@ -1286,6 +1305,8 @@ int domain_kill(struct domain *d)rspin_barrier (&d->domain_lock);argo_destroy (d);vnuma_destroy (d->vnuma); +domain_free_pending_scrub (d); +rspin_unlock (&d->page_alloc_lock);
This is a double unlock, isn't it?
The freeing wants to be in domain_kill() (ish), or _domain_destroy() but not both.
In this case we can't have anything using pending scrubbing until the domain is the domlist (i.e. can be the target of other hypercalls), so this wants to be in domain_relinquish_resources() (rather than domain_kill() which we're trying to empty).
In principle we could assert that it's already NULL in _domain_destroy() which might help catch an error if it gets set early but the domain destroyed before getting into the domlist, but that seems like a rather slim case.
~Andrew
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
Quoted 37 lines · Andrew Cooper
Quoted 36 lines · Roger Pau Monne
xen/common/domain.c +19
-0@@ -710,6 +710,23 @@ static int domain_teardown(struct domain *d)return 0 ; } +/* +* Called multiple times during domain destruction, to attempt to early free +* any stashed pages to be scrubbed. The call from _domain_destroy() is done +* when the toolstack can no longer stash any pages. +*/ +static void domain_free_pending_scrub (struct domain *d) +{ +rspin_lock (&d->page_alloc_lock); +if ( d->pending_scrub ) + { +FREE_DOMHEAP_PAGES (d->pending_scrub, d->pending_scrub_order); + d->pending_scrub_order =0 ; + d->pending_scrub_index =0 ; + } +rspin_unlock (&d->page_alloc_lock); +} +/* * Destroy a domain once all references to it have been dropped. Used either * from the RCU path, or from the domain_create() error path before the domain @@ -722,6 +739,8 @@ static void _domain_destroy(struct domain *d)XVFREE (d->console); +domain_free_pending_scrub (d); +argo_destroy (d);
Quoted 9 lines
xen/common/domain.c int domain_kill(struct domain *d) · line 1309
Quoted 7 lines
xen/common/domain.c +2
-0rangeset_domain_destroy (d);@@ -1286,6 +1305,8 @@ int domain_kill(struct domain *d)rspin_barrier (&d->domain_lock);argo_destroy (d);vnuma_destroy (d->vnuma); +domain_free_pending_scrub (d); +rspin_unlock (&d->page_alloc_lock);This is a double unlock, isn't it?
Bah, this was a leftover from the previous version, sorry.
Quoted 3 lines
The freeing wants to be in
domain_kill()(ish), or_domain_destroy()but not both.
Jan specifically asked for the cleanup to be in both. _domain_destroy() is the must have one, as that's done once the domain is unhooked from the domain list and hence can no longer be the target of populate physmap hypercalls. Jan asked for the domain_kill() instance to attempt to free the pending page as early as possible.
Quoted 4 lines
In this case we can't have anything using pending scrubbing until the domain is the domlist (i.e. can be the target of other hypercalls), so this wants to be in domain_relinquish_resources() (rather than
domain_kill()which we're trying to empty).
domain_relinquish_resources() is arch-specific, while the pending_scrub stuff is common with all arches. It seemed better placed in domain_kill() because that's generic code shared by all arches.
I could place it in domain_teardown() instead if that's better than domain_kill().
Quoted 4 lines
In principle we could assert that it's already NULL in
_domain_destroy()which might help catch an error if it gets set early but the domain destroyed before getting into the domlist, but that seems like a rather slim case.
Given my understanding of the logic in the XENMEM_ hypercalls, I think toolstack can still target domains in the process of being destroyed, at which point we need to keep a final cleanup instance _domain_destroy(), or otherwise adjust XENMEM_populate_physmap hypercall (and others?) so they can't target dying domains.
Thanks, Roger.
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
Quoted 11 lines · Roger Pau Monné
Quoted 4 lines · Andrew Cooper
In principle we could assert that it's already NULL in
_domain_destroy()which might help catch an error if it gets set early but the domain destroyed before getting into the domlist, but that seems like a rather slim case.Given my understanding of the logic in the
XENMEM_hypercalls, I think toolstack can still target domains in the process of being destroyed, at which point we need to keep a final cleanup instance_domain_destroy(), or otherwise adjustXENMEM_populate_physmaphypercall (and others?) so they can't target dying domains.
Considering that these requests will fail for dying domains because of the check in assign_pages(), it may indeed make sense to have another earlier check for the purposes here. Otoh doing this early may not buy us very much, as the domain may become "dying" immediately after the check. Whereas switching stash_allocation()'s if() to
if ( d->pending_scrub || d->is_dying )
looks like it might do.
Jan
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
Quoted 19 lines · Jan Beulich
Quoted 11 lines · Roger Pau Monné
Quoted 4 lines · Andrew Cooper
In principle we could assert that it's already NULL in
_domain_destroy()which might help catch an error if it gets set early but the domain destroyed before getting into the domlist, but that seems like a rather slim case.Given my understanding of the logic in the
XENMEM_hypercalls, I think toolstack can still target domains in the process of being destroyed, at which point we need to keep a final cleanup instance_domain_destroy(), or otherwise adjustXENMEM_populate_physmaphypercall (and others?) so they can't target dying domains.Considering that these requests will fail for dying domains because of the check in
assign_pages(), it may indeed make sense to have another earlier check for the purposes here. Otoh doing this early may not buy us very much, as the domain may become "dying" immediately after the check. Whereas switching stash_allocation()'sif()to
Quoted 3 lines
if ( d->pending_scrub || d->is_dying )looks like it might do.
Oh, I didn't notice the check in assign_pages().
I think a check in stash_allocation() together with a call to domain_pending_scrub_free() in domain_teardown() (which is after setting d->is_dying) should be enough to ensure the field is clear when reaching _domain_destroy(). The call to domain_pending_scrub_free() in _domain_destroy() can be replaced with an ASSERT(!d->pending_scrub) then.
Thanks, Roger.
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
In reply to Roger Pau MonneQuoted 7 lines · Roger Pau Monne
xen/common/memory.c +2
-1@@ -275,7 +339,18 @@ static void populate_physmap(struct memop_args *a) }else { - page =alloc_domheap_pages (d, a->extent_order, a->memflags); +unsigned int scrub_start =0 ; + nodeid_t node =
xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 353
Quoted 10 lines
xen/common/memory.c +10
-0+ (a->memflags & MEMF_exact_node) ?MEMF_get_node (a->memflags) + :NUMA_NO_NODE ; + + page =get_stashed_allocation (d, a->extent_order, node, + &scrub_start); + +if ( !page ) + page =alloc_domheap_pages (d, a->extent_order, + a->memflags | (d->creation_finished ?0 + : MEMF_no_scrub));
I fear there's a more basic issue here that so far we didn't pay attention to:alloc_domheap_pages() is what invokes assign_page(), which in turn resets ->count_info for each of the pages. This reset includes setting PGC_allocated, which ...
Quoted 4 lines
xen/common/memory.c +0
-0@@ -286,6 +361,30 @@ static void populate_physmap(struct memop_args *a) goto out; }
xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 373
Quoted 10 lines
xen/common/memory.c +10
-0+if ( !d->creation_finished ) + { +unsigned int dirty_cnt =0 ; + +/* Check if there's anything to scrub. */ +for ( j = scrub_start; j < (1U << a->extent_order); j++ ) + { +if ( !test_and_clear_bit (_PGC_need_scrub, + &page[j].count_info) ) +continue ;
... means we will now scrub every page in the block, not just those which weren't scrubbed yet, and we end up clearing PGC_allocated. All because of PGC_need_scrub aliasing PGC_allocated. I wonder how this didn't end up screwing any testing you surely will have done. Or maybe I'm completely off here?
Jan
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
Quoted 28 lines · Jan Beulich
Quoted 7 lines · Roger Pau Monne
xen/common/memory.c +2
-1@@ -275,7 +339,18 @@ static void populate_physmap(struct memop_args *a) }else { - page =alloc_domheap_pages (d, a->extent_order, a->memflags); +unsigned int scrub_start =0 ; + nodeid_t node =xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 353
Quoted 10 lines
xen/common/memory.c +10
-0+ (a->memflags & MEMF_exact_node) ?MEMF_get_node (a->memflags) + :NUMA_NO_NODE ; + + page =get_stashed_allocation (d, a->extent_order, node, + &scrub_start); + +if ( !page ) + page =alloc_domheap_pages (d, a->extent_order, + a->memflags | (d->creation_finished ?0 + : MEMF_no_scrub));I fear there's a more basic issue here that so far we didn't pay attention to:
alloc_domheap_pages()is what invokesassign_page(), which in turn resets ->count_info for each of the pages. This reset includes settingPGC_allocated, which ...Quoted 4 lines
xen/common/memory.c +0
-0@@ -286,6 +361,30 @@ static void populate_physmap(struct memop_args *a) goto out; }
Quoted 15 lines
xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 373
Quoted 10 lines
xen/common/memory.c +10
-0+if ( !d->creation_finished ) + { +unsigned int dirty_cnt =0 ; + +/* Check if there's anything to scrub. */ +for ( j = scrub_start; j < (1U << a->extent_order); j++ ) + { +if ( !test_and_clear_bit (_PGC_need_scrub, + &page[j].count_info) ) +continue ;... means we will now scrub every page in the block, not just those which weren't scrubbed yet, and we end up clearing
PGC_allocated. All because ofPGC_need_scrubaliasingPGC_allocated. I wonder how this didn't end up screwing any testing you surely will have done. Or maybe I'm completely off here?
Thanks for spotting this! No, I didn't see any issues. I don't see any check for PGC_allocated in free_domheap_pages(), which could explain the lack of failures?
I will have to allocate with MEMF_no_owner and then do the assign_pages() call from populate_physmap() after the scrubbing is done. Maybe that would work. Memory allocated using MEMF_no_owner still consumes the claim pool if a domain parameter is passed to alloc_heap_pages().
Roger.
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
Quoted 45 lines · Roger Pau Monné
Quoted 28 lines · Jan Beulich
Quoted 7 lines · Roger Pau Monne
xen/common/memory.c +2
-1@@ -275,7 +339,18 @@ static void populate_physmap(struct memop_args *a) }else { - page =alloc_domheap_pages (d, a->extent_order, a->memflags); +unsigned int scrub_start =0 ; + nodeid_t node =xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 353
Quoted 10 lines
xen/common/memory.c +10
-0+ (a->memflags & MEMF_exact_node) ?MEMF_get_node (a->memflags) + :NUMA_NO_NODE ; + + page =get_stashed_allocation (d, a->extent_order, node, + &scrub_start); + +if ( !page ) + page =alloc_domheap_pages (d, a->extent_order, + a->memflags | (d->creation_finished ?0 + : MEMF_no_scrub));I fear there's a more basic issue here that so far we didn't pay attention to:
alloc_domheap_pages()is what invokesassign_page(), which in turn resets ->count_info for each of the pages. This reset includes settingPGC_allocated, which ...Quoted 4 lines
xen/common/memory.c +0
-0@@ -286,6 +361,30 @@ static void populate_physmap(struct memop_args *a) goto out; }Quoted 15 lines
xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 373
Quoted 10 lines
xen/common/memory.c +10
-0+if ( !d->creation_finished ) + { +unsigned int dirty_cnt =0 ; + +/* Check if there's anything to scrub. */ +for ( j = scrub_start; j < (1U << a->extent_order); j++ ) + { +if ( !test_and_clear_bit (_PGC_need_scrub, + &page[j].count_info) ) +continue ;... means we will now scrub every page in the block, not just those which weren't scrubbed yet, and we end up clearing
PGC_allocated. All because ofPGC_need_scrubaliasingPGC_allocated. I wonder how this didn't end up screwing any testing you surely will have done. Or maybe I'm completely off here?
Quoted 3 lines
Thanks for spotting this! No, I didn't see any issues. I don't see any check for
PGC_allocatedinfree_domheap_pages(), which could explain the lack of failures?
Maybe. PGC_allocated consumes a page ref, so I would have expected accounting issues.
Quoted 5 lines
I will have to allocate with
MEMF_no_ownerand then do theassign_pages()call frompopulate_physmap()after the scrubbing is done. Maybe that would work. Memory allocated usingMEMF_no_ownerstill consumes the claim pool if a domain parameter is passed toalloc_heap_pages().
Technically this looks like it might work, but it's feeling as if this was getting increasingly fragile. I'm also not quite sure whether MEMF_no_owner allocations should consume claimed pages. Imo there are arguments both in favor and against such behavior.
We may want to explore the alternative of un-aliasing the two PGC_*.
Jan
Re: [PATCH v4 2/3] xen/mm: allow deferred scrub of physmap populate allocated pages
Quoted 59 lines · Jan Beulich
Quoted 45 lines · Roger Pau Monné
Quoted 28 lines · Jan Beulich
Quoted 7 lines · Roger Pau Monne
xen/common/memory.c +2
-1@@ -275,7 +339,18 @@ static void populate_physmap(struct memop_args *a) }else { - page =alloc_domheap_pages (d, a->extent_order, a->memflags); +unsigned int scrub_start =0 ; + nodeid_t node =xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 353
Quoted 10 lines
xen/common/memory.c +10
-0+ (a->memflags & MEMF_exact_node) ?MEMF_get_node (a->memflags) + :NUMA_NO_NODE ; + + page =get_stashed_allocation (d, a->extent_order, node, + &scrub_start); + +if ( !page ) + page =alloc_domheap_pages (d, a->extent_order, + a->memflags | (d->creation_finished ?0 + : MEMF_no_scrub));I fear there's a more basic issue here that so far we didn't pay attention to:
alloc_domheap_pages()is what invokesassign_page(), which in turn resets ->count_info for each of the pages. This reset includes settingPGC_allocated, which ...Quoted 4 lines
xen/common/memory.c +0
-0@@ -286,6 +361,30 @@ static void populate_physmap(struct memop_args *a) goto out; }Quoted 15 lines
xen/common/memory.c static void populate_physmap(struct memop_args *a) · line 373
Quoted 10 lines
xen/common/memory.c +10
-0+if ( !d->creation_finished ) + { +unsigned int dirty_cnt =0 ; + +/* Check if there's anything to scrub. */ +for ( j = scrub_start; j < (1U << a->extent_order); j++ ) + { +if ( !test_and_clear_bit (_PGC_need_scrub, + &page[j].count_info) ) +continue ;... means we will now scrub every page in the block, not just those which weren't scrubbed yet, and we end up clearing
PGC_allocated. All because ofPGC_need_scrubaliasingPGC_allocated. I wonder how this didn't end up screwing any testing you surely will have done. Or maybe I'm completely off here?Quoted 3 lines
Thanks for spotting this! No, I didn't see any issues. I don't see any check for
PGC_allocatedinfree_domheap_pages(), which could explain the lack of failures?Maybe.
PGC_allocatedconsumes a page ref, so I would have expected accounting issues.Quoted 5 lines
I will have to allocate with
MEMF_no_ownerand then do theassign_pages()call frompopulate_physmap()after the scrubbing is done. Maybe that would work. Memory allocated usingMEMF_no_ownerstill consumes the claim pool if a domain parameter is passed toalloc_heap_pages().
Quoted 6 lines
Technically this looks like it might work, but it's feeling as if this was getting increasingly fragile. I'm also not quite sure whether
MEMF_no_ownerallocations should consume claimed pages. Imo there are arguments both in favor and against such behavior.We may want to explore the alternative of un-aliasing the two PGC_*.
I expected the PGC_ bits would be all consumed, but I see there's a range that are still empty, so it might indeed be easier to remove the alias. Let me give that a try.
Thanks, Roger.
PATCH v4 3/3 xen/mm: limit non-scrubbed allocations to a specific order
In reply to Roger Pau MonneThe current logic allows for up to 1G pages to be scrubbed in place, which can cause the watchdog to trigger in practice. Reduce the limit for in-place scrubbed allocations to a newly introduced define:CONFIG_DIRTY_MAX_ORDER. This currently defaults to CONFIG_PTDOM_MAX_ORDER on all architectures. Also introduce a command line option to set the value.
---
Changes since v3:
- Note the order limitation is only post-boot.
Changes since v2:
- Move placement of the max-order-dirty option help.
- Add note in memop-max-order about interactions.
- Use CONFIG_PTDOM_MAX_ORDER as the default.
Changes since v1:
- Split from previous patch.
- Introduce a command line option to set the limit.
--- docs/misc/xen-command-line.pandoc | 13 +++++++++++++ xen/common/memory.c | 3 --- xen/common/page_alloc.c | 23 ++++++++++++++++++++++- xen/include/xen/mm.h | 4 ++++ 4 files changed, 39 insertions(+), 4 deletions(-)
docs/misc/xen-command-line.pandoc +13 -0
@@ -1837,6 +1837,16 @@ presented as the number of bits needed to encode it. This must be at least one pending bit to be allocated. Defaults to 20 bits (to cover at most 1048576 interrupts). +### max-order-dirty +> `= <integer>` + +Specify the maximum allocation order allowed when scrubbing allocated pages +in-place. The allocation is non-preemptive, and hence the value must be keep +low enough to avoid hogging the CPU for too long. + +Defaults to `CONFIG_DIRTY_MAX_ORDER` or if unset to `CONFIG_PTDOM_MAX_ORDER`. +Note those are internal per-architecture defines not available from Kconfig. + ### mce (x86) > `= <boolean>`
@@ -1878,6 +1888,9 @@ requests issued by the various kinds of domains (in this order: ordinary DomU, control domain, hardware domain, and - when supported by the platform - DomU with pass-through device assigned). +Note orders here can be further limited by the value in `max-order-dirty` for +allocations requesting pages to be scrubbed in-place. + ### mmcfg (x86) > `= <boolean>[,amd-fam10]`
xen/common/memory.c +0 -3
@@ -56,9 +56,6 @@ struct memop_args {#ifndef CONFIG_CTLDOM_MAX_ORDER #define CONFIG_CTLDOM_MAX_ORDER CONFIG_PAGEALLOC_MAX_ORDER#endif -#ifndef CONFIG_PTDOM_MAX_ORDER -#define CONFIG_PTDOM_MAX_ORDER CONFIG_HWDOM_MAX_ORDER -#endif static unsigned int __read_mostly domu_max_order =CONFIG_DOMU_MAX_ORDER ;static unsigned int __read_mostly ctldom_max_order =CONFIG_CTLDOM_MAX_ORDER ;
xen/common/page_alloc.c +22 -1
@@ -267,6 +267,13 @@ static PAGE_LIST_HEAD(page_offlined_list);/* Broken page list, protected by heap_lock. */ static PAGE_LIST_HEAD (page_broken_list); +/* Maximum order allowed for post-boot allocations with MEMF_no_scrub. */ +#ifndef CONFIG_DIRTY_MAX_ORDER +# define CONFIG_DIRTY_MAX_ORDER CONFIG_PTDOM_MAX_ORDER +#endif +static unsigned int __ro_after_init dirty_max_order =CONFIG_DIRTY_MAX_ORDER ; +integer_param ("max-order-dirty" , dirty_max_order); +/************************* * BOOT-TIME ALLOCATOR */
@@ -1008,7 +1015,13 @@ static struct page_info *alloc_heap_pages( pg =get_free_buddy (zone_lo, zone_hi, order, memflags, d);/* Try getting a dirty buddy if we couldn't get a clean one. */ -if ( !pg && !(memflags & MEMF_no_scrub) ) +if ( !pg && !(memflags & MEMF_no_scrub) && +/* +* Allow any order unscrubbed allocations during boot time, we +* compensate by processing softirqs in the scrubbing loop below once +* irqs are enabled. +*/ + (order <= dirty_max_order || system_state < SYS_STATE_active) ) pg =get_free_buddy (zone_lo, zone_hi, order, memflags | MEMF_no_scrub, d);if ( !pg )
@@ -1117,6 +1130,14 @@ static struct page_info *alloc_heap_pages(scrub_one_page (&pg[i], cold); dirty_cnt++; + +/* +* Use SYS_STATE_smp_boot explicitly; ahead of that state +* interrupts are disabled. +*/ +if ( system_state == SYS_STATE_smp_boot && + !(dirty_cnt &0xff ) ) +process_pending_softirqs (); }else check_one_page (&pg[i]);
xen/include/xen/mm.h +4 -0
@@ -232,6 +232,10 @@ struct npfec {#else #define MAX_ORDER 20/* 2^20 contiguous pages */ #endif +#ifndef CONFIG_PTDOM_MAX_ORDER +# define CONFIG_PTDOM_MAX_ORDER CONFIG_HWDOM_MAX_ORDER +#endif +mfn_t acquire_reserved_page (struct domain *d,unsigned int memflags);/* Private domain structs for DOMID_XEN, DOMID_IO, etc. */
Signature · 1 line
2.51.0