xen/riscv: introduce p2m functionality
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
In reply to Oleksii KurochkoQuoted 127 lines · Oleksii Kurochko
Quoted 62 lines · Jan Beulich
Quoted 53 lines · Oleksii Kurochko
Quoted 52 lines · Jan Beulich
Quoted 28 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Quoted 14 lines · Jan Beulich
xen/arch/riscv/p2m.c static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn) · line 348
Quoted 7 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +1
-0@@ -345,6 +345,26 @@ static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn)return __map_domain_page (p2m->root + root_table_indx); } +static int p2m_type_radix_set (struct p2m_domain *p2m,pte_t pte,p2m_type_t t)See comments on the earlier patch regarding naming.
xen/arch/riscv/p2m.c static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn) · line 351
Quoted 3 lines
xen/arch/riscv/p2m.c +3
-0+{ +int rc; +gfn_t gfn =mfn_to_gfn (p2m->domain,mfn_from_pte (pte));How does this work, when you record GFNs only for Xenheap pages?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think I don't understand what is an issue. Could you please provide some extra details?
Counter question: The
mfn_to_gfn()you currently have is only a stub. It only works for 1:1 mapped domains. Can you show me the eventual final implementation of the function, making it possible to use it here?At the moment, I planned to support only 1:1 mapped domains, so it is final implementation.
Isn't that on overly severe limitation?
I wouldn't say that it's a severe limitation, as it's just a matter of how
Quoted 8 lines
|mfn_to_gfn()| is implemented. When non-1:1 mapped domains are supported, |mfn_to_gfn()| can be implemented differently, while the code where it’s calledwill likely remain unchanged.
What I meant in my reply is that, for the current state and current limitations, this is the final implementation of|mfn_to_gfn()|. But that doesn't mean I don't see the value in, or the need for, non-1:1 mapped domains—it's just that this limitation simplifies development at the current stage of the RISC-V port.
Simplification is fine in some cases, but not supporting the "normal" way of domain construction looks like a pretty odd restriction. I'm also curious how you envision to implement
mfn_to_gfn()then, suitable for generic use like the one here. Imo, current limitation or not, you simply want to avoid use of that function outside of the special gnttab case.Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
In this context (not sure if I asked before): With this use of a radix tree, how do you intend to bound the amount of memory that a domain can use, by making Xen insert very many entries?
I didn’t think about that. I assumed it would be enough to set the amount of memory a guest domain can use by specifying|xen,domain-p2m-mem-mb| in the DTS, or using some predefined value if|xen,domain-p2m-mem-mb| isn’t explicitly set.
Which would require these allocations to come from that pool.
Yes, and it is true only for non-hardware domains with the current implementation.
???
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
I meant that pool is used now only for non-hardware domains at the moment.
And how does this matter here? The memory required for the radix tree doesn't come from that pool anyway.
I thought that is possible to do that somehow, but looking at a code of
radix-tree.cit seems like the only one way to allocate memroy for the radix treeisradix_tree_node_alloc()->xzalloc(struct rcu_node).Then it is needed to introduce
radix_tree_node_allocate(domain)Quoted 8 lines
That would be a possibility, but you may have seen that less than half a year ago we got rid of something along these lines. So it would require some pretty good justification to re-introduce.
Quoted 2 lines
or radix tree
can't be used at all for mentioned in the previous replies security reason, no?(Very) careful use may still be possible. But the downside of using this (potentially long lookup times) would always remain.
Could you please clarify what do you mean here by "(Very) careful"?
I thought about an introduction of an amount of possible keys in radix tree and if this amount is 0 then stop domain. And it is also unclear what should be a value for this amount. Probably, you have better idea.But generally your idea below ...
Quoted 42 lines
Quoted 20 lines
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines
Also, it seems this would just lead to the issue you mentioned earlier: when the memory runs out,|domain_crash()| will be called or PTE will be zapped.
Or one domain exhausting memory would cause another domain to fail. A domain impacting just itself may be tolerable. But a domain affecting other domains isn't.
But it seems like this issue could happen in any implementation. It won't happen only if we will have only pre-populated pool for any domain type (hardware, control, guest domain) without ability to extend them or allocate extra pages from domheap in runtime. Otherwise, if extra pages allocation is allowed then we can't really do something with this issue.
Quoted 4 lines
Quoted 2 lines
But that's why I brought this up: You simply have to. Or, as indicated, the moment you mark Xen security-supported on RISC-V, there will be an XSA needed.
Why it isn't XSA for other architectures? At least, Arm then should have such XSA.
Does Arm use a radix tree for storing types? It uses one for mem-access, but it's not clear to me whether that's actually a supported feature.
Quoted 3 lines
I don't understand why x86 won't have the same issue. Memory is the limited and shared resource, so if one of the domain will use to much memory then it could happen that other domains won't have enough memory for its purpose...
Quoted 7 lines
Quoted 4 lines
The question is whether allocations are bounded. With this use of a radix tree, you give domains a way to have Xen allocate pretty much arbitrary amounts of memory to populate that tree. That unbounded-ness is the problem, not memory allocations in general.
Isn't radix tree key bounded to an amount of GFNs given for a domain? We can't have more keys then a max GFN number for a domain. So a potential amount of necessary memory for radix tree is also bounded to an amount of GFNs.
To some degree yes, hence why I said "pretty much arbitrary amounts". But recall that "amount of GFNs" is a fuzzy term; I think you mean to use it to describe the amount of memory pages given to the guest. GFNs can be used for other purposes, though. Guests could e.g. grant themselves access to their own memory, then map those grants at otherwise unused GFNs.
Quoted 3 lines
Anyway, IIUC I just can't use radix tree for p2m types at all, right?
If yes, does it make sense to borrow 2 bits from structpage_info->type_infoas now it is used 9-bits for count of a frame?struct
page_infodescribes MFNs, when you want to describe GFNs. As you mentioned earlier, multiple GFNs can in principle map to the same MFN. You would force them to all have the same properties, which would be in direct conflict with e.g. the grant P2M types.Quoted 6 lines
Just to mention one possible alternative to using radix trees: You could maintain a 2nd set of intermediate "page tables", just that leaf entries would hold meta data for the respective GFN. The memory for those "page tables" could come from the normal P2M pool (and allocation would thus only consume domain-specific resources). Of course in any model like this the question of lookup times (as mentioned above) would remain.
Quoted 8 lines
...looks like an optimal option.
The only thing I worry about is that it will require some code duplication (I will think how to re-use the current one code), as for example, when setting/getting metadata, TLB flushing isn’t needed at all as we aren't working with with real P2M page tables.
Agree that lookup won't be the best one, but nothing can be done with such models.
Probably, instead of having a second set of intermediate "page tables", we could just allocate two consecutive pages within the real P2M page tables for the intermediate page table. The first page would serve as the actual page table to which the intermediate page table points, and the second page would store metadata for each entry of the page table that the intermediate page table references.
As we are supporting only 1gb, 2mb and 4kb mappings we could do a little optimization and start allocate these consecutive pages only for PT levels which corresponds to 1gb, 2mb, 4kb mappings.
Does it make sense?
~ Oleksii
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 145 lines · Oleksii Kurochko
Quoted 127 lines · Oleksii Kurochko
Quoted 62 lines · Jan Beulich
Quoted 53 lines · Oleksii Kurochko
Quoted 52 lines · Jan Beulich
Quoted 28 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Quoted 14 lines · Jan Beulich
xen/arch/riscv/p2m.c static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn) · line 348
Quoted 7 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +1
-0@@ -345,6 +345,26 @@ static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn)return __map_domain_page (p2m->root + root_table_indx); } +static int p2m_type_radix_set (struct p2m_domain *p2m,pte_t pte,p2m_type_t t)See comments on the earlier patch regarding naming.
xen/arch/riscv/p2m.c static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn) · line 351
Quoted 3 lines
xen/arch/riscv/p2m.c +3
-0+{ +int rc; +gfn_t gfn =mfn_to_gfn (p2m->domain,mfn_from_pte (pte));How does this work, when you record GFNs only for Xenheap pages?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think I don't understand what is an issue. Could you please provide some extra details?
Counter question: The
mfn_to_gfn()you currently have is only a stub. It only works for 1:1 mapped domains. Can you show me the eventual final implementation of the function, making it possible to use it here?At the moment, I planned to support only 1:1 mapped domains, so it is final implementation.
Isn't that on overly severe limitation?
I wouldn't say that it's a severe limitation, as it's just a matter of how
Quoted 8 lines
|mfn_to_gfn()| is implemented. When non-1:1 mapped domains are supported, |mfn_to_gfn()| can be implemented differently, while the code where it’s calledwill likely remain unchanged.
What I meant in my reply is that, for the current state and current limitations, this is the final implementation of|mfn_to_gfn()|. But that doesn't mean I don't see the value in, or the need for, non-1:1 mapped domains—it's just that this limitation simplifies development at the current stage of the RISC-V port.
Simplification is fine in some cases, but not supporting the "normal" way of domain construction looks like a pretty odd restriction. I'm also curious how you envision to implement
mfn_to_gfn()then, suitable for generic use like the one here. Imo, current limitation or not, you simply want to avoid use of that function outside of the special gnttab case.Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
In this context (not sure if I asked before): With this use of a radix tree, how do you intend to bound the amount of memory that a domain can use, by making Xen insert very many entries?
I didn’t think about that. I assumed it would be enough to set the amount of memory a guest domain can use by specifying|xen,domain-p2m-mem-mb| in the DTS, or using some predefined value if|xen,domain-p2m-mem-mb| isn’t explicitly set.
Which would require these allocations to come from that pool.
Yes, and it is true only for non-hardware domains with the current implementation.
???
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
I meant that pool is used now only for non-hardware domains at the moment.
And how does this matter here? The memory required for the radix tree doesn't come from that pool anyway.
I thought that is possible to do that somehow, but looking at a code of
radix-tree.cit seems like the only one way to allocate memroy for the radix treeisradix_tree_node_alloc()->xzalloc(struct rcu_node).Then it is needed to introduce
radix_tree_node_allocate(domain)Quoted 8 lines
That would be a possibility, but you may have seen that less than half a year ago we got rid of something along these lines. So it would require some pretty good justification to re-introduce.
Quoted 2 lines
or radix tree
can't be used at all for mentioned in the previous replies security reason, no?(Very) careful use may still be possible. But the downside of using this (potentially long lookup times) would always remain.
Could you please clarify what do you mean here by "(Very) careful"?
I thought about an introduction of an amount of possible keys in radix tree and if this amount is 0 then stop domain. And it is also unclear what should be a value for this amount. Probably, you have better idea.But generally your idea below ...
Quoted 42 lines
Quoted 20 lines
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines
Also, it seems this would just lead to the issue you mentioned earlier: when the memory runs out,|domain_crash()| will be called or PTE will be zapped.
Or one domain exhausting memory would cause another domain to fail. A domain impacting just itself may be tolerable. But a domain affecting other domains isn't.
But it seems like this issue could happen in any implementation. It won't happen only if we will have only pre-populated pool for any domain type (hardware, control, guest domain) without ability to extend them or allocate extra pages from domheap in runtime. Otherwise, if extra pages allocation is allowed then we can't really do something with this issue.
Quoted 4 lines
Quoted 2 lines
But that's why I brought this up: You simply have to. Or, as indicated, the moment you mark Xen security-supported on RISC-V, there will be an XSA needed.
Why it isn't XSA for other architectures? At least, Arm then should have such XSA.
Does Arm use a radix tree for storing types? It uses one for mem-access, but it's not clear to me whether that's actually a supported feature.
Quoted 3 lines
I don't understand why x86 won't have the same issue. Memory is the limited and shared resource, so if one of the domain will use to much memory then it could happen that other domains won't have enough memory for its purpose...
Quoted 7 lines
Quoted 4 lines
The question is whether allocations are bounded. With this use of a radix tree, you give domains a way to have Xen allocate pretty much arbitrary amounts of memory to populate that tree. That unbounded-ness is the problem, not memory allocations in general.
Isn't radix tree key bounded to an amount of GFNs given for a domain? We can't have more keys then a max GFN number for a domain. So a potential amount of necessary memory for radix tree is also bounded to an amount of GFNs.
To some degree yes, hence why I said "pretty much arbitrary amounts". But recall that "amount of GFNs" is a fuzzy term; I think you mean to use it to describe the amount of memory pages given to the guest. GFNs can be used for other purposes, though. Guests could e.g. grant themselves access to their own memory, then map those grants at otherwise unused GFNs.
Quoted 3 lines
Anyway, IIUC I just can't use radix tree for p2m types at all, right?
If yes, does it make sense to borrow 2 bits from structpage_info->type_infoas now it is used 9-bits for count of a frame?struct
page_infodescribes MFNs, when you want to describe GFNs. As you mentioned earlier, multiple GFNs can in principle map to the same MFN. You would force them to all have the same properties, which would be in direct conflict with e.g. the grant P2M types.Quoted 6 lines
Just to mention one possible alternative to using radix trees: You could maintain a 2nd set of intermediate "page tables", just that leaf entries would hold meta data for the respective GFN. The memory for those "page tables" could come from the normal P2M pool (and allocation would thus only consume domain-specific resources). Of course in any model like this the question of lookup times (as mentioned above) would remain.
Quoted 8 lines
...looks like an optimal option.
The only thing I worry about is that it will require some code duplication (I will think how to re-use the current one code), as for example, when setting/getting metadata, TLB flushing isn’t needed at all as we aren't working with with real P2M page tables.
Agree that lookup won't be the best one, but nothing can be done with such models.Probably, instead of having a second set of intermediate "page tables", we could just allocate two consecutive pages within the real P2M page tables for the intermediate page table. The first page would serve as the actual page table to which the intermediate page table points, and the second page would store metadata for each entry of the page table that the intermediate page table references.
Quoted 5 lines
As we are supporting only 1gb, 2mb and 4kb mappings we could do a little optimization and start allocate these consecutive pages only for PT levels which corresponds to 1gb, 2mb, 4kb mappings.
Does it make sense?
I was indeed entertaining this idea, but I couldn't conclude for myself if that would indeed be without any rough edges. Hence I didn't want to suggest such. For example, the need to have adjacent pairs of pages could result in a higher rate of allocation failures (while populating or re-sizing the P2M pool). This would be possible to avoid by still using entirely separate pages, and then merely linking them together via some unused struct page_info fields (the "normal" linking fields can't be used, afaict).
Jan
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 148 lines · Jan Beulich
Quoted 142 lines · Oleksii Kurochko
Quoted 126 lines · Oleksii Kurochko
Quoted 62 lines · Jan Beulich
Quoted 53 lines · Oleksii Kurochko
Quoted 52 lines · Jan Beulich
Quoted 28 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Quoted 14 lines · Jan Beulich
Quoted 7 lines · Oleksii Kurochko
--- a/xen/arch/riscv/p2m.c
+++ b/xen/arch/riscv/p2m.c
@@ -345,6 +345,26 @@ static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn)
return __map_domain_page(p2m->root + root_table_indx);
}+static int p2m_type_radix_set(struct p2m_domain *p2m, pte_t pte, p2m_type_t t)
See comments on the earlier patch regarding naming.
Quoted 3 lines
+{
+ int rc;
+ gfn_t gfn = mfn_to_gfn(p2m->domain, mfn_from_pte(pte));How does this work, when you record GFNs only for Xenheap pages?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think I don't understand what is an issue. Could you please provide some extra details?
Counter question: The
mfn_to_gfn()you currently have is only a stub. It only works for 1:1 mapped domains. Can you show me the eventual final implementation of the function, making it possible to use it here?At the moment, I planned to support only 1:1 mapped domains, so it is final implementation.
Isn't that on overly severe limitation?
I wouldn't say that it's a severe limitation, as it's just a matter of how
Quoted 8 lines
|mfn_to_gfn()| is implemented. When non-1:1 mapped domains are supported, |mfn_to_gfn()| can be implemented differently, while the code where it’s calledwill likely remain unchanged.
What I meant in my reply is that, for the current state and current limitations, this is the final implementation of|mfn_to_gfn()|. But that doesn't mean I don't see the value in, or the need for, non-1:1 mapped domains—it's just that this limitation simplifies development at the current stage of the RISC-V port.
Simplification is fine in some cases, but not supporting the "normal" way of domain construction looks like a pretty odd restriction. I'm also curious how you envision to implement
mfn_to_gfn()then, suitable for generic use like the one here. Imo, current limitation or not, you simply want to avoid use of that function outside of the special gnttab case.Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
In this context (not sure if I asked before): With this use of a radix tree, how do you intend to bound the amount of memory that a domain can use, by making Xen insert very many entries?
I didn’t think about that. I assumed it would be enough to set the amount of memory a guest domain can use by specifying|xen,domain-p2m-mem-mb| in the DTS, or using some predefined value if|xen,domain-p2m-mem-mb| isn’t explicitly set.
Which would require these allocations to come from that pool.
Yes, and it is true only for non-hardware domains with the current implementation.
???
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
I meant that pool is used now only for non-hardware domains at the moment.
And how does this matter here? The memory required for the radix tree doesn't come from that pool anyway.
I thought that is possible to do that somehow, but looking at a code of
radix-tree.cit seems like the only one way to allocate memroy for the radix treeisradix_tree_node_alloc()->xzalloc(struct rcu_node).Then it is needed to introduce
radix_tree_node_allocate(domain)Quoted 8 lines
That would be a possibility, but you may have seen that less than half a year ago we got rid of something along these lines. So it would require some pretty good justification to re-introduce.
Quoted 2 lines
or radix tree
can't be used at all for mentioned in the previous replies security reason, no?(Very) careful use may still be possible. But the downside of using this (potentially long lookup times) would always remain.
Could you please clarify what do you mean here by "(Very) careful"?
I thought about an introduction of an amount of possible keys in radix tree and if this amount is 0 then stop domain. And it is also unclear what should be a value for this amount. Probably, you have better idea.But generally your idea below ...
Quoted 42 lines
Quoted 20 lines
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines
Also, it seems this would just lead to the issue you mentioned earlier: when the memory runs out,|domain_crash()| will be called or PTE will be zapped.
Or one domain exhausting memory would cause another domain to fail. A domain impacting just itself may be tolerable. But a domain affecting other domains isn't.
But it seems like this issue could happen in any implementation. It won't happen only if we will have only pre-populated pool for any domain type (hardware, control, guest domain) without ability to extend them or allocate extra pages from domheap in runtime. Otherwise, if extra pages allocation is allowed then we can't really do something with this issue.
Quoted 4 lines
Quoted 2 lines
But that's why I brought this up: You simply have to. Or, as indicated, the moment you mark Xen security-supported on RISC-V, there will be an XSA needed.
Why it isn't XSA for other architectures? At least, Arm then should have such XSA.
Does Arm use a radix tree for storing types? It uses one for mem-access, but it's not clear to me whether that's actually a supported feature.
Quoted 3 lines
I don't understand why x86 won't have the same issue. Memory is the limited and shared resource, so if one of the domain will use to much memory then it could happen that other domains won't have enough memory for its purpose...
Quoted 7 lines
Quoted 4 lines
The question is whether allocations are bounded. With this use of a radix tree, you give domains a way to have Xen allocate pretty much arbitrary amounts of memory to populate that tree. That unbounded-ness is the problem, not memory allocations in general.
Isn't radix tree key bounded to an amount of GFNs given for a domain? We can't have more keys then a max GFN number for a domain. So a potential amount of necessary memory for radix tree is also bounded to an amount of GFNs.
To some degree yes, hence why I said "pretty much arbitrary amounts". But recall that "amount of GFNs" is a fuzzy term; I think you mean to use it to describe the amount of memory pages given to the guest. GFNs can be used for other purposes, though. Guests could e.g. grant themselves access to their own memory, then map those grants at otherwise unused GFNs.
Quoted 3 lines
Anyway, IIUC I just can't use radix tree for p2m types at all, right?
If yes, does it make sense to borrow 2 bits from structpage_info->type_infoas now it is used 9-bits for count of a frame?struct
page_infodescribes MFNs, when you want to describe GFNs. As you mentioned earlier, multiple GFNs can in principle map to the same MFN. You would force them to all have the same properties, which would be in direct conflict with e.g. the grant P2M types.Quoted 6 lines
Just to mention one possible alternative to using radix trees: You could maintain a 2nd set of intermediate "page tables", just that leaf entries would hold meta data for the respective GFN. The memory for those "page tables" could come from the normal P2M pool (and allocation would thus only consume domain-specific resources). Of course in any model like this the question of lookup times (as mentioned above) would remain.
Quoted 8 lines
...looks like an optimal option.
The only thing I worry about is that it will require some code duplication (I will think how to re-use the current one code), as for example, when setting/getting metadata, TLB flushing isn’t needed at all as we aren't working with with real P2M page tables.
Agree that lookup won't be the best one, but nothing can be done with such models.Probably, instead of having a second set of intermediate "page tables", we could just allocate two consecutive pages within the real P2M page tables for the intermediate page table. The first page would serve as the actual page table to which the intermediate page table points, and the second page would store metadata for each entry of the page table that the intermediate page table references.
Quoted 5 lines
As we are supporting only 1gb, 2mb and 4kb mappings we could do a little optimization and start allocate these consecutive pages only for PT levels which corresponds to 1gb, 2mb, 4kb mappings.
Does it make sense?
Quoted 8 lines
I was indeed entertaining this idea, but I couldn't conclude for myself if that would indeed be without any rough edges. Hence I didn't want to suggest such. For example, the need to have adjacent pairs of pages could result in a higher rate of allocation failures (while populating or re-sizing the P2M pool). This would be possible to avoid by still using entirely separate pages, and then merely linking them together via some unused struct
page_infofields (the "normal" linking fields can't be used, afaict).
I think that all the fields are used, so it will be needed to introduce new "struct page_list_entry metadata_list;".
Can't we introduce new PGT_METADATA type and then add metadata page to struct page_info->list and when a metadata will be needed just iterate through page_info->list and find a page with PGT_METADATA type?
~ Oleksii
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 157 lines · Oleksii Kurochko
Quoted 147 lines · Jan Beulich
Quoted 141 lines · Oleksii Kurochko
Quoted 125 lines · Oleksii Kurochko
Quoted 62 lines · Jan Beulich
Quoted 53 lines · Oleksii Kurochko
Quoted 52 lines · Jan Beulich
Quoted 28 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Quoted 14 lines · Jan Beulich
On 10.06.2025 15:05, Oleksii Kurochko wrote:
--- a/xen/arch/riscv/p2m.c
+++ b/xen/arch/riscv/p2m.c
@@ -345,6 +345,26 @@ static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn)
return __map_domain_page(p2m->root + root_table_indx);
}+static int p2m_type_radix_set(struct p2m_domain *p2m, pte_t pte, p2m_type_t t)
See comments on the earlier patch regarding naming.+{
+ int rc;
+ gfn_t gfn = mfn_to_gfn(p2m->domain, mfn_from_pte(pte));
How does this work, when you record GFNs only for Xenheap pages?Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think I don't understand what is an issue. Could you please provide some extra details?
Counter question: The
mfn_to_gfn()you currently have is only a stub. It only works for 1:1 mapped domains. Can you show me the eventual final implementation of the function, making it possible to use it here?At the moment, I planned to support only 1:1 mapped domains, so it is final implementation.
Isn't that on overly severe limitation?
I wouldn't say that it's a severe limitation, as it's just a matter of how
Quoted 8 lines
|mfn_to_gfn()| is implemented. When non-1:1 mapped domains are supported, |mfn_to_gfn()| can be implemented differently, while the code where it’s calledwill likely remain unchanged.
What I meant in my reply is that, for the current state and current limitations, this is the final implementation of|mfn_to_gfn()|. But that doesn't mean I don't see the value in, or the need for, non-1:1 mapped domains—it's just that this limitation simplifies development at the current stage of the RISC-V port.
Simplification is fine in some cases, but not supporting the "normal" way of domain construction looks like a pretty odd restriction. I'm also curious how you envision to implement
mfn_to_gfn()then, suitable for generic use like the one here. Imo, current limitation or not, you simply want to avoid use of that function outside of the special gnttab case.Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
In this context (not sure if I asked before): With this use of a radix tree,
how do you intend to bound the amount of memory that a domain can use, by
making Xen insert very many entries?I didn’t think about that. I assumed it would be enough to set the amount of memory a guest domain can use by specifying|xen,domain-p2m-mem-mb| in the DTS, or using some predefined value if|xen,domain-p2m-mem-mb| isn’t explicitly set.
Which would require these allocations to come from that pool.
Yes, and it is true only for non-hardware domains with the current implementation.
???
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
I meant that pool is used now only for non-hardware domains at the moment.
And how does this matter here? The memory required for the radix tree doesn't come from that pool anyway.
I thought that is possible to do that somehow, but looking at a code of
radix-tree.cit seems like the only one way to allocate memroy for the radix treeisradix_tree_node_alloc()->xzalloc(struct rcu_node).Then it is needed to introduce
radix_tree_node_allocate(domain)Quoted 8 lines
That would be a possibility, but you may have seen that less than half a year ago we got rid of something along these lines. So it would require some pretty good justification to re-introduce.
Quoted 2 lines
or radix tree
can't be used at all for mentioned in the previous replies security reason, no?(Very) careful use may still be possible. But the downside of using this (potentially long lookup times) would always remain.
Could you please clarify what do you mean here by "(Very) careful"?
I thought about an introduction of an amount of possible keys in radix tree and if this amount is 0 then stop domain. And it is also unclear what should be a value for this amount. Probably, you have better idea.But generally your idea below ...
Quoted 42 lines
Quoted 20 lines
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines
Also, it seems this would just lead to the issue you mentioned earlier: when the memory runs out,|domain_crash()| will be called or PTE will be zapped.
Or one domain exhausting memory would cause another domain to fail. A domain impacting just itself may be tolerable. But a domain affecting other domains isn't.
But it seems like this issue could happen in any implementation. It won't happen only if we will have only pre-populated pool for any domain type (hardware, control, guest domain) without ability to extend them or allocate extra pages from domheap in runtime. Otherwise, if extra pages allocation is allowed then we can't really do something with this issue.
Quoted 4 lines
Quoted 2 lines
But that's why I brought this up: You simply have to. Or, as indicated, the moment you mark Xen security-supported on RISC-V, there will be an XSA needed.
Why it isn't XSA for other architectures? At least, Arm then should have such XSA.
Does Arm use a radix tree for storing types? It uses one for mem-access, but it's not clear to me whether that's actually a supported feature.
Quoted 3 lines
I don't understand why x86 won't have the same issue. Memory is the limited and shared resource, so if one of the domain will use to much memory then it could happen that other domains won't have enough memory for its purpose...
Quoted 7 lines
Quoted 4 lines
The question is whether allocations are bounded. With this use of a radix tree, you give domains a way to have Xen allocate pretty much arbitrary amounts of memory to populate that tree. That unbounded-ness is the problem, not memory allocations in general.
Isn't radix tree key bounded to an amount of GFNs given for a domain? We can't have more keys then a max GFN number for a domain. So a potential amount of necessary memory for radix tree is also bounded to an amount of GFNs.
To some degree yes, hence why I said "pretty much arbitrary amounts". But recall that "amount of GFNs" is a fuzzy term; I think you mean to use it to describe the amount of memory pages given to the guest. GFNs can be used for other purposes, though. Guests could e.g. grant themselves access to their own memory, then map those grants at otherwise unused GFNs.
Quoted 3 lines
Anyway, IIUC I just can't use radix tree for p2m types at all, right?
If yes, does it make sense to borrow 2 bits from structpage_info->type_infoas now it is used 9-bits for count of a frame?struct
page_infodescribes MFNs, when you want to describe GFNs. As you mentioned earlier, multiple GFNs can in principle map to the same MFN. You would force them to all have the same properties, which would be in direct conflict with e.g. the grant P2M types.Quoted 6 lines
Just to mention one possible alternative to using radix trees: You could maintain a 2nd set of intermediate "page tables", just that leaf entries would hold meta data for the respective GFN. The memory for those "page tables" could come from the normal P2M pool (and allocation would thus only consume domain-specific resources). Of course in any model like this the question of lookup times (as mentioned above) would remain.
Quoted 8 lines
...looks like an optimal option.
The only thing I worry about is that it will require some code duplication (I will think how to re-use the current one code), as for example, when setting/getting metadata, TLB flushing isn’t needed at all as we aren't working with with real P2M page tables.
Agree that lookup won't be the best one, but nothing can be done with such models.Probably, instead of having a second set of intermediate "page tables", we could just allocate two consecutive pages within the real P2M page tables for the intermediate page table. The first page would serve as the actual page table to which the intermediate page table points, and the second page would store metadata for each entry of the page table that the intermediate page table references.
Quoted 5 lines
As we are supporting only 1gb, 2mb and 4kb mappings we could do a little optimization and start allocate these consecutive pages only for PT levels which corresponds to 1gb, 2mb, 4kb mappings.
Does it make sense?
Quoted 8 lines
I was indeed entertaining this idea, but I couldn't conclude for myself if that would indeed be without any rough edges. Hence I didn't want to suggest such. For example, the need to have adjacent pairs of pages could result in a higher rate of allocation failures (while populating or re-sizing the P2M pool). This would be possible to avoid by still using entirely separate pages, and then merely linking them together via some unused struct
page_infofields (the "normal" linking fields can't be used, afaict).
Quoted 2 lines
I think that all the fields are used, so it will be needed to introduce new "struct
page_list_entrymetadata_list;".
All the fields are used _somewhere_, sure. But once you have allocated a page (and that page isn't assigned to a domain), you control what the fields are used for. Or else enlisting pages on private lists wouldn't be legitimate either.
Quoted 3 lines
Can't we introduce new
PGT_METADATAtype and then add metadata page to structpage_info->listand when a metadata will be needed just iterate throughpage_info->listand find a page withPGT_METADATAtype?
I'd be careful with the introduction of new page types. All handling of page types everywhere in the (affected part of the) code base would then need auditing.
Jan
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 157 lines · Jan Beulich
Quoted 156 lines · Oleksii Kurochko
Quoted 155 lines · Jan Beulich
Quoted 141 lines · Oleksii Kurochko
Quoted 125 lines · Oleksii Kurochko
Quoted 62 lines · Jan Beulich
Quoted 53 lines · Oleksii Kurochko
Quoted 52 lines · Jan Beulich
Quoted 28 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
On 7/1/25 5:08 PM, Jan Beulich wrote:
On 10.06.2025 15:05, Oleksii Kurochko wrote:
--- a/xen/arch/riscv/p2m.c
+++ b/xen/arch/riscv/p2m.c
@@ -345,6 +345,26 @@ static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn)
return __map_domain_page(p2m->root + root_table_indx);
}+static int p2m_type_radix_set(struct p2m_domain *p2m, pte_t pte, p2m_type_t t)
See comments on the earlier patch regarding naming.+{
+ int rc;
+ gfn_t gfn = mfn_to_gfn(p2m->domain, mfn_from_pte(pte));
How does this work, when you record GFNs only for Xenheap pages?Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think I don't understand what is an issue. Could you please provide
some extra details?Counter question: The
mfn_to_gfn()you currently have is only a stub. It only works for 1:1 mapped domains. Can you show me the eventual final implementation of the function, making it possible to use it here?At the moment, I planned to support only 1:1 mapped domains, so it is final implementation.
Isn't that on overly severe limitation?
I wouldn't say that it's a severe limitation, as it's just a matter of how
Quoted 8 lines
|mfn_to_gfn()| is implemented. When non-1:1 mapped domains are supported, |mfn_to_gfn()| can be implemented differently, while the code where it’s calledwill likely remain unchanged.
What I meant in my reply is that, for the current state and current limitations, this is the final implementation of|mfn_to_gfn()|. But that doesn't mean I don't see the value in, or the need for, non-1:1 mapped domains—it's just that this limitation simplifies development at the current stage of the RISC-V port.
Simplification is fine in some cases, but not supporting the "normal" way of domain construction looks like a pretty odd restriction. I'm also curious how you envision to implement
mfn_to_gfn()then, suitable for generic use like the one here. Imo, current limitation or not, you simply want to avoid use of that function outside of the special gnttab case.Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
In this context (not sure if I asked before): With this use of a radix tree,
how do you intend to bound the amount of memory that a domain can use, by
making Xen insert very many entries?
I didn’t think about that. I assumed it would be enough to set the amount of
memory a guest domain can use by specifying|xen,domain-p2m-mem-mb| in the DTS,
or using some predefined value if|xen,domain-p2m-mem-mb| isn’t explicitly set.Which would require these allocations to come from that pool.
Yes, and it is true only for non-hardware domains with the current implementation.
???
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
I meant that pool is used now only for non-hardware domains at the moment.
And how does this matter here? The memory required for the radix tree doesn't come from that pool anyway.
I thought that is possible to do that somehow, but looking at a code of
radix-tree.cit seems like the only one way to allocate memroy for the radix treeisradix_tree_node_alloc()->xzalloc(struct rcu_node).Then it is needed to introduce
radix_tree_node_allocate(domain)Quoted 8 lines
That would be a possibility, but you may have seen that less than half a year ago we got rid of something along these lines. So it would require some pretty good justification to re-introduce.
Quoted 2 lines
or radix tree
can't be used at all for mentioned in the previous replies security reason, no?(Very) careful use may still be possible. But the downside of using this (potentially long lookup times) would always remain.
Could you please clarify what do you mean here by "(Very) careful"?
I thought about an introduction of an amount of possible keys in radix tree and if this amount is 0 then stop domain. And it is also unclear what should be a value for this amount. Probably, you have better idea.But generally your idea below ...
Quoted 42 lines
Quoted 20 lines
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Quoted 2 lines
Also, it seems this would just lead to the issue you mentioned earlier: when
the memory runs out,|domain_crash()| will be called or PTE will be zapped.Or one domain exhausting memory would cause another domain to fail. A domain impacting just itself may be tolerable. But a domain affecting other domains isn't.
But it seems like this issue could happen in any implementation. It won't happen only if we will have only pre-populated pool for any domain type (hardware, control, guest domain) without ability to extend them or allocate extra pages from domheap in runtime. Otherwise, if extra pages allocation is allowed then we can't really do something with this issue.
Quoted 4 lines
Quoted 2 lines
But that's why I brought this up: You simply have to. Or, as indicated, the moment you mark Xen security-supported on RISC-V, there will be an XSA needed.
Why it isn't XSA for other architectures? At least, Arm then should have such XSA.
Does Arm use a radix tree for storing types? It uses one for mem-access, but it's not clear to me whether that's actually a supported feature.
Quoted 3 lines
I don't understand why x86 won't have the same issue. Memory is the limited and shared resource, so if one of the domain will use to much memory then it could happen that other domains won't have enough memory for its purpose...
Quoted 7 lines
Quoted 4 lines
The question is whether allocations are bounded. With this use of a radix tree, you give domains a way to have Xen allocate pretty much arbitrary amounts of memory to populate that tree. That unbounded-ness is the problem, not memory allocations in general.
Isn't radix tree key bounded to an amount of GFNs given for a domain? We can't have more keys then a max GFN number for a domain. So a potential amount of necessary memory for radix tree is also bounded to an amount of GFNs.
To some degree yes, hence why I said "pretty much arbitrary amounts". But recall that "amount of GFNs" is a fuzzy term; I think you mean to use it to describe the amount of memory pages given to the guest. GFNs can be used for other purposes, though. Guests could e.g. grant themselves access to their own memory, then map those grants at otherwise unused GFNs.
Quoted 3 lines
Anyway, IIUC I just can't use radix tree for p2m types at all, right?
If yes, does it make sense to borrow 2 bits from structpage_info->type_infoas now it is used 9-bits for count of a frame?struct
page_infodescribes MFNs, when you want to describe GFNs. As you mentioned earlier, multiple GFNs can in principle map to the same MFN. You would force them to all have the same properties, which would be in direct conflict with e.g. the grant P2M types.Quoted 6 lines
Just to mention one possible alternative to using radix trees: You could maintain a 2nd set of intermediate "page tables", just that leaf entries would hold meta data for the respective GFN. The memory for those "page tables" could come from the normal P2M pool (and allocation would thus only consume domain-specific resources). Of course in any model like this the question of lookup times (as mentioned above) would remain.
Quoted 8 lines
...looks like an optimal option.
The only thing I worry about is that it will require some code duplication (I will think how to re-use the current one code), as for example, when setting/getting metadata, TLB flushing isn’t needed at all as we aren't working with with real P2M page tables.
Agree that lookup won't be the best one, but nothing can be done with such models.Probably, instead of having a second set of intermediate "page tables", we could just allocate two consecutive pages within the real P2M page tables for the intermediate page table. The first page would serve as the actual page table to which the intermediate page table points, and the second page would store metadata for each entry of the page table that the intermediate page table references.
Quoted 5 lines
As we are supporting only 1gb, 2mb and 4kb mappings we could do a little optimization and start allocate these consecutive pages only for PT levels which corresponds to 1gb, 2mb, 4kb mappings.
Does it make sense?
I was indeed entertaining this idea, but I couldn't conclude for myself if that would indeed be without any rough edges. Hence I didn't want to suggest such. For example, the need to have adjacent pairs of pages could result in a higher rate of allocation failures (while populating or re-sizing the P2M pool). This would be possible to avoid by still using entirely separate pages, and then merely linking them together via some unused struct
page_infofields (the "normal" linking fields can't be used, afaict).
Quoted 5 lines
Quoted 2 lines
I think that all the fields are used, so it will be needed to introduce new "struct
page_list_entrymetadata_list;".All the fields are used _somewhere_, sure. But once you have allocated a page (and that page isn't assigned to a domain), you control what the fields are used for.
I thought that the whole idea is to use domain's pages from P2M pool freelist, pages for which is allocated by alloc_domheap_page(d, MEMF_no_owner), so an allocated page is assigned to a domain.
I assume that I have in this case to take some pages for an intermediate page table from freelist P2M pool, set an owner domain to NULL (pg->inuse.domain = NULL).
Then in this case it isn't clear why pg->list can't be re-used to link several pages for intermediate page table purposes + metadata? Is it because pg->list can be not empty? In this case it isn't clear if I could use a page, which has threaded pages.
page_info->count_info can't be re-used as it will break put_page_*() connected stuff. And for similar reason page_info->v.{...} can't be re-used as then page_get_owner() will be broken.
And page_info->tlbflush_timestamp still need for a common alloc algo to handle when to do TLB flush.
So if I will add something to page_info->v.{...} or page_info->u.{...} then mentioned above functions can't be used anymore for such pages which are used for intermediate page tables.
Quoted 2 lines
Or else enlisting pages on private lists wouldn't be
legitimate either.
Hmm, but I still should have link several pages somehow.
Or you meant just to have a field which just store a physical address to metadata? (what still looks like a list)
Quoted 7 lines
Quoted 4 lines
Can't we introduce new
PGT_METADATAtype and then add metadata page to structpage_info->listand when a metadata will be needed just iterate throughpage_info->listand find a page withPGT_METADATAtype?I'd be careful with the introduction of new page types. All handling of page types everywhere in the (affected part of the) code base would then need auditing.
~ Oleksii
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 161 lines · Oleksii Kurochko
Quoted 159 lines · Jan Beulich
Quoted 158 lines · Oleksii Kurochko
Quoted 147 lines · Jan Beulich
Quoted 141 lines · Oleksii Kurochko
Quoted 125 lines · Oleksii Kurochko
Quoted 62 lines · Jan Beulich
Quoted 53 lines · Oleksii Kurochko
Quoted 52 lines · Jan Beulich
Quoted 28 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
On 15.07.2025 16:47, Oleksii Kurochko wrote:
On 7/1/25 5:08 PM, Jan Beulich wrote:
On 10.06.2025 15:05, Oleksii Kurochko wrote:
--- a/xen/arch/riscv/p2m.c
+++ b/xen/arch/riscv/p2m.c
@@ -345,6 +345,26 @@ static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn)
return __map_domain_page(p2m->root + root_table_indx);
}+static int p2m_type_radix_set(struct p2m_domain *p2m, pte_t pte, p2m_type_t t)
See comments on the earlier patch regarding naming.+{
+ int rc;
+ gfn_t gfn = mfn_to_gfn(p2m->domain, mfn_from_pte(pte));
How does this work, when you record GFNs only for Xenheap pages?Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
I think I don't understand what is an issue. Could you please provide
some extra details?
Counter question: The mfn_to_gfn() you currently have is only a stub. It only
works for 1:1 mapped domains. Can you show me the eventual final implementation
of the function, making it possible to use it here?At the moment, I planned to support only 1:1 mapped domains, so it is final implementation.
Isn't that on overly severe limitation?
I wouldn't say that it's a severe limitation, as it's just a matter of how
Quoted 8 lines
|mfn_to_gfn()| is implemented. When non-1:1 mapped domains are supported, |mfn_to_gfn()| can be implemented differently, while the code where it’s calledwill likely remain unchanged.
What I meant in my reply is that, for the current state and current limitations, this is the final implementation of|mfn_to_gfn()|. But that doesn't mean I don't see the value in, or the need for, non-1:1 mapped domains—it's just that this limitation simplifies development at the current stage of the RISC-V port.
Simplification is fine in some cases, but not supporting the "normal" way of domain construction looks like a pretty odd restriction. I'm also curious how you envision to implement
mfn_to_gfn()then, suitable for generic use like the one here. Imo, current limitation or not, you simply want to avoid use of that function outside of the special gnttab case.Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
In this context (not sure if I asked before): With this use of a radix tree,
how do you intend to bound the amount of memory that a domain can use, by
making Xen insert very many entries?
I didn’t think about that. I assumed it would be enough to set the amount of
memory a guest domain can use by specifying|xen,domain-p2m-mem-mb| in the DTS,
or using some predefined value if|xen,domain-p2m-mem-mb| isn’t explicitly set.
Which would require these allocations to come from that pool.Yes, and it is true only for non-hardware domains with the current implementation.
???
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
I meant that pool is used now only for non-hardware domains at the moment.
And how does this matter here? The memory required for the radix tree doesn't come from that pool anyway.
I thought that is possible to do that somehow, but looking at a code of
radix-tree.cit seems like the only one way to allocate memroy for the radix treeisradix_tree_node_alloc()->xzalloc(struct rcu_node).Then it is needed to introduce
radix_tree_node_allocate(domain)Quoted 8 lines
That would be a possibility, but you may have seen that less than half a year ago we got rid of something along these lines. So it would require some pretty good justification to re-introduce.
Quoted 2 lines
or radix tree
can't be used at all for mentioned in the previous replies security reason, no?(Very) careful use may still be possible. But the downside of using this (potentially long lookup times) would always remain.
Could you please clarify what do you mean here by "(Very) careful"?
I thought about an introduction of an amount of possible keys in radix tree and if this amount is 0 then stop domain. And it is also unclear what should be a value for this amount. Probably, you have better idea.But generally your idea below ...
Quoted 42 lines
Quoted 20 lines
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Also, it seems this would just lead to the issue you mentioned earlier: when
the memory runs out,|domain_crash()| will be called or PTE will be zapped.
Or one domain exhausting memory would cause another domain to fail. A domain
impacting just itself may be tolerable. But a domain affecting other domains
isn't.But it seems like this issue could happen in any implementation. It won't happen only if we will have only pre-populated pool for any domain type (hardware, control, guest domain) without ability to extend them or allocate extra pages from domheap in runtime. Otherwise, if extra pages allocation is allowed then we can't really do something with this issue.
Quoted 4 lines
Quoted 2 lines
But that's why I brought this up: You simply have to. Or, as indicated, the moment you mark Xen security-supported on RISC-V, there will be an XSA needed.
Why it isn't XSA for other architectures? At least, Arm then should have such XSA.
Does Arm use a radix tree for storing types? It uses one for mem-access, but it's not clear to me whether that's actually a supported feature.
Quoted 3 lines
I don't understand why x86 won't have the same issue. Memory is the limited and shared resource, so if one of the domain will use to much memory then it could happen that other domains won't have enough memory for its purpose...
Quoted 7 lines
Quoted 4 lines
The question is whether allocations are bounded. With this use of a radix tree, you give domains a way to have Xen allocate pretty much arbitrary amounts of memory to populate that tree. That unbounded-ness is the problem, not memory allocations in general.
Isn't radix tree key bounded to an amount of GFNs given for a domain? We can't have more keys then a max GFN number for a domain. So a potential amount of necessary memory for radix tree is also bounded to an amount of GFNs.
To some degree yes, hence why I said "pretty much arbitrary amounts". But recall that "amount of GFNs" is a fuzzy term; I think you mean to use it to describe the amount of memory pages given to the guest. GFNs can be used for other purposes, though. Guests could e.g. grant themselves access to their own memory, then map those grants at otherwise unused GFNs.
Quoted 3 lines
Anyway, IIUC I just can't use radix tree for p2m types at all, right?
If yes, does it make sense to borrow 2 bits from structpage_info->type_infoas now it is used 9-bits for count of a frame?struct
page_infodescribes MFNs, when you want to describe GFNs. As you mentioned earlier, multiple GFNs can in principle map to the same MFN. You would force them to all have the same properties, which would be in direct conflict with e.g. the grant P2M types.Quoted 6 lines
Just to mention one possible alternative to using radix trees: You could maintain a 2nd set of intermediate "page tables", just that leaf entries would hold meta data for the respective GFN. The memory for those "page tables" could come from the normal P2M pool (and allocation would thus only consume domain-specific resources). Of course in any model like this the question of lookup times (as mentioned above) would remain.
Quoted 8 lines
...looks like an optimal option.
The only thing I worry about is that it will require some code duplication (I will think how to re-use the current one code), as for example, when setting/getting metadata, TLB flushing isn’t needed at all as we aren't working with with real P2M page tables.
Agree that lookup won't be the best one, but nothing can be done with such models.Probably, instead of having a second set of intermediate "page tables", we could just allocate two consecutive pages within the real P2M page tables for the intermediate page table. The first page would serve as the actual page table to which the intermediate page table points, and the second page would store metadata for each entry of the page table that the intermediate page table references.
Quoted 5 lines
As we are supporting only 1gb, 2mb and 4kb mappings we could do a little optimization and start allocate these consecutive pages only for PT levels which corresponds to 1gb, 2mb, 4kb mappings.
Does it make sense?
Quoted 8 lines
I was indeed entertaining this idea, but I couldn't conclude for myself if that would indeed be without any rough edges. Hence I didn't want to suggest such. For example, the need to have adjacent pairs of pages could result in a higher rate of allocation failures (while populating or re-sizing the P2M pool). This would be possible to avoid by still using entirely separate pages, and then merely linking them together via some unused struct
page_infofields (the "normal" linking fields can't be used, afaict).I think that all the fields are used, so it will be needed to introduce new "struct
page_list_entrymetadata_list;".
Quoted 7 lines
Quoted 3 lines
All the fields are used _somewhere_, sure. But once you have allocated a page (and that page isn't assigned to a domain), you control what the fields are used for.
I thought that the whole idea is to use domain's pages from P2M pool freelist, pages for which is allocated by
alloc_domheap_page(d, MEMF_no_owner), so an allocated page is assigned to a domain.
You did check what effect MEMF_no_owner has, didn't you? Such pages are _not_ assigned to the domain.
Quoted 6 lines
I assume that I have in this case to take some pages for an intermediate page table from freelist P2M pool, set an owner domain to NULL (
pg->inuse.domain= NULL).Then in this case it isn't clear why
pg->listcan't be re-used to link several pages for intermediate page table purposes + metadata? Is it becausepg->listcan be not empty? In this case it isn't clear if I could use a page, which has threaded pages.
Actually looks like I was mis-remembering. Pages removed from freelist indeed aren't put on any other list, so the linking fields are available for use. I guess I had x86 shadow code in mind, where the linking fields are further used.
Jan
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 9 lines · Jan Beulich
Quoted 6 lines
I assume that I have in this case to take some pages for an intermediate page table from freelist P2M pool, set an owner domain to NULL (
pg->inuse.domain= NULL).Then in this case it isn't clear why
pg->listcan't be re-used to link several pages for intermediate page table purposes + metadata? Is it becausepg->listcan be not empty? In this case it isn't clear if I could use a page, which has threaded pages.Actually looks like I was mis-remembering. Pages removed from freelist indeed aren't put on any other list, so the linking fields are available for use. I guess I had x86 shadow code in mind, where the linking fields are further used.
Perhaps, I misunderstood you about "linking fields", but it seems like I can't reuse struct page_info->list as it is used by page_list_add() which is called by p2m_alloc_page() to allocate page(s) for an intermediate page table:
static inline void
page_list_add(struct page_info *page, struct page_list_head *head)
{
list_add(&page->list, head);}
struct page_info * paging_alloc_page(struct domain *d)
{
struct page_info *pg;
spin_lock(&d->arch.paging.lock);
pg = page_list_remove_head(&d->arch.paging.freelist);
spin_unlock(&d->arch.paging.lock);
INIT_LIST_HEAD(&pg->list);
return pg;
}
static struct page_info *p2m_alloc_page(struct domain *d)
{
struct page_info *pg = paging_alloc_page(d);
if ( pg )
page_list_add(pg, &p2m_get_hostp2m(d)->pages);
return pg;
}So I have to reuse another field from struct page_info. It seems like it won't be an issue if to add a new struct page_list_entry metadata_list to 'union v':
union {
/* Page is in use */
struct {
/* Owner of this page (NULL if page is anonymous). */
struct domain *domain;
} inuse;
/* Page is on a free list. */
struct {
/* Order-size of the free chunk this page is the head of. */
unsigned int order;
} free;
+
+ struct page_list_entry metadata_list;
} v;Am I missing something?
~ Oleksii
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 61 lines · Oleksii Kurochko
Quoted 9 lines · Jan Beulich
Quoted 6 lines
I assume that I have in this case to take some pages for an intermediate page table from freelist P2M pool, set an owner domain to NULL (
pg->inuse.domain= NULL).Then in this case it isn't clear why
pg->listcan't be re-used to link several pages for intermediate page table purposes + metadata? Is it becausepg->listcan be not empty? In this case it isn't clear if I could use a page, which has threaded pages.Actually looks like I was mis-remembering. Pages removed from freelist indeed aren't put on any other list, so the linking fields are available for use. I guess I had x86 shadow code in mind, where the linking fields are further used.
Perhaps, I misunderstood you about "linking fields", but it seems like I can't reuse struct
page_info->listas it is used bypage_list_add()which is called byp2m_alloc_page()to allocate page(s) for an intermediate page table:static inline void page_list_add(struct page_info *page, struct page_list_head *head) { list_add(&page->list, head); } struct page_info * paging_alloc_page(struct domain *d) { struct page_info *pg; spin_lock(&d->arch.paging.lock); pg = page_list_remove_head(&d->arch.paging.freelist); spin_unlock(&d->arch.paging.lock); INIT_LIST_HEAD(&pg->list); return pg; } static struct page_info *p2m_alloc_page(struct domain *d) { struct page_info *pg = paging_alloc_page(d); if ( pg ) page_list_add(pg, &p2m_get_hostp2m(d)->pages); return pg; }So I have to reuse another field from struct
page_info. It seems like it won't be an issue if to add a new structpage_list_entrymetadata_listto 'union v':union { /* Page is in use */ struct { /* Owner of this page (NULL if page is anonymous). */ struct domain *domain; } inuse; /* Page is on a free list. */ struct { /* Order-size of the free chunk this page is the head of. */ unsigned int order; } free;(quoted diff) +2
-0+ + struct page_list_entry metadata_list; } v;
Quoted 1 line
Am I missing something?
Well, you're doubling the size of that union then, aren't you? As was mentioned quite some time ago, struct page_info needs quite a bit of care when you mean to add new fields there. Question is whether for the purpose here you actually need a doubly-linked list. A single pointer would be fine to put there.
Jan
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 61 lines · Jan Beulich
Quoted 60 lines · Oleksii Kurochko
Quoted 9 lines · Jan Beulich
Quoted 6 lines
I assume that I have in this case to take some pages for an intermediate page table from freelist P2M pool, set an owner domain to NULL (
pg->inuse.domain= NULL).Then in this case it isn't clear why
pg->listcan't be re-used to link several pages for intermediate page table purposes + metadata? Is it becausepg->listcan be not empty? In this case it isn't clear if I could use a page, which has threaded pages.Actually looks like I was mis-remembering. Pages removed from freelist indeed aren't put on any other list, so the linking fields are available for use. I guess I had x86 shadow code in mind, where the linking fields are further used.
Perhaps, I misunderstood you about "linking fields", but it seems like I can't reuse struct
page_info->listas it is used bypage_list_add()which is called byp2m_alloc_page()to allocate page(s) for an intermediate page table:static inline void page_list_add(struct page_info *page, struct page_list_head *head) { list_add(&page->list, head); } struct page_info * paging_alloc_page(struct domain *d) { struct page_info *pg; spin_lock(&d->arch.paging.lock); pg = page_list_remove_head(&d->arch.paging.freelist); spin_unlock(&d->arch.paging.lock); INIT_LIST_HEAD(&pg->list); return pg; } static struct page_info *p2m_alloc_page(struct domain *d) { struct page_info *pg = paging_alloc_page(d); if ( pg ) page_list_add(pg, &p2m_get_hostp2m(d)->pages); return pg; }So I have to reuse another field from struct
page_info. It seems like it won't be an issue if to add a new structpage_list_entrymetadata_listto 'union v':union { /* Page is in use */ struct { /* Owner of this page (NULL if page is anonymous). */ struct domain *domain; } inuse; /* Page is on a free list. */ struct { /* Order-size of the free chunk this page is the head of. */ unsigned int order; } free;(quoted diff) +2
-0+ + struct page_list_entry metadata_list; } v;
Quoted 5 lines
Quoted 1 line
Am I missing something?
Well, you're doubling the size of that union then, aren't you? As was mentioned quite some time ago, struct
page_infoneeds quite a bit of care when you mean to add new fields there. Question is whether for the purpose here you actually need a doubly-linked list. A single pointer would be fine to put there.
Agree, a single pointer will be more then enough.
I'm thinking if it is possible to do something with the case if someone will try to use:
#define page_get_owner(p) (p)->v.inuse.domain
for a page which was allocated for metadata storage. Shouldn't I have a separate list for such pages and a macro which will check if a page is in this list? Similar a list which we have for p2m pages in struct p2m_domain:
...
/* Pages used to construct the p2m */
struct page_list_head pages;
...Of course, such pages are allocated by alloc_domheap_page(d, MEMF_no_owner), so there is no owner. But if someone will accidentally use this macro for such pages then it will be an issue as ->domain likely won't be a NULL anymore.
~ Oleksii
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
Quoted 82 lines · Oleksii Kurochko
Quoted 61 lines · Jan Beulich
Quoted 60 lines · Oleksii Kurochko
Quoted 9 lines · Jan Beulich
Quoted 6 lines
I assume that I have in this case to take some pages for an intermediate page table from freelist P2M pool, set an owner domain to NULL (
pg->inuse.domain= NULL).Then in this case it isn't clear why
pg->listcan't be re-used to link several pages for intermediate page table purposes + metadata? Is it becausepg->listcan be not empty? In this case it isn't clear if I could use a page, which has threaded pages.Actually looks like I was mis-remembering. Pages removed from freelist indeed aren't put on any other list, so the linking fields are available for use. I guess I had x86 shadow code in mind, where the linking fields are further used.
Perhaps, I misunderstood you about "linking fields", but it seems like I can't reuse struct
page_info->listas it is used bypage_list_add()which is called byp2m_alloc_page()to allocate page(s) for an intermediate page table:static inline void page_list_add(struct page_info *page, struct page_list_head *head) { list_add(&page->list, head); } struct page_info * paging_alloc_page(struct domain *d) { struct page_info *pg; spin_lock(&d->arch.paging.lock); pg = page_list_remove_head(&d->arch.paging.freelist); spin_unlock(&d->arch.paging.lock); INIT_LIST_HEAD(&pg->list); return pg; } static struct page_info *p2m_alloc_page(struct domain *d) { struct page_info *pg = paging_alloc_page(d); if ( pg ) page_list_add(pg, &p2m_get_hostp2m(d)->pages); return pg; }So I have to reuse another field from struct
page_info. It seems like it won't be an issue if to add a new structpage_list_entrymetadata_listto 'union v':union { /* Page is in use */ struct { /* Owner of this page (NULL if page is anonymous). */ struct domain *domain; } inuse; /* Page is on a free list. */ struct { /* Order-size of the free chunk this page is the head of. */ unsigned int order; } free;(quoted diff) +2
-0+ + struct page_list_entry metadata_list; } v;Quoted 5 lines
Quoted 1 line
Am I missing something?
Well, you're doubling the size of that union then, aren't you? As was mentioned quite some time ago, struct
page_infoneeds quite a bit of care when you mean to add new fields there. Question is whether for the purpose here you actually need a doubly-linked list. A single pointer would be fine to put there.Agree, a single pointer will be more then enough.
I'm thinking if it is possible to do something with the case if someone will try to use:
#define page_get_owner(p) (p)->v.inuse.domain for a page which was allocated for metadata storage. Shouldn't I have a separate list for such pages and a macro which will check if a page is in this list? Similar a list which we have for p2m pages in struct p2m_domain: ... /* Pages used to construct the p2m */ struct page_list_head pages; ...
Quoted 3 lines
Of course, such pages are allocated by
alloc_domheap_page(d, MEMF_no_owner), so there is no owner. But if someone will accidentally use this macro for such pages then it will be an issue as ->domain likely won't be a NULL anymore.
It's the nature of using unions that such a risk exists. Take a look at x86'es structure, where several of the fields are re-purposed for shadow pages. It's something similar you'd do here, in the end.
Jan
Re: [PATCH v2 13/17] xen/riscv: Implement p2m_entry_from_mfn() and support PBMT configuration
In reply to Oleksii KurochkoQuoted 69 lines · Oleksii Kurochko
Quoted 68 lines · Jan Beulich
Quoted 53 lines · Oleksii Kurochko
Quoted 52 lines · Jan Beulich
Quoted 28 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Quoted 14 lines · Jan Beulich
xen/arch/riscv/p2m.c static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn) · line 348
Quoted 7 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +1
-0@@ -345,6 +345,26 @@ static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn)return __map_domain_page (p2m->root + root_table_indx); } +static int p2m_type_radix_set (struct p2m_domain *p2m,pte_t pte,p2m_type_t t)See comments on the earlier patch regarding naming.
xen/arch/riscv/p2m.c static pte_t *p2m_get_root_pointer(struct p2m_domain *p2m, gfn_t gfn) · line 351
Quoted 3 lines
xen/arch/riscv/p2m.c +3
-0+{ +int rc; +gfn_t gfn =mfn_to_gfn (p2m->domain,mfn_from_pte (pte));How does this work, when you record GFNs only for Xenheap pages?
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think I don't understand what is an issue. Could you please provide some extra details?
Counter question: The
mfn_to_gfn()you currently have is only a stub. It only works for 1:1 mapped domains. Can you show me the eventual final implementation of the function, making it possible to use it here?At the moment, I planned to support only 1:1 mapped domains, so it is final implementation.
Isn't that on overly severe limitation?
I wouldn't say that it's a severe limitation, as it's just a matter of how
Quoted 8 lines
|mfn_to_gfn()| is implemented. When non-1:1 mapped domains are supported, |mfn_to_gfn()| can be implemented differently, while the code where it’s calledwill likely remain unchanged.
What I meant in my reply is that, for the current state and current limitations, this is the final implementation of|mfn_to_gfn()|. But that doesn't mean I don't see the value in, or the need for, non-1:1 mapped domains—it's just that this limitation simplifies development at the current stage of the RISC-V port.
Simplification is fine in some cases, but not supporting the "normal" way of domain construction looks like a pretty odd restriction. I'm also curious how you envision to implement
mfn_to_gfn()then, suitable for generic use like the one here. Imo, current limitation or not, you simply want to avoid use of that function outside of the special gnttab case.Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
In this context (not sure if I asked before): With this use of a radix tree, how do you intend to bound the amount of memory that a domain can use, by making Xen insert very many entries?
I didn’t think about that. I assumed it would be enough to set the amount of memory a guest domain can use by specifying|xen,domain-p2m-mem-mb| in the DTS, or using some predefined value if|xen,domain-p2m-mem-mb| isn’t explicitly set.
Which would require these allocations to come from that pool.
Yes, and it is true only for non-hardware domains with the current implementation.
???
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
I meant that pool is used now only for non-hardware domains at the moment.
And how does this matter here? The memory required for the radix tree doesn't come from that pool anyway.
I thought that is possible to do that somehow, but looking at a code of
radix-tree.cit seems like the only one way to allocate memroy for the radix treeisradix_tree_node_alloc()->xzalloc(struct rcu_node).Then it is needed to introduce
radix_tree_node_allocate(domain)That would be a possibility, but you may have seen that less than half a year ago we got rid of something along these lines. So it would require some pretty good justification to re-introduce.
Quoted 2 lines
or radix tree
can't be used at all for mentioned in the previous replies security reason, no?
Quoted 7 lines
Quoted 2 lines
(Very) careful use may still be possible. But the downside of using this (potentially long lookup times) would always remain.
Could you please clarify what do you mean here by "(Very) careful"?
I thought about an introduction of an amount of possible keys in radix tree and if this amount is 0 then stop domain. And it is also unclear what should be a value for this amount. Probably, you have better idea.
I had no particular idea in mind. I said "(very) careful" merely to clarify that whatever model is chosen, it would need to satisfy certain needs.
Jan
PATCH v2 14/17 xen/riscv: implement p2m_next_level()
In reply to Oleksii KurochkoImplement the p2m_next_level() function, which enables traversal and dynamic allocation of intermediate levels (if necessary) in the RISC-V p2m (physical-to-machine) page table hierarchy.
To support this, the following helpers are introduced:
- p2me_is_mapping(): Determines whether a PTE represents a valid mapping.
- page_to_p2m_table(): Constructs non-leaf PTEs pointing to next-level page tables with correct attributes.
- p2m_alloc_page(): Allocates page table pages, supporting both hardware and guest domains.
- p2m_create_table(): Allocates and initializes a new page table page and installs it into the hierarchy.
---
Changes in V2:
- New patch. It was a part of a big patch "xen/riscv: implement p2m mapping functionality" which was splitted to smaller.
- s/p2m_is_mapping/p2me_is_mapping.
--- xen/arch/riscv/p2m.c | 103 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 2 deletions(-)
xen/arch/riscv/p2m.c +101 -2
@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; } +/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +} +static inline bool p2me_is_superpage (struct p2m_domain *p2m,pte_t pte,unsigned int level) {
@@ -492,6 +503,70 @@ static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t,return e; } +/* Generate table entry with correct attributes. */ +static pte_t page_to_p2m_table (struct p2m_domain *p2m,struct page_info *page) +{ +/* +* Since this function generates a table entry, according to "Encoding +* of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0 +* to point to the next level of the page table. +* Therefore, to ensure that an entry is a page table entry, +* `p2m_access_n2rwx` is passed to `mfn_to_p2m_entry()` as the access value, +* which overrides whatever was passed as `p2m_type_t` and guarantees that +* the entry is a page table entry by setting r = w = x = 0. +*/ +return p2m_entry_from_mfn (p2m,page_to_mfn (page), p2m_ram_rw, p2m_access_n2rwx); +} + +static struct page_info *p2m_alloc_page (struct domain *d) +{ +struct page_info *pg; + +/* +* For hardware domain, there should be no limit in the number of pages that +* can be allocated, so that the kernel may take advantage of the extended +* regions. Hence, allocate p2m pages for hardware domains from heap. +*/ +if (is_hardware_domain (d) ) + { + pg =alloc_domheap_page (d, MEMF_no_owner); +if ( pg ==NULL ) +printk (XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n" ); + } +else + { +spin_lock (&d->arch.paging.lock); + pg =page_list_remove_head (&d->arch.paging.p2m_freelist); +spin_unlock (&d->arch.paging.lock); + } + +return pg; +} + +/* Allocate a new page table page and hook it in via the given entry. */ +static int p2m_create_table (struct p2m_domain *p2m,pte_t *entry) +{ +struct page_info *page; +pte_t *p; + +ASSERT (!p2me_is_valid (p2m, *entry)); + + page =p2m_alloc_page (p2m->domain); +if ( page ==NULL ) +return -ENOMEM ; + +page_list_add (page, &p2m->pages); + + p =__map_domain_page (page); +clear_page (p); + +unmap_domain_page (p); + +p2m_write_pte (entry,page_to_p2m_table (p2m, page), p2m->clean_pte); + +return 0 ; +} +#define GUEST_TABLE_MAP_NONE 0#define GUEST_TABLE_MAP_NOMEM 1#define GUEST_TABLE_SUPER_PAGE 2
@@ -516,9 +591,33 @@ static int p2m_next_level(struct p2m_domain *p2m, bool alloc_tbl, unsigned int level, pte_t **table, unsigned int offset) { - panic("%s: hasn't been implemented yet\n", __func__); + pte_t *entry; +int ret; +mfn_t mfn; + + entry = *table + offset; + +if ( !p2me_is_valid (p2m, *entry) ) + { +if ( !alloc_tbl ) +return GUEST_TABLE_MAP_NONE ; + + ret =p2m_create_table (p2m, entry); +if ( ret ) +return GUEST_TABLE_MAP_NOMEM ; + } + +/* The function p2m_next_level() is never called at the last level */ +ASSERT (level !=0 ); +if (p2me_is_mapping (p2m, *entry) ) +return GUEST_TABLE_SUPER_PAGE ; + + mfn =mfn_from_pte (*entry); + +unmap_domain_page (*table); + *table =map_domain_page (mfn); -return GUEST_TABLE_MAP_NONE ; +return GUEST_TABLE_NORMAL ; }static void p2m_put_foreign_page (struct page_info *pg)
Signature · 1 line
2.49.0
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }
xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}
Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
Quoted 7 lines
xen/arch/riscv/p2m.c +3
-0@@ -492,6 +503,70 @@ static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t,return e; } +/* Generate table entry with correct attributes. */ +static pte_t page_to_p2m_table (struct p2m_domain *p2m,struct page_info *page) +{
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 518
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* Since this function generates a table entry, according to "Encoding +* of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0 +* to point to the next level of the page table. +* Therefore, to ensure that an entry is a page table entry, +* `p2m_access_n2rwx` is passed to `mfn_to_p2m_entry()` as the access value, +* which overrides whatever was passed as `p2m_type_t` and guarantees that +* the entry is a page table entry by setting r = w = x = 0. +*/ +return p2m_entry_from_mfn (p2m,page_to_mfn (page), p2m_ram_rw, p2m_access_n2rwx);
Similarly P2M access shouldn't apply to intermediate page tables. (Moot with that, but (ab)using p2m_access_n2rwx would also look wrong: You did read what it means, didn't you?)
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+} + +static struct page_info *p2m_alloc_page (struct domain *d) +{ +struct page_info *pg; + +/*
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 535
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+* For hardware domain, there should be no limit in the number of pages that +* can be allocated, so that the kernel may take advantage of the extended +* regions. Hence, allocate p2m pages for hardware domains from heap. +*/ +if (is_hardware_domain (d) ) + { + pg =alloc_domheap_page (d, MEMF_no_owner); +if ( pg ==NULL ) +printk (XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n" ); + }
The comment looks to have been taken verbatim from Arm. Whatever "extended regions" are, does the same concept even exist on RISC-V?
Also, special casing Dom0 like this has benefits, but also comes with a
pitfall: If the system's out of memory, allocations will fail. A pre- populated pool would avoid that (until exhausted, of course). If special- casing of Dom0 is needed, I wonder whether ...
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 541
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+else + { +spin_lock (&d->arch.paging.lock); + pg =page_list_remove_head (&d->arch.paging.p2m_freelist); +spin_unlock (&d->arch.paging.lock); + }
... going this path but with a Dom0-only fallback to general allocation wouldn't be the better route.
Quoted 11 lines
xen/arch/riscv/p2m.c +11
-0+return pg; +} + +/* Allocate a new page table page and hook it in via the given entry. */ +static int p2m_create_table (struct p2m_domain *p2m,pte_t *entry) +{ +struct page_info *page; +pte_t *p; + +ASSERT (!p2me_is_valid (p2m, *entry)); +
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 563
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ page =p2m_alloc_page (p2m->domain); +if ( page ==NULL ) +return -ENOMEM ; + +page_list_add (page, &p2m->pages); + + p =__map_domain_page (page); +clear_page (p); + +unmap_domain_page (p);
clear_domain_page()? Or actually clear_and_clean_page()?
Quoted 13 lines
xen/arch/riscv/p2m.c +8
-1@@ -516,9 +591,33 @@ static int p2m_next_level(struct p2m_domain *p2m, bool alloc_tbl, unsigned int level, pte_t **table, unsigned int offset) { - panic("%s: hasn't been implemented yet\n", __func__); + pte_t *entry; +int ret; +mfn_t mfn; + + entry = *table + offset; + +if ( !p2me_is_valid (p2m, *entry) ) + {
xen/arch/riscv/p2m.c static int p2m_next_level(struct p2m_domain *p2m, bool alloc_tbl, · line 611
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+if ( !alloc_tbl ) +return GUEST_TABLE_MAP_NONE ; + + ret =p2m_create_table (p2m, entry); +if ( ret ) +return GUEST_TABLE_MAP_NOMEM ; + } + +/* The function p2m_next_level() is never called at the last level */ +ASSERT (level !=0 );
Logically you would perhaps better do this ahead of trying to allocate a page table. Calls here with level == 0 are invalid in all cases aiui, not just when you make it here.
xen/arch/riscv/p2m.c static int p2m_next_level(struct p2m_domain *p2m, bool alloc_tbl, · line 618
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+if (p2me_is_mapping (p2m, *entry) ) +return GUEST_TABLE_SUPER_PAGE ; + + mfn =mfn_from_pte (*entry); + +unmap_domain_page (*table); + *table =map_domain_page (mfn);
Just to mention it (may not need taking care of right away), there's an inefficiency here: In p2m_create_table() you map the page to clear it. Then you tear down that mapping, just to re-establish it here.
Jan
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }
Quoted 12 lines
xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
It doesn't matter whether it is an intermediate page table or a leaf PTE pointing to a page — PTE should be valid. Considering that in the current implementation it’s possible for PTE.v = 0 but P2M.v = 1, it is better to check P2M.v instead of PTE.v.
Quoted 5 lines
Quoted 5 lines
xen/arch/riscv/p2m.c +0
-0@@ -492,6 +503,70 @@ static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t,return e; }
Quoted 16 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 518
Quoted 13 lines
xen/arch/riscv/p2m.c +13
-0+/* Generate table entry with correct attributes. */ +static pte_t page_to_p2m_table (struct p2m_domain *p2m,struct page_info *page) +{ +/* +* Since this function generates a table entry, according to "Encoding +* of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0 +* to point to the next level of the page table. +* Therefore, to ensure that an entry is a page table entry, +* `p2m_access_n2rwx` is passed to `mfn_to_p2m_entry()` as the access value, +* which overrides whatever was passed as `p2m_type_t` and guarantees that +* the entry is a page table entry by setting r = w = x = 0. +*/ +return p2m_entry_from_mfn (p2m,page_to_mfn (page), p2m_ram_rw, p2m_access_n2rwx);Similarly P2M access shouldn't apply to intermediate page tables. (Moot with that, but (ab)using
p2m_access_n2rwxwould also look wrong: You did read what it means, didn't you?)
|p2m_access_n2rwx| was chosen not really because of the description mentioned near its declaration, but because it sets r=w=x=0, which RISC-V expects for a PTE that points to the next-level page table.
Generally, I agree that P2M access shouldn't be applied to intermediate page tables.
What I can suggest in this case is to use|p2m_access_rwx| instead of|p2m_access_n2rwx|,
which will ensure that the P2M access type isn't applied when|p2m_entry_from_mfn() |is called, and then, after calling|p2m_entry_from_mfn()|, simply set|PTE.r,w,x=0|. So this function will look like:
/* Generate table entry with correct attributes. */
static pte_t page_to_p2m_table(struct p2m_domain *p2m, struct page_info *page)
{
/*
* p2m_ram_rw is chosen for a table entry as p2m table should be valid
* from both P2M and hardware point of view.
*
* p2m_access_rwx is chosen to restrict access permissions, what mean
* do not apply access permission for a table entry
*/
pte_t pte = p2m_pte_from_mfn(p2m, page_to_mfn(page), _gfn(0), p2m_ram_rw,
p2m_access_rwx);
/*
* Since this function generates a table entry, according to "Encoding
* of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0
* to point to the next level of the page table.
*/
pte.pte &= ~PTE_ACCESS_MASK;
return pte;
}Does this make sense? Or would it be better to keep the current version of |page_to_p2m_table()| and just improve the comment explaining why|p2m_access_n2rwx |is used for a table entry?
Quoted 12 lines
Quoted 12 lines
xen/arch/riscv/p2m.c +11
-0+} + +static struct page_info *p2m_alloc_page (struct domain *d) +{ +struct page_info *pg; + +/* +* For hardware domain, there should be no limit in the number of pages that +* can be allocated, so that the kernel may take advantage of the extended +* regions. Hence, allocate p2m pages for hardware domains from heap. +*/
Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 535
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+if (is_hardware_domain (d) ) + { + pg =alloc_domheap_page (d, MEMF_no_owner); +if ( pg ==NULL ) +printk (XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n" ); + }The comment looks to have been taken verbatim from Arm. Whatever "extended regions" are, does the same concept even exist on RISC-V?
Initially, I missed that it’s used only for Arm. Since it was mentioned in |doc/misc/xen-command-line.pandoc|, I assumed it applied to all architectures. But now I see that it’s Arm-specific:: ### ext_regions (Arm)
Quoted 6 lines
Also, special casing Dom0 like this has benefits, but also comes with a
pitfall: If the system's out of memory, allocations will fail. A pre- populated pool would avoid that (until exhausted, of course). If special- casing of Dom0 is needed, I wonder whether ...
Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 541
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+else + { +spin_lock (&d->arch.paging.lock); + pg =page_list_remove_head (&d->arch.paging.p2m_freelist); +spin_unlock (&d->arch.paging.lock); + }... going this path but with a Dom0-only fallback to general allocation wouldn't be the better route.
IIUC, then it should be something like:
static struct page_info *p2m_alloc_page(struct domain *d)
{
struct page_info *pg;
spin_lock(&d->arch.paging.lock);
pg = page_list_remove_head(&d->arch.paging.p2m_freelist);
spin_unlock(&d->arch.paging.lock);
if ( !pg && is_hardware_domain(d) )
{
/* Need to allocate more memory from domheap */
pg = alloc_domheap_page(d, MEMF_no_owner);
if ( pg == NULL )
{
printk(XENLOG_ERR "Failed to allocate pages.\n");
return pg;
}
ACCESS_ONCE(d->arch.paging.total_pages)++;
page_list_add_tail(pg, &d->arch.paging.freelist);
}
return pg;
}And basically use|d->arch.paging.freelist| for both dom0less and dom0 domains,
with the only difference being that in the case of Dom0,|d->arch.paging.freelist |could be extended.
Do I understand your idea correctly?
(
Probably, this is the reply you’re referring to:
https://lore.kernel.org/xen-devel/[email protected]/,
at the moment, I can't find a better one.
)
Quoted 15 lines
Quoted 15 lines
xen/arch/riscv/p2m.c +14
-0+return pg; +} + +/* Allocate a new page table page and hook it in via the given entry. */ +static int p2m_create_table (struct p2m_domain *p2m,pte_t *entry) +{ +struct page_info *page; +pte_t *p; + +ASSERT (!p2me_is_valid (p2m, *entry)); + + page =p2m_alloc_page (p2m->domain); +if ( page ==NULL ) +return -ENOMEM ;
Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 563
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+ +page_list_add (page, &p2m->pages); + + p =__map_domain_page (page); +clear_page (p); + +unmap_domain_page (p);
clear_domain_page()? Or actuallyclear_and_clean_page()?
Agree, clear_and_clean_page() would be better here.
Quoted 19 lines
Quoted 19 lines
xen/arch/riscv/p2m.c +13
-1@@ -516,9 +591,33 @@ static int p2m_next_level(struct p2m_domain *p2m, bool alloc_tbl, unsigned int level, pte_t **table, unsigned int offset) { - panic("%s: hasn't been implemented yet\n", __func__); + pte_t *entry; +int ret; +mfn_t mfn; + + entry = *table + offset; + +if ( !p2me_is_valid (p2m, *entry) ) + { +if ( !alloc_tbl ) +return GUEST_TABLE_MAP_NONE ; + + ret =p2m_create_table (p2m, entry); +if ( ret )
Quoted 8 lines
xen/arch/riscv/p2m.c static int p2m_next_level(struct p2m_domain *p2m, bool alloc_tbl, · line 611
Quoted 5 lines
xen/arch/riscv/p2m.c +5
-0+return GUEST_TABLE_MAP_NOMEM ; + } + +/* The function p2m_next_level() is never called at the last level */ +ASSERT (level !=0 );Logically you would perhaps better do this ahead of trying to allocate a page table. Calls here with level == 0 are invalid in all cases aiui, not just when you make it here.
It makes sense. I will move ASSERT() to the start of the function p2m_next_level().
Quoted 10 lines
xen/arch/riscv/p2m.c static int p2m_next_level(struct p2m_domain *p2m, bool alloc_tbl, · line 618
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+if (p2me_is_mapping (p2m, *entry) ) +return GUEST_TABLE_SUPER_PAGE ; + + mfn =mfn_from_pte (*entry); + +unmap_domain_page (*table); + *table =map_domain_page (mfn);Just to mention it (may not need taking care of right away), there's an inefficiency here: In
p2m_create_table()you map the page to clear it. Then you tear down that mapping, just to re-establish it here.
I will add:
/*
* TODO: There's an inefficiency here:
* In p2m_create_table(), the page is mapped to clear it.
* Then that mapping is torn down in p2m_create_table(),
* only to be re-established here.
*/
*table = map_domain_page(mfn);
Thanks.
~ Oleksii
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 21 lines · Oleksii Kurochko
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }Quoted 12 lines
xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
Quoted 4 lines
It doesn't matter whether it is an intermediate page table or a leaf PTE pointing to a page — PTE should be valid. Considering that in the current implementation it’s possible for PTE.v = 0 but P2M.v = 1, it is better to check P2M.v instead of PTE.v.
I'm confused by this reply. If you want to name 2nd level page table entries P2M - fine (but unhelpful). But then for any memory access there's only one of the two involved: A PTE (Xen accesses) or a P2M (guest accesses). Hence how can there be "PTE.v = 0 but P2M.v = 1"?
An intermediate page table entry is something Xen controls entirely. Hence it has no (guest induced) type.
Quoted 21 lines
Quoted 4 lines
Quoted 4 lines
xen/arch/riscv/p2m.c +0
-0@@ -492,6 +503,70 @@ static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t,return e; }Quoted 16 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 518
Quoted 13 lines
xen/arch/riscv/p2m.c +13
-0+/* Generate table entry with correct attributes. */ +static pte_t page_to_p2m_table (struct p2m_domain *p2m,struct page_info *page) +{ +/* +* Since this function generates a table entry, according to "Encoding +* of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0 +* to point to the next level of the page table. +* Therefore, to ensure that an entry is a page table entry, +* `p2m_access_n2rwx` is passed to `mfn_to_p2m_entry()` as the access value, +* which overrides whatever was passed as `p2m_type_t` and guarantees that +* the entry is a page table entry by setting r = w = x = 0. +*/ +return p2m_entry_from_mfn (p2m,page_to_mfn (page), p2m_ram_rw, p2m_access_n2rwx);Similarly P2M access shouldn't apply to intermediate page tables. (Moot with that, but (ab)using
p2m_access_n2rwxwould also look wrong: You did read what it means, didn't you?)
Quoted 7 lines
|p2m_access_n2rwx| was chosen not really because of the description mentioned near its declaration, but because it sets r=w=x=0, which RISC-V expects for a PTE that points to the next-level page table.
Generally, I agree that P2M access shouldn't be applied to intermediate page tables.
What I can suggest in this case is to use|p2m_access_rwx| instead of|p2m_access_n2rwx|,
No. p2m_access_* shouldn't come into play here at all. Period. Just like P2M types shouldn't. As per above - intermediate page tables are Xen internal constructs.
Quoted 25 lines
which will ensure that the P2M access type isn't applied when|p2m_entry_from_mfn() |is called, and then, after calling|p2m_entry_from_mfn()|, simply set|PTE.r,w,x=0|. So this function will look like:
/* Generate table entry with correct attributes. */ static pte_t page_to_p2m_table(struct p2m_domain *p2m, struct page_info *page) { /* * p2m_ram_rw is chosen for a table entry as p2m table should be valid * from both P2M and hardware point of view. * * p2m_access_rwx is chosen to restrict access permissions, what mean * do not apply access permission for a table entry */ pte_t pte = p2m_pte_from_mfn(p2m, page_to_mfn(page), _gfn(0), p2m_ram_rw, p2m_access_rwx); /* * Since this function generates a table entry, according to "Encoding * of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0 * to point to the next level of the page table. */ pte.pte &= ~PTE_ACCESS_MASK; return pte; }
Quoted 2 lines
Does this make sense? Or would it be better to keep the current version of |page_to_p2m_table()| and just improve the comment explaining why|p2m_access_n2rwx |is used for a table entry?
No to both, as per above.
Quoted 62 lines
Quoted 9 lines
Quoted 9 lines
xen/arch/riscv/p2m.c +9
-0+static struct page_info *p2m_alloc_page (struct domain *d) +{ +struct page_info *pg; + +/* +* For hardware domain, there should be no limit in the number of pages that +* can be allocated, so that the kernel may take advantage of the extended +* regions. Hence, allocate p2m pages for hardware domains from heap. +*/ Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 535
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+if (is_hardware_domain (d) ) + { + pg =alloc_domheap_page (d, MEMF_no_owner); +if ( pg ==NULL ) +printk (XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n" ); + }The comment looks to have been taken verbatim from Arm. Whatever "extended regions" are, does the same concept even exist on RISC-V?
Initially, I missed that it’s used only for Arm. Since it was mentioned in |doc/misc/xen-command-line.pandoc|, I assumed it applied to all architectures. But now I see that it’s Arm-specific:: ###
ext_regions(Arm)Quoted 6 lines
Also, special casing Dom0 like this has benefits, but also comes with a
pitfall: If the system's out of memory, allocations will fail. A pre- populated pool would avoid that (until exhausted, of course). If special- casing of Dom0 is needed, I wonder whether ...Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 541
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+else + { +spin_lock (&d->arch.paging.lock); + pg =page_list_remove_head (&d->arch.paging.p2m_freelist); +spin_unlock (&d->arch.paging.lock); + }... going this path but with a Dom0-only fallback to general allocation wouldn't be the better route.
IIUC, then it should be something like:
static struct page_info *p2m_alloc_page(struct domain *d) { struct page_info *pg; spin_lock(&d->arch.paging.lock); pg = page_list_remove_head(&d->arch.paging.p2m_freelist); spin_unlock(&d->arch.paging.lock); if ( !pg && is_hardware_domain(d) ) { /* Need to allocate more memory from domheap */ pg = alloc_domheap_page(d, MEMF_no_owner); if ( pg == NULL ) { printk(XENLOG_ERR "Failed to allocate pages.\n"); return pg; } ACCESS_ONCE(d->arch.paging.total_pages)++; page_list_add_tail(pg, &d->arch.paging.freelist); } return pg; }
Quoted 4 lines
And basically use|d->arch.paging.freelist| for both dom0less and dom0 domains,
with the only difference being that in the case of Dom0,|d->arch.paging.freelist |could be extended.Do I understand your idea correctly?
Broadly yes, but not in the details. For example, I don't think such a page allocated from the general heap would want appending to freelist. Commentary and alike also would want tidying.
And of course going forward, for split hardware and control domains the latter may want similar treatment.
Jan
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 21 lines · Jan Beulich
Quoted 20 lines · Oleksii Kurochko
Quoted 19 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
Quoted 8 lines
Quoted 4 lines
It doesn't matter whether it is an intermediate page table or a leaf PTE pointing to a page — PTE should be valid. Considering that in the current implementation it’s possible for PTE.v = 0 but P2M.v = 1, it is better to check P2M.v instead of PTE.v.
I'm confused by this reply. If you want to name 2nd level page table entries P2M - fine (but unhelpful). But then for any memory access there's only one of the two involved: A PTE (Xen accesses) or a P2M (guest accesses). Hence how can there be "PTE.v = 0 but P2M.v = 1"?
I think I understand your confusion, let me try to rephrase.
The reason for having both|p2m_is_valid()| and|pte_is_valid()| is that I want to have the ability to use the P2M PTE valid bit to track which pages were accessed by a vCPU, so that cleaning and invalidating RAM associated with the guest vCPU won't be too expensive, for example.
In this case, the P2M PTE valid bit will be set to 0, but the P2M PTE type bits will be set to something other than|p2m_invalid| (even for a table entries), so when an MMU fault occurs, we can properly resolve it.
So, if the P2M PTE type (what|p2m_is_valid()| checks) is set to|p2m_invalid|, it means that the valid bit (what|pte_is_valid()| checks) should be set to 0, so the P2M PTE is genuinely invalid.
It could also be the case that the P2M PTE type isn't|p2m_invalid (and P2M PTE valid will be intentionally set to 0 to have ability to track which pages were accessed for the reason I wrote above)|, and when MMU fault occurs we could properly handle it and set to 1 P2M PTE valid bit to 1...
Quoted 3 lines
An intermediate page table entry is something Xen controls entirely. Hence it has no (guest induced) type.
... And actually it is a reason why it is needed to set a type even for an intermediate page table entry.
I hope now it is a lit bit clearer what and why was done.
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
Quoted 8 lines
xen/arch/riscv/p2m.c +3
-0@@ -492,6 +503,70 @@ static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t,return e; } +/* Generate table entry with correct attributes. */ +static pte_t page_to_p2m_table (struct p2m_domain *p2m,struct page_info *page) +{xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 518
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* Since this function generates a table entry, according to "Encoding +* of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0 +* to point to the next level of the page table. +* Therefore, to ensure that an entry is a page table entry, +* `p2m_access_n2rwx` is passed to `mfn_to_p2m_entry()` as the access value, +* which overrides whatever was passed as `p2m_type_t` and guarantees that +* the entry is a page table entry by setting r = w = x = 0. +*/ +return p2m_entry_from_mfn (p2m,page_to_mfn (page), p2m_ram_rw, p2m_access_n2rwx);Similarly P2M access shouldn't apply to intermediate page tables. (Moot with that, but (ab)using
p2m_access_n2rwxwould also look wrong: You did read what it means, didn't you?)
Quoted 8 lines
Quoted 7 lines
|p2m_access_n2rwx| was chosen not really because of the description mentioned near its declaration, but because it sets r=w=x=0, which RISC-V expects for a PTE that points to the next-level page table.
Generally, I agree that P2M access shouldn't be applied to intermediate page tables.
What I can suggest in this case is to use|p2m_access_rwx| instead of|p2m_access_n2rwx|,
No. p2m_access_* shouldn't come into play here at all.
Okay, then it seems like I just can't explicitly re-use p2m_pte_from_mfn() in page_to_p2m_table() and have to open-code p2m_pte_from_mfn() or add another one argument is_table to decide if p2m_access_t and/or p2m_type_t should be applied.
Quoted 2 lines
Period. Just like P2M types
shouldn't. As per above - intermediate page tables are Xen internal constructs.
Look please at the explaining above why p2m types is needed despite of the fact that logically it isn't really needed.
Quoted 89 lines
Quoted 26 lines
which will ensure that the P2M access type isn't applied when|p2m_entry_from_mfn() |is called, and then, after calling|p2m_entry_from_mfn()|, simply set|PTE.r,w,x=0|. So this function will look like:
/* Generate table entry with correct attributes. */ static pte_t page_to_p2m_table(struct p2m_domain *p2m, struct page_info *page) { /* * p2m_ram_rw is chosen for a table entry as p2m table should be valid * from both P2M and hardware point of view. * * p2m_access_rwx is chosen to restrict access permissions, what mean * do not apply access permission for a table entry */ pte_t pte = p2m_pte_from_mfn(p2m, page_to_mfn(page), _gfn(0), p2m_ram_rw, p2m_access_rwx); /* * Since this function generates a table entry, according to "Encoding * of PTE R/W/X fields," the entry's r, w, and x fields must be set to 0 * to point to the next level of the page table. */ pte.pte &= ~PTE_ACCESS_MASK; return pte; }Quoted 2 lines
Does this make sense? Or would it be better to keep the current version of |page_to_p2m_table()| and just improve the comment explaining why|p2m_access_n2rwx |is used for a table entry?
No to both, as per above.
Quoted 59 lines
Quoted 9 lines
Quoted 9 lines
xen/arch/riscv/p2m.c +9
-0+static struct page_info *p2m_alloc_page (struct domain *d) +{ +struct page_info *pg; + +/* +* For hardware domain, there should be no limit in the number of pages that +* can be allocated, so that the kernel may take advantage of the extended +* regions. Hence, allocate p2m pages for hardware domains from heap. +*/ Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 535
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+if (is_hardware_domain (d) ) + { + pg =alloc_domheap_page (d, MEMF_no_owner); +if ( pg ==NULL ) +printk (XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n" ); + }The comment looks to have been taken verbatim from Arm. Whatever "extended regions" are, does the same concept even exist on RISC-V?
Initially, I missed that it’s used only for Arm. Since it was mentioned in |doc/misc/xen-command-line.pandoc|, I assumed it applied to all architectures. But now I see that it’s Arm-specific:: ###
ext_regions(Arm)Quoted 5 lines
Also, special casing Dom0 like this has benefits, but also comes with a
pitfall: If the system's out of memory, allocations will fail. A pre- populated pool would avoid that (until exhausted, of course). If special- casing of Dom0 is needed, I wonder whether ...Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 541
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+else + { +spin_lock (&d->arch.paging.lock); + pg =page_list_remove_head (&d->arch.paging.p2m_freelist); +spin_unlock (&d->arch.paging.lock); + }... going this path but with a Dom0-only fallback to general allocation wouldn't be the better route.
IIUC, then it should be something like:
static struct page_info *p2m_alloc_page(struct domain *d) { struct page_info *pg; spin_lock(&d->arch.paging.lock); pg = page_list_remove_head(&d->arch.paging.p2m_freelist); spin_unlock(&d->arch.paging.lock); if ( !pg && is_hardware_domain(d) ) { /* Need to allocate more memory from domheap */ pg = alloc_domheap_page(d, MEMF_no_owner); if ( pg == NULL ) { printk(XENLOG_ERR "Failed to allocate pages.\n"); return pg; } ACCESS_ONCE(d->arch.paging.total_pages)++; page_list_add_tail(pg, &d->arch.paging.freelist); } return pg; }
Quoted 7 lines
Quoted 4 lines
And basically use|d->arch.paging.freelist| for both dom0less and dom0 domains,
with the only difference being that in the case of Dom0,|d->arch.paging.freelist |could be extended.Do I understand your idea correctly?
Broadly yes, but not in the details. For example, I don't think such a page allocated from the general heap would want appending to freelist. Commentary and alike also would want tidying.
Could you please explain why it wouldn't want appending to freelist?
Quoted 3 lines
And of course going forward, for split hardware and control domains the latter may want similar treatment.
Could you please clarify what is the difference between hardware and control domains?
I thought that it is the same or is it for the case when we have dom0 (control domain) which runs domD (hardware domain) and guest domain?
Thanks.
~ Oleksii
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 31 lines · Oleksii Kurochko
Quoted 21 lines · Jan Beulich
Quoted 20 lines · Oleksii Kurochko
Quoted 19 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
Quoted 8 lines
Quoted 4 lines
It doesn't matter whether it is an intermediate page table or a leaf PTE pointing to a page — PTE should be valid. Considering that in the current implementation it’s possible for PTE.v = 0 but P2M.v = 1, it is better to check P2M.v instead of PTE.v.
I'm confused by this reply. If you want to name 2nd level page table entries P2M - fine (but unhelpful). But then for any memory access there's only one of the two involved: A PTE (Xen accesses) or a P2M (guest accesses). Hence how can there be "PTE.v = 0 but P2M.v = 1"?
Quoted 6 lines
I think I understand your confusion, let me try to rephrase.
The reason for having both|p2m_is_valid()| and|pte_is_valid()| is that I want to have the ability to use the P2M PTE valid bit to track which pages were accessed by a vCPU, so that cleaning and invalidating RAM associated with the guest vCPU won't be too expensive, for example.
I don't know what you're talking about here.
Quoted 13 lines
In this case, the P2M PTE valid bit will be set to 0, but the P2M PTE type bits will be set to something other than|p2m_invalid| (even for a table entries), so when an MMU fault occurs, we can properly resolve it.
So, if the P2M PTE type (what|p2m_is_valid()| checks) is set to|p2m_invalid|, it means that the valid bit (what|pte_is_valid()| checks) should be set to 0, so the P2M PTE is genuinely invalid.
It could also be the case that the P2M PTE type isn't|p2m_invalid (and P2M PTE valid will be intentionally set to 0 to have ability to track which pages were accessed for the reason I wrote above)|, and when MMU fault occurs we could properly handle it and set to 1 P2M PTE valid bit to 1...
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
An intermediate page table entry is something Xen controls entirely. Hence it has no (guest induced) type.
... And actually it is a reason why it is needed to set a type even for an intermediate page table entry.
I hope now it is a lit bit clearer what and why was done.
Sadly not. I still don't see what use the P2M type in of an intermediate page table is going to be. It surely can't reliably describe all of the entries that page table holds. Intermediate page tables and leaf pages are just too different to share a concept like this, I think. That said, I'll be happy to be shown code demonstrating the contrary.
Quoted 34 lines
Quoted 34 lines
Quoted 34 lines
Quoted 9 lines
Quoted 9 lines
xen/arch/riscv/p2m.c +9
-0+static struct page_info *p2m_alloc_page (struct domain *d) +{ +struct page_info *pg; + +/* +* For hardware domain, there should be no limit in the number of pages that +* can be allocated, so that the kernel may take advantage of the extended +* regions. Hence, allocate p2m pages for hardware domains from heap. +*/ Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 535
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+if (is_hardware_domain (d) ) + { + pg =alloc_domheap_page (d, MEMF_no_owner); +if ( pg ==NULL ) +printk (XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n" ); + }The comment looks to have been taken verbatim from Arm. Whatever "extended regions" are, does the same concept even exist on RISC-V?
Initially, I missed that it’s used only for Arm. Since it was mentioned in |doc/misc/xen-command-line.pandoc|, I assumed it applied to all architectures. But now I see that it’s Arm-specific:: ###
ext_regions(Arm)Quoted 13 lines
Also, special casing Dom0 like this has benefits, but also comes with a
pitfall: If the system's out of memory, allocations will fail. A pre- populated pool would avoid that (until exhausted, of course). If special- casing of Dom0 is needed, I wonder whether ...xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 541
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+else + { +spin_lock (&d->arch.paging.lock); + pg =page_list_remove_head (&d->arch.paging.p2m_freelist); +spin_unlock (&d->arch.paging.lock); + }... going this path but with a Dom0-only fallback to general allocation wouldn't be the better route.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
IIUC, then it should be something like:
static struct page_info *p2m_alloc_page(struct domain *d) { struct page_info *pg; spin_lock(&d->arch.paging.lock); pg = page_list_remove_head(&d->arch.paging.p2m_freelist);
Note this: Here you _remove_ from freelist, because you want to actually use the page. Then clearly ...
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
spin_unlock(&d->arch.paging.lock); if ( !pg && is_hardware_domain(d) ) { /* Need to allocate more memory from domheap */ pg = alloc_domheap_page(d, MEMF_no_owner); if ( pg == NULL ) { printk(XENLOG_ERR "Failed to allocate pages.\n"); return pg; } ACCESS_ONCE(d->arch.paging.total_pages)++; page_list_add_tail(pg, &d->arch.paging.freelist); } return pg; }And basically use|d->arch.paging.freelist| for both dom0less and dom0 domains,
with the only difference being that in the case of Dom0,|d->arch.paging.freelist |could be extended.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Do I understand your idea correctly?
Broadly yes, but not in the details. For example, I don't think such a page allocated from the general heap would want appending to freelist. Commentary and alike also would want tidying.
Could you please explain why it wouldn't want appending to freelist?
... adding to freelist here is wrong: You want to use this separately allocated page, too. Else once it is freed it'll be added to freelist a 2nd time, leading to a corrupt list.
Quoted 7 lines
Quoted 2 lines
And of course going forward, for split hardware and control domains the latter may want similar treatment.
Could you please clarify what is the difference between hardware and control domains?
I thought that it is the same or is it for the case when we have dom0 (control domain) which runs domD (hardware domain) and guest domain?
That's the common case, yes, but conceptually the two can be separate. And if you've followed recent discussions on the list you would also have noticed that work is being done in that direction. (But this was really a forward-looking comment; I didn't mean to make you cover that case right away. Just wanted you to be aware.)
Jan
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 31 lines · Jan Beulich
Quoted 30 lines · Oleksii Kurochko
Quoted 29 lines · Jan Beulich
Quoted 20 lines · Oleksii Kurochko
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }Quoted 12 lines
xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
Quoted 4 lines
It doesn't matter whether it is an intermediate page table or a leaf PTE pointing to a page — PTE should be valid. Considering that in the current implementation it’s possible for PTE.v = 0 but P2M.v = 1, it is better to check P2M.v instead of PTE.v.
I'm confused by this reply. If you want to name 2nd level page table entries P2M - fine (but unhelpful). But then for any memory access there's only one of the two involved: A PTE (Xen accesses) or a P2M (guest accesses). Hence how can there be "PTE.v = 0 but P2M.v = 1"?
Quoted 7 lines
Quoted 6 lines
I think I understand your confusion, let me try to rephrase.
The reason for having both|p2m_is_valid()| and|pte_is_valid()| is that I want to have the ability to use the P2M PTE valid bit to track which pages were accessed by a vCPU, so that cleaning and invalidating RAM associated with the guest vCPU won't be too expensive, for example.
I don't know what you're talking about here.
https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/arm/mmu/p2m.c#L1649
Quoted 18 lines
Quoted 18 lines
In this case, the P2M PTE valid bit will be set to 0, but the P2M PTE type bits will be set to something other than|p2m_invalid| (even for a table entries), so when an MMU fault occurs, we can properly resolve it.
So, if the P2M PTE type (what|p2m_is_valid()| checks) is set to|p2m_invalid|, it means that the valid bit (what|pte_is_valid()| checks) should be set to 0, so the P2M PTE is genuinely invalid.
It could also be the case that the P2M PTE type isn't|p2m_invalid (and P2M PTE valid will be intentionally set to 0 to have ability to track which pages were accessed for the reason I wrote above)|, and when MMU fault occurs we could properly handle it and set to 1 P2M PTE valid bit to 1...
Quoted 2 lines
An intermediate page table entry is something Xen controls entirely. Hence it has no (guest induced) type.
... And actually it is a reason why it is needed to set a type even for an intermediate page table entry.
Quoted 6 lines
Quoted 1 line
I hope now it is a lit bit clearer what and why was done.
Sadly not. I still don't see what use the P2M type in of an intermediate page table is going to be. It surely can't reliably describe all of the entries that page table holds. Intermediate page tables and leaf pages are just too different to share a concept like this, I think. That said, I'll be happy to be shown code demonstrating the contrary.
Then it is needed to introduce new p2m_type_t - p2m_table and use it. Would it be better?
I still need some type to have ability to distinguish if p2m is valid or not from p2m management and hardware point of view.
If there is no need for such distinguish why all archs introduce p2m_invalid? Isn't enough just to use P2M PTE valid bit?
Quoted 66 lines
Quoted 35 lines
Quoted 35 lines
Quoted 35 lines
Quoted 10 lines
Quoted 10 lines
xen/arch/riscv/p2m.c +9
-0+static struct page_info *p2m_alloc_page (struct domain *d) +{ +struct page_info *pg; + +/* +* For hardware domain, there should be no limit in the number of pages that +* can be allocated, so that the kernel may take advantage of the extended +* regions. Hence, allocate p2m pages for hardware domains from heap. +*/ Quoted 8 lines
xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 535
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+if (is_hardware_domain (d) ) + { + pg =alloc_domheap_page (d, MEMF_no_owner); +if ( pg ==NULL ) +printk (XENLOG_G_ERR "Failed to allocate P2M pages for hwdom.\n" ); + }The comment looks to have been taken verbatim from Arm. Whatever "extended regions" are, does the same concept even exist on RISC-V?
Initially, I missed that it’s used only for Arm. Since it was mentioned in |doc/misc/xen-command-line.pandoc|, I assumed it applied to all architectures. But now I see that it’s Arm-specific:: ###
ext_regions(Arm)Quoted 13 lines
Also, special casing Dom0 like this has benefits, but also comes with a
pitfall: If the system's out of memory, allocations will fail. A pre- populated pool would avoid that (until exhausted, of course). If special- casing of Dom0 is needed, I wonder whether ...xen/arch/riscv/p2m.c static pte_t p2m_entry_from_mfn(struct p2m_domain *p2m, mfn_t mfn, p2m_type_t t, · line 541
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+else + { +spin_lock (&d->arch.paging.lock); + pg =page_list_remove_head (&d->arch.paging.p2m_freelist); +spin_unlock (&d->arch.paging.lock); + }... going this path but with a Dom0-only fallback to general allocation wouldn't be the better route.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
IIUC, then it should be something like:
static struct page_info *p2m_alloc_page(struct domain *d) { struct page_info *pg; spin_lock(&d->arch.paging.lock); pg = page_list_remove_head(&d->arch.paging.p2m_freelist);Note this: Here you _remove_ from freelist, because you want to actually use the page. Then clearly ...
Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
spin_unlock(&d->arch.paging.lock); if ( !pg && is_hardware_domain(d) ) { /* Need to allocate more memory from domheap */ pg = alloc_domheap_page(d, MEMF_no_owner); if ( pg == NULL ) { printk(XENLOG_ERR "Failed to allocate pages.\n"); return pg; } ACCESS_ONCE(d->arch.paging.total_pages)++; page_list_add_tail(pg, &d->arch.paging.freelist); } return pg; }And basically use|d->arch.paging.freelist| for both dom0less and dom0 domains,
with the only difference being that in the case of Dom0,|d->arch.paging.freelist |could be extended.
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
Do I understand your idea correctly?
Broadly yes, but not in the details. For example, I don't think such a page allocated from the general heap would want appending to freelist. Commentary and alike also would want tidying.
Could you please explain why it wouldn't want appending to freelist?
... adding to freelist here is wrong: You want to use this separately allocated page, too. Else once it is freed it'll be added to freelist a 2nd time, leading to a corrupt list.
Got it, I understand why it shouldn’t be added to the freelist.
Incrementing total_pages still makes sense, right?
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
And of course going forward, for split hardware and control domains the latter may want similar treatment.
Could you please clarify what is the difference between hardware and control domains?
I thought that it is the same or is it for the case when we have dom0 (control domain) which runs domD (hardware domain) and guest domain?
Quoted 5 lines
That's the common case, yes, but conceptually the two can be separate. And if you've followed recent discussions on the list you would also have noticed that work is being done in that direction. (But this was really a forward-looking comment; I didn't mean to make you cover that case right away. Just wanted you to be aware.)
Thanks.
~ Oleksii
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 34 lines · Oleksii Kurochko
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Oleksii Kurochko
Quoted 21 lines · Jan Beulich
Quoted 20 lines · Oleksii Kurochko
Quoted 19 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
Quoted 8 lines
Quoted 4 lines
It doesn't matter whether it is an intermediate page table or a leaf PTE pointing to a page — PTE should be valid. Considering that in the current implementation it’s possible for PTE.v = 0 but P2M.v = 1, it is better to check P2M.v instead of PTE.v.
I'm confused by this reply. If you want to name 2nd level page table entries P2M - fine (but unhelpful). But then for any memory access there's only one of the two involved: A PTE (Xen accesses) or a P2M (guest accesses). Hence how can there be "PTE.v = 0 but P2M.v = 1"?
I think I understand your confusion, let me try to rephrase.
Quoted 7 lines
Quoted 5 lines
Quoted 4 lines
The reason for having both|p2m_is_valid()| and|pte_is_valid()| is that I want to have the ability to use the P2M PTE valid bit to track which pages were accessed by a vCPU, so that cleaning and invalidating RAM associated with the guest vCPU won't be too expensive, for example.
I don't know what you're talking about here.
https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/arm/mmu/p2m.c#L1649
How does that Arm function matter here? Aiui you don't need anything like that in RISC-V, as there caches don't need disabling temporarily.
Quoted 24 lines
Quoted 17 lines
Quoted 17 lines
In this case, the P2M PTE valid bit will be set to 0, but the P2M PTE type bits will be set to something other than|p2m_invalid| (even for a table entries), so when an MMU fault occurs, we can properly resolve it.
So, if the P2M PTE type (what|p2m_is_valid()| checks) is set to|p2m_invalid|, it means that the valid bit (what|pte_is_valid()| checks) should be set to 0, so the P2M PTE is genuinely invalid.
It could also be the case that the P2M PTE type isn't|p2m_invalid (and P2M PTE valid will be intentionally set to 0 to have ability to track which pages were accessed for the reason I wrote above)|, and when MMU fault occurs we could properly handle it and set to 1 P2M PTE valid bit to 1...
Quoted 2 lines
An intermediate page table entry is something Xen controls entirely. Hence it has no (guest induced) type.
... And actually it is a reason why it is needed to set a type even for an intermediate page table entry.
Quoted 6 lines
Quoted 1 line
I hope now it is a lit bit clearer what and why was done.
Sadly not. I still don't see what use the P2M type in of an intermediate page table is going to be. It surely can't reliably describe all of the entries that page table holds. Intermediate page tables and leaf pages are just too different to share a concept like this, I think. That said, I'll be happy to be shown code demonstrating the contrary.
Quoted 7 lines
Then it is needed to introduce new
p2m_type_t-p2m_tableand use it. Would it be better?I still need some type to have ability to distinguish if p2m is valid or not from p2m management and hardware point of view.
If there is no need for such distinguish why all archs introducep2m_invalid? Isn't enough just to use P2M PTE valid bit?
At least on x86 we don't tag intermediate page tables with P2M types. For ordinary leaf entries the situation is different, as there may be varying reasons why a PTE has its valid (on x86: present) bit cleared. Hence the type is relevant there, just to know what to do when a page is accessed through such a not-present PTE.
Jan
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 35 lines · Jan Beulich
Quoted 34 lines · Oleksii Kurochko
Quoted 33 lines · Jan Beulich
Quoted 32 lines · Oleksii Kurochko
Quoted 21 lines · Jan Beulich
Quoted 20 lines · Oleksii Kurochko
Quoted 19 lines · Jan Beulich
Quoted 6 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +0
-0@@ -387,6 +387,17 @@ static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte)return p2m_type_radix_get (p2m, pte) != p2m_invalid; }xen/arch/riscv/p2m.c static inline bool p2me_is_valid(struct p2m_domain *p2m, pte_t pte) · line 399
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+/* +* pte_is_* helpers are checking the valid bit set in the +* PTE but we have to check p2m_type instead (look at the comment above +* p2me_is_valid()) +* Provide our own overlay to check the valid bit. +*/ +static inline bool p2me_is_mapping (struct p2m_domain *p2m,pte_t pte) +{ +return p2me_is_valid (p2m, pte) && (pte.pte &PTE_ACCESS_MASK ); +}Same question as on the earlier patch - does P2M type apply to intermediate page tables at all? (Conceptually it shouldn't.)
Quoted 8 lines
Quoted 4 lines
It doesn't matter whether it is an intermediate page table or a leaf PTE pointing to a page — PTE should be valid. Considering that in the current implementation it’s possible for PTE.v = 0 but P2M.v = 1, it is better to check P2M.v instead of PTE.v.
I'm confused by this reply. If you want to name 2nd level page table entries P2M - fine (but unhelpful). But then for any memory access there's only one of the two involved: A PTE (Xen accesses) or a P2M (guest accesses). Hence how can there be "PTE.v = 0 but P2M.v = 1"?
I think I understand your confusion, let me try to rephrase.
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
The reason for having both|p2m_is_valid()| and|pte_is_valid()| is that I want to have the ability to use the P2M PTE valid bit to track which pages were accessed by a vCPU, so that cleaning and invalidating RAM associated with the guest vCPU won't be too expensive, for example.
I don't know what you're talking about here.
https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/arm/mmu/p2m.c#L1649
How does that Arm function matter here? Aiui you don't need anything like that in RISC-V, as there caches don't need disabling temporarily.
I thought that it could be needed not only in the case when a d-cache is disabled temporarily, but it seems like that I was just wrong and all other cases are handled by cache coherency protocol.
Quoted 31 lines
Quoted 24 lines
Quoted 18 lines
Quoted 18 lines
In this case, the P2M PTE valid bit will be set to 0, but the P2M PTE type bits will be set to something other than|p2m_invalid| (even for a table entries), so when an MMU fault occurs, we can properly resolve it.
So, if the P2M PTE type (what|p2m_is_valid()| checks) is set to|p2m_invalid|, it means that the valid bit (what|pte_is_valid()| checks) should be set to 0, so the P2M PTE is genuinely invalid.
It could also be the case that the P2M PTE type isn't|p2m_invalid (and P2M PTE valid will be intentionally set to 0 to have ability to track which pages were accessed for the reason I wrote above)|, and when MMU fault occurs we could properly handle it and set to 1 P2M PTE valid bit to 1...
Quoted 2 lines
An intermediate page table entry is something Xen controls entirely. Hence it has no (guest induced) type.
... And actually it is a reason why it is needed to set a type even for an intermediate page table entry.
Quoted 6 lines
Quoted 1 line
I hope now it is a lit bit clearer what and why was done.
Sadly not. I still don't see what use the P2M type in of an intermediate page table is going to be. It surely can't reliably describe all of the entries that page table holds. Intermediate page tables and leaf pages are just too different to share a concept like this, I think. That said, I'll be happy to be shown code demonstrating the contrary.
Quoted 7 lines
Then it is needed to introduce new
p2m_type_t-p2m_tableand use it. Would it be better?I still need some type to have ability to distinguish if p2m is valid or not from p2m management and hardware point of view.
If there is no need for such distinguish why all archs introducep2m_invalid? Isn't enough just to use P2M PTE valid bit?
Quoted 5 lines
At least on x86 we don't tag intermediate page tables with P2M types. For ordinary leaf entries the situation is different, as there may be varying reasons why a PTE has its valid (on x86: present) bit cleared. Hence the type is relevant there, just to know what to do when a page is accessed through such a not-present PTE.
I think that I got your idea now.
Does it make sense to have such optimization when we have 2Mb memory range and it was mapped using 4k pages instead of 1 super-page, could it be useful to invalidate just just page table entry of L1 which corresponds to the start of this 2mb memory range, instead of invalidating each entry on L0? If it could useful then intermediate page tables should be tagged too. Arm has such use cases:
https://gitlab.com/xen-project/people/olkur/xen/-/blob/staging/xen/arch/arm/mmu/p2m.c#L1286
~ OLeksii
Re: [PATCH v2 14/17] xen/riscv: implement p2m_next_level()
Quoted 42 lines · Oleksii Kurochko
Quoted 33 lines · Jan Beulich
Quoted 25 lines · Oleksii Kurochko
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
In this case, the P2M PTE valid bit will be set to 0, but the P2M PTE type bits will be set to something other than|p2m_invalid| (even for a table entries), so when an MMU fault occurs, we can properly resolve it.
So, if the P2M PTE type (what|p2m_is_valid()| checks) is set to|p2m_invalid|, it means that the valid bit (what|pte_is_valid()| checks) should be set to 0, so the P2M PTE is genuinely invalid.
It could also be the case that the P2M PTE type isn't|p2m_invalid (and P2M PTE valid will be intentionally set to 0 to have ability to track which pages were accessed for the reason I wrote above)|, and when MMU fault occurs we could properly handle it and set to 1 P2M PTE valid bit to 1...
Quoted 2 lines
An intermediate page table entry is something Xen controls entirely. Hence it has no (guest induced) type.
... And actually it is a reason why it is needed to set a type even for an intermediate page table entry.
Quoted 6 lines
Quoted 1 line
I hope now it is a lit bit clearer what and why was done.
Sadly not. I still don't see what use the P2M type in of an intermediate page table is going to be. It surely can't reliably describe all of the entries that page table holds. Intermediate page tables and leaf pages are just too different to share a concept like this, I think. That said, I'll be happy to be shown code demonstrating the contrary.
Quoted 7 lines
Then it is needed to introduce new
p2m_type_t-p2m_tableand use it. Would it be better?I still need some type to have ability to distinguish if p2m is valid or not from p2m management and hardware point of view.
If there is no need for such distinguish why all archs introducep2m_invalid? Isn't enough just to use P2M PTE valid bit?Quoted 5 lines
At least on x86 we don't tag intermediate page tables with P2M types. For ordinary leaf entries the situation is different, as there may be varying reasons why a PTE has its valid (on x86: present) bit cleared. Hence the type is relevant there, just to know what to do when a page is accessed through such a not-present PTE.
I think that I got your idea now.
Quoted 7 lines
Does it make sense to have such optimization when we have 2Mb memory range and it was mapped using 4k pages instead of 1 super-page, could it be useful to invalidate just just page table entry of L1 which corresponds to the start of this 2mb memory range, instead of invalidating each entry on L0? If it could useful then intermediate page tables should be tagged too. Arm has such use cases:
https://gitlab.com/xen-project/people/olkur/xen/-/blob/staging/xen/arch/arm/mmu/p2m.c#L1286
I don't currently see how that's related to the topic at hand.
Furthermore range-constrained TLB flushing is never at just an address, i.e. "L1 which corresponds to the start of this 2mb memory range" isn't meaningful here. It's always a range (typically expressed by address and size), and it always needs to be the full range that is invalidated. This can be a solitary low-level flush operation when you know a large page mapping would _not_ be split. When splitting is done in software or when hardware may split behind your back, you always need to invalidate the entire range. Or else, in your example, 4k TLB entries may remain for any but the first page of the 2M super-page. (Whether such a range can still be done in a single invalidation operation is a separate question. But I don't see how maintaining the type at the L1 level would help there.)
Jan
PATCH v2 15/17 xen/riscv: Implement superpage splitting for p2m mappings
In reply to Oleksii KurochkoAdd support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introduce p2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
- __p2m_set_entry() updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.
This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table. Additionally, the page table walk logic has been adjusted, as ARM uses the opposite walk order compared to RISC-V.
---
Changes in V2:
- New patch. It was a part of a big patch "xen/riscv: implement p2m mapping functionality" which was splitted to smaller.
- Update the commit above the cycle which creates new page table as RISC-V travserse page tables in an opposite to ARM order.
- RISC-V doesn't require BBM so there is no needed for invalidating and TLB flushing before updating PTE.
--- xen/arch/riscv/p2m.c | 102 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-)
xen/arch/riscv/p2m.c +101 -1
@@ -743,6 +743,77 @@ static void p2m_free_entry(struct p2m_domain *p2m,p2m_free_page (p2m->domain, pg); } +static bool p2m_split_superpage (struct p2m_domain *p2m,pte_t *entry, +unsigned int level,unsigned int target, +const unsigned int *offsets) +{ +struct page_info *page; +unsigned int i; +pte_t pte, *table; +bool rv = true; + +/* Convenience aliases */ +mfn_t mfn =pte_get_mfn (*entry); +unsigned int next_level = level -1 ; +unsigned int level_order =XEN_PT_LEVEL_ORDER (next_level); + +/* +* This should only be called with target != level and the entry is +* a superpage. +*/ +ASSERT (level > target); +ASSERT (p2me_is_superpage (p2m, *entry, level)); + + page =p2m_alloc_page (p2m->domain); +if ( !page ) +return false; + +page_list_add (page, &p2m->pages); + table =__map_domain_page (page); + +/* +* We are either splitting a second level 1G page into 512 first level +* 2M pages, or a first level 2M page into 512 zero level 4K pages. +*/ +for ( i =0 ; i <XEN_PT_ENTRIES ; i++ ) + { +pte_t *new_entry = table + i; + +/* +* Use the content of the superpage entry and override +* the necessary fields. So the correct permission are kept. +*/ + pte = *entry; +pte_set_mfn (&pte,mfn_add (mfn, i << level_order)); + +write_pte (new_entry, pte); + } + +/* +* Shatter superpage in the page to the level we want to make the +* changes. +* This is done outside the loop to avoid checking the offset to +* know whether the entry should be shattered for every entry. +*/ +if ( next_level != target ) + rv =p2m_split_superpage (p2m, table + offsets[next_level], + level -1 , target, offsets); + +/* TODO: why it is necessary to have clean here? Not somewhere in the caller */ +if ( p2m->clean_pte ) +clean_dcache_va_range (table,PAGE_SIZE ); + +unmap_domain_page (table); + +/* +* Even if we failed, we should install the newly allocated PTE +* entry. The caller will be in charge to free the sub-tree. +*/ +p2m_write_pte (entry,page_to_p2m_table (p2m, page), p2m->clean_pte); + +return rv; +} +/* * Insert an entry in the p2m. This should be called with a mapping * equal to a page/superpage.
@@ -806,7 +877,36 @@ static int __p2m_set_entry(struct p2m_domain *p2m,*/ if ( level > target ) { -panic ("Shattering isn't implemented\n" ); +/* We need to split the original page. */ +pte_t split_pte = *entry; + +ASSERT (p2me_is_superpage (p2m, *entry, level)); + +if ( !p2m_split_superpage (p2m, &split_pte, level, target, offsets) ) + { +/* Free the allocated sub-tree */ +p2m_free_entry (p2m, split_pte, level); + + rc = -ENOMEM ; + goto out; + } + +p2m_write_pte (entry, split_pte, p2m->clean_pte); + +/* Then move to the level we want to make real changes */ +for ( ; level < target; level++ ) + { + rc =p2m_next_level (p2m, true, level, &table, offsets[level]); + +/* +* The entry should be found and either be a table +* or a superpage if level 0 is not targeted +*/ +ASSERT (rc ==GUEST_TABLE_NORMAL || + (rc ==GUEST_TABLE_SUPER_PAGE && target >0 )); + } + + entry = table + offsets[level]; }/*
Signature · 1 line
2.49.0
Re: [PATCH v2 15/17] xen/riscv: Implement superpage splitting for p2m mappings
Quoted 11 lines · Oleksii Kurochko
Add support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introducep2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
-__p2m_set_entry()updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.
Quoted 4 lines
This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table.
But some flushing is going to be necessary. As long as you only ever do global flushes, the one after the individual PTE modification (within the split table) will do (if BBM isn't required, see below), but once you move to more fine-grained flushing, that's not going to be enough anymore. Not sure it's a good idea to leave such a pitfall.
As to (no need for) BBM: I couldn't find anything to that effect in the privileged spec. Can you provide some pointer? What I found instead is e.g. this sentence: "To ensure that implicit reads observe writes to the same memory locations, an SFENCE.VMA instruction must be executed after the writes to flush the relevant cached translations." And this: "Accessing the same location using different cacheability attributes may cause loss of coherence." (This may not only occur when the same physical address is mapped twice at different VAs, but also after the shattering of a superpage when the new entry differs in cacheability.)
Quoted 2 lines
Additionally, the page table walk logic has been adjusted, as ARM uses the opposite walk order compared to RISC-V.
I think you used some similar wording already in an earlier patch. I find this confusing: Walk order is, aiui, the same. It's merely the numbering of levels that is the opposite way round, isn't it?
Quoted 37 lines
Signed-off-by Oleksii Kurochko <[email protected]>---
Changes in V2:
- New patch. It was a part of a big patch "xen/riscv: implement p2m mapping functionality" which was splitted to smaller.
- Update the commit above the cycle which creates new page table as RISC-V travserse page tables in an opposite to ARM order.
- RISC-V doesn't require BBM so there is no needed for invalidating and TLB flushing before updating PTE.--- xen/arch/riscv/p2m.c | 102 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-)xen/arch/riscv/p2m.c +16
-0@@ -743,6 +743,77 @@ static void p2m_free_entry(struct p2m_domain *p2m,p2m_free_page (p2m->domain, pg); } +static bool p2m_split_superpage (struct p2m_domain *p2m,pte_t *entry, +unsigned int level,unsigned int target, +const unsigned int *offsets) +{ +struct page_info *page; +unsigned int i; +pte_t pte, *table; +bool rv = true; + +/* Convenience aliases */ +mfn_t mfn =pte_get_mfn (*entry); +unsigned int next_level = level -1 ; +unsigned int level_order =XEN_PT_LEVEL_ORDER (next_level); + +/* +* This should only be called with target != level and the entry is
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 771
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+* a superpage. +*/ +ASSERT (level > target); +ASSERT (p2me_is_superpage (p2m, *entry, level)); + + page =p2m_alloc_page (p2m->domain); +if ( !page ) +return false; + +page_list_add (page, &p2m->pages);
Is there a reason this list maintenance isn't done in p2m_alloc_page()?
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+ table =__map_domain_page (page); + +/* +* We are either splitting a second level 1G page into 512 first level +* 2M pages, or a first level 2M page into 512 zero level 4K pages. +*/
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 787
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+for ( i =0 ; i <XEN_PT_ENTRIES ; i++ ) + { +pte_t *new_entry = table + i; + +/* +* Use the content of the superpage entry and override +* the necessary fields. So the correct permission are kept. +*/ + pte = *entry; +pte_set_mfn (&pte,mfn_add (mfn, i << level_order));
While okay as long as you only permit superpages up to 1G, this is another trap for someone to fall into: Imo i would better be unsigned long right away, considering that RISC-V permits large pages at all levels.
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 800
Quoted 12 lines
xen/arch/riscv/p2m.c +12
-0+write_pte (new_entry, pte); + } + +/* +* Shatter superpage in the page to the level we want to make the +* changes. +* This is done outside the loop to avoid checking the offset to +* know whether the entry should be shattered for every entry. +*/ +if ( next_level != target ) + rv =p2m_split_superpage (p2m, table + offsets[next_level], + level -1 , target, offsets);
I don't understand the comment: Under what conditions would every entry need (further) shattering? And where's that happening? Or is this merely a word ordering issue in the sentence, and "for every entry" wants moving ahead? (In that case I'm unconvinced this is in need of commenting upon.)
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 806
Quoted 5 lines
xen/arch/riscv/p2m.c +5
-0+/* TODO: why it is necessary to have clean here? Not somewhere in the caller */ +if ( p2m->clean_pte ) +clean_dcache_va_range (table,PAGE_SIZE ); + +unmap_domain_page (table);
Again likely not something that wants taking care of right away, but there again is an inefficiency here: The caller almost certainly wants to map the same page again, to update the one entry that caused the request to shatter the page.
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 812
Quoted 5 lines
xen/arch/riscv/p2m.c +5
-0+/* +* Even if we failed, we should install the newly allocated PTE +* entry. The caller will be in charge to free the sub-tree. +*/ +p2m_write_pte (entry,page_to_p2m_table (p2m, page), p2m->clean_pte);
Why would it be wrong to free the page right here, vacating the entry at the same time (or leaving just that to the caller)? (IOW - if this is an implementation decision of yours, I think the word "should" would want dropping.) After all, the caller invoking p2m_free_entry() on the thus split PTE is less efficient (needs to iterate over all entries) than on the original one (where it's just a single superpage).
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 878
Quoted 3 lines
xen/arch/riscv/p2m.c +0
-0@@ -806,7 +877,36 @@ static int __p2m_set_entry(struct p2m_domain *p2m,*/ if ( level > target )
This condition is likely too strong, unless you actually mean to also split a superpage if it really wouldn't need splitting (new entry written still fitting with the superpage mapping, i.e. suitable MFN and same attributes).
Quoted 10 lines
xen/arch/riscv/p2m.c +8
-1{ -panic ("Shattering isn't implemented\n" ); +/* We need to split the original page. */ +pte_t split_pte = *entry; + +ASSERT (p2me_is_superpage (p2m, *entry, level)); + +if ( !p2m_split_superpage (p2m, &split_pte, level, target, offsets) ) + { +/* Free the allocated sub-tree */
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 897
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+p2m_free_entry (p2m, split_pte, level); + + rc = -ENOMEM ; + goto out; + } + +p2m_write_pte (entry, split_pte, p2m->clean_pte); + +/* Then move to the level we want to make real changes */ +for ( ; level < target; level++ )
Don't you mean to move downwards here? At which point I wonder: Did you test this code?
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 907
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ { + rc =p2m_next_level (p2m, true, level, &table, offsets[level]); + +/* +* The entry should be found and either be a table +* or a superpage if level 0 is not targeted +*/ +ASSERT (rc ==GUEST_TABLE_NORMAL || + (rc ==GUEST_TABLE_SUPER_PAGE && target >0 )); + }
This, too, is inefficient (but likely good enough as a starting point): You walk tables twice - first when splitting, and then again when finding the target level.
Considering the enclosing if(), this also again is a do/while() candidate.
Jan
Re: [PATCH v2 15/17] xen/riscv: Implement superpage splitting for p2m mappings
Quoted 16 lines · Jan Beulich
Quoted 11 lines · Oleksii Kurochko
Add support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introducep2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
-__p2m_set_entry()updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.Quoted 4 lines
This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table.
Quoted 5 lines
But some flushing is going to be necessary. As long as you only ever do global flushes, the one after the individual PTE modification (within the split table) will do (if BBM isn't required, see below), but once you move to more fine-grained flushing, that's not going to be enough anymore. Not sure it's a good idea to leave such a pitfall.
I think that I don't fully understand what is an issue.
Quoted 10 lines
As to (no need for) BBM: I couldn't find anything to that effect in the privileged spec. Can you provide some pointer? What I found instead is e.g. this sentence: "To ensure that implicit reads observe writes to the same memory locations, an SFENCE.VMA instruction must be executed after the writes to flush the relevant cached translations." And this: "Accessing the same location using different cacheability attributes may cause loss of coherence." (This may not only occur when the same physical address is mapped twice at different VAs, but also after the shattering of a superpage when the new entry differs in cacheability.)
I also couldn't find that RISC-V spec mandates BBM explicitly as Arm does it.
What I meant that on RISC-V can do:
- Write new PTE
- Flush TLB
While on Arm it is almost always needed to do:
- Write zero to PTE
- Flush TLB
- Write new PTE
For example, the common CoW code path where you copy from a read only page to a new page, then map that new page as writable just works on RISC-V without extra considerations and on Arm it requires BBM.
It seems to me that BBM is mandated for Arm only because that TLB is shared among cores, so there is no any guarantee that no prior entry for same VA remains in TLB. In case of RISC-V's TLB isn't shared and after a flush it is guaranteed that no prior entry for the same VA remains in the TLB.
But in the same time it could be cases, I guess, where BBM will be needed for RISC-V too. Even the case of CoW mentioned above will need some kind of BBM, but nothing that'll the CPU misbehave by doing CoW without BBM on RISC-V.
Quoted 6 lines
Quoted 3 lines
Additionally, the page table walk logic has been adjusted, as ARM uses the opposite walk order compared to RISC-V.
I think you used some similar wording already in an earlier patch. I find this confusing: Walk order is, aiui, the same. It's merely the numbering of levels that is the opposite way round, isn't it?
Yes, the numbering of levels is different and I counted that as a different walk order. If it is too confusing, I will reword it and use numbering of levels.
Quoted 41 lines
Quoted 41 lines
Signed-off-by Oleksii Kurochko<[email protected]>---
Changes in V2:
- New patch. It was a part of a big patch "xen/riscv: implement p2m mapping functionality" which was splitted to smaller.
- Update the commit above the cycle which creates new page table as RISC-V travserse page tables in an opposite to ARM order.
- RISC-V doesn't require BBM so there is no needed for invalidating and TLB flushing before updating PTE.--- xen/arch/riscv/p2m.c | 102 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 101 insertions(+), 1 deletion(-)xen/arch/riscv/p2m.c +19
-0@@ -743,6 +743,77 @@ static void p2m_free_entry(struct p2m_domain *p2m,p2m_free_page (p2m->domain, pg); } +static bool p2m_split_superpage (struct p2m_domain *p2m,pte_t *entry, +unsigned int level,unsigned int target, +const unsigned int *offsets) +{ +struct page_info *page; +unsigned int i; +pte_t pte, *table; +bool rv = true; + +/* Convenience aliases */ +mfn_t mfn =pte_get_mfn (*entry); +unsigned int next_level = level -1 ; +unsigned int level_order =XEN_PT_LEVEL_ORDER (next_level); + +/* +* This should only be called with target != level and the entry is +* a superpage. +*/ +ASSERT (level > target);
Quoted 8 lines
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 771
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+ASSERT (p2me_is_superpage (p2m, *entry, level)); + + page =p2m_alloc_page (p2m->domain); +if ( !page ) +return false; + +page_list_add (page, &p2m->pages);Is there a reason this list maintenance isn't done in
p2m_alloc_page()?
No there is no any reason, I will move that inside p2m_alloc_page().
Quoted 33 lines
Quoted 7 lines
xen/arch/riscv/p2m.c +6
-0+ table =__map_domain_page (page); + +/* +* We are either splitting a second level 1G page into 512 first level +* 2M pages, or a first level 2M page into 512 zero level 4K pages. +*/ xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 787
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+for ( i =0 ; i <XEN_PT_ENTRIES ; i++ ) + { +pte_t *new_entry = table + i; + +/* +* Use the content of the superpage entry and override +* the necessary fields. So the correct permission are kept. +*/ + pte = *entry; +pte_set_mfn (&pte,mfn_add (mfn, i << level_order));While okay as long as you only permit superpages up to 1G, this is another trap for someone to fall into: Imo i would better be unsigned long right away, considering that RISC-V permits large pages at all levels.
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 800
Quoted 12 lines
xen/arch/riscv/p2m.c +12
-0+write_pte (new_entry, pte); + } + +/* +* Shatter superpage in the page to the level we want to make the +* changes. +* This is done outside the loop to avoid checking the offset to +* know whether the entry should be shattered for every entry. +*/ +if ( next_level != target ) + rv =p2m_split_superpage (p2m, table + offsets[next_level], + level -1 , target, offsets);
Quoted 5 lines
I don't understand the comment: Under what conditions would every entry need (further) shattering? And where's that happening? Or is this merely a word ordering issue in the sentence, and "for every entry" wants moving ahead? (In that case I'm unconvinced this is in need of commenting upon.)
It is wording question. It should be something like:
+ /*
+ * Shatter superpage in the page to the level we want to make the + * changes. + * This is done outside the loop to avoid checking the offset for every + * entry (of new page table) to know whether the entry should be shattered. + */
Quoted 10 lines
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 806
Quoted 6 lines
xen/arch/riscv/p2m.c +5
-0+/* TODO: why it is necessary to have clean here? Not somewhere in the caller */ +if ( p2m->clean_pte ) +clean_dcache_va_range (table,PAGE_SIZE ); + +unmap_domain_page (table);Again likely not something that wants taking care of right away, but there again is an inefficiency here: The caller almost certainly wants to map the same page again, to update the one entry that caused the request to shatter the page.
I'll add TODO.
Quoted 6 lines
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 812
Quoted 6 lines
xen/arch/riscv/p2m.c +5
-0+/* +* Even if we failed, we should install the newly allocated PTE +* entry. The caller will be in charge to free the sub-tree. +*/ +p2m_write_pte (entry,page_to_p2m_table (p2m, page), p2m->clean_pte);
Quoted 6 lines
Why would it be wrong to free the page right here, vacating the entry at the same time (or leaving just that to the caller)? (IOW - if this is an implementation decision of yours, I think the word "should" would want dropping.) After all, the caller invoking
p2m_free_entry()on the thus split PTE is less efficient (needs to iterate over all entries) than on the original one (where it's just a single superpage).
I think that I didn't get your idea.
Quoted 8 lines
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 878
Quoted 4 lines
xen/arch/riscv/p2m.c +0
-0@@ -806,7 +877,36 @@ static int __p2m_set_entry(struct p2m_domain *p2m,*/ if ( level > target )This condition is likely too strong, unless you actually mean to also split a superpage if it really wouldn't need splitting (new entry written still fitting with the superpage mapping, i.e. suitable MFN and same attributes).
I am not really sure that I fully understand.
My understanding is if level != target then the splitting is needed, I am not really get the part "split a superpage if it really wouldn't need splitting".
Quoted 15 lines
Quoted 15 lines
xen/arch/riscv/p2m.c +12
-1{ -panic ("Shattering isn't implemented\n" ); +/* We need to split the original page. */ +pte_t split_pte = *entry; + +ASSERT (p2me_is_superpage (p2m, *entry, level)); + +if ( !p2m_split_superpage (p2m, &split_pte, level, target, offsets) ) + { +/* Free the allocated sub-tree */ +p2m_free_entry (p2m, split_pte, level); + + rc = -ENOMEM ; + goto out;
Quoted 8 lines
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 897
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+ } + +p2m_write_pte (entry, split_pte, p2m->clean_pte); + +/* Then move to the level we want to make real changes */ +for ( ; level < target; level++ )Don't you mean to move downwards here? At which point I wonder: Did you test this code?
No as the test for this case was missed. I will add one.
Quoted 11 lines
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 907
Quoted 11 lines
xen/arch/riscv/p2m.c +10
-0+ { + rc =p2m_next_level (p2m, true, level, &table, offsets[level]); + +/* +* The entry should be found and either be a table +* or a superpage if level 0 is not targeted +*/ +ASSERT (rc ==GUEST_TABLE_NORMAL || + (rc ==GUEST_TABLE_SUPER_PAGE && target >0 )); + }
Quoted 4 lines
This, too, is inefficient (but likely good enough as a starting point): You walk tables twice - first when splitting, and then again when finding the target level.
Considering the enclosing
if(), this also again is a do/while() candidate.
I will add TODO to make that part more efficient. And based on your reply regarding statement inside if(), I'll likely to use do/while().
Thanks.
~ Oleksii
Re: [PATCH v2 15/17] xen/riscv: Implement superpage splitting for p2m mappings
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Add support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introducep2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
-__p2m_set_entry()updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table.
Quoted 7 lines
Quoted 5 lines
But some flushing is going to be necessary. As long as you only ever do global flushes, the one after the individual PTE modification (within the split table) will do (if BBM isn't required, see below), but once you move to more fine-grained flushing, that's not going to be enough anymore. Not sure it's a good idea to leave such a pitfall.
I think that I don't fully understand what is an issue.
Whether a flush is necessary after solely breaking up a superpage is arch- defined. I don't know how much restrictions the spec on possible hardware behavior for RISC-V. However, the eventual change of (at least) one entry of fulfill the original request will surely require a flush. What I was trying to say is that this required flush would better not also cover for the flushes that may or may not be required by the spec. IOW if the spec leaves any room for flushes to possibly be needed, those flushes would better be explicit.
Quoted 16 lines
Quoted 9 lines
As to (no need for) BBM: I couldn't find anything to that effect in the privileged spec. Can you provide some pointer? What I found instead is e.g. this sentence: "To ensure that implicit reads observe writes to the same memory locations, an SFENCE.VMA instruction must be executed after the writes to flush the relevant cached translations." And this: "Accessing the same location using different cacheability attributes may cause loss of coherence." (This may not only occur when the same physical address is mapped twice at different VAs, but also after the shattering of a superpage when the new entry differs in cacheability.)
I also couldn't find that RISC-V spec mandates BBM explicitly as Arm does it.
What I meant that on RISC-V can do:
- Write new PTE
- Flush TLB
Quoted 8 lines
While on Arm it is almost always needed to do:
- Write zero to PTE
- Flush TLB
- Write new PTEFor example, the common CoW code path where you copy from a read only page to a new page, then map that new page as writable just works on RISC-V without extra considerations and on Arm it requires BBM.
CoW is a specific sub-case with increasing privilege.
Quoted 4 lines
It seems to me that BBM is mandated for Arm only because that TLB is shared among cores, so there is no any guarantee that no prior entry for same VA remains in TLB. In case of RISC-V's TLB isn't shared and after a flush it is guaranteed that no prior entry for the same VA remains in the TLB.
Not sure that's the sole reason. But again the question is: Is this written down explicitly anywhere? Generally there can be multiple levels of TLBs, and while some of them may be per-core, others may be shared.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
xen/arch/riscv/p2m.c +5
-0+/* +* Even if we failed, we should install the newly allocated PTE +* entry. The caller will be in charge to free the sub-tree. +*/ +p2m_write_pte (entry,page_to_p2m_table (p2m, page), p2m->clean_pte);
Quoted 8 lines
Quoted 6 lines
Why would it be wrong to free the page right here, vacating the entry at the same time (or leaving just that to the caller)? (IOW - if this is an implementation decision of yours, I think the word "should" would want dropping.) After all, the caller invoking
p2m_free_entry()on the thus split PTE is less efficient (needs to iterate over all entries) than on the original one (where it's just a single superpage).I think that I didn't get your idea.
Well, first and foremost this was a question to you, as it's not clear to me whether "should" is the correct word to use here. It would be appropriate if the spec mandated this behavior. It would seem less appropriate if this arrangement was an implementation choice of yours. And it looks to me as if the latter was the case here.
Quoted 11 lines
Quoted 7 lines
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 878
Quoted 3 lines
xen/arch/riscv/p2m.c +0
-0@@ -806,7 +877,36 @@ static int __p2m_set_entry(struct p2m_domain *p2m,*/ if ( level > target )This condition is likely too strong, unless you actually mean to also split a superpage if it really wouldn't need splitting (new entry written still fitting with the superpage mapping, i.e. suitable MFN and same attributes).
I am not really sure that I fully understand.
My understanding is if level != target then the splitting is needed, I am not really get the part "split a superpage if it really wouldn't need splitting".
Hmm, maybe I was wrong here. The caller determines at what level the actual change needs to occur? In which case what you have may be right.
Jan
Re: [PATCH v2 15/17] xen/riscv: Implement superpage splitting for p2m mappings
Quoted 24 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Add support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introducep2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
-__p2m_set_entry()updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table.
Quoted 6 lines
Quoted 5 lines
But some flushing is going to be necessary. As long as you only ever do global flushes, the one after the individual PTE modification (within the split table) will do (if BBM isn't required, see below), but once you move to more fine-grained flushing, that's not going to be enough anymore. Not sure it's a good idea to leave such a pitfall.
I think that I don't fully understand what is an issue.
Quoted 8 lines
Whether a flush is necessary after solely breaking up a superpage is arch- defined. I don't know how much restrictions the spec on possible hardware behavior for RISC-V. However, the eventual change of (at least) one entry of fulfill the original request will surely require a flush. What I was trying to say is that this required flush would better not also cover for the flushes that may or may not be required by the spec. IOW if the spec leaves any room for flushes to possibly be needed, those flushes would better be explicit.
I think that I still don't understand why what I wrote above will work as long as global flushes is working and will stop to work when more fine-grained flushing is used.
Inside p2m_split_superpage() we don't need any kind of TLB flush operation as it is allocation a new page for a table and works with it, so no one could use this page at the moment and cache it in TLB.
The only question is that if it is needed BBM before staring using splitted entry:
....
if ( !p2m_split_superpage(p2m, &split_pte, level, target, offsets) )
{
/* Free the allocated sub-tree */
p2m_free_subtree(p2m, split_pte, level);
rc = -ENOMEM;
goto out;
}------> /* Should be BBM used here ? */
p2m_write_pte(entry, split_pte, p2m->clean_pte);
And I can't find anything in the spec what tells me to use BBM here like Arm does:
/*
* Follow the break-before-sequence to update the entry.
* For more details see (D4.7.1 in ARM DDI 0487A.j).
*/
p2m_remove_pte(entry, p2m->clean_pte);
p2m_force_tlb_flush_sync(p2m);
p2m_write_pte(entry, split_pte, p2m->clean_pte);I agree that even BBM isn't mandated explicitly sometime it could be useful, but here I am not really sure what is the point to do TLB flush before p2m_write_pte() as nothing serious will happen if and old mapping will be used for a some time as we are keeping the same rights for smaller (splited) mapping.
The reason why Arm does p2m_remove_pte() & p2m_force_tlb_flush() before updating an entry here as it is doesn't guarantee that everything will be okay and they explicitly tells:
This situation can possibly break coherency and ordering guarantees, leading to inconsistent unwanted behavior in our Processing Element (PE).
Quoted 20 lines
Quoted 20 lines
Quoted 9 lines
As to (no need for) BBM: I couldn't find anything to that effect in the privileged spec. Can you provide some pointer? What I found instead is e.g. this sentence: "To ensure that implicit reads observe writes to the same memory locations, an SFENCE.VMA instruction must be executed after the writes to flush the relevant cached translations." And this: "Accessing the same location using different cacheability attributes may cause loss of coherence." (This may not only occur when the same physical address is mapped twice at different VAs, but also after the shattering of a superpage when the new entry differs in cacheability.)
I also couldn't find that RISC-V spec mandates BBM explicitly as Arm does it.
What I meant that on RISC-V can do:
- Write new PTE
- Flush TLBWhile on Arm it is almost always needed to do:
- Write zero to PTE
- Flush TLB
- Write new PTE
Quoted 4 lines
Quoted 3 lines
For example, the common CoW code path where you copy from a read only page to a new page, then map that new page as writable just works on RISC-V without extra considerations and on Arm it requires BBM.
CoW is a specific sub-case with increasing privilege.
Could you please explain why it matters increasing of privilege in terms of TLB flushing and PTE clearing before writing a new PTE?
Quoted 8 lines
Quoted 5 lines
It seems to me that BBM is mandated for Arm only because that TLB is shared among cores, so there is no any guarantee that no prior entry for same VA remains in TLB. In case of RISC-V's TLB isn't shared and after a flush it is guaranteed that no prior entry for the same VA remains in the TLB.
Not sure that's the sole reason. But again the question is: Is this written down explicitly anywhere? Generally there can be multiple levels of TLBs, and while some of them may be per-core, others may be shared.
Spec. mentions that:
If a hart employs an address-translation cache, that cache must appear to be private to that hart.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
xen/arch/riscv/p2m.c static void p2m_free_entry(struct p2m_domain *p2m, · line 812
Quoted 6 lines
xen/arch/riscv/p2m.c +5
-0+/* +* Even if we failed, we should install the newly allocated PTE +* entry. The caller will be in charge to free the sub-tree. +*/ +p2m_write_pte (entry,page_to_p2m_table (p2m, page), p2m->clean_pte);Why would it be wrong to free the page right here, vacating the entry at the same time (or leaving just that to the caller)? (IOW - if this is an implementation decision of yours, I think the word "should" would want dropping.) After all, the caller invoking
p2m_free_entry()on the thus split PTE is less efficient (needs to iterate over all entries) than on the original one (where it's just a single superpage).
Quoted 6 lines
Quoted 1 line
I think that I didn't get your idea.
Well, first and foremost this was a question to you, as it's not clear to me whether "should" is the correct word to use here. It would be appropriate if the spec mandated this behavior. It would seem less appropriate if this arrangement was an implementation choice of yours. And it looks to me as if the latter was the case here.
No, the spec doesn't mandate such behavior, it is just implementation specific. I can mention that in the comment.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
xen/arch/riscv/p2m.c static int __p2m_set_entry(struct p2m_domain *p2m, · line 878
Quoted 4 lines
xen/arch/riscv/p2m.c +0
-0@@ -806,7 +877,36 @@ static int __p2m_set_entry(struct p2m_domain *p2m,*/ if ( level > target )This condition is likely too strong, unless you actually mean to also split a superpage if it really wouldn't need splitting (new entry written still fitting with the superpage mapping, i.e. suitable MFN and same attributes).
Quoted 5 lines
Quoted 3 lines
I am not really sure that I fully understand.
My understanding is if level != target then the splitting is needed, I am not really get the part "split a superpage if it really wouldn't need splitting".Hmm, maybe I was wrong here. The caller determines at what level the actual change needs to occur? In which case what you have may be right.
Yes, the caller determines expected level and then asks __p2m_set_entry() to map on this level.
~ Oleksii
Re: [PATCH v2 15/17] xen/riscv: Implement superpage splitting for p2m mappings
Quoted 54 lines · Oleksii Kurochko
Quoted 24 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Add support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introducep2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
-__p2m_set_entry()updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table.
Quoted 6 lines
Quoted 5 lines
But some flushing is going to be necessary. As long as you only ever do global flushes, the one after the individual PTE modification (within the split table) will do (if BBM isn't required, see below), but once you move to more fine-grained flushing, that's not going to be enough anymore. Not sure it's a good idea to leave such a pitfall.
I think that I don't fully understand what is an issue.
Quoted 8 lines
Whether a flush is necessary after solely breaking up a superpage is arch- defined. I don't know how much restrictions the spec on possible hardware behavior for RISC-V. However, the eventual change of (at least) one entry of fulfill the original request will surely require a flush. What I was trying to say is that this required flush would better not also cover for the flushes that may or may not be required by the spec. IOW if the spec leaves any room for flushes to possibly be needed, those flushes would better be explicit.
I think that I still don't understand why what I wrote above will work as long as global flushes is working and will stop to work when more fine-grained flushing is used.
Inside
p2m_split_superpage()we don't need any kind of TLB flush operation as it is allocation a new page for a table and works with it, so no one could use this page at the moment and cache it in TLB.The only question is that if it is needed BBM before staring using splitted entry:
.... if ( !p2m_split_superpage(p2m, &split_pte, level, target, offsets) ) { /* Free the allocated sub-tree */ p2m_free_subtree(p2m, split_pte, level); rc = -ENOMEM; goto out; }
Quoted 5 lines
------> /* Should be BBM used here ? */
p2m_write_pte(entry, split_pte, p2m->clean_pte);And I can't find anything in the spec what tells me to use BBM here like Arm does:
But what you need is a statement in the spec that you can get away without. Imo at least.
Quoted 40 lines
/* * Follow the break-before-sequence to update the entry. * For more details see (D4.7.1 in ARM DDI 0487A.j). */ p2m_remove_pte(entry, p2m->clean_pte); p2m_force_tlb_flush_sync(p2m); p2m_write_pte(entry, split_pte, p2m->clean_pte);I agree that even BBM isn't mandated explicitly sometime it could be useful, but here I am not really sure what is the point to do TLB flush before
p2m_write_pte()as nothing serious will happen if and old mapping will be used for a some time as we are keeping the same rights for smaller (splited) mapping.
The reason why Arm doesp2m_remove_pte()&p2m_force_tlb_flush()before updating an entry here as it is doesn't guarantee that everything will be okay and they explicitly tells:
This situation can possibly break coherency and ordering guarantees, leading to inconsistent unwanted behavior in our Processing Element (PE).Quoted 20 lines
Quoted 20 lines
Quoted 9 lines
As to (no need for) BBM: I couldn't find anything to that effect in the privileged spec. Can you provide some pointer? What I found instead is e.g. this sentence: "To ensure that implicit reads observe writes to the same memory locations, an SFENCE.VMA instruction must be executed after the writes to flush the relevant cached translations." And this: "Accessing the same location using different cacheability attributes may cause loss of coherence." (This may not only occur when the same physical address is mapped twice at different VAs, but also after the shattering of a superpage when the new entry differs in cacheability.)
I also couldn't find that RISC-V spec mandates BBM explicitly as Arm does it.
What I meant that on RISC-V can do:
- Write new PTE
- Flush TLBWhile on Arm it is almost always needed to do:
- Write zero to PTE
- Flush TLB
- Write new PTE
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
For example, the common CoW code path where you copy from a read only page to a new page, then map that new page as writable just works on RISC-V without extra considerations and on Arm it requires BBM.
CoW is a specific sub-case with increasing privilege.
Could you please explain why it matters increasing of privilege in terms of TLB flushing and PTE clearing before writing a new PTE?
Some architectures automatically re-walk page tables when encountering a permission violation based on TLB contents. Hence increasing privilege can be a special case.
Quoted 11 lines
Quoted 7 lines
Quoted 4 lines
It seems to me that BBM is mandated for Arm only because that TLB is shared among cores, so there is no any guarantee that no prior entry for same VA remains in TLB. In case of RISC-V's TLB isn't shared and after a flush it is guaranteed that no prior entry for the same VA remains in the TLB.
Not sure that's the sole reason. But again the question is: Is this written down explicitly anywhere? Generally there can be multiple levels of TLBs, and while some of them may be per-core, others may be shared.
Spec. mentions that:
If a hart employs an address-translation cache, that cache must appear to be private to that hart.
Hmm, okay, that indeed pretty much excludes shared TLBs.
Jan
Re: [PATCH v2 15/17] xen/riscv: Implement superpage splitting for p2m mappings
Quoted 53 lines · Jan Beulich
Quoted 52 lines · Oleksii Kurochko
Quoted 24 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Add support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introducep2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
-__p2m_set_entry()updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table.
Quoted 6 lines
Quoted 5 lines
But some flushing is going to be necessary. As long as you only ever do global flushes, the one after the individual PTE modification (within the split table) will do (if BBM isn't required, see below), but once you move to more fine-grained flushing, that's not going to be enough anymore. Not sure it's a good idea to leave such a pitfall.
I think that I don't fully understand what is an issue.
Quoted 8 lines
Whether a flush is necessary after solely breaking up a superpage is arch- defined. I don't know how much restrictions the spec on possible hardware behavior for RISC-V. However, the eventual change of (at least) one entry of fulfill the original request will surely require a flush. What I was trying to say is that this required flush would better not also cover for the flushes that may or may not be required by the spec. IOW if the spec leaves any room for flushes to possibly be needed, those flushes would better be explicit.
I think that I still don't understand why what I wrote above will work as long as global flushes is working and will stop to work when more fine-grained flushing is used.
Inside
p2m_split_superpage()we don't need any kind of TLB flush operation as it is allocation a new page for a table and works with it, so no one could use this page at the moment and cache it in TLB.The only question is that if it is needed BBM before staring using splitted entry:
.... if ( !p2m_split_superpage(p2m, &split_pte, level, target, offsets) ) { /* Free the allocated sub-tree */ p2m_free_subtree(p2m, split_pte, level); rc = -ENOMEM; goto out; }
Quoted 7 lines
Quoted 5 lines
------> /* Should be BBM used here ? */
p2m_write_pte(entry, split_pte, p2m->clean_pte);And I can't find anything in the spec what tells me to use BBM here like Arm does:
But what you need is a statement in the spec that you can get away without. Imo at least.
In the spec. it is mentioned that:
It is permitted for multiple address-translation cache entries to co-exist for the same address. This represents the fact that in a conventional TLB hierarchy, it is possible for multiple entries to match a single address if, for example, a page is upgraded to a superpage without first clearing the original non-leaf PTE’s valid bit and executing an SFENCE.VMA with rs1=x0, or if multiple TLBs exist in parallel at a given level of the hierarchy. In this case, just as if an SFENCE.VMA is not executed between a write to the memory-management tables and subsequent implicit read of the same address: it is unpredictable whether the old non-leaf PTE or the new leaf PTE is used, but the behavior is otherwise well defined.
The phrase*"but the behavior is otherwise well defined"* emphasizes that even if the TLB sees two versions (the old and the new), the architecture guarantees stability, and the behavior remains safe — though unpredictable in terms of which translation will be used. And I think that this unpredictability is okay, at least, in the case if superpage splitting and therefore TLB flushing can be deferred since the old pages (which are used for old mapping) still exist and the permissions of the new entries match those of the original ones. Also, it seems like there clearing PTE before TLB flushing isn't need too.
Does it make sense?
Re: [PATCH v2 15/17] xen/riscv: Implement superpage splitting for p2m mappings
Quoted 81 lines · Oleksii Kurochko
Quoted 53 lines · Jan Beulich
Quoted 52 lines · Oleksii Kurochko
Quoted 24 lines · Jan Beulich
Quoted 17 lines · Oleksii Kurochko
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Oleksii Kurochko
Add support for down large memory mappings ("superpages") in the RISC-V p2m mapping so that smaller, more precise mappings ("finer-grained entries") can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introducep2m_split_superpage(): Recursively shatters a superpage into smaller page table entries down to the target level, preserving original permissions and attributes.
-__p2m_set_entry()updated to invoke superpage splitting when inserting entries at lower levels within a superpage-mapped region.This implementation is based on the ARM code, with modifications to the part that follows the BBM (break-before-make) approach. Unlike ARM, RISC-V does not require BBM, so there is no need to invalidate the PTE and flush the TLB before updating it with the newly created, split page table.
Quoted 6 lines
Quoted 5 lines
But some flushing is going to be necessary. As long as you only ever do global flushes, the one after the individual PTE modification (within the split table) will do (if BBM isn't required, see below), but once you move to more fine-grained flushing, that's not going to be enough anymore. Not sure it's a good idea to leave such a pitfall.
I think that I don't fully understand what is an issue.
Quoted 8 lines
Whether a flush is necessary after solely breaking up a superpage is arch- defined. I don't know how much restrictions the spec on possible hardware behavior for RISC-V. However, the eventual change of (at least) one entry of fulfill the original request will surely require a flush. What I was trying to say is that this required flush would better not also cover for the flushes that may or may not be required by the spec. IOW if the spec leaves any room for flushes to possibly be needed, those flushes would better be explicit.
I think that I still don't understand why what I wrote above will work as long as global flushes is working and will stop to work when more fine-grained flushing is used.
Inside
p2m_split_superpage()we don't need any kind of TLB flush operation as it is allocation a new page for a table and works with it, so no one could use this page at the moment and cache it in TLB.The only question is that if it is needed BBM before staring using splitted entry:
.... if ( !p2m_split_superpage(p2m, &split_pte, level, target, offsets) ) { /* Free the allocated sub-tree */ p2m_free_subtree(p2m, split_pte, level); rc = -ENOMEM; goto out; }Quoted 7 lines
Quoted 5 lines
------> /* Should be BBM used here ? */
p2m_write_pte(entry, split_pte, p2m->clean_pte);And I can't find anything in the spec what tells me to use BBM here like Arm does:
But what you need is a statement in the spec that you can get away without. Imo at least.
In the spec. it is mentioned that:
It is permitted for multiple address-translation cache entries to co-exist for the same address. This represents the fact that in a conventional TLB hierarchy, it is possible for multiple entries to match a single address if, for example, a page is upgraded to a superpage without first clearing the original non-leaf PTE’s valid bit and executing an SFENCE.VMA with rs1=x0, or if multiple TLBs exist in parallel at a given level of the hierarchy. In this case, just as if an SFENCE.VMA is not executed between a write to the memory-management tables and subsequent implicit read of the same address: it is unpredictable whether the old non-leaf PTE or the new leaf PTE is used, but the behavior is otherwise well defined.
The phrase*"but the behavior is otherwise well defined"* emphasizes that even if the TLB sees two versions (the old and the new), the architecture guarantees stability, and the behavior remains safe — though unpredictable in terms of which translation will be used. And I think that this unpredictability is okay, at least, in the case if superpage splitting and therefore TLB flushing can be deferred since the old pages (which are used for old mapping) still exist and the permissions of the new entries match those of the original ones. Also, it seems like there clearing PTE before TLB flushing isn't need too.
Quoted 1 line
Does it make sense?
Yes, I think this indeed is sufficient as a spec requirement.
Jan
PATCH v2 16/17 xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
In reply to Oleksii KurochkoImplement the mfn_valid() macro to verify whether a given MFN is valid by checking that it falls within the range [start_page, max_page). These bounds are initialized based on the start and end addresses of RAM.
As part of this patch, start_page is introduced and initialized with the PFN of the first RAM page.
Also, after providing a non-stub implementation of the mfn_valid() macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `__next_node':
/build/xen/./include/xen/nodemask.h:202: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `get_free_buddy':
/build/xen/common/page_alloc.c:881: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1043: undefined reference to `page_get_owner_and_reference'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1098: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `ns16550_interrupt':
/build/xen/drivers/char/ns16550.c:205: undefined reference to `get_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_get_owner_and_reference' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:35: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
- page_get_owner_and_reference() and its variant to safely acquire a
reference to a page and retrieve its owner.
- put_page() and put_page_nr() to release page references and free the page
when the count drops to zero.
For put_page_nr(), code related to static memory configuration is wrapped
with CONFIG_STATIC_MEMORY, as this configuration has not yet been moved to
common code. Therefore, PGC_static and free_domstatic_page() are not
introduced for RISC-V. However, since this configuration could be useful
in the future, the relevant code is retained and conditionally compiled.
- A stub for page_is_ram_type() that currently always returns 0 and asserts unreachable, as RAM type checking is not yet implemented.
---
Changes in V2:
- New patch.
--- xen/arch/riscv/include/asm/mm.h | 9 ++- xen/arch/riscv/mm.c | 97 +++++++++++++++++++++++++++++++-- 2 files changed, 99 insertions(+), 7 deletions(-)
xen/arch/riscv/include/asm/mm.h +7 -2
@@ -5,6 +5,7 @@#include <public/xen.h> #include <xen/bug.h> +#include <xen/compiler.h> #include <xen/const.h> #include <xen/mm-frame.h> #include <xen/pdx.h>
@@ -288,8 +289,12 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)#define page_get_owner (p) (p)->v.inuse.domain#define page_set_owner (p, d) ((p)->v.inuse.domain = (d)) -/* TODO: implement */ -#define mfn_valid (mfn) ({ (void)(mfn); 0; }) +extern unsigned long start_page; + +#define mfn_valid (mfn) ({ \ + unsigned long mfn__ = mfn_x(mfn); \ + likely((mfn__ >= start_page) && (mfn__ < max_page)); \ +})#define domain_set_alloc_bitsize (d) ((void)(d))#define domain_clamp_alloc_bitsize (d, b) ((void)(d), (b))
xen/arch/riscv/mm.c +92 -5
118 lines not on this page · raw message
Signature · 1 line
2.49.0
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 26 lines · Oleksii Kurochko
Implement the
mfn_valid()macro to verify whether a given MFN is valid by checking that it falls within the range [start_page,max_page). These bounds are initialized based on the start and end addresses of RAM.As part of this patch,
start_pageis introduced and initialized with the PFN of the first RAM page.Also, after providing a non-stub implementation of the
mfn_valid()macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `__next_node':
/build/xen/./include/xen/nodemask.h:202: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `get_free_buddy':
/build/xen/common/page_alloc.c:881: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1043: undefined reference to `page_get_owner_and_reference'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1098: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `ns16550_interrupt':
/build/xen/drivers/char/ns16550.c:205: undefined reference to `get_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_get_owner_and_reference' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:35: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
-page_get_owner_and_reference()and its variant to safely acquire a
reference to a page and retrieve its owner.
-put_page()andput_page_nr()to release page references and free the page
Quoted 8 lines
when the count drops to zero.
Forput_page_nr(), code related to static memory configuration is wrapped withCONFIG_STATIC_MEMORY, as this configuration has not yet been moved to common code. Therefore,PGC_staticandfree_domstatic_page()are not introduced for RISC-V. However, since this configuration could be useful in the future, the relevant code is retained and conditionally compiled.
- A stub forpage_is_ram_type()that currently always returns 0 and asserts unreachable, as RAM type checking is not yet implemented.
How does this end up working when common code references the function?
xen/arch/riscv/include/asm/mm.h static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) · line 297
Quoted 12 lines
xen/arch/riscv/include/asm/mm.h +6
-2@@ -288,8 +289,12 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)#define page_get_owner (p) (p)->v.inuse.domain#define page_set_owner (p, d) ((p)->v.inuse.domain = (d)) -/* TODO: implement */ -#define mfn_valid (mfn) ({ (void)(mfn); 0; }) +extern unsigned long start_page; + +#define mfn_valid (mfn) ({ \ + unsigned long mfn__ = mfn_x(mfn); \ + likely((mfn__ >= start_page) && (mfn__ < max_page)); \ +})
I don't think you should try to be clever and avoid using __mfn_valid() here, at least not without an easily identifiable TODO. Surely you've seen that both Arm and x86 use it.
Also, according to all I know, likely() doesn't work very well when used like this, except for architectures supporting conditionally executed insns (like Arm32 or IA-64, i.e. beyond conditional branches). I.e. if you want to use likely() here, I think you need two of them.
xen/arch/riscv/mm.c static void __init setup_directmap_mappings(unsigned long base_mfn, · line 523
Quoted 5 lines
xen/arch/riscv/mm.c +1
-0@@ -525,6 +520,8 @@ static void __init setup_directmap_mappings(unsigned long base_mfn,#error setup_{directmap,frametable}_mapping() should be implemented for RV_32#endif +unsigned long __read_mostly start_page;
Memory hotplug question again: __read_mostly or __ro_after_init?
Quoted 14 lines
xen/arch/riscv/mm.c +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr)
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + { +ASSERT_UNREACHABLE (); +return NULL ; + }
I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee, ASSERT_UNREACHABLE() is wrong to use. All I can see is process_shm_node() incrementing shmem_extra[].nr_shm_borrowers, without any limit check.
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 6 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */
May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 24 lines
xen/arch/riscv/mm.c +24
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +} + +struct domain *page_get_owner_and_reference (struct page_info *page) +{ +return page_get_owner_and_nr_reference (page,1 ); +} + +void put_page_nr (struct page_info *page,unsigned long nr) +{ +unsigned long nx, x, y = page->count_info; + + do { + ASSERT((y & PGC_count_mask) >= nr); + x = y; + nx = x - nr;
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 675
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+ } +while (unlikely ((y =cmpxchg (&page->count_info, x, nx)) != x) ); + +if (unlikely ((nx & PGC_count_mask) ==0 ) ) + { +#ifdef CONFIG_STATIC_MEMORY +if (unlikely (nx & PGC_static) ) +free_domstatic_page (page); + else +#endif
Such #ifdef-ed-out code is liable to go stale. Minimally use IS_ENABLED(). Even better would imo be if you introduced a "stub" PGC_static, resolving to 0 (i.e. for now unconditionally).
Jan
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 43 lines · Jan Beulich
Quoted 14 lines · Oleksii Kurochko
xen/arch/riscv/mm.c +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr)xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + { +ASSERT_UNREACHABLE (); +return NULL ; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 6 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 10 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 651
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +}
There also looks to be a dead code concern here (towards the "nr" parameters here and elsewhere, when STATIC_SHM=n). Just that apparently we decided to leave out Misra rule 2.2 entirely.
Jan
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 42 lines · Jan Beulich
Quoted 41 lines · Jan Beulich
Quoted 14 lines · Oleksii Kurochko
xen/arch/riscv/mm.c +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr)xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + { +ASSERT_UNREACHABLE (); +return NULL ; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 6 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 13 lines
Quoted 10 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 651
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +}There also looks to be a dead code concern here (towards the "nr" parameters here and elsewhere, when STATIC_SHM=n). Just that apparently we decided to leave out Misra rule 2.2 entirely.
I think that I didn't get what is an issue when STATIC_SHM=n, functions is still going to be called through page_get_owner_and_reference(), at least, in page_alloc.c .
~ Oleksii
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 58 lines · Oleksii Kurochko
Quoted 42 lines · Jan Beulich
Quoted 41 lines · Jan Beulich
Quoted 14 lines · Oleksii Kurochko
xen/arch/riscv/mm.c +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr)xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + { +ASSERT_UNREACHABLE (); +return NULL ; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 6 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 13 lines
Quoted 10 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 651
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +}There also looks to be a dead code concern here (towards the "nr" parameters here and elsewhere, when STATIC_SHM=n). Just that apparently we decided to leave out Misra rule 2.2 entirely.
Quoted 2 lines
I think that I didn't get what is an issue when STATIC_SHM=n, functions is still going to be called through
page_get_owner_and_reference(), at least, inpage_alloc.c.
Yes, but will "nr" ever be anything other than 1 then? IOW omitting the parameter would be fine. And that's what "dead code" is about.
Jan
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 57 lines · Jan Beulich
Quoted 56 lines · Oleksii Kurochko
Quoted 55 lines · Jan Beulich
Quoted 41 lines · Jan Beulich
Quoted 14 lines · Oleksii Kurochko
xen/arch/riscv/mm.c +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr)xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + { +ASSERT_UNREACHABLE (); +return NULL ; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 6 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 10 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 651
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +}There also looks to be a dead code concern here (towards the "nr" parameters here and elsewhere, when STATIC_SHM=n). Just that apparently we decided to leave out Misra rule 2.2 entirely.
Quoted 4 lines
Quoted 2 lines
I think that I didn't get what is an issue when STATIC_SHM=n, functions is still going to be called through
page_get_owner_and_reference(), at least, inpage_alloc.c.Yes, but will "nr" ever be anything other than 1 then? IOW omitting the parameter would be fine. And that's what "dead code" is about.
Got it.
So we don't have any SAF-x tag to mark this function as safe. What is the best one solution for now if nr argument will be needed in the future for STATIC_SHM=y?
~ Oleksii
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 60 lines · Oleksii Kurochko
Quoted 59 lines · Jan Beulich
Quoted 58 lines · Oleksii Kurochko
Quoted 42 lines · Jan Beulich
Quoted 41 lines · Jan Beulich
Quoted 14 lines · Oleksii Kurochko
xen/arch/riscv/mm.c +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr)xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + { +ASSERT_UNREACHABLE (); +return NULL ; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 6 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 13 lines
Quoted 10 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 651
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +}There also looks to be a dead code concern here (towards the "nr" parameters here and elsewhere, when STATIC_SHM=n). Just that apparently we decided to leave out Misra rule 2.2 entirely.
I think that I didn't get what is an issue when STATIC_SHM=n, functions is still going to be called through
page_get_owner_and_reference(), at least, inpage_alloc.c.
Quoted 7 lines
Quoted 2 lines
Yes, but will "nr" ever be anything other than 1 then? IOW omitting the parameter would be fine. And that's what "dead code" is about.
Got it.
So we don't have any SAF-x tag to mark this function as safe. What is the best one solution for now if nr argument will be needed in the future for STATIC_SHM=y?
Add the parameter at that point. Just like was done for Arm.
Jan
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 61 lines · Jan Beulich
Quoted 60 lines · Oleksii Kurochko
Quoted 59 lines · Jan Beulich
Quoted 58 lines · Oleksii Kurochko
Quoted 42 lines · Jan Beulich
Quoted 41 lines · Jan Beulich
Quoted 14 lines · Oleksii Kurochko
(quoted diff) +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) { return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE); } + +int page_is_ram_type(unsigned long mfn, unsigned long mem_type) +{ + ASSERT_UNREACHABLE(); + + return 0; +} + +static struct domain *page_get_owner_and_nr_reference(struct page_info *page, + unsigned long nr)Review void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
(quoted diff) +10
-0+{ + unsigned long x, y = page->count_info; + struct domain *owner; + + /* Restrict nr to avoid "double" overflow */ + if ( nr >= PGC_count_mask ) + { + ASSERT_UNREACHABLE(); + return NULL; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.Review void __iomem *ioremap(paddr_t pa, size_t len) · line 640
Quoted 6 lines
(quoted diff) +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 13 lines
Quoted 10 lines
Review void __iomem *ioremap(paddr_t pa, size_t len) · line 650
Quoted 10 lines
(quoted diff) +10
-0+ if ( unlikely(((x + nr) & PGC_count_mask) <= nr) ) + return NULL; + } + while ( (y = cmpxchg(&page->count_info, x, x + nr)) != x ); + + owner = page_get_owner(page); + ASSERT(owner); + + return owner; +}There also looks to be a dead code concern here (towards the "nr" parameters here and elsewhere, when STATIC_SHM=n). Just that apparently we decided to leave out Misra rule 2.2 entirely.
I think that I didn't get what is an issue when STATIC_SHM=n, functions is still going to be called through
page_get_owner_and_reference(), at least, inpage_alloc.c.
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
Yes, but will "nr" ever be anything other than 1 then? IOW omitting the parameter would be fine. And that's what "dead code" is about.
Got it.
So we don't have any SAF-x tag to mark this function as safe. What is the best one solution for now if nr argument will be needed in the future for STATIC_SHM=y?
Add the parameter at that point. Just like was done for Arm.
Hmm, it seems like I am confusing something... Arm has the same defintion and declaration of page_get_owner_and_nr_reference().
~ Oleksii
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 65 lines · Oleksii Kurochko
Quoted 63 lines · Jan Beulich
Quoted 62 lines · Oleksii Kurochko
Quoted 61 lines · Jan Beulich
Quoted 56 lines · Oleksii Kurochko
Quoted 42 lines · Jan Beulich
Quoted 41 lines · Jan Beulich
Quoted 14 lines · Oleksii Kurochko
(quoted diff) +10
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) { return ioremap_attr(pa, len, PAGE_HYPERVISOR_NOCACHE); } + +int page_is_ram_type(unsigned long mfn, unsigned long mem_type) +{ + ASSERT_UNREACHABLE(); + + return 0; +} + +static struct domain *page_get_owner_and_nr_reference(struct page_info *page, + unsigned long nr)Review void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 10 lines
(quoted diff) +10
-0+{ + unsigned long x, y = page->count_info; + struct domain *owner; + + /* Restrict nr to avoid "double" overflow */ + if ( nr >= PGC_count_mask ) + { + ASSERT_UNREACHABLE(); + return NULL; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.Review void __iomem *ioremap(paddr_t pa, size_t len) · line 640
Quoted 6 lines
(quoted diff) +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 13 lines
Quoted 10 lines
Review void __iomem *ioremap(paddr_t pa, size_t len) · line 650
Quoted 10 lines
(quoted diff) +10
-0+ if ( unlikely(((x + nr) & PGC_count_mask) <= nr) ) + return NULL; + } + while ( (y = cmpxchg(&page->count_info, x, x + nr)) != x ); + + owner = page_get_owner(page); + ASSERT(owner); + + return owner; +}There also looks to be a dead code concern here (towards the "nr" parameters here and elsewhere, when STATIC_SHM=n). Just that apparently we decided to leave out Misra rule 2.2 entirely.
Quoted 2 lines
I think that I didn't get what is an issue when STATIC_SHM=n, functions is still going to be called through
page_get_owner_and_reference(), at least, inpage_alloc.c.Yes, but will "nr" ever be anything other than 1 then? IOW omitting the parameter would be fine. And that's what "dead code" is about.
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
Got it.
So we don't have any SAF-x tag to mark this function as safe. What is the best one solution for now if nr argument will be needed in the future for STATIC_SHM=y?
Add the parameter at that point. Just like was done for Arm.
Hmm, it seems like I am confusing something... Arm has the same defintion and declaration of
page_get_owner_and_nr_reference().
But it didn't always have it. And there is at least one pending issue there. Hence my request to use the simpler variant until someone actually makes STATIC_SHM work on RISC-V. And hopefully by then the issue in Arm code is sorted, and you can clone the code without raising questions.
Jan
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
In reply to Jan BeulichQuoted 90 lines · Jan Beulich
Quoted 26 lines · Oleksii Kurochko
Implement the
mfn_valid()macro to verify whether a given MFN is valid by checking that it falls within the range [start_page,max_page). These bounds are initialized based on the start and end addresses of RAM.As part of this patch,
start_pageis introduced and initialized with the PFN of the first RAM page.Also, after providing a non-stub implementation of the
mfn_valid()macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `__next_node':
/build/xen/./include/xen/nodemask.h:202: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `get_free_buddy':
/build/xen/common/page_alloc.c:881: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1043: undefined reference to `page_get_owner_and_reference'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1098: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `ns16550_interrupt':
/build/xen/drivers/char/ns16550.c:205: undefined reference to `get_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_get_owner_and_reference' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:35: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
-page_get_owner_and_reference()and its variant to safely acquire a
reference to a page and retrieve its owner.
-put_page()andput_page_nr()to release page references and free the pageQuoted 8 lines
when the count drops to zero.
Forput_page_nr(), code related to static memory configuration is wrapped withCONFIG_STATIC_MEMORY, as this configuration has not yet been moved to common code. Therefore,PGC_staticandfree_domstatic_page()are not introduced for RISC-V. However, since this configuration could be useful in the future, the relevant code is retained and conditionally compiled.
- A stub forpage_is_ram_type()that currently always returns 0 and asserts unreachable, as RAM type checking is not yet implemented.How does this end up working when common code references the function?
xen/arch/riscv/include/asm/mm.h static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) · line 297
Quoted 12 lines
xen/arch/riscv/include/asm/mm.h +6
-2@@ -288,8 +289,12 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)#define page_get_owner (p) (p)->v.inuse.domain#define page_set_owner (p, d) ((p)->v.inuse.domain = (d)) -/* TODO: implement */ -#define mfn_valid (mfn) ({ (void)(mfn); 0; }) +extern unsigned long start_page; + +#define mfn_valid (mfn) ({ \ + unsigned long mfn__ = mfn_x(mfn); \ + likely((mfn__ >= start_page) && (mfn__ < max_page)); \ +})I don't think you should try to be clever and avoid using
__mfn_valid()here, at least not without an easily identifiable TODO. Surely you've seen that both Arm and x86 use it.Also, according to all I know,
likely()doesn't work very well when used like this, except for architectures supporting conditionally executed insns (like Arm32 or IA-64, i.e. beyond conditional branches). I.e. if you want to uselikely()here, I think you need two of them.xen/arch/riscv/mm.c static void __init setup_directmap_mappings(unsigned long base_mfn, · line 523
Quoted 5 lines
xen/arch/riscv/mm.c +1
-0@@ -525,6 +520,8 @@ static void __init setup_directmap_mappings(unsigned long base_mfn,#error setup_{directmap,frametable}_mapping() should be implemented for RV_32#endif +unsigned long __read_mostly start_page;Memory hotplug question again:
__read_mostlyor__ro_after_init?Quoted 22 lines
xen/arch/riscv/mm.c +18
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr) +{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + { +ASSERT_UNREACHABLE ();
Quoted 8 lines
Review void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 2 lines
(quoted diff) +2
-0+ return NULL; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.
Honestly I don't know why this assert was placed here. I checked the code and we don't limit nr_shm_borrowers in any place, so in theory it's possible to end up here.
~Michal
Quoted 52 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 7 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 24 lines
xen/arch/riscv/mm.c +24
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +} + +struct domain *page_get_owner_and_reference (struct page_info *page) +{ +return page_get_owner_and_nr_reference (page,1 ); +} + +void put_page_nr (struct page_info *page,unsigned long nr) +{ +unsigned long nx, x, y = page->count_info; + + do { + ASSERT((y & PGC_count_mask) >= nr); + x = y; + nx = x - nr;xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 675
Quoted 10 lines
xen/arch/riscv/mm.c +10
-0+ } +while (unlikely ((y =cmpxchg (&page->count_info, x, nx)) != x) ); + +if (unlikely ((nx & PGC_count_mask) ==0 ) ) + { +#ifdef CONFIG_STATIC_MEMORY +if (unlikely (nx & PGC_static) ) +free_domstatic_page (page); + else +#endif Such #ifdef-ed-out code is liable to go stale. Minimally use
IS_ENABLED(). Even better would imo be if you introduced a "stub"PGC_static, resolving to 0 (i.e. for now unconditionally).Jan
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
In reply to Jan BeulichQuoted 35 lines · Jan Beulich
Quoted 26 lines · Oleksii Kurochko
Implement the
mfn_valid()macro to verify whether a given MFN is valid by checking that it falls within the range [start_page,max_page). These bounds are initialized based on the start and end addresses of RAM.As part of this patch,
start_pageis introduced and initialized with the PFN of the first RAM page.Also, after providing a non-stub implementation of the
mfn_valid()macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `__next_node':
/build/xen/./include/xen/nodemask.h:202: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `get_free_buddy':
/build/xen/common/page_alloc.c:881: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1043: undefined reference to `page_get_owner_and_reference'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1098: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `ns16550_interrupt':
/build/xen/drivers/char/ns16550.c:205: undefined reference to `get_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_get_owner_and_reference' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:35: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
-page_get_owner_and_reference()and its variant to safely acquire a
reference to a page and retrieve its owner.
-put_page()andput_page_nr()to release page references and free the pageQuoted 8 lines
when the count drops to zero.
Forput_page_nr(), code related to static memory configuration is wrapped withCONFIG_STATIC_MEMORY, as this configuration has not yet been moved to common code. Therefore,PGC_staticandfree_domstatic_page()are not introduced for RISC-V. However, since this configuration could be useful in the future, the relevant code is retained and conditionally compiled.
- A stub forpage_is_ram_type()that currently always returns 0 and asserts unreachable, as RAM type checking is not yet implemented.
Quoted 1 line
How does this end up working when common code references the function?
Based on the following commit message:
Callers are VT-d (so x86 specific) and various bits of page offlining support, which although it looks generic (and is in xen/common) does things like diving into page_info->count_info which is not generic.
In any case on this is only reachable via XEN_SYSCTL_page_offline_op, which clearly shouldn't be called on ARM just yet.
There is no callers for page_is_ram_type() for Arm now, and I expect something similar for RISC-V. As we don't even introduce hypercalls for RISC-V, we can just live without it.
Quoted 5 lines
Quoted 5 lines
xen/arch/riscv/include/asm/mm.h +0
-0@@ -288,8 +289,12 @@ static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr)#define page_get_owner (p) (p)->v.inuse.domain#define page_set_owner (p, d) ((p)->v.inuse.domain = (d))
Quoted 11 lines
xen/arch/riscv/include/asm/mm.h static inline bool arch_mfns_in_directmap(unsigned long mfn, unsigned long nr) · line 297
Quoted 8 lines
xen/arch/riscv/include/asm/mm.h +6
-2-/* TODO: implement */ -#define mfn_valid (mfn) ({ (void)(mfn); 0; }) +extern unsigned long start_page; + +#define mfn_valid (mfn) ({ \ + unsigned long mfn__ = mfn_x(mfn); \ + likely((mfn__ >= start_page) && (mfn__ < max_page)); \ +})I don't think you should try to be clever and avoid using
__mfn_valid()here, at least not without an easily identifiable TODO. Surely you've seen that both Arm and x86 use it.
Overlooked that pdx.c is compiled unconditionally, so I thought that __mfn_valid() common implementation isn't avaiable as, at the moment, RISC-V doesn't support PDX_COMPRESSION...
Quoted 4 lines
Also, according to all I know,
likely()doesn't work very well when used like this, except for architectures supporting conditionally executed insns (like Arm32 or IA-64, i.e. beyond conditional branches). I.e. if you want to uselikely()here, I think you need two of them.
... I will update mfn_valid() definition according to your recommendations.
Quoted 29 lines
xen/arch/riscv/mm.c static void __init setup_directmap_mappings(unsigned long base_mfn, · line 523
Quoted 6 lines
xen/arch/riscv/mm.c +1
-0@@ -525,6 +520,8 @@ static void __init setup_directmap_mappings(unsigned long base_mfn,#error setup_{directmap,frametable}_mapping() should be implemented for RV_32#endif +unsigned long __read_mostly start_page;Memory hotplug question again:
__read_mostlyor__ro_after_init?Quoted 21 lines
xen/arch/riscv/mm.c +17
-0@@ -613,3 +612,91 @@ void __iomem *ioremap(paddr_t pa, size_t len) {return ioremap_attr (pa, len,PAGE_HYPERVISOR_NOCACHE ); } + +int page_is_ram_type (unsigned long mfn,unsigned long mem_type) +{ +ASSERT_UNREACHABLE (); + +return 0 ; +} + +static struct domain *page_get_owner_and_nr_reference (struct page_info *page, +unsigned long nr) +{ +unsigned long x, y = page->count_info; +struct domain *owner; + +/* Restrict nr to avoid "double" overflow */ +if ( nr >= PGC_count_mask ) + {
Quoted 8 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 634
Quoted 3 lines
xen/arch/riscv/mm.c +3
-0+ASSERT_UNREACHABLE (); +return NULL ; + }I question the validity of this, already in the Arm original: I can't spot how the caller guarantees to stay below that limit. Without such an (attempted) guarantee,
ASSERT_UNREACHABLE()is wrong to use. All I can see isprocess_shm_node()incrementing shmem_extra[].nr_shm_borrowers, without any limit check.
Agree, it could be really dropped.
Quoted 40 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 641
Quoted 7 lines
xen/arch/riscv/mm.c +6
-0+ do { + x = y; + /* + * Count == 0: Page is not allocated, so we cannot take a reference. + * Count == -1: Reference count would wrap, which is invalid. + */May I once again ask that you look carefully at comments (as much as at code) you copy. Clearly this comment wasn't properly updated when the bumping by 1 was changed to bumping by nr.
Quoted 29 lines
xen/arch/riscv/mm.c +29
-0+if (unlikely (((x + nr) & PGC_count_mask) <= nr) ) +return NULL ; + } +while ( (y =cmpxchg (&page->count_info, x, x + nr)) != x ); + + owner =page_get_owner (page); +ASSERT (owner); + +return owner; +} + +struct domain *page_get_owner_and_reference (struct page_info *page) +{ +return page_get_owner_and_nr_reference (page,1 ); +} + +void put_page_nr (struct page_info *page,unsigned long nr) +{ +unsigned long nx, x, y = page->count_info; + +do { +ASSERT ((y & PGC_count_mask) >= nr); + x = y; + nx = x - nr; + } +while (unlikely ((y =cmpxchg (&page->count_info, x, nx)) != x) ); + +if (unlikely ((nx & PGC_count_mask) ==0 ) ) + {
Quoted 8 lines
xen/arch/riscv/mm.c void __iomem *ioremap(paddr_t pa, size_t len) · line 675
Quoted 5 lines
xen/arch/riscv/mm.c +5
-0+#ifdef CONFIG_STATIC_MEMORY +if (unlikely (nx & PGC_static) ) +free_domstatic_page (page); + else +#endif Such #ifdef-ed-out code is liable to go stale. Minimally use
IS_ENABLED(). Even better would imo be if you introduced a "stub"PGC_static, resolving to 0 (i.e. for now unconditionally).
An introduction of PGC_static would be better.
Thanks.
~ Oleksii
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 38 lines · Oleksii Kurochko
Quoted 35 lines · Jan Beulich
Quoted 26 lines · Oleksii Kurochko
Implement the
mfn_valid()macro to verify whether a given MFN is valid by checking that it falls within the range [start_page,max_page). These bounds are initialized based on the start and end addresses of RAM.As part of this patch,
start_pageis introduced and initialized with the PFN of the first RAM page.Also, after providing a non-stub implementation of the
mfn_valid()macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `__next_node':
/build/xen/./include/xen/nodemask.h:202: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `get_free_buddy':
/build/xen/common/page_alloc.c:881: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1043: undefined reference to `page_get_owner_and_reference'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1098: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `ns16550_interrupt':
/build/xen/drivers/char/ns16550.c:205: undefined reference to `get_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_get_owner_and_reference' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:35: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
-page_get_owner_and_reference()and its variant to safely acquire a
reference to a page and retrieve its owner.
-put_page()andput_page_nr()to release page references and free the pageQuoted 8 lines
when the count drops to zero.
Forput_page_nr(), code related to static memory configuration is wrapped withCONFIG_STATIC_MEMORY, as this configuration has not yet been moved to common code. Therefore,PGC_staticandfree_domstatic_page()are not introduced for RISC-V. However, since this configuration could be useful in the future, the relevant code is retained and conditionally compiled.
- A stub forpage_is_ram_type()that currently always returns 0 and asserts unreachable, as RAM type checking is not yet implemented.Quoted 1 line
How does this end up working when common code references the function?
Quoted 7 lines
Based on the following commit message:
Callers are VT-d (so x86 specific) and various bits of page offlining support, which although it looks generic (and is in xen/common) does things like diving intopage_info->count_infowhich is not generic.In any case on this is only reachable via
XEN_SYSCTL_page_offline_op, which clearly shouldn't be called on ARM just yet.
What commit message are you talking about? Nothing like the above is anywhere in this patch.
Quoted 3 lines
There is no callers for
page_is_ram_type()for Arm now, and I expect something similar for RISC-V. As we don't even introduce hypercalls for RISC-V, we can just live without it.
If there's no caller, why the stub?
Jan
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
Quoted 43 lines · Jan Beulich
Quoted 42 lines · Oleksii Kurochko
Quoted 35 lines · Jan Beulich
Quoted 26 lines · Oleksii Kurochko
Implement the
mfn_valid()macro to verify whether a given MFN is valid by checking that it falls within the range [start_page,max_page). These bounds are initialized based on the start and end addresses of RAM.As part of this patch,
start_pageis introduced and initialized with the PFN of the first RAM page.Also, after providing a non-stub implementation of the
mfn_valid()macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `__next_node':
/build/xen/./include/xen/nodemask.h:202: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `get_free_buddy':
/build/xen/common/page_alloc.c:881: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1043: undefined reference to `page_get_owner_and_reference'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1098: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `ns16550_interrupt':
/build/xen/drivers/char/ns16550.c:205: undefined reference to `get_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_get_owner_and_reference' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:35: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
-page_get_owner_and_reference()and its variant to safely acquire a
reference to a page and retrieve its owner.
-put_page()andput_page_nr()to release page references and free the pageQuoted 8 lines
when the count drops to zero.
Forput_page_nr(), code related to static memory configuration is wrapped withCONFIG_STATIC_MEMORY, as this configuration has not yet been moved to common code. Therefore,PGC_staticandfree_domstatic_page()are not introduced for RISC-V. However, since this configuration could be useful in the future, the relevant code is retained and conditionally compiled.
- A stub forpage_is_ram_type()that currently always returns 0 and asserts unreachable, as RAM type checking is not yet implemented.Quoted 1 line
How does this end up working when common code references the function?
Based on the following commit message:
Callers are VT-d (so x86 specific) and various bits of page offlining support, which although it looks generic (and is in xen/common) does things like diving intopage_info->count_infowhich is not generic.
Quoted 4 lines
Quoted 2 lines
In any case on this is only reachable via
XEN_SYSCTL_page_offline_op, which clearly shouldn't be called on ARM just yet.What commit message are you talking about? Nothing like the above is anywhere in this patch.
It's pretty old commit:
commit 214c4cd94a80bcaf042f25158eaa7d0e5bbc3b5b
Author: Ian Campbell<[email protected]>
Date: Wed Dec 19 14:16:23 2012 +0000
But since that time page_is_ram_type() hasn't been changed for Arm.
Quoted 5 lines
Quoted 4 lines
There is no callers for
page_is_ram_type()for Arm now, and I expect something similar for RISC-V. As we don't even introduce hypercalls for RISC-V, we can just live without it.If there's no caller, why the stub?
Because that parts of common code which are using it aren't under ifdef, for example, this one function:
int offline_page(mfn_t mfn, int broken, uint32_t *status)
And specifically this function is called when XEN_SYSCTL_page_offline_op is handled.
But I agree that it seems like nothing prevents to call XEN_SYSCTL_page_offline_op.
~ Oleksii
Re: [PATCH v2 16/17] xen/riscv: implement mfn_valid() and page reference, ownership handling helpers
In reply to Oleksii KurochkoQuoted 38 lines · Oleksii Kurochko
Quoted 35 lines · Jan Beulich
Quoted 26 lines · Oleksii Kurochko
Implement the
mfn_valid()macro to verify whether a given MFN is valid by checking that it falls within the range [start_page,max_page). These bounds are initialized based on the start and end addresses of RAM.As part of this patch,
start_pageis introduced and initialized with the PFN of the first RAM page.Also, after providing a non-stub implementation of the
mfn_valid()macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `__next_node':
/build/xen/./include/xen/nodemask.h:202: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `get_free_buddy':
/build/xen/common/page_alloc.c:881: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1043: undefined reference to `page_get_owner_and_reference'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1098: undefined reference to `page_is_ram_type'
riscv64-linux-gnu-ld: prelink.o: in function `ns16550_interrupt':
/build/xen/drivers/char/ns16550.c:205: undefined reference to `get_page'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_get_owner_and_reference' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:35: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
-page_get_owner_and_reference()and its variant to safely acquire a
reference to a page and retrieve its owner.
-put_page()andput_page_nr()to release page references and free the pageQuoted 8 lines
when the count drops to zero.
Forput_page_nr(), code related to static memory configuration is wrapped withCONFIG_STATIC_MEMORY, as this configuration has not yet been moved to common code. Therefore,PGC_staticandfree_domstatic_page()are not introduced for RISC-V. However, since this configuration could be useful in the future, the relevant code is retained and conditionally compiled.
- A stub forpage_is_ram_type()that currently always returns 0 and asserts unreachable, as RAM type checking is not yet implemented.Quoted 1 line
How does this end up working when common code references the function?
Quoted 7 lines
Based on the following commit message:
Callers are VT-d (so x86 specific) and various bits of page offlining support, which although it looks generic (and is in xen/common) does things like diving intopage_info->count_infowhich is not generic.In any case on this is only reachable via
XEN_SYSCTL_page_offline_op, which clearly shouldn't be called on ARM just yet.
Assuming this is from an old commit, then I have to question this justification. I see nothing preventing XEN_SYSCTL_page_offline_op to be invoked on an Arm system. Hence (unless I'm overlooking somthing) ASSERT_UNREACHABLE() is simply inappropriate (and wants fixing). Luckily it being sysctl-s only, there's no need for an XSA. In no case should known flawed code be copied into another port.
Jan
PATCH v2 17/17 xen/riscv: add support of page lookup by GFN
In reply to Oleksii KurochkoIntroduce helper functions for safely querying the P2M (physical-to-machine) mapping:
- add p2m_read_lock(), p2m_read_unlock(), and p2m_is_locked() for managing P2M lock state.
- Implement p2m_get_entry() to retrieve mapping details for a given GFN, including MFN, page order, and validity.
- Add p2m_lookup() to encapsulate read-locked MFN retrieval.
- Introduce p2m_get_page_from_gfn() to convert a GFN into a page_info pointer, acquiring a reference to the page if valid.
Implementations are based on Arm's functions with some minor modifications:
- p2m_get_entry():
- Reverse traversal of page tables, as RISC-V uses the opposite order compared to Arm.
- Removed the return of p2m_access_t from p2m_get_entry() since mem_access_settings is not introduced for RISC-V.
- Updated BUILD_BUG_ON() to check using the level 0 mask, which corresponds to Arm's THIRD_MASK.
- Replaced open-coded bit shifts with the BIT() macro.
- Other minor changes, such as using RISC-V-specific functions to validate P2M PTEs, and replacing Arm-specific GUEST_* macros with their RISC-V equivalents.
- p2m_get_page_from_gfn():
- Removed p2m_is_foreign() and related logic, as this functionality is not implemented for RISC-V.
---
Changes in V2:
- New patch.
--- xen/arch/riscv/include/asm/p2m.h | 18 +++++ xen/arch/riscv/p2m.c | 131 +++++++++++++++++++++++++++++++ 2 files changed, 149 insertions(+)
xen/arch/riscv/include/asm/p2m.h +18 -0
@@ -184,6 +184,24 @@ static inline int p2m_is_write_locked(struct p2m_domain *p2m) return rw_is_write_locked(&p2m->lock); } +static inline void p2m_read_lock (struct p2m_domain *p2m) +{ +read_lock (&p2m->lock); +} + +static inline void p2m_read_unlock (struct p2m_domain *p2m) +{ +read_unlock (&p2m->lock); +} + +static inline int p2m_is_locked (struct p2m_domain *p2m) +{ +return rw_is_locked (&p2m->lock); +} + +struct page_info *p2m_get_page_from_gfn (struct domain *d,gfn_t gfn, +p2m_type_t *t); +#endif /* ASM__RISCV__P2M_H */ /*
xen/arch/riscv/p2m.c +131 -0
134 lines not on this page · raw message
Signature · 1 line
2.49.0
Re: [PATCH v2 17/17] xen/riscv: add support of page lookup by GFN
Quoted 10 lines · Oleksii Kurochko
Introduce helper functions for safely querying the P2M (physical-to-machine) mapping:
- addp2m_read_lock(),p2m_read_unlock(), andp2m_is_locked()for managing P2M lock state.
- Implementp2m_get_entry()to retrieve mapping details for a given GFN, including MFN, page order, and validity.
- Addp2m_lookup()to encapsulate read-locked MFN retrieval.
- Introducep2m_get_page_from_gfn()to convert a GFN into apage_infopointer, acquiring a reference to the page if valid.
Quoted 6 lines
Implementations are based on Arm's functions with some minor modifications:
-p2m_get_entry():
- Reverse traversal of page tables, as RISC-V uses the opposite order compared to Arm.
- Removed the return ofp2m_access_tfromp2m_get_entry()sincemem_access_settingsis not introduced for RISC-V.
Didn't I see uses of p2m_access in earlier patches? If you don't mean to have that, then please consistently {every,no}where.
Quoted 9 lines
- Updated
BUILD_BUG_ON()to check using the level 0 mask, which corresponds to Arm'sTHIRD_MASK.
- Replaced open-coded bit shifts with theBIT()macro.
- Other minor changes, such as using RISC-V-specific functions to validate P2M PTEs, and replacing Arm-specific GUEST_* macros with their RISC-V equivalents.
-p2m_get_page_from_gfn():
- Removedp2m_is_foreign()and related logic, as this functionality is not implemented for RISC-V.
Yet I expect you'll need this, sooner or later.
Quoted 13 lines
xen/arch/riscv/include/asm/p2m.h +7
-0@@ -184,6 +184,24 @@ static inline int p2m_is_write_locked(struct p2m_domain *p2m)return rw_is_write_locked (&p2m->lock); } +static inline void p2m_read_lock (struct p2m_domain *p2m) +{ +read_lock (&p2m->lock); +} + +static inline void p2m_read_unlock (struct p2m_domain *p2m) +{
xen/arch/riscv/include/asm/p2m.h static inline int p2m_is_write_locked(struct p2m_domain *p2m) · line 203
Quoted 10 lines
xen/arch/riscv/include/asm/p2m.h +10
-0+read_unlock (&p2m->lock); +} + +static inline int p2m_is_locked (struct p2m_domain *p2m) +{ +return rw_is_locked (&p2m->lock); +} + +struct page_info *p2m_get_page_from_gfn (struct domain *d,gfn_t gfn, +p2m_type_t *t);
Once again I don't think you can pass struct domain * here, when in the long run a domain can have multiple P2Ms.
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1063
Quoted 12 lines
xen/arch/riscv/p2m.c +6
-0@@ -1055,3 +1055,134 @@ int guest_physmap_add_entry(struct domain *d, {return p2m_insert_mapping (d, gfn, (1 << page_order), mfn, t); } + +/* +* Get the details of a given gfn. +* +* If the entry is present, the associated MFN will be returned and the +* access and type filled up. The page_order will correspond to the
You removed p2m_access_t * from the parameters; you need to also update the comment then accordingly.
Quoted 9 lines
xen/arch/riscv/p2m.c +9
-0+* order of the mapping in the page table (i.e it could be a superpage). +* +* If the entry is not present, INVALID_MFN will be returned and the +* page_order will be set according to the order of the invalid range. +* +* valid will contain the value of bit[0] (e.g valid bit) of the +* entry. +*/ +static mfn_t p2m_get_entry(struct p2m_domain *p2m, gfn_t gfn,
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1082
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+p2m_type_t *t, + unsigned int *page_order, + bool *valid) +{ + paddr_t addr = gfn_to_gaddr(gfn); +unsigned int level =0 ; +pte_t entry, *table; +int rc; +mfn_t mfn =INVALID_MFN ; +p2m_type_t _t;
Please no local variables with leading underscores. In x86 we commonly name such variables p2mt.
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1083
Quoted 1 line
xen/arch/riscv/p2m.c +1
-0+DECLARE_OFFSETS (offsets, addr);
This is the sole use of "addr". Is such a local variable really worth having?
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+ASSERT (p2m_is_locked (p2m)); +BUILD_BUG_ON (XEN_PT_LEVEL_MAP_MASK (0 ) !=PAGE_MASK ); + +/* Allow t to be NULL */ + t = t ?: &_t; + + *t = p2m_invalid;
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1101
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ +if ( valid ) + *valid = false; + +/* XXX: Check if the mapping is lower than the mapped gfn */ + +/* This gfn is higher than the highest the p2m map currently holds */ +if (gfn_x (gfn) >gfn_x (p2m->max_mapped_gfn) ) + { +for ( level =P2M_ROOT_LEVEL ; level ; level-- )
Nit: Stray blank before the 2nd semicolon. (Again at least once below.)
Quoted 14 lines
xen/arch/riscv/p2m.c +14
-0+if ( (gfn_x (gfn) & (XEN_PT_LEVEL_MASK (level) >>PAGE_SHIFT )) > +gfn_x (p2m->max_mapped_gfn) ) +break ; + + goto out; + } + + table =p2m_get_root_pointer (p2m, gfn); + +/* +* the table should always be non-NULL because the gfn is below +* p2m->max_mapped_gfn and the root table pages are always present. +*/ +if ( !table )
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1125
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ { +ASSERT_UNREACHABLE (); + level =P2M_ROOT_LEVEL ; + goto out; + } + +for ( level =P2M_ROOT_LEVEL ; level ; level-- ) + { + rc =p2m_next_level (p2m, true, level, &table, offsets[level]); +if ( (rc ==GUEST_TABLE_MAP_NONE ) && (rc !=GUEST_TABLE_MAP_NOMEM ) )
This condition looks odd. As written the rhs of the && is redundant.
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1127
Quoted 2 lines
xen/arch/riscv/p2m.c +2
-0+ goto out_unmap; + else if ( rc != GUEST_TABLE_NORMAL )
As before, no real need for "else" in such cases.
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1135
Quoted 8 lines
xen/arch/riscv/p2m.c +8
-0+break ; + } + + entry = table[offsets[level]]; + +if (p2me_is_valid (p2m, entry) ) + { + *t =p2m_type_radix_get (p2m, entry);
If the incoming argument is NULL, the somewhat expensive radix tree lookup is unnecessary here.
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1146
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ mfn =pte_get_mfn (entry); +/* +* The entry may point to a superpage. Find the MFN associated +* to the GFN. +*/ + mfn =mfn_add (mfn, +gfn_x (gfn) & (BIT (XEN_PT_LEVEL_ORDER (level),UL ) -1 )); + +if ( valid ) + *valid =pte_is_valid (entry);
Interesting. Why not the P2M counterpart of the function? Yes, the comment ahead of the function says so, but I don't see why the valid bit suddenly is relevant here (besides the P2M type).
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1152
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+ } + +out_unmap: +unmap_domain_page (table); + +out:
Nit: Style (bot labels).
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1159
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+if ( page_order ) + *page_order =XEN_PT_LEVEL_ORDER (level); + +return mfn; +} + +static mfn_t p2m_lookup (struct domain *d,gfn_t gfn,p2m_type_t *t)
pointer-to-const for the 1st arg? But again more likely struct p2m_domain * anyway?
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1171
Quoted 12 lines
xen/arch/riscv/p2m.c +12
-0+{ +mfn_t mfn; +struct p2m_domain *p2m =p2m_get_hostp2m (d); + +p2m_read_lock (p2m); + mfn =p2m_get_entry (p2m, gfn, t,NULL ,NULL ); +p2m_read_unlock (p2m); + +return mfn; +} + +struct page_info *p2m_get_page_from_gfn(struct domain *d, gfn_t gfn,
Same here - likely you mean struct p2m_domain * instead.
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1174
Quoted 3 lines
xen/arch/riscv/p2m.c +3
-0+ p2m_type_t *t) +{ +p2m_type_t p2mt = {0 };
Why a compound initializer for something that isn't a compound object? And why plain 0 for something that is an enumerated type?
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1180
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+struct page_info *page; + +mfn_t mfn =p2m_lookup (d, gfn, &p2mt); + +if ( t ) + *t = p2mt;
What's wrong with passing t directly to p2m_lookup()?
Jan
Re: [PATCH v2 17/17] xen/riscv: add support of page lookup by GFN
Quoted 11 lines · Jan Beulich
Quoted 10 lines · Oleksii Kurochko
Introduce helper functions for safely querying the P2M (physical-to-machine) mapping:
- addp2m_read_lock(),p2m_read_unlock(), andp2m_is_locked()for managing P2M lock state.
- Implementp2m_get_entry()to retrieve mapping details for a given GFN, including MFN, page order, and validity.
- Addp2m_lookup()to encapsulate read-locked MFN retrieval.
- Introducep2m_get_page_from_gfn()to convert a GFN into apage_infopointer, acquiring a reference to the page if valid.
Quoted 8 lines
Quoted 6 lines
Implementations are based on Arm's functions with some minor modifications:
-p2m_get_entry():
- Reverse traversal of page tables, as RISC-V uses the opposite order compared to Arm.
- Removed the return ofp2m_access_tfromp2m_get_entry()sincemem_access_settingsis not introduced for RISC-V.Didn't I see uses of
p2m_accessin earlier patches? If you don't mean to have that, then please consistently {every,no}where.
Yes, it was used. I think it would be better just usage of p2m_access from earlier patches.
Quoted 10 lines
Quoted 9 lines
- Updated
BUILD_BUG_ON()to check using the level 0 mask, which corresponds to Arm'sTHIRD_MASK.
- Replaced open-coded bit shifts with theBIT()macro.
- Other minor changes, such as using RISC-V-specific functions to validate P2M PTEs, and replacing Arm-specific GUEST_* macros with their RISC-V equivalents.
-p2m_get_page_from_gfn():
- Removedp2m_is_foreign()and related logic, as this functionality is not implemented for RISC-V.Yet I expect you'll need this, sooner or later.
Then I'll add correspondent code in this patch.
Quoted 37 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1063
Quoted 12 lines
xen/arch/riscv/p2m.c +6
-0@@ -1055,3 +1055,134 @@ int guest_physmap_add_entry(struct domain *d, {return p2m_insert_mapping (d, gfn, (1 << page_order), mfn, t); } + +/* +* Get the details of a given gfn. +* +* If the entry is present, the associated MFN will be returned and the +* access and type filled up. The page_order will correspond to the You removed
p2m_access_t* from the parameters; you need to also update the comment then accordingly.Quoted 9 lines
xen/arch/riscv/p2m.c +9
-0+* order of the mapping in the page table (i.e it could be a superpage). +* +* If the entry is not present, INVALID_MFN will be returned and the +* page_order will be set according to the order of the invalid range. +* +* valid will contain the value of bit[0] (e.g valid bit) of the +* entry. +*/ +static mfn_t p2m_get_entry(struct p2m_domain *p2m, gfn_t gfn,xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1082
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+p2m_type_t *t, + unsigned int *page_order, + bool *valid) +{ + paddr_t addr = gfn_to_gaddr(gfn); +unsigned int level =0 ; +pte_t entry, *table; +int rc; +mfn_t mfn =INVALID_MFN ; +p2m_type_t _t;Please no local variables with leading underscores. In x86 we commonly name such variables p2mt.
Quoted 2 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1083
Quoted 1 line
xen/arch/riscv/p2m.c +1
-0+DECLARE_OFFSETS (offsets, addr);This is the sole use of "addr". Is such a local variable really worth having?
Not really, I'll drop it.
Quoted 36 lines
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+ASSERT (p2m_is_locked (p2m)); +BUILD_BUG_ON (XEN_PT_LEVEL_MAP_MASK (0 ) !=PAGE_MASK ); + +/* Allow t to be NULL */ + t = t ?: &_t; + + *t = p2m_invalid;xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1101
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ +if ( valid ) + *valid = false; + +/* XXX: Check if the mapping is lower than the mapped gfn */ + +/* This gfn is higher than the highest the p2m map currently holds */ +if (gfn_x (gfn) >gfn_x (p2m->max_mapped_gfn) ) + { +for ( level =P2M_ROOT_LEVEL ; level ; level-- )Nit: Stray blank before the 2nd semicolon. (Again at least once below.)
Quoted 17 lines
xen/arch/riscv/p2m.c +17
-0+if ( (gfn_x (gfn) & (XEN_PT_LEVEL_MASK (level) >>PAGE_SHIFT )) > +gfn_x (p2m->max_mapped_gfn) ) +break ; + + goto out; + } + + table =p2m_get_root_pointer (p2m, gfn); + +/* +* the table should always be non-NULL because the gfn is below +* p2m->max_mapped_gfn and the root table pages are always present. +*/ +if ( !table ) + { +ASSERT_UNREACHABLE (); + level =P2M_ROOT_LEVEL ;
Quoted 8 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1125
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+ goto out; + } + +for ( level =P2M_ROOT_LEVEL ; level ; level-- ) + { + rc =p2m_next_level (p2m, true, level, &table, offsets[level]); +if ( (rc ==GUEST_TABLE_MAP_NONE ) && (rc !=GUEST_TABLE_MAP_NOMEM ) )This condition looks odd. As written the rhs of the && is redundant.
And it is wrong. It should:
if ( (rc == P2M_TABLE_MAP_NONE) || (rc == P2M_TABLE_MAP_NOMEM) )
Quoted 4 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1127
Quoted 2 lines
xen/arch/riscv/p2m.c +2
-0+ goto out_unmap; + else if ( rc != GUEST_TABLE_NORMAL )As before, no real need for "else" in such cases.
Quoted 10 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1135
Quoted 8 lines
xen/arch/riscv/p2m.c +8
-0+break ; + } + + entry = table[offsets[level]]; + +if (p2me_is_valid (p2m, entry) ) + { + *t =p2m_type_radix_get (p2m, entry);If the incoming argument is NULL, the somewhat expensive radix tree lookup is unnecessary here.
Good point.
Quoted 10 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1146
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ mfn =pte_get_mfn (entry); +/* +* The entry may point to a superpage. Find the MFN associated +* to the GFN. +*/ + mfn =mfn_add (mfn, +gfn_x (gfn) & (BIT (XEN_PT_LEVEL_ORDER (level),UL ) -1 )); + +if ( valid ) + *valid =pte_is_valid (entry);
Quoted 3 lines
Interesting. Why not the P2M counterpart of the function? Yes, the comment ahead of the function says so, but I don't see why the valid bit suddenly is relevant here (besides the P2M type).
This valid variable is expected to be used in the caller (something what Arm does here
https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/arm/p2m.c#L375) to check if it is needed to do flush_page_to_ram(), so if the the valid bit in PTE was set to 0 then it means nothing should be flushed as entry wasn't used as it marked invalid.
Quoted 8 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1152
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+ } + +out_unmap: +unmap_domain_page (table); + +out:Nit: Style (bot labels).
Quoted 9 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1159
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+if ( page_order ) + *page_order =XEN_PT_LEVEL_ORDER (level); + +return mfn; +} + +static mfn_t p2m_lookup (struct domain *d,gfn_t gfn,p2m_type_t *t)pointer-to-const for the 1st arg? But again more likely struct
p2m_domain* anyway?
|struct p2_domain| would be better, but I’m not really sure that a pointer-to-const can be used here. I expect that, in that case,
|p2m_read_lock()| would also need to accept a pointer-to-const, and since it calls|read_lock()| internally, that could be a problem because|read_lock() |accepts a|rwlock_t *l|.
Quoted 14 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1171
Quoted 12 lines
xen/arch/riscv/p2m.c +12
-0+{ +mfn_t mfn; +struct p2m_domain *p2m =p2m_get_hostp2m (d); + +p2m_read_lock (p2m); + mfn =p2m_get_entry (p2m, gfn, t,NULL ,NULL ); +p2m_read_unlock (p2m); + +return mfn; +} + +struct page_info *p2m_get_page_from_gfn(struct domain *d, gfn_t gfn,Same here - likely you mean struct
p2m_domain* instead.
Quoted 5 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1174
Quoted 3 lines
xen/arch/riscv/p2m.c +3
-0+ p2m_type_t *t) +{ +p2m_type_t p2mt = {0 };Why a compound initializer for something that isn't a compound object? And why plain 0 for something that is an enumerated type?
Agree, it should be a compound initializer. I'll drop it.
Quoted 8 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1180
Quoted 7 lines
xen/arch/riscv/p2m.c +6
-0+struct page_info *page; + +mfn_t mfn =p2m_lookup (d, gfn, &p2mt); + +if ( t ) + *t = p2mt;What's wrong with passing t directly to
p2m_lookup()?
It was needed before when the code of p2m_get_page_from_gfn() looked like:
struct page_info *p2m_get_page_from_gfn(struct domain *d, gfn_t gfn,
p2m_type_t *t)
{
struct page_info *page;
p2m_type_t p2mt;
mfn_t mfn = p2m_lookup(d, gfn, &p2mt);
if ( t )
*t = p2mt;
if ( !p2m_is_any_ram(p2mt) )
return NULL;So it was needed to make sure that p2m_is_any_ram(*t) doesn't try to dereference a NULL pointer.
Even with the current one implementation the similar issue could be with if use *t instead of p2mt:
struct page_info *p2m_get_page_from_gfn(struct p2m_domain *p2m, gfn_t gfn,
p2m_type_t *t)
{
...
if ( p2m_is_foreign(p2mt) )
{
struct domain *fdom = page_get_owner_and_reference(page);And the second reason it is because of p2m_get_entry() (which is used inside p2m_lookup()) could return for t a pointer to local variable:
static mfn_t p2m_get_entry(struct p2m_domain *p2m, gfn_t gfn,
p2m_type_t *t,
unsigned int *page_order,
bool *valid)
{
...
p2m_type_t p2mt;
...
/* Allow t to be NULL */
t = t ?: &p2mt;
...What looks wrong. I will remove this part of the code and pass t directly to p2m_lookup().
And after p2m_lookup() call will just check if t argument is NULL then init it with p2mt:
struct page_info *p2m_get_page_from_gfn(struct p2m_domain *p2m, gfn_t gfn,
p2m_type_t *t)
{
struct page_info *page;
p2m_type_t p2mt = p2m_invalid;
mfn_t mfn = p2m_lookup(p2m, gfn, t);
if ( !mfn_valid(mfn) )
return NULL;
if ( !t )
p2mt = *t;
...}
Thanks.
~ Oleksii
Re: [PATCH v2 17/17] xen/riscv: add support of page lookup by GFN
Quoted 119 lines · Oleksii Kurochko
Quoted 38 lines · Jan Beulich
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1063
Quoted 12 lines · Oleksii Kurochko
xen/arch/riscv/p2m.c +6
-0@@ -1055,3 +1055,134 @@ int guest_physmap_add_entry(struct domain *d, {return p2m_insert_mapping (d, gfn, (1 << page_order), mfn, t); } + +/* +* Get the details of a given gfn. +* +* If the entry is present, the associated MFN will be returned and the +* access and type filled up. The page_order will correspond to the You removed
p2m_access_t* from the parameters; you need to also update the comment then accordingly.Quoted 9 lines
xen/arch/riscv/p2m.c +9
-0+* order of the mapping in the page table (i.e it could be a superpage). +* +* If the entry is not present, INVALID_MFN will be returned and the +* page_order will be set according to the order of the invalid range. +* +* valid will contain the value of bit[0] (e.g valid bit) of the +* entry. +*/ +static mfn_t p2m_get_entry(struct p2m_domain *p2m, gfn_t gfn,xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1082
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+p2m_type_t *t, + unsigned int *page_order, + bool *valid) +{ + paddr_t addr = gfn_to_gaddr(gfn); +unsigned int level =0 ; +pte_t entry, *table; +int rc; +mfn_t mfn =INVALID_MFN ; +p2m_type_t _t;Please no local variables with leading underscores. In x86 we commonly name such variables p2mt.
Quoted 2 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1083
Quoted 1 line
xen/arch/riscv/p2m.c +1
-0+DECLARE_OFFSETS (offsets, addr);This is the sole use of "addr". Is such a local variable really worth having?
Not really, I'll drop it.
Quoted 36 lines
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+ASSERT (p2m_is_locked (p2m)); +BUILD_BUG_ON (XEN_PT_LEVEL_MAP_MASK (0 ) !=PAGE_MASK ); + +/* Allow t to be NULL */ + t = t ?: &_t; + + *t = p2m_invalid;xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1101
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ +if ( valid ) + *valid = false; + +/* XXX: Check if the mapping is lower than the mapped gfn */ + +/* This gfn is higher than the highest the p2m map currently holds */ +if (gfn_x (gfn) >gfn_x (p2m->max_mapped_gfn) ) + { +for ( level =P2M_ROOT_LEVEL ; level ; level-- )Nit: Stray blank before the 2nd semicolon. (Again at least once below.)
Quoted 17 lines
xen/arch/riscv/p2m.c +17
-0+if ( (gfn_x (gfn) & (XEN_PT_LEVEL_MASK (level) >>PAGE_SHIFT )) > +gfn_x (p2m->max_mapped_gfn) ) +break ; + + goto out; + } + + table =p2m_get_root_pointer (p2m, gfn); + +/* +* the table should always be non-NULL because the gfn is below +* p2m->max_mapped_gfn and the root table pages are always present. +*/ +if ( !table ) + { +ASSERT_UNREACHABLE (); + level =P2M_ROOT_LEVEL ;Quoted 8 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1125
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+ goto out; + } + +for ( level =P2M_ROOT_LEVEL ; level ; level-- ) + { + rc =p2m_next_level (p2m, true, level, &table, offsets[level]); +if ( (rc ==GUEST_TABLE_MAP_NONE ) && (rc !=GUEST_TABLE_MAP_NOMEM ) )This condition looks odd. As written the rhs of the && is redundant.
And it is wrong. It should:
if ( (rc ==P2M_TABLE_MAP_NONE) || (rc ==P2M_TABLE_MAP_NOMEM) )Quoted 4 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1127
Quoted 2 lines
xen/arch/riscv/p2m.c +2
-0+ goto out_unmap; + else if ( rc != GUEST_TABLE_NORMAL )As before, no real need for "else" in such cases.
Quoted 10 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1135
Quoted 8 lines
xen/arch/riscv/p2m.c +8
-0+break ; + } + + entry = table[offsets[level]]; + +if (p2me_is_valid (p2m, entry) ) + { + *t =p2m_type_radix_get (p2m, entry);If the incoming argument is NULL, the somewhat expensive radix tree lookup is unnecessary here.
Good point.
Quoted 10 lines
Quoted 10 lines
xen/arch/riscv/p2m.c +10
-0+ mfn =pte_get_mfn (entry); +/* +* The entry may point to a superpage. Find the MFN associated +* to the GFN. +*/ + mfn =mfn_add (mfn, +gfn_x (gfn) & (BIT (XEN_PT_LEVEL_ORDER (level),UL ) -1 )); + +if ( valid ) + *valid =pte_is_valid (entry);
Quoted 8 lines
Quoted 3 lines
Interesting. Why not the P2M counterpart of the function? Yes, the comment ahead of the function says so, but I don't see why the valid bit suddenly is relevant here (besides the P2M type).
This valid variable is expected to be used in the caller (something what Arm does here
https://gitlab.com/xen-project/xen/-/blob/staging/xen/arch/arm/p2m.c#L375) to check if it is needed to doflush_page_to_ram(), so if the the valid bit in PTE was set to 0 then it means nothing should be flushed as entry wasn't used as it marked invalid.
I hope you see what a mess you get if you have two flavors of "valid" for a PTE.
Quoted 18 lines
Quoted 8 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1152
Quoted 6 lines
xen/arch/riscv/p2m.c +6
-0+ } + +out_unmap: +unmap_domain_page (table); + +out:Nit: Style (bot labels).
Quoted 9 lines
xen/arch/riscv/p2m.c int guest_physmap_add_entry(struct domain *d, · line 1159
Quoted 7 lines
xen/arch/riscv/p2m.c +7
-0+if ( page_order ) + *page_order =XEN_PT_LEVEL_ORDER (level); + +return mfn; +} + +static mfn_t p2m_lookup (struct domain *d,gfn_t gfn,p2m_type_t *t)pointer-to-const for the 1st arg? But again more likely struct
p2m_domain* anyway?
Quoted 2 lines
|struct p2_domain| would be better, but I’m not really sure that a pointer-to-const can be used here.
Note how I also didn't suggest const there.
Quoted 3 lines
I expect that, in that case,
|p2m_read_lock()| would also need to accept a pointer-to-const, and since it calls|read_lock()| internally, that could be a problem because|read_lock() |accepts a|rwlock_t *l|.
Correct.
Jan