Skip to contentSkip to conversation

Add AMD Secure Nested Paging (SEV-SNP)

PATCH In review Ashish Kalra 22 participants 306 messages Jun 20, 2022 – Jan 5, 2023 Page 2 of 3

Subsystems (12)
lore
Share & tools
lore
author maintainer
#176 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

In reply to Kalra, Ashish
Quoted 68 lines · Ashish

Hello Vlastimil,

Quoted 54 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra%40amd.com%2F&data=05%7C01%7Cashish.kalra%40amd.com%7C944b59f239c541a52ac808dac71c2089%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041220947600149%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=do9zzyMlAErkKx5rguqnL2GoG4lhsWHDI74zgwLWaZU%3D&reserved=0

Quoted 11 lines · Ashish

Hello Boris,

Quoted 8 lines · Borislav Petkov
Quoted 3 lines · Ashish

       if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Will look at adding our own recovery function for the same, but that will again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

Quoted 9 lines

But there is still added complexity of handling hugepages as part of reclamation failures (both HugeTLB and transparent hugepages) and that means calling more static functions in mm/memory_failure.c

There is probably a more appropriate handler in mm/memory-failure.c:

soft_offline_page() - this will mark the page as HWPoisoned and also has handling for hugepages. And we can avoid adding a new page flag too.

Quoted 7 lines

soft_offline_page - Soft offline a page.
Soft offline a page, by migration or invalidation, without killing anything.

So, this looks like a good option to call
soft_offline_page() instead of memory_failure() in case of failure to transition the page back to HV/shared state via SNP_RECLAIM_CMD and/or RMPUPDATE instruction.

So it's a bit unclear to me what exact situation we are handling here. The original patch here seems to me to be just leaking back pages that are unsafe for further use. soft_offline_page() seems to fit that scenario of a graceful leak before something is irrepairably corrupt and we page fault on it. But then in the thread you discus PF handling and killing. So what is the case here? If we detect this need to call snp_leak_pages() does it mean:

Quoted 2 lines

a) nobody that could page fault at them (the guest?) is running anymore, we are tearing it down, we just can't reuse the pages further on the host

The host can page fault on them, if anything on the host tries to write to these pages. Host reads will return garbage data.

Quoted 6 lines

- seem like soft_offline_page() could work, but maybe we could just put the
pages on some leaked lists without special page? The only thing that should matter is not to free the pages to the page allocator so they would be reused by something else.

Quoted 5 lines
Quoted 2 lines

b) something can stil page fault at them (what?) - AFAIU can't be resolved without killing something, memory_failure() might limit the damage

As i mentioned above, host writes will cause RMP violation page fault.

And to add here, if its a guest private page, then the above fault cannot be resolved, so the faulting process is terminated.

Thanks,
Ashish

Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

Still waiting for some/more feedback from mm folks on the same.

Just send the patch and they'll give it.

Thx.

#177 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 71 lines · Ashish
Quoted 69 lines · Ashish

Hello Vlastimil,

Quoted 55 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra%40amd.com%2F&data=05%7C01%7Cashish.kalra%40amd.com%7C944b59f239c541a52ac808dac71c2089%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041220947600149%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=do9zzyMlAErkKx5rguqnL2GoG4lhsWHDI74zgwLWaZU%3D&reserved=0

Quoted 11 lines · Ashish

Hello Boris,

Quoted 8 lines · Borislav Petkov
Quoted 3 lines · Ashish

       if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Will look at adding our own recovery function for the same, but that will again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

Quoted 9 lines

But there is still added complexity of handling hugepages as part of reclamation failures (both HugeTLB and transparent hugepages) and that means calling more static functions in mm/memory_failure.c

There is probably a more appropriate handler in mm/memory-failure.c:

soft_offline_page() - this will mark the page as HWPoisoned and also has handling for hugepages. And we can avoid adding a new page flag too.

Quoted 8 lines

soft_offline_page - Soft offline a page.
Soft offline a page, by migration or invalidation, without killing anything.

So, this looks like a good option to call
soft_offline_page() instead of memory_failure() in case of failure to transition the page back to HV/shared state via SNP_RECLAIM_CMD and/or RMPUPDATE instruction.

So it's a bit unclear to me what exact situation we are handling here. The original patch here seems to me to be just leaking back pages that are unsafe for further use. soft_offline_page() seems to fit that scenario of a graceful leak before something is irrepairably corrupt and we page fault on it.
But then in the thread you discus PF handling and killing. So what is the case here? If we detect this need to call snp_leak_pages() does it mean:

Quoted 2 lines

a) nobody that could page fault at them (the guest?) is running anymore, we are tearing it down, we just can't reuse the pages further on the host

The host can page fault on them, if anything on the host tries to write to these pages. Host reads will return garbage data.

Quoted 5 lines

- seem like soft_offline_page() could work, but maybe we could just put the pages on some leaked lists without special page? The only thing that should matter is not to free the pages to the page allocator so they would be reused by something else.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

b) something can stil page fault at them (what?) - AFAIU can't be resolved without killing something, memory_failure() might limit the damage

As i mentioned above, host writes will cause RMP violation page fault.

And to add here, if its a guest private page, then the above fault cannot be resolved, so the faulting process is terminated.

BTW would this not be mostly resolved as part of rebasing to UPM?
- host will not have these pages mapped in the first place (both kernel directmap and qemu userspace)
- guest will have them mapped, but I assume that the conversion from private to shared (that might fail?) can only happen after guest's mappings are invalidated in the first place?

Quoted 13 lines

Thanks,
Ashish

Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

Still waiting for some/more feedback from mm folks on the same.

Just send the patch and they'll give it.

Thx.

author maintainer
#178 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 81 lines · Vlastimil Babka
Quoted 71 lines · Ashish
Quoted 69 lines · Ashish

Hello Vlastimil,

Quoted 55 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra%40amd.com%2F&data=05%7C01%7Cashish.kalra%40amd.com%7C38f8b76238014c67049308dac7b213a5%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041865033588985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DDm7tZhUJLy%2BMzS1SXUnsBmBQnAI5dqR6tWZhCKRMEI%3D&reserved=0

Quoted 11 lines · Ashish

Hello Boris,

Quoted 8 lines · Borislav Petkov
Quoted 3 lines · Ashish

       if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Will look at adding our own recovery function for the same, but that will again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

Quoted 9 lines

But there is still added complexity of handling hugepages as part of reclamation failures (both HugeTLB and transparent hugepages) and that means calling more static functions in mm/memory_failure.c

There is probably a more appropriate handler in mm/memory-failure.c:

soft_offline_page() - this will mark the page as HWPoisoned and also has handling for hugepages. And we can avoid adding a new page flag too.

Quoted 8 lines

soft_offline_page - Soft offline a page.
Soft offline a page, by migration or invalidation, without killing anything.

So, this looks like a good option to call
soft_offline_page() instead of memory_failure() in case of failure to transition the page back to HV/shared state via SNP_RECLAIM_CMD and/or RMPUPDATE instruction.

So it's a bit unclear to me what exact situation we are handling here. The original patch here seems to me to be just leaking back pages that are unsafe for further use. soft_offline_page() seems to fit that scenario of a graceful leak before something is irrepairably corrupt and we page fault on it.
But then in the thread you discus PF handling and killing. So what is the case here? If we detect this need to call snp_leak_pages() does it mean:

Quoted 2 lines

a) nobody that could page fault at them (the guest?) is running anymore, we are tearing it down, we just can't reuse the pages further on the host

The host can page fault on them, if anything on the host tries to write to these pages. Host reads will return garbage data.

Quoted 5 lines

- seem like soft_offline_page() could work, but maybe we could just put the pages on some leaked lists without special page? The only thing that should matter is not to free the pages to the page allocator so they would be reused by something else.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

b) something can stil page fault at them (what?) - AFAIU can't be resolved without killing something, memory_failure() might limit the damage

As i mentioned above, host writes will cause RMP violation page fault.

And to add here, if its a guest private page, then the above fault cannot be resolved, so the faulting process is terminated.

Quoted 7 lines

BTW would this not be mostly resolved as part of rebasing to UPM?
- host will not have these pages mapped in the first place (both kernel directmap and qemu userspace)
- guest will have them mapped, but I assume that the conversion from private to shared (that might fail?) can only happen after guest's mappings are invalidated in the first place?

Yes, that will be true for guest private pages. But then there are host allocated pages for firmware use which will remain in firmware page state or reclaim state if they can't be transitioned back to HV/shared state once the firmware releases them back to the host and accessing them at this point can potentially cause RMP violation #PF.

Again i don't think this is going to happen regularly or frequently so it will be a rare error case where the page reclamation, i.e., the transition back to HV/shared state fails and now these pages are no longer safe to be used.

Referring back to your thoughts about putting these pages on some leaked pages list, do any such leaked pages list exist currently ?

Thanks,
Ashish

Quoted 11 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

Still waiting for some/more feedback from mm folks on the same.

Just send the patch and they'll give it.

Thx.

#179 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 102 lines · Ashish
Quoted 87 lines · Vlastimil Babka
Quoted 77 lines · Ashish
Quoted 75 lines · Ashish

Hello Vlastimil,

Quoted 58 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra%40amd.com%2F&data=05%7C01%7Cashish.kalra%40amd.com%7C38f8b76238014c67049308dac7b213a5%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041865033588985%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DDm7tZhUJLy%2BMzS1SXUnsBmBQnAI5dqR6tWZhCKRMEI%3D&reserved=0

Quoted 11 lines · Ashish

Hello Boris,

Quoted 8 lines · Borislav Petkov
Quoted 3 lines · Ashish

        if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Will look at adding our own recovery function for the same, but that will
again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

Quoted 13 lines

But there is still added complexity of handling hugepages as part of reclamation failures (both HugeTLB and transparent hugepages) and that means calling more static functions in mm/memory_failure.c

There is probably a more appropriate handler in mm/memory-failure.c:

soft_offline_page() - this will mark the page as HWPoisoned and also has handling for hugepages. And we can avoid adding a new page flag too.

soft_offline_page - Soft offline a page.
Soft offline a page, by migration or invalidation, without killing anything.

Quoted 5 lines

So, this looks like a good option to call soft_offline_page() instead of memory_failure() in case of failure to transition the page back to HV/shared state via SNP_RECLAIM_CMD
and/or RMPUPDATE instruction.

So it's a bit unclear to me what exact situation we are handling here. The original patch here seems to me to be just leaking back pages that are unsafe for further use. soft_offline_page() seems to fit that scenario of a
graceful leak before something is irrepairably corrupt and we page fault on it.
But then in the thread you discus PF handling and killing. So what is the case here? If we detect this need to call snp_leak_pages() does it mean:

Quoted 3 lines

a) nobody that could page fault at them (the guest?) is running anymore, we
are tearing it down, we just can't reuse the pages further on the host

The host can page fault on them, if anything on the host tries to write to these pages. Host reads will return garbage data.

Quoted 7 lines

- seem like soft_offline_page() could work, but maybe we could just put the
pages on some leaked lists without special page? The only thing that should
matter is not to free the pages to the page allocator so they would be reused by something else.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

b) something can stil page fault at them (what?) - AFAIU can't be resolved without killing something, memory_failure() might limit the damage

As i mentioned above, host writes will cause RMP violation page fault.

And to add here, if its a guest private page, then the above fault cannot be resolved, so the faulting process is terminated.

Quoted 7 lines

BTW would this not be mostly resolved as part of rebasing to UPM?
- host will not have these pages mapped in the first place (both kernel directmap and qemu userspace)
- guest will have them mapped, but I assume that the conversion from private to shared (that might fail?) can only happen after guest's mappings are invalidated in the first place?

Yes, that will be true for guest private pages. But then there are host allocated pages for firmware use which will remain in firmware page state or reclaim state if they can't be transitioned back to HV/shared state once the firmware releases them back to the host and accessing them at this point can potentially cause RMP violation #PF.

Quoted 7 lines

Again i don't think this is going to happen regularly or frequently so it will be a rare error case where the page reclamation, i.e., the transition back to HV/shared state fails and now these pages are no longer safe to be used.

Referring back to your thoughts about putting these pages on some leaked pages list, do any such leaked pages list exist currently ?

Not AFAIK, you could just create a list_head somewhere appropriate (some snp state structure?) and put the pages there, maybe with a counter exposed in debugs. The point would be mostly that if something goes so wrong it would be leaking substantial amounts of memory, we can at least recognize the cause (but I suppose the dmesg will be also full of messages) and e.g. find the pages in a crash dump.

Quoted 14 lines

Thanks,
Ashish

Quoted 11 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

Still waiting for some/more feedback from mm folks on the same.

Just send the patch and they'll give it.

Thx.

author maintainer
#180 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 111 lines · Vlastimil Babka
Quoted 102 lines · Ashish
Quoted 87 lines · Vlastimil Babka
Quoted 77 lines · Ashish
Quoted 75 lines · Ashish

Hello Vlastimil,

Quoted 58 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra%40amd.com%2F&data=05%7C01%7Cashish.kalra%40amd.com%7C174b7caaf99a473194cd08dac7bcebf3%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041911481429347%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CFkAXNQqangvCqhnwDyIUJUkfiUrX67OpKDTtLGj6PU%3D&reserved=0

Quoted 11 lines · Ashish

Hello Boris,

Quoted 8 lines · Borislav Petkov
Quoted 3 lines · Ashish

        if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Will look at adding our own recovery function for the same, but that will
again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

Quoted 13 lines

But there is still added complexity of handling hugepages as part of reclamation failures (both HugeTLB and transparent hugepages) and that means calling more static functions in mm/memory_failure.c

There is probably a more appropriate handler in mm/memory-failure.c:

soft_offline_page() - this will mark the page as HWPoisoned and also has handling for hugepages. And we can avoid adding a new page flag too.

soft_offline_page - Soft offline a page.
Soft offline a page, by migration or invalidation, without killing anything.

Quoted 5 lines

So, this looks like a good option to call soft_offline_page() instead of memory_failure() in case of failure to transition the page back to HV/shared state via SNP_RECLAIM_CMD
and/or RMPUPDATE instruction.

So it's a bit unclear to me what exact situation we are handling here. The original patch here seems to me to be just leaking back pages that are unsafe for further use. soft_offline_page() seems to fit that scenario of a
graceful leak before something is irrepairably corrupt and we page fault on it.
But then in the thread you discus PF handling and killing. So what is the case here? If we detect this need to call snp_leak_pages() does it mean:

Quoted 3 lines

a) nobody that could page fault at them (the guest?) is running anymore, we
are tearing it down, we just can't reuse the pages further on the host

The host can page fault on them, if anything on the host tries to write to these pages. Host reads will return garbage data.

Quoted 7 lines

- seem like soft_offline_page() could work, but maybe we could just put the
pages on some leaked lists without special page? The only thing that should
matter is not to free the pages to the page allocator so they would be reused by something else.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

b) something can stil page fault at them (what?) - AFAIU can't be resolved without killing something, memory_failure() might limit the damage

As i mentioned above, host writes will cause RMP violation page fault.

And to add here, if its a guest private page, then the above fault cannot be resolved, so the faulting process is terminated.

Quoted 7 lines

BTW would this not be mostly resolved as part of rebasing to UPM?
- host will not have these pages mapped in the first place (both kernel directmap and qemu userspace)
- guest will have them mapped, but I assume that the conversion from private to shared (that might fail?) can only happen after guest's mappings are invalidated in the first place?

Yes, that will be true for guest private pages. But then there are host allocated pages for firmware use which will remain in firmware page state or reclaim state if they can't be transitioned back to HV/shared state once the firmware releases them back to the host and accessing them at this point can potentially cause RMP violation #PF.

Quoted 7 lines

Again i don't think this is going to happen regularly or frequently so it will be a rare error case where the page reclamation, i.e., the transition back to HV/shared state fails and now these pages are no longer safe to be used.

Referring back to your thoughts about putting these pages on some leaked pages list, do any such leaked pages list exist currently ?

Quoted 7 lines

Not AFAIK, you could just create a list_head somewhere appropriate (some snp state structure?) and put the pages there, maybe with a counter exposed in debugs. The point would be mostly that if something goes so wrong it would be leaking substantial amounts of memory, we can at least recognize the cause (but I suppose the dmesg will be also full of messages) and e.g. find the pages in a crash dump.

Ok, so i will work on implementing this leaked pages list and put it on a sev/snp associated structure.

Also to add here, we will actually get a not-present #PF instead of the RMP violation #PF on writing to these leaked pages, as these pages would have been removed from the kernel direct map.

Thanks,
Ashish

Quoted 13 lines
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines

Still waiting for some/more feedback from mm folks on the same.

Just send the patch and they'll give it.

Thx.

maintainer
#181 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 2 lines · Ashish

Ok, so i will work on implementing this leaked pages list and put it on a sev/snp associated structure.

See __sgx_sanitize_pages() and the poison list there, for an example.

Quoted 3 lines

Also to add here, we will actually get a not-present #PF instead of the RMP violation #PF on writing to these leaked pages, as these pages would have been removed from the kernel direct map.

So if you do the list and still have the kernel raise a RMP fault for those pages, traversing that list in the RMP handler to check whether the page is there on it, should be a lot faster operation than doing the #PF thing and removing them from the direct map.

And sorry for misleading you about UPM - we were thinking wrong yesterday.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#182 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 6 lines · Borislav Petkov
Quoted 2 lines · Ashish

Ok, so i will work on implementing this leaked pages list and put it on a sev/snp associated structure.

See __sgx_sanitize_pages() and the poison list there, for an example.

Quoted 9 lines
Quoted 3 lines

Also to add here, we will actually get a not-present #PF instead of the RMP violation #PF on writing to these leaked pages, as these pages would have been removed from the kernel direct map.

So if you do the list and still have the kernel raise a RMP fault for those pages, traversing that list in the RMP handler to check whether the page is there on it, should be a lot faster operation than doing the #PF thing and removing them from the direct map.

Actually, these host allocated pages would have already been removed from the kernel direct map, when they were transitioned to the firmware state. So actually the not-present #PF fault will happen on any read/write access to these leaked pages instead of the RMP violation #PF (not-present #PF has higher priority than RMP violation #PF).

If these pages cannot be reclaimed, they are unsafe to use and cannot be added back to the kernel direct map.

Thanks,
Ashish

Quoted 5 lines

And sorry for misleading you about UPM - we were thinking wrong yesterday.

Thx.

maintainer
#183 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 2 lines · Ashish

Actually, these host allocated pages would have already been removed from the kernel direct map,

And, as I said above, it would be a lot easier to handle any potential faults resulting from the host touching them by having it raise a *RMP* fault instead of normal *PF* fault where the latter code is a crazy mess.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#184 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 7 lines · Borislav Petkov
Quoted 2 lines · Ashish

Actually, these host allocated pages would have already been removed from the kernel direct map,

And, as I said above, it would be a lot easier to handle any potential faults resulting from the host touching them by having it raise a *RMP* fault instead of normal *PF* fault where the latter code is a crazy mess.

Just to reiterate here, we won't be getting a *RMP* fault but will instead get a normal (not-present) #PF fault when the host touches these pages.

Sorry for any confusion about the fault signaled, earlier i mentioned we will get a RMP violation #PF, but actually as these pages are also removed from the kernel direct-map, therefore, we will get the not-present #PF and not the RMP #PF (core will check and signal not-present #PF before it performs the RMP checks).

Thanks,
Ashish

#185 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

In reply to Vlastimil Babka
Quoted 8 lines · Vlastimil Babka
Quoted 2 lines

Referring back to your thoughts about putting these pages on some leaked pages list, do any such leaked pages list exist currently ?

Not AFAIK, you could just create a list_head somewhere appropriate (some snp state structure?) and put the pages there, maybe with a counter exposed in debugs. The point would be mostly that if something goes so wrong it would be leaking substantial amounts of memory, we can at least recognize the cause (but I suppose the dmesg will be also full of messages) and e.g. find the pages in a crash dump.

It might also be worth looking through the places that check PageHWPoison() and making sure that none of them are poking into the page contents.

It's also the kind of thing that adding some CONFIG_DEBUG_VM checks might help with. For instance, nobody should ever be kmap*()'ing a private page. The same might even go for pin_user_pages().

#186 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

In reply to Vlastimil Babka
Quoted 22 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://lore.kernel.org/all/3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra@amd.com/

Quoted 17 lines · Ashish

Hello Boris,

Quoted 13 lines · Borislav Petkov
Quoted 3 lines · Ashish

      if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 2 lines

Will look at adding our own recovery function for the same, but that will again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Quoted 6 lines
Quoted 1 line

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

I agree with this. Just defining PG_offlimits as an alias of PG_hwpoison could break current hwpoison workload. So if you finally decide to go forward in this direction, you may as well have some indicator to distinguish the new kind of leaked pages from hwpoisoned pages.

I don't remember exact thread, but I've read someone writing about similar kind of suggestion of using memory_failure() to make pages inaccessible in non-memory error usecase. I feel that it could be possible to generalize memory_failure() as general-purpose page offlining (by renaming it with hard_offline_page() and making memory_failure() one of the user of it).

Thanks,
Naoya Horiguchi

author maintainer
#187 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 35 lines · HORIGUCHI NAOYA(堀口 直也)
Quoted 22 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra%40amd.com%2F&data=05%7C01%7Cashish.kalra%40amd.com%7C7b2d39d6e2504a8f923608dac792224b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041727879125176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KBJLKhPQP23vmvY%2FNnbjZs8wTJs%2FrF%2BiU54Sdc4Ldx4%3D&reserved=0

Quoted 17 lines · Ashish

Hello Boris,

Quoted 13 lines · Borislav Petkov
Quoted 3 lines · Ashish

      if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 2 lines

Will look at adding our own recovery function for the same, but that will again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Quoted 6 lines
Quoted 1 line

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

I agree with this. Just defining PG_offlimits as an alias of PG_hwpoison could break current hwpoison workload. So if you finally decide to go forward in this direction, you may as well have some indicator to distinguish the new kind of leaked pages from hwpoisoned pages.

Quoted 4 lines

I don't remember exact thread, but I've read someone writing about similar kind of suggestion of using memory_failure() to make pages inaccessible in non-memory error usecase. I feel that it could be possible to generalize memory_failure() as general-purpose page offlining (by renaming it with

But, doesn't memory_failure() also mark the pages as PG_hwpoison, and then using it for these leaked pages will again cause confusion with actual hwpoison ?

Thanks,
Ashish

Quoted 5 lines

hard_offline_page() and making memory_failure() one of the user of it).

Thanks,
Naoya Horiguchi

#188 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 36 lines · Ashish
Quoted 35 lines · HORIGUCHI NAOYA(堀口 直也)
Quoted 22 lines · Vlastimil Babka

Cc'ing memory failure folks, the beinning of this subthread is here:

https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F3a51840f6a80c87b39632dc728dbd9b5dd444cd7.1655761627.git.ashish.kalra%40amd.com%2F&data=05%7C01%7Cashish.kalra%40amd.com%7C7b2d39d6e2504a8f923608dac792224b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638041727879125176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=KBJLKhPQP23vmvY%2FNnbjZs8wTJs%2FrF%2BiU54Sdc4Ldx4%3D&reserved=0

Quoted 17 lines · Ashish

Hello Boris,

Quoted 13 lines · Borislav Petkov
Quoted 3 lines · Ashish

      if (snp_lookup_rmpentry(pfn, &rmp_level)) { do_sigbus(regs, error_code, address, VM_FAULT_SIGBUS); return RMP_PF_RETRY;

Does this issue some halfway understandable error message why the process got killed?

Quoted 2 lines

Will look at adding our own recovery function for the same, but that will again mark the pages as poisoned, right ?

Well, not poisoned but PG_offlimits or whatever the mm folks agree upon. Semantically, it'll be handled the same way, ofc.

Quoted 6 lines
Quoted 1 line

Added a new PG_offlimits flag and a simple corresponding handler for it.

One thing is, there's not enough page flags to be adding more (except aliases for existing) for cases that can avoid it, but as Boris says, if using alias to PG_hwpoison it depends what will become confused with the actual hwpoison.

I agree with this. Just defining PG_offlimits as an alias of PG_hwpoison could break current hwpoison workload. So if you finally decide to go forward in this direction, you may as well have some indicator to distinguish the new kind of leaked pages from hwpoisoned pages.

Quoted 8 lines
Quoted 4 lines

I don't remember exact thread, but I've read someone writing about similar kind of suggestion of using memory_failure() to make pages inaccessible in non-memory error usecase. I feel that it could be possible to generalize memory_failure() as general-purpose page offlining (by renaming it with

But, doesn't memory_failure() also mark the pages as PG_hwpoison, and then using it for these leaked pages will again cause confusion with actual hwpoison ?

Yes, so we might need modification of memory_failure code for this approach like renaming PG_hwpoison to more generic one (although some possible names like PageOffline and PageIsolated are already used) and/or somehow showing "which kind of leaked pages" info.

Thanks,
Naoya Horiguchi

author maintainer
#189 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

In reply to Kalra, Ashish

Hello Boris,

Quoted 25 lines
Quoted 25 lines
Quoted 10 lines

+        if (ret)
+            goto cleanup;

drivers/crypto/ccp/sev-dev.c +7 -0
+
+        ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+        if (ret)
+            goto cleanup;
+
+        pfn++;
+        n++;

drivers/crypto/ccp/sev-dev.c static int sev_cmd_buffer_len(int cmd) · line 229

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+    }
+
+    return 0;
+
+cleanup:
+    /*
+     * If failed to reclaim the page then page is no longer safe to
+     * be released, leak it.
+     */
+    snp_leak_pages(pfn, npages - n);

So this looks real weird: we go and reclaim pages, we hit an error during reclaiming a page X somewhere in-between and then we go and mark the *remaining* pages as not to be used?!

Quoted 5 lines
Quoted 4 lines

Why?

Why not only that *one* page which failed and then we continue with the rest?!

I had a re-look at this while fixing the memory_failure() handling and realized that we can't do a *partial* recovery here if we fail to reclaim a single page, i.e., if we hit an error during reclaiming a single page we need to mark the remaining pages as not usable.

This is because this page could be a part of larger buffer which would have been transitioned to firmware state and need to be restored back in *full* to HV/shared state, any access to a partially transitioned buffer will still cause failures, basically the callers won't be able to do any kind of a recovery/access on a partially restored/reclaimed buffer and now potentially fragmented buffer, which anyway means failure due to data loss on non reclaimed page(s).

So we need to be able to reclaim all the pages or none.

Also this failure won't be happening regularly/frequently, it is a *rare* error case and if there is a reclamation failure on a single page, there is a high probability that there will be reclamation failures on subsequent pages.

Thanks,
Ashish

maintainer
#190 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 1 line · Ashish

So we need to be able to reclaim all the pages or none.

/me goes and looks at SNP_PAGE_RECLAIM's retvals:

- INVALID_PLATFORM_STATE - platform is not in INIT state. That's certainly not a reason to leak pages.

- INVALID_ADDRESS - PAGE_PADDR is not a valid system physical address. That's botched command buffer but not a broken page so no reason to leak them either.

- INVALID_PAGE_STATE - the page is neither of those types: metadata, firmware, pre-guest nor pre-swap. So if you issue page reclaim on the wrong range of pages that looks again like a user error but no need to leak pages.

- INVALID_PAGE_SIZE - a size mismatch. Still sounds to me like a user error of sev-guest instead of anything wrong deeper in the FW or HW.

So in all those, if you end up supplying the wrong range of addresses, you most certainly will end up leaking the wrong pages.

So it sounds to me like you wanna say: "Error reclaiming range, check your driver" instead of punishing any innocent pages.

Now, if the retval from the fw were FIRMWARE_INTERNAL_ERROR or so, then sure, by all means. But not for the above. All the error conditions above sound like the kernel has supplied the wrong range/botched command buffer to the firmware so there's no need to leak pages.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#191 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Hello Boris,

Quoted 7 lines · Borislav Petkov
Quoted 1 line · Ashish

So we need to be able to reclaim all the pages or none.

/me goes and looks at SNP_PAGE_RECLAIM's retvals:

- INVALID_PLATFORM_STATE - platform is not in INIT state. That's certainly not a reason to leak pages.

This should not happen, as there are sev->snp_initialized checks before any firmware page allocation or snp page transitions.

Quoted 10 lines

- INVALID_ADDRESS - PAGE_PADDR is not a valid system physical address. That's botched command buffer but not a broken page so no reason to leak them either.

- INVALID_PAGE_STATE - the page is neither of those types: metadata, firmware, pre-guest nor pre-swap. So if you issue page reclaim on the wrong range of pages that looks again like a user error but no need to leak pages.

Quoted 8 lines

- INVALID_PAGE_SIZE - a size mismatch. Still sounds to me like a user error of sev-guest instead of anything wrong deeper in the FW or HW.

So in all those, if you end up supplying the wrong range of addresses, you most certainly will end up leaking the wrong pages.

So it sounds to me like you wanna say: "Error reclaiming range, check your driver" instead of punishing any innocent pages.

I agree, but these pages are not in the right state to be released back to the system or accessed by the host, because they have already been transitioned successfully to firmware state and the reclaim has failed. If we release them back to page-allocator and whenever the host accesses them, it will get a not-present #PF and it will panic/crash the host process.

It might be a user/sev-guest error, but these pages are now unsafe to use. So is a kernel panic justified here, instead of not releasing the pages back to host and logging errors for the same.

Thanks,
Ashish

Quoted 8 lines

Now, if the retval from the fw were FIRMWARE_INTERNAL_ERROR or so, then sure, by all means. But not for the above. All the error conditions above sound like the kernel has supplied the wrong range/botched command buffer to the firmware so there's no need to leak pages.

Thx.

maintainer
#192 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 1 line · Ashish

I agree, but these pages are not in the right state to be released back to

Which pages exactly?

Some pages' state has really changed underneath or you've given the wrong range?

Quoted 3 lines

It might be a user/sev-guest error, but these pages are now unsafe to use. So is a kernel panic justified here, instead of not releasing the pages back to host and logging errors for the same.

Ok, there are two cases:

* kernel error: I guess a big fat warning is the least we can issue here. Not sure about panic considering this should almost never happen and a warning would allow for people to catch dumps and debug the issue.

* firmware error: I don't think you can know that that is really the case on a production system without additional fw debugging capabilities. Dumping a warning would be the least we can do here too, to signal that something's out of the ordinary and so people can look into it further.

So yeah, a big fat warning is a good start. And then you don't need any memory poisoning etc gunk.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#193 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Hello Boris,

Quoted 18 lines · Borislav Petkov
Quoted 1 line · Ashish

I agree, but these pages are not in the right state to be released back to

Which pages exactly?

Some pages' state has really changed underneath or you've given the wrong range?

Quoted 3 lines

It might be a user/sev-guest error, but these pages are now unsafe to use. So is a kernel panic justified here, instead of not releasing the pages back to host and logging errors for the same.

Ok, there are two cases:

* kernel error: I guess a big fat warning is the least we can issue here. Not sure about panic considering this should almost never happen and a warning would allow for people to catch dumps and debug the issue.

Quoted 5 lines

* firmware error: I don't think you can know that that is really the case on a production system without additional fw debugging capabilities. Dumping a warning would be the least we can do here too, to signal that something's out of the ordinary and so people can look into it further.

Please note that in both cases, these non-reclaimed pages cannot be freed/returned back to the page allocator. Anytime the kernel accesses these pages it will cause a panic or host process crash.

So along with warning, the pages will be added to a leaked pages list, but there is no poisoning or anything, only need to ensure that these pages are not touched/accessed again.

Thanks,
Ashish

Quoted 4 lines

So yeah, a big fat warning is a good start. And then you don't need any memory poisoning etc gunk.

maintainer
#194 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 2 lines · Ashish

Please note that in both cases, these non-reclaimed pages cannot be freed/returned back to the page allocator.

You keep repeating "these pages". Which pages?

What if you specify the wrong, innocent pages because the kernel is wrong?

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#195 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 5 lines · Borislav Petkov
Quoted 2 lines · Ashish

Please note that in both cases, these non-reclaimed pages cannot be freed/returned back to the page allocator.

You keep repeating "these pages". Which pages?

The pages which have been allocated for firmware use (such as for SNP_INIT command, TMR memory for SEV-ES usage, etc), command buffers used for SEV legacy commands when SNP is enabled.

Here is a detailed description of the SEV legacy command handling when SNP is enabled:
The behavior of the SEV-legacy commands is altered when the SNP firmware
is in the INIT state. When SNP is in INIT state, all the SEV-legacy
commands that cause the firmware to write to memory must be in the
firmware state before issuing the command. A command buffer may contain a system physical address that the firmware may write to. In case the command buffer contains a system physical address points to a guest memory, we need to change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.

Then there are host buffers allocated for SNP platform status command, SNP launch update and SNP launch update vmsa command.

The other pages which can be user or guest provided are SNP guest requests and SNP guest debug helpers.

It is important to note that if invalid address/len are supplied, the failure will happen at the initial stage itself of transitioning these pages to firmware state.

But if the above pages have been successfully transitioned to firmware state and passed on to the SNP firmware, then after return, they need to be restored to shared state. If this restoration/reclamation fails, then accessing these pages will cause the kernel to panic.

Quoted 4 lines

What if you specify the wrong, innocent pages because the kernel is wrong?

In such a case the kernel panic is justifiable, but again if incorrect addresses are supplied, the failure will happen at the initial stage of transitioning these pages to firmware state and there is no need to reclaim.

Or, otherwise dump a warning and let the pages not be freed/returned back to the page allocator.

It is either innocent pages or kernel panic or an innocent host process crash (these are the choices to make).

Thanks,
Ashish

maintainer
#196 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Quoted 3 lines · Ashish

It is important to note that if invalid address/len are supplied, the failure will happen at the initial stage itself of transitioning these pages to firmware state.

/me goes and checks out your v6 tree based on 5.18.

Lemme choose one:

static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)

{
	...

        inpages = sev_pin_memory(kvm, params.uaddr, params.len, &npages, 1);

	...

        for (i = 0; i < npages; i++) {
                pfn = page_to_pfn(inpages[i]);

		...

                ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE, &data, error);
                if (ret) {
                        /*
                         * If the command failed then need to reclaim the page.
                         */
                        snp_page_reclaim(pfn);

and here it would leak the pages if it cannot reclaim them.

Now how did you get those?

Through params.uaddr and params.len which come from userspace:

        if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
                return -EFAULT;

Now, think about it, can userspace be trusted?

Exactly.

Yeah, yeah, I see it does is_hva_registered() but userspace can just as well supply the wrong region which fits.

Quoted 1 line

In such a case the kernel panic is justifiable,

So userspace can supply whatever it wants and you'd panic?

You surely don't mean that.

Quoted 3 lines

but again if incorrect addresses are supplied, the failure will happen at the initial stage of transitioning these pages to firmware state and there is no need to reclaim.

See above.

Quoted 5 lines

Or, otherwise dump a warning and let the pages not be freed/returned back to the page allocator.

It is either innocent pages or kernel panic or an innocent host process crash (these are the choices to make).

No, it is make the kernel as resilient as possible. Which means, no panic, add the pages to a not-to-be-used-anymore list and scream loudly with warning messages when it must leak pages so that people can fix the issue.

Ok?

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#197 lore raw

Re: [PATCH Part2 v6 14/49] crypto: ccp: Handle the legacy TMR allocation when SNP is enabled

Hello Boris,

Quoted 39 lines · Borislav Petkov
Quoted 3 lines · Ashish

It is important to note that if invalid address/len are supplied, the failure will happen at the initial stage itself of transitioning these pages to firmware state.

/me goes and checks out your v6 tree based on 5.18.

Lemme choose one:

static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)

{
	...

         inpages = sev_pin_memory(kvm, params.uaddr, params.len, &npages, 1);

	...

         for (i = 0; i < npages; i++) {
                 pfn = page_to_pfn(inpages[i]);

		...

                 ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE, &data, error);
                 if (ret) {
                         /*
                          * If the command failed then need to reclaim the page.
                          */
                         snp_page_reclaim(pfn);

and here it would leak the pages if it cannot reclaim them.

Now how did you get those?

Through params.uaddr and params.len which come from userspace:

         if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
                 return -EFAULT;
Quoted 6 lines

Now, think about it, can userspace be trusted?

Exactly.

Yeah, yeah, I see it does is_hva_registered() but userspace can just as well supply the wrong region which fits.

Yes, that's right.

Also, before sev_issue_cmd() above, there is a call to rmp_make_private() to make these pages transition to firmware state before we issue the LAUNCH_UPDATE command as below:

           ret = rmp_make_private(pfn, gfn << PAGE_SHIFT, level,
				  sev_get_asid(kvm), true);
                if (ret) {
                        ret = -EFAULT;
                        goto e_unpin;

                }
...
 	ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE,
			      &data, error);

So in case, the userspace provided an invalid/incorrect range, this
transition would have failed and there would not have been a need to do any reclaim, so there are no pages leaked here.

This is also the reason why we need to reclaim pages if the subsequent LAUNCH_UPDATE command fails, as now the pages are in F/W state because of the rmp_make_private() call and they are now unsafe to be used by the host.

Quoted 7 lines
Quoted 2 lines

In such a case the kernel panic is justifiable,

So userspace can supply whatever it wants and you'd panic?

You surely don't mean that.

No, we don't want to do that.

Quoted 3 lines
Quoted 3 lines

but again if incorrect addresses are supplied, the failure will happen at the initial stage of transitioning these pages to firmware state and there is no need to reclaim.

This is the case i mentioned above, rmp_make_private() before the firmware command is the initial stage of transitioning the pages to firmware state before issuing the firmware command.

Quoted 9 lines

See above.

Quoted 5 lines

Or, otherwise dump a warning and let the pages not be freed/returned back to the page allocator.

It is either innocent pages or kernel panic or an innocent host process crash (these are the choices to make).

Quoted 7 lines

No, it is make the kernel as resilient as possible. Which means, no panic, add the pages to a not-to-be-used-anymore list and scream loudly with warning messages when it must leak pages so that people can fix the issue.

Ok?

Right, yes, i totally agree.

So now we are adding these pages to an internal not-to-be-used-anymore list and printing warnings and ensuring no panics as we don't allow these pages to be released back to the page allocator.

Thanks,
Ashish

author maintainer
#198 lore raw

PATCH v6 15/49 crypto: ccp: Handle the legacy SEV command when SNP is enabled

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The behavior of the SEV-legacy commands is altered when the SNP firmware is in the INIT state. When SNP is in INIT state, all the SEV-legacy commands that cause the firmware to write to memory must be in the firmware state before issuing the command..

A command buffer may contains a system physical address that the firmware may write to. There are two cases that need to be handled:

1) system physical address points to a guest memory
2) system physical address points to a host memory

To handle the case #1, change the page state to the firmware in the RMP table before issuing the command and restore the state to shared after the command completes.

For the case #2, use a bounce buffer to complete the request.

---
 drivers/crypto/ccp/sev-dev.c | 346 ++++++++++++++++++++++++++++++++++-
 drivers/crypto/ccp/sev-dev.h |  12 ++
 2 files changed, 348 insertions(+), 10 deletions(-)
drivers/crypto/ccp/sev-dev.c +336 -10
index 0dbd99f29b25..75f5c4ed9ac3 100644
@@ -441,12 +441,295 @@ static void sev_write_init_ex_file_if_required(int cmd_id)
 	sev_write_init_ex_file();
 }
 
+static int alloc_snp_host_map(struct sev_device *sev)
+{
+	struct page *page;
+	int i;
+
+	for (i = 0; i < MAX_SNP_HOST_MAP_BUFS; i++) {
+		struct snp_host_map *map = &sev->snp_host_map[i];
+
+		memset(map, 0, sizeof(*map));
+
+		page = alloc_pages(GFP_KERNEL_ACCOUNT, get_order(SEV_FW_BLOB_MAX_SIZE));
+		if (!page)
+			return -ENOMEM;
+
+		map->host = page_address(page);
+	}
+
+	return 0;
+}
+
+static void free_snp_host_map(struct sev_device *sev)
+{
+	int i;
+
+	for (i = 0; i < MAX_SNP_HOST_MAP_BUFS; i++) {
+		struct snp_host_map *map = &sev->snp_host_map[i];
+
+		if (map->host) {
+			__free_pages(virt_to_page(map->host), get_order(SEV_FW_BLOB_MAX_SIZE));
+			memset(map, 0, sizeof(*map));
+		}
+	}
+}
+
+static int map_firmware_writeable(u64 *paddr, u32 len, bool guest, struct snp_host_map *map)
+{
+	unsigned int npages = PAGE_ALIGN(len) >> PAGE_SHIFT;
+
+	map->active = false;
+
+	if (!paddr || !len)
+		return 0;
+
+	map->paddr = *paddr;
+	map->len = len;
+
+	/* If paddr points to a guest memory then change the page state to firmwware. */
+	if (guest) {
+		if (snp_set_rmp_state(*paddr, npages, true, true, false))
+			return -EFAULT;
+
+		goto done;
+	}
+
+	if (!map->host)
+		return -ENOMEM;
+
+	/* Check if the pre-allocated buffer can be used to fullfil the request. */
+	if (len > SEV_FW_BLOB_MAX_SIZE)
+		return -EINVAL;
+
+	/* Transition the pre-allocated buffer to the firmware state. */
+	if (snp_set_rmp_state(__pa(map->host), npages, true, true, false))
+		return -EFAULT;
+
+	/* Set the paddr to use pre-allocated firmware buffer */
+	*paddr = __psp_pa(map->host);
+
+done:
+	map->active = true;
+	return 0;
+}
+
+static int unmap_firmware_writeable(u64 *paddr, u32 len, bool guest, struct snp_host_map *map)
+{
+	unsigned int npages = PAGE_ALIGN(len) >> PAGE_SHIFT;
+
+	if (!map->active)
+		return 0;
+
+	/* If paddr points to a guest memory then restore the page state to hypervisor. */
+	if (guest) {
+		if (snp_set_rmp_state(*paddr, npages, false, true, true))
+			return -EFAULT;
+
+		goto done;
+	}
+
+	/*
+	 * Transition the pre-allocated buffer to hypervisor state before the access.
+	 *
+	 * This is because while changing the page state to firmware, the kernel unmaps
+	 * the pages from the direct map, and to restore the direct map we must
+	 * transition the pages to shared state.
+	 */
+	if (snp_set_rmp_state(__pa(map->host), npages, false, true, true))
+		return -EFAULT;
+
+	/* Copy the response data firmware buffer to the callers buffer. */
+	memcpy(__va(__sme_clr(map->paddr)), map->host, min_t(size_t, len, map->len));
+	*paddr = map->paddr;
+
+done:
+	map->active = false;
+	return 0;
+}
+
+static bool sev_legacy_cmd_buf_writable(int cmd)
+{
+	switch (cmd) {
+	case SEV_CMD_PLATFORM_STATUS:
+	case SEV_CMD_GUEST_STATUS:
+	case SEV_CMD_LAUNCH_START:
+	case SEV_CMD_RECEIVE_START:
+	case SEV_CMD_LAUNCH_MEASURE:
+	case SEV_CMD_SEND_START:
+	case SEV_CMD_SEND_UPDATE_DATA:
+	case SEV_CMD_SEND_UPDATE_VMSA:
+	case SEV_CMD_PEK_CSR:
+	case SEV_CMD_PDH_CERT_EXPORT:
+	case SEV_CMD_GET_ID:
+	case SEV_CMD_ATTESTATION_REPORT:
+		return true;
+	default:
+		return false;
+	}
+}
+
+#define prep_buffer(name, addr, len, guest, map) \
+	func(&((typeof(name *))cmd_buf)->addr, ((typeof(name *))cmd_buf)->len, guest, map)
+
+static int __snp_cmd_buf_copy(int cmd, void *cmd_buf, bool to_fw, int fw_err)
+{
+	int (*func)(u64 *paddr, u32 len, bool guest, struct snp_host_map *map);
+	struct sev_device *sev = psp_master->sev_data;
+	bool from_fw = !to_fw;
+
+	/*
+	 * After the command is completed, change the command buffer memory to
+	 * hypervisor state.
+	 *
+	 * The immutable bit is automatically cleared by the firmware, so
+	 * no not need to reclaim the page.
+	 */
+	if (from_fw && sev_legacy_cmd_buf_writable(cmd)) {
+		if (snp_set_rmp_state(__pa(cmd_buf), 1, false, true, false))
+			return -EFAULT;
+
+		/* No need to go further if firmware failed to execute command. */
+		if (fw_err)
+			return 0;
+	}
+
+	if (to_fw)
+		func = map_firmware_writeable;
+	else
+		func = unmap_firmware_writeable;
+
+	/*
+	 * A command buffer may contains a system physical address. If the address
+	 * points to a host memory then use an intermediate firmware page otherwise
+	 * change the page state in the RMP table.
+	 */
+	switch (cmd) {
+	case SEV_CMD_PDH_CERT_EXPORT:
+		if (prep_buffer(struct sev_data_pdh_cert_export, pdh_cert_address,
+				pdh_cert_len, false, &sev->snp_host_map[0]))
+			goto err;
+		if (prep_buffer(struct sev_data_pdh_cert_export, cert_chain_address,
+				cert_chain_len, false, &sev->snp_host_map[1]))
+			goto err;
+		break;
+	case SEV_CMD_GET_ID:
+		if (prep_buffer(struct sev_data_get_id, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_PEK_CSR:
+		if (prep_buffer(struct sev_data_pek_csr, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_UPDATE_DATA:
+		if (prep_buffer(struct sev_data_launch_update_data, address, len,
+				true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_UPDATE_VMSA:
+		if (prep_buffer(struct sev_data_launch_update_vmsa, address, len,
+				true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_MEASURE:
+		if (prep_buffer(struct sev_data_launch_measure, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_LAUNCH_UPDATE_SECRET:
+		if (prep_buffer(struct sev_data_launch_secret, guest_address, guest_len,
+				true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_DBG_DECRYPT:
+		if (prep_buffer(struct sev_data_dbg, dst_addr, len, false,
+				&sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_DBG_ENCRYPT:
+		if (prep_buffer(struct sev_data_dbg, dst_addr, len, true,
+				&sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_ATTESTATION_REPORT:
+		if (prep_buffer(struct sev_data_attestation_report, address, len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_SEND_START:
+		if (prep_buffer(struct sev_data_send_start, session_address,
+				session_len, false, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_SEND_UPDATE_DATA:
+		if (prep_buffer(struct sev_data_send_update_data, hdr_address, hdr_len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		if (prep_buffer(struct sev_data_send_update_data, trans_address,
+				trans_len, false, &sev->snp_host_map[1]))
+			goto err;
+		break;
+	case SEV_CMD_SEND_UPDATE_VMSA:
+		if (prep_buffer(struct sev_data_send_update_vmsa, hdr_address, hdr_len,
+				false, &sev->snp_host_map[0]))
+			goto err;
+		if (prep_buffer(struct sev_data_send_update_vmsa, trans_address,
+				trans_len, false, &sev->snp_host_map[1]))
+			goto err;
+		break;
+	case SEV_CMD_RECEIVE_UPDATE_DATA:
+		if (prep_buffer(struct sev_data_receive_update_data, guest_address,
+				guest_len, true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	case SEV_CMD_RECEIVE_UPDATE_VMSA:
+		if (prep_buffer(struct sev_data_receive_update_vmsa, guest_address,
+				guest_len, true, &sev->snp_host_map[0]))
+			goto err;
+		break;
+	default:
+		break;
+	}
+
+	/* The command buffer need to be in the firmware state. */
+	if (to_fw && sev_legacy_cmd_buf_writable(cmd)) {
+		if (snp_set_rmp_state(__pa(cmd_buf), 1, true, true, false))
+			return -EFAULT;
+	}
+
+	return 0;
+
+err:
+	return -EINVAL;
+}
+
+static inline bool need_firmware_copy(int cmd)
+{
+	struct sev_device *sev = psp_master->sev_data;
+
+	/* After SNP is INIT'ed, the behavior of legacy SEV command is changed. */
+	return ((cmd < SEV_CMD_SNP_INIT) && sev->snp_inited) ? true : false;
+}
+
+static int snp_aware_copy_to_firmware(int cmd, void *data)
+{
+	return __snp_cmd_buf_copy(cmd, data, true, 0);
+}
+
+static int snp_aware_copy_from_firmware(int cmd, void *data, int fw_err)
+{
+	return __snp_cmd_buf_copy(cmd, data, false, fw_err);
+}
+
 static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
 {
 	struct psp_device *psp = psp_master;
 	struct sev_device *sev;
 	unsigned int phys_lsb, phys_msb;
 	unsigned int reg, ret = 0;
+	void *cmd_buf;
 	int buf_len;
 
 	if (!psp || !psp->sev_data)
@@ -466,12 +749,28 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
 	 * work for some memory, e.g. vmalloc'd addresses, and @data may not be
 	 * physically contiguous.
 	 */
-	if (data)
-		memcpy(sev->cmd_buf, data, buf_len);
+	if (data) {
+		if (sev->cmd_buf_active > 2)
+			return -EBUSY;
+
+		cmd_buf = sev->cmd_buf_active ? sev->cmd_buf_backup : sev->cmd_buf;
+
+		memcpy(cmd_buf, data, buf_len);
+		sev->cmd_buf_active++;
+
+		/*
+		 * The behavior of the SEV-legacy commands is altered when the
+		 * SNP firmware is in the INIT state.
+		 */
+		if (need_firmware_copy(cmd) && snp_aware_copy_to_firmware(cmd, sev->cmd_buf))
+			return -EFAULT;
+	} else {
+		cmd_buf = sev->cmd_buf;
+	}
 
 	/* Get the physical address of the command buffer */
-	phys_lsb = data ? lower_32_bits(__psp_pa(sev->cmd_buf)) : 0;
-	phys_msb = data ? upper_32_bits(__psp_pa(sev->cmd_buf)) : 0;
+	phys_lsb = data ? lower_32_bits(__psp_pa(cmd_buf)) : 0;
+	phys_msb = data ? upper_32_bits(__psp_pa(cmd_buf)) : 0;
 
 	dev_dbg(sev->dev, "sev command id %#x buffer 0x%08x%08x timeout %us\n",
 		cmd, phys_msb, phys_lsb, psp_timeout);
@@ -514,15 +813,24 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
 		sev_write_init_ex_file_if_required(cmd);
 	}
 
-	print_hex_dump_debug("(out): ", DUMP_PREFIX_OFFSET, 16, 2, data,
-			     buf_len, false);
-
 	/*
 	 * Copy potential output from the PSP back to data.  Do this even on
 	 * failure in case the caller wants to glean something from the error.
 	 */
-	if (data)
-		memcpy(data, sev->cmd_buf, buf_len);
+	if (data) {
+		/*
+		 * Restore the page state after the command completes.
+		 */
+		if (need_firmware_copy(cmd) &&
+		    snp_aware_copy_from_firmware(cmd, cmd_buf, ret))
+			return -EFAULT;
+
+		memcpy(data, cmd_buf, buf_len);
+		sev->cmd_buf_active--;
+	}
+
+	print_hex_dump_debug("(out): ", DUMP_PREFIX_OFFSET, 16, 2, data,
+			     buf_len, false);
 
 	return ret;
 }
@@ -1451,10 +1759,12 @@ int sev_dev_init(struct psp_device *psp)
 	if (!sev)
 		goto e_err;
 
-	sev->cmd_buf = (void *)devm_get_free_pages(dev, GFP_KERNEL, 0);
+	sev->cmd_buf = (void *)devm_get_free_pages(dev, GFP_KERNEL, 1);
 	if (!sev->cmd_buf)
 		goto e_sev;
 
+	sev->cmd_buf_backup = (uint8_t *)sev->cmd_buf + PAGE_SIZE;
+
 	psp->sev_data = sev;
 
 	sev->dev = dev;
@@ -1513,6 +1823,12 @@ static void sev_firmware_shutdown(struct sev_device *sev)
 		sev_init_ex_buffer = NULL;
 	}
 
+	/*
+	 * The host map need to clear the immutable bit so it must be free'd before the
+	 * SNP firmware shutdown.
+	 */
+	free_snp_host_map(sev);
+
 	sev_snp_shutdown(NULL);
 }
 
@@ -1588,6 +1904,14 @@ void sev_pci_init(void)
 				dev_err(sev->dev, "SEV-SNP: failed to INIT error %#x\n", error);
 			}
 		}
+
+		/*
+		 * Allocate the intermediate buffers used for the legacy command handling.
+		 */
+		if (alloc_snp_host_map(sev)) {
+			dev_notice(sev->dev, "Failed to alloc host map (disabling legacy SEV)\n");
+			goto skip_legacy;
+		}
 	}
 
 	/* Obtain the TMR memory area for SEV-ES use */
@@ -1605,12 +1929,14 @@ void sev_pci_init(void)
 		dev_err(sev->dev, "SEV: failed to INIT error %#x, rc %d\n",
 			error, rc);
 
+skip_legacy:
 	dev_info(sev->dev, "SEV%s API:%d.%d build:%d\n", sev->snp_inited ?
 		"-SNP" : "", sev->api_major, sev->api_minor, sev->build);
 
 	return;
 
 err:
+	free_snp_host_map(sev);
 	psp_master->sev_data = NULL;
 }
 
drivers/crypto/ccp/sev-dev.h +12 -0
index 186ad20cbd24..fe5d7a3ebace 100644
@@ -29,11 +29,20 @@
 #define SEV_CMDRESP_CMD_SHIFT		16
 #define SEV_CMDRESP_IOC			BIT(0)
 
+#define MAX_SNP_HOST_MAP_BUFS		2
+
 struct sev_misc_dev {
 	struct kref refcount;
 	struct miscdevice misc;
 };
 
+struct snp_host_map {
+	u64 paddr;
+	u32 len;
+	void *host;
+	bool active;
+};
+
 struct sev_device {
 	struct device *dev;
 	struct psp_device *psp;
@@ -52,8 +61,11 @@ struct sev_device {
 	u8 build;
 
 	void *cmd_buf;
+	void *cmd_buf_backup;
+	int cmd_buf_active;
 
 	bool snp_inited;
+	struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
 };
 
 int sev_dev_init(struct psp_device *psp);
Signature · 1 line
2.25.1
author maintainer
#199 lore raw

PATCH v6 16/49 crypto: ccp: Add the SNP_PLATFORM_STATUS command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The command can be used by the userspace to query the SNP platform status report. See the SEV-SNP spec for more details.

---
 Documentation/virt/coco/sevguest.rst | 27 +++++++++++++++++
 drivers/crypto/ccp/sev-dev.c         | 45 ++++++++++++++++++++++++++++
 include/uapi/linux/psp-sev.h         |  1 +
 3 files changed, 73 insertions(+)
Documentation/virt/coco/sevguest.rst +27 -0
index bf593e88cfd9..11ea67c944df 100644
@@ -61,6 +61,22 @@ counter (e.g. counter overflow), then -EIO will be returned.
                 __u64 fw_err;
         };
 
+The host ioctl should be called to /dev/sev device. The ioctl accepts command
+id and command input structure.
+
+::
+        struct sev_issue_cmd {
+                /* Command ID */
+                __u32 cmd;
+
+                /* Command request structure */
+                __u64 data;
+
+                /* firmware error code on failure (see psp-sev.h) */
+                __u32 error;
+        };
+
+
 2.1 SNP_GET_REPORT
 ------------------
 
@@ -118,6 +134,17 @@ be updated with the expected value.
 
 See GHCB specification for further detail on how to parse the certificate blob.
 
+2.4 SNP_PLATFORM_STATUS
+-----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_platform_status
+:Returns (out): 0 on success, -negative on error
+
+The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The
+status includes API major, minor version and more. See the SEV-SNP
+specification for further details.
+
 3. SEV-SNP CPUID Enforcement
 ============================
 
drivers/crypto/ccp/sev-dev.c +45 -0

57 lines not on this page · raw message

include/uapi/linux/psp-sev.h +1 -0
index bed65a891223..ffd60e8b0a31 100644
@@ -28,6 +28,7 @@ enum {
 	SEV_PEK_CERT_IMPORT,
 	SEV_GET_ID,	/* This command is deprecated, use SEV_GET_ID2 */
 	SEV_GET_ID2,
+	SNP_PLATFORM_STATUS,
 
 	SEV_MAX,
 };
Signature · 1 line
2.25.1
author maintainer
#200 lore raw

PATCH v6 17/49 crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP firmware provides the SNP_CONFIG command used to set the system-wide configuration value for SNP guests. The information includes the TCB version string to be reported in guest attestation reports.

Version 2 of the GHCB specification adds an NAE (SNP extended guest request) that a guest can use to query the reports that include additional certificates.

In both cases, userspace provided additional data is included in the attestation reports. The userspace will use the SNP_SET_EXT_CONFIG command to give the certificate blob and the reported TCB version string at once. Note that the specification defines certificate blob with a specific GUID format; the userspace is responsible for building the proper certificate blob. The ioctl treats it an opaque blob.

While it is not defined in the spec, but let's add SNP_GET_EXT_CONFIG command that can be used to obtain the data programmed through the SNP_SET_EXT_CONFIG.

---
 Documentation/virt/coco/sevguest.rst |  27 +++++++
 drivers/crypto/ccp/sev-dev.c         | 115 +++++++++++++++++++++++++++
 drivers/crypto/ccp/sev-dev.h         |   3 +
 include/uapi/linux/psp-sev.h         |  17 ++++
 4 files changed, 162 insertions(+)
Documentation/virt/coco/sevguest.rst +27 -0
index 11ea67c944df..3014de47e4ce 100644
@@ -145,6 +145,33 @@ The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The
 status includes API major, minor version and more. See the SEV-SNP
 specification for further details.
 
+2.5 SNP_SET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to set the system-wide configuration such as
+reported TCB version in the attestation report. The command is similar to
+SNP_CONFIG command defined in the SEV-SNP spec. The main difference is the
+command also accepts an additional certificate blob defined in the GHCB
+specification.
+
+If the certs_address is zero, then previous certificate blob will deleted.
+For more information on the certificate blob layout, see the GHCB spec
+(extended guest request message).
+
+2.6 SNP_GET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to query the system-wide configuration set
+through the SNP_SET_EXT_CONFIG.
+
 3. SEV-SNP CPUID Enforcement
 ============================
 
drivers/crypto/ccp/sev-dev.c +115 -0

133 lines not on this page · raw message

drivers/crypto/ccp/sev-dev.h +3 -0
index fe5d7a3ebace..d2fe1706311a 100644
@@ -66,6 +66,9 @@ struct sev_device {
 
 	bool snp_inited;
 	struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
+	void *snp_certs_data;
+	u32 snp_certs_len;
+	struct sev_user_data_snp_config snp_config;
 };
 
 int sev_dev_init(struct psp_device *psp);
include/uapi/linux/psp-sev.h +17 -0
index ffd60e8b0a31..60e7a8d1a18e 100644
@@ -29,6 +29,8 @@ enum {
 	SEV_GET_ID,	/* This command is deprecated, use SEV_GET_ID2 */
 	SEV_GET_ID2,
 	SNP_PLATFORM_STATUS,
+	SNP_SET_EXT_CONFIG,
+	SNP_GET_EXT_CONFIG,
 
 	SEV_MAX,
 };
@@ -190,6 +192,21 @@ struct sev_user_data_snp_config {
 	__u8 rsvd[52];
 } __packed;
 
+/**
+ * struct sev_data_snp_ext_config - system wide configuration value for SNP.
+ *
+ * @config_address: address of the struct sev_user_data_snp_config or 0 when
+ *		reported_tcb does not need to be updated.
+ * @certs_address: address of extended guest request certificate chain or
+ *              0 when previous certificate should be removed on SNP_SET_EXT_CONFIG.
+ * @certs_len: length of the certs
+ */
+struct sev_user_data_ext_snp_config {
+	__u64 config_address;		/* In */
+	__u64 certs_address;		/* In */
+	__u32 certs_len;		/* In */
+};
+
 /**
  * struct sev_issue_cmd - SEV ioctl parameters
  *
Signature · 1 line
2.25.1
#201 lore raw

Re: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

Quoted 42 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP firmware provides the SNP_CONFIG command used to set the system-wide configuration value for SNP guests. The information includes the TCB version string to be reported in guest attestation reports.

Version 2 of the GHCB specification adds an NAE (SNP extended guest request) that a guest can use to query the reports that include additional certificates.

In both cases, userspace provided additional data is included in the attestation reports. The userspace will use the SNP_SET_EXT_CONFIG command to give the certificate blob and the reported TCB version string at once. Note that the specification defines certificate blob with a specific GUID format; the userspace is responsible for building the proper certificate blob. The ioctl treats it an opaque blob.

While it is not defined in the spec, but let's add SNP_GET_EXT_CONFIG command that can be used to obtain the data programmed through the SNP_SET_EXT_CONFIG.

---
 Documentation/virt/coco/sevguest.rst |  27 +++++++
 drivers/crypto/ccp/sev-dev.c         | 115 +++++++++++++++++++++++++++
 drivers/crypto/ccp/sev-dev.h         |   3 +
 include/uapi/linux/psp-sev.h         |  17 ++++
 4 files changed, 162 insertions(+)
Documentation/virt/coco/sevguest.rst +4 -0
index 11ea67c944df..3014de47e4ce 100644
@@ -145,6 +145,33 @@ The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The
 status includes API major, minor version and more. See the SEV-SNP
 specification for further details.
 
+2.5 SNP_SET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd

Documentation/virt/coco/sevguest.rst The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The · line 161

Quoted 10 lines
Documentation/virt/coco/sevguest.rst +10 -0
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to set the system-wide configuration such as
+reported TCB version in the attestation report. The command is similar to
+SNP_CONFIG command defined in the SEV-SNP spec. The main difference is the
+command also accepts an additional certificate blob defined in the GHCB
+specification.
+
+If the certs_address is zero, then previous certificate blob will deleted.

... then the previous certificate blob will be deleted.

Quoted 32 lines
Documentation/virt/coco/sevguest.rst +13 -0
+For more information on the certificate blob layout, see the GHCB spec
+(extended guest request message).
+
+2.6 SNP_GET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to query the system-wide configuration set
+through the SNP_SET_EXT_CONFIG.
+
 3. SEV-SNP CPUID Enforcement
 ============================
 
drivers/crypto/ccp/sev-dev.c +4 -0
index b9b6fab31a82..97b479d5aa86 100644
@@ -1312,6 +1312,10 @@ static int __sev_snp_shutdown_locked(int *error)
        if (!sev->snp_inited)
                return 0;
 
+       /* Free the memory used for caching the certificate data */
+       kfree(sev->snp_certs_data);
+       sev->snp_certs_data = NULL;
+
        /* SHUTDOWN requires the DF_FLUSH */
        wbinvd_on_all_cpus();
        __sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, NULL);

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1626

Quoted 7 lines
drivers/crypto/ccp/sev-dev.c +4 -0
@@ -1616,6 +1620,111 @@ static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp)
        return ret;
 }
 
+static int sev_ioctl_snp_get_config(struct sev_issue_cmd *argp)
+{
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;

Lets memset |input| to zero to avoid leaking kernel memory, see "crypto: ccp - Use kzalloc for sev ioctl interfaces to prevent kernel memory leak"

Quoted 52 lines
drivers/crypto/ccp/sev-dev.c +52 -0
+       int ret;
+
+       if (!sev->snp_inited || !argp->data)
+               return -EINVAL;
+
+       if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+               return -EFAULT;
+
+       /* Copy the TCB version programmed through the SET_CONFIG to userspace */
+       if (input.config_address) {
+               if (copy_to_user((void * __user)input.config_address,
+                                &sev->snp_config, sizeof(struct sev_user_data_snp_config)))
+                       return -EFAULT;
+       }
+
+       /* Copy the extended certs programmed through the SNP_SET_CONFIG */
+       if (input.certs_address && sev->snp_certs_data) {
+               if (input.certs_len < sev->snp_certs_len) {
+                       /* Return the certs length to userspace */
+                       input.certs_len = sev->snp_certs_len;
+
+                       ret = -ENOSR;
+                       goto e_done;
+               }
+
+               if (copy_to_user((void * __user)input.certs_address,
+                                sev->snp_certs_data, sev->snp_certs_len))
+                       return -EFAULT;
+       }
+
+       ret = 0;
+
+e_done:
+       if (copy_to_user((void __user *)argp->data, &input, sizeof(input)))
+               ret = -EFAULT;
+
+       return ret;
+}
+
+static int sev_ioctl_snp_set_config(struct sev_issue_cmd *argp, bool writable)
+{
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;
+       struct sev_user_data_snp_config config;
+       void *certs = NULL;
+       int ret = 0;
+
+       if (!sev->snp_inited || !argp->data)
+               return -EINVAL;
+
+       if (!writable)
+               return -EPERM;

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1688

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+
+       if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+               return -EFAULT;
+
+       /* Copy the certs from userspace */
+       if (input.certs_address) {
+               if (!input.certs_len || !IS_ALIGNED(input.certs_len, PAGE_SIZE))
+                       return -EINVAL;
+
+               certs = psp_copy_user_blob(input.certs_address, input.certs_len);

I see that psp_copy_user_blob() uses memdup_user() which tracks the allocated memory to GFP_USER. Given this memory is long lived and now belongs to the PSP driver in perpetuity, should this be tracked with GFP_KERNEL?

Quoted 21 lines
drivers/crypto/ccp/sev-dev.c +21 -0
+               if (IS_ERR(certs))
+                       return PTR_ERR(certs);
+       }
+
+       /* Issue the PSP command to update the TCB version using the SNP_CONFIG. */
+       if (input.config_address) {
+               if (copy_from_user(&config,
+                                  (void __user *)input.config_address, sizeof(config))) {
+                       ret = -EFAULT;
+                       goto e_free;
+               }
+
+               ret = __sev_do_cmd_locked(SEV_CMD_SNP_CONFIG, &config, &argp->error);
+               if (ret)
+                       goto e_free;
+
+               memcpy(&sev->snp_config, &config, sizeof(config));
+       }
+
+       /*
+        * If the new certs are passed then cache it else free the old certs.

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1719

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+        */
+       if (certs) {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = certs;
+               sev->snp_certs_len = input.certs_len;
+       } else {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = NULL;
+               sev->snp_certs_len = 0;
+       }

Do we need another lock here? When I look at 18/49 it seems like snp_guest_ext_guest_request() it seems like we have a race for |sev->snp_certs_data|

Quoted 29 lines
drivers/crypto/ccp/sev-dev.c +14 -0
+
+       return 0;
+
+e_free:
+       kfree(certs);
+       return ret;
+}
+
 static long sev_ioctl(struct file *file, unsigned int ioctl, unsigned long arg)
 {
        void __user *argp = (void __user *)arg;
@@ -1670,6 +1779,12 @@ static long sev_ioctl(struct file *file, unsigned int ioctl, unsigned long arg)
        case SNP_PLATFORM_STATUS:
                ret = sev_ioctl_snp_platform_status(&input);
                break;
+       case SNP_SET_EXT_CONFIG:
+               ret = sev_ioctl_snp_set_config(&input, writable);
+               break;
+       case SNP_GET_EXT_CONFIG:
+               ret = sev_ioctl_snp_get_config(&input);
+               break;
        default:
                ret = -EINVAL;
                goto out;
drivers/crypto/ccp/sev-dev.h +0 -0
index fe5d7a3ebace..d2fe1706311a 100644

drivers/crypto/ccp/sev-dev.h struct sev_device { · line 71

Quoted 6 lines
drivers/crypto/ccp/sev-dev.h +3 -0
@@ -66,6 +66,9 @@ struct sev_device {
 
        bool snp_inited;
        struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
+       void *snp_certs_data;
+       u32 snp_certs_len;
+       struct sev_user_data_snp_config snp_config;

Since this gets copy_to_user'd can we memset this to 0 to prevent leaking kernel uninitialized memory? Similar to recent patches with kzalloc and __GPF_ZERO usage.

Quoted 41 lines
drivers/crypto/ccp/sev-dev.h +0 -0
 };
 
 int sev_dev_init(struct psp_device *psp);
include/uapi/linux/psp-sev.h +17 -1
index ffd60e8b0a31..60e7a8d1a18e 100644
@@ -29,6 +29,8 @@ enum {
        SEV_GET_ID,     /* This command is deprecated, use SEV_GET_ID2 */
        SEV_GET_ID2,
        SNP_PLATFORM_STATUS,
+       SNP_SET_EXT_CONFIG,
+       SNP_GET_EXT_CONFIG,
 
        SEV_MAX,
 };
@@ -190,6 +192,21 @@ struct sev_user_data_snp_config {
        __u8 rsvd[52];
 } __packed;
 
+/**
+ * struct sev_data_snp_ext_config - system wide configuration value for SNP.
+ *
+ * @config_address: address of the struct sev_user_data_snp_config or 0 when
+ *             reported_tcb does not need to be updated.
+ * @certs_address: address of extended guest request certificate chain or
+ *              0 when previous certificate should be removed on SNP_SET_EXT_CONFIG.
+ * @certs_len: length of the certs
+ */
+struct sev_user_data_ext_snp_config {
+       __u64 config_address;           /* In */
+       __u64 certs_address;            /* In */
+       __u32 certs_len;                /* In */
+};
+
 /**
  * struct sev_issue_cmd - SEV ioctl parameters
  *
--

2.25.1

author maintainer
#202 lore raw

RE: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

[AMD Official Use Only - General]

Hello Peter,

Quoted 6 lines

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1626

Quoted 3 lines
drivers/crypto/ccp/sev-dev.c +3 -0
+static int sev_ioctl_snp_get_config(struct sev_issue_cmd *argp) {
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;

Lets memset |input| to zero to avoid leaking kernel memory, see "crypto: ccp - Use kzalloc for sev ioctl interfaces to prevent kernel memory leak"

Yes.

Quoted 18 lines
Quoted 18 lines
drivers/crypto/ccp/sev-dev.c +18 -0
+static int sev_ioctl_snp_set_config(struct sev_issue_cmd *argp, bool 
+writable) {
+       struct sev_device *sev = psp_master->sev_data;
+       struct sev_user_data_ext_snp_config input;
+       struct sev_user_data_snp_config config;
+       void *certs = NULL;
+       int ret = 0;
+
+       if (!sev->snp_inited || !argp->data)
+               return -EINVAL;
+
+       if (!writable)
+               return -EPERM;
+
+       if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+               return -EFAULT;
+
+       /* Copy the certs from userspace */
Quoted 8 lines

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp)

Quoted 6 lines
drivers/crypto/ccp/sev-dev.c +6 -0
+       if (input.certs_address) {
+               if (!input.certs_len || !IS_ALIGNED(input.certs_len, PAGE_SIZE))
+                       return -EINVAL;
+
+               certs = psp_copy_user_blob(input.certs_address, 
+ input.certs_len);

I see that psp_copy_user_blob() uses memdup_user() which tracks the allocated memory to GFP_USER. Given this memory is long lived and now belongs to the PSP driver in perpetuity, should this be tracked with GFP_KERNEL?

But we need to copy from user space address, so what is the alternative here ?

Quoted 13 lines

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1719

Quoted 12 lines
drivers/crypto/ccp/sev-dev.c +12 -0
+       /*
+        * If the new certs are passed then cache it else free the old certs.
+        */
+       if (certs) {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = certs;
+               sev->snp_certs_len = input.certs_len;
+       } else {
+               kfree(sev->snp_certs_data);
+               sev->snp_certs_data = NULL;
+               sev->snp_certs_len = 0;
+       }
Quoted 3 lines

Do we need another lock here? When I look at 18/49 it seems like snp_guest_ext_guest_request() it seems like we have a race for |sev->snp_certs_data|

The certificate blob in snp_guest_ext_guest_request() will depend on the certificate blob provided here by SNP_SET_EXT_CONFIG. There might be a potential race with the SNP extended guest request NAE, let me have a look at it.

Quoted 7 lines

drivers/crypto/ccp/sev-dev.h struct sev_device { · line 71

Quoted 5 lines
        bool snp_inited;
        struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
drivers/crypto/ccp/sev-dev.h +3 -0
+       void *snp_certs_data;
+       u32 snp_certs_len;
+       struct sev_user_data_snp_config snp_config;

Since this gets copy_to_user'd can we memset this to 0 to prevent leaking kernel uninitialized memory? Similar to recent patches with kzalloc and __GPF_ZERO usage.

Yes.

Thanks,
Ashish

#203 lore raw

Re: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

In reply to Ashish Kalra
Quoted 9 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP firmware provides the SNP_CONFIG command used to set the system-wide configuration value for SNP guests. The information includes the TCB version string to be reported in guest attestation reports.

Version 2 of the GHCB specification adds an NAE (SNP extended guest request) that a guest can use to query the reports that include additional certificates.

Neither in the commit message nor in the documentation is GHCB open coded.

Quoted 140 lines

In both cases, userspace provided additional data is included in the attestation reports. The userspace will use the SNP_SET_EXT_CONFIG command to give the certificate blob and the reported TCB version string at once. Note that the specification defines certificate blob with a specific GUID format; the userspace is responsible for building the proper certificate blob. The ioctl treats it an opaque blob.

While it is not defined in the spec, but let's add SNP_GET_EXT_CONFIG command that can be used to obtain the data programmed through the SNP_SET_EXT_CONFIG.

---
 Documentation/virt/coco/sevguest.rst |  27 +++++++
 drivers/crypto/ccp/sev-dev.c         | 115 +++++++++++++++++++++++++++
 drivers/crypto/ccp/sev-dev.h         |   3 +
 include/uapi/linux/psp-sev.h         |  17 ++++
 4 files changed, 162 insertions(+)
Documentation/virt/coco/sevguest.rst +27 -0
index 11ea67c944df..3014de47e4ce 100644
@@ -145,6 +145,33 @@ The SNP_PLATFORM_STATUS command is used to query the SNP platform status. The
 status includes API major, minor version and more. See the SEV-SNP
 specification for further details.
 
+2.5 SNP_SET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to set the system-wide configuration such as
+reported TCB version in the attestation report. The command is similar to
+SNP_CONFIG command defined in the SEV-SNP spec. The main difference is the
+command also accepts an additional certificate blob defined in the GHCB
+specification.
+
+If the certs_address is zero, then previous certificate blob will deleted.
+For more information on the certificate blob layout, see the GHCB spec
+(extended guest request message).
+
+2.6 SNP_GET_EXT_CONFIG
+----------------------
+:Technology: sev-snp
+:Type: hypervisor ioctl cmd
+:Parameters (in): struct sev_data_snp_ext_config
+:Returns (out): 0 on success, -negative on error
+
+The SNP_SET_EXT_CONFIG is used to query the system-wide configuration set
+through the SNP_SET_EXT_CONFIG.
+
 3. SEV-SNP CPUID Enforcement
 ============================
 
drivers/crypto/ccp/sev-dev.c +68 -0
index b9b6fab31a82..97b479d5aa86 100644
@@ -1312,6 +1312,10 @@ static int __sev_snp_shutdown_locked(int *error)
 	if (!sev->snp_inited)
 		return 0;
 
+	/* Free the memory used for caching the certificate data */
+	kfree(sev->snp_certs_data);
+	sev->snp_certs_data = NULL;
+
 	/* SHUTDOWN requires the DF_FLUSH */
 	wbinvd_on_all_cpus();
 	__sev_do_cmd_locked(SEV_CMD_SNP_DF_FLUSH, NULL, NULL);
@@ -1616,6 +1620,111 @@ static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp)
 	return ret;
 }
 
+static int sev_ioctl_snp_get_config(struct sev_issue_cmd *argp)
+{
+	struct sev_device *sev = psp_master->sev_data;
+	struct sev_user_data_ext_snp_config input;
+	int ret;
+
+	if (!sev->snp_inited || !argp->data)
+		return -EINVAL;
+
+	if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+		return -EFAULT;
+
+	/* Copy the TCB version programmed through the SET_CONFIG to userspace */
+	if (input.config_address) {
+		if (copy_to_user((void * __user)input.config_address,
+				 &sev->snp_config, sizeof(struct sev_user_data_snp_config)))
+			return -EFAULT;
+	}
+
+	/* Copy the extended certs programmed through the SNP_SET_CONFIG */
+	if (input.certs_address && sev->snp_certs_data) {
+		if (input.certs_len < sev->snp_certs_len) {
+			/* Return the certs length to userspace */
+			input.certs_len = sev->snp_certs_len;
+
+			ret = -ENOSR;
+			goto e_done;
+		}
+
+		if (copy_to_user((void * __user)input.certs_address,
+				 sev->snp_certs_data, sev->snp_certs_len))
+			return -EFAULT;
+	}
+
+	ret = 0;
+
+e_done:
+	if (copy_to_user((void __user *)argp->data, &input, sizeof(input)))
+		ret = -EFAULT;
+
+	return ret;
+}
+
+static int sev_ioctl_snp_set_config(struct sev_issue_cmd *argp, bool writable)
+{
+	struct sev_device *sev = psp_master->sev_data;
+	struct sev_user_data_ext_snp_config input;
+	struct sev_user_data_snp_config config;
+	void *certs = NULL;
+	int ret = 0;
+
+	if (!sev->snp_inited || !argp->data)
+		return -EINVAL;
+
+	if (!writable)
+		return -EPERM;
+
+	if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+		return -EFAULT;
+
+	/* Copy the certs from userspace */
+	if (input.certs_address) {
+		if (!input.certs_len || !IS_ALIGNED(input.certs_len, PAGE_SIZE))
+			return -EINVAL;

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1696

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+
+		certs = psp_copy_user_blob(input.certs_address, input.certs_len);
+		if (IS_ERR(certs))
+			return PTR_ERR(certs);
+	}
+
+	/* Issue the PSP command to update the TCB version using the SNP_CONFIG. */
+	if (input.config_address) {
+		if (copy_from_user(&config,
+				   (void __user *)input.config_address, sizeof(config))) {

You can put this into a single line, and it's still less than 100 characters:

                if (copy_from_user(&config, (void __user *)input.config_address, sizeof(config))) {
Quoted 4 lines
drivers/crypto/ccp/sev-dev.c +4 -0
+			ret = -EFAULT;
+			goto e_free;
+		}
+

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1710

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+		ret = __sev_do_cmd_locked(SEV_CMD_SNP_CONFIG, &config, &argp->error);
+		if (ret)
+			goto e_free;
+
+		memcpy(&sev->snp_config, &config, sizeof(config));
+	}
+
+	/*
+	 * If the new certs are passed then cache it else free the old certs.
+	 */
        kfree(sev->snp_certs_data);

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1712

Quoted 2 lines
drivers/crypto/ccp/sev-dev.c +2 -0
+	if (certs) {
+		kfree(sev->snp_certs_data);

Remove kfree().

drivers/crypto/ccp/sev-dev.c static int sev_ioctl_snp_platform_status(struct sev_issue_cmd *argp) · line 1716

Quoted 4 lines
drivers/crypto/ccp/sev-dev.c +4 -0
+		sev->snp_certs_data = certs;
+		sev->snp_certs_len = input.certs_len;
+	} else {
+		kfree(sev->snp_certs_data);

Remove kfree().

Quoted 79 lines
drivers/crypto/ccp/sev-dev.c +17 -0
+		sev->snp_certs_data = NULL;
+		sev->snp_certs_len = 0;
+	}
+
+	return 0;
+
+e_free:
+	kfree(certs);
+	return ret;
+}
+
 static long sev_ioctl(struct file *file, unsigned int ioctl, unsigned long arg)
 {
 	void __user *argp = (void __user *)arg;
@@ -1670,6 +1779,12 @@ static long sev_ioctl(struct file *file, unsigned int ioctl, unsigned long arg)
 	case SNP_PLATFORM_STATUS:
 		ret = sev_ioctl_snp_platform_status(&input);
 		break;
+	case SNP_SET_EXT_CONFIG:
+		ret = sev_ioctl_snp_set_config(&input, writable);
+		break;
+	case SNP_GET_EXT_CONFIG:
+		ret = sev_ioctl_snp_get_config(&input);
+		break;
 	default:
 		ret = -EINVAL;
 		goto out;
drivers/crypto/ccp/sev-dev.h +3 -0
index fe5d7a3ebace..d2fe1706311a 100644
@@ -66,6 +66,9 @@ struct sev_device {
 
 	bool snp_inited;
 	struct snp_host_map snp_host_map[MAX_SNP_HOST_MAP_BUFS];
+	void *snp_certs_data;
+	u32 snp_certs_len;
+	struct sev_user_data_snp_config snp_config;
 };
 
 int sev_dev_init(struct psp_device *psp);
include/uapi/linux/psp-sev.h +17 -0
index ffd60e8b0a31..60e7a8d1a18e 100644
@@ -29,6 +29,8 @@ enum {
 	SEV_GET_ID,	/* This command is deprecated, use SEV_GET_ID2 */
 	SEV_GET_ID2,
 	SNP_PLATFORM_STATUS,
+	SNP_SET_EXT_CONFIG,
+	SNP_GET_EXT_CONFIG,
 
 	SEV_MAX,
 };
@@ -190,6 +192,21 @@ struct sev_user_data_snp_config {
 	__u8 rsvd[52];
 } __packed;
 
+/**
+ * struct sev_data_snp_ext_config - system wide configuration value for SNP.
+ *
+ * @config_address: address of the struct sev_user_data_snp_config or 0 when
+ *		reported_tcb does not need to be updated.
+ * @certs_address: address of extended guest request certificate chain or
+ *              0 when previous certificate should be removed on SNP_SET_EXT_CONFIG.
+ * @certs_len: length of the certs
+ */
+struct sev_user_data_ext_snp_config {
+	__u64 config_address;		/* In */
+	__u64 certs_address;		/* In */
+	__u32 certs_len;		/* In */
+};
+
 /**
  * struct sev_issue_cmd - SEV ioctl parameters
  *
Signature · 1 line
2.25.1

BR, Jarkko

#204 lore raw

Re: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

To preface, I don't want to delay this patch set, only have the conversation at the most appropriate place.

Quoted 5 lines
Quoted 4 lines

The SEV-SNP firmware provides the SNP_CONFIG command used to set the system-wide configuration value for SNP guests. The information includes the TCB version string to be reported in guest attestation reports.

The system-wide aspect of this makes me wonder if we can also have a VM instance-specific extension. This is important for the use case that we may see secure boot variables included in the launch measurement, making offline signing of the UEFI image impossible. We can't sign the cross-product of all UEFI builds and every user's EFI variables. We'd like to include an instance-specific certificate that specifies the platform-endorsed golden measurement of the UEFI.

An alternative that doesn't require a change to the kernel is to just make this certificate fetchable from a FAMILY_ID-keyed, predetermined URL prefix + IMAGE_ID + '.crt', but this requires a download (and continuous hosting) to do something as routine as collecting an attestation report. It's up to the upstream community to determine if that is an acceptable cost to keep the complexity of a certificate table merge operation out of the kernel.

The SNP API specification gives an interpretation to the data blob here as a table of GUID/offset pairs followed by data blobs that presumably are at the appropriate offsets into the data pages. The spec allows for the host to add any number of GUID/offset pairs it wants, with 3 specific GUIDs recommended for the AMD PSP certificate chain.

The snp_guest_ext_guest_request function in ccp is what passes back the certificate data that was previously stored, so I'm wondering if it can take an extra (pointer,len) pair of VM instance certificate data to merge with the host certificate data before returning to the guest. The new required length is the sum total of both the header certs and instance certs. The operation to copy the data is no longer a memcpy but a header merge that tracks the offset shifts caused by a larger header and other certificates in the remaining data pages.

I can propose my own patch on top of this v6 patch set that adds a KVM ioctl like KVM_{GET,SET}_INSTANCE_SNP_EXT_CONFIG and then pass along the stored certificate blob in the request call. I'd prefer to have the design agreed upon upfront though.

Signature · 1 line
-Dionna Glaze, PhD (she/her)
maintainer
#205 lore raw

Re: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

Quoted 25 lines · Dionna Amalie Glaze

To preface, I don't want to delay this patch set, only have the conversation at the most appropriate place.

Quoted 5 lines
Quoted 4 lines

The SEV-SNP firmware provides the SNP_CONFIG command used to set the system-wide configuration value for SNP guests. The information includes the TCB version string to be reported in guest attestation reports.

The system-wide aspect of this makes me wonder if we can also have a VM instance-specific extension. This is important for the use case that we may see secure boot variables included in the launch measurement, making offline signing of the UEFI image impossible. We can't sign the cross-product of all UEFI builds and every user's EFI variables. We'd like to include an instance-specific certificate that specifies the platform-endorsed golden measurement of the UEFI.

An alternative that doesn't require a change to the kernel is to just make this certificate fetchable from a FAMILY_ID-keyed, predetermined URL prefix + IMAGE_ID + '.crt', but this requires a download (and continuous hosting) to do something as routine as collecting an attestation report. It's up to the upstream community to determine if that is an acceptable cost to keep the complexity of a certificate table merge operation out of the kernel.

Quoted 1 line

The SNP API specification gives an interpretation to the data blob

That's the GHCB specification, not the SNP API.

Quoted 15 lines

here as a table of GUID/offset pairs followed by data blobs that presumably are at the appropriate offsets into the data pages. The spec allows for the host to add any number of GUID/offset pairs it wants, with 3 specific GUIDs recommended for the AMD PSP certificate chain.

The snp_guest_ext_guest_request function in ccp is what passes back the certificate data that was previously stored, so I'm wondering if it can take an extra (pointer,len) pair of VM instance certificate data to merge with the host certificate data before returning to the guest. The new required length is the sum total of both the header certs and instance certs. The operation to copy the data is no longer a memcpy but a header merge that tracks the offset shifts caused by a larger header and other certificates in the remaining data pages.

Quoted 2 lines

I can propose my own patch on top of this v6 patch set that adds a KVM ioctl like KVM_{GET,SET}_INSTANCE_SNP_EXT_CONFIG and then pass along

Would it be burden to supply all the certificates, both system and per-VM, in this KVM call? On the SNP Extended Guest Request, the hypervisor could just check if there is a per-VM blob and return that or else return the system-wide blob (if present).

Thanks,
Tom

Quoted 3 lines

the stored certificate blob in the request call. I'd prefer to have the design agreed upon upfront though.

#206 lore raw

Re: [PATCH Part2 v6 17/49] crypto: ccp: Add the SNP_{SET,GET}_EXT_CONFIG command

Quoted 5 lines

Would it be burden to supply all the certificates, both system and per-VM, in this KVM call? On the SNP Extended Guest Request, the hypervisor could just check if there is a per-VM blob and return that or else return the system-wide blob (if present).

I think that's fine by me. We can use SNP_GET_EXT_CONFIG, merge in user space, and create an instance override with a KVM ioctl without touching ccp.

Signature · 1 line
-Dionna Glaze, PhD (she/her)
author maintainer
#207 lore raw

PATCH v6 18/49 crypto: ccp: Provide APIs to query extended attestation report

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of the GHCB specification defines VMGEXIT that is used to get the extended attestation report. The extended attestation report includes the certificate blobs provided through the SNP_SET_EXT_CONFIG.

The snp_guest_ext_guest_request() will be used by the hypervisor to get the extended attestation report. See the GHCB specification for more details.

---
 drivers/crypto/ccp/sev-dev.c | 43 ++++++++++++++++++++++++++++++++++++
 include/linux/psp-sev.h      | 24 ++++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/crypto/ccp/sev-dev.c +43 -0
index 97b479d5aa86..f6306b820b86 100644
@@ -25,6 +25,7 @@
 #include <linux/fs.h>
 
 #include <asm/smp.h>
+#include <asm/sev.h>
 
 #include "psp-dev.h"
 #include "sev-dev.h"
@@ -1857,6 +1858,48 @@ int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
 }
 EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
 
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+				unsigned long vaddr, unsigned long *npages, unsigned long *fw_err)
+{
+	unsigned long expected_npages;
+	struct sev_device *sev;
+	int rc;
+
+	if (!psp_master || !psp_master->sev_data)
+		return -ENODEV;
+
+	sev = psp_master->sev_data;
+
+	if (!sev->snp_inited)
+		return -EINVAL;
+
+	/*
+	 * Check if there is enough space to copy the certificate chain. Otherwise
+	 * return ERROR code defined in the GHCB specification.
+	 */
+	expected_npages = sev->snp_certs_len >> PAGE_SHIFT;
+	if (*npages < expected_npages) {
+		*npages = expected_npages;
+		*fw_err = SNP_GUEST_REQ_INVALID_LEN;
+		return -EINVAL;
+	}
+
+	rc = sev_do_cmd(SEV_CMD_SNP_GUEST_REQUEST, data, (int *)&fw_err);
+	if (rc)
+		return rc;
+
+	/* Copy the certificate blob */
+	if (sev->snp_certs_data) {
+		*npages = expected_npages;
+		memcpy((void *)vaddr, sev->snp_certs_data, *npages << PAGE_SHIFT);
+	} else {
+		*npages = 0;
+	}
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(snp_guest_ext_guest_request);
+
 static void sev_exit(struct kref *ref)
 {
 	misc_deregister(&misc_dev->misc);
include/linux/psp-sev.h +24 -0
index a3bb792bb842..cd37ccd1fa1f 100644
@@ -945,6 +945,23 @@ void *psp_copy_user_blob(u64 uaddr, u32 len);
 void *snp_alloc_firmware_page(gfp_t mask);
 void snp_free_firmware_page(void *addr);
 
+/**
+ * snp_guest_ext_guest_request - perform the SNP extended guest request command
+ *  defined in the GHCB specification.
+ *
+ * @data: the input guest request structure
+ * @vaddr: address where the certificate blob need to be copied.
+ * @npages: number of pages for the certificate blob.
+ *    If the specified page count is less than the certificate blob size, then the
+ *    required page count is returned with error code defined in the GHCB spec.
+ *    If the specified page count is more than the certificate blob size, then
+ *    page count is updated to reflect the amount of valid data copied in the
+ *    vaddr.
+ */
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+				unsigned long vaddr, unsigned long *npages,
+				unsigned long *error);
+
 #else	/* !CONFIG_CRYPTO_DEV_SP_PSP */
 
 static inline int
@@ -992,6 +1009,13 @@ static inline void *snp_alloc_firmware_page(gfp_t mask)
 
 static inline void snp_free_firmware_page(void *addr) { }
 
+static inline int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+					      unsigned long vaddr, unsigned long *n,
+					      unsigned long *error)
+{
+	return -ENODEV;
+}
+
 #endif	/* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif	/* __PSP_SEV_H__ */
Signature · 1 line
2.25.1
#208 lore raw

Re: [PATCH Part2 v6 18/49] crypto: ccp: Provide APIs to query extended attestation report

Quoted 50 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of the GHCB specification defines VMGEXIT that is used to get the extended attestation report. The extended attestation report includes the certificate blobs provided through the SNP_SET_EXT_CONFIG.

The snp_guest_ext_guest_request() will be used by the hypervisor to get the extended attestation report. See the GHCB specification for more details.

---
 drivers/crypto/ccp/sev-dev.c | 43 ++++++++++++++++++++++++++++++++++++
 include/linux/psp-sev.h      | 24 ++++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/crypto/ccp/sev-dev.c +18 -0
index 97b479d5aa86..f6306b820b86 100644
@@ -25,6 +25,7 @@
 #include <linux/fs.h>
 
 #include <asm/smp.h>
+#include <asm/sev.h>
 
 #include "psp-dev.h"
 #include "sev-dev.h"
@@ -1857,6 +1858,48 @@ int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
 }
 EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
 
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+                               unsigned long vaddr, unsigned long *npages, unsigned long *fw_err)
+{
+       unsigned long expected_npages;
+       struct sev_device *sev;
+       int rc;
+
+       if (!psp_master || !psp_master->sev_data)
+               return -ENODEV;
+
+       sev = psp_master->sev_data;
+
+       if (!sev->snp_inited)
+               return -EINVAL;
+
+       /*
+        * Check if there is enough space to copy the certificate chain. Otherwise

drivers/crypto/ccp/sev-dev.c int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error) · line 1887

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+        * return ERROR code defined in the GHCB specification.
+        */
+       expected_npages = sev->snp_certs_len >> PAGE_SHIFT;
+       if (*npages < expected_npages) {
+               *npages = expected_npages;
+               *fw_err = SNP_GUEST_REQ_INVALID_LEN;
+               return -EINVAL;
+       }
+
+       rc = sev_do_cmd(SEV_CMD_SNP_GUEST_REQUEST, data, (int *)&fw_err);

We can just pass |fw_error| here (with the cast) here right? Not need to do &fw_err.

      rc = sev_do_cmd(SEV_CMD_SNP_GUEST_REQUEST, data, (int *)fw_err);

drivers/crypto/ccp/sev-dev.c int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error) · line 1894

Quoted 7 lines
drivers/crypto/ccp/sev-dev.c +7 -0
+       if (rc)
+               return rc;
+
+       /* Copy the certificate blob */
+       if (sev->snp_certs_data) {
+               *npages = expected_npages;
+               memcpy((void *)vaddr, sev->snp_certs_data, *npages << PAGE_SHIFT);

Why don't we just make |vaddr| into a void* instead of an unsigned long?

Quoted 56 lines
drivers/crypto/ccp/sev-dev.c +8 -0
+       } else {
+               *npages = 0;
+       }
+
+       return rc;
+}
+EXPORT_SYMBOL_GPL(snp_guest_ext_guest_request);
+
 static void sev_exit(struct kref *ref)
 {
        misc_deregister(&misc_dev->misc);
include/linux/psp-sev.h +24 -1
index a3bb792bb842..cd37ccd1fa1f 100644
@@ -945,6 +945,23 @@ void *psp_copy_user_blob(u64 uaddr, u32 len);
 void *snp_alloc_firmware_page(gfp_t mask);
 void snp_free_firmware_page(void *addr);
 
+/**
+ * snp_guest_ext_guest_request - perform the SNP extended guest request command
+ *  defined in the GHCB specification.
+ *
+ * @data: the input guest request structure
+ * @vaddr: address where the certificate blob need to be copied.
+ * @npages: number of pages for the certificate blob.
+ *    If the specified page count is less than the certificate blob size, then the
+ *    required page count is returned with error code defined in the GHCB spec.
+ *    If the specified page count is more than the certificate blob size, then
+ *    page count is updated to reflect the amount of valid data copied in the
+ *    vaddr.
+ */
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+                               unsigned long vaddr, unsigned long *npages,
+                               unsigned long *error);
+
 #else  /* !CONFIG_CRYPTO_DEV_SP_PSP */
 
 static inline int
@@ -992,6 +1009,13 @@ static inline void *snp_alloc_firmware_page(gfp_t mask)
 
 static inline void snp_free_firmware_page(void *addr) { }
 
+static inline int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+                                             unsigned long vaddr, unsigned long *n,
+                                             unsigned long *error)
+{
+       return -ENODEV;
+}
+
 #endif /* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif /* __PSP_SEV_H__ */
--

2.25.1

#209 lore raw

Re: [PATCH Part2 v6 18/49] crypto: ccp: Provide APIs to query extended attestation report

In reply to Ashish Kalra

I'd rephrase "Provide in-kernel API..." (e.g. not uapi).

Quoted 9 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of the GHCB specification defines VMGEXIT that is used to get the extended attestation report. The extended attestation report includes the certificate blobs provided through the SNP_SET_EXT_CONFIG.

The snp_guest_ext_guest_request() will be used by the hypervisor to get the extended attestation report. See the GHCB specification for more details.

What is "the hypersivor"? Could it be replaced with e.g. KVM for clarity?

Quoted 54 lines
---
 drivers/crypto/ccp/sev-dev.c | 43 ++++++++++++++++++++++++++++++++++++
 include/linux/psp-sev.h      | 24 ++++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/crypto/ccp/sev-dev.c +32 -0
index 97b479d5aa86..f6306b820b86 100644
@@ -25,6 +25,7 @@
 #include <linux/fs.h>
 
 #include <asm/smp.h>
+#include <asm/sev.h>
 
 #include "psp-dev.h"
 #include "sev-dev.h"
@@ -1857,6 +1858,48 @@ int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error)
 }
 EXPORT_SYMBOL_GPL(snp_guest_dbg_decrypt);
 
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+				unsigned long vaddr, unsigned long *npages, unsigned long *fw_err)
+{
+	unsigned long expected_npages;
+	struct sev_device *sev;
+	int rc;
+
+	if (!psp_master || !psp_master->sev_data)
+		return -ENODEV;
+
+	sev = psp_master->sev_data;
+
+	if (!sev->snp_inited)
+		return -EINVAL;
+
+	/*
+	 * Check if there is enough space to copy the certificate chain. Otherwise
+	 * return ERROR code defined in the GHCB specification.
+	 */
+	expected_npages = sev->snp_certs_len >> PAGE_SHIFT;
+	if (*npages < expected_npages) {
+		*npages = expected_npages;
+		*fw_err = SNP_GUEST_REQ_INVALID_LEN;
+		return -EINVAL;
+	}
+
+	rc = sev_do_cmd(SEV_CMD_SNP_GUEST_REQUEST, data, (int *)&fw_err);
+	if (rc)
+		return rc;
+
+	/* Copy the certificate blob */

drivers/crypto/ccp/sev-dev.c int snp_guest_dbg_decrypt(struct sev_data_snp_dbg *data, int *error) · line 1901

Quoted 10 lines
drivers/crypto/ccp/sev-dev.c +10 -0
+	if (sev->snp_certs_data) {
+		*npages = expected_npages;
+		memcpy((void *)vaddr, sev->snp_certs_data, *npages << PAGE_SHIFT);
+	} else {
+		*npages = 0;
+	}
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(snp_guest_ext_guest_request);

Undocumented export.

Quoted 15 lines
drivers/crypto/ccp/sev-dev.c +1 -0
+
 static void sev_exit(struct kref *ref)
 {
 	misc_deregister(&misc_dev->misc);
include/linux/psp-sev.h +3 -0
index a3bb792bb842..cd37ccd1fa1f 100644
@@ -945,6 +945,23 @@ void *psp_copy_user_blob(u64 uaddr, u32 len);
 void *snp_alloc_firmware_page(gfp_t mask);
 void snp_free_firmware_page(void *addr);
 
+/**
+ * snp_guest_ext_guest_request - perform the SNP extended guest request command
+ *  defined in the GHCB specification.

include/linux/psp-sev.h void *psp_copy_user_blob(u64 uaddr, u32 len); · line 960

Quoted 10 lines
include/linux/psp-sev.h +10 -0
+ *
+ * @data: the input guest request structure
+ * @vaddr: address where the certificate blob need to be copied.
+ * @npages: number of pages for the certificate blob.
+ *    If the specified page count is less than the certificate blob size, then the
+ *    required page count is returned with error code defined in the GHCB spec.
+ *    If the specified page count is more than the certificate blob size, then
+ *    page count is updated to reflect the amount of valid data copied in the
+ *    vaddr.
+ */

This kdoc is misplaced: it should be in sev-dev.c, right before the implementation. Also it does not say anything about return value, and still the return type is "int".

Quoted 24 lines
include/linux/psp-sev.h +11 -0
+int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+				unsigned long vaddr, unsigned long *npages,
+				unsigned long *error);
+
 #else	/* !CONFIG_CRYPTO_DEV_SP_PSP */
 
 static inline int
@@ -992,6 +1009,13 @@ static inline void *snp_alloc_firmware_page(gfp_t mask)
 
 static inline void snp_free_firmware_page(void *addr) { }
 
+static inline int snp_guest_ext_guest_request(struct sev_data_snp_guest_request *data,
+					      unsigned long vaddr, unsigned long *n,
+					      unsigned long *error)
+{
+	return -ENODEV;
+}
+
 #endif	/* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif	/* __PSP_SEV_H__ */
Signature · 1 line
2.25.1

BR, Jarkko

author maintainer
#210 lore raw

PATCH v6 19/49 KVM: SVM: Add support to handle AP reset MSR protocol

In reply to Ashish Kalra

From: Tom Lendacky <[email protected]>

Add support for AP Reset Hold being invoked using the GHCB MSR protocol, available in version 2 of the GHCB specification.

---
 arch/x86/include/asm/sev-common.h |  2 ++
 arch/x86/kvm/svm/sev.c            | 56 ++++++++++++++++++++++++++-----
 arch/x86/kvm/svm/svm.h            |  1 +
 3 files changed, 51 insertions(+), 8 deletions(-)
arch/x86/include/asm/sev-common.h +2 -0
index b8357d6ecd47..e15548d88f2a 100644
@@ -56,6 +56,8 @@
 /* AP Reset Hold */
 #define GHCB_MSR_AP_RESET_HOLD_REQ	0x006
 #define GHCB_MSR_AP_RESET_HOLD_RESP	0x007
+#define GHCB_MSR_AP_RESET_HOLD_RESULT_POS	12
+#define GHCB_MSR_AP_RESET_HOLD_RESULT_MASK	GENMASK_ULL(51, 0)
 
 /* GHCB GPA Register */
 #define GHCB_MSR_REG_GPA_REQ		0x012
arch/x86/kvm/svm/sev.c +48 -8

85 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +1 -0
index bb9ec9139af3..9f7eb1f18893 100644
@@ -186,6 +186,7 @@ struct vcpu_sev_es_state {
 	struct ghcb *ghcb;
 	struct kvm_host_map ghcb_map;
 	bool received_first_sipi;
+	unsigned int ap_reset_hold_type;
 
 	/* SEV-ES scratch area support */
 	void *ghcb_sa;
Signature · 1 line
2.25.1
author maintainer
#211 lore raw

PATCH v6 20/49 KVM: SVM: Provide the Hypervisor Feature support VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of the GHCB specification introduced advertisement of features that are supported by the Hypervisor.

Now that KVM supports version 2 of the GHCB specification, bump the maximum supported protocol version.

---
 arch/x86/include/asm/sev-common.h |  2 ++
 arch/x86/kvm/svm/sev.c            | 14 ++++++++++++++
 arch/x86/kvm/svm/svm.h            |  3 ++-
 3 files changed, 18 insertions(+), 1 deletion(-)
arch/x86/include/asm/sev-common.h +2 -0
index e15548d88f2a..539de6b93420 100644
@@ -101,6 +101,8 @@ enum psc_op {
 /* GHCB Hypervisor Feature Request/Response */
 #define GHCB_MSR_HV_FT_REQ		0x080
 #define GHCB_MSR_HV_FT_RESP		0x081
+#define GHCB_MSR_HV_FT_POS		12
+#define GHCB_MSR_HV_FT_MASK		GENMASK_ULL(51, 0)
 #define GHCB_MSR_HV_FT_RESP_VAL(v)			\
 	/* GHCBData[63:12] */				\
 	(((u64)(v) & GENMASK_ULL(63, 12)) >> 12)
arch/x86/kvm/svm/sev.c +14 -0
index a1318236acd2..b49c370d5ae9 100644
@@ -2480,6 +2480,7 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 	case SVM_VMGEXIT_AP_HLT_LOOP:
 	case SVM_VMGEXIT_AP_JUMP_TABLE:
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
+	case SVM_VMGEXIT_HV_FEATURES:
 		break;
 	default:
 		reason = GHCB_ERR_INVALID_EVENT;
@@ -2746,6 +2747,13 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 				  GHCB_MSR_INFO_MASK,
 				  GHCB_MSR_INFO_POS);
 		break;
+	case GHCB_MSR_HV_FT_REQ: {
+		set_ghcb_msr_bits(svm, GHCB_HV_FT_SUPPORTED,
+				  GHCB_MSR_HV_FT_MASK, GHCB_MSR_HV_FT_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_HV_FT_RESP,
+				  GHCB_MSR_INFO_MASK, GHCB_MSR_INFO_POS);
+		break;
+	}
 	case GHCB_MSR_TERM_REQ: {
 		u64 reason_set, reason_code;
 
@@ -2871,6 +2879,12 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		ret = 1;
 		break;
 	}
+	case SVM_VMGEXIT_HV_FEATURES: {
+		ghcb_set_sw_exit_info_2(ghcb, GHCB_HV_FT_SUPPORTED);
+
+		ret = 1;
+		break;
+	}
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
 		vcpu_unimpl(vcpu,
 			    "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n",
arch/x86/kvm/svm/svm.h +2 -1
index 9f7eb1f18893..1f4a8bd09c9e 100644
@@ -629,9 +629,10 @@ unsigned long avic_vcpu_get_apicv_inhibit_reasons(struct kvm_vcpu *vcpu);
 
 /* sev.c */
 
-#define GHCB_VERSION_MAX	1ULL
+#define GHCB_VERSION_MAX	2ULL
 #define GHCB_VERSION_MIN	1ULL
 
+#define GHCB_HV_FT_SUPPORTED	0
 
 extern unsigned int max_sev_asid;
 
Signature · 1 line
2.25.1
author maintainer
#212 lore raw

PATCH v6 21/49 KVM: SVM: Make AVIC backing, VMSA and VMCB memory allocation SNP safe

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Implement a workaround for an SNP erratum where the CPU will incorrectly signal an RMP violation #PF if a hugepage (2mb or 1gb) collides with the RMP entry of a VMCB, VMSA or AVIC backing page.

When SEV-SNP is globally enabled, the CPU marks the VMCB, VMSA, and AVIC
backing pages as "in-use" in the RMP after a successful VMRUN. This
is done for _all_ VMs, not just SNP-Active VMs.

If the hypervisor accesses an in-use page through a writable translation, the CPU will throw an RMP violation #PF. On early SNP hardware, if an in-use page is 2mb aligned and software accesses any part of the associated 2mb region with a hupage, the CPU will incorrectly treat the entire 2mb region as in-use and signal a spurious RMP violation #PF.

The recommended is to not use the hugepage for the VMCB, VMSA or AVIC backing page. Add a generic allocator that will ensure that the page returns is not hugepage (2mb or 1gb) and is safe to be used when SEV-SNP is enabled.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  2 ++
 arch/x86/kvm/lapic.c               |  5 ++++-
 arch/x86/kvm/svm/sev.c             | 35 ++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             | 16 ++++++++++++--
 arch/x86/kvm/svm/svm.h             |  1 +
 6 files changed, 57 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index da47f60a4650..a66292dae698 100644
@@ -128,6 +128,7 @@ KVM_X86_OP(msr_filter_changed)
 KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
+KVM_X86_OP(alloc_apic_backing_page)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +2 -0
index c24a72ddc93b..0205e2944067 100644
@@ -1512,6 +1512,8 @@ struct kvm_x86_ops {
 	 * Returns vCPU specific APICv inhibit reasons
 	 */
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
+
+	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/lapic.c +4 -1
index 66b0eb0bda94..7c7fc6c4a7f9 100644
@@ -2506,7 +2506,10 @@ int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns)
 
 	vcpu->arch.apic = apic;
 
-	apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
+	if (kvm_x86_ops.alloc_apic_backing_page)
+		apic->regs = static_call(kvm_x86_alloc_apic_backing_page)(vcpu);
+	else
+		apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
 	if (!apic->regs) {
 		printk(KERN_ERR "malloc apic regs error for vcpu %x\n",
 		       vcpu->vcpu_id);
arch/x86/kvm/svm/sev.c +35 -0
index b49c370d5ae9..93365996bd59 100644
@@ -3030,3 +3030,38 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
 		break;
 	}
 }
+
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu)
+{
+	unsigned long pfn;
+	struct page *p;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+
+	/*
+	 * Allocate an SNP safe page to workaround the SNP erratum where
+	 * the CPU will incorrectly signal an RMP violation  #PF if a
+	 * hugepage (2mb or 1gb) collides with the RMP entry of VMCB, VMSA
+	 * or AVIC backing page. The recommeded workaround is to not use the
+	 * hugepage.
+	 *
+	 * Allocate one extra page, use a page which is not 2mb aligned
+	 * and free the other.
+	 */
+	p = alloc_pages(GFP_KERNEL_ACCOUNT | __GFP_ZERO, 1);
+	if (!p)
+		return NULL;
+
+	split_page(p, 1);
+
+	pfn = page_to_pfn(p);
+	if (IS_ALIGNED(__pfn_to_phys(pfn), PMD_SIZE)) {
+		pfn++;
+		__free_page(p);
+	} else {
+		__free_page(pfn_to_page(pfn + 1));
+	}
+
+	return pfn_to_page(pfn);
+}
arch/x86/kvm/svm/svm.c +14 -2
index efc7623d0f90..b4bd64f94d3a 100644
@@ -1260,7 +1260,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
 	svm = to_svm(vcpu);
 
 	err = -ENOMEM;
-	vmcb01_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+	vmcb01_page = snp_safe_alloc_page(vcpu);
 	if (!vmcb01_page)
 		goto out;
 
@@ -1269,7 +1269,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
 		 * SEV-ES guests require a separate VMSA page used to contain
 		 * the encrypted register state of the guest.
 		 */
-		vmsa_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+		vmsa_page = snp_safe_alloc_page(vcpu);
 		if (!vmsa_page)
 			goto error_free_vmcb_page;
 
@@ -4598,6 +4598,16 @@ static int svm_vm_init(struct kvm *kvm)
 	return 0;
 }
 
+static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu)
+{
+	struct page *page = snp_safe_alloc_page(vcpu);
+
+	if (!page)
+		return NULL;
+
+	return page_address(page);
+}
+
 static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.name = "kvm_amd",
 
@@ -4722,6 +4732,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 
 	.vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
+
+	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
 };
 
 /*
arch/x86/kvm/svm/svm.h +1 -0
index 1f4a8bd09c9e..9672e25a338d 100644
@@ -659,6 +659,7 @@ void sev_es_vcpu_reset(struct vcpu_svm *svm);
 void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
 void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
 
 /* vmenter.S */
 
Signature · 1 line
2.25.1
#213 lore raw

Re: [PATCH Part2 v6 21/49] KVM: SVM: Make AVIC backing, VMSA and VMCB memory allocation SNP safe

Quoted 100 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Implement a workaround for an SNP erratum where the CPU will incorrectly signal an RMP violation #PF if a hugepage (2mb or 1gb) collides with the RMP entry of a VMCB, VMSA or AVIC backing page.

When SEV-SNP is globally enabled, the CPU marks the VMCB, VMSA, and AVIC
backing pages as "in-use" in the RMP after a successful VMRUN. This
is done for _all_ VMs, not just SNP-Active VMs.

If the hypervisor accesses an in-use page through a writable translation, the CPU will throw an RMP violation #PF. On early SNP hardware, if an in-use page is 2mb aligned and software accesses any part of the associated 2mb region with a hupage, the CPU will incorrectly treat the entire 2mb region as in-use and signal a spurious RMP violation #PF.

The recommended is to not use the hugepage for the VMCB, VMSA or AVIC backing page. Add a generic allocator that will ensure that the page returns is not hugepage (2mb or 1gb) and is safe to be used when SEV-SNP is enabled.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  2 ++
 arch/x86/kvm/lapic.c               |  5 ++++-
 arch/x86/kvm/svm/sev.c             | 35 ++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             | 16 ++++++++++++--
 arch/x86/kvm/svm/svm.h             |  1 +
 6 files changed, 57 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index da47f60a4650..a66292dae698 100644
@@ -128,6 +128,7 @@ KVM_X86_OP(msr_filter_changed)
 KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
+KVM_X86_OP(alloc_apic_backing_page)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +2 -0
index c24a72ddc93b..0205e2944067 100644
@@ -1512,6 +1512,8 @@ struct kvm_x86_ops {
 	 * Returns vCPU specific APICv inhibit reasons
 	 */
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
+
+	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/lapic.c +4 -1
index 66b0eb0bda94..7c7fc6c4a7f9 100644
@@ -2506,7 +2506,10 @@ int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns)
 
 	vcpu->arch.apic = apic;
 
-	apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
+	if (kvm_x86_ops.alloc_apic_backing_page)
+		apic->regs = static_call(kvm_x86_alloc_apic_backing_page)(vcpu);
+	else
+		apic->regs = (void *)get_zeroed_page(GFP_KERNEL_ACCOUNT);
 	if (!apic->regs) {
 		printk(KERN_ERR "malloc apic regs error for vcpu %x\n",
 		       vcpu->vcpu_id);
arch/x86/kvm/svm/sev.c +17 -0
index b49c370d5ae9..93365996bd59 100644
@@ -3030,3 +3030,38 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
 		break;
 	}
 }
+
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu)
+{
+	unsigned long pfn;
+	struct page *p;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+
+	/*
+	 * Allocate an SNP safe page to workaround the SNP erratum where
+	 * the CPU will incorrectly signal an RMP violation  #PF if a
+	 * hugepage (2mb or 1gb) collides with the RMP entry of VMCB, VMSA
+	 * or AVIC backing page. The recommeded workaround is to not use the
+	 * hugepage.
+	 *
+	 * Allocate one extra page, use a page which is not 2mb aligned

arch/x86/kvm/svm/sev.c void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) · line 3059

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+	 * and free the other.
+	 */
+	p = alloc_pages(GFP_KERNEL_ACCOUNT | __GFP_ZERO, 1);
+	if (!p)
+		return NULL;
+
+	split_page(p, 1);
+
+	pfn = page_to_pfn(p);
+	if (IS_ALIGNED(__pfn_to_phys(pfn), PMD_SIZE)) {

I think you could simply IS_ALIGNED(pfn, PTRS_PER_PMD), no need to expand to full phys.

arch/x86/kvm/svm/sev.c void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) · line 3061

Quoted 2 lines
arch/x86/kvm/svm/sev.c +2 -0
+		pfn++;
+		__free_page(p);

To avoid pfn_to_page(), drop 'pfn++' and

__free_page(p++);

arch/x86/kvm/svm/sev.c void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) · line 3063

Quoted 2 lines
arch/x86/kvm/svm/sev.c +2 -0
+	} else {
+		__free_page(pfn_to_page(pfn + 1));

__free_page(p+1);

arch/x86/kvm/svm/sev.c void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) · line 3066

Quoted 3 lines
arch/x86/kvm/svm/sev.c +3 -0
+	}
+
+	return pfn_to_page(pfn);

return p;

Quoted 61 lines
arch/x86/kvm/svm/sev.c +1 -0
+}
arch/x86/kvm/svm/svm.c +14 -2
index efc7623d0f90..b4bd64f94d3a 100644
@@ -1260,7 +1260,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
 	svm = to_svm(vcpu);
 
 	err = -ENOMEM;
-	vmcb01_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+	vmcb01_page = snp_safe_alloc_page(vcpu);
 	if (!vmcb01_page)
 		goto out;
 
@@ -1269,7 +1269,7 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu)
 		 * SEV-ES guests require a separate VMSA page used to contain
 		 * the encrypted register state of the guest.
 		 */
-		vmsa_page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+		vmsa_page = snp_safe_alloc_page(vcpu);
 		if (!vmsa_page)
 			goto error_free_vmcb_page;
 
@@ -4598,6 +4598,16 @@ static int svm_vm_init(struct kvm *kvm)
 	return 0;
 }
 
+static void *svm_alloc_apic_backing_page(struct kvm_vcpu *vcpu)
+{
+	struct page *page = snp_safe_alloc_page(vcpu);
+
+	if (!page)
+		return NULL;
+
+	return page_address(page);
+}
+
 static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.name = "kvm_amd",
 
@@ -4722,6 +4732,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 
 	.vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
+
+	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
 };
 
 /*
arch/x86/kvm/svm/svm.h +1 -0
index 1f4a8bd09c9e..9672e25a338d 100644
@@ -659,6 +659,7 @@ void sev_es_vcpu_reset(struct vcpu_svm *svm);
 void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
 void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
+struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
 
 /* vmenter.S */
 
author maintainer
#214 lore raw

PATCH v6 22/49 KVM: SVM: Add initial SEV-SNP support

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The next generation of SEV is called SEV-SNP (Secure Nested Paging). SEV-SNP builds upon existing SEV and SEV-ES functionality while adding new hardware based security protection. SEV-SNP adds strong memory encryption integrity protection to help prevent malicious hypervisor-based attacks such as data replay, memory re-mapping, and more, to create an isolated execution environment.

The SNP feature is added incrementally, the later patches adds a new module parameters that can be used to enabled SEV-SNP in the KVM.

---
 arch/x86/kvm/svm/sev.c | 10 +++++++++-
 arch/x86/kvm/svm/svm.h |  8 ++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/sev.c +9 -1
index 93365996bd59..dc1f69a28aa7 100644
@@ -56,6 +56,9 @@ module_param_named(sev_es, sev_es_enabled, bool, 0444);
 #define sev_es_enabled false
 #endif /* CONFIG_KVM_AMD_SEV */
 
+/* enable/disable SEV-SNP support */
+static bool sev_snp_enabled;
+
 #define AP_RESET_HOLD_NONE		0
 #define AP_RESET_HOLD_NAE_EVENT		1
 #define AP_RESET_HOLD_MSR_PROTO		2
@@ -2120,6 +2123,7 @@ void __init sev_hardware_setup(void)
 {
 #ifdef CONFIG_KVM_AMD_SEV
 	unsigned int eax, ebx, ecx, edx, sev_asid_count, sev_es_asid_count;
+	bool sev_snp_supported = false;
 	bool sev_es_supported = false;
 	bool sev_supported = false;
 
@@ -2190,12 +2194,16 @@ void __init sev_hardware_setup(void)
 	if (misc_cg_set_capacity(MISC_CG_RES_SEV_ES, sev_es_asid_count))
 		goto out;
 
-	pr_info("SEV-ES supported: %u ASIDs\n", sev_es_asid_count);
 	sev_es_supported = true;
+	sev_snp_supported = sev_snp_enabled && cpu_feature_enabled(X86_FEATURE_SEV_SNP);
+
+	pr_info("SEV-ES %ssupported: %u ASIDs\n",
+		sev_snp_supported ? "and SEV-SNP " : "", sev_es_asid_count);
 
 out:
 	sev_enabled = sev_supported;
 	sev_es_enabled = sev_es_supported;
+	sev_snp_enabled = sev_snp_supported;
 #endif
 }
 
arch/x86/kvm/svm/svm.h +8 -0
index 9672e25a338d..edecc5066517 100644
@@ -75,6 +75,7 @@ enum {
 struct kvm_sev_info {
 	bool active;		/* SEV enabled guest */
 	bool es_active;		/* SEV-ES enabled guest */
+	bool snp_active;	/* SEV-SNP enabled guest */
 	unsigned int asid;	/* ASID used for this guest */
 	unsigned int handle;	/* SEV firmware handle */
 	int fd;			/* SEV device fd */
@@ -314,6 +315,13 @@ static __always_inline bool sev_es_guest(struct kvm *kvm)
 #endif
 }
 
+static inline bool sev_snp_guest(struct kvm *kvm)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+
+	return sev_es_guest(kvm) && sev->snp_active;
+}
+
 static inline void vmcb_mark_all_dirty(struct vmcb *vmcb)
 {
 	vmcb->control.clean = 0;
Signature · 1 line
2.25.1
author maintainer
#215 lore raw

PATCH v6 23/49 KVM: SVM: Add KVM_SNP_INIT command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SNP_INIT command is used by the hypervisor to initialize the SEV-SNP platform context. In a typical workflow, this command should be the first command issued. When creating SEV-SNP guest, the VMM must use this command instead of the KVM_SEV_INIT or KVM_SEV_ES_INIT.

The flags value must be zero, it will be extended in future SNP support to communicate the optional features (such as restricted INT injection etc).

---
 .../virt/kvm/x86/amd-memory-encryption.rst    | 27 ++++++++++++
 arch/x86/include/asm/svm.h                    |  1 +
 arch/x86/kvm/svm/sev.c                        | 44 ++++++++++++++++++-
 arch/x86/kvm/svm/svm.h                        |  4 ++
 include/uapi/linux/kvm.h                      | 13 ++++++
 5 files changed, 87 insertions(+), 2 deletions(-)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +27 -0
index 2d307811978c..903023f524af 100644
@@ -435,6 +435,33 @@ issued by the hypervisor to make the guest ready for execution.
 
 Returns: 0 on success, -negative on error
 
+18. KVM_SNP_INIT
+----------------
+
+The KVM_SNP_INIT command can be used by the hypervisor to initialize SEV-SNP
+context. In a typical workflow, this command should be the first command issued.
+
+Parameters (in/out): struct kvm_snp_init
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_snp_init {
+                __u64 flags;
+        };
+
+The flags bitmap is defined as::
+
+   /* enable the restricted injection */
+   #define KVM_SEV_SNP_RESTRICTED_INJET   (1<<0)
+
+   /* enable the restricted injection timer */
+   #define KVM_SEV_SNP_RESTRICTED_TIMER_INJET   (1<<1)
+
+If the specified flags is not supported then return -EOPNOTSUPP, and the supported
+flags are returned.
+
 References
 ==========
 
arch/x86/include/asm/svm.h +1 -0
index 1b07fba11704..284a8113227e 100644
@@ -263,6 +263,7 @@ enum avic_ipi_failure_cause {
 #define AVIC_HPA_MASK	~((0xFFFULL << 52) | 0xFFF)
 #define VMCB_AVIC_APIC_BAR_MASK		0xFFFFFFFFFF000ULL
 
+#define SVM_SEV_FEAT_SNP_ACTIVE		BIT(0)
 
 struct vmcb_seg {
 	u16 selector;
arch/x86/kvm/svm/sev.c +42 -2

79 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +4 -0
index edecc5066517..2f45589ee596 100644
@@ -72,6 +72,9 @@ enum {
 /* TPR and CR2 are always written before VMRUN */
 #define VMCB_ALWAYS_DIRTY_MASK	((1U << VMCB_INTR) | (1U << VMCB_CR2))
 
+/* Supported init feature flags */
+#define SEV_SNP_SUPPORTED_FLAGS		0x0
+
 struct kvm_sev_info {
 	bool active;		/* SEV enabled guest */
 	bool es_active;		/* SEV-ES enabled guest */
@@ -87,6 +90,7 @@ struct kvm_sev_info {
 	struct list_head mirror_entry; /* Use as a list entry of mirrors */
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
 	atomic_t migration_in_progress;
+	u64 snp_init_flags;
 };
 
 struct kvm_svm {
include/uapi/linux/kvm.h +13 -0
index 68ce07185f03..0f912cefc544 100644
@@ -1810,6 +1810,9 @@ enum sev_cmd_id {
 	/* Guest Migration Extension */
 	KVM_SEV_SEND_CANCEL,
 
+	/* SNP specific commands */
+	KVM_SEV_SNP_INIT,
+
 	KVM_SEV_NR_MAX,
 };
 
@@ -1906,6 +1909,16 @@ struct kvm_sev_receive_update_data {
 	__u32 trans_len;
 };
 
+/* enable the restricted injection */
+#define KVM_SEV_SNP_RESTRICTED_INJET   (1 << 0)
+
+/* enable the restricted injection timer */
+#define KVM_SEV_SNP_RESTRICTED_TIMER_INJET   (1 << 1)
+
+struct kvm_snp_init {
+	__u64 flags;
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
author maintainer
#216 lore raw

PATCH v6 24/49 KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

KVM_SEV_SNP_LAUNCH_START begins the launch process for an SEV-SNP guest. The command initializes a cryptographic digest context used to construct the measurement of the guest. If the guest is expected to be migrated, the command also binds a migration agent (MA) to the guest.

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  24 ++++
 arch/x86/kvm/svm/sev.c                        | 115 +++++++++++++++++-
 arch/x86/kvm/svm/svm.h                        |   1 +
 include/uapi/linux/kvm.h                      |  10 ++
 4 files changed, 147 insertions(+), 3 deletions(-)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +24 -0
index 903023f524af..878711f2dca6 100644
@@ -462,6 +462,30 @@ The flags bitmap is defined as::
 If the specified flags is not supported then return -EOPNOTSUPP, and the supported
 flags are returned.
 
+19. KVM_SNP_LAUNCH_START
+------------------------
+
+The KVM_SNP_LAUNCH_START command is used for creating the memory encryption
+context for the SEV-SNP guest. To create the encryption context, user must
+provide a guest policy, migration agent (if any) and guest OS visible
+workarounds value as defined SEV-SNP specification.
+
+Parameters (in): struct  kvm_snp_launch_start
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_start {
+                __u64 policy;           /* Guest policy to use. */
+                __u64 ma_uaddr;         /* userspace address of migration agent */
+                __u8 ma_en;             /* 1 if the migtation agent is enabled */
+                __u8 imi_en;            /* set IMI to 1. */
+                __u8 gosvw[16];         /* guest OS visible workarounds */
+        };
+
+See the SEV-SNP specification for further detail on the launch input.
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +112 -3

161 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +1 -0
index 2f45589ee596..71c011af098e 100644
@@ -91,6 +91,7 @@ struct kvm_sev_info {
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
 	atomic_t migration_in_progress;
 	u64 snp_init_flags;
+	void *snp_context;      /* SNP guest context page */
 };
 
 struct kvm_svm {
include/uapi/linux/kvm.h +10 -0
index 0f912cefc544..0cb119d66ae5 100644
@@ -1812,6 +1812,7 @@ enum sev_cmd_id {
 
 	/* SNP specific commands */
 	KVM_SEV_SNP_INIT,
+	KVM_SEV_SNP_LAUNCH_START,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1919,6 +1920,15 @@ struct kvm_snp_init {
 	__u64 flags;
 };
 
+struct kvm_sev_snp_launch_start {
+	__u64 policy;
+	__u64 ma_uaddr;
+	__u8 ma_en;
+	__u8 imi_en;
+	__u8 gosvw[16];
+	__u8 pad[6];
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
#217 lore raw

Re: [PATCH Part2 v6 24/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command

Quoted 9 lines

+19. KVM_SNP_LAUNCH_START
+------------------------

Documentation/virt/kvm/x86/amd-memory-encryption.rst +6 -0
+
+The KVM_SNP_LAUNCH_START command is used for creating the memory encryption
+context for the SEV-SNP guest. To create the encryption context, user must
+provide a guest policy, migration agent (if any) and guest OS visible
+workarounds value as defined SEV-SNP specification.
+

Documentation/virt/kvm/x86/amd-memory-encryption.rst The flags bitmap is defined as:: · line 482

Quoted 10 lines
Documentation/virt/kvm/x86/amd-memory-encryption.rst +10 -0
+Parameters (in): struct  kvm_snp_launch_start
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_start {
+                __u64 policy;           /* Guest policy to use. */
+                __u64 ma_uaddr;         /* userspace address of migration agent */
+                __u8 ma_en;             /* 1 if the migtation agent is enabled */

migration

Quoted 13 lines
(quoted diff) +8 -0
+                __u8 imi_en;            /* set IMI to 1. */
+                __u8 gosvw[16];         /* guest OS visible workarounds */
+        };
+
+See the SEV-SNP specification for further detail on the launch input.
+
 References
 ==========
 
 
 
+static int snp_decommission_context(struct kvm *kvm)
+{

Review

Quoted 10 lines
(quoted diff) +10 -0
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_decommission data = {};
+       int ret;
+
+       /* If context is not created then do nothing */
+       if (!sev->snp_context)
+               return 0;
+
+       data.gctx_paddr = __sme_pa(sev->snp_context);
+       ret = snp_guest_decommission(&data, NULL);

Do we have a similar race like in sev_unbind_asid() with DEACTIVATE and WBINVD/DF_FLUSH? The SNP_DECOMMISSION spec looks quite similar to DEACTIVATE.

Quoted 13 lines
arch/x86/kvm/svm/sev.c +10 -0
+       if (WARN_ONCE(ret, "failed to release guest context"))
+               return ret;
+
+       /* free the context page now */
+       snp_free_firmware_page(sev->snp_context);
+       sev->snp_context = NULL;
+
+       return 0;
+}
+
 void sev_vm_destroy(struct kvm *kvm)
 {
        struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
author maintainer
#218 lore raw

RE: [PATCH Part2 v6 24/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command

[AMD Official Use Only - General]

Quoted 5 lines
Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+static int snp_decommission_context(struct kvm *kvm) {
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_decommission data = {};
+       int ret;
+
Quoted 8 lines

arch/x86/kvm/svm/sev.c int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 2199

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+       /* If context is not created then do nothing */
+       if (!sev->snp_context)
+               return 0;
+
+       data.gctx_paddr = __sme_pa(sev->snp_context);
+       ret = snp_guest_decommission(&data, NULL);

Do we have a similar race like in sev_unbind_asid() with DEACTIVATE and WBINVD/DF_FLUSH? The SNP_DECOMMISSION spec looks quite similar to DEACTIVATE.

Yes, SNP_DECOMMISION also marks the ASID as invalid and require a WBINVD/DF_FLUSH before the ASID is re-used/re-cycled, so we need to prevent against DECOMMISION and ASID re-cycling happening at the same time. Can reuse the same RWSEM (sev_deactivate_lock) here too.

Thanks,
Ashish

Quoted 13 lines
arch/x86/kvm/svm/sev.c +10 -0
+       if (WARN_ONCE(ret, "failed to release guest context"))
+               return ret;
+
+       /* free the context page now */
+       snp_free_firmware_page(sev->snp_context);
+       sev->snp_context = NULL;
+
+       return 0;
+}
+
 void sev_vm_destroy(struct kvm *kvm)
 {
        struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
#219 lore raw

Re: [PATCH Part2 v6 24/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_START command

In reply to Ashish Kalra
Quoted 94 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

KVM_SEV_SNP_LAUNCH_START begins the launch process for an SEV-SNP guest. The command initializes a cryptographic digest context used to construct the measurement of the guest. If the guest is expected to be migrated, the command also binds a migration agent (MA) to the guest.

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  24 ++++
 arch/x86/kvm/svm/sev.c                        | 115 +++++++++++++++++-
 arch/x86/kvm/svm/svm.h                        |   1 +
 include/uapi/linux/kvm.h                      |  10 ++
 4 files changed, 147 insertions(+), 3 deletions(-)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +24 -0
index 903023f524af..878711f2dca6 100644
@@ -462,6 +462,30 @@ The flags bitmap is defined as::
 If the specified flags is not supported then return -EOPNOTSUPP, and the supported
 flags are returned.
 
+19. KVM_SNP_LAUNCH_START
+------------------------
+
+The KVM_SNP_LAUNCH_START command is used for creating the memory encryption
+context for the SEV-SNP guest. To create the encryption context, user must
+provide a guest policy, migration agent (if any) and guest OS visible
+workarounds value as defined SEV-SNP specification.
+
+Parameters (in): struct  kvm_snp_launch_start
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_start {
+                __u64 policy;           /* Guest policy to use. */
+                __u64 ma_uaddr;         /* userspace address of migration agent */
+                __u8 ma_en;             /* 1 if the migtation agent is enabled */
+                __u8 imi_en;            /* set IMI to 1. */
+                __u8 gosvw[16];         /* guest OS visible workarounds */
+        };
+
+See the SEV-SNP specification for further detail on the launch input.
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +10 -2
index 813bda7f7b55..9e6fc7a94ed7 100644
@@ -21,6 +21,7 @@
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
 #include <asm/fpu/xcr.h>
+#include <asm/sev.h>
 
 #include "x86.h"
 #include "svm.h"
@@ -73,6 +74,8 @@ static unsigned int nr_asids;
 static unsigned long *sev_asid_bitmap;
 static unsigned long *sev_reclaim_asid_bitmap;
 
+static int snp_decommission_context(struct kvm *kvm);
+
 struct enc_region {
 	struct list_head list;
 	unsigned long npages;
@@ -98,12 +101,17 @@ static int sev_flush_asids(int min_asid, int max_asid)
 	down_write(&sev_deactivate_lock);
 
 	wbinvd_on_all_cpus();
-	ret = sev_guest_df_flush(&error);
+
+	if (sev_snp_enabled)
+		ret = snp_guest_df_flush(&error);
+	else
+		ret = sev_guest_df_flush(&error);
 
 	up_write(&sev_deactivate_lock);
 
 	if (ret)
-		pr_err("SEV: DF_FLUSH failed, ret=%d, error=%#x\n", ret, error);
+		pr_err("SEV%s: DF_FLUSH failed, ret=%d, error=%#x\n",
+			sev_snp_enabled ? "-SNP" : "", ret, error);
 
 	return ret;
 }

arch/x86/kvm/svm/sev.c int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 1842

Quoted 10 lines
arch/x86/kvm/svm/sev.c +7 -0
@@ -1825,6 +1833,74 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 	return ret;
 }
 
+static void *snp_context_create(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct sev_data_snp_gctx_create data = {};
+	void *context;
+	int rc;
+
+	/* Allocate memory for context page */

Nit: this comment has very little value, if any. It's just stating the obvious.

Instead, I'd add a description for the function:

/*
* Allocate and initialize a digest for the guest measurement. */
static void *snp_context_create(struct kvm *kvm, struct kvm_sev_cmd *argp)

This would be much more helpful to get a grasp on "what I'm looking at".

Quoted 163 lines
arch/x86/kvm/svm/sev.c +95 -1
+	context = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
+	if (!context)
+		return NULL;
+
+	data.gctx_paddr = __psp_pa(context);
+	rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_GCTX_CREATE, &data, &argp->error);
+	if (rc) {
+		snp_free_firmware_page(context);
+		return NULL;
+	}
+
+	return context;
+}
+
+static int snp_bind_asid(struct kvm *kvm, int *error)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_activate data = {0};
+
+	data.gctx_paddr = __psp_pa(sev->snp_context);
+	data.asid   = sev_get_asid(kvm);
+	return sev_issue_cmd(kvm, SEV_CMD_SNP_ACTIVATE, &data, error);
+}
+
+static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_start start = {0};
+	struct kvm_sev_snp_launch_start params;
+	int rc;
+
+	if (!sev_snp_guest(kvm))
+		return -ENOTTY;
+
+	if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+		return -EFAULT;
+
+	sev->snp_context = snp_context_create(kvm, argp);
+	if (!sev->snp_context)
+		return -ENOTTY;
+
+	start.gctx_paddr = __psp_pa(sev->snp_context);
+	start.policy = params.policy;
+	memcpy(start.gosvw, params.gosvw, sizeof(params.gosvw));
+	rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_START, &start, &argp->error);
+	if (rc)
+		goto e_free_context;
+
+	sev->fd = argp->sev_fd;
+	rc = snp_bind_asid(kvm, &argp->error);
+	if (rc)
+		goto e_free_context;
+
+	return 0;
+
+e_free_context:
+	snp_decommission_context(kvm);
+
+	return rc;
+}
+
 int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_sev_cmd sev_cmd;
@@ -1915,6 +1991,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 	case KVM_SEV_RECEIVE_FINISH:
 		r = sev_receive_finish(kvm, &sev_cmd);
 		break;
+	case KVM_SEV_SNP_LAUNCH_START:
+		r = snp_launch_start(kvm, &sev_cmd);
+		break;
 	default:
 		r = -EINVAL;
 		goto out;
@@ -2106,6 +2185,28 @@ int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 	return ret;
 }
 
+static int snp_decommission_context(struct kvm *kvm)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_decommission data = {};
+	int ret;
+
+	/* If context is not created then do nothing */
+	if (!sev->snp_context)
+		return 0;
+
+	data.gctx_paddr = __sme_pa(sev->snp_context);
+	ret = snp_guest_decommission(&data, NULL);
+	if (WARN_ONCE(ret, "failed to release guest context"))
+		return ret;
+
+	/* free the context page now */
+	snp_free_firmware_page(sev->snp_context);
+	sev->snp_context = NULL;
+
+	return 0;
+}
+
 void sev_vm_destroy(struct kvm *kvm)
 {
 	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
@@ -2147,7 +2248,15 @@ void sev_vm_destroy(struct kvm *kvm)
 		}
 	}
 
-	sev_unbind_asid(kvm, sev->handle);
+	if (sev_snp_guest(kvm)) {
+		if (snp_decommission_context(kvm)) {
+			WARN_ONCE(1, "Failed to free SNP guest context, leaking asid!\n");
+			return;
+		}
+	} else {
+		sev_unbind_asid(kvm, sev->handle);
+	}
+
 	sev_asid_free(sev);
 }
 
arch/x86/kvm/svm/svm.h +1 -0
index 2f45589ee596..71c011af098e 100644
@@ -91,6 +91,7 @@ struct kvm_sev_info {
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
 	atomic_t migration_in_progress;
 	u64 snp_init_flags;
+	void *snp_context;      /* SNP guest context page */
 };
 
 struct kvm_svm {
include/uapi/linux/kvm.h +10 -0
index 0f912cefc544..0cb119d66ae5 100644
@@ -1812,6 +1812,7 @@ enum sev_cmd_id {
 
 	/* SNP specific commands */
 	KVM_SEV_SNP_INIT,
+	KVM_SEV_SNP_LAUNCH_START,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1919,6 +1920,15 @@ struct kvm_snp_init {
 	__u64 flags;
 };
 
+struct kvm_sev_snp_launch_start {
+	__u64 policy;
+	__u64 ma_uaddr;
+	__u8 ma_en;
+	__u8 imi_en;
+	__u8 gosvw[16];
+	__u8 pad[6];
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1

BR, Jarkko

author maintainer
#220 lore raw

PATCH v6 25/49 KVM: SVM: Disallow registering memory range from HugeTLB for SNP guest

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

While creating the VM, userspace call the KVM_MEMORY_ENCRYPT_REG_REGION ioctl to register the memory regions for the guest. This registered memory region is typically used as a guest RAM. Later, the guest may issue the page state change (PSC) request that will require splitting the large page into smaller page. If the memory is allocated from the HugeTLB then hypervisor will not be able to split it.

Do not allow registering the memory range backed by the HugeTLB until the hypervisor support is added to handle the case.

---
 arch/x86/kvm/svm/sev.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
arch/x86/kvm/svm/sev.c +37 -0
index 9e6fc7a94ed7..41b83aa6b5f4 100644
@@ -17,6 +17,7 @@
 #include <linux/misc_cgroup.h>
 #include <linux/processor.h>
 #include <linux/trace_events.h>
+#include <linux/hugetlb.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
@@ -2007,6 +2008,35 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 	return r;
 }
 
+static bool is_range_hugetlb(struct kvm *kvm, struct kvm_enc_region *range)
+{
+	struct vm_area_struct *vma;
+	u64 start, end;
+	bool ret = true;
+
+	start = range->addr;
+	end = start + range->size;
+
+	mmap_read_lock(kvm->mm);
+
+	do {
+		vma = find_vma_intersection(kvm->mm, start, end);
+		if (!vma)
+			goto unlock;
+
+		if (is_vm_hugetlb_page(vma))
+			goto unlock;
+
+		start = vma->vm_end;
+	} while (end > vma->vm_end);
+
+	ret = false;
+
+unlock:
+	mmap_read_unlock(kvm->mm);
+	return ret;
+}
+
 int sev_mem_enc_register_region(struct kvm *kvm,
 				struct kvm_enc_region *range)
 {
@@ -2024,6 +2054,13 @@ int sev_mem_enc_register_region(struct kvm *kvm,
 	if (range->addr > ULONG_MAX || range->size > ULONG_MAX)
 		return -EINVAL;
 
+	/*
+	 * SEV-SNP does not support the backing pages from the HugeTLB. Verify
+	 * that the registered memory range is not from the HugeTLB.
+	 */
+	if (sev_snp_guest(kvm) && is_range_hugetlb(kvm, range))
+		return -EINVAL;
+
 	region = kzalloc(sizeof(*region), GFP_KERNEL_ACCOUNT);
 	if (!region)
 		return -ENOMEM;
Signature · 1 line
2.25.1
#221 lore raw

Re: [PATCH Part2 v6 25/49] KVM: SVM: Disallow registering memory range from HugeTLB for SNP guest

Quoted 44 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

While creating the VM, userspace call the KVM_MEMORY_ENCRYPT_REG_REGION ioctl to register the memory regions for the guest. This registered memory region is typically used as a guest RAM. Later, the guest may issue the page state change (PSC) request that will require splitting the large page into smaller page. If the memory is allocated from the HugeTLB then hypervisor will not be able to split it.

Do not allow registering the memory range backed by the HugeTLB until the hypervisor support is added to handle the case.

---
 arch/x86/kvm/svm/sev.c | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
arch/x86/kvm/svm/sev.c +12 -0
index 9e6fc7a94ed7..41b83aa6b5f4 100644
@@ -17,6 +17,7 @@
 #include <linux/misc_cgroup.h>
 #include <linux/processor.h>
 #include <linux/trace_events.h>
+#include <linux/hugetlb.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
@@ -2007,6 +2008,35 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 	return r;
 }
 
+static bool is_range_hugetlb(struct kvm *kvm, struct kvm_enc_region *range)
+{
+	struct vm_area_struct *vma;
+	u64 start, end;
+	bool ret = true;
+
+	start = range->addr;
+	end = start + range->size;
+
+	mmap_read_lock(kvm->mm);
+

arch/x86/kvm/svm/sev.c int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp) · line 2031

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+	do {
+		vma = find_vma_intersection(kvm->mm, start, end);
+		if (!vma)
+			goto unlock;
+
+		if (is_vm_hugetlb_page(vma))
+			goto unlock;
+
+		start = vma->vm_end;
+	} while (end > vma->vm_end);

Note it's more efficient to only find the first vma and then iterate using vma->vm_next. But the details will change when maple tree is merged, and likely this patch won't exist after rebasing to UPM anyway...

Quoted 24 lines
arch/x86/kvm/svm/sev.c +14 -0
+	ret = false;
+
+unlock:
+	mmap_read_unlock(kvm->mm);
+	return ret;
+}
+
 int sev_mem_enc_register_region(struct kvm *kvm,
 				struct kvm_enc_region *range)
 {
@@ -2024,6 +2054,13 @@ int sev_mem_enc_register_region(struct kvm *kvm,
 	if (range->addr > ULONG_MAX || range->size > ULONG_MAX)
 		return -EINVAL;
 
+	/*
+	 * SEV-SNP does not support the backing pages from the HugeTLB. Verify
+	 * that the registered memory range is not from the HugeTLB.
+	 */
+	if (sev_snp_guest(kvm) && is_range_hugetlb(kvm, range))
+		return -EINVAL;
+
 	region = kzalloc(sizeof(*region), GFP_KERNEL_ACCOUNT);
 	if (!region)
 		return -ENOMEM;
author maintainer
#222 lore raw

PATCH v6 26/49 KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_UPDATE command can be used to insert data into the guest's memory. The data is encrypted with the cryptographic context created with the KVM_SEV_SNP_LAUNCH_START.

In addition to the inserting data, it can insert a two special pages into the guests memory: the secrets page and the CPUID page.

While terminating the guest, reclaim the guest pages added in the RMP table. If the reclaim fails, then the page is no longer safe to be released back to the system and leak them.

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  29 +++
 arch/x86/kvm/svm/sev.c                        | 187 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  19 ++
 3 files changed, 235 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +29 -0
index 878711f2dca6..62abd5c1f72b 100644
@@ -486,6 +486,35 @@ Returns: 0 on success, -negative on error
 
 See the SEV-SNP specification for further detail on the launch input.
 
+20. KVM_SNP_LAUNCH_UPDATE
+-------------------------
+
+The KVM_SNP_LAUNCH_UPDATE is used for encrypting a memory region. It also
+calculates a measurement of the memory contents. The measurement is a signature
+of the memory contents that can be sent to the guest owner as an attestation
+that the memory was encrypted correctly by the firmware.
+
+Parameters (in): struct  kvm_snp_launch_update
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_update {
+                __u64 start_gfn;        /* Guest page number to start from. */
+                __u64 uaddr;            /* userspace address need to be encrypted */
+                __u32 len;              /* length of memory region */
+                __u8 imi_page;          /* 1 if memory is part of the IMI */
+                __u8 page_type;         /* page type */
+                __u8 vmpl3_perms;       /* VMPL3 permission mask */
+                __u8 vmpl2_perms;       /* VMPL2 permission mask */
+                __u8 vmpl1_perms;       /* VMPL1 permission mask */
+        };
+
+See the SEV-SNP spec for further details on how to build the VMPL permission
+mask and page type.
+
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +187 -0

217 lines not on this page · raw message

include/uapi/linux/kvm.h +19 -0
index 0cb119d66ae5..9b36b07414ea 100644
@@ -1813,6 +1813,7 @@ enum sev_cmd_id {
 	/* SNP specific commands */
 	KVM_SEV_SNP_INIT,
 	KVM_SEV_SNP_LAUNCH_START,
+	KVM_SEV_SNP_LAUNCH_UPDATE,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1929,6 +1930,24 @@ struct kvm_sev_snp_launch_start {
 	__u8 pad[6];
 };
 
+#define KVM_SEV_SNP_PAGE_TYPE_NORMAL		0x1
+#define KVM_SEV_SNP_PAGE_TYPE_VMSA		0x2
+#define KVM_SEV_SNP_PAGE_TYPE_ZERO		0x3
+#define KVM_SEV_SNP_PAGE_TYPE_UNMEASURED	0x4
+#define KVM_SEV_SNP_PAGE_TYPE_SECRETS		0x5
+#define KVM_SEV_SNP_PAGE_TYPE_CPUID		0x6
+
+struct kvm_sev_snp_launch_update {
+	__u64 start_gfn;
+	__u64 uaddr;
+	__u32 len;
+	__u8 imi_page;
+	__u8 page_type;
+	__u8 vmpl3_perms;
+	__u8 vmpl2_perms;
+	__u8 vmpl1_perms;
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
#223 lore raw

Re: [PATCH Part2 v6 26/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

Quoted 82 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_UPDATE command can be used to insert data into the guest's memory. The data is encrypted with the cryptographic context created with the KVM_SEV_SNP_LAUNCH_START.

In addition to the inserting data, it can insert a two special pages into the guests memory: the secrets page and the CPUID page.

While terminating the guest, reclaim the guest pages added in the RMP table. If the reclaim fails, then the page is no longer safe to be released back to the system and leak them.

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  29 +++
 arch/x86/kvm/svm/sev.c                        | 187 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  19 ++
 3 files changed, 235 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +29 -0
index 878711f2dca6..62abd5c1f72b 100644
@@ -486,6 +486,35 @@ Returns: 0 on success, -negative on error
 
 See the SEV-SNP specification for further detail on the launch input.
 
+20. KVM_SNP_LAUNCH_UPDATE
+-------------------------
+
+The KVM_SNP_LAUNCH_UPDATE is used for encrypting a memory region. It also
+calculates a measurement of the memory contents. The measurement is a signature
+of the memory contents that can be sent to the guest owner as an attestation
+that the memory was encrypted correctly by the firmware.
+
+Parameters (in): struct  kvm_snp_launch_update
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_update {
+                __u64 start_gfn;        /* Guest page number to start from. */
+                __u64 uaddr;            /* userspace address need to be encrypted */
+                __u32 len;              /* length of memory region */
+                __u8 imi_page;          /* 1 if memory is part of the IMI */
+                __u8 page_type;         /* page type */
+                __u8 vmpl3_perms;       /* VMPL3 permission mask */
+                __u8 vmpl2_perms;       /* VMPL2 permission mask */
+                __u8 vmpl1_perms;       /* VMPL1 permission mask */
+        };
+
+See the SEV-SNP spec for further details on how to build the VMPL permission
+mask and page type.
+
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +4 -0
index 41b83aa6b5f4..b5f0707d7ed6 100644
@@ -18,6 +18,7 @@
 #include <linux/processor.h>
 #include <linux/trace_events.h>
 #include <linux/hugetlb.h>
+#include <linux/sev.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
@@ -233,6 +234,49 @@ static void sev_decommission(unsigned int handle)
        sev_guest_decommission(&decommission, NULL);
 }
 
+static inline void snp_leak_pages(u64 pfn, enum pg_level level)
+{
+       unsigned int npages = page_level_size(level) >> PAGE_SHIFT;

arch/x86/kvm/svm/sev.c static void sev_decommission(unsigned int handle) · line 249

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       WARN(1, "psc failed pfn 0x%llx pages %d (leaking)\n", pfn, npages);
+
+       while (npages) {
+               memory_failure(pfn, 0);
+               dump_rmpentry(pfn);
+               npages--;
+               pfn++;
+       }
+}

Should this be deduplicated with the snp_leak_pages() in "crypto: ccp:
Handle the legacy TMR allocation when SNP is enabled" ?

Quoted 42 lines
arch/x86/kvm/svm/sev.c +35 -0
+
+static int snp_page_reclaim(u64 pfn)
+{
+       struct sev_data_snp_page_reclaim data = {0};
+       int err, rc;
+
+       data.paddr = __sme_set(pfn << PAGE_SHIFT);
+       rc = snp_guest_page_reclaim(&data, &err);
+       if (rc) {
+               /*
+                * If the reclaim failed, then page is no longer safe
+                * to use.
+                */
+               snp_leak_pages(pfn, PG_LEVEL_4K);
+       }
+
+       return rc;
+}
+
+static int host_rmp_make_shared(u64 pfn, enum pg_level level, bool leak)
+{
+       int rc;
+
+       rc = rmp_make_shared(pfn, level);
+       if (rc && leak)
+               snp_leak_pages(pfn, level);
+
+       return rc;
+}
+
 static void sev_unbind_asid(struct kvm *kvm, unsigned int handle)
 {
        struct sev_data_deactivate deactivate;
@@ -1902,6 +1946,123 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
        return rc;
 }
 
+static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct list_head *head = &sev->regions_list;
+       struct enc_region *i;

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 1963

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       lockdep_assert_held(&kvm->lock);
+
+       list_for_each_entry(i, head, list) {
+               u64 start = i->uaddr;
+               u64 end = start + i->size;
+
+               if (start <= hva && end >= (hva + len))
+                       return true;
+       }

Given that usersapce could load sev->regions_list with any # of any sized regions. Should we add a cond_resched() like in sev_vm_destroy()?

Quoted 75 lines
arch/x86/kvm/svm/sev.c +75 -0
+
+       return false;
+}
+
+static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_launch_update data = {0};
+       struct kvm_sev_snp_launch_update params;
+       unsigned long npages, pfn, n = 0;
+       int *error = &argp->error;
+       struct page **inpages;
+       int ret, i, level;
+       u64 gfn;
+
+       if (!sev_snp_guest(kvm))
+               return -ENOTTY;
+
+       if (!sev->snp_context)
+               return -EINVAL;
+
+       if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+               return -EFAULT;
+
+       /* Verify that the specified address range is registered. */
+       if (!is_hva_registered(kvm, params.uaddr, params.len))
+               return -EINVAL;
+
+       /*
+        * The userspace memory is already locked so technically we don't
+        * need to lock it again. Later part of the function needs to know
+        * pfn so call the sev_pin_memory() so that we can get the list of
+        * pages to iterate through.
+        */
+       inpages = sev_pin_memory(kvm, params.uaddr, params.len, &npages, 1);
+       if (!inpages)
+               return -ENOMEM;
+
+       /*
+        * Verify that all the pages are marked shared in the RMP table before
+        * going further. This is avoid the cases where the userspace may try
+        * updating the same page twice.
+        */
+       for (i = 0; i < npages; i++) {
+               if (snp_lookup_rmpentry(page_to_pfn(inpages[i]), &level) != 0) {
+                       sev_unpin_memory(kvm, inpages, npages);
+                       return -EFAULT;
+               }
+       }
+
+       gfn = params.start_gfn;
+       level = PG_LEVEL_4K;
+       data.gctx_paddr = __psp_pa(sev->snp_context);
+
+       for (i = 0; i < npages; i++) {
+               pfn = page_to_pfn(inpages[i]);
+
+               ret = rmp_make_private(pfn, gfn << PAGE_SHIFT, level, sev_get_asid(kvm), true);
+               if (ret) {
+                       ret = -EFAULT;
+                       goto e_unpin;
+               }
+
+               n++;
+               data.address = __sme_page_pa(inpages[i]);
+               data.page_size = X86_TO_RMP_PG_LEVEL(level);
+               data.page_type = params.page_type;
+               data.vmpl3_perms = params.vmpl3_perms;
+               data.vmpl2_perms = params.vmpl2_perms;
+               data.vmpl1_perms = params.vmpl1_perms;
+               ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE, &data, error);
+               if (ret) {
+                       /*
+                        * If the command failed then need to reclaim the page.
+                        */

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2048

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+                       snp_page_reclaim(pfn);
+                       goto e_unpin;
+               }
+
+               gfn++;
+       }
+
+e_unpin:
+       /* Content of memory is updated, mark pages dirty */
+       for (i = 0; i < n; i++) {

Since |n| is not only a loop variable but actually carries the number of private pages over to e_unpin can we use a more descriptive name? How about something like 'nprivate_pages'?

Quoted 100 lines
arch/x86/kvm/svm/sev.c +43 -0
+               set_page_dirty_lock(inpages[i]);
+               mark_page_accessed(inpages[i]);
+
+               /*
+                * If its an error, then update RMP entry to change page ownership
+                * to the hypervisor.
+                */
+               if (ret)
+                       host_rmp_make_shared(pfn, level, true);
+       }
+
+       /* Unlock the user pages */
+       sev_unpin_memory(kvm, inpages, npages);
+
+       return ret;
+}
+
 int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
        struct kvm_sev_cmd sev_cmd;
@@ -1995,6 +2156,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
        case KVM_SEV_SNP_LAUNCH_START:
                r = snp_launch_start(kvm, &sev_cmd);
                break;
+       case KVM_SEV_SNP_LAUNCH_UPDATE:
+               r = snp_launch_update(kvm, &sev_cmd);
+               break;
        default:
                r = -EINVAL;
                goto out;
@@ -2113,6 +2277,29 @@ find_enc_region(struct kvm *kvm, struct kvm_enc_region *range)
 static void __unregister_enc_region_locked(struct kvm *kvm,
                                           struct enc_region *region)
 {
+       unsigned long i, pfn;
+       int level;
+
+       /*
+        * The guest memory pages are assigned in the RMP table. Unassign it
+        * before releasing the memory.
+        */
+       if (sev_snp_guest(kvm)) {
+               for (i = 0; i < region->npages; i++) {
+                       pfn = page_to_pfn(region->pages[i]);
+
+                       if (!snp_lookup_rmpentry(pfn, &level))
+                               continue;
+
+                       cond_resched();
+
+                       if (level > PG_LEVEL_4K)
+                               pfn &= ~(KVM_PAGES_PER_HPAGE(PG_LEVEL_2M) - 1);
+
+                       host_rmp_make_shared(pfn, level, true);
+               }
+       }
+
        sev_unpin_memory(kvm, region->pages, region->npages);
        list_del(&region->list);
        kfree(region);
include/uapi/linux/kvm.h +19 -1
index 0cb119d66ae5..9b36b07414ea 100644
@@ -1813,6 +1813,7 @@ enum sev_cmd_id {
        /* SNP specific commands */
        KVM_SEV_SNP_INIT,
        KVM_SEV_SNP_LAUNCH_START,
+       KVM_SEV_SNP_LAUNCH_UPDATE,
 
        KVM_SEV_NR_MAX,
 };
@@ -1929,6 +1930,24 @@ struct kvm_sev_snp_launch_start {
        __u8 pad[6];
 };
 
+#define KVM_SEV_SNP_PAGE_TYPE_NORMAL           0x1
+#define KVM_SEV_SNP_PAGE_TYPE_VMSA             0x2
+#define KVM_SEV_SNP_PAGE_TYPE_ZERO             0x3
+#define KVM_SEV_SNP_PAGE_TYPE_UNMEASURED       0x4
+#define KVM_SEV_SNP_PAGE_TYPE_SECRETS          0x5
+#define KVM_SEV_SNP_PAGE_TYPE_CPUID            0x6
+
+struct kvm_sev_snp_launch_update {
+       __u64 start_gfn;
+       __u64 uaddr;
+       __u32 len;
+       __u8 imi_page;
+       __u8 page_type;
+       __u8 vmpl3_perms;
+       __u8 vmpl2_perms;
+       __u8 vmpl1_perms;
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU    (1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3         (1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX       (1 << 2)
--

2.25.1

author maintainer
#224 lore raw

RE: [PATCH Part2 v6 26/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

[Public]

Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/svm/sev.c +8 -0
+static inline void snp_leak_pages(u64 pfn, enum pg_level level) {
+       unsigned int npages = page_level_size(level) >> PAGE_SHIFT;
+
+       WARN(1, "psc failed pfn 0x%llx pages %d (leaking)\n", pfn, 
+ npages);
+
+       while (npages) {
+               memory_failure(pfn, 0);
Quoted 8 lines

arch/x86/kvm/svm/sev.c static void sev_decommission(unsigned int handle) · line 249

Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+               dump_rmpentry(pfn);
+               npages--;
+               pfn++;
+       }
+}

Should this be deduplicated with the snp_leak_pages() in "crypto: ccp:
Handle the legacy TMR allocation when SNP is enabled" ?

Yes, probably should.

Quoted 9 lines
Quoted 9 lines
arch/x86/kvm/svm/sev.c +9 -0
+static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len) 
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct list_head *head = &sev->regions_list;
+       struct enc_region *i;
+
+       lockdep_assert_held(&kvm->lock);
+
+       list_for_each_entry(i, head, list) {
Quoted 8 lines

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 1963

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+               u64 start = i->uaddr;
+               u64 end = start + i->size;
+
+               if (start <= hva && end >= (hva + len))
+                       return true;
+       }

Given that usersapce could load sev->regions_list with any # of any sized regions. Should we add a cond_resched() like in sev_vm_destroy()?

Actually, is_hva_registered() is also called from PSC handler with kvm->lock mutex held. Even though it is a mutex, I am not really sure if it is a good idea to do cond_resched() with the kvm->lock mutex held ?

Quoted 6 lines

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2048

Quoted 3 lines
arch/x86/kvm/svm/sev.c +3 -0
+e_unpin:
+       /* Content of memory is updated, mark pages dirty */
+       for (i = 0; i < n; i++) {

Since |n| is not only a loop variable but actually carries the number of private pages over to e_unpin can we use a more descriptive name? How about something like 'nprivate_pages'?

Yes.

Thanks,
Ashish

#225 lore raw

Re: [PATCH Part2 v6 26/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

In reply to Ashish Kalra
Quoted 6 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_UPDATE command can be used to insert data into the guest's memory. The data is encrypted with the cryptographic context created with the KVM_SEV_SNP_LAUNCH_START.

Quoted 6 lines

In addition to the inserting data, it can insert a two special pages into the guests memory: the secrets page and the CPUID page.

While terminating the guest, reclaim the guest pages added in the RMP table. If the reclaim fails, then the page is no longer safe to be released back to the system and leak them.

From this paragraph I get a picture that reclaimer is failing "all the time", and that is totally normal and legit behaviour. Is this the case?

Stimuli/conditions/something is mandatory if failure is mentioned in any context.

Quoted 316 lines

For more information see the SEV-SNP specification.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  29 +++
 arch/x86/kvm/svm/sev.c                        | 187 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  19 ++
 3 files changed, 235 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +29 -0
index 878711f2dca6..62abd5c1f72b 100644
@@ -486,6 +486,35 @@ Returns: 0 on success, -negative on error
 
 See the SEV-SNP specification for further detail on the launch input.
 
+20. KVM_SNP_LAUNCH_UPDATE
+-------------------------
+
+The KVM_SNP_LAUNCH_UPDATE is used for encrypting a memory region. It also
+calculates a measurement of the memory contents. The measurement is a signature
+of the memory contents that can be sent to the guest owner as an attestation
+that the memory was encrypted correctly by the firmware.
+
+Parameters (in): struct  kvm_snp_launch_update
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_update {
+                __u64 start_gfn;        /* Guest page number to start from. */
+                __u64 uaddr;            /* userspace address need to be encrypted */
+                __u32 len;              /* length of memory region */
+                __u8 imi_page;          /* 1 if memory is part of the IMI */
+                __u8 page_type;         /* page type */
+                __u8 vmpl3_perms;       /* VMPL3 permission mask */
+                __u8 vmpl2_perms;       /* VMPL2 permission mask */
+                __u8 vmpl1_perms;       /* VMPL1 permission mask */
+        };
+
+See the SEV-SNP spec for further details on how to build the VMPL permission
+mask and page type.
+
+
 References
 ==========
 
arch/x86/kvm/svm/sev.c +187 -0
index 41b83aa6b5f4..b5f0707d7ed6 100644
@@ -18,6 +18,7 @@
 #include <linux/processor.h>
 #include <linux/trace_events.h>
 #include <linux/hugetlb.h>
+#include <linux/sev.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
@@ -233,6 +234,49 @@ static void sev_decommission(unsigned int handle)
 	sev_guest_decommission(&decommission, NULL);
 }
 
+static inline void snp_leak_pages(u64 pfn, enum pg_level level)
+{
+	unsigned int npages = page_level_size(level) >> PAGE_SHIFT;
+
+	WARN(1, "psc failed pfn 0x%llx pages %d (leaking)\n", pfn, npages);
+
+	while (npages) {
+		memory_failure(pfn, 0);
+		dump_rmpentry(pfn);
+		npages--;
+		pfn++;
+	}
+}
+
+static int snp_page_reclaim(u64 pfn)
+{
+	struct sev_data_snp_page_reclaim data = {0};
+	int err, rc;
+
+	data.paddr = __sme_set(pfn << PAGE_SHIFT);
+	rc = snp_guest_page_reclaim(&data, &err);
+	if (rc) {
+		/*
+		 * If the reclaim failed, then page is no longer safe
+		 * to use.
+		 */
+		snp_leak_pages(pfn, PG_LEVEL_4K);
+	}
+
+	return rc;
+}
+
+static int host_rmp_make_shared(u64 pfn, enum pg_level level, bool leak)
+{
+	int rc;
+
+	rc = rmp_make_shared(pfn, level);
+	if (rc && leak)
+		snp_leak_pages(pfn, level);
+
+	return rc;
+}
+
 static void sev_unbind_asid(struct kvm *kvm, unsigned int handle)
 {
 	struct sev_data_deactivate deactivate;
@@ -1902,6 +1946,123 @@ static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	return rc;
 }
 
+static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct list_head *head = &sev->regions_list;
+	struct enc_region *i;
+
+	lockdep_assert_held(&kvm->lock);
+
+	list_for_each_entry(i, head, list) {
+		u64 start = i->uaddr;
+		u64 end = start + i->size;
+
+		if (start <= hva && end >= (hva + len))
+			return true;
+	}
+
+	return false;
+}
+
+static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_update data = {0};
+	struct kvm_sev_snp_launch_update params;
+	unsigned long npages, pfn, n = 0;
+	int *error = &argp->error;
+	struct page **inpages;
+	int ret, i, level;
+	u64 gfn;
+
+	if (!sev_snp_guest(kvm))
+		return -ENOTTY;
+
+	if (!sev->snp_context)
+		return -EINVAL;
+
+	if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+		return -EFAULT;
+
+	/* Verify that the specified address range is registered. */
+	if (!is_hva_registered(kvm, params.uaddr, params.len))
+		return -EINVAL;
+
+	/*
+	 * The userspace memory is already locked so technically we don't
+	 * need to lock it again. Later part of the function needs to know
+	 * pfn so call the sev_pin_memory() so that we can get the list of
+	 * pages to iterate through.
+	 */
+	inpages = sev_pin_memory(kvm, params.uaddr, params.len, &npages, 1);
+	if (!inpages)
+		return -ENOMEM;
+
+	/*
+	 * Verify that all the pages are marked shared in the RMP table before
+	 * going further. This is avoid the cases where the userspace may try
+	 * updating the same page twice.
+	 */
+	for (i = 0; i < npages; i++) {
+		if (snp_lookup_rmpentry(page_to_pfn(inpages[i]), &level) != 0) {
+			sev_unpin_memory(kvm, inpages, npages);
+			return -EFAULT;
+		}
+	}
+
+	gfn = params.start_gfn;
+	level = PG_LEVEL_4K;
+	data.gctx_paddr = __psp_pa(sev->snp_context);
+
+	for (i = 0; i < npages; i++) {
+		pfn = page_to_pfn(inpages[i]);
+
+		ret = rmp_make_private(pfn, gfn << PAGE_SHIFT, level, sev_get_asid(kvm), true);
+		if (ret) {
+			ret = -EFAULT;
+			goto e_unpin;
+		}
+
+		n++;
+		data.address = __sme_page_pa(inpages[i]);
+		data.page_size = X86_TO_RMP_PG_LEVEL(level);
+		data.page_type = params.page_type;
+		data.vmpl3_perms = params.vmpl3_perms;
+		data.vmpl2_perms = params.vmpl2_perms;
+		data.vmpl1_perms = params.vmpl1_perms;
+		ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE, &data, error);
+		if (ret) {
+			/*
+			 * If the command failed then need to reclaim the page.
+			 */
+			snp_page_reclaim(pfn);
+			goto e_unpin;
+		}
+
+		gfn++;
+	}
+
+e_unpin:
+	/* Content of memory is updated, mark pages dirty */
+	for (i = 0; i < n; i++) {
+		set_page_dirty_lock(inpages[i]);
+		mark_page_accessed(inpages[i]);
+
+		/*
+		 * If its an error, then update RMP entry to change page ownership
+		 * to the hypervisor.
+		 */
+		if (ret)
+			host_rmp_make_shared(pfn, level, true);
+	}
+
+	/* Unlock the user pages */
+	sev_unpin_memory(kvm, inpages, npages);
+
+	return ret;
+}
+
 int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_sev_cmd sev_cmd;
@@ -1995,6 +2156,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 	case KVM_SEV_SNP_LAUNCH_START:
 		r = snp_launch_start(kvm, &sev_cmd);
 		break;
+	case KVM_SEV_SNP_LAUNCH_UPDATE:
+		r = snp_launch_update(kvm, &sev_cmd);
+		break;
 	default:
 		r = -EINVAL;
 		goto out;
@@ -2113,6 +2277,29 @@ find_enc_region(struct kvm *kvm, struct kvm_enc_region *range)
 static void __unregister_enc_region_locked(struct kvm *kvm,
 					   struct enc_region *region)
 {
+	unsigned long i, pfn;
+	int level;
+
+	/*
+	 * The guest memory pages are assigned in the RMP table. Unassign it
+	 * before releasing the memory.
+	 */
+	if (sev_snp_guest(kvm)) {
+		for (i = 0; i < region->npages; i++) {
+			pfn = page_to_pfn(region->pages[i]);
+
+			if (!snp_lookup_rmpentry(pfn, &level))
+				continue;
+
+			cond_resched();
+
+			if (level > PG_LEVEL_4K)
+				pfn &= ~(KVM_PAGES_PER_HPAGE(PG_LEVEL_2M) - 1);
+
+			host_rmp_make_shared(pfn, level, true);
+		}
+	}
+
 	sev_unpin_memory(kvm, region->pages, region->npages);
 	list_del(&region->list);
 	kfree(region);
include/uapi/linux/kvm.h +19 -0
index 0cb119d66ae5..9b36b07414ea 100644
@@ -1813,6 +1813,7 @@ enum sev_cmd_id {
 	/* SNP specific commands */
 	KVM_SEV_SNP_INIT,
 	KVM_SEV_SNP_LAUNCH_START,
+	KVM_SEV_SNP_LAUNCH_UPDATE,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1929,6 +1930,24 @@ struct kvm_sev_snp_launch_start {
 	__u8 pad[6];
 };
 
+#define KVM_SEV_SNP_PAGE_TYPE_NORMAL		0x1
+#define KVM_SEV_SNP_PAGE_TYPE_VMSA		0x2
+#define KVM_SEV_SNP_PAGE_TYPE_ZERO		0x3
+#define KVM_SEV_SNP_PAGE_TYPE_UNMEASURED	0x4
+#define KVM_SEV_SNP_PAGE_TYPE_SECRETS		0x5
+#define KVM_SEV_SNP_PAGE_TYPE_CPUID		0x6
+
+struct kvm_sev_snp_launch_update {
+	__u64 start_gfn;
+	__u64 uaddr;
+	__u32 len;
+	__u8 imi_page;
+	__u8 page_type;
+	__u8 vmpl3_perms;
+	__u8 vmpl2_perms;
+	__u8 vmpl1_perms;
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1

BR, Jarkko

#226 lore raw

Re: [PATCH Part2 v6 26/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

In reply to Ashish Kalra
Quoted 8 lines
arch/x86/kvm/svm/sev.c +8 -0
+static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct list_head *head = &sev->regions_list;
+	struct enc_region *i;
+
+	lockdep_assert_held(&kvm->lock);
+

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 1966

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+	list_for_each_entry(i, head, list) {
+		u64 start = i->uaddr;
+		u64 end = start + i->size;
+
+		if (start <= hva && end >= (hva + len))
+			return true;
+	}
+
+	return false;
+}

Since KVM_MEMORY_ENCRYPT_REG_REGION should be called for every memory region the user gives to kvm, is the regions_list any different from kvm's memslots?

Quoted 24 lines
arch/x86/kvm/svm/sev.c +24 -0
+
+static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_update data = {0};
+	struct kvm_sev_snp_launch_update params;
+	unsigned long npages, pfn, n = 0;
+	int *error = &argp->error;
+	struct page **inpages;
+	int ret, i, level;
+	u64 gfn;
+
+	if (!sev_snp_guest(kvm))
+		return -ENOTTY;
+
+	if (!sev->snp_context)
+		return -EINVAL;
+
+	if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+		return -EFAULT;
+
+	/* Verify that the specified address range is registered. */
+	if (!is_hva_registered(kvm, params.uaddr, params.len))
+		return -EINVAL;

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2000

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+	/*
+	 * The userspace memory is already locked so technically we don't
+	 * need to lock it again. Later part of the function needs to know
+	 * pfn so call the sev_pin_memory() so that we can get the list of
+	 * pages to iterate through.
+	 */
+	inpages = sev_pin_memory(kvm, params.uaddr, params.len, &npages, 1);
+	if (!inpages)
+		return -ENOMEM;

sev_pin_memory will call pin_user_pages() which fails for PFNMAP vmas that you would get if you use memory allocated from an IO driver.
Using gfn_to_pfn instead will make this work with vmas backed by pages or raw pfn mappings.

Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.

author maintainer
#227 lore raw

RE: [PATCH Part2 v6 26/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_UPDATE command

[AMD Official Use Only - General]

Hello Sabin,

Quoted 12 lines
Quoted 12 lines
arch/x86/kvm/svm/sev.c +12 -0
+static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len) 
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct list_head *head = &sev->regions_list;
+	struct enc_region *i;
+
+	lockdep_assert_held(&kvm->lock);
+
+	list_for_each_entry(i, head, list) {
+		u64 start = i->uaddr;
+		u64 end = start + i->size;
+
Quoted 8 lines

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 1966

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+		if (start <= hva && end >= (hva + len))
+			return true;
+	}
+
+	return false;
+}

Since KVM_MEMORY_ENCRYPT_REG_REGION should be called for every memory region the user gives to kvm, is the regions_list any different from kvm's memslots?

Actually, the KVM_MEMORY_ENCRYPT_REG_REGION is being called and the regions_list is only being setup for the guest RAM blocks.

Quoted 29 lines
Quoted 29 lines
arch/x86/kvm/svm/sev.c +29 -0
+
+static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd 
+*argp) {
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_update data = {0};
+	struct kvm_sev_snp_launch_update params;
+	unsigned long npages, pfn, n = 0;
+	int *error = &argp->error;
+	struct page **inpages;
+	int ret, i, level;
+	u64 gfn;
+
+	if (!sev_snp_guest(kvm))
+		return -ENOTTY;
+
+	if (!sev->snp_context)
+		return -EINVAL;
+
+	if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+		return -EFAULT;
+
+	/* Verify that the specified address range is registered. */
+	if (!is_hva_registered(kvm, params.uaddr, params.len))
+		return -EINVAL;
+
+	/*
+	 * The userspace memory is already locked so technically we don't
+	 * need to lock it again. Later part of the function needs to know
+	 * pfn so call the sev_pin_memory() so that we can get the list of
Quoted 8 lines

arch/x86/kvm/svm/sev.c static int snp_launch_start(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2000

Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+	 * pages to iterate through.
+	 */
+	inpages = sev_pin_memory(kvm, params.uaddr, params.len, &npages, 1);
+	if (!inpages)
+		return -ENOMEM;

sev_pin_memory will call pin_user_pages() which fails for PFNMAP vmas that you would get if you use memory allocated from an IO driver. Using gfn_to_pfn instead will make this work with vmas backed by pages or raw pfn mappings.

All the guest memory is being allocated via the userspace VMM, so how and where will we get memory allocated from an IO driver ?

Thanks,
Ashish

author maintainer
#228 lore raw

PATCH v6 27/49 KVM: SVM: Mark the private vma unmerable for SEV-SNP guests

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled, the guest private pages are added in the RMP table; while adding the pages, the rmp_make_private() unmaps the pages from the direct map. If KSM attempts to access those unmapped pages then it will trigger #PF (page-not-present).

Encrypted guest pages cannot be shared between the process, so an userspace should not mark the region mergeable but to be safe, mark the process vma unmerable before adding the pages in the RMP table.

---
 arch/x86/kvm/svm/sev.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
arch/x86/kvm/svm/sev.c +32 -0
index b5f0707d7ed6..a9461d352eda 100644
@@ -19,11 +19,13 @@
 #include <linux/trace_events.h>
 #include <linux/hugetlb.h>
 #include <linux/sev.h>
+#include <linux/ksm.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
 #include <asm/fpu/xcr.h>
 #include <asm/sev.h>
+#include <asm/mman.h>
 
 #include "x86.h"
 #include "svm.h"
@@ -1965,6 +1967,30 @@ static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
 	return false;
 }
 
+static int snp_mark_unmergable(struct kvm *kvm, u64 start, u64 size)
+{
+	struct vm_area_struct *vma;
+	u64 end = start + size;
+	int ret;
+
+	do {
+		vma = find_vma_intersection(kvm->mm, start, end);
+		if (!vma) {
+			ret = -EINVAL;
+			break;
+		}
+
+		ret = ksm_madvise(vma, vma->vm_start, vma->vm_end,
+				  MADV_UNMERGEABLE, &vma->vm_flags);
+		if (ret)
+			break;
+
+		start = vma->vm_end;
+	} while (end > vma->vm_end);
+
+	return ret;
+}
+
 static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
 	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
@@ -1989,6 +2015,12 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	if (!is_hva_registered(kvm, params.uaddr, params.len))
 		return -EINVAL;
 
+	mmap_write_lock(kvm->mm);
+	ret = snp_mark_unmergable(kvm, params.uaddr, params.len);
+	mmap_write_unlock(kvm->mm);
+	if (ret)
+		return -EFAULT;
+
 	/*
 	 * The userspace memory is already locked so technically we don't
 	 * need to lock it again. Later part of the function needs to know
Signature · 1 line
2.25.1
#229 lore raw

Re: [PATCH Part2 v6 27/49] KVM: SVM: Mark the private vma unmerable for SEV-SNP guests

Quoted 10 lines · * Ashish Kalra ([email protected]) wrote:

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled, the guest private pages are added in the RMP table; while adding the pages, the rmp_make_private() unmaps the pages from the direct map. If KSM attempts to access those unmapped pages then it will trigger #PF (page-not-present).

Encrypted guest pages cannot be shared between the process, so an userspace should not mark the region mergeable but to be safe, mark the process vma unmerable before adding the pages in the RMP table.

^
Typo 'unmergable' (also in title)

Quoted 70 lines
---
 arch/x86/kvm/svm/sev.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
arch/x86/kvm/svm/sev.c +32 -0
index b5f0707d7ed6..a9461d352eda 100644
@@ -19,11 +19,13 @@
 #include <linux/trace_events.h>
 #include <linux/hugetlb.h>
 #include <linux/sev.h>
+#include <linux/ksm.h>
 
 #include <asm/pkru.h>
 #include <asm/trapnr.h>
 #include <asm/fpu/xcr.h>
 #include <asm/sev.h>
+#include <asm/mman.h>
 
 #include "x86.h"
 #include "svm.h"
@@ -1965,6 +1967,30 @@ static bool is_hva_registered(struct kvm *kvm, hva_t hva, size_t len)
 	return false;
 }
 
+static int snp_mark_unmergable(struct kvm *kvm, u64 start, u64 size)
+{
+	struct vm_area_struct *vma;
+	u64 end = start + size;
+	int ret;
+
+	do {
+		vma = find_vma_intersection(kvm->mm, start, end);
+		if (!vma) {
+			ret = -EINVAL;
+			break;
+		}
+
+		ret = ksm_madvise(vma, vma->vm_start, vma->vm_end,
+				  MADV_UNMERGEABLE, &vma->vm_flags);
+		if (ret)
+			break;
+
+		start = vma->vm_end;
+	} while (end > vma->vm_end);
+
+	return ret;
+}
+
 static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
 	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
@@ -1989,6 +2015,12 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	if (!is_hva_registered(kvm, params.uaddr, params.len))
 		return -EINVAL;
 
+	mmap_write_lock(kvm->mm);
+	ret = snp_mark_unmergable(kvm, params.uaddr, params.len);
+	mmap_write_unlock(kvm->mm);
+	if (ret)
+		return -EFAULT;
+
 	/*
 	 * The userspace memory is already locked so technically we don't
 	 * need to lock it again. Later part of the function needs to know
Signature · 1 line
2.25.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
#230 lore raw

Re: [PATCH Part2 v6 27/49] KVM: SVM: Mark the private vma unmerable for SEV-SNP guests

In reply to Ashish Kalra
Quoted 7 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled, the guest private pages are added in the RMP table; while adding the pages, the rmp_make_private() unmaps the pages from the direct map. If KSM attempts to access those unmapped pages then it will trigger #PF (page-not-present).

Quoted 5 lines

Encrypted guest pages cannot be shared between the process, so an userspace should not mark the region mergeable but to be safe, mark the process vma unmerable before adding the pages in the RMP table.

Note this doesn't really mark the vma unmergeable, rather it unmarks it as mergeable, and unmerges any already merged pages.
Which seems like a good idea. Is snp_launch_update() the only place that needs it or can private pages be added elsewhere too?

However, AFAICS nothing stops userspace to do another madvise(MADV_MERGEABLE) afterwards, so we should make somehow sure that ksm will still be prevented, as we should protect the kernel even from a buggy userspace. So either we stop it with a flag at vma level (see ksm_madvise() for which flags currently stop it), or page level - currently only PageAnon() pages are handled. The vma level is probably easier/cheaper.

It's also possible that this will solve itself with the switch to UPM as those vma's or pages might be incompatible with ksm naturally (didn't check closely), and then this patch can be just dropped. But we should double-check.

author maintainer
#231 lore raw

PATCH v6 28/49 KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

If its an SNP guest, then VMSA was added in the RMP entry as a guest owned page and also removed from the kernel direct map so flush it later after it is transitioned back to hypervisor state and restored in the direct map.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  22 ++++
 arch/x86/kvm/svm/sev.c                        | 119 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  14 +++
 3 files changed, 155 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +22 -0
index 62abd5c1f72b..750162cff87b 100644
@@ -514,6 +514,28 @@ Returns: 0 on success, -negative on error
 See the SEV-SNP spec for further details on how to build the VMPL permission
 mask and page type.
 
+21. KVM_SNP_LAUNCH_FINISH
+-------------------------
+
+After completion of the SNP guest launch flow, the KVM_SNP_LAUNCH_FINISH command can be
+issued to make the guest ready for the execution.
+
+Parameters (in): struct kvm_sev_snp_launch_finish
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_finish {
+                __u64 id_block_uaddr;
+                __u64 id_auth_uaddr;
+                __u8 id_block_en;
+                __u8 auth_key_en;
+                __u8 host_data[32];
+        };
+
+
+See SEV-SNP specification for further details on launch finish input parameters.
 
 References
 ==========
arch/x86/kvm/svm/sev.c +119 -0

142 lines not on this page · raw message

include/uapi/linux/kvm.h +14 -0
index 9b36b07414ea..5a4662716b6a 100644
@@ -1814,6 +1814,7 @@ enum sev_cmd_id {
 	KVM_SEV_SNP_INIT,
 	KVM_SEV_SNP_LAUNCH_START,
 	KVM_SEV_SNP_LAUNCH_UPDATE,
+	KVM_SEV_SNP_LAUNCH_FINISH,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1948,6 +1949,19 @@ struct kvm_sev_snp_launch_update {
 	__u8 vmpl1_perms;
 };
 
+#define KVM_SEV_SNP_ID_BLOCK_SIZE	96
+#define KVM_SEV_SNP_ID_AUTH_SIZE	4096
+#define KVM_SEV_SNP_FINISH_DATA_SIZE	32
+
+struct kvm_sev_snp_launch_finish {
+	__u64 id_block_uaddr;
+	__u64 id_auth_uaddr;
+	__u8 id_block_en;
+	__u8 auth_key_en;
+	__u8 host_data[KVM_SEV_SNP_FINISH_DATA_SIZE];
+	__u8 pad[6];
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1
#232 lore raw

Re: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

Quoted 6 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

Quoted 7 lines

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

If its an SNP guest, then VMSA was added in the RMP entry as a guest owned page and also removed from the kernel direct map so flush it later after it is transitioned back to hypervisor state and restored in the direct map.

Given the guest uses the SNP NAE AP boot protocol we were expecting that there would be some option to add vCPUs to the VM but mark them as "pending AP boot creation protocol" state. This would allow the LaunchDigest of a VM doesn't change just because its vCPU count changes. Would it be possible to add a new add an argument to KVM_SNP_LAUNCH_FINISH to tell it which vCPUs to LAUNCH_UPDATE VMSA pages for or similarly a new argument for KVM_CREATE_VCPU?

Quoted 51 lines
---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  22 ++++
 arch/x86/kvm/svm/sev.c                        | 119 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  14 +++
 3 files changed, 155 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +22 -0
index 62abd5c1f72b..750162cff87b 100644
@@ -514,6 +514,28 @@ Returns: 0 on success, -negative on error
 See the SEV-SNP spec for further details on how to build the VMPL permission
 mask and page type.
 
+21. KVM_SNP_LAUNCH_FINISH
+-------------------------
+
+After completion of the SNP guest launch flow, the KVM_SNP_LAUNCH_FINISH command can be
+issued to make the guest ready for the execution.
+
+Parameters (in): struct kvm_sev_snp_launch_finish
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_finish {
+                __u64 id_block_uaddr;
+                __u64 id_auth_uaddr;
+                __u8 id_block_en;
+                __u8 auth_key_en;
+                __u8 host_data[32];
+        };
+
+
+See SEV-SNP specification for further details on launch finish input parameters.
 
 References
 ==========
arch/x86/kvm/svm/sev.c +1 -0
index a9461d352eda..a5b90469683f 100644
@@ -2095,6 +2095,106 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
        return ret;
 }
 
+static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)

arch/x86/kvm/svm/sev.c static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2108

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_launch_update data = {};
+       int i, ret;
+
+       data.gctx_paddr = __psp_pa(sev->snp_context);
+       data.page_type = SNP_PAGE_TYPE_VMSA;
+
+       for (i = 0; i < kvm->created_vcpus; i++) {
+               struct vcpu_svm *svm = to_svm(xa_load(&kvm->vcpu_array, i));

Why are we iterating over |created_vcpus| rather than using kvm_for_each_vcpu?

arch/x86/kvm/svm/sev.c static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2114

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+               u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+               /* Perform some pre-encryption checks against the VMSA */
+               ret = sev_es_sync_vmsa(svm);
+               if (ret)
+                       return ret;

Do we need to take the 'vcpu->mutex' lock before modifying the vcpu,like we do for SEV-ES in sev_launch_update_vmsa()?

Quoted 103 lines
arch/x86/kvm/svm/sev.c +89 -0
+
+               /* Transition the VMSA page to a firmware state. */
+               ret = rmp_make_private(pfn, -1, PG_LEVEL_4K, sev->asid, true);
+               if (ret)
+                       return ret;
+
+               /* Issue the SNP command to encrypt the VMSA */
+               data.address = __sme_pa(svm->sev_es.vmsa);
+               ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE,
+                                     &data, &argp->error);
+               if (ret) {
+                       snp_page_reclaim(pfn);
+                       return ret;
+               }
+
+               svm->vcpu.arch.guest_state_protected = true;
+       }
+
+       return 0;
+}
+
+static int snp_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_launch_finish *data;
+       void *id_block = NULL, *id_auth = NULL;
+       struct kvm_sev_snp_launch_finish params;
+       int ret;
+
+       if (!sev_snp_guest(kvm))
+               return -ENOTTY;
+
+       if (!sev->snp_context)
+               return -EINVAL;
+
+       if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+               return -EFAULT;
+
+       /* Measure all vCPUs using LAUNCH_UPDATE before we finalize the launch flow. */
+       ret = snp_launch_update_vmsa(kvm, argp);
+       if (ret)
+               return ret;
+
+       data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+       if (!data)
+               return -ENOMEM;
+
+       if (params.id_block_en) {
+               id_block = psp_copy_user_blob(params.id_block_uaddr, KVM_SEV_SNP_ID_BLOCK_SIZE);
+               if (IS_ERR(id_block)) {
+                       ret = PTR_ERR(id_block);
+                       goto e_free;
+               }
+
+               data->id_block_en = 1;
+               data->id_block_paddr = __sme_pa(id_block);
+       }
+
+       if (params.auth_key_en) {
+               id_auth = psp_copy_user_blob(params.id_auth_uaddr, KVM_SEV_SNP_ID_AUTH_SIZE);
+               if (IS_ERR(id_auth)) {
+                       ret = PTR_ERR(id_auth);
+                       goto e_free_id_block;
+               }
+
+               data->auth_key_en = 1;
+               data->id_auth_paddr = __sme_pa(id_auth);
+       }
+
+       data->gctx_paddr = __psp_pa(sev->snp_context);
+       ret = sev_issue_cmd(kvm, SEV_CMD_SNP_LAUNCH_FINISH, data, &argp->error);
+
+       kfree(id_auth);
+
+e_free_id_block:
+       kfree(id_block);
+
+e_free:
+       kfree(data);
+
+       return ret;
+}
+
 int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
        struct kvm_sev_cmd sev_cmd;
@@ -2191,6 +2291,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
        case KVM_SEV_SNP_LAUNCH_UPDATE:
                r = snp_launch_update(kvm, &sev_cmd);
                break;
+       case KVM_SEV_SNP_LAUNCH_FINISH:
+               r = snp_launch_finish(kvm, &sev_cmd);
+               break;
        default:
                r = -EINVAL;
                goto out;
@@ -2696,11 +2799,27 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 
        svm = to_svm(vcpu);
 
+       /*
+        * If its an SNP guest, then VMSA was added in the RMP entry as
+        * a guest owned page. Transition the page to hypervisor state

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2814

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+        * before releasing it back to the system.
+        * Also the page is removed from the kernel direct map, so flush it
+        * later after it is transitioned back to hypervisor state and
+        * restored in the direct map.
+        */
+       if (sev_snp_guest(vcpu->kvm)) {
+               u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+               if (host_rmp_make_shared(pfn, PG_LEVEL_4K, false))
+                       goto skip_vmsa_free;

Why not call host_rmp_make_shared with leak==true? This old VMSA page is now unusable IIUC.

Quoted 46 lines
arch/x86/kvm/svm/sev.c +3 -0
+       }
+
        if (vcpu->arch.guest_state_protected)
                sev_flush_encrypted_page(vcpu, svm->sev_es.vmsa);
 
        __free_page(virt_to_page(svm->sev_es.vmsa));
 
+skip_vmsa_free:
        if (svm->sev_es.ghcb_sa_free)
                kvfree(svm->sev_es.ghcb_sa);
 }
include/uapi/linux/kvm.h +14 -1
index 9b36b07414ea..5a4662716b6a 100644
@@ -1814,6 +1814,7 @@ enum sev_cmd_id {
        KVM_SEV_SNP_INIT,
        KVM_SEV_SNP_LAUNCH_START,
        KVM_SEV_SNP_LAUNCH_UPDATE,
+       KVM_SEV_SNP_LAUNCH_FINISH,
 
        KVM_SEV_NR_MAX,
 };
@@ -1948,6 +1949,19 @@ struct kvm_sev_snp_launch_update {
        __u8 vmpl1_perms;
 };
 
+#define KVM_SEV_SNP_ID_BLOCK_SIZE      96
+#define KVM_SEV_SNP_ID_AUTH_SIZE       4096
+#define KVM_SEV_SNP_FINISH_DATA_SIZE   32
+
+struct kvm_sev_snp_launch_finish {
+       __u64 id_block_uaddr;
+       __u64 id_auth_uaddr;
+       __u8 id_block_en;
+       __u8 auth_key_en;
+       __u8 host_data[KVM_SEV_SNP_FINISH_DATA_SIZE];
+       __u8 pad[6];
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU    (1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3         (1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX       (1 << 2)
--

2.25.1

author maintainer
#233 lore raw

RE: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

[AMD Official Use Only - General]

Hello Peter,

Quoted 7 lines
Quoted 5 lines

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

Given the guest uses the SNP NAE AP boot protocol we were expecting that there would be some option to add vCPUs to the VM but mark them as "pending AP boot creation protocol" state. This would allow the LaunchDigest of a VM doesn't change >just because its vCPU count changes. Would it be possible to add a new add an argument to KVM_SNP_LAUNCH_FINISH to tell it which vCPUs to LAUNCH_UPDATE VMSA pages for or similarly a new argument for KVM_CREATE_VCPU?

But don't we want/need to measure all vCPUs using LAUNCH_UPDATE_VMSA before we issue SNP_LAUNCH_FINISH command ?

If we are going to add vCPUs and mark them as "pending AP boot creation" state then how are we going to do LAUNCH_UPDATE_VMSAs for them after SNP_LAUNCH_FINISH ?

int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd

Quoted 5 lines
Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+*argp) {
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       struct sev_data_snp_launch_update data = {};
+       int i, ret;
+
Quoted 8 lines

arch/x86/kvm/svm/sev.c static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+       data.gctx_paddr = __psp_pa(sev->snp_context);
+       data.page_type = SNP_PAGE_TYPE_VMSA;
+
+       for (i = 0; i < kvm->created_vcpus; i++) {
+               struct vcpu_svm *svm = 
+ to_svm(xa_load(&kvm->vcpu_array, i));

Why are we iterating over |created_vcpus| rather than using kvm_for_each_vcpu?

Yes we should be using kvm_for_each_vcpu(), that will also help avoid touching implementation specific details and hide complexities such as xa_load(), locking requirements, etc.

Additionally, kvm_for_each_vcpu() works on online_cpus, but I think that is what we should be considering at LAUNCH_UPDATE_VMSA time, via-a-vis created_vcpus.

Quoted 8 lines

arch/x86/kvm/svm/sev.c static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2114

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+               u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+               /* Perform some pre-encryption checks against the VMSA */
+               ret = sev_es_sync_vmsa(svm);
+               if (ret)
+                       return ret;

Do we need to take the 'vcpu->mutex' lock before modifying the vcpu,like we do for SEV-ES in sev_launch_update_vmsa()?

This is using the per-cpu vcpu_svm structure, but we may need to guard against the KVM vCPU ioctl requests, so yes it is safer to take the 'vcpu->mutex' lock here.

Quoted 14 lines

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2814

Quoted 13 lines
arch/x86/kvm/svm/sev.c +13 -0
+       /*
+        * If its an SNP guest, then VMSA was added in the RMP entry as
+        * a guest owned page. Transition the page to hypervisor state
+        * before releasing it back to the system.
+        * Also the page is removed from the kernel direct map, so flush it
+        * later after it is transitioned back to hypervisor state and
+        * restored in the direct map.
+        */
+       if (sev_snp_guest(vcpu->kvm)) {
+               u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+               if (host_rmp_make_shared(pfn, PG_LEVEL_4K, false))
+                       goto skip_vmsa_free;
Quoted 1 line

Why not call host_rmp_make_shared with leak==true? This old VMSA page is now unusable IIUC.

Yes the old VMSA page is now unavailable and lost, so makes sense to call host_rmp_make_shared() with leak==true.

Thanks,
Ashish

#234 lore raw

Re: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

Quoted 8 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 3 lines
Quoted 3 lines

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

Given the guest uses the SNP NAE AP boot protocol we were expecting that there would be some option to add vCPUs to the VM but mark them as "pending AP boot creation protocol" state. This would allow the LaunchDigest of a VM doesn't change >just because its vCPU count changes. Would it be possible to add a new add an argument to KVM_SNP_LAUNCH_FINISH to tell it which vCPUs to LAUNCH_UPDATE VMSA pages for or similarly a new argument for KVM_CREATE_VCPU?

But don't we want/need to measure all vCPUs using LAUNCH_UPDATE_VMSA before we issue SNP_LAUNCH_FINISH command ?

If we are going to add vCPUs and mark them as "pending AP boot creation" state then how are we going to do LAUNCH_UPDATE_VMSAs for them after SNP_LAUNCH_FINISH ?

If I understand correctly we don't need or even want the APs to be LAUNCH_UPDATE_VMSA'd. LAUNCH_UPDATEing all the VMSAs causes VMs with different numbers of vCPUs to have different launch digests. Its my understanding the SNP AP Creation protocol was to solve this so that VMs with different vcpu counts have the same launch digest.

Looking at patch "[Part2,v6,44/49] KVM: SVM: Support SEV-SNP AP Creation NAE event" and section "4.1.9 SNP AP Creation" of the GHCB spec. There is no need to mark the LAUNCH_UPDATE the AP's VMSA or mark the vCPUs runnable. Instead we can do that only for the BSP. Then in the guest UEFI the BSP can: create new VMSAs from guest pages, RMPADJUST them into the RMP state VMSA, then use the SNP AP Creation NAE to get the hypervisor to mark them runnable. I believe this is all setup in the UEFI patch:
https://www.mail-archive.com/[email protected]/msg38460.html.

author maintainer
#235 lore raw

RE: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

[AMD Official Use Only - General]

Hello Peter,

Quoted 10 lines
Quoted 5 lines
Quoted 1 line

Given the guest uses the SNP NAE AP boot protocol we were expecting that there would be some option to add vCPUs to the VM but mark them as "pending AP boot creation protocol" state. This would allow the LaunchDigest of a VM doesn't change >just because its vCPU count changes. Would it be possible to add a new add an argument to KVM_SNP_LAUNCH_FINISH to tell it which vCPUs to LAUNCH_UPDATE VMSA pages for or similarly a new argument for KVM_CREATE_VCPU?

But don't we want/need to measure all vCPUs using LAUNCH_UPDATE_VMSA before we issue SNP_LAUNCH_FINISH command ?

If we are going to add vCPUs and mark them as "pending AP boot creation" state then how are we going to do LAUNCH_UPDATE_VMSAs for them after SNP_LAUNCH_FINISH ?

If I understand correctly we don't need or even want the APs to be LAUNCH_UPDATE_VMSA'd. LAUNCH_UPDATEing all the VMSAs causes VMs with different numbers of vCPUs to have different launch digests. Its my understanding the SNP AP >Creation protocol was to solve this so that VMs with different vcpu counts have the same launch digest.

Looking at patch "[Part2,v6,44/49] KVM: SVM: Support SEV-SNP AP Creation NAE event" and section "4.1.9 SNP AP Creation" of the GHCB spec. There is no need to mark the LAUNCH_UPDATE the AP's VMSA or mark the vCPUs runnable. Instead we >can do that only for the BSP. Then in the guest UEFI the BSP can: create new VMSAs from guest pages, RMPADJUST them into the RMP state VMSA, then use the SNP AP Creation NAE to get the hypervisor to mark them runnable. I believe this is all >setup in the UEFI patch:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fdevel%40edk2.groups.io%2Fmsg38460.html&amp;data=05%7C01%7CAshish.Kalra%40amd.com%7Ca40178ac6f284a9e33aa08da64152baa%>7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637932339382401133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=ZaiHHo9S24f9BB6E%>2FjexOt5TdKJQXxQDJI5QoYdDDHc%3D&amp;reserved=0.

Yes, I discussed the same with Tom, and this will be supported going forward, only the BSP will need to go through the LAUNCH_UPDATE_VMSA and at runtime the guest can dynamically create more APs using the SNP AP Creation NAE event.

Now, coming back to the original question, why do we need a separate vCPU count argument for SNP_LAUNCH_FINISH, won't the statically created vCPUs in kvm->created_vcpus/online_vcpus be sufficient for that, any dynamically created vCPU's won't be part of the initial measurement or LaunchDigest of the VM, right ?

Thanks,
Ashish

#236 lore raw

Re: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

Quoted 13 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Given the guest uses the SNP NAE AP boot protocol we were expecting that there would be some option to add vCPUs to the VM but mark them as "pending AP boot creation protocol" state. This would allow the LaunchDigest of a VM doesn't change >just because its vCPU count changes. Would it be possible to add a new add an argument to KVM_SNP_LAUNCH_FINISH to tell it which vCPUs to LAUNCH_UPDATE VMSA pages for or similarly a new argument for KVM_CREATE_VCPU?

But don't we want/need to measure all vCPUs using LAUNCH_UPDATE_VMSA before we issue SNP_LAUNCH_FINISH command ?

If we are going to add vCPUs and mark them as "pending AP boot creation" state then how are we going to do LAUNCH_UPDATE_VMSAs for them after SNP_LAUNCH_FINISH ?

If I understand correctly we don't need or even want the APs to be LAUNCH_UPDATE_VMSA'd. LAUNCH_UPDATEing all the VMSAs causes VMs with different numbers of vCPUs to have different launch digests. Its my understanding the SNP AP >Creation protocol was to solve this so that VMs with different vcpu counts have the same launch digest.

Quoted 7 lines
Quoted 2 lines

Looking at patch "[Part2,v6,44/49] KVM: SVM: Support SEV-SNP AP Creation NAE event" and section "4.1.9 SNP AP Creation" of the GHCB spec. There is no need to mark the LAUNCH_UPDATE the AP's VMSA or mark the vCPUs runnable. Instead we >can do that only for the BSP. Then in the guest UEFI the BSP can: create new VMSAs from guest pages, RMPADJUST them into the RMP state VMSA, then use the SNP AP Creation NAE to get the hypervisor to mark them runnable. I believe this is all >setup in the UEFI patch:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fdevel%40edk2.groups.io%2Fmsg38460.html&amp;data=05%7C01%7CAshish.Kalra%40amd.com%7Ca40178ac6f284a9e33aa08da64152baa%>7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637932339382401133%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=ZaiHHo9S24f9BB6E%>2FjexOt5TdKJQXxQDJI5QoYdDDHc%3D&amp;reserved=0.

Yes, I discussed the same with Tom, and this will be supported going forward, only the BSP will need to go through the LAUNCH_UPDATE_VMSA and at runtime the guest can dynamically create more APs using the SNP AP Creation NAE event.

Now, coming back to the original question, why do we need a separate vCPU count argument for SNP_LAUNCH_FINISH, won't the statically created vCPUs in kvm->created_vcpus/online_vcpus be sufficient for that, any dynamically created vCPU's won't be part of the initial measurement or LaunchDigest of the VM, right ?

Are you suggesting that QEMU will KVM_CREATE_VCPU the BSP, then LAUNCH_FINISH, then KVM_CREATE_VCPU all the APs to their VMSAs were not LAUNCH_UPDATED? If so, it seems annoying to have to create vCPUs at different times to get their VMSAs into different states. That's why I was suggesting some other mechanism so we can continue to KVM_CREATE_VCPU all the vCPUs at the same time.

Quoted 3 lines

Thanks,
Ashish

maintainer
#237 lore raw

Re: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

In reply to Peter Gonda
Quoted 24 lines · Peter Gonda
Quoted 8 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 3 lines
Quoted 3 lines

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

Given the guest uses the SNP NAE AP boot protocol we were expecting that there would be some option to add vCPUs to the VM but mark them as "pending AP boot creation protocol" state. This would allow the LaunchDigest of a VM doesn't change >just because its vCPU count changes. Would it be possible to add a new add an argument to KVM_SNP_LAUNCH_FINISH to tell it which vCPUs to LAUNCH_UPDATE VMSA pages for or similarly a new argument for KVM_CREATE_VCPU?

But don't we want/need to measure all vCPUs using LAUNCH_UPDATE_VMSA before we issue SNP_LAUNCH_FINISH command ?

If we are going to add vCPUs and mark them as "pending AP boot creation" state then how are we going to do LAUNCH_UPDATE_VMSAs for them after SNP_LAUNCH_FINISH ?

If I understand correctly we don't need or even want the APs to be LAUNCH_UPDATE_VMSA'd. LAUNCH_UPDATEing all the VMSAs causes VMs with different numbers of vCPUs to have different launch digests. Its my understanding the SNP AP Creation protocol was to solve this so that VMs with different vcpu counts have the same launch digest.

Quoted 9 lines

Looking at patch "[Part2,v6,44/49] KVM: SVM: Support SEV-SNP AP Creation NAE event" and section "4.1.9 SNP AP Creation" of the GHCB spec. There is no need to mark the LAUNCH_UPDATE the AP's VMSA or mark the vCPUs runnable. Instead we can do that only for the BSP. Then in the guest UEFI the BSP can: create new VMSAs from guest pages, RMPADJUST them into the RMP state VMSA, then use the SNP AP Creation NAE to get the hypervisor to mark them runnable. I believe this is all setup in the UEFI patch:
https://www.mail-archive.com/[email protected]/msg38460.html.

Not quite... there isn't a way to (easily) retrieve the APIC IDs for all of the vCPUs, which are required in order to use the AP Create event.

For this version of SNP, all of the vCPUs are measured and started by OVMF in the same way as SEV-ES. However, once the vCPUs have run, we now have the APIC ID associated with each vCPU and the AP Create event can be used going forward.

The SVSM support will introduce a new NAE event to the GHCB spec to retrieve all of the APIC IDs from the hypervisor. With that, then you would be able be required to perform a LAUNCH_UPDATE_VMSA against the BSP.

Thanks,
Tom

#238 lore raw

Re: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

Quoted 39 lines · Tom Lendacky
Quoted 24 lines · Peter Gonda
Quoted 8 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 3 lines
Quoted 3 lines

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

Given the guest uses the SNP NAE AP boot protocol we were expecting that there would be some option to add vCPUs to the VM but mark them as "pending AP boot creation protocol" state. This would allow the LaunchDigest of a VM doesn't change >just because its vCPU count changes. Would it be possible to add a new add an argument to KVM_SNP_LAUNCH_FINISH to tell it which vCPUs to LAUNCH_UPDATE VMSA pages for or similarly a new argument for KVM_CREATE_VCPU?

But don't we want/need to measure all vCPUs using LAUNCH_UPDATE_VMSA before we issue SNP_LAUNCH_FINISH command ?

If we are going to add vCPUs and mark them as "pending AP boot creation" state then how are we going to do LAUNCH_UPDATE_VMSAs for them after SNP_LAUNCH_FINISH ?

If I understand correctly we don't need or even want the APs to be LAUNCH_UPDATE_VMSA'd. LAUNCH_UPDATEing all the VMSAs causes VMs with different numbers of vCPUs to have different launch digests. Its my understanding the SNP AP Creation protocol was to solve this so that VMs with different vcpu counts have the same launch digest.

Quoted 9 lines

Looking at patch "[Part2,v6,44/49] KVM: SVM: Support SEV-SNP AP Creation NAE event" and section "4.1.9 SNP AP Creation" of the GHCB spec. There is no need to mark the LAUNCH_UPDATE the AP's VMSA or mark the vCPUs runnable. Instead we can do that only for the BSP. Then in the guest UEFI the BSP can: create new VMSAs from guest pages, RMPADJUST them into the RMP state VMSA, then use the SNP AP Creation NAE to get the hypervisor to mark them runnable. I believe this is all setup in the UEFI patch:
https://www.mail-archive.com/[email protected]/msg38460.html.

Not quite... there isn't a way to (easily) retrieve the APIC IDs for all of the vCPUs, which are required in order to use the AP Create event.

Quoted 8 lines

For this version of SNP, all of the vCPUs are measured and started by OVMF in the same way as SEV-ES. However, once the vCPUs have run, we now have the APIC ID associated with each vCPU and the AP Create event can be used going forward.

The SVSM support will introduce a new NAE event to the GHCB spec to retrieve all of the APIC IDs from the hypervisor. With that, then you would be able be required to perform a LAUNCH_UPDATE_VMSA against the BSP.

Thank you Tom I missed that we needed to run the APs to set up their APIC IDs for OVMF. Is there any reason we need to wait for the SVSM to do what you describe? Couldn't the OVMF use an NAE to get all the APIC IDs?

Quoted 4 lines

Thanks,
Tom

#239 lore raw

Re: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

In reply to Ashish Kalra
Quoted 7 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

Nit: for completeness sake it would nice to fully conclude whether LAUNCH_UPDATE is usable after LAUNCH_FINISH in this paragraph.

Quoted 19 lines

If its an SNP guest, then VMSA was added in the RMP entry as a guest owned page and also removed from the kernel direct map so flush it later after it is transitioned back to hypervisor state and restored in the direct map.

---
 .../virt/kvm/x86/amd-memory-encryption.rst    |  22 ++++
 arch/x86/kvm/svm/sev.c                        | 119 ++++++++++++++++++
 include/uapi/linux/kvm.h                      |  14 +++
 3 files changed, 155 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +0 -0
index 62abd5c1f72b..750162cff87b 100644

Documentation/virt/kvm/x86/amd-memory-encryption.rst Returns: 0 on success, -negative on error · line 521

Quoted 8 lines
Documentation/virt/kvm/x86/amd-memory-encryption.rst +5 -0
@@ -514,6 +514,28 @@ Returns: 0 on success, -negative on error
 See the SEV-SNP spec for further details on how to build the VMPL permission
 mask and page type.
 
+21. KVM_SNP_LAUNCH_FINISH
+-------------------------
+
+After completion of the SNP guest launch flow, the KVM_SNP_LAUNCH_FINISH command can be
+issued to make the guest ready for the execution.

Some remark about LAUNCH_UPDATE post-LAUNCH_FINISH would be nice.

Quoted 62 lines
Documentation/virt/kvm/x86/amd-memory-encryption.rst +17 -0
+
+Parameters (in): struct kvm_sev_snp_launch_finish
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_finish {
+                __u64 id_block_uaddr;
+                __u64 id_auth_uaddr;
+                __u8 id_block_en;
+                __u8 auth_key_en;
+                __u8 host_data[32];
+        };
+
+
+See SEV-SNP specification for further details on launch finish input parameters.
 
 References
 ==========
arch/x86/kvm/svm/sev.c +34 -0
index a9461d352eda..a5b90469683f 100644
@@ -2095,6 +2095,106 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	return ret;
 }
 
+static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_update data = {};
+	int i, ret;
+
+	data.gctx_paddr = __psp_pa(sev->snp_context);
+	data.page_type = SNP_PAGE_TYPE_VMSA;
+
+	for (i = 0; i < kvm->created_vcpus; i++) {
+		struct vcpu_svm *svm = to_svm(xa_load(&kvm->vcpu_array, i));
+		u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+		/* Perform some pre-encryption checks against the VMSA */
+		ret = sev_es_sync_vmsa(svm);
+		if (ret)
+			return ret;
+
+		/* Transition the VMSA page to a firmware state. */
+		ret = rmp_make_private(pfn, -1, PG_LEVEL_4K, sev->asid, true);
+		if (ret)
+			return ret;
+
+		/* Issue the SNP command to encrypt the VMSA */
+		data.address = __sme_pa(svm->sev_es.vmsa);
+		ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE,
+				      &data, &argp->error);
+		if (ret) {
+			snp_page_reclaim(pfn);
+			return ret;
+		}
+
+		svm->vcpu.arch.guest_state_protected = true;
+	}

arch/x86/kvm/svm/sev.c static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2141

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+	return 0;
+}
+
+static int snp_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_finish *data;
+	void *id_block = NULL, *id_auth = NULL;
+	struct kvm_sev_snp_launch_finish params;

Nit: "params" should be the 2nd declaration (reverse christmas tree order).

Quoted 132 lines
arch/x86/kvm/svm/sev.c +75 -0
+	int ret;
+
+	if (!sev_snp_guest(kvm))
+		return -ENOTTY;
+
+	if (!sev->snp_context)
+		return -EINVAL;
+
+	if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+		return -EFAULT;
+
+	/* Measure all vCPUs using LAUNCH_UPDATE before we finalize the launch flow. */
+	ret = snp_launch_update_vmsa(kvm, argp);
+	if (ret)
+		return ret;
+
+	data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+	if (!data)
+		return -ENOMEM;
+
+	if (params.id_block_en) {
+		id_block = psp_copy_user_blob(params.id_block_uaddr, KVM_SEV_SNP_ID_BLOCK_SIZE);
+		if (IS_ERR(id_block)) {
+			ret = PTR_ERR(id_block);
+			goto e_free;
+		}
+
+		data->id_block_en = 1;
+		data->id_block_paddr = __sme_pa(id_block);
+	}
+
+	if (params.auth_key_en) {
+		id_auth = psp_copy_user_blob(params.id_auth_uaddr, KVM_SEV_SNP_ID_AUTH_SIZE);
+		if (IS_ERR(id_auth)) {
+			ret = PTR_ERR(id_auth);
+			goto e_free_id_block;
+		}
+
+		data->auth_key_en = 1;
+		data->id_auth_paddr = __sme_pa(id_auth);
+	}
+
+	data->gctx_paddr = __psp_pa(sev->snp_context);
+	ret = sev_issue_cmd(kvm, SEV_CMD_SNP_LAUNCH_FINISH, data, &argp->error);
+
+	kfree(id_auth);
+
+e_free_id_block:
+	kfree(id_block);
+
+e_free:
+	kfree(data);
+
+	return ret;
+}
+
 int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	struct kvm_sev_cmd sev_cmd;
@@ -2191,6 +2291,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 	case KVM_SEV_SNP_LAUNCH_UPDATE:
 		r = snp_launch_update(kvm, &sev_cmd);
 		break;
+	case KVM_SEV_SNP_LAUNCH_FINISH:
+		r = snp_launch_finish(kvm, &sev_cmd);
+		break;
 	default:
 		r = -EINVAL;
 		goto out;
@@ -2696,11 +2799,27 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 
 	svm = to_svm(vcpu);
 
+	/*
+	 * If its an SNP guest, then VMSA was added in the RMP entry as
+	 * a guest owned page. Transition the page to hypervisor state
+	 * before releasing it back to the system.
+	 * Also the page is removed from the kernel direct map, so flush it
+	 * later after it is transitioned back to hypervisor state and
+	 * restored in the direct map.
+	 */
+	if (sev_snp_guest(vcpu->kvm)) {
+		u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+		if (host_rmp_make_shared(pfn, PG_LEVEL_4K, false))
+			goto skip_vmsa_free;
+	}
+
 	if (vcpu->arch.guest_state_protected)
 		sev_flush_encrypted_page(vcpu, svm->sev_es.vmsa);
 
 	__free_page(virt_to_page(svm->sev_es.vmsa));
 
+skip_vmsa_free:
 	if (svm->sev_es.ghcb_sa_free)
 		kvfree(svm->sev_es.ghcb_sa);
 }
include/uapi/linux/kvm.h +14 -0
index 9b36b07414ea..5a4662716b6a 100644
@@ -1814,6 +1814,7 @@ enum sev_cmd_id {
 	KVM_SEV_SNP_INIT,
 	KVM_SEV_SNP_LAUNCH_START,
 	KVM_SEV_SNP_LAUNCH_UPDATE,
+	KVM_SEV_SNP_LAUNCH_FINISH,
 
 	KVM_SEV_NR_MAX,
 };
@@ -1948,6 +1949,19 @@ struct kvm_sev_snp_launch_update {
 	__u8 vmpl1_perms;
 };
 
+#define KVM_SEV_SNP_ID_BLOCK_SIZE	96
+#define KVM_SEV_SNP_ID_AUTH_SIZE	4096
+#define KVM_SEV_SNP_FINISH_DATA_SIZE	32
+
+struct kvm_sev_snp_launch_finish {
+	__u64 id_block_uaddr;
+	__u64 id_auth_uaddr;
+	__u8 id_block_en;
+	__u8 auth_key_en;
+	__u8 host_data[KVM_SEV_SNP_FINISH_DATA_SIZE];
+	__u8 pad[6];
+};
+
 #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
 #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
 #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
Signature · 1 line
2.25.1

BR, Jarkko

#240 lore raw

PATCH v6 1/1 ] KVM: SEV: fix snp_launch_finish

In reply to Ashish Kalra

The params.auth_key_en indicator does _not_ specify, whether an ID_AUTH struct should be sent or not, but, wheter the ID_AUTH struct contains an author key or not. The firmware always expects an ID_AUTH block.

---
 arch/x86/kvm/svm/sev.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/sev.c +4 -3
index 85357dc4d231..5cf4be6a33ba 100644
@@ -2242,17 +2242,18 @@ static int snp_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 		data->id_block_en = 1;
 		data->id_block_paddr = __sme_pa(id_block);
-	}
 
-	if (params.auth_key_en) {
 		id_auth = psp_copy_user_blob(params.id_auth_uaddr, KVM_SEV_SNP_ID_AUTH_SIZE);
 		if (IS_ERR(id_auth)) {
 			ret = PTR_ERR(id_auth);
 			goto e_free_id_block;
 		}
 
-		data->auth_key_en = 1;
 		data->id_auth_paddr = __sme_pa(id_auth);
+
+		if (params.auth_key_en) {
+			data->auth_key_en = 1;
+		}
 	}
 
 	data->gctx_paddr = __psp_pa(sev->snp_context);
Signature · 1 line
2.37.1
#241 lore raw

Re: [[PATCH for v6]] KVM: SEV: fix snp_launch_finish

Quoted 5 lines · Harald Hoyer

The params.auth_key_en indicator does _not_ specify, whether an ID_AUTH struct should be sent or not, but, wheter the ID_AUTH struct contains an author key or not. The firmware always expects an ID_AUTH block.

Please provide feedback by directly responding to whatever patch/email is buggy. Or if that's too complicated for some reason (unlikely in this case), provide the fixup patch to the author *off-list*.

Quoted 16 lines
---
 arch/x86/kvm/svm/sev.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/sev.c +0 -2
index 85357dc4d231..5cf4be6a33ba 100644
@@ -2242,17 +2242,18 @@ static int snp_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 		data->id_block_en = 1;
 		data->id_block_paddr = __sme_pa(id_block);
-	}
 
-	if (params.auth_key_en) {

arch/x86/kvm/svm/sev.c static int snp_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2254

Quoted 10 lines
arch/x86/kvm/svm/sev.c +2 -1
 		id_auth = psp_copy_user_blob(params.id_auth_uaddr, KVM_SEV_SNP_ID_AUTH_SIZE);
 		if (IS_ERR(id_auth)) {
 			ret = PTR_ERR(id_auth);
 			goto e_free_id_block;
 		}
 
-		data->auth_key_en = 1;
 		data->id_auth_paddr = __sme_pa(id_auth);
+
+		if (params.auth_key_en) {

While I'm here though... Single line if-statements don't need curly braces.

Quoted 8 lines
arch/x86/kvm/svm/sev.c +2 -0
+			data->auth_key_en = 1;
+		}
 	}
 
 	data->gctx_paddr = __psp_pa(sev->snp_context);
Signature · 1 line
2.37.1
#242 lore raw

Re: [[PATCH for v6]] KVM: SEV: fix snp_launch_finish

Quoted 10 lines · Sean Christopherson
Quoted 5 lines · Harald Hoyer

The params.auth_key_en indicator does _not_ specify, whether an ID_AUTH struct should be sent or not, but, wheter the ID_AUTH struct contains an author key or not. The firmware always expects an ID_AUTH block.

Please provide feedback by directly responding to whatever patch/email is buggy. Or if that's too complicated for some reason (unlikely in this case), provide the fixup patch to the author *off-list*.

I'd guess that'd be:

https://lore.kernel.org/all/6a513cf79bf71c479dbd72165faf1d804d77b3af.1655761627.git.ashish.kalra@amd.com/

BR, Jarkko

#243 lore raw

Re: [PATCH Part2 v6 28/49] KVM: SVM: Add KVM_SEV_SNP_LAUNCH_FINISH command

In reply to Ashish Kalra

Replying inline to the patch (and not with a in-reply-to patch, as nitted by Sean Christopherson).

Am 21.06.22 um 01:08 schrieb Ashish Kalra:

Quoted 109 lines

From: Brijesh Singh <[email protected]>

The KVM_SEV_SNP_LAUNCH_FINISH finalize the cryptographic digest and stores it as the measurement of the guest at launch.

While finalizing the launch flow, it also issues the LAUNCH_UPDATE command to encrypt the VMSA pages.

If its an SNP guest, then VMSA was added in the RMP entry as a guest owned page and also removed from the kernel direct map so flush it later after it is transitioned back to hypervisor state and restored in the direct map.

---
  .../virt/kvm/x86/amd-memory-encryption.rst    |  22 ++++
  arch/x86/kvm/svm/sev.c                        | 119 ++++++++++++++++++
  include/uapi/linux/kvm.h                      |  14 +++
  3 files changed, 155 insertions(+)
Documentation/virt/kvm/x86/amd-memory-encryption.rst +22 -0
index 62abd5c1f72b..750162cff87b 100644
@@ -514,6 +514,28 @@ Returns: 0 on success, -negative on error
  See the SEV-SNP spec for further details on how to build the VMPL permission
  mask and page type.
 
+21. KVM_SNP_LAUNCH_FINISH
+-------------------------
+
+After completion of the SNP guest launch flow, the KVM_SNP_LAUNCH_FINISH command can be
+issued to make the guest ready for the execution.
+
+Parameters (in): struct kvm_sev_snp_launch_finish
+
+Returns: 0 on success, -negative on error
+
+::
+
+        struct kvm_sev_snp_launch_finish {
+                __u64 id_block_uaddr;
+                __u64 id_auth_uaddr;
+                __u8 id_block_en;
+                __u8 auth_key_en;
+                __u8 host_data[32];
+        };
+
+
+See SEV-SNP specification for further details on launch finish input parameters.
 
  References
  ==========
arch/x86/kvm/svm/sev.c +47 -0
index a9461d352eda..a5b90469683f 100644
@@ -2095,6 +2095,106 @@ static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp)
  	return ret;
  }
 
+static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_update data = {};
+	int i, ret;
+
+	data.gctx_paddr = __psp_pa(sev->snp_context);
+	data.page_type = SNP_PAGE_TYPE_VMSA;
+
+	for (i = 0; i < kvm->created_vcpus; i++) {
+		struct vcpu_svm *svm = to_svm(xa_load(&kvm->vcpu_array, i));
+		u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+		/* Perform some pre-encryption checks against the VMSA */
+		ret = sev_es_sync_vmsa(svm);
+		if (ret)
+			return ret;
+
+		/* Transition the VMSA page to a firmware state. */
+		ret = rmp_make_private(pfn, -1, PG_LEVEL_4K, sev->asid, true);
+		if (ret)
+			return ret;
+
+		/* Issue the SNP command to encrypt the VMSA */
+		data.address = __sme_pa(svm->sev_es.vmsa);
+		ret = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_LAUNCH_UPDATE,
+				      &data, &argp->error);
+		if (ret) {
+			snp_page_reclaim(pfn);
+			return ret;
+		}
+
+		svm->vcpu.arch.guest_state_protected = true;
+	}
+
+	return 0;
+}
+
+static int snp_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+	struct sev_data_snp_launch_finish *data;
+	void *id_block = NULL, *id_auth = NULL;
+	struct kvm_sev_snp_launch_finish params;
+	int ret;
+
+	if (!sev_snp_guest(kvm))

arch/x86/kvm/svm/sev.c static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2154

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+		return -ENOTTY;
+
+	if (!sev->snp_context)
+		return -EINVAL;
+
+	if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
+		return -EFAULT;
+
+	/* Measure all vCPUs using LAUNCH_UPDATE before we finalize the launch flow. */
+	ret = snp_launch_update_vmsa(kvm, argp);

This poses a real problem for those, who want to precalculate the digest beforehand and sign their TEE without loading the TEE:
1. We don't know the contents of the VMSA, nor the hash of it.
2. Who guarantees, that future kernels have the same VMSA contents?

I would propose at least one additional ioctl parameter specifying the final VMSA for the SNP_PAGE_TYPE_VMSA snp_launch_update_vmsa. This parameter could specify to use:
- the current VMSA
- or a VMSA resembling the CPU state on reset, where the contents is guaranteed to never change and have a defined digest
- or a user provided VMSA

Quoted 9 lines
arch/x86/kvm/svm/sev.c +9 -0
+	if (ret)
+		return ret;
+
+	data = kzalloc(sizeof(*data), GFP_KERNEL_ACCOUNT);
+	if (!data)
+		return -ENOMEM;
+
+	if (params.id_block_en) {
+		id_block = psp_copy_user_blob(params.id_block_uaddr, KVM_SEV_SNP_ID_BLOCK_SIZE);

arch/x86/kvm/svm/sev.c static int snp_launch_update(struct kvm *kvm, struct kvm_sev_cmd *argp) · line 2173

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+		if (IS_ERR(id_block)) {
+			ret = PTR_ERR(id_block);
+			goto e_free;
+		}
+
+		data->id_block_en = 1;
+		data->id_block_paddr = __sme_pa(id_block);
+	}
+
+	if (params.auth_key_en) {

The params.auth_key_en indicator does _not_ specify, whether an ID_AUTH struct should be sent or not, but wheter the ID_AUTH struct contains an author key or not. The firmware always expects an ID_AUTH block.

So, please move the upper if to enclose only data->auth_key_en = 1;, or use my patch sent in-reply to this mail yesterday.

Quoted 97 lines
arch/x86/kvm/svm/sev.c +43 -0
+		id_auth = psp_copy_user_blob(params.id_auth_uaddr, KVM_SEV_SNP_ID_AUTH_SIZE);
+		if (IS_ERR(id_auth)) {
+			ret = PTR_ERR(id_auth);
+			goto e_free_id_block;
+		}
+
+		data->auth_key_en = 1;
+		data->id_auth_paddr = __sme_pa(id_auth);
+	}
+
+	data->gctx_paddr = __psp_pa(sev->snp_context);
+	ret = sev_issue_cmd(kvm, SEV_CMD_SNP_LAUNCH_FINISH, data, &argp->error);
+
+	kfree(id_auth);
+
+e_free_id_block:
+	kfree(id_block);
+
+e_free:
+	kfree(data);
+
+	return ret;
+}
+
  int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
  {
  	struct kvm_sev_cmd sev_cmd;
@@ -2191,6 +2291,9 @@ int sev_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
  	case KVM_SEV_SNP_LAUNCH_UPDATE:
  		r = snp_launch_update(kvm, &sev_cmd);
  		break;
+	case KVM_SEV_SNP_LAUNCH_FINISH:
+		r = snp_launch_finish(kvm, &sev_cmd);
+		break;
  	default:
  		r = -EINVAL;
  		goto out;
@@ -2696,11 +2799,27 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 
  	svm = to_svm(vcpu);
 
+	/*
+	 * If its an SNP guest, then VMSA was added in the RMP entry as
+	 * a guest owned page. Transition the page to hypervisor state
+	 * before releasing it back to the system.
+	 * Also the page is removed from the kernel direct map, so flush it
+	 * later after it is transitioned back to hypervisor state and
+	 * restored in the direct map.
+	 */
+	if (sev_snp_guest(vcpu->kvm)) {
+		u64 pfn = __pa(svm->sev_es.vmsa) >> PAGE_SHIFT;
+
+		if (host_rmp_make_shared(pfn, PG_LEVEL_4K, false))
+			goto skip_vmsa_free;
+	}
+
  	if (vcpu->arch.guest_state_protected)
  		sev_flush_encrypted_page(vcpu, svm->sev_es.vmsa);
 
  	__free_page(virt_to_page(svm->sev_es.vmsa));
 
+skip_vmsa_free:
  	if (svm->sev_es.ghcb_sa_free)
  		kvfree(svm->sev_es.ghcb_sa);
  }
include/uapi/linux/kvm.h +14 -0
index 9b36b07414ea..5a4662716b6a 100644
@@ -1814,6 +1814,7 @@ enum sev_cmd_id {
  	KVM_SEV_SNP_INIT,
  	KVM_SEV_SNP_LAUNCH_START,
  	KVM_SEV_SNP_LAUNCH_UPDATE,
+	KVM_SEV_SNP_LAUNCH_FINISH,
 
  	KVM_SEV_NR_MAX,
  };
@@ -1948,6 +1949,19 @@ struct kvm_sev_snp_launch_update {
  	__u8 vmpl1_perms;
  };
 
+#define KVM_SEV_SNP_ID_BLOCK_SIZE	96
+#define KVM_SEV_SNP_ID_AUTH_SIZE	4096
+#define KVM_SEV_SNP_FINISH_DATA_SIZE	32
+
+struct kvm_sev_snp_launch_finish {
+	__u64 id_block_uaddr;
+	__u64 id_auth_uaddr;
+	__u8 id_block_en;
+	__u8 auth_key_en;
+	__u8 host_data[KVM_SEV_SNP_FINISH_DATA_SIZE];
+	__u8 pad[6];
+};
+
  #define KVM_DEV_ASSIGN_ENABLE_IOMMU	(1 << 0)
  #define KVM_DEV_ASSIGN_PCI_2_3		(1 << 1)
  #define KVM_DEV_ASSIGN_MASK_INTX	(1 << 2)
author maintainer
#244 lore raw

PATCH v6 29/49 KVM: X86: Keep the NPT and RMP page level in sync

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When running an SEV-SNP VM, the sPA used to index the RMP entry is obtained through the NPT translation (gva->gpa->spa). The NPT page level is checked against the page level programmed in the RMP entry. If the page level does not match, then it will cause a nested page fault with the RMP bit set to indicate the RMP violation.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  1 +
 arch/x86/kvm/mmu/mmu.c             |  5 ++++
 arch/x86/kvm/svm/sev.c             | 46 ++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             |  1 +
 arch/x86/kvm/svm/svm.h             |  1 +
 6 files changed, 55 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index a66292dae698..e0068e702692 100644
@@ -129,6 +129,7 @@ KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
+KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -0
index 0205e2944067..2748c69609e3 100644
@@ -1514,6 +1514,7 @@ struct kvm_x86_ops {
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
 	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+	void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/mmu/mmu.c +5 -0
index c623019929a7..997318ecebd1 100644
@@ -43,6 +43,7 @@
 #include <linux/hash.h>
 #include <linux/kern_levels.h>
 #include <linux/kthread.h>
+#include <linux/sev.h>
 
 #include <asm/page.h>
 #include <asm/memtype.h>
@@ -2824,6 +2825,10 @@ static int host_pfn_mapping_level(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
 	if (unlikely(!pte))
 		return PG_LEVEL_4K;
 
+	/* Adjust the page level based on the SEV-SNP RMP page level. */
+	if (kvm_x86_ops.rmp_page_level_adjust)
+		static_call(kvm_x86_rmp_page_level_adjust)(kvm, pfn, &level);
+
 	return level;
 }
 
arch/x86/kvm/svm/sev.c +46 -0

49 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +1 -0
index b4bd64f94d3a..18e2cd4d9559 100644
@@ -4734,6 +4734,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
 	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
+	.rmp_page_level_adjust = sev_rmp_page_level_adjust,
 };
 
 /*
arch/x86/kvm/svm/svm.h +1 -0
index 71c011af098e..7782312a1cda 100644
@@ -673,6 +673,7 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
 void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
+void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
 
 /* vmenter.S */
 
Signature · 1 line
2.25.1
#245 lore raw

Re: [PATCH Part2 v6 29/49] KVM: X86: Keep the NPT and RMP page level in sync

s/X86/x86/

Quoted 78 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When running an SEV-SNP VM, the sPA used to index the RMP entry is obtained through the NPT translation (gva->gpa->spa). The NPT page level is checked against the page level programmed in the RMP entry. If the page level does not match, then it will cause a nested page fault with the RMP bit set to indicate the RMP violation.

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  1 +
 arch/x86/kvm/mmu/mmu.c             |  5 ++++
 arch/x86/kvm/svm/sev.c             | 46 ++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c             |  1 +
 arch/x86/kvm/svm/svm.h             |  1 +
 6 files changed, 55 insertions(+)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index a66292dae698..e0068e702692 100644
@@ -129,6 +129,7 @@ KVM_X86_OP(complete_emulated_msr)
 KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
+KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +1 -0
index 0205e2944067..2748c69609e3 100644
@@ -1514,6 +1514,7 @@ struct kvm_x86_ops {
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
 	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+	void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/mmu/mmu.c +5 -0
index c623019929a7..997318ecebd1 100644
@@ -43,6 +43,7 @@
 #include <linux/hash.h>
 #include <linux/kern_levels.h>
 #include <linux/kthread.h>
+#include <linux/sev.h>
 
 #include <asm/page.h>
 #include <asm/memtype.h>
@@ -2824,6 +2825,10 @@ static int host_pfn_mapping_level(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn,
 	if (unlikely(!pte))
 		return PG_LEVEL_4K;
 
+	/* Adjust the page level based on the SEV-SNP RMP page level. */
+	if (kvm_x86_ops.rmp_page_level_adjust)
+		static_call(kvm_x86_rmp_page_level_adjust)(kvm, pfn, &level);
+
 	return level;
 }
 
arch/x86/kvm/svm/sev.c +5 -0
index a5b90469683f..91d3d24e60d2 100644
@@ -3597,3 +3597,49 @@ struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu)
 
 	return pfn_to_page(pfn);
 }
+
+static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end)
+{
+	int level;
+

arch/x86/kvm/svm/sev.c struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu) · line 3614

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+	while (end > start) {
+		if (snp_lookup_rmpentry(start, &level) != 0)
+			return false;
+		start++;
+	}
+
+	return true;
+}
+
+void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level)

Would not do harm to document this, given that it is not a static fuction.

Quoted 58 lines
arch/x86/kvm/svm/sev.c +31 -0
+{
+	int rmp_level, assigned;
+
+	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
+		return;
+
+	assigned = snp_lookup_rmpentry(pfn, &rmp_level);
+	if (unlikely(assigned < 0))
+		return;
+
+	if (!assigned) {
+		/*
+		 * If all the pages are shared then no need to keep the RMP
+		 * and NPT in sync.
+		 */
+		pfn = pfn & ~(PTRS_PER_PMD - 1);
+		if (is_pfn_range_shared(pfn, pfn + PTRS_PER_PMD))
+			return;
+	}
+
+	/*
+	 * The hardware installs 2MB TLB entries to access to 1GB pages,
+	 * therefore allow NPT to use 1GB pages when pfn was added as 2MB
+	 * in the RMP table.
+	 */
+	if (rmp_level == PG_LEVEL_2M && (*level == PG_LEVEL_1G))
+		return;
+
+	/* Adjust the level to keep the NPT and RMP in sync */
+	*level = min_t(size_t, *level, rmp_level);
+}
arch/x86/kvm/svm/svm.c +1 -0
index b4bd64f94d3a..18e2cd4d9559 100644
@@ -4734,6 +4734,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
 	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
+	.rmp_page_level_adjust = sev_rmp_page_level_adjust,
 };
 
 /*
arch/x86/kvm/svm/svm.h +1 -0
index 71c011af098e..7782312a1cda 100644
@@ -673,6 +673,7 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector);
 void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
+void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
 
 /* vmenter.S */
 
Signature · 1 line
2.25.1

BR, Jarkko

author maintainer
#246 lore raw

PATCH v6 30/49 KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX and SNP

In reply to Ashish Kalra

From: Sean Christopherson <[email protected]>

Introduce a helper to directly (pun intended) fault-in a TDP page without having to go through the full page fault path. This allows TDX to get the resulting pfn and also allows the RET_PF_* enums to stay in mmu.c where they belong.

---
 arch/x86/kvm/mmu.h     |  3 +++
 arch/x86/kvm/mmu/mmu.c | 51 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
arch/x86/kvm/mmu.h +3 -0
index e6cae6f22683..c99b15e97a0a 100644
@@ -204,6 +204,9 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 	return vcpu->arch.mmu->page_fault(vcpu, &fault);
 }
 
+kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
+			       u32 error_code, int max_level);
+
 /*
  * Check if a given access (described through the I/D, W/R and U/S bits of a
  * page fault error code pfec) causes a permission fault with the given PTE
arch/x86/kvm/mmu/mmu.c +51 -0

57 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#247 lore raw

PATCH v6 31/49 KVM: x86: Introduce kvm_mmu_get_tdp_walk() for SEV-SNP use

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP VMs may call the page state change VMGEXIT to add the GPA as private or shared in the RMP table. The page state change VMGEXIT will contain the RMP page level to be used in the RMP entry. If the page level between the TDP and RMP does not match then, it will result in nested-page-fault (RMP violation).

The SEV-SNP VMGEXIT handler will use the kvm_mmu_get_tdp_walk() to get the current page-level in the TDP for the given GPA and calculate a workable page level. If a GPA is mapped as a 4K-page in the TDP, but the guest requested to add the GPA as a 2M in the RMP entry then the 2M request will be broken into 4K-pages to keep the RMP and TDP page-levels in sync.

TDP SPTEs are RCU protected so need to put kvm_mmu_get_tdp_walk() in RCU read-side critical section by using walk_shadow_page_lockless_begin() and walk_lockless_shadow_page_lockless_end(). This fixes the "suspicious RCU usage" message seen with lockdep kernel build.

Signed-off by: Ashish Kalra <[email protected]>

---
 arch/x86/kvm/mmu.h     |  2 ++
 arch/x86/kvm/mmu/mmu.c | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)
arch/x86/kvm/mmu.h +2 -0
index c99b15e97a0a..d55b5166389a 100644
@@ -178,6 +178,8 @@ static inline bool is_nx_huge_page_enabled(void)
 	return READ_ONCE(nx_huge_pages);
 }
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level);
+
 static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 					u32 err, bool prefetch)
 {
arch/x86/kvm/mmu/mmu.c +33 -0
index 569021af349a..c1ac486e096e 100644
@@ -4151,6 +4151,39 @@ kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_map_tdp_page);
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level)
+{
+	u64 sptes[PT64_ROOT_MAX_LEVEL + 1];
+	int leaf, root;
+
+	walk_shadow_page_lockless_begin(vcpu);
+
+	if (is_tdp_mmu(vcpu->arch.mmu))
+		leaf = kvm_tdp_mmu_get_walk(vcpu, gpa, sptes, &root);
+	else
+		leaf = get_walk(vcpu, gpa, sptes, &root);
+
+	walk_shadow_page_lockless_end(vcpu);
+
+	if (unlikely(leaf < 0))
+		return false;
+
+	/* Check if the leaf SPTE is present */
+	if (!is_shadow_present_pte(sptes[leaf]))
+		return false;
+
+	*pfn = spte_to_pfn(sptes[leaf]);
+	if (leaf > PG_LEVEL_4K) {
+		u64 page_mask = KVM_PAGES_PER_HPAGE(leaf) - KVM_PAGES_PER_HPAGE(leaf - 1);
+		*pfn |= (gpa_to_gfn(gpa) & page_mask);
+	}
+
+	*level = leaf;
+
+	return true;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_get_tdp_walk);
+
 static void nonpaging_init_context(struct kvm_mmu *context)
 {
 	context->page_fault = nonpaging_page_fault;
Signature · 1 line
2.25.1
#248 lore raw

Re: [PATCH Part2 v6 31/49] KVM: x86: Introduce kvm_mmu_get_tdp_walk() for SEV-SNP use

Quoted 64 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The SEV-SNP VMs may call the page state change VMGEXIT to add the GPA as private or shared in the RMP table. The page state change VMGEXIT will contain the RMP page level to be used in the RMP entry. If the page level between the TDP and RMP does not match then, it will result in nested-page-fault (RMP violation).

The SEV-SNP VMGEXIT handler will use the kvm_mmu_get_tdp_walk() to get the current page-level in the TDP for the given GPA and calculate a workable page level. If a GPA is mapped as a 4K-page in the TDP, but the guest requested to add the GPA as a 2M in the RMP entry then the 2M request will be broken into 4K-pages to keep the RMP and TDP page-levels in sync.

TDP SPTEs are RCU protected so need to put kvm_mmu_get_tdp_walk() in RCU read-side critical section by using walk_shadow_page_lockless_begin() and walk_lockless_shadow_page_lockless_end(). This fixes the "suspicious RCU usage" message seen with lockdep kernel build.

Signed-off by: Ashish Kalra <[email protected]>

---
 arch/x86/kvm/mmu.h     |  2 ++
 arch/x86/kvm/mmu/mmu.c | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)
arch/x86/kvm/mmu.h +2 -0
index c99b15e97a0a..d55b5166389a 100644
@@ -178,6 +178,8 @@ static inline bool is_nx_huge_page_enabled(void)
        return READ_ONCE(nx_huge_pages);
 }
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level);
+
 static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
                                        u32 err, bool prefetch)
 {
arch/x86/kvm/mmu/mmu.c +15 -0
index 569021af349a..c1ac486e096e 100644
@@ -4151,6 +4151,39 @@ kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
 }
 EXPORT_SYMBOL_GPL(kvm_mmu_map_tdp_page);
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level)
+{
+       u64 sptes[PT64_ROOT_MAX_LEVEL + 1];
+       int leaf, root;
+
+       walk_shadow_page_lockless_begin(vcpu);
+
+       if (is_tdp_mmu(vcpu->arch.mmu))
+               leaf = kvm_tdp_mmu_get_walk(vcpu, gpa, sptes, &root);
+       else
+               leaf = get_walk(vcpu, gpa, sptes, &root);
+
+       walk_shadow_page_lockless_end(vcpu);
+
+       if (unlikely(leaf < 0))

arch/x86/kvm/mmu/mmu.c kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa, · line 4178

Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +10 -0
+               return false;
+
+       /* Check if the leaf SPTE is present */
+       if (!is_shadow_present_pte(sptes[leaf]))
+               return false;
+
+       *pfn = spte_to_pfn(sptes[leaf]);
+       if (leaf > PG_LEVEL_4K) {
+               u64 page_mask = KVM_PAGES_PER_HPAGE(leaf) - KVM_PAGES_PER_HPAGE(leaf - 1);
+               *pfn |= (gpa_to_gfn(gpa) & page_mask);

Similar to the discussion in the other patch, I believe you should apply the same fix here as well.
It should be
u64 page_mask = KVM_PAGES_PER_HPAGE(leaf) - 1;

Quoted 14 lines
arch/x86/kvm/mmu/mmu.c +8 -1
+       }
+
+       *level = leaf;
+
+       return true;
+}
+EXPORT_SYMBOL_GPL(kvm_mmu_get_tdp_walk);
+
 static void nonpaging_init_context(struct kvm_mmu *context)
 {
        context->page_fault = nonpaging_page_fault;
--

2.25.1

author maintainer
#249 lore raw

PATCH v6 32/49 KVM: x86: Define RMP page fault error bits for #NPF

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled globally, the hardware places restrictions on all memory accesses based on the RMP entry, whether the hypervisor or a VM, performs the accesses. When hardware encounters an RMP access violation during a guest access, it will cause a #VMEXIT(NPF).

See APM2 section 16.36.10 for more details.

---
 arch/x86/include/asm/kvm_host.h | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/x86/include/asm/kvm_host.h +8 -0
index 2748c69609e3..49b217dc8d7e 100644
@@ -247,9 +247,13 @@ enum x86_intercept_stage;
 #define PFERR_FETCH_BIT 4
 #define PFERR_PK_BIT 5
 #define PFERR_SGX_BIT 15
+#define PFERR_GUEST_RMP_BIT 31
 #define PFERR_GUEST_FINAL_BIT 32
 #define PFERR_GUEST_PAGE_BIT 33
 #define PFERR_IMPLICIT_ACCESS_BIT 48
+#define PFERR_GUEST_ENC_BIT 34
+#define PFERR_GUEST_SIZEM_BIT 35
+#define PFERR_GUEST_VMPL_BIT 36
 
 #define PFERR_PRESENT_MASK (1U << PFERR_PRESENT_BIT)
 #define PFERR_WRITE_MASK (1U << PFERR_WRITE_BIT)
@@ -261,6 +265,10 @@ enum x86_intercept_stage;
 #define PFERR_GUEST_FINAL_MASK (1ULL << PFERR_GUEST_FINAL_BIT)
 #define PFERR_GUEST_PAGE_MASK (1ULL << PFERR_GUEST_PAGE_BIT)
 #define PFERR_IMPLICIT_ACCESS (1ULL << PFERR_IMPLICIT_ACCESS_BIT)
+#define PFERR_GUEST_RMP_MASK (1ULL << PFERR_GUEST_RMP_BIT)
+#define PFERR_GUEST_ENC_MASK (1ULL << PFERR_GUEST_ENC_BIT)
+#define PFERR_GUEST_SIZEM_MASK (1ULL << PFERR_GUEST_SIZEM_BIT)
+#define PFERR_GUEST_VMPL_MASK (1ULL << PFERR_GUEST_VMPL_BIT)
 
 #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK |	\
 				 PFERR_WRITE_MASK |		\
Signature · 1 line
2.25.1
author maintainer
#250 lore raw

PATCH v6 33/49 KVM: x86: Update page-fault trace to log full 64-bit error code

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The #NPT error code is a 64-bit value but the trace prints only the lower 32-bits. Some of the fault error code (e.g PFERR_GUEST_FINAL_MASK) are available in the upper 32-bits.

---
 arch/x86/kvm/trace.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/trace.h +3 -3
index e3a24b8f04be..9b9bc5468103 100644
@@ -383,12 +383,12 @@ TRACE_EVENT(kvm_inj_exception,
  * Tracepoint for page fault.
  */
 TRACE_EVENT(kvm_page_fault,
-	TP_PROTO(unsigned long fault_address, unsigned int error_code),
+	TP_PROTO(unsigned long fault_address, u64 error_code),
 	TP_ARGS(fault_address, error_code),
 
 	TP_STRUCT__entry(
 		__field(	unsigned long,	fault_address	)
-		__field(	unsigned int,	error_code	)
+		__field(	u64,		error_code	)
 	),
 
 	TP_fast_assign(
@@ -396,7 +396,7 @@ TRACE_EVENT(kvm_page_fault,
 		__entry->error_code	= error_code;
 	),
 
-	TP_printk("address %lx error_code %x",
+	TP_printk("address %lx error_code %llx",
 		  __entry->fault_address, __entry->error_code)
 );
 
Signature · 1 line
2.25.1
#251 lore raw

Re: [PATCH Part2 v6 33/49] KVM: x86: Update page-fault trace to log full 64-bit error code

Quoted 7 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

The #NPT error code is a 64-bit value but the trace prints only the lower 32-bits. Some of the fault error code (e.g PFERR_GUEST_FINAL_MASK) are available in the upper 32-bits.

Why stable?

Quoted 33 lines
---
 arch/x86/kvm/trace.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/trace.h +3 -3
index e3a24b8f04be..9b9bc5468103 100644
@@ -383,12 +383,12 @@ TRACE_EVENT(kvm_inj_exception,
  * Tracepoint for page fault.
  */
 TRACE_EVENT(kvm_page_fault,
-	TP_PROTO(unsigned long fault_address, unsigned int error_code),
+	TP_PROTO(unsigned long fault_address, u64 error_code),
 	TP_ARGS(fault_address, error_code),
 
 	TP_STRUCT__entry(
 		__field(	unsigned long,	fault_address	)
-		__field(	unsigned int,	error_code	)
+		__field(	u64,		error_code	)
 	),
 
 	TP_fast_assign(
@@ -396,7 +396,7 @@ TRACE_EVENT(kvm_page_fault,
 		__entry->error_code	= error_code;
 	),
 
-	TP_printk("address %lx error_code %x",
+	TP_printk("address %lx error_code %llx",
 		  __entry->fault_address, __entry->error_code)
 );
 
author maintainer
#252 lore raw

PATCH v6 34/49 KVM: SVM: Do not use long-lived GHCB map while setting scratch area

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

The setup_vmgexit_scratch() function may rely on a long-lived GHCB mapping if the GHCB shared buffer area was used for the scratch area. In preparation for eliminating the long-lived GHCB mapping, always allocate a buffer for the scratch area so it can be accessed without the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 74 +++++++++++++++++++-----------------------
 arch/x86/kvm/svm/svm.h |  3 +-
 2 files changed, 36 insertions(+), 41 deletions(-)
arch/x86/kvm/svm/sev.c +34 -40
index 91d3d24e60d2..01ea257e17d6 100644
@@ -2820,8 +2820,7 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 	__free_page(virt_to_page(svm->sev_es.vmsa));
 
 skip_vmsa_free:
-	if (svm->sev_es.ghcb_sa_free)
-		kvfree(svm->sev_es.ghcb_sa);
+	kvfree(svm->sev_es.ghcb_sa);
 }
 
 static void dump_ghcb(struct vcpu_svm *svm)
@@ -2909,6 +2908,9 @@ static void sev_es_sync_from_ghcb(struct vcpu_svm *svm)
 	control->exit_info_1 = ghcb_get_sw_exit_info_1(ghcb);
 	control->exit_info_2 = ghcb_get_sw_exit_info_2(ghcb);
 
+	/* Copy the GHCB scratch area GPA */
+	svm->sev_es.ghcb_sa_gpa = ghcb_get_sw_scratch(ghcb);
+
 	/* Clear the valid entries fields */
 	memset(ghcb->save.valid_bitmap, 0, sizeof(ghcb->save.valid_bitmap));
 }
@@ -3054,23 +3056,12 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm)
 	if (!svm->sev_es.ghcb)
 		return;
 
-	if (svm->sev_es.ghcb_sa_free) {
-		/*
-		 * The scratch area lives outside the GHCB, so there is a
-		 * buffer that, depending on the operation performed, may
-		 * need to be synced, then freed.
-		 */
-		if (svm->sev_es.ghcb_sa_sync) {
-			kvm_write_guest(svm->vcpu.kvm,
-					ghcb_get_sw_scratch(svm->sev_es.ghcb),
-					svm->sev_es.ghcb_sa,
-					svm->sev_es.ghcb_sa_len);
-			svm->sev_es.ghcb_sa_sync = false;
-		}
-
-		kvfree(svm->sev_es.ghcb_sa);
-		svm->sev_es.ghcb_sa = NULL;
-		svm->sev_es.ghcb_sa_free = false;
+	 /* Sync the scratch buffer area. */
+	if (svm->sev_es.ghcb_sa_sync) {
+		kvm_write_guest(svm->vcpu.kvm,
+				ghcb_get_sw_scratch(svm->sev_es.ghcb),
+				svm->sev_es.ghcb_sa, svm->sev_es.ghcb_sa_len);
+		svm->sev_es.ghcb_sa_sync = false;
 	}
 
 	trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, svm->sev_es.ghcb);
@@ -3111,9 +3102,8 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 	struct ghcb *ghcb = svm->sev_es.ghcb;
 	u64 ghcb_scratch_beg, ghcb_scratch_end;
 	u64 scratch_gpa_beg, scratch_gpa_end;
-	void *scratch_va;
 
-	scratch_gpa_beg = ghcb_get_sw_scratch(ghcb);
+	scratch_gpa_beg = svm->sev_es.ghcb_sa_gpa;
 	if (!scratch_gpa_beg) {
 		pr_err("vmgexit: scratch gpa not provided\n");
 		goto e_scratch;
@@ -3143,9 +3133,6 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 			       scratch_gpa_beg, scratch_gpa_end);
 			goto e_scratch;
 		}
-
-		scratch_va = (void *)svm->sev_es.ghcb;
-		scratch_va += (scratch_gpa_beg - control->ghcb_gpa);
 	} else {
 		/*
 		 * The guest memory must be read into a kernel buffer, so
@@ -3156,29 +3143,36 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 			       len, GHCB_SCRATCH_AREA_LIMIT);
 			goto e_scratch;
 		}
-		scratch_va = kvzalloc(len, GFP_KERNEL_ACCOUNT);
-		if (!scratch_va)
-			return -ENOMEM;
+	}
 
-		if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, scratch_va, len)) {
-			/* Unable to copy scratch area from guest */
-			pr_err("vmgexit: kvm_read_guest for scratch area failed\n");
+	if (svm->sev_es.ghcb_sa_alloc_len < len) {
+		void *scratch_va = kvzalloc(len, GFP_KERNEL_ACCOUNT);
 
-			kvfree(scratch_va);
-			return -EFAULT;
-		}
+		if (!scratch_va)
+			return -ENOMEM;
 
 		/*
-		 * The scratch area is outside the GHCB. The operation will
-		 * dictate whether the buffer needs to be synced before running
-		 * the vCPU next time (i.e. a read was requested so the data
-		 * must be written back to the guest memory).
+		 * Free the old scratch area and switch to using newly
+		 * allocated.
 		 */
-		svm->sev_es.ghcb_sa_sync = sync;
-		svm->sev_es.ghcb_sa_free = true;
+		kvfree(svm->sev_es.ghcb_sa);
+
+		svm->sev_es.ghcb_sa_alloc_len = len;
+		svm->sev_es.ghcb_sa = scratch_va;
 	}
 
-	svm->sev_es.ghcb_sa = scratch_va;
+	if (kvm_read_guest(svm->vcpu.kvm, scratch_gpa_beg, svm->sev_es.ghcb_sa, len)) {
+		/* Unable to copy scratch area from guest */
+		pr_err("vmgexit: kvm_read_guest for scratch area failed\n");
+		return -EFAULT;
+	}
+
+	/*
+	 * The operation will dictate whether the buffer needs to be synced
+	 * before running the vCPU next time (i.e. a read was requested so
+	 * the data must be written back to the guest memory).
+	 */
+	svm->sev_es.ghcb_sa_sync = sync;
 	svm->sev_es.ghcb_sa_len = len;
 
 	return 0;
arch/x86/kvm/svm/svm.h +2 -1
index 7782312a1cda..bd0db4d4a61e 100644
@@ -197,8 +197,9 @@ struct vcpu_sev_es_state {
 	/* SEV-ES scratch area support */
 	void *ghcb_sa;
 	u32 ghcb_sa_len;
+	u64 ghcb_sa_gpa;
+	u32 ghcb_sa_alloc_len;
 	bool ghcb_sa_sync;
-	bool ghcb_sa_free;
 };
 
 struct vcpu_svm {
Signature · 1 line
2.25.1
author maintainer
#253 lore raw

PATCH v6 35/49 KVM: SVM: Remove the long-lived GHCB host map

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131 ++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c +83 -48
index 01ea257e17d6..c70f3f7e06a8 100644
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
 	kvfree(svm->sev_es.ghcb_sa);
 }
 
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
+{
+	struct vmcb_control_area *control = &svm->vmcb->control;
+	u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+	if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+		/* Unable to map GHCB from guest */
+		pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+		return -EFAULT;
+	}
+
+	return 0;
+}
+
+static inline void svm_unmap_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
+{
+	kvm_vcpu_unmap(&svm->vcpu, map, true);
+}
+
 static void dump_ghcb(struct vcpu_svm *svm)
 {
-	struct ghcb *ghcb = svm->sev_es.ghcb;
+	struct kvm_host_map map;
 	unsigned int nbits;
+	struct ghcb *ghcb;
+
+	if (svm_map_ghcb(svm, &map))
+		return;
+
+	ghcb = map.hva;
 
 	/* Re-use the dump_invalid_vmcb module parameter */
 	if (!dump_invalid_vmcb) {
 		pr_warn_ratelimited("set kvm_amd.dump_invalid_vmcb=1 to dump internal KVM state.\n");
-		return;
+		goto e_unmap;
 	}
 
 	nbits = sizeof(ghcb->save.valid_bitmap) * 8;
@@ -2846,12 +2871,21 @@ static void dump_ghcb(struct vcpu_svm *svm)
 	pr_err("%-20s%016llx is_valid: %u\n", "sw_scratch",
 	       ghcb->save.sw_scratch, ghcb_sw_scratch_is_valid(ghcb));
 	pr_err("%-20s%*pb\n", "valid_bitmap", nbits, ghcb->save.valid_bitmap);
+
+e_unmap:
+	svm_unmap_ghcb(svm, &map);
 }
 
-static void sev_es_sync_to_ghcb(struct vcpu_svm *svm)
+static bool sev_es_sync_to_ghcb(struct vcpu_svm *svm)
 {
 	struct kvm_vcpu *vcpu = &svm->vcpu;
-	struct ghcb *ghcb = svm->sev_es.ghcb;
+	struct kvm_host_map map;
+	struct ghcb *ghcb;
+
+	if (svm_map_ghcb(svm, &map))
+		return false;
+
+	ghcb = map.hva;
 
 	/*
 	 * The GHCB protocol so far allows for the following data
@@ -2865,13 +2899,24 @@ static void sev_es_sync_to_ghcb(struct vcpu_svm *svm)
 	ghcb_set_rbx(ghcb, vcpu->arch.regs[VCPU_REGS_RBX]);
 	ghcb_set_rcx(ghcb, vcpu->arch.regs[VCPU_REGS_RCX]);
 	ghcb_set_rdx(ghcb, vcpu->arch.regs[VCPU_REGS_RDX]);
+
+	/*
+	 * Copy the return values from the exit_info_{1,2}.
+	 */
+	ghcb_set_sw_exit_info_1(ghcb, svm->sev_es.ghcb_sw_exit_info_1);
+	ghcb_set_sw_exit_info_2(ghcb, svm->sev_es.ghcb_sw_exit_info_2);
+
+	trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, ghcb);
+
+	svm_unmap_ghcb(svm, &map);
+
+	return true;
 }
 
-static void sev_es_sync_from_ghcb(struct vcpu_svm *svm)
+static void sev_es_sync_from_ghcb(struct vcpu_svm *svm, struct ghcb *ghcb)
 {
 	struct vmcb_control_area *control = &svm->vmcb->control;
 	struct kvm_vcpu *vcpu = &svm->vcpu;
-	struct ghcb *ghcb = svm->sev_es.ghcb;
 	u64 exit_code;
 
 	/*
@@ -2915,20 +2960,25 @@ static void sev_es_sync_from_ghcb(struct vcpu_svm *svm)
 	memset(ghcb->save.valid_bitmap, 0, sizeof(ghcb->save.valid_bitmap));
 }
 
-static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
+static int sev_es_validate_vmgexit(struct vcpu_svm *svm, u64 *exit_code)
 {
-	struct kvm_vcpu *vcpu;
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm_host_map map;
 	struct ghcb *ghcb;
-	u64 exit_code;
 	u64 reason;
 
-	ghcb = svm->sev_es.ghcb;
+	if (svm_map_ghcb(svm, &map))
+		return -EFAULT;
+
+	ghcb = map.hva;
+
+	trace_kvm_vmgexit_enter(vcpu->vcpu_id, ghcb);
 
 	/*
 	 * Retrieve the exit code now even though it may not be marked valid
 	 * as it could help with debugging.
 	 */
-	exit_code = ghcb_get_sw_exit_code(ghcb);
+	*exit_code = ghcb_get_sw_exit_code(ghcb);
 
 	/* Only GHCB Usage code 0 is supported */
 	if (ghcb->ghcb_usage) {
@@ -3021,6 +3071,9 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 		goto vmgexit_err;
 	}
 
+	sev_es_sync_from_ghcb(svm, ghcb);
+
+	svm_unmap_ghcb(svm, &map);
 	return 0;
 
 vmgexit_err:
@@ -3031,10 +3084,10 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 			    ghcb->ghcb_usage);
 	} else if (reason == GHCB_ERR_INVALID_EVENT) {
 		vcpu_unimpl(vcpu, "vmgexit: exit code %#llx is not valid\n",
-			    exit_code);
+			    *exit_code);
 	} else {
 		vcpu_unimpl(vcpu, "vmgexit: exit code %#llx input is not valid\n",
-			    exit_code);
+			    *exit_code);
 		dump_ghcb(svm);
 	}
 
@@ -3044,6 +3097,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm)
 	ghcb_set_sw_exit_info_1(ghcb, 2);
 	ghcb_set_sw_exit_info_2(ghcb, reason);
 
+	svm_unmap_ghcb(svm, &map);
+
 	/* Resume the guest to "return" the error code. */
 	return 1;
 }
@@ -3053,23 +3108,20 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm)
 	/* Clear any indication that the vCPU is in a type of AP Reset Hold */
 	svm->sev_es.ap_reset_hold_type = AP_RESET_HOLD_NONE;
 
-	if (!svm->sev_es.ghcb)
+	if (!svm->sev_es.ghcb_in_use)
 		return;
 
 	 /* Sync the scratch buffer area. */
 	if (svm->sev_es.ghcb_sa_sync) {
 		kvm_write_guest(svm->vcpu.kvm,
-				ghcb_get_sw_scratch(svm->sev_es.ghcb),
+				svm->sev_es.ghcb_sa_gpa,
 				svm->sev_es.ghcb_sa, svm->sev_es.ghcb_sa_len);
 		svm->sev_es.ghcb_sa_sync = false;
 	}
 
-	trace_kvm_vmgexit_exit(svm->vcpu.vcpu_id, svm->sev_es.ghcb);
-
 	sev_es_sync_to_ghcb(svm);
 
-	kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true);
-	svm->sev_es.ghcb = NULL;
+	svm->sev_es.ghcb_in_use = false;
 }
 
 void pre_sev_run(struct vcpu_svm *svm, int cpu)
@@ -3099,7 +3151,6 @@ void pre_sev_run(struct vcpu_svm *svm, int cpu)
 static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 {
 	struct vmcb_control_area *control = &svm->vmcb->control;
-	struct ghcb *ghcb = svm->sev_es.ghcb;
 	u64 ghcb_scratch_beg, ghcb_scratch_end;
 	u64 scratch_gpa_beg, scratch_gpa_end;
 
@@ -3178,8 +3229,8 @@ static int setup_vmgexit_scratch(struct vcpu_svm *svm, bool sync, u64 len)
 	return 0;
 
 e_scratch:
-	ghcb_set_sw_exit_info_1(ghcb, 2);
-	ghcb_set_sw_exit_info_2(ghcb, GHCB_ERR_INVALID_SCRATCH_AREA);
+	svm_set_ghcb_sw_exit_info_1(&svm->vcpu, 2);
+	svm_set_ghcb_sw_exit_info_2(&svm->vcpu, GHCB_ERR_INVALID_SCRATCH_AREA);
 
 	return 1;
 }
@@ -3316,7 +3367,6 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 	struct vcpu_svm *svm = to_svm(vcpu);
 	struct vmcb_control_area *control = &svm->vmcb->control;
 	u64 ghcb_gpa, exit_code;
-	struct ghcb *ghcb;
 	int ret;
 
 	/* Validate the GHCB */
@@ -3331,29 +3381,14 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		return 1;
 	}
 
-	if (kvm_vcpu_map(vcpu, ghcb_gpa >> PAGE_SHIFT, &svm->sev_es.ghcb_map)) {
-		/* Unable to map GHCB from guest */
-		vcpu_unimpl(vcpu, "vmgexit: error mapping GHCB [%#llx] from guest\n",
-			    ghcb_gpa);
-
-		/* Without a GHCB, just return right back to the guest */
-		return 1;
-	}
-
-	svm->sev_es.ghcb = svm->sev_es.ghcb_map.hva;
-	ghcb = svm->sev_es.ghcb_map.hva;
-
-	trace_kvm_vmgexit_enter(vcpu->vcpu_id, ghcb);
-
-	exit_code = ghcb_get_sw_exit_code(ghcb);
-
-	ret = sev_es_validate_vmgexit(svm);
+	ret = sev_es_validate_vmgexit(svm, &exit_code);
 	if (ret)
 		return ret;
 
-	sev_es_sync_from_ghcb(svm);
-	ghcb_set_sw_exit_info_1(ghcb, 0);
-	ghcb_set_sw_exit_info_2(ghcb, 0);
+	svm->sev_es.ghcb_in_use = true;
+
+	svm_set_ghcb_sw_exit_info_1(vcpu, 0);
+	svm_set_ghcb_sw_exit_info_2(vcpu, 0);
 
 	switch (exit_code) {
 	case SVM_VMGEXIT_MMIO_READ:
@@ -3393,20 +3428,20 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 			break;
 		case 1:
 			/* Get AP jump table address */
-			ghcb_set_sw_exit_info_2(ghcb, sev->ap_jump_table);
+			svm_set_ghcb_sw_exit_info_2(vcpu, sev->ap_jump_table);
 			break;
 		default:
 			pr_err("svm: vmgexit: unsupported AP jump table request - exit_info_1=%#llx\n",
 			       control->exit_info_1);
-			ghcb_set_sw_exit_info_1(ghcb, 2);
-			ghcb_set_sw_exit_info_2(ghcb, GHCB_ERR_INVALID_INPUT);
+			svm_set_ghcb_sw_exit_info_1(vcpu, 2);
+			svm_set_ghcb_sw_exit_info_2(vcpu, GHCB_ERR_INVALID_INPUT);
 		}
 
 		ret = 1;
 		break;
 	}
 	case SVM_VMGEXIT_HV_FEATURES: {
-		ghcb_set_sw_exit_info_2(ghcb, GHCB_HV_FT_SUPPORTED);
+		svm_set_ghcb_sw_exit_info_2(vcpu, GHCB_HV_FT_SUPPORTED);
 
 		ret = 1;
 		break;
@@ -3537,7 +3572,7 @@ void sev_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector)
 		 * Return from an AP Reset Hold VMGEXIT, where the guest will
 		 * set the CS and RIP. Set SW_EXIT_INFO_2 to a non-zero value.
 		 */
-		ghcb_set_sw_exit_info_2(svm->sev_es.ghcb, 1);
+		svm_set_ghcb_sw_exit_info_2(vcpu, 1);
 		break;
 	case AP_RESET_HOLD_MSR_PROTO:
 		/*
arch/x86/kvm/svm/svm.c +6 -6
index 18e2cd4d9559..b24e0171cbf2 100644
@@ -2720,14 +2720,14 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 static int svm_complete_emulated_msr(struct kvm_vcpu *vcpu, int err)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
-	if (!err || !sev_es_guest(vcpu->kvm) || WARN_ON_ONCE(!svm->sev_es.ghcb))
+	if (!err || !sev_es_guest(vcpu->kvm) || WARN_ON_ONCE(!svm->sev_es.ghcb_in_use))
 		return kvm_complete_insn_gp(vcpu, err);
 
-	ghcb_set_sw_exit_info_1(svm->sev_es.ghcb, 1);
-	ghcb_set_sw_exit_info_2(svm->sev_es.ghcb,
-				X86_TRAP_GP |
-				SVM_EVTINJ_TYPE_EXEPT |
-				SVM_EVTINJ_VALID);
+	svm_set_ghcb_sw_exit_info_1(vcpu, 1);
+	svm_set_ghcb_sw_exit_info_2(vcpu,
+				    X86_TRAP_GP |
+				    SVM_EVTINJ_TYPE_EXEPT |
+				    SVM_EVTINJ_VALID);
 	return 1;
 }
 
arch/x86/kvm/svm/svm.h +22 -2
index bd0db4d4a61e..c80352c9c0d6 100644
@@ -189,8 +189,7 @@ struct svm_nested_state {
 struct vcpu_sev_es_state {
 	/* SEV-ES support */
 	struct sev_es_save_area *vmsa;
-	struct ghcb *ghcb;
-	struct kvm_host_map ghcb_map;
+	bool ghcb_in_use;
 	bool received_first_sipi;
 	unsigned int ap_reset_hold_type;
 
@@ -200,6 +199,13 @@ struct vcpu_sev_es_state {
 	u64 ghcb_sa_gpa;
 	u32 ghcb_sa_alloc_len;
 	bool ghcb_sa_sync;
+
+	/*
+	 * SEV-ES support to hold the sw_exit_info return values to be
+	 * sync'ed to the GHCB when mapped.
+	 */
+	u64 ghcb_sw_exit_info_1;
+	u64 ghcb_sw_exit_info_2;
 };
 
 struct vcpu_svm {
@@ -614,6 +620,20 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm);
 void nested_vmcb02_compute_g_pat(struct vcpu_svm *svm);
 void svm_switch_vmcb(struct vcpu_svm *svm, struct kvm_vmcb_info *target_vmcb);
 
+static inline void svm_set_ghcb_sw_exit_info_1(struct kvm_vcpu *vcpu, u64 val)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	svm->sev_es.ghcb_sw_exit_info_1 = val;
+}
+
+static inline void svm_set_ghcb_sw_exit_info_2(struct kvm_vcpu *vcpu, u64 val)
+{
+	struct vcpu_svm *svm = to_svm(vcpu);
+
+	svm->sev_es.ghcb_sw_exit_info_2 = val;
+}
+
 extern struct kvm_x86_nested_ops svm_nested_ops;
 
 /* avic.c */
Signature · 1 line
2.25.1
#254 lore raw

Re: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

Quoted 36 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131 ++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c +3 -0
index 01ea257e17d6..c70f3f7e06a8 100644
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
        kvfree(svm->sev_es.ghcb_sa);
 }
 
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
+{
+       struct vmcb_control_area *control = &svm->vmcb->control;

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2838

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+       u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+               /* Unable to map GHCB from guest */
+               pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+               return -EFAULT;
+       }
+
+       return 0;
+}

There is a perf cost to this suggestion but it might make accessing the GHCB safer for KVM. Have you thought about just using kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf improvement in a follow up.

Since we cannot unmap GHCBs I don't think UPM will help here so we probably want to make these patches safe against malicious guests making GHCBs private. But maybe UPM does help?

author maintainer
#255 lore raw

RE: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

[AMD Official Use Only - General]

Hello Peter,

Quoted 33 lines
Quoted 33 lines

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131 
++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c index renamed from arch/x86/kvm/svm/sev.c +0 -0

01ea257e17d6..c70f3f7e06a8 100644

arch/x86/kvm/svm/sev.c +0 -0
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
        kvfree(svm->sev_es.ghcb_sa);
 }
 
Quoted 16 lines

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2838

Quoted 13 lines
arch/x86/kvm/svm/sev.c +13 -0
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct 
+kvm_host_map *map) {
+       struct vmcb_control_area *control = &svm->vmcb->control;
+       u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+               /* Unable to map GHCB from guest */
+               pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+               return -EFAULT;
+       }
+
+       return 0;
+}

There is a perf cost to this suggestion but it might make accessing the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here will be the GHCB write-back path (copying it back) before VMRUN: this will again hit the issue we have currently with kvm_write_guest() / copy_to_user(), when we use it to sync the scratch buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Thanks,
Ashish

Quoted 1 line

Since we cannot unmap GHCBs I don't think UPM will help here so we probably want to make these patches safe against malicious guests making GHCBs private. But maybe UPM does help?

#256 lore raw

Re: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

Quoted 63 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 33 lines
Quoted 33 lines

From: Brijesh Singh <[email protected]>

On VMGEXIT, sev_handle_vmgexit() creates a host mapping for the GHCB GPA, and unmaps it just before VM-entry. This long-lived GHCB map is used by the VMGEXIT handler through accessors such as ghcb_{set_get}_xxx().

A long-lived GHCB map can cause issue when SEV-SNP is enabled. When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change.

To eliminate the long-lived GHCB mapping, update the GHCB sync operations to explicitly map the GHCB before access and unmap it after access is complete. This requires that the setting of the GHCBs sw_exit_info_{1,2} fields be done during sev_es_sync_to_ghcb(), so create two new fields in the vcpu_svm struct to hold these values when required to be set outside of the GHCB mapping.

---
 arch/x86/kvm/svm/sev.c | 131
++++++++++++++++++++++++++---------------
 arch/x86/kvm/svm/svm.c |  12 ++--
 arch/x86/kvm/svm/svm.h |  24 +++++++-
 3 files changed, 111 insertions(+), 56 deletions(-)
arch/x86/kvm/svm/sev.c index renamed from arch/x86/kvm/svm/sev.c +0 -0

01ea257e17d6..c70f3f7e06a8 100644

arch/x86/kvm/svm/sev.c +0 -0
@@ -2823,15 +2823,40 @@ void sev_free_vcpu(struct kvm_vcpu *vcpu)
        kvfree(svm->sev_es.ghcb_sa);
 }
 
Quoted 16 lines

arch/x86/kvm/svm/sev.c void sev_free_vcpu(struct kvm_vcpu *vcpu) · line 2838

Quoted 13 lines
arch/x86/kvm/svm/sev.c +13 -0
+static inline int svm_map_ghcb(struct vcpu_svm *svm, struct
+kvm_host_map *map) {
+       struct vmcb_control_area *control = &svm->vmcb->control;
+       u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+
+       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+               /* Unable to map GHCB from guest */
+               pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
+               return -EFAULT;
+       }
+
+       return 0;
+}

There is a perf cost to this suggestion but it might make accessing the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here will be the GHCB write-back path (copying it back) before VMRUN: this will again hit the issue we have currently with kvm_write_guest() / copy_to_user(), when we use it to sync the scratch buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000
Quoted 8 lines

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Ah I didn't see that issue thanks for the pointer.

The patch description says "When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change." since if the guest were to convert the GHCB page to private when the host is using the GHCB the host could get an RMP violation right? That RMP violation would cause the host to crash unless we use some copy_to_user() type protections. I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

I was wrong about the importance of this though seanjc@ walked me through how UPM will solve this issue so no worries about this until the series is rebased on to UPM.

Quoted 5 lines

Thanks,
Ashish

Quoted 1 line

Since we cannot unmap GHCBs I don't think UPM will help here so we probably want to make these patches safe against malicious guests making GHCBs private. But maybe UPM does help?

author maintainer
#257 lore raw

RE: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

[AMD Official Use Only - General]

Hello Peter,

Quoted 23 lines
Quoted 14 lines
Quoted 3 lines

There is a perf cost to this suggestion but it might make accessing
the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here
will be the GHCB write-back path (copying it back) before VMRUN: this
will again hit the issue we have currently with
kvm_write_guest() / copy_to_user(), when we use it to sync the scratch
buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000
Quoted 8 lines

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail
due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Quoted 3 lines

Ah I didn't see that issue thanks for the pointer.

The patch description says "When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change." since if the guest were to convert the GHCB page to private when the host is using the GHCB the host could get an RMP violation right?

Right.

Quoted 1 line

That RMP violation would cause the host to crash unless we use some copy_to_user() type protections.

As such copy_to_user() will only swallow the RMP violation and return failure, so the host can retry the write.

Quoted 1 line

I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

We do have the protections for GHCB getting mapped to private specifically, there are new post_{map|unmap}_gfn functions added to verify if it is safe to map GHCB pages. There is a PSC spinlock added which protects again page state change for these mapped pages.
Below is the reference to this patch:
https://lore.kernel.org/lkml/[email protected]/T/#mafcaac7296eb9a92c0ea58730dbd3ca47a8e0756

But do note that there is protection only for GHCB pages and there is a need to add generic post_{map,unmap}_gfn() ops that can be used to verify that it's safe to map a given guest page in the hypervisor. This is a TODO right now and probably this is something which UPM can address more cleanly.

Quoted 1 line

I was wrong about the importance of this though seanjc@ walked me through how UPM will solve this issue so no worries about this until the series is rebased on to UPM.

Thanks,
Ashish

#258 lore raw

Re: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

Quoted 40 lines · Ashish

[AMD Official Use Only - General]

Hello Peter,

Quoted 23 lines
Quoted 14 lines
Quoted 3 lines

There is a perf cost to this suggestion but it might make accessing
the GHCB safer for KVM. Have you thought about just using
kvm_read_guest() or copy_from_user() to fully copy out the GCHB into a KVM owned buffer, then copying it back before the VMRUN. That way the KVM doesn't need to guard against page_state_changes on the GHCBs, that could be a perf ?>>improvement in a follow up.

Along with the performance costs you mentioned, the main concern here
will be the GHCB write-back path (copying it back) before VMRUN: this
will again hit the issue we have currently with
kvm_write_guest() / copy_to_user(), when we use it to sync the scratch
buffer back to GHCB. This can fail if guest RAM is mapped using huge-page(s) and RMP is 4K. Please refer to the patch/fix mentioned below, kvm_write_guest() potentially can fail before VMRUN in case of SNP :

commit 94ed878c2669532ebae8eb9b4503f19aa33cd7aa

Author: Ashish Kalra <[email protected]>
Date:   Mon Jun 6 22:28:01 2022 +0000
Quoted 8 lines

KVM: SVM: Sync the GHCB scratch buffer using already mapped ghcb

Using kvm_write_guest() to sync the GHCB scratch buffer can fail
due to host mapping being 2M, but RMP being 4K. The page fault handling in do_user_addr_fault() fails to split the 2M page to handle RMP fault due to it being called here in a non-preemptible context. Instead use the already kernel mapped ghcb to sync the scratch buffer when the scratch buffer is contained within the GHCB.

Quoted 3 lines

Ah I didn't see that issue thanks for the pointer.

The patch description says "When SEV-SNP is enabled the mapped GPA needs to be protected against a page state change." since if the guest were to convert the GHCB page to private when the host is using the GHCB the host could get an RMP violation right?

Right.

Quoted 1 line

That RMP violation would cause the host to crash unless we use some copy_to_user() type protections.

As such copy_to_user() will only swallow the RMP violation and return failure, so the host can retry the write.

Quoted 1 line

I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

Quoted 7 lines

We do have the protections for GHCB getting mapped to private specifically, there are new post_{map|unmap}_gfn functions added to verify if it is safe to map GHCB pages. There is a PSC spinlock added which protects again page state change for these mapped pages.
Below is the reference to this patch:
https://lore.kernel.org/lkml/[email protected]/T/#mafcaac7296eb9a92c0ea58730dbd3ca47a8e0756

But do note that there is protection only for GHCB pages and there is a need to add generic post_{map,unmap}_gfn() ops that can be used to verify that it's safe to map a given guest page in the hypervisor. This is a TODO right now and probably this is something which UPM can address more cleanly.

Thank you Ashish. I had missed that.

Can you help me understand why its OK to use kvm_write_guest() for the |snp_certs_data| inside of snp_handle_ext_guest_request() in patch 42/49? I would have thought we'd have the same 2M vs 4K mapping issues.

Quoted 5 lines
Quoted 2 lines

I was wrong about the importance of this though seanjc@ walked me through how UPM will solve this issue so no worries about this until the series is rebased on to UPM.

Thanks,
Ashish

author maintainer
#259 lore raw

RE: [PATCH Part2 v6 35/49] KVM: SVM: Remove the long-lived GHCB host map

[AMD Official Use Only - General]

Hello Peter,

Quoted 14 lines
Quoted 14 lines
Quoted 1 line

I don't see anything mechanism for this patch to add the page state change protection discussed. Can't another vCPU still convert the GHCB to private?

We do have the protections for GHCB getting mapped to private
specifically, there are new post_{map|unmap}_gfn functions added to verify if it is safe to map GHCB pages. There is a PSC spinlock added which protects again page state change for these mapped pages. Below is the reference to this patch:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
.kernel.org%2Flkml%2Fcover.1655761627.git.ashish.kalra%40amd.com%2FT%2
F%23mafcaac7296eb9a92c0ea58730dbd3ca47a8e0756&amp;data=05%7C01%7CAshis
h.Kalra%40amd.com%7C647218cdb2a040bf354e08da60fa2968%7C3dd8961fe4884e6
08e11a82d994e183d%7C0%7C0%7C637928924845082803%7CUnknown%7CTWFpbGZsb3d
8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
3000%7C%7C%7C&amp;sdata=ss8%2F5qualccXQero9phARIG2wvYhtp8SMdve3GglZeU%
3D&amp;reserved=0

Quoted 8 lines
Quoted 2 lines

But do note that there is protection only for GHCB pages and there is
a need to add generic post_{map,unmap}_gfn() ops that can be used to verify that it's safe to map a given guest page in the hypervisor. This is a TODO right now and probably this is something which UPM can address more cleanly.

Thank you Ashish. I had missed that.

Can you help me understand why its OK to use kvm_write_guest() for the |snp_certs_data| inside of snp_handle_ext_guest_request() in patch 42/49? I would have thought we'd have the same 2M vs 4K mapping issues.

Preemption is not disabled there, hence the RMP page fault handler can do the split of 2M to 4K on host pages without any issues.

Thanks,
Ashish

author maintainer
#260 lore raw

PATCH v6 36/49 KVM: SVM: Add support to handle GHCB GPA register VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP guests are required to perform a GHCB GPA registration. Before using a GHCB GPA for a vCPU the first time, a guest must register the vCPU GHCB GPA. If hypervisor can work with the guest requested GPA then it must respond back with the same GPA otherwise return -1.

On VMEXIT, Verify that GHCB GPA matches with the registered value. If a mismatch is detected then abort the guest.

---
 arch/x86/include/asm/sev-common.h |  8 ++++++++
 arch/x86/kvm/svm/sev.c            | 27 +++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.h            |  7 +++++++
 3 files changed, 42 insertions(+)
arch/x86/include/asm/sev-common.h +8 -0
index 539de6b93420..0a9055cdfae2 100644
@@ -59,6 +59,14 @@
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_POS	12
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_MASK	GENMASK_ULL(51, 0)
 
+/* Preferred GHCB GPA Request */
+#define GHCB_MSR_PREF_GPA_REQ		0x010
+#define GHCB_MSR_GPA_VALUE_POS		12
+#define GHCB_MSR_GPA_VALUE_MASK		GENMASK_ULL(51, 0)
+
+#define GHCB_MSR_PREF_GPA_RESP		0x011
+#define GHCB_MSR_PREF_GPA_NONE		0xfffffffffffff
+
 /* GHCB GPA Register */
 #define GHCB_MSR_REG_GPA_REQ		0x012
 #define GHCB_MSR_REG_GPA_REQ_VAL(v)			\
arch/x86/kvm/svm/sev.c +27 -0
index c70f3f7e06a8..6de48130e414 100644
@@ -3331,6 +3331,27 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 				  GHCB_MSR_INFO_MASK, GHCB_MSR_INFO_POS);
 		break;
 	}
+	case GHCB_MSR_PREF_GPA_REQ: {
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_NONE, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
+	case GHCB_MSR_REG_GPA_REQ: {
+		u64 gfn;
+
+		gfn = get_ghcb_msr_bits(svm, GHCB_MSR_GPA_VALUE_MASK,
+					GHCB_MSR_GPA_VALUE_POS);
+
+		svm->sev_es.ghcb_registered_gpa = gfn_to_gpa(gfn);
+
+		set_ghcb_msr_bits(svm, gfn, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_REG_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
 	case GHCB_MSR_TERM_REQ: {
 		u64 reason_set, reason_code;
 
@@ -3381,6 +3402,12 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		return 1;
 	}
 
+	/* SEV-SNP guest requires that the GHCB GPA must be registered */
+	if (sev_snp_guest(svm->vcpu.kvm) && !ghcb_gpa_is_registered(svm, ghcb_gpa)) {
+		vcpu_unimpl(&svm->vcpu, "vmgexit: GHCB GPA [%#llx] is not registered.\n", ghcb_gpa);
+		return -EINVAL;
+	}
+
 	ret = sev_es_validate_vmgexit(svm, &exit_code);
 	if (ret)
 		return ret;
arch/x86/kvm/svm/svm.h +7 -0
index c80352c9c0d6..54ff56cb6125 100644
@@ -206,6 +206,8 @@ struct vcpu_sev_es_state {
 	 */
 	u64 ghcb_sw_exit_info_1;
 	u64 ghcb_sw_exit_info_2;
+
+	u64 ghcb_registered_gpa;
 };
 
 struct vcpu_svm {
@@ -334,6 +336,11 @@ static inline bool sev_snp_guest(struct kvm *kvm)
 	return sev_es_guest(kvm) && sev->snp_active;
 }
 
+static inline bool ghcb_gpa_is_registered(struct vcpu_svm *svm, u64 val)
+{
+	return svm->sev_es.ghcb_registered_gpa == val;
+}
+
 static inline void vmcb_mark_all_dirty(struct vmcb *vmcb)
 {
 	vmcb->control.clean = 0;
Signature · 1 line
2.25.1
#261 lore raw

Re: [PATCH Part2 v6 36/49] KVM: SVM: Add support to handle GHCB GPA register VMGEXIT

Quoted 22 lines · * Ashish Kalra ([email protected]) wrote:

From: Brijesh Singh <[email protected]>

SEV-SNP guests are required to perform a GHCB GPA registration. Before using a GHCB GPA for a vCPU the first time, a guest must register the vCPU GHCB GPA. If hypervisor can work with the guest requested GPA then it must respond back with the same GPA otherwise return -1.

On VMEXIT, Verify that GHCB GPA matches with the registered value. If a mismatch is detected then abort the guest.

---
 arch/x86/include/asm/sev-common.h |  8 ++++++++
 arch/x86/kvm/svm/sev.c            | 27 +++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.h            |  7 +++++++
 3 files changed, 42 insertions(+)
arch/x86/include/asm/sev-common.h +0 -0
index 539de6b93420..0a9055cdfae2 100644

arch/x86/include/asm/sev-common.h line 65

Quoted 7 lines
arch/x86/include/asm/sev-common.h +4 -0
@@ -59,6 +59,14 @@
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_POS	12
 #define GHCB_MSR_AP_RESET_HOLD_RESULT_MASK	GENMASK_ULL(51, 0)
 
+/* Preferred GHCB GPA Request */
+#define GHCB_MSR_PREF_GPA_REQ		0x010
+#define GHCB_MSR_GPA_VALUE_POS		12
+#define GHCB_MSR_GPA_VALUE_MASK		GENMASK_ULL(51, 0)

Are the magic 51's in here fixed ?

Dave

Quoted 79 lines
arch/x86/include/asm/sev-common.h +3 -0
+#define GHCB_MSR_PREF_GPA_RESP		0x011
+#define GHCB_MSR_PREF_GPA_NONE		0xfffffffffffff
+
 /* GHCB GPA Register */
 #define GHCB_MSR_REG_GPA_REQ		0x012
 #define GHCB_MSR_REG_GPA_REQ_VAL(v)			\
arch/x86/kvm/svm/sev.c +27 -0
index c70f3f7e06a8..6de48130e414 100644
@@ -3331,6 +3331,27 @@ static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 				  GHCB_MSR_INFO_MASK, GHCB_MSR_INFO_POS);
 		break;
 	}
+	case GHCB_MSR_PREF_GPA_REQ: {
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_NONE, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_PREF_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
+	case GHCB_MSR_REG_GPA_REQ: {
+		u64 gfn;
+
+		gfn = get_ghcb_msr_bits(svm, GHCB_MSR_GPA_VALUE_MASK,
+					GHCB_MSR_GPA_VALUE_POS);
+
+		svm->sev_es.ghcb_registered_gpa = gfn_to_gpa(gfn);
+
+		set_ghcb_msr_bits(svm, gfn, GHCB_MSR_GPA_VALUE_MASK,
+				  GHCB_MSR_GPA_VALUE_POS);
+		set_ghcb_msr_bits(svm, GHCB_MSR_REG_GPA_RESP, GHCB_MSR_INFO_MASK,
+				  GHCB_MSR_INFO_POS);
+		break;
+	}
 	case GHCB_MSR_TERM_REQ: {
 		u64 reason_set, reason_code;
 
@@ -3381,6 +3402,12 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		return 1;
 	}
 
+	/* SEV-SNP guest requires that the GHCB GPA must be registered */
+	if (sev_snp_guest(svm->vcpu.kvm) && !ghcb_gpa_is_registered(svm, ghcb_gpa)) {
+		vcpu_unimpl(&svm->vcpu, "vmgexit: GHCB GPA [%#llx] is not registered.\n", ghcb_gpa);
+		return -EINVAL;
+	}
+
 	ret = sev_es_validate_vmgexit(svm, &exit_code);
 	if (ret)
 		return ret;
arch/x86/kvm/svm/svm.h +7 -0
index c80352c9c0d6..54ff56cb6125 100644
@@ -206,6 +206,8 @@ struct vcpu_sev_es_state {
 	 */
 	u64 ghcb_sw_exit_info_1;
 	u64 ghcb_sw_exit_info_2;
+
+	u64 ghcb_registered_gpa;
 };
 
 struct vcpu_svm {
@@ -334,6 +336,11 @@ static inline bool sev_snp_guest(struct kvm *kvm)
 	return sev_es_guest(kvm) && sev->snp_active;
 }
 
+static inline bool ghcb_gpa_is_registered(struct vcpu_svm *svm, u64 val)
+{
+	return svm->sev_es.ghcb_registered_gpa == val;
+}
+
 static inline void vmcb_mark_all_dirty(struct vmcb *vmcb)
 {
 	vmcb->control.clean = 0;
Signature · 1 line
2.25.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
author maintainer
#262 lore raw

PATCH v6 37/49 KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP VMs can ask the hypervisor to change the page state in the RMP table to be private or shared using the Page State Change MSR protocol as defined in the GHCB specification.

Before changing the page state in the RMP entry, lookup the page in the NPT to make sure that there is a valid mapping for it. If the mapping exist then try to find a workable page level between the NPT and RMP for the page. If the page is not mapped in the NPT, then create a fault such that it gets mapped before we change the page state in the RMP entry.

---
 arch/x86/include/asm/sev-common.h |   9 ++
 arch/x86/kvm/svm/sev.c            | 197 ++++++++++++++++++++++++++++++
 arch/x86/kvm/trace.h              |  34 ++++++
 arch/x86/kvm/x86.c                |   1 +
 4 files changed, 241 insertions(+)
arch/x86/include/asm/sev-common.h +9 -0
index 0a9055cdfae2..ee38f7408470 100644
@@ -93,6 +93,10 @@ enum psc_op {
 };
 
 #define GHCB_MSR_PSC_REQ		0x014
+#define GHCB_MSR_PSC_GFN_POS		12
+#define GHCB_MSR_PSC_GFN_MASK		GENMASK_ULL(39, 0)
+#define GHCB_MSR_PSC_OP_POS		52
+#define GHCB_MSR_PSC_OP_MASK		0xf
 #define GHCB_MSR_PSC_REQ_GFN(gfn, op)			\
 	/* GHCBData[55:52] */				\
 	(((u64)((op) & 0xf) << 52) |			\
@@ -102,6 +106,11 @@ enum psc_op {
 	GHCB_MSR_PSC_REQ)
 
 #define GHCB_MSR_PSC_RESP		0x015
+#define GHCB_MSR_PSC_ERROR_POS		32
+#define GHCB_MSR_PSC_ERROR_MASK		GENMASK_ULL(31, 0)
+#define GHCB_MSR_PSC_ERROR		GENMASK_ULL(31, 0)
+#define GHCB_MSR_PSC_RSVD_POS		12
+#define GHCB_MSR_PSC_RSVD_MASK		GENMASK_ULL(19, 0)
 #define GHCB_MSR_PSC_RESP_VAL(val)			\
 	/* GHCBData[63:32] */				\
 	(((u64)(val) & GENMASK_ULL(63, 32)) >> 32)
arch/x86/kvm/svm/sev.c +197 -0

215 lines not on this page · raw message

arch/x86/kvm/trace.h +34 -0
index 9b9bc5468103..79801e50344a 100644
@@ -7,6 +7,7 @@
 #include <asm/svm.h>
 #include <asm/clocksource.h>
 #include <asm/pvclock-abi.h>
+#include <asm/sev-common.h>
 
 #undef TRACE_SYSTEM
 #define TRACE_SYSTEM kvm
@@ -1755,6 +1756,39 @@ TRACE_EVENT(kvm_vmgexit_msr_protocol_exit,
 		  __entry->vcpu_id, __entry->ghcb_gpa, __entry->result)
 );
 
+/*
+ * Tracepoint for the SEV-SNP page state change processing
+ */
+#define psc_operation					\
+	{SNP_PAGE_STATE_PRIVATE, "private"},		\
+	{SNP_PAGE_STATE_SHARED,  "shared"}		\
+
+TRACE_EVENT(kvm_snp_psc,
+	TP_PROTO(unsigned int vcpu_id, u64 pfn, u64 gpa, u8 op, int level),
+	TP_ARGS(vcpu_id, pfn, gpa, op, level),
+
+	TP_STRUCT__entry(
+		__field(int, vcpu_id)
+		__field(u64, pfn)
+		__field(u64, gpa)
+		__field(u8, op)
+		__field(int, level)
+	),
+
+	TP_fast_assign(
+		__entry->vcpu_id = vcpu_id;
+		__entry->pfn = pfn;
+		__entry->gpa = gpa;
+		__entry->op = op;
+		__entry->level = level;
+	),
+
+	TP_printk("vcpu %u, pfn %llx, gpa %llx, op %s, level %d",
+		  __entry->vcpu_id, __entry->pfn, __entry->gpa,
+		  __print_symbolic(__entry->op, psc_operation),
+		  __entry->level)
+);
+
 #endif /* _TRACE_KVM_H */
 
 #undef TRACE_INCLUDE_PATH
arch/x86/kvm/x86.c +1 -0
index 50fff5202e7e..4a1d16231e30 100644
@@ -13066,6 +13066,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_enter);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_exit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_msr_protocol_enter);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_vmgexit_msr_protocol_exit);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_snp_psc);
 
 static int __init kvm_x86_init(void)
 {
Signature · 1 line
2.25.1
#263 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 50 lines
arch/x86/kvm/svm/sev.c +50 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, gpa_t gpa,
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then build the NPT.
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is registered. This enforcement is
+                        * required to avoid the cases where a page is marked private
+                        * in the RMP table but never gets cleanup during the VM
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva, page_level_size(level));
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable before adding the pages
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva, page_level_size(level));
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);

arch/x86/kvm/svm/sev.c static void set_ghcb_msr(struct vcpu_svm *svm, u64 value) · line 3395

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU unmapped the page
+                        * before we acquire the lock. Retry the PSC.
+                        */
+                       write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 35 lines
arch/x86/kvm/svm/sev.c +35 -0
+
+               /*
+                * Adjust the level so that we don't go higher than the backing
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op, level);
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn, level);
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level, sev->asid, false);
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
#264 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 56 lines · Peter Gonda
Quoted 56 lines
arch/x86/kvm/svm/sev.c +55 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, gpa_t gpa,
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then build the NPT.
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is registered. This enforcement is
+                        * required to avoid the cases where a page is marked private
+                        * in the RMP table but never gets cleanup during the VM
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva, page_level_size(level));
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable before adding the pages
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva, page_level_size(level));
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU unmapped the page
Quoted 9 lines

arch/x86/kvm/svm/sev.c static void set_ghcb_msr(struct vcpu_svm *svm, u64 value) · line 3395

Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+                        * before we acquire the lock. Retry the PSC.
+                        */
+                       write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 35 lines
Quoted 35 lines
arch/x86/kvm/svm/sev.c +35 -0
+
+               /*
+                * Adjust the level so that we don't go higher than the backing
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op, level);
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn, level);
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level, sev->asid, false);
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
maintainer
#265 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 67 lines · Alper Gun
Quoted 56 lines · Peter Gonda
Quoted 56 lines
arch/x86/kvm/svm/sev.c +55 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, gpa_t gpa,
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then build the NPT.
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is registered. This enforcement is
+                        * required to avoid the cases where a page is marked private
+                        * in the RMP table but never gets cleanup during the VM
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva, page_level_size(level));
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable before adding the pages
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva, page_level_size(level));
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU unmapped the page
Quoted 9 lines

arch/x86/kvm/svm/sev.c static void set_ghcb_msr(struct vcpu_svm *svm, u64 value) · line 3395

Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+                        * before we acquire the lock. Retry the PSC.
+                        */
+                       write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 6 lines

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning

Returning 0 here is ok because the PSC current index into the PSC structure will not be updated and the guest will then retry (see the loop in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

Quoted 41 lines

an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 35 lines
Quoted 35 lines
arch/x86/kvm/svm/sev.c +35 -0
+
+               /*
+                * Adjust the level so that we don't go higher than the backing
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op, level);
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn, level);
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level, sev->asid, false);
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
#266 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 76 lines · Tom Lendacky
Quoted 67 lines · Alper Gun
Quoted 56 lines · Peter Gonda
Quoted 56 lines
arch/x86/kvm/svm/sev.c +55 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, gpa_t gpa,
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then build the NPT.
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is registered. This enforcement is
+                        * required to avoid the cases where a page is marked private
+                        * in the RMP table but never gets cleanup during the VM
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva, page_level_size(level));
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable before adding the pages
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva, page_level_size(level));
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU unmapped the page
Quoted 9 lines

arch/x86/kvm/svm/sev.c static void set_ghcb_msr(struct vcpu_svm *svm, u64 value) · line 3395

Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+                        * before we acquire the lock. Retry the PSC.
+                        */
+                       write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 6 lines

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning

Quoted 6 lines

Returning 0 here is ok because the PSC current index into the PSC structure will not be updated and the guest will then retry (see the loop in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

But the host code updates the index. It doesn't leave the loop because rc is 0. The guest will think that it is successful.
rc = __snp_handle_page_state_change(vcpu, op, gpa, level); if (rc)
goto out;

Also the page state change request with MSR is not retried. It terminates the VM if the MSR request fails.

Quoted 42 lines
Quoted 42 lines

an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 35 lines
Quoted 35 lines
arch/x86/kvm/svm/sev.c +35 -0
+
+               /*
+                * Adjust the level so that we don't go higher than the backing
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op, level);
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn, level);
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level, sev->asid, false);
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
maintainer
#267 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 84 lines · Alper Gun
Quoted 76 lines · Tom Lendacky
Quoted 67 lines · Alper Gun
Quoted 56 lines · Peter Gonda
Quoted 56 lines
arch/x86/kvm/svm/sev.c +55 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, gpa_t gpa,
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then build the NPT.
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is registered. This enforcement is
+                        * required to avoid the cases where a page is marked private
+                        * in the RMP table but never gets cleanup during the VM
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva, page_level_size(level));
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable before adding the pages
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva, page_level_size(level));
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU unmapped the page
Quoted 9 lines

arch/x86/kvm/svm/sev.c static void set_ghcb_msr(struct vcpu_svm *svm, u64 value) · line 3395

Quoted 5 lines
arch/x86/kvm/svm/sev.c +5 -0
+                        * before we acquire the lock. Retry the PSC.
+                        */
+                       write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 6 lines

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning

Quoted 6 lines

Returning 0 here is ok because the PSC current index into the PSC structure will not be updated and the guest will then retry (see the loop in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

Quoted 8 lines

But the host code updates the index. It doesn't leave the loop because rc is 0. The guest will think that it is successful.
rc = __snp_handle_page_state_change(vcpu, op, gpa, level); if (rc)
goto out;

Also the page state change request with MSR is not retried. It terminates the VM if the MSR request fails.

Ah, right. I see what you mean. It should probably return a -EAGAIN instead of 0 and then the if (rc) check should be modified to specifically look for -EAGAIN and goto out after setting rc to 0.

But that does leave the MSR protocol open to the problem that you mention, so, yes, retry logic in snp_handle_page_state_change() for a -EAGAIN seems reasonable.

Thanks,
Tom

Quoted 43 lines
Quoted 43 lines
Quoted 43 lines

an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 35 lines
Quoted 35 lines
arch/x86/kvm/svm/sev.c +35 -0
+
+               /*
+                * Adjust the level so that we don't go higher than the backing
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op, level);
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn, level);
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level, sev->asid, false);
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
#268 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 105 lines · Tom Lendacky
Quoted 95 lines · Alper Gun
Quoted 87 lines · Tom Lendacky
Quoted 78 lines · Alper Gun
Quoted 67 lines · Peter Gonda
Quoted 67 lines
(quoted diff) +2 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, 

enum psc_op op, gpa_t gpa,
+                                         int level)

arch/x86/kvm/svm/sev.c +12 -0
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then 

build the NPT.
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level); +               if (rc)
+                       return -EINVAL;

arch/x86/kvm/svm/sev.c +9 -0
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is 

registered. This enforcement is
+                        * required to avoid the cases where a page is marked private
+                        * in the RMP table but never gets cleanup during the VM
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva, page_level_size(level));
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable before adding the pages
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva, page_level_size(level));
+                       mmap_write_unlock(kvm->mm); +                       if (rc)
+                               return -EINVAL;
+               }

arch/x86/kvm/svm/sev.c +4 -0
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, 

&npt_level);
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU unmapped the page
+                        * before we acquire the lock. Retry the

Quoted 9 lines
Quoted 5 lines

PSC.
+                        */
+ write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 6 lines

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning

Quoted 6 lines

Returning 0 here is ok because the PSC current index into the PSC structure will not be updated and the guest will then retry (see the loop in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

Quoted 8 lines

But the host code updates the index. It doesn't leave the loop because rc is 0. The guest will think that it is successful.
rc = __snp_handle_page_state_change(vcpu, op, gpa, level); if (rc)
goto out;

Also the page state change request with MSR is not retried. It terminates the VM if the MSR request fails.

Quoted 7 lines

Ah, right. I see what you mean. It should probably return a -EAGAIN instead of 0 and then the if (rc) check should be modified to specifically look for -EAGAIN and goto out after setting rc to 0.

But that does leave the MSR protocol open to the problem that you mention, so, yes, retry logic in snp_handle_page_state_change() for a -EAGAIN seems reasonable.

Thanks,
Tom

I believe it makes more sense to add the retry logic within __snp_handle_page_state_change() itself, as that will make it work for both the GHCB MSR protocol and the GHCB VMGEXIT requests.

Thanks, Ashish

Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines

an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 40 lines
Quoted 40 lines

+
+               /*
+                * Adjust the level so that we don't go higher than the backing
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level); +
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op, level);
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn, level);
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level, sev->asid, false);
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }

arch/x86/kvm/svm/sev.c +5 -0
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx 

pfn %llx level %d rc %d\n",
+                                          op, gpa, pfn, level, rc); +                       return rc;
+               }

arch/x86/kvm/svm/sev.c +7 -0
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
#269 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 118 lines · Ashish Kalra
Quoted 115 lines · Tom Lendacky
Quoted 97 lines · Alper Gun
Quoted 87 lines · Tom Lendacky
Quoted 78 lines · Alper Gun
Quoted 67 lines · Peter Gonda
Quoted 67 lines
(quoted diff) +2 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu,

enum psc_op op, gpa_t gpa,

(quoted diff) +13 -0
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then

build the NPT.

(quoted diff) +13 -0
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is

registered. This enforcement is
+ * required to avoid the cases where a
page is marked private
+ * in the RMP table but never gets
cleanup during the VM

(quoted diff) +4 -0
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva,

page_level_size(level));

(quoted diff) +6 -0
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable

before adding the pages

(quoted diff) +4 -0
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva,

page_level_size(level));

(quoted diff) +8 -0
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn,

&npt_level);

(quoted diff) +3 -0
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU

unmapped the page
+ * before we acquire the lock. Retry the

Quoted 9 lines

Review

Quoted 5 lines

PSC.

(quoted diff) +4 -0
+                        */
+ write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 6 lines

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning

Quoted 7 lines

Returning 0 here is ok because the PSC current index into the PSC structure will not be updated and the guest will then retry (see the loop
in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

Quoted 8 lines

But the host code updates the index. It doesn't leave the loop because rc is 0. The guest will think that it is successful.
rc = __snp_handle_page_state_change(vcpu, op, gpa, level); if (rc)
goto out;

Also the page state change request with MSR is not retried. It terminates the VM if the MSR request fails.

Ah, right. I see what you mean. It should probably return a -EAGAIN instead of 0 and then the if (rc) check should be modified to specifically look for -EAGAIN and goto out after setting rc to 0.

But that does leave the MSR protocol open to the problem that you mention, so, yes, retry logic in snp_handle_page_state_change() for a -EAGAIN seems reasonable.

Quoted 6 lines
Quoted 2 lines

Thanks,
Tom

I believe it makes more sense to add the retry logic within __snp_handle_page_state_change() itself, as that will make it work for both the GHCB MSR protocol and the GHCB VMGEXIT requests.

You are suggesting we just retry 'kvm_mmu_get_tdp_walk' inside of __snp_handle_page_state_change()? That should work but how many times do we retry? If we return EAGAIN or error we can leave it up to the caller

Quoted 52 lines

Thanks, Ashish

Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines

an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 40 lines
Quoted 40 lines
(quoted diff) +3 -0
+
+               /*
+                * Adjust the level so that we don't go higher

than the backing

(quoted diff) +5 -0
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op,

level);

(quoted diff) +4 -0
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn,

level);

(quoted diff) +3 -0
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level,

sev->asid, false);

(quoted diff) +10 -0
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx

pfn %llx level %d rc %d\n",

(quoted diff) +10 -0
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
author maintainer
#270 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 122 lines · Peter Gonda
Quoted 121 lines · Ashish Kalra
Quoted 117 lines · Tom Lendacky
Quoted 97 lines · Alper Gun
Quoted 87 lines · Tom Lendacky
Quoted 78 lines · Alper Gun
Quoted 67 lines · Peter Gonda
Quoted 67 lines
(quoted diff) +2 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu,

enum psc_op op, gpa_t gpa,

(quoted diff) +13 -0
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then

build the NPT.

(quoted diff) +13 -0
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is

registered. This enforcement is
+ * required to avoid the cases where a
page is marked private
+ * in the RMP table but never gets
cleanup during the VM

(quoted diff) +4 -0
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva,

page_level_size(level));

(quoted diff) +6 -0
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable

before adding the pages

(quoted diff) +4 -0
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva,

page_level_size(level));

(quoted diff) +8 -0
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn,

&npt_level);

(quoted diff) +3 -0
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU

unmapped the page
+ * before we acquire the lock. Retry the

Quoted 9 lines

Review

Quoted 5 lines

PSC.

(quoted diff) +4 -0
+                        */
+ write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 6 lines

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning

Quoted 7 lines

Returning 0 here is ok because the PSC current index into the PSC structure will not be updated and the guest will then retry (see the loop
in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

Quoted 8 lines

But the host code updates the index. It doesn't leave the loop because rc is 0. The guest will think that it is successful.
rc = __snp_handle_page_state_change(vcpu, op, gpa, level); if (rc)
goto out;

Also the page state change request with MSR is not retried. It terminates the VM if the MSR request fails.

Ah, right. I see what you mean. It should probably return a -EAGAIN instead of 0 and then the if (rc) check should be modified to specifically look for -EAGAIN and goto out after setting rc to 0.

But that does leave the MSR protocol open to the problem that you mention, so, yes, retry logic in snp_handle_page_state_change() for a -EAGAIN seems reasonable.

Thanks,
Tom

Quoted 9 lines
Quoted 3 lines

I believe it makes more sense to add the retry logic within __snp_handle_page_state_change() itself, as that will make it work for both the GHCB MSR protocol and the GHCB VMGEXIT requests.

You are suggesting we just retry 'kvm_mmu_get_tdp_walk' inside of __snp_handle_page_state_change()? That should work but how many times do we retry? If we return EAGAIN or error we can leave it up to the caller

Ok, we return -EAGAIN here and then let the caller in snp_handle_page_state_change() or sev_handle_vmgexit_msr_protocol() (in case of GHCB MSR protocol) do the retries.

But, the question still remains, how may retry attempts should the caller attempt ?

Thanks,
Ashish

Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines

an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 40 lines
Quoted 40 lines
(quoted diff) +3 -0
+
+               /*
+                * Adjust the level so that we don't go higher

than the backing

(quoted diff) +5 -0
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op,

level);

(quoted diff) +4 -0
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn,

level);

(quoted diff) +3 -0
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level,

sev->asid, false);

(quoted diff) +10 -0
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx

pfn %llx level %d rc %d\n",

(quoted diff) +10 -0
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
#271 lore raw

Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based Page State Change VMGEXIT

Quoted 133 lines · Ashish
Quoted 122 lines · Peter Gonda
Quoted 121 lines · Ashish Kalra
Quoted 117 lines · Tom Lendacky
Quoted 97 lines · Alper Gun
Quoted 87 lines · Tom Lendacky
Quoted 78 lines · Alper Gun
Quoted 67 lines · Peter Gonda
Quoted 67 lines
(quoted diff) +2 -0
+
+static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu,

enum psc_op op, gpa_t gpa,

(quoted diff) +13 -0
+                                         int level)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
+       struct kvm *kvm = vcpu->kvm;
+       int rc, npt_level;
+       kvm_pfn_t pfn;
+       gpa_t gpa_end;
+
+       gpa_end = gpa + page_level_size(level);
+
+       while (gpa < gpa_end) {
+               /*
+                * If the gpa is not present in the NPT then

build the NPT.

(quoted diff) +13 -0
+                */
+               rc = snp_check_and_build_npt(vcpu, gpa, level);
+               if (rc)
+                       return -EINVAL;
+
+               if (op == SNP_PAGE_STATE_PRIVATE) {
+                       hva_t hva;
+
+                       if (snp_gpa_to_hva(kvm, gpa, &hva))
+                               return -EINVAL;
+
+                       /*
+                        * Verify that the hva range is

registered. This enforcement is
+ * required to avoid the cases where a
page is marked private
+ * in the RMP table but never gets
cleanup during the VM

(quoted diff) +4 -0
+                        * termination path.
+                        */
+                       mutex_lock(&kvm->lock);
+                       rc = is_hva_registered(kvm, hva,

page_level_size(level));

(quoted diff) +6 -0
+                       mutex_unlock(&kvm->lock);
+                       if (!rc)
+                               return -EINVAL;
+
+                       /*
+                        * Mark the userspace range unmerable

before adding the pages

(quoted diff) +4 -0
+                        * in the RMP table.
+                        */
+                       mmap_write_lock(kvm->mm);
+                       rc = snp_mark_unmergable(kvm, hva,

page_level_size(level));

(quoted diff) +8 -0
+                       mmap_write_unlock(kvm->mm);
+                       if (rc)
+                               return -EINVAL;
+               }
+
+               write_lock(&kvm->mmu_lock);
+
+               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn,

&npt_level);

(quoted diff) +3 -0
+               if (!rc) {
+                       /*
+                        * This may happen if another vCPU

unmapped the page
+ * before we acquire the lock. Retry the

Quoted 9 lines

Review

Quoted 5 lines

PSC.

(quoted diff) +4 -0
+                        */
+ write_unlock(&kvm->mmu_lock);
+                       return 0;
+               }

I think we want to return -EAGAIN or similar if we want the caller to retry, right? I think returning 0 here hides the error.

Quoted 6 lines

The problem here is that the caller(linux guest kernel) doesn't retry if PSC fails. The current implementation in the guest kernel is that if a page state change request fails, it terminates the VM with GHCB_TERM_PSC reason.
Returning 0 here is not a good option because it will fail the PSC silently and will probably cause a nested RMP fault later. Returning

Quoted 7 lines

Returning 0 here is ok because the PSC current index into the PSC structure will not be updated and the guest will then retry (see the loop
in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

Quoted 8 lines

But the host code updates the index. It doesn't leave the loop because rc is 0. The guest will think that it is successful.
rc = __snp_handle_page_state_change(vcpu, op, gpa, level); if (rc)
goto out;

Also the page state change request with MSR is not retried. It terminates the VM if the MSR request fails.

Ah, right. I see what you mean. It should probably return a -EAGAIN instead of 0 and then the if (rc) check should be modified to specifically look for -EAGAIN and goto out after setting rc to 0.

But that does leave the MSR protocol open to the problem that you mention, so, yes, retry logic in snp_handle_page_state_change() for a -EAGAIN seems reasonable.

Thanks,
Tom

Quoted 9 lines
Quoted 3 lines

I believe it makes more sense to add the retry logic within __snp_handle_page_state_change() itself, as that will make it work for both the GHCB MSR protocol and the GHCB VMGEXIT requests.

You are suggesting we just retry 'kvm_mmu_get_tdp_walk' inside of __snp_handle_page_state_change()? That should work but how many times do we retry? If we return EAGAIN or error we can leave it up to the caller

Quoted 6 lines

Ok, we return -EAGAIN here and then let the caller in snp_handle_page_state_change() or sev_handle_vmgexit_msr_protocol() (in case of GHCB MSR protocol) do the retries.

But, the question still remains, how may retry attempts should the caller attempt ?

With UPM I don't think we need to deal with this particular case, since we don't need to walk the NPT to determine the PFN. The PSC will simply get forward to userspace, and userspace will (generally):

for shared->private:
- deallocate page in shared pool
- allocate page in private pool
- issue KVM_MEM_ENCRYPT_REG_REGION on the GFN to switch it to private in the KVM xarray and RMP table (and zap current NPT mapping)
- resume guest
- guest faults on GFN and KVM MMU sees that it is private and maps the GFN to the corresponding PFN in the private pool, which should be reliably obtainable since it is pinned

for private->shared:
- deallocate page in private pool (which will switch it to shared in RMP table so it can be safely released back to host)
- allocate page in shared pool
- issue KVM_MEM_ENCRYPT_UNREG_REGION on the GFN to switch it to shared in the KVM xarray (and zap current NPT mapping)
- resume guest
- guest faults on GFN and KVM MMU sees that it is shared and handles it just like it would a normal non-SEV guest, so we don't ever need to acquire the specific PFN backing the HVA since they are implicitly shared, so no need anymore for kvm_mmu_get_tdp_walk() helper

(also no need for pre-mapping into TDP via kvm_mmu_map_tdp_page() in this case, but not sure that was needed even without UPM, seems more like an optimization to avoid a 2nd #NPF. I guess we still have the option with UPM though if it seems justified, but it would likely happen in {REG,UNREG}_REGION in that case rather than SNP-specific hooks)

There may be some other edge cases to consider, but I'm not aware of any sequences that aren't clearly misbehavior on the part of userspace/guest, in which case terminating at the host/guest level seems reasonable.

-Mike

Quoted 53 lines

Thanks,
Ashish

Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines
Quoted 49 lines

an error also terminates the guest immediately with current guest implementation. I think the best approach here is adding a retry logic to this function. Retrying without returning an error should help it work because snp_check_and_build_npt will be called again and in the second attempt this should work.

Quoted 40 lines
Quoted 40 lines
(quoted diff) +3 -0
+
+               /*
+                * Adjust the level so that we don't go higher

than the backing

(quoted diff) +5 -0
+                * page level.
+                */
+               level = min_t(size_t, level, npt_level);
+
+               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op,

level);

(quoted diff) +4 -0
+
+               switch (op) {
+               case SNP_PAGE_STATE_SHARED:
+                       rc = snp_make_page_shared(kvm, gpa, pfn,

level);

(quoted diff) +3 -0
+                       break;
+               case SNP_PAGE_STATE_PRIVATE:
+                       rc = rmp_make_private(pfn, gpa, level,

sev->asid, false);

(quoted diff) +10 -0
+                       break;
+               default:
+                       rc = -EINVAL;
+                       break;
+               }
+
+               write_unlock(&kvm->mmu_lock);
+
+               if (rc) {
+                       pr_err_ratelimited("Error op %d gpa %llx

pfn %llx level %d rc %d\n",

(quoted diff) +10 -0
+                                          op, gpa, pfn, level, rc);
+                       return rc;
+               }
+
+               gpa = gpa + page_level_size(level);
+       }
+
+       return 0;
+}
+
author maintainer
#272 lore raw

PATCH v6 38/49 KVM: SVM: Add support to handle Page State Change VMGEXIT

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

SEV-SNP VMs can ask the hypervisor to change the page state in the RMP table to be private or shared using the Page State Change NAE event as defined in the GHCB specification version 2.

---
 arch/x86/include/asm/sev-common.h |  7 +++
 arch/x86/kvm/svm/sev.c            | 79 +++++++++++++++++++++++++++++--
 2 files changed, 81 insertions(+), 5 deletions(-)
arch/x86/include/asm/sev-common.h +7 -0
index ee38f7408470..1b111cde8c82 100644
@@ -130,6 +130,13 @@ enum psc_op {
 /* SNP Page State Change NAE event */
 #define VMGEXIT_PSC_MAX_ENTRY		253
 
+/* The page state change hdr structure in not valid */
+#define PSC_INVALID_HDR			1
+/* The hdr.cur_entry or hdr.end_entry is not valid */
+#define PSC_INVALID_ENTRY		2
+/* Page state change encountered undefined error */
+#define PSC_UNDEF_ERR			3
+
 struct psc_hdr {
 	u16 cur_entry;
 	u16 end_entry;
arch/x86/kvm/svm/sev.c +74 -5

126 lines not on this page · raw message

Signature · 1 line
2.25.1
author maintainer
#273 lore raw

PATCH v6 39/49 KVM: SVM: Introduce ops for the post gfn map and unmap

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest VM, the guest memory pages can either be a private or shared. A write from the hypervisor goes through the RMP checks. If hardware sees that hypervisor is attempting to write to a guest private page, then it triggers an RMP violation #PF.

To avoid the RMP violation with GHCB pages, added new post_{map,unmap}_gfn functions to verify if its safe to map GHCB pages. Uses a spinlock to protect against the page state change for existing mapped pages.

Need to add generic post_{map,unmap}_gfn() ops that can be used to verify that its safe to map a given guest page in the hypervisor.

This patch will need to be revisited later after consensus is reached on how to manage guest private memory as probably UPM private memslots will be able to handle this page state change more gracefully.

Signed-off by: Ashish Kalra <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  3 ++
 arch/x86/kvm/svm/sev.c             | 48 ++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.c             |  3 ++
 arch/x86/kvm/svm/svm.h             | 11 +++++++
 5 files changed, 64 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index e0068e702692..2dd2bc0cf4c3 100644
@@ -130,6 +130,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
 KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
+KVM_X86_OP(update_protected_guest_state)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +3 -0
index 49b217dc8d7e..8abc0e724f5c 100644
@@ -1522,7 +1522,10 @@ struct kvm_x86_ops {
 	unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
 	void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+
 	void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+
+	int (*update_protected_guest_state)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/svm/sev.c +46 -2

86 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +3 -0
index b24e0171cbf2..1c8e035ba011 100644
@@ -4734,7 +4734,10 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
 	.alloc_apic_backing_page = svm_alloc_apic_backing_page,
+
 	.rmp_page_level_adjust = sev_rmp_page_level_adjust,
+
+	.update_protected_guest_state = sev_snp_update_protected_guest_state,
 };
 
 /*
arch/x86/kvm/svm/svm.h +11 -0
index 54ff56cb6125..3fd95193ed8d 100644
@@ -79,19 +79,25 @@ struct kvm_sev_info {
 	bool active;		/* SEV enabled guest */
 	bool es_active;		/* SEV-ES enabled guest */
 	bool snp_active;	/* SEV-SNP enabled guest */
+
 	unsigned int asid;	/* ASID used for this guest */
 	unsigned int handle;	/* SEV firmware handle */
 	int fd;			/* SEV device fd */
+
 	unsigned long pages_locked; /* Number of pages locked */
 	struct list_head regions_list;  /* List of registered regions */
+
 	u64 ap_jump_table;	/* SEV-ES AP Jump Table address */
+
 	struct kvm *enc_context_owner; /* Owner of copied encryption context */
 	struct list_head mirror_vms; /* List of VMs mirroring */
 	struct list_head mirror_entry; /* Use as a list entry of mirrors */
 	struct misc_cg *misc_cg; /* For misc cgroup accounting */
 	atomic_t migration_in_progress;
+
 	u64 snp_init_flags;
 	void *snp_context;      /* SNP guest context page */
+	spinlock_t psc_lock;
 };
 
 struct kvm_svm {
@@ -702,6 +708,11 @@ void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
 void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code);
+void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu);
+int sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu);
 
 /* vmenter.S */
 
Signature · 1 line
2.25.1
#274 lore raw

Re: [PATCH Part2 v6 39/49] KVM: SVM: Introduce ops for the post gfn map and unmap

Quoted 109 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest VM, the guest memory pages can either be a private or shared. A write from the hypervisor goes through the RMP checks. If hardware sees that hypervisor is attempting to write to a guest private page, then it triggers an RMP violation #PF.

To avoid the RMP violation with GHCB pages, added new post_{map,unmap}_gfn functions to verify if its safe to map GHCB pages. Uses a spinlock to protect against the page state change for existing mapped pages.

Need to add generic post_{map,unmap}_gfn() ops that can be used to verify that its safe to map a given guest page in the hypervisor.

This patch will need to be revisited later after consensus is reached on how to manage guest private memory as probably UPM private memslots will be able to handle this page state change more gracefully.

Signed-off by: Ashish Kalra <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  3 ++
 arch/x86/kvm/svm/sev.c             | 48 ++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.c             |  3 ++
 arch/x86/kvm/svm/svm.h             | 11 +++++++
 5 files changed, 64 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index e0068e702692..2dd2bc0cf4c3 100644
@@ -130,6 +130,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
 KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
+KVM_X86_OP(update_protected_guest_state)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +3 -0
index 49b217dc8d7e..8abc0e724f5c 100644
@@ -1522,7 +1522,10 @@ struct kvm_x86_ops {
        unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
        void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+
        void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+
+       int (*update_protected_guest_state)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/svm/sev.c +14 -2
index cb2d1bbb862b..4ed90331bca0 100644
@@ -341,6 +341,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
                if (ret)
                        goto e_free;
 
+               spin_lock_init(&sev->psc_lock);
                ret = sev_snp_init(&argp->error);
        } else {
                ret = sev_platform_init(&argp->error);
@@ -2828,19 +2829,28 @@ static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
 {
        struct vmcb_control_area *control = &svm->vmcb->control;
        u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
 
-       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+       if (kvm_vcpu_map(vcpu, gfn, map)) {
                /* Unable to map GHCB from guest */
                pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
                return -EFAULT;
        }
 
+       if (sev_post_map_gfn(vcpu->kvm, map->gfn, map->pfn)) {
+               kvm_vcpu_unmap(vcpu, map, false);
+               return -EBUSY;
+       }
+
        return 0;
 }
 
 static inline void svm_unmap_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
 {
-       kvm_vcpu_unmap(&svm->vcpu, map, true);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+
+       kvm_vcpu_unmap(vcpu, map, true);
+       sev_post_unmap_gfn(vcpu->kvm, map->gfn, map->pfn);
 }
 
 static void dump_ghcb(struct vcpu_svm *svm)
@@ -3383,6 +3393,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
                                return PSC_UNDEF_ERR;
                }
 
+               spin_lock(&sev->psc_lock);
+
                write_lock(&kvm->mmu_lock);
 
                rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);

arch/x86/kvm/svm/sev.c static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, · line 3432

Quoted 4 lines
arch/x86/kvm/svm/sev.c +1 -0
@@ -3417,6 +3429,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
 
                write_unlock(&kvm->mmu_lock);
 
+               spin_unlock(&sev->psc_lock);

There is a corner case where the psc_lock is not released. If kvm_mmu_get_tdp_walk fails, the lock will be kept and will cause soft lockup.

Quoted 98 lines
arch/x86/kvm/svm/sev.c +31 -0
+
                if (rc) {
                        pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
                                           op, gpa, pfn, level, rc);
@@ -3965,3 +3979,33 @@ void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level)
        /* Adjust the level to keep the NPT and RMP in sync */
        *level = min_t(size_t, *level, rmp_level);
 }
+
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       int level;
+
+       if (!sev_snp_guest(kvm))
+               return 0;
+
+       spin_lock(&sev->psc_lock);
+
+       /* If pfn is not added as private then fail */
+       if (snp_lookup_rmpentry(pfn, &level) == 1) {
+               spin_unlock(&sev->psc_lock);
+               pr_err_ratelimited("failed to map private gfn 0x%llx pfn 0x%llx\n", gfn, pfn);
+               return -EBUSY;
+       }
+
+       return 0;
+}
+
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+
+       if (!sev_snp_guest(kvm))
+               return;
+
+       spin_unlock(&sev->psc_lock);
+}
arch/x86/kvm/svm/svm.c +3 -0
index b24e0171cbf2..1c8e035ba011 100644
@@ -4734,7 +4734,10 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
        .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
        .alloc_apic_backing_page = svm_alloc_apic_backing_page,
+
        .rmp_page_level_adjust = sev_rmp_page_level_adjust,
+
+       .update_protected_guest_state = sev_snp_update_protected_guest_state,
 };
 
 /*
arch/x86/kvm/svm/svm.h +11 -1
index 54ff56cb6125..3fd95193ed8d 100644
@@ -79,19 +79,25 @@ struct kvm_sev_info {
        bool active;            /* SEV enabled guest */
        bool es_active;         /* SEV-ES enabled guest */
        bool snp_active;        /* SEV-SNP enabled guest */
+
        unsigned int asid;      /* ASID used for this guest */
        unsigned int handle;    /* SEV firmware handle */
        int fd;                 /* SEV device fd */
+
        unsigned long pages_locked; /* Number of pages locked */
        struct list_head regions_list;  /* List of registered regions */
+
        u64 ap_jump_table;      /* SEV-ES AP Jump Table address */
+
        struct kvm *enc_context_owner; /* Owner of copied encryption context */
        struct list_head mirror_vms; /* List of VMs mirroring */
        struct list_head mirror_entry; /* Use as a list entry of mirrors */
        struct misc_cg *misc_cg; /* For misc cgroup accounting */
        atomic_t migration_in_progress;
+
        u64 snp_init_flags;
        void *snp_context;      /* SNP guest context page */
+       spinlock_t psc_lock;
 };
 
 struct kvm_svm {
@@ -702,6 +708,11 @@ void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
 void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code);
+void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu);
+int sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu);
 
 /* vmenter.S */
 
--

2.25.1

#275 lore raw

Re: [PATCH Part2 v6 39/49] KVM: SVM: Introduce ops for the post gfn map and unmap

Quoted 165 lines · Alper Gun
Quoted 109 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest VM, the guest memory pages can either be a private or shared. A write from the hypervisor goes through the RMP checks. If hardware sees that hypervisor is attempting to write to a guest private page, then it triggers an RMP violation #PF.

To avoid the RMP violation with GHCB pages, added new post_{map,unmap}_gfn functions to verify if its safe to map GHCB pages. Uses a spinlock to protect against the page state change for existing mapped pages.

Need to add generic post_{map,unmap}_gfn() ops that can be used to verify that its safe to map a given guest page in the hypervisor.

This patch will need to be revisited later after consensus is reached on how to manage guest private memory as probably UPM private memslots will be able to handle this page state change more gracefully.

Signed-off by: Ashish Kalra <[email protected]>

---
 arch/x86/include/asm/kvm-x86-ops.h |  1 +
 arch/x86/include/asm/kvm_host.h    |  3 ++
 arch/x86/kvm/svm/sev.c             | 48 ++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.c             |  3 ++
 arch/x86/kvm/svm/svm.h             | 11 +++++++
 5 files changed, 64 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index e0068e702692..2dd2bc0cf4c3 100644
@@ -130,6 +130,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)
 KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
 KVM_X86_OP(alloc_apic_backing_page)
 KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
+KVM_X86_OP(update_protected_guest_state)
 
 #undef KVM_X86_OP
 #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +3 -0
index 49b217dc8d7e..8abc0e724f5c 100644
@@ -1522,7 +1522,10 @@ struct kvm_x86_ops {
        unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
        void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+
        void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+
+       int (*update_protected_guest_state)(struct kvm_vcpu *vcpu);
 };
 
 struct kvm_x86_nested_ops {
arch/x86/kvm/svm/sev.c +14 -2
index cb2d1bbb862b..4ed90331bca0 100644
@@ -341,6 +341,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
                if (ret)
                        goto e_free;
 
+               spin_lock_init(&sev->psc_lock);
                ret = sev_snp_init(&argp->error);
        } else {
                ret = sev_platform_init(&argp->error);
@@ -2828,19 +2829,28 @@ static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
 {
        struct vmcb_control_area *control = &svm->vmcb->control;
        u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
 
-       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+       if (kvm_vcpu_map(vcpu, gfn, map)) {
                /* Unable to map GHCB from guest */
                pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
                return -EFAULT;
        }
 
+       if (sev_post_map_gfn(vcpu->kvm, map->gfn, map->pfn)) {
+               kvm_vcpu_unmap(vcpu, map, false);
+               return -EBUSY;
+       }
+
        return 0;
 }
 
 static inline void svm_unmap_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
 {
-       kvm_vcpu_unmap(&svm->vcpu, map, true);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+
+       kvm_vcpu_unmap(vcpu, map, true);
+       sev_post_unmap_gfn(vcpu->kvm, map->gfn, map->pfn);
 }
 
 static void dump_ghcb(struct vcpu_svm *svm)
@@ -3383,6 +3393,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
                                return PSC_UNDEF_ERR;
                }
 
+               spin_lock(&sev->psc_lock);
+
                write_lock(&kvm->mmu_lock);
 
                rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);

arch/x86/kvm/svm/sev.c static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, · line 3432

Quoted 4 lines
arch/x86/kvm/svm/sev.c +1 -0
@@ -3417,6 +3429,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
 
                write_unlock(&kvm->mmu_lock);
 
+               spin_unlock(&sev->psc_lock);

There is a corner case where the psc_lock is not released. If kvm_mmu_get_tdp_walk fails, the lock will be kept and will cause soft lockup.

Quoted 45 lines
arch/x86/kvm/svm/sev.c +31 -0
+
                if (rc) {
                        pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
                                           op, gpa, pfn, level, rc);
@@ -3965,3 +3979,33 @@ void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level)
        /* Adjust the level to keep the NPT and RMP in sync */
        *level = min_t(size_t, *level, rmp_level);
 }
+
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       int level;
+
+       if (!sev_snp_guest(kvm))
+               return 0;
+
+       spin_lock(&sev->psc_lock);
+
+       /* If pfn is not added as private then fail */
+       if (snp_lookup_rmpentry(pfn, &level) == 1) {
+               spin_unlock(&sev->psc_lock);
+               pr_err_ratelimited("failed to map private gfn 0x%llx pfn 0x%llx\n", gfn, pfn);
+               return -EBUSY;
+       }
+
+       return 0;
+}
+
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+
+       if (!sev_snp_guest(kvm))
+               return;
+
+       spin_unlock(&sev->psc_lock);
+}
arch/x86/kvm/svm/svm.c +0 -0
index b24e0171cbf2..1c8e035ba011 100644
@@ -4734,7 +4734,10 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
        .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
Quoted 6 lines

arch/x86/kvm/svm/svm.c static struct kvm_x86_ops svm_x86_ops __initdata = { · line 4741

Quoted 6 lines
arch/x86/kvm/svm/svm.c +3 -0
        .alloc_apic_backing_page = svm_alloc_apic_backing_page,
+
        .rmp_page_level_adjust = sev_rmp_page_level_adjust,
+
+       .update_protected_guest_state = sev_snp_update_protected_guest_state,
 };

I don't see this function sev_snp_update_protected_guest_state() being defined anywhere in this series.

Then this line is removed in 'KVM: SVM: Support SEV-SNP AP Creation NAE event'. Should this line just be removed from this patch in the first place?

Quoted 31 lines
Quoted 31 lines
arch/x86/kvm/svm/svm.c +0 -0
 /*
arch/x86/kvm/svm/svm.h +6 -0
index 54ff56cb6125..3fd95193ed8d 100644
@@ -79,19 +79,25 @@ struct kvm_sev_info {
        bool active;            /* SEV enabled guest */
        bool es_active;         /* SEV-ES enabled guest */
        bool snp_active;        /* SEV-SNP enabled guest */
+
        unsigned int asid;      /* ASID used for this guest */
        unsigned int handle;    /* SEV firmware handle */
        int fd;                 /* SEV device fd */
+
        unsigned long pages_locked; /* Number of pages locked */
        struct list_head regions_list;  /* List of registered regions */
+
        u64 ap_jump_table;      /* SEV-ES AP Jump Table address */
+
        struct kvm *enc_context_owner; /* Owner of copied encryption context */
        struct list_head mirror_vms; /* List of VMs mirroring */
        struct list_head mirror_entry; /* Use as a list entry of mirrors */
        struct misc_cg *misc_cg; /* For misc cgroup accounting */
        atomic_t migration_in_progress;
+
        u64 snp_init_flags;
        void *snp_context;      /* SNP guest context page */
+       spinlock_t psc_lock;
 };
 
Quoted 10 lines

arch/x86/kvm/svm/svm.h void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa); · line 715

Quoted 10 lines
arch/x86/kvm/svm/svm.h +5 -0
 struct kvm_svm {
@@ -702,6 +708,11 @@ void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
 void sev_es_unmap_ghcb(struct vcpu_svm *svm);
 struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
 void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code);
+void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu);
+int sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu);

Ditto should this be removed?

Quoted 6 lines
Quoted 6 lines
arch/x86/kvm/svm/svm.h +0 -1
 /* vmenter.S */
 
--

2.25.1

author maintainer
#276 lore raw

Re: [PATCH Part2 v6 39/49] KVM: SVM: Introduce ops for the post gfn map and unmap

Quoted 176 lines · Peter Gonda
Quoted 165 lines · Alper Gun
Quoted 109 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest VM, the guest memory pages can either be a private or shared. A write from the hypervisor goes through the RMP checks. If hardware sees that hypervisor is attempting to write to a guest private page, then it triggers an RMP violation #PF.

To avoid the RMP violation with GHCB pages, added new post_{map,unmap}_gfn functions to verify if its safe to map GHCB pages. Uses a spinlock to protect against the page state change for existing mapped pages.

Need to add generic post_{map,unmap}_gfn() ops that can be used to verify that its safe to map a given guest page in the hypervisor.

This patch will need to be revisited later after consensus is reached on how to manage guest private memory as probably UPM private memslots will be able to handle this page state change more gracefully.

Signed-off by: Ashish Kalra <[email protected]>

---
  arch/x86/include/asm/kvm-x86-ops.h |  1 +
  arch/x86/include/asm/kvm_host.h    |  3 ++
  arch/x86/kvm/svm/sev.c             | 48 ++++++++++++++++++++++++++++--
  arch/x86/kvm/svm/svm.c             |  3 ++
  arch/x86/kvm/svm/svm.h             | 11 +++++++
  5 files changed, 64 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index e0068e702692..2dd2bc0cf4c3 100644
@@ -130,6 +130,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)
  KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
  KVM_X86_OP(alloc_apic_backing_page)
  KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
+KVM_X86_OP(update_protected_guest_state)
 
  #undef KVM_X86_OP
  #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +3 -0
index 49b217dc8d7e..8abc0e724f5c 100644
@@ -1522,7 +1522,10 @@ struct kvm_x86_ops {
         unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
         void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+
         void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+
+       int (*update_protected_guest_state)(struct kvm_vcpu *vcpu);
  };
 
  struct kvm_x86_nested_ops {
arch/x86/kvm/svm/sev.c +14 -2
index cb2d1bbb862b..4ed90331bca0 100644
@@ -341,6 +341,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
                 if (ret)
                         goto e_free;
 
+               spin_lock_init(&sev->psc_lock);
                 ret = sev_snp_init(&argp->error);
         } else {
                 ret = sev_platform_init(&argp->error);
@@ -2828,19 +2829,28 @@ static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
  {
         struct vmcb_control_area *control = &svm->vmcb->control;
         u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
 
-       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+       if (kvm_vcpu_map(vcpu, gfn, map)) {
                 /* Unable to map GHCB from guest */
                 pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
                 return -EFAULT;
         }
 
+       if (sev_post_map_gfn(vcpu->kvm, map->gfn, map->pfn)) {
+               kvm_vcpu_unmap(vcpu, map, false);
+               return -EBUSY;
+       }
+
         return 0;
  }
 
  static inline void svm_unmap_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
  {
-       kvm_vcpu_unmap(&svm->vcpu, map, true);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+
+       kvm_vcpu_unmap(vcpu, map, true);
+       sev_post_unmap_gfn(vcpu->kvm, map->gfn, map->pfn);
  }
 
  static void dump_ghcb(struct vcpu_svm *svm)
@@ -3383,6 +3393,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
                                 return PSC_UNDEF_ERR;
                 }
 
+               spin_lock(&sev->psc_lock);
+
                 write_lock(&kvm->mmu_lock);
 
                 rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);

arch/x86/kvm/svm/sev.c static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, · line 3432

Quoted 4 lines
arch/x86/kvm/svm/sev.c +1 -0
@@ -3417,6 +3429,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
 
                 write_unlock(&kvm->mmu_lock);
 
+               spin_unlock(&sev->psc_lock);

There is a corner case where the psc_lock is not released. If kvm_mmu_get_tdp_walk fails, the lock will be kept and will cause soft lockup.

Quoted 45 lines
arch/x86/kvm/svm/sev.c +31 -0
+
                 if (rc) {
                         pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
                                            op, gpa, pfn, level, rc);
@@ -3965,3 +3979,33 @@ void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level)
         /* Adjust the level to keep the NPT and RMP in sync */
         *level = min_t(size_t, *level, rmp_level);
  }
+
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+       int level;
+
+       if (!sev_snp_guest(kvm))
+               return 0;
+
+       spin_lock(&sev->psc_lock);
+
+       /* If pfn is not added as private then fail */
+       if (snp_lookup_rmpentry(pfn, &level) == 1) {
+               spin_unlock(&sev->psc_lock);
+               pr_err_ratelimited("failed to map private gfn 0x%llx pfn 0x%llx\n", gfn, pfn);
+               return -EBUSY;
+       }
+
+       return 0;
+}
+
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
+{
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+
+       if (!sev_snp_guest(kvm))
+               return;
+
+       spin_unlock(&sev->psc_lock);
+}
arch/x86/kvm/svm/svm.c +0 -0
index b24e0171cbf2..1c8e035ba011 100644
@@ -4734,7 +4734,10 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
         .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons,
 
Quoted 6 lines

arch/x86/kvm/svm/svm.c static struct kvm_x86_ops svm_x86_ops __initdata = { · line 4741

Quoted 6 lines
arch/x86/kvm/svm/svm.c +3 -0
         .alloc_apic_backing_page = svm_alloc_apic_backing_page,
+
         .rmp_page_level_adjust = sev_rmp_page_level_adjust,
+
+       .update_protected_guest_state = sev_snp_update_protected_guest_state,
  };

I don't see this function sev_snp_update_protected_guest_state() being defined anywhere in this series.

Quoted 3 lines

Then this line is removed in 'KVM: SVM: Support SEV-SNP AP Creation NAE event'. Should this line just be removed from this patch in the first place?

Yes, already fixed for v7.

Quoted 32 lines
Quoted 32 lines
Quoted 32 lines
arch/x86/kvm/svm/svm.c +0 -0
  /*
arch/x86/kvm/svm/svm.h +6 -0
index 54ff56cb6125..3fd95193ed8d 100644
@@ -79,19 +79,25 @@ struct kvm_sev_info {
         bool active;            /* SEV enabled guest */
         bool es_active;         /* SEV-ES enabled guest */
         bool snp_active;        /* SEV-SNP enabled guest */
+
         unsigned int asid;      /* ASID used for this guest */
         unsigned int handle;    /* SEV firmware handle */
         int fd;                 /* SEV device fd */
+
         unsigned long pages_locked; /* Number of pages locked */
         struct list_head regions_list;  /* List of registered regions */
+
         u64 ap_jump_table;      /* SEV-ES AP Jump Table address */
+
         struct kvm *enc_context_owner; /* Owner of copied encryption context */
         struct list_head mirror_vms; /* List of VMs mirroring */
         struct list_head mirror_entry; /* Use as a list entry of mirrors */
         struct misc_cg *misc_cg; /* For misc cgroup accounting */
         atomic_t migration_in_progress;
+
         u64 snp_init_flags;
         void *snp_context;      /* SNP guest context page */
+       spinlock_t psc_lock;
  };
 
Quoted 13 lines
Quoted 10 lines

arch/x86/kvm/svm/svm.h void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa); · line 715

Quoted 10 lines
arch/x86/kvm/svm/svm.h +5 -0
  struct kvm_svm {
@@ -702,6 +708,11 @@ void sev_es_prepare_switch_to_guest(struct sev_es_save_area *hostsa);
  void sev_es_unmap_ghcb(struct vcpu_svm *svm);
  struct page *snp_safe_alloc_page(struct kvm_vcpu *vcpu);
  void sev_rmp_page_level_adjust(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+int sev_post_map_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn);
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code);
+void sev_snp_init_protected_guest_state(struct kvm_vcpu *vcpu);
+int sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu);

Ditto should this be removed?

Yes, already fixed for v7.

Thanks,
Ashish

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
arch/x86/kvm/svm/svm.h +0 -1
  /* vmenter.S */
 
--

2.25.1

author maintainer
#277 lore raw

Re: [PATCH Part2 v6 39/49] KVM: SVM: Introduce ops for the post gfn map and unmap

In reply to Peter Gonda
Quoted 113 lines · Peter Gonda
Quoted 112 lines · Alper Gun
Quoted 110 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest VM, the guest memory pages can either be a private or shared. A write from the hypervisor goes through the RMP checks. If hardware sees that hypervisor is attempting to write to a guest private page, then it triggers an RMP violation #PF.

To avoid the RMP violation with GHCB pages, added new post_{map,unmap}_gfn functions to verify if its safe to map GHCB pages. Uses a spinlock to protect against the page state change for existing mapped pages.

Need to add generic post_{map,unmap}_gfn() ops that can be used to verify that its safe to map a given guest page in the hypervisor.

This patch will need to be revisited later after consensus is reached on how to manage guest private memory as probably UPM private memslots will be able to handle this page state change more gracefully.

Signed-off by: Ashish Kalra <[email protected]>

---
  arch/x86/include/asm/kvm-x86-ops.h |  1 +
  arch/x86/include/asm/kvm_host.h    |  3 ++
  arch/x86/kvm/svm/sev.c             | 48 ++++++++++++++++++++++++++++--
  arch/x86/kvm/svm/svm.c             |  3 ++
  arch/x86/kvm/svm/svm.h             | 11 +++++++
  5 files changed, 64 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -0
index e0068e702692..2dd2bc0cf4c3 100644
@@ -130,6 +130,7 @@ KVM_X86_OP(vcpu_deliver_sipi_vector)
  KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons);
  KVM_X86_OP(alloc_apic_backing_page)
  KVM_X86_OP_OPTIONAL(rmp_page_level_adjust)
+KVM_X86_OP(update_protected_guest_state)
 
  #undef KVM_X86_OP
  #undef KVM_X86_OP_OPTIONAL
arch/x86/include/asm/kvm_host.h +3 -0
index 49b217dc8d7e..8abc0e724f5c 100644
@@ -1522,7 +1522,10 @@ struct kvm_x86_ops {
         unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu);
 
         void *(*alloc_apic_backing_page)(struct kvm_vcpu *vcpu);
+
         void (*rmp_page_level_adjust)(struct kvm *kvm, kvm_pfn_t pfn, int *level);
+
+       int (*update_protected_guest_state)(struct kvm_vcpu *vcpu);
  };
 
  struct kvm_x86_nested_ops {
arch/x86/kvm/svm/sev.c +14 -2
index cb2d1bbb862b..4ed90331bca0 100644
@@ -341,6 +341,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
                 if (ret)
                         goto e_free;
 
+               spin_lock_init(&sev->psc_lock);
                 ret = sev_snp_init(&argp->error);
         } else {
                 ret = sev_platform_init(&argp->error);
@@ -2828,19 +2829,28 @@ static inline int svm_map_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
  {
         struct vmcb_control_area *control = &svm->vmcb->control;
         u64 gfn = gpa_to_gfn(control->ghcb_gpa);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
 
-       if (kvm_vcpu_map(&svm->vcpu, gfn, map)) {
+       if (kvm_vcpu_map(vcpu, gfn, map)) {
                 /* Unable to map GHCB from guest */
                 pr_err("error mapping GHCB GFN [%#llx] from guest\n", gfn);
                 return -EFAULT;
         }
 
+       if (sev_post_map_gfn(vcpu->kvm, map->gfn, map->pfn)) {
+               kvm_vcpu_unmap(vcpu, map, false);
+               return -EBUSY;
+       }
+
         return 0;
  }
 
  static inline void svm_unmap_ghcb(struct vcpu_svm *svm, struct kvm_host_map *map)
  {
-       kvm_vcpu_unmap(&svm->vcpu, map, true);
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+
+       kvm_vcpu_unmap(vcpu, map, true);
+       sev_post_unmap_gfn(vcpu->kvm, map->gfn, map->pfn);
  }
 
  static void dump_ghcb(struct vcpu_svm *svm)
@@ -3383,6 +3393,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
                                 return PSC_UNDEF_ERR;
                 }
 
+               spin_lock(&sev->psc_lock);
+
                 write_lock(&kvm->mmu_lock);
 
                 rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);
@@ -3417,6 +3429,8 @@ static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op,
 
Quoted 8 lines
Quoted 8 lines

arch/x86/kvm/svm/sev.c static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, · line 3432

Quoted 3 lines
arch/x86/kvm/svm/sev.c +1 -0
                 write_unlock(&kvm->mmu_lock);
 
+               spin_unlock(&sev->psc_lock);

There is a corner case where the psc_lock is not released. If kvm_mmu_get_tdp_walk fails, the lock will be kept and will cause soft lockup.

This is also already fixed for v7.

Thanks,
Ashish

author maintainer
#278 lore raw

PATCH v6 40/49 KVM: x86: Export the kvm_zap_gfn_range() for the SNP use

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

While resolving the RMP page fault, we may run into cases where the page level between the RMP entry and TDP does not match and the 2M RMP entry must be split into 4K RMP entries. Or a 2M TDP page need to be broken into multiple of 4K pages.

To keep the RMP and TDP page level in sync, we will zap the gfn range after splitting the pages in the RMP entry. The zap should force the TDP to gets rebuilt with the new page level.

---
 arch/x86/include/asm/kvm_host.h | 2 ++
 arch/x86/kvm/mmu.h              | 2 --
 arch/x86/kvm/mmu/mmu.c          | 1 +
 3 files changed, 3 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 8abc0e724f5c..1db4d178eb1d 100644
@@ -1627,6 +1627,8 @@ void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,
 void kvm_mmu_zap_all(struct kvm *kvm);
 void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen);
 void kvm_mmu_change_mmu_pages(struct kvm *kvm, unsigned long kvm_nr_mmu_pages);
+void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end);
+
 
 int load_pdptrs(struct kvm_vcpu *vcpu, unsigned long cr3);
 
arch/x86/kvm/mmu.h +0 -2
index d55b5166389a..c5044958a0fa 100644
@@ -267,8 +267,6 @@ static inline u8 permission_fault(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu,
 	return -(u32)fault & errcode;
 }
 
-void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end);
-
 int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu);
 
 int kvm_mmu_post_init_vm(struct kvm *kvm);
arch/x86/kvm/mmu/mmu.c +1 -0
index c1ac486e096e..67120bfeb667 100644
@@ -6084,6 +6084,7 @@ static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,
 
 	return need_tlb_flush;
 }
+EXPORT_SYMBOL_GPL(kvm_zap_gfn_range);
 
 void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm,
 				   const struct kvm_memory_slot *slot)
Signature · 1 line
2.25.1
author maintainer
#279 lore raw

PATCH v6 41/49 KVM: SVM: Add support to handle the RMP nested page fault

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
 arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c | 14 +++++---
 2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +76 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
 	spin_unlock(&sev->psc_lock);
 }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+	int rmp_level, npt_level, rc, assigned;
+	struct kvm *kvm = vcpu->kvm;
+	gfn_t gfn = gpa_to_gfn(gpa);
+	bool need_psc = false;
+	enum psc_op psc_op;
+	kvm_pfn_t pfn;
+	bool private;
+
+	write_lock(&kvm->mmu_lock);
+
+	if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))
+		goto unlock;
+
+	assigned = snp_lookup_rmpentry(pfn, &rmp_level);
+	if (unlikely(assigned < 0))
+		goto unlock;
+
+	private = !!(error_code & PFERR_GUEST_ENC_MASK);
+
+	/*
+	 * If the fault was due to size mismatch, or NPT and RMP page level's
+	 * are not in sync, then use PSMASH to split the RMP entry into 4K.
+	 */
+	if ((error_code & PFERR_GUEST_SIZEM_MASK) ||
+	    (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) {
+		rc = snp_rmptable_psmash(kvm, pfn);
+		if (rc)
+			pr_err_ratelimited("psmash failed, gpa 0x%llx pfn 0x%llx rc %d\n",
+					   gpa, pfn, rc);
+		goto out;
+	}
+
+	/*
+	 * If it's a private access, and the page is not assigned in the
+	 * RMP table, create a new private RMP entry. This can happen if
+	 * guest did not use the PSC VMGEXIT to transition the page state
+	 * before the access.
+	 */
+	if (!assigned && private) {
+		need_psc = 1;
+		psc_op = SNP_PAGE_STATE_PRIVATE;
+		goto out;
+	}
+
+	/*
+	 * If it's a shared access, but the page is private in the RMP table
+	 * then make the page shared in the RMP table. This can happen if
+	 * the guest did not use the PSC VMGEXIT to transition the page
+	 * state before the access.
+	 */
+	if (assigned && !private) {
+		need_psc = 1;
+		psc_op = SNP_PAGE_STATE_SHARED;
+	}
+
+out:
+	write_unlock(&kvm->mmu_lock);
+
+	if (need_psc)
+		rc = __snp_handle_page_state_change(vcpu, psc_op, gpa, PG_LEVEL_4K);
+
+	/*
+	 * The fault handler has updated the RMP pagesize, zap the existing
+	 * rmaps for large entry ranges so that nested page table gets rebuilt
+	 * with the updated RMP pagesize.
+	 */
+	gfn = gpa_to_gfn(gpa) & ~(KVM_PAGES_PER_HPAGE(PG_LEVEL_2M) - 1);
+	kvm_zap_gfn_range(kvm, gfn, gfn + PTRS_PER_PMD);
+	return;
+
+unlock:
+	write_unlock(&kvm->mmu_lock);
+}
arch/x86/kvm/svm/svm.c +10 -4
index 1c8e035ba011..7742bc986afc 100644
@@ -1866,15 +1866,21 @@ static int pf_interception(struct kvm_vcpu *vcpu)
 static int npf_interception(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_svm *svm = to_svm(vcpu);
+	int rc;
 
 	u64 fault_address = svm->vmcb->control.exit_info_2;
 	u64 error_code = svm->vmcb->control.exit_info_1;
 
 	trace_kvm_page_fault(fault_address, error_code);
-	return kvm_mmu_page_fault(vcpu, fault_address, error_code,
-			static_cpu_has(X86_FEATURE_DECODEASSISTS) ?
-			svm->vmcb->control.insn_bytes : NULL,
-			svm->vmcb->control.insn_len);
+	rc = kvm_mmu_page_fault(vcpu, fault_address, error_code,
+				static_cpu_has(X86_FEATURE_DECODEASSISTS) ?
+				svm->vmcb->control.insn_bytes : NULL,
+				svm->vmcb->control.insn_len);
+
+	if (error_code & PFERR_GUEST_RMP_MASK)
+		handle_rmp_page_fault(vcpu, fault_address, error_code);
+
+	return rc;
 }
 
 static int db_interception(struct kvm_vcpu *vcpu)
Signature · 1 line
2.25.1
#280 lore raw

Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

Quoted 26 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
 arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c | 14 +++++---
 2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +4 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
 	spin_unlock(&sev->psc_lock);
 }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+	int rmp_level, npt_level, rc, assigned;

arch/x86/kvm/svm/sev.c void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) · line 4025

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+	struct kvm *kvm = vcpu->kvm;
+	gfn_t gfn = gpa_to_gfn(gpa);
+	bool need_psc = false;
+	enum psc_op psc_op;
+	kvm_pfn_t pfn;
+	bool private;
+
+	write_lock(&kvm->mmu_lock);
+
+	if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))

This function does not exist. Should it be kvm_mmu_get_tdp_page?

BR, Jarkko

#281 lore raw

Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

Quoted 31 lines · Jarkko Sakkinen
Quoted 30 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
 arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c | 14 +++++---
 2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +8 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
 	spin_unlock(&sev->psc_lock);
 }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+	int rmp_level, npt_level, rc, assigned;
+	struct kvm *kvm = vcpu->kvm;
+	gfn_t gfn = gpa_to_gfn(gpa);
+	bool need_psc = false;
+	enum psc_op psc_op;
Quoted 8 lines

arch/x86/kvm/svm/sev.c void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) · line 4025

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+	kvm_pfn_t pfn;
+	bool private;
+
+	write_lock(&kvm->mmu_lock);
+
+	if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))

This function does not exist. Should it be kvm_mmu_get_tdp_page?

Ugh, ignore that.

This the actual issue:

$ git grep  kvm_mmu_get_tdp_walk
arch/x86/kvm/mmu/mmu.c:bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level)
arch/x86/kvm/mmu/mmu.c:EXPORT_SYMBOL_GPL(kvm_mmu_get_tdp_walk);
arch/x86/kvm/svm/sev.c:         rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);

It's not declared in any header.

BR, Jarkko

#282 lore raw

Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

Quoted 43 lines · Jarkko Sakkinen
Quoted 31 lines · Jarkko Sakkinen
Quoted 30 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
 arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c | 14 +++++---
 2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +8 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
 	spin_unlock(&sev->psc_lock);
 }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+	int rmp_level, npt_level, rc, assigned;
+	struct kvm *kvm = vcpu->kvm;
+	gfn_t gfn = gpa_to_gfn(gpa);
+	bool need_psc = false;
+	enum psc_op psc_op;
Quoted 8 lines

arch/x86/kvm/svm/sev.c void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) · line 4025

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+	kvm_pfn_t pfn;
+	bool private;
+
+	write_lock(&kvm->mmu_lock);
+
+	if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))

This function does not exist. Should it be kvm_mmu_get_tdp_page?

Ugh, ignore that.

Quoted 8 lines

This the actual issue:

$ git grep  kvm_mmu_get_tdp_walk
arch/x86/kvm/mmu/mmu.c:bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level)
arch/x86/kvm/mmu/mmu.c:EXPORT_SYMBOL_GPL(kvm_mmu_get_tdp_walk);
arch/x86/kvm/svm/sev.c:         rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);

It's not declared in any header.

arch/x86/kvm/mmu.h +2 -0
index 0e1f4d92b89b..33267f619e61 100644
@@ -164,6 +164,8 @@ static inline void kvm_mmu_load_pgd(struct kvm_vcpu *vcpu)
 kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
                               u32 error_code, int max_level);
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level):
+
 /*
  * Check if a given access (described through the I/D, W/R and U/S bits of a
  * page fault error code pfec) causes a permission fault with the given PTE

BTW, kvm_mmu_map_tdp_page() ought to be in single line since it's less than 100 characters.

BR, Jarkko

author maintainer
#283 lore raw

RE: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

[AMD Official Use Only - General]

Yes, this is fixed in 5.19 rebase.

Thanks,
Ashish

-----Original Message-----
From: Jarkko Sakkinen <[email protected]>
Sent: Tuesday, July 12, 2022 7:49 AM
To: Kalra, Ashish <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Roth, Michael <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

Quoted 46 lines · Jarkko Sakkinen
Quoted 34 lines · Jarkko Sakkinen
Quoted 33 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places
restrictions on all memory accesses based on the contents of the
RMP table. When hardware encounters RMP check failure caused by
the guest memory access it raises the #NPF. The error code
contains additional information on the access type. See the APM volume 2 for additional information.

---
 arch/x86/kvm/svm/sev.c | 76 
++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c | 14 +++++---
 2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c index renamed from arch/x86/kvm/svm/sev.c +0 -0

4ed90331bca0..7fc0fad87054 100644

arch/x86/kvm/svm/sev.c +9 -0
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, 
 gfn_t gfn, kvm_pfn_t pfn)
 
 	spin_unlock(&sev->psc_lock);
 }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 
+error_code) {
+	int rmp_level, npt_level, rc, assigned;
+	struct kvm *kvm = vcpu->kvm;
+	gfn_t gfn = gpa_to_gfn(gpa);
+	bool need_psc = false;
+	enum psc_op psc_op;
+	kvm_pfn_t pfn;
Quoted 8 lines

arch/x86/kvm/svm/sev.c

Quoted 6 lines
arch/x86/kvm/svm/sev.c +6 -0
+	bool private;
+
+	write_lock(&kvm->mmu_lock);
+
+	if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, 
+&npt_level)))

This function does not exist. Should it be kvm_mmu_get_tdp_page?

Ugh, ignore that.

Quoted 8 lines

This the actual issue:

$ git grep  kvm_mmu_get_tdp_walk
arch/x86/kvm/mmu/mmu.c:bool kvm_mmu_get_tdp_walk(struct kvm_vcpu 
*vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level) arch/x86/kvm/mmu/mmu.c:EXPORT_SYMBOL_GPL(kvm_mmu_get_tdp_walk);
arch/x86/kvm/svm/sev.c:         rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);

It's not declared in any header.

arch/x86/kvm/mmu.h +2 -0
@@ -164,6 +164,8 @@ static inline void kvm_mmu_load_pgd(struct kvm_vcpu *vcpu)  kvm_pfn_t kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa,
                               u32 error_code, int max_level);
 
+bool kvm_mmu_get_tdp_walk(struct kvm_vcpu *vcpu, gpa_t gpa, kvm_pfn_t *pfn, int *level):
+
 /*
  * Check if a given access (described through the I/D, W/R and U/S bits of a
  * page fault error code pfec) causes a permission fault with the given PTE
 

BTW, kvm_mmu_map_tdp_page() ought to be in single line since it's less than 100 characters.

BR, Jarkko

#284 lore raw

Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

In reply to Ashish Kalra
Quoted 42 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
 arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c | 14 +++++---
 2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +19 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
        spin_unlock(&sev->psc_lock);
 }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+       int rmp_level, npt_level, rc, assigned;
+       struct kvm *kvm = vcpu->kvm;
+       gfn_t gfn = gpa_to_gfn(gpa);
+       bool need_psc = false;
+       enum psc_op psc_op;
+       kvm_pfn_t pfn;
+       bool private;
+
+       write_lock(&kvm->mmu_lock);
+
+       if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))
+               goto unlock;
+
+       assigned = snp_lookup_rmpentry(pfn, &rmp_level);
+       if (unlikely(assigned < 0))
+               goto unlock;

arch/x86/kvm/svm/sev.c void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) · line 4040

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       private = !!(error_code & PFERR_GUEST_ENC_MASK);
+
+       /*
+        * If the fault was due to size mismatch, or NPT and RMP page level's
+        * are not in sync, then use PSMASH to split the RMP entry into 4K.
+        */
+       if ((error_code & PFERR_GUEST_SIZEM_MASK) ||
+           (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) {
+               rc = snp_rmptable_psmash(kvm, pfn);

Regarding this case:
RMP level is 4K
Page table level is 2M

Does this also cause a page fault with size mismatch? If so, we shouldn't try psmash because the rmp entry is already 4K.

I see these errors in our tests and I think it may be happening because rmp size is already 4K.

[ 1848.752952] psmash failed, gpa 0x191560000 pfn 0x536cd60 rc 7
[ 2922.879635] psmash failed, gpa 0x102830000 pfn 0x37c8230 rc 7
[ 3010.983090] psmash failed, gpa 0x104220000 pfn 0x6cf1e20 rc 7
[ 3170.792050] psmash failed, gpa 0x108a80000 pfn 0x20e0080 rc 7
[ 3345.955147] psmash failed, gpa 0x11b480000 pfn 0x1545e480 rc 7

Shouldn't we use AND instead of OR in the if statement?

if ((error_code & PFERR_GUEST_SIZEM_MASK) && ...

Quoted 81 lines
arch/x86/kvm/svm/sev.c +47 -0
+               if (rc)
+                       pr_err_ratelimited("psmash failed, gpa 0x%llx pfn 0x%llx rc %d\n",
+                                          gpa, pfn, rc);
+               goto out;
+       }
+
+       /*
+        * If it's a private access, and the page is not assigned in the
+        * RMP table, create a new private RMP entry. This can happen if
+        * guest did not use the PSC VMGEXIT to transition the page state
+        * before the access.
+        */
+       if (!assigned && private) {
+               need_psc = 1;
+               psc_op = SNP_PAGE_STATE_PRIVATE;
+               goto out;
+       }
+
+       /*
+        * If it's a shared access, but the page is private in the RMP table
+        * then make the page shared in the RMP table. This can happen if
+        * the guest did not use the PSC VMGEXIT to transition the page
+        * state before the access.
+        */
+       if (assigned && !private) {
+               need_psc = 1;
+               psc_op = SNP_PAGE_STATE_SHARED;
+       }
+
+out:
+       write_unlock(&kvm->mmu_lock);
+
+       if (need_psc)
+               rc = __snp_handle_page_state_change(vcpu, psc_op, gpa, PG_LEVEL_4K);
+
+       /*
+        * The fault handler has updated the RMP pagesize, zap the existing
+        * rmaps for large entry ranges so that nested page table gets rebuilt
+        * with the updated RMP pagesize.
+        */
+       gfn = gpa_to_gfn(gpa) & ~(KVM_PAGES_PER_HPAGE(PG_LEVEL_2M) - 1);
+       kvm_zap_gfn_range(kvm, gfn, gfn + PTRS_PER_PMD);
+       return;
+
+unlock:
+       write_unlock(&kvm->mmu_lock);
+}
arch/x86/kvm/svm/svm.c +10 -5
index 1c8e035ba011..7742bc986afc 100644
@@ -1866,15 +1866,21 @@ static int pf_interception(struct kvm_vcpu *vcpu)
 static int npf_interception(struct kvm_vcpu *vcpu)
 {
        struct vcpu_svm *svm = to_svm(vcpu);
+       int rc;
 
        u64 fault_address = svm->vmcb->control.exit_info_2;
        u64 error_code = svm->vmcb->control.exit_info_1;
 
        trace_kvm_page_fault(fault_address, error_code);
-       return kvm_mmu_page_fault(vcpu, fault_address, error_code,
-                       static_cpu_has(X86_FEATURE_DECODEASSISTS) ?
-                       svm->vmcb->control.insn_bytes : NULL,
-                       svm->vmcb->control.insn_len);
+       rc = kvm_mmu_page_fault(vcpu, fault_address, error_code,
+                               static_cpu_has(X86_FEATURE_DECODEASSISTS) ?
+                               svm->vmcb->control.insn_bytes : NULL,
+                               svm->vmcb->control.insn_len);
+
+       if (error_code & PFERR_GUEST_RMP_MASK)
+               handle_rmp_page_fault(vcpu, fault_address, error_code);
+
+       return rc;
 }
 
 static int db_interception(struct kvm_vcpu *vcpu)
--

2.25.1

author maintainer
#285 lore raw

Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

Hello Alper,

Quoted 65 lines · Alper Gun
Quoted 42 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
  arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
  arch/x86/kvm/svm/svm.c | 14 +++++---
  2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +19 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
         spin_unlock(&sev->psc_lock);
  }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+       int rmp_level, npt_level, rc, assigned;
+       struct kvm *kvm = vcpu->kvm;
+       gfn_t gfn = gpa_to_gfn(gpa);
+       bool need_psc = false;
+       enum psc_op psc_op;
+       kvm_pfn_t pfn;
+       bool private;
+
+       write_lock(&kvm->mmu_lock);
+
+       if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))
+               goto unlock;
+
+       assigned = snp_lookup_rmpentry(pfn, &rmp_level);
+       if (unlikely(assigned < 0))
+               goto unlock;

arch/x86/kvm/svm/sev.c void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) · line 4040

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       private = !!(error_code & PFERR_GUEST_ENC_MASK);
+
+       /*
+        * If the fault was due to size mismatch, or NPT and RMP page level's
+        * are not in sync, then use PSMASH to split the RMP entry into 4K.
+        */
+       if ((error_code & PFERR_GUEST_SIZEM_MASK) ||
+           (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) {
+               rc = snp_rmptable_psmash(kvm, pfn);

Regarding this case:
RMP level is 4K
Page table level is 2M

Does this also cause a page fault with size mismatch? If so, we shouldn't try psmash because the rmp entry is already 4K.

I see these errors in our tests and I think it may be happening because rmp size is already 4K.

Quoted 8 lines
[ 1848.752952] psmash failed, gpa 0x191560000 pfn 0x536cd60 rc 7
[ 2922.879635] psmash failed, gpa 0x102830000 pfn 0x37c8230 rc 7
[ 3010.983090] psmash failed, gpa 0x104220000 pfn 0x6cf1e20 rc 7
[ 3170.792050] psmash failed, gpa 0x108a80000 pfn 0x20e0080 rc 7
[ 3345.955147] psmash failed, gpa 0x11b480000 pfn 0x1545e480 rc 7

Shouldn't we use AND instead of OR in the if statement?

I believe this we can't do, looking at the typical usage case below :

[   37.243969] #VMEXIT (NPF) - SIZEM, err 0xc80000005 npt_level 2, rmp_level 2, private 1
[   37.243973] trying psmash gpa 0x7f790000 pfn 0x1f5d90

This is typically the case with #VMEXIT(NPF) with SIZEM error code, when the guest tries to do PVALIDATE on 4K GHCB pages, in this case both the RMP table and NPT will be optimally setup to 2M hugepage as can be seen.

Is it possible to investigate in more depth, when is the this case being observed:
RMP level is 4K
Page table level is 2M
We shouldn't try psmash because the rmp entry is already 4K.

Thanks,
Ashish

Quoted 2 lines

if ((error_code & PFERR_GUEST_SIZEM_MASK) && ...

#286 lore raw

Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

Quoted 84 lines · Ashish

Hello Alper,

Quoted 65 lines · Alper Gun
Quoted 42 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
  arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
  arch/x86/kvm/svm/svm.c | 14 +++++---
  2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +19 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
         spin_unlock(&sev->psc_lock);
  }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+       int rmp_level, npt_level, rc, assigned;
+       struct kvm *kvm = vcpu->kvm;
+       gfn_t gfn = gpa_to_gfn(gpa);
+       bool need_psc = false;
+       enum psc_op psc_op;
+       kvm_pfn_t pfn;
+       bool private;
+
+       write_lock(&kvm->mmu_lock);
+
+       if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))
+               goto unlock;
+
+       assigned = snp_lookup_rmpentry(pfn, &rmp_level);
+       if (unlikely(assigned < 0))
+               goto unlock;

arch/x86/kvm/svm/sev.c void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) · line 4040

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       private = !!(error_code & PFERR_GUEST_ENC_MASK);
+
+       /*
+        * If the fault was due to size mismatch, or NPT and RMP page level's
+        * are not in sync, then use PSMASH to split the RMP entry into 4K.
+        */
+       if ((error_code & PFERR_GUEST_SIZEM_MASK) ||
+           (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) {
+               rc = snp_rmptable_psmash(kvm, pfn);

Regarding this case:
RMP level is 4K
Page table level is 2M

Does this also cause a page fault with size mismatch? If so, we shouldn't try psmash because the rmp entry is already 4K.

I see these errors in our tests and I think it may be happening because rmp size is already 4K.

Quoted 8 lines
[ 1848.752952] psmash failed, gpa 0x191560000 pfn 0x536cd60 rc 7
[ 2922.879635] psmash failed, gpa 0x102830000 pfn 0x37c8230 rc 7
[ 3010.983090] psmash failed, gpa 0x104220000 pfn 0x6cf1e20 rc 7
[ 3170.792050] psmash failed, gpa 0x108a80000 pfn 0x20e0080 rc 7
[ 3345.955147] psmash failed, gpa 0x11b480000 pfn 0x1545e480 rc 7

Shouldn't we use AND instead of OR in the if statement?

I believe this we can't do, looking at the typical usage case below :

[   37.243969] #VMEXIT (NPF) - SIZEM, err 0xc80000005 npt_level 2,
rmp_level 2, private 1
[   37.243973] trying psmash gpa 0x7f790000 pfn 0x1f5d90
Quoted 6 lines

This is typically the case with #VMEXIT(NPF) with SIZEM error code, when the guest tries to do PVALIDATE on 4K GHCB pages, in this case both the RMP table and NPT will be optimally setup to 2M hugepage as can be seen.

Is it possible to investigate in more depth, when is the this case being observed:

Yes, I added more logs and I can see that these errors happen when RMP level is 4K and NPT level is 2M.
psmash fails as expected. I think it is just a log, there is no real issue but the best is not trying psmash if rmp level is 4K.

Quoted 9 lines

RMP level is 4K
Page table level is 2M
We shouldn't try psmash because the rmp entry is already 4K.

Thanks,
Ashish

Quoted 2 lines

if ((error_code & PFERR_GUEST_SIZEM_MASK) && ...

author maintainer
#287 lore raw

Re: [PATCH Part2 v6 41/49] KVM: SVM: Add support to handle the RMP nested page fault

Quoted 89 lines · Alper Gun
Quoted 88 lines · Ashish

Hello Alper,

Quoted 65 lines · Alper Gun
Quoted 42 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

When SEV-SNP is enabled in the guest, the hardware places restrictions on all memory accesses based on the contents of the RMP table. When hardware encounters RMP check failure caused by the guest memory access it raises the #NPF. The error code contains additional information on the access type. See the APM volume 2 for additional information.

---
   arch/x86/kvm/svm/sev.c | 76 ++++++++++++++++++++++++++++++++++++++++++
   arch/x86/kvm/svm/svm.c | 14 +++++---
   2 files changed, 86 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/sev.c +19 -0
index 4ed90331bca0..7fc0fad87054 100644
@@ -4009,3 +4009,79 @@ void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn)
 
          spin_unlock(&sev->psc_lock);
   }
+
+void handle_rmp_page_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code)
+{
+       int rmp_level, npt_level, rc, assigned;
+       struct kvm *kvm = vcpu->kvm;
+       gfn_t gfn = gpa_to_gfn(gpa);
+       bool need_psc = false;
+       enum psc_op psc_op;
+       kvm_pfn_t pfn;
+       bool private;
+
+       write_lock(&kvm->mmu_lock);
+
+       if (unlikely(!kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level)))
+               goto unlock;
+
+       assigned = snp_lookup_rmpentry(pfn, &rmp_level);
+       if (unlikely(assigned < 0))
+               goto unlock;

arch/x86/kvm/svm/sev.c void sev_post_unmap_gfn(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn) · line 4040

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+       private = !!(error_code & PFERR_GUEST_ENC_MASK);
+
+       /*
+        * If the fault was due to size mismatch, or NPT and RMP page level's
+        * are not in sync, then use PSMASH to split the RMP entry into 4K.
+        */
+       if ((error_code & PFERR_GUEST_SIZEM_MASK) ||
+           (npt_level == PG_LEVEL_4K && rmp_level == PG_LEVEL_2M && private)) {
+               rc = snp_rmptable_psmash(kvm, pfn);

Regarding this case:
RMP level is 4K
Page table level is 2M

Does this also cause a page fault with size mismatch? If so, we shouldn't try psmash because the rmp entry is already 4K.

I see these errors in our tests and I think it may be happening because rmp size is already 4K.

Quoted 8 lines
[ 1848.752952] psmash failed, gpa 0x191560000 pfn 0x536cd60 rc 7
[ 2922.879635] psmash failed, gpa 0x102830000 pfn 0x37c8230 rc 7
[ 3010.983090] psmash failed, gpa 0x104220000 pfn 0x6cf1e20 rc 7
[ 3170.792050] psmash failed, gpa 0x108a80000 pfn 0x20e0080 rc 7
[ 3345.955147] psmash failed, gpa 0x11b480000 pfn 0x1545e480 rc 7

Shouldn't we use AND instead of OR in the if statement?

I believe this we can't do, looking at the typical usage case below :

[   37.243969] #VMEXIT (NPF) - SIZEM, err 0xc80000005 npt_level 2,
rmp_level 2, private 1
[   37.243973] trying psmash gpa 0x7f790000 pfn 0x1f5d90

This is typically the case with #VMEXIT(NPF) with SIZEM error code, when the guest tries to do PVALIDATE on 4K GHCB pages, in this case both the RMP table and NPT will be optimally setup to 2M hugepage as can be seen.

Quoted 8 lines
Quoted 2 lines

Is it possible to investigate in more depth, when is the this case being observed:

Yes, I added more logs and I can see that these errors happen when RMP level is 4K and NPT level is 2M.
psmash fails as expected. I think it is just a log, there is no real issue but the best is not trying psmash if rmp level is 4K.

Now, the SIZEM bit is only set when PVALIDATE or RMPADJUST fails due to
guest attempting to validate a 4K page that is backed by a 2MB RMP entry, which is not the case here as RMP level is 4K.

Also, this does not fall into the second case for the same reason.

#NPF will happen during Guest page table walk if RMP checks fail for 2M nested page and RMP.SubPage_Count !=0 OR RMP.PageSize != Nested table page size, but then that shouldn't have the SIZEM fault bit set.

This raises concern about some existing race condition, it probably
can race with
snp_handle_page_state_change()->snp_make_page_shared()->snp_rmptable_psmash(),
but that code path seems to be protected from this nested RMP #PF handler as they both acquire the kvm->mmu_lock.

So, this still needs more investigation.

Can you share what kind of tests are you running to reproduce this issue ?

Thanks,
Ashish

Quoted 9 lines
Quoted 9 lines

RMP level is 4K
Page table level is 2M
We shouldn't try psmash because the rmp entry is already 4K.

Thanks,
Ashish

Quoted 2 lines

if ((error_code & PFERR_GUEST_SIZEM_MASK) && ...

author maintainer
#288 lore raw

PATCH v6 42/49 KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

In reply to Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of GHCB specification added the support for two SNP Guest Request Message NAE events. The events allows for an SEV-SNP guest to make request to the SEV-SNP firmware through hypervisor using the SNP_GUEST_REQUEST API define in the SEV-SNP firmware specification.

The SNP_EXT_GUEST_REQUEST is similar to SNP_GUEST_REQUEST with the difference of an additional certificate blob that can be passed through the SNP_SET_CONFIG ioctl defined in the CCP driver. The CCP driver provides snp_guest_ext_guest_request() that is used by the KVM to get both the report and certificate data at once.

---
 arch/x86/kvm/svm/sev.c | 196 +++++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.h |   2 +
 2 files changed, 192 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/sev.c +190 -6
index 7fc0fad87054..089af21a4efe 100644
@@ -343,6 +343,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
 		spin_lock_init(&sev->psc_lock);
 		ret = sev_snp_init(&argp->error);
+		mutex_init(&sev->guest_req_lock);
 	} else {
 		ret = sev_platform_init(&argp->error);
 	}
@@ -1884,23 +1885,39 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 
 static void *snp_context_create(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
+	void *context = NULL, *certs_data = NULL, *resp_page = NULL;
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
 	struct sev_data_snp_gctx_create data = {};
-	void *context;
 	int rc;
 
+	/* Allocate memory used for the certs data in SNP guest request */
+	certs_data = kmalloc(SEV_FW_BLOB_MAX_SIZE, GFP_KERNEL_ACCOUNT);
+	if (!certs_data)
+		return NULL;
+
 	/* Allocate memory for context page */
 	context = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
 	if (!context)
-		return NULL;
+		goto e_free;
+
+	/* Allocate a firmware buffer used during the guest command handling. */
+	resp_page = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
+	if (!resp_page)
+		goto e_free;
 
 	data.gctx_paddr = __psp_pa(context);
 	rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_GCTX_CREATE, &data, &argp->error);
-	if (rc) {
-		snp_free_firmware_page(context);
-		return NULL;
-	}
+	if (rc)
+		goto e_free;
+
+	sev->snp_certs_data = certs_data;
 
 	return context;
+
+e_free:
+	snp_free_firmware_page(context);
+	kfree(certs_data);
+	return NULL;
 }
 
 static int snp_bind_asid(struct kvm *kvm, int *error)
@@ -2565,6 +2582,8 @@ static int snp_decommission_context(struct kvm *kvm)
 	snp_free_firmware_page(sev->snp_context);
 	sev->snp_context = NULL;
 
+	kfree(sev->snp_certs_data);
+
 	return 0;
 }
 
@@ -3077,6 +3096,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm, u64 *exit_code)
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
 	case SVM_VMGEXIT_HV_FEATURES:
 	case SVM_VMGEXIT_PSC:
+	case SVM_VMGEXIT_GUEST_REQUEST:
+	case SVM_VMGEXIT_EXT_GUEST_REQUEST:
 		break;
 	default:
 		reason = GHCB_ERR_INVALID_EVENT;
@@ -3502,6 +3523,155 @@ static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm)
 	return rc ? map_to_psc_vmgexit_code(rc) : 0;
 }
 
+static unsigned long snp_setup_guest_buf(struct vcpu_svm *svm,
+					 struct sev_data_snp_guest_request *data,
+					 gpa_t req_gpa, gpa_t resp_gpa)
+{
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	kvm_pfn_t req_pfn, resp_pfn;
+	struct kvm_sev_info *sev;
+
+	sev = &to_kvm_svm(kvm)->sev_info;
+
+	if (!IS_ALIGNED(req_gpa, PAGE_SIZE) || !IS_ALIGNED(resp_gpa, PAGE_SIZE))
+		return SEV_RET_INVALID_PARAM;
+
+	req_pfn = gfn_to_pfn(kvm, gpa_to_gfn(req_gpa));
+	if (is_error_noslot_pfn(req_pfn))
+		return SEV_RET_INVALID_ADDRESS;
+
+	resp_pfn = gfn_to_pfn(kvm, gpa_to_gfn(resp_gpa));
+	if (is_error_noslot_pfn(resp_pfn))
+		return SEV_RET_INVALID_ADDRESS;
+
+	if (rmp_make_private(resp_pfn, 0, PG_LEVEL_4K, 0, true))
+		return SEV_RET_INVALID_ADDRESS;
+
+	data->gctx_paddr = __psp_pa(sev->snp_context);
+	data->req_paddr = __sme_set(req_pfn << PAGE_SHIFT);
+	data->res_paddr = __sme_set(resp_pfn << PAGE_SHIFT);
+
+	return 0;
+}
+
+static void snp_cleanup_guest_buf(struct sev_data_snp_guest_request *data, unsigned long *rc)
+{
+	u64 pfn = __sme_clr(data->res_paddr) >> PAGE_SHIFT;
+	int ret;
+
+	ret = snp_page_reclaim(pfn);
+	if (ret)
+		*rc = SEV_RET_INVALID_ADDRESS;
+
+	ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+	if (ret)
+		*rc = SEV_RET_INVALID_ADDRESS;
+}
+
+static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+	struct sev_data_snp_guest_request data = {0};
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_sev_info *sev;
+	unsigned long rc;
+	int err;
+
+	if (!sev_snp_guest(vcpu->kvm)) {
+		rc = SEV_RET_INVALID_GUEST;
+		goto e_fail;
+	}
+
+	sev = &to_kvm_svm(kvm)->sev_info;
+
+	mutex_lock(&sev->guest_req_lock);
+
+	rc = snp_setup_guest_buf(svm, &data, req_gpa, resp_gpa);
+	if (rc)
+		goto unlock;
+
+	rc = sev_issue_cmd(kvm, SEV_CMD_SNP_GUEST_REQUEST, &data, &err);
+	if (rc)
+		/* use the firmware error code */
+		rc = err;
+
+	snp_cleanup_guest_buf(&data, &rc);
+
+unlock:
+	mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+	svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
+static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+	struct sev_data_snp_guest_request req = {0};
+	struct kvm_vcpu *vcpu = &svm->vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	unsigned long data_npages;
+	struct kvm_sev_info *sev;
+	unsigned long rc, err;
+	u64 data_gpa;
+
+	if (!sev_snp_guest(vcpu->kvm)) {
+		rc = SEV_RET_INVALID_GUEST;
+		goto e_fail;
+	}
+
+	sev = &to_kvm_svm(kvm)->sev_info;
+
+	data_gpa = vcpu->arch.regs[VCPU_REGS_RAX];
+	data_npages = vcpu->arch.regs[VCPU_REGS_RBX];
+
+	if (!IS_ALIGNED(data_gpa, PAGE_SIZE)) {
+		rc = SEV_RET_INVALID_ADDRESS;
+		goto e_fail;
+	}
+
+	/* Verify that requested blob will fit in certificate buffer */
+	if ((data_npages << PAGE_SHIFT) > SEV_FW_BLOB_MAX_SIZE) {
+		rc = SEV_RET_INVALID_PARAM;
+		goto e_fail;
+	}
+
+	mutex_lock(&sev->guest_req_lock);
+
+	rc = snp_setup_guest_buf(svm, &req, req_gpa, resp_gpa);
+	if (rc)
+		goto unlock;
+
+	rc = snp_guest_ext_guest_request(&req, (unsigned long)sev->snp_certs_data,
+					 &data_npages, &err);
+	if (rc) {
+		/*
+		 * If buffer length is small then return the expected
+		 * length in rbx.
+		 */
+		if (err == SNP_GUEST_REQ_INVALID_LEN)
+			vcpu->arch.regs[VCPU_REGS_RBX] = data_npages;
+
+		/* pass the firmware error code */
+		rc = err;
+		goto cleanup;
+	}
+
+	/* Copy the certificate blob in the guest memory */
+	if (data_npages &&
+	    kvm_write_guest(kvm, data_gpa, sev->snp_certs_data, data_npages << PAGE_SHIFT))
+		rc = SEV_RET_INVALID_ADDRESS;
+
+cleanup:
+	snp_cleanup_guest_buf(&req, &rc);
+
+unlock:
+	mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+	svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
 static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 {
 	struct vmcb_control_area *control = &svm->vmcb->control;
@@ -3753,6 +3923,20 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
 		svm_set_ghcb_sw_exit_info_2(vcpu, rc);
 		break;
 	}
+	case SVM_VMGEXIT_GUEST_REQUEST: {
+		snp_handle_guest_request(svm, control->exit_info_1, control->exit_info_2);
+
+		ret = 1;
+		break;
+	}
+	case SVM_VMGEXIT_EXT_GUEST_REQUEST: {
+		snp_handle_ext_guest_request(svm,
+					     control->exit_info_1,
+					     control->exit_info_2);
+
+		ret = 1;
+		break;
+	}
 	case SVM_VMGEXIT_UNSUPPORTED_EVENT:
 		vcpu_unimpl(vcpu,
 			    "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n",
arch/x86/kvm/svm/svm.h +2 -0
index 3fd95193ed8d..3be24da1a743 100644
@@ -98,6 +98,8 @@ struct kvm_sev_info {
 	u64 snp_init_flags;
 	void *snp_context;      /* SNP guest context page */
 	spinlock_t psc_lock;
+	void *snp_certs_data;
+	struct mutex guest_req_lock;
 };
 
 struct kvm_svm {
Signature · 1 line
2.25.1
#289 lore raw

Re: [PATCH Part2 v6 42/49] KVM: SVM: Provide support for SNP_GUEST_REQUEST NAE event

Quoted 33 lines · Ashish Kalra

From: Brijesh Singh <[email protected]>

Version 2 of GHCB specification added the support for two SNP Guest Request Message NAE events. The events allows for an SEV-SNP guest to make request to the SEV-SNP firmware through hypervisor using the SNP_GUEST_REQUEST API define in the SEV-SNP firmware specification.

The SNP_EXT_GUEST_REQUEST is similar to SNP_GUEST_REQUEST with the difference of an additional certificate blob that can be passed through the SNP_SET_CONFIG ioctl defined in the CCP driver. The CCP driver provides snp_guest_ext_guest_request() that is used by the KVM to get both the report and certificate data at once.

---
 arch/x86/kvm/svm/sev.c | 196 +++++++++++++++++++++++++++++++++++++++--
 arch/x86/kvm/svm/svm.h |   2 +
 2 files changed, 192 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/sev.c +1 -0
index 7fc0fad87054..089af21a4efe 100644
@@ -343,6 +343,7 @@ static int sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp)
 
                spin_lock_init(&sev->psc_lock);
                ret = sev_snp_init(&argp->error);
+               mutex_init(&sev->guest_req_lock);
        } else {
                ret = sev_platform_init(&argp->error);
        }

arch/x86/kvm/svm/sev.c int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 1888

Quoted 4 lines
arch/x86/kvm/svm/sev.c +1 -0
@@ -1884,23 +1885,39 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
 
 static void *snp_context_create(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
+       void *context = NULL, *certs_data = NULL, *resp_page = NULL;

Is the NULL setting here unnecessary since all of these are set via functions snp_alloc_firmware_page(), kmalloc(), and snp_alloc_firmware_page() respectively?

arch/x86/kvm/svm/sev.c int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 1896

Quoted 9 lines
arch/x86/kvm/svm/sev.c +5 -1
+       struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
        struct sev_data_snp_gctx_create data = {};
-       void *context;
        int rc;
 
+       /* Allocate memory used for the certs data in SNP guest request */
+       certs_data = kmalloc(SEV_FW_BLOB_MAX_SIZE, GFP_KERNEL_ACCOUNT);
+       if (!certs_data)
+               return NULL;

I think we want to use kzalloc() here to ensure we never give the guest uninitialized kernel memory.

arch/x86/kvm/svm/sev.c int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd) · line 1906

Quoted 11 lines
arch/x86/kvm/svm/sev.c +7 -1
+
        /* Allocate memory for context page */
        context = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
        if (!context)
-               return NULL;
+               goto e_free;
+
+       /* Allocate a firmware buffer used during the guest command handling. */
+       resp_page = snp_alloc_firmware_page(GFP_KERNEL_ACCOUNT);
+       if (!resp_page)
+               goto e_free;

|resp_page| doesn't appear to be used anywhere?

Quoted 43 lines
arch/x86/kvm/svm/sev.c +13 -4
        data.gctx_paddr = __psp_pa(context);
        rc = __sev_issue_cmd(argp->sev_fd, SEV_CMD_SNP_GCTX_CREATE, &data, &argp->error);
-       if (rc) {
-               snp_free_firmware_page(context);
-               return NULL;
-       }
+       if (rc)
+               goto e_free;
+
+       sev->snp_certs_data = certs_data;
 
        return context;
+
+e_free:
+       snp_free_firmware_page(context);
+       kfree(certs_data);
+       return NULL;
 }
 
 static int snp_bind_asid(struct kvm *kvm, int *error)
@@ -2565,6 +2582,8 @@ static int snp_decommission_context(struct kvm *kvm)
        snp_free_firmware_page(sev->snp_context);
        sev->snp_context = NULL;
 
+       kfree(sev->snp_certs_data);
+
        return 0;
 }
 
@@ -3077,6 +3096,8 @@ static int sev_es_validate_vmgexit(struct vcpu_svm *svm, u64 *exit_code)
        case SVM_VMGEXIT_UNSUPPORTED_EVENT:
        case SVM_VMGEXIT_HV_FEATURES:
        case SVM_VMGEXIT_PSC:
+       case SVM_VMGEXIT_GUEST_REQUEST:
+       case SVM_VMGEXIT_EXT_GUEST_REQUEST:
                break;
        default:
                reason = GHCB_ERR_INVALID_EVENT;
@@ -3502,6 +3523,155 @@ static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm)
        return rc ? map_to_psc_vmgexit_code(rc) : 0;
 }
 

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3535

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+static unsigned long snp_setup_guest_buf(struct vcpu_svm *svm,
+                                        struct sev_data_snp_guest_request *data,
+                                        gpa_t req_gpa, gpa_t resp_gpa)
+{
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       kvm_pfn_t req_pfn, resp_pfn;
+       struct kvm_sev_info *sev;
+
+       sev = &to_kvm_svm(kvm)->sev_info;

This is normally done at declaration in this file. Why not here?

      struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
Quoted 20 lines
arch/x86/kvm/svm/sev.c +20 -0
+
+       if (!IS_ALIGNED(req_gpa, PAGE_SIZE) || !IS_ALIGNED(resp_gpa, PAGE_SIZE))
+               return SEV_RET_INVALID_PARAM;
+
+       req_pfn = gfn_to_pfn(kvm, gpa_to_gfn(req_gpa));
+       if (is_error_noslot_pfn(req_pfn))
+               return SEV_RET_INVALID_ADDRESS;
+
+       resp_pfn = gfn_to_pfn(kvm, gpa_to_gfn(resp_gpa));
+       if (is_error_noslot_pfn(resp_pfn))
+               return SEV_RET_INVALID_ADDRESS;
+
+       if (rmp_make_private(resp_pfn, 0, PG_LEVEL_4K, 0, true))
+               return SEV_RET_INVALID_ADDRESS;
+
+       data->gctx_paddr = __psp_pa(sev->snp_context);
+       data->req_paddr = __sme_set(req_pfn << PAGE_SHIFT);
+       data->res_paddr = __sme_set(resp_pfn << PAGE_SHIFT);
+
+       return 0;

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3565

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+}
+
+static void snp_cleanup_guest_buf(struct sev_data_snp_guest_request *data, unsigned long *rc)
+{
+       u64 pfn = __sme_clr(data->res_paddr) >> PAGE_SHIFT;
+       int ret;
+
+       ret = snp_page_reclaim(pfn);
+       if (ret)
+               *rc = SEV_RET_INVALID_ADDRESS;

Do we need a diff error code here? This means the page the guest gives us is now "stuck" in the FW owned state. How would the guest know this is the case? We return the exact same error in snp_setup_guest_buf() if the resp_gpa isn't page aligned so now if the guest ever sees a SEV_RET_INVALID_ADDRESS I think its only safe option is to either try and page_state_change it to a know state or mark it as unusable memory.

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3569

Quoted 4 lines
arch/x86/kvm/svm/sev.c +4 -0
+
+       ret = rmp_make_shared(pfn, PG_LEVEL_4K);
+       if (ret)
+               *rc = SEV_RET_INVALID_ADDRESS;

Ditto here I think we need some way to signal to the guest what state this page is on return to guest execution.

Also these errors shadow over FW successes, this means the guest's guest-request-sequence-numbers are now out of sync meaning this VMPCK is unusable less the guest risk reusing the AES IV (which would break the confidentiality/integrity). Should we have a way to signal to the guest the FW has successfully run your command but we could not change the page states back correctly, so the guest should increment their sequence numbers.

Quoted 7 lines
arch/x86/kvm/svm/sev.c +7 -0
+}
+
+static void snp_handle_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+       struct sev_data_snp_guest_request data = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3586

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+       struct kvm_sev_info *sev;
+       unsigned long rc;
+       int err;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;

Ditto why not due this above?

Quoted 67 lines
arch/x86/kvm/svm/sev.c +67 -0
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &data, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = sev_issue_cmd(kvm, SEV_CMD_SNP_GUEST_REQUEST, &data, &err);
+       if (rc)
+               /* use the firmware error code */
+               rc = err;
+
+       snp_cleanup_guest_buf(&data, &rc);
+
+unlock:
+       mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+       svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
+static void snp_handle_ext_guest_request(struct vcpu_svm *svm, gpa_t req_gpa, gpa_t resp_gpa)
+{
+       struct sev_data_snp_guest_request req = {0};
+       struct kvm_vcpu *vcpu = &svm->vcpu;
+       struct kvm *kvm = vcpu->kvm;
+       unsigned long data_npages;
+       struct kvm_sev_info *sev;
+       unsigned long rc, err;
+       u64 data_gpa;
+
+       if (!sev_snp_guest(vcpu->kvm)) {
+               rc = SEV_RET_INVALID_GUEST;
+               goto e_fail;
+       }
+
+       sev = &to_kvm_svm(kvm)->sev_info;
+
+       data_gpa = vcpu->arch.regs[VCPU_REGS_RAX];
+       data_npages = vcpu->arch.regs[VCPU_REGS_RBX];
+
+       if (!IS_ALIGNED(data_gpa, PAGE_SIZE)) {
+               rc = SEV_RET_INVALID_ADDRESS;
+               goto e_fail;
+       }
+
+       /* Verify that requested blob will fit in certificate buffer */
+       if ((data_npages << PAGE_SHIFT) > SEV_FW_BLOB_MAX_SIZE) {
+               rc = SEV_RET_INVALID_PARAM;
+               goto e_fail;
+       }
+
+       mutex_lock(&sev->guest_req_lock);
+
+       rc = snp_setup_guest_buf(svm, &req, req_gpa, resp_gpa);
+       if (rc)
+               goto unlock;
+
+       rc = snp_guest_ext_guest_request(&req, (unsigned long)sev->snp_certs_data,
+                                        &data_npages, &err);
+       if (rc) {
+               /*
+                * If buffer length is small then return the expected
+                * length in rbx.
+                */
+               if (err == SNP_GUEST_REQ_INVALID_LEN)
+                       vcpu->arch.regs[VCPU_REGS_RBX] = data_npages;

arch/x86/kvm/svm/sev.c static unsigned long snp_handle_page_state_change(struct vcpu_svm *svm) · line 3663

Quoted 10 lines
arch/x86/kvm/svm/sev.c +10 -0
+
+               /* pass the firmware error code */
+               rc = err;
+               goto cleanup;
+       }
+
+       /* Copy the certificate blob in the guest memory */
+       if (data_npages &&
+           kvm_write_guest(kvm, data_gpa, sev->snp_certs_data, data_npages << PAGE_SHIFT))
+               rc = SEV_RET_INVALID_ADDRESS;

Since at this point the PSP FW has correctly executed the command and incremented the VMPCK sequence number I think we need another error signal here since this will tell the guest the PSP had an error so it will not know if the VMPCK sequence number should be incremented.

Quoted 51 lines
arch/x86/kvm/svm/sev.c +25 -0
+
+cleanup:
+       snp_cleanup_guest_buf(&req, &rc);
+
+unlock:
+       mutex_unlock(&sev->guest_req_lock);
+
+e_fail:
+       svm_set_ghcb_sw_exit_info_2(vcpu, rc);
+}
+
 static int sev_handle_vmgexit_msr_protocol(struct vcpu_svm *svm)
 {
        struct vmcb_control_area *control = &svm->vmcb->control;
@@ -3753,6 +3923,20 @@ int sev_handle_vmgexit(struct kvm_vcpu *vcpu)
                svm_set_ghcb_sw_exit_info_2(vcpu, rc);
                break;
        }
+       case SVM_VMGEXIT_GUEST_REQUEST: {
+               snp_handle_guest_request(svm, control->exit_info_1, control->exit_info_2);
+
+               ret = 1;
+               break;
+       }
+       case SVM_VMGEXIT_EXT_GUEST_REQUEST: {
+               snp_handle_ext_guest_request(svm,
+                                            control->exit_info_1,
+                                            control->exit_info_2);
+
+               ret = 1;
+               break;
+       }
        case SVM_VMGEXIT_UNSUPPORTED_EVENT:
                vcpu_unimpl(vcpu,
                            "vmgexit: unsupported event - exit_info_1=%#llx, exit_info_2=%#llx\n",
arch/x86/kvm/svm/svm.h +2 -1
index 3fd95193ed8d..3be24da1a743 100644
@@ -98,6 +98,8 @@ struct kvm_sev_info {
        u64 snp_init_flags;
        void *snp_context;      /* SNP guest context page */
        spinlock_t psc_lock;
+       void *snp_certs_data;
+       struct mutex guest_req_lock;
 };
 
 struct kvm_svm {
--

2.25.1