Skip to contentSkip to conversation

Add Secure TSC support for SNP guests

PATCH Superseded Nikunj A Dadhania 3 participants 45 messages Feb 15, 2024 – Apr 23, 2024

Subsystems (5)
x86 KVM KVM x86 x86 MM +1 more subsystems

v8 of 16 Superseded Latest is v16 →

lore
Share & tools
lore
#0 lore raw

v8 cover Add Secure TSC support for SNP guests

This patchset is also available at:

https://github.com/AMDESE/linux-kvm/tree/sectsc-guest-latest

and is based on tip/x86/sev base commit ee8ff8768735 ("crypto: ccp - Have it depend on AMD_IOMMU")

Overview
--------

Secure TSC allows guests to securely use RDTSC/RDTSCP instructions as the parameters being used cannot be changed by hypervisor once the guest is launched. More details in the AMD64 APM Vol 2, Section "Secure TSC".

During the boot-up of the secondary cpus, SecureTSC enabled guests need to query TSC info from AMD Security Processor. This communication channel is encrypted between the AMD Security Processor and the guest, the hypervisor is just the conduit to deliver the guest messages to the AMD Security Processor. Each message is protected with an AEAD (AES-256 GCM). See "SEV Secure Nested Paging Firmware ABI Specification" document (currently at
https://www.amd.com/system/files/TechDocs/56860.pdf) section "TSC Info"

Use a minimal GCM library to encrypt/decrypt SNP Guest messages to communicate with the AMD Security Processor which is available at early boot.

SEV-guest driver has the implementation for guest and AMD Security Processor communication. As the TSC_INFO needs to be initialized during early boot before smp cpus are started, move most of the sev-guest driver code to kernel/sev.c and provide well defined APIs to the sev-guest driver to use the interface to avoid code-duplication.

Patches:
01-08: Preparation and movement of sev-guest driver code 09-16: SecureTSC enablement patches.

Testing SecureTSC
-----------------

SecureTSC hypervisor patches based on top of SEV-SNP Guest MEMFD series:
https://github.com/AMDESE/linux-kvm/tree/sectsc-host-latest

QEMU changes:
https://github.com/nikunjad/qemu/tree/snp-securetsc-latest

QEMU commandline SEV-SNP with SecureTSC:

  qemu-system-x86_64 -cpu EPYC-Milan-v2,+securetsc,+invtsc -smp 4 \
    -object memory-backend-memfd,id=ram1,size=1G,share=true,prealloc=false,reserve=false \
    -object sev-snp-guest,id=sev0,cbitpos=51,reduced-phys-bits=1,secure-tsc=on \
    -machine q35,confidential-guest-support=sev0,memory-backend=ram1 \
    ...

Changelog:
----------
v8:
* Rebased on top of tip/x86/sev
* Use minimum size of IV or msg_seqno in memcpy
* Use arch/x86/include/asm/sev.h instead of sev-guest.h
* Use DEFINE_MUTEX for snp_guest_cmd_mutex
* Added Reviewed-by from Tom.
* Dropped Tested-by from patch 3/16

https://lore.kernel.org/kvm/[email protected]/

v7:
* Drop mutex from the snp_dev and add snp_guest_cmd_{lock,unlock} API
* Added comments for secrets page failure
* Added define for maximum supported VMPCK
* Updated comments why sev_status is used directly instead of cpu_feature_enabled()

https://lore.kernel.org/kvm/[email protected]/

v6: https://lore.kernel.org/lkml/[email protected]/
v5: https://lore.kernel.org/lkml/[email protected]/
v4: https://lore.kernel.org/lkml/[email protected]/
v3: https://lore.kernel.org/lkml/[email protected]/
v2: https://lore.kernel.org/r/[email protected]/
v1: https://lore.kernel.org/r/[email protected]

Nikunj A Dadhania (16):
virt: sev-guest: Use AES GCM crypto library
virt: sev-guest: Replace dev_dbg with pr_debug
virt: sev-guest: Add SNP guest request structure
virt: sev-guest: Add vmpck_id to snp_guest_dev struct x86/sev: Cache the secrets page address
virt: sev-guest: Move SNP Guest command mutex
x86/sev: Move and reorganize sev guest request api x86/mm: Add generic guest initialization hook x86/cpufeatures: Add synthetic Secure TSC bit
x86/sev: Add Secure TSC support for SNP guests x86/sev: Change TSC MSR behavior for Secure TSC enabled guests x86/sev: Prevent RDTSC/RDTSCP interception for Secure TSC enabled
guests
x86/kvmclock: Skip kvmclock when Secure TSC is available x86/sev: Mark Secure TSC as reliable
x86/cpu/amd: Do not print FW_BUG for Secure TSC x86/sev: Enable Secure TSC for SNP guests

 arch/x86/Kconfig                        |   1 +
 arch/x86/boot/compressed/sev.c          |   3 +-
 arch/x86/include/asm/cpufeatures.h      |   1 +
 arch/x86/include/asm/sev-common.h       |   1 +
 arch/x86/include/asm/sev.h              | 206 ++++++-
 arch/x86/include/asm/svm.h              |   6 +-
 arch/x86/include/asm/x86_init.h         |   2 +
 arch/x86/kernel/cpu/amd.c               |   3 +-
 arch/x86/kernel/kvmclock.c              |   2 +-
 arch/x86/kernel/sev-shared.c            |  10 +
 arch/x86/kernel/sev.c                   | 648 +++++++++++++++++++--
 arch/x86/kernel/x86_init.c              |   2 +
 arch/x86/mm/mem_encrypt.c               |  12 +-
 arch/x86/mm/mem_encrypt_amd.c           |  12 +
 drivers/virt/coco/sev-guest/Kconfig     |   3 -
 drivers/virt/coco/sev-guest/sev-guest.c | 725 ++++--------------------
 drivers/virt/coco/sev-guest/sev-guest.h |  63 --
 17 files changed, 937 insertions(+), 763 deletions(-)
 delete mode 100644 drivers/virt/coco/sev-guest/sev-guest.h

base-commit: ee8ff8768735edc3e013837c4416f819543ddc17

Signature · 1 line
2.34.1
#1 lore raw

PATCH v8 1/16 virt: sev-guest: Use AES GCM crypto library

The sev-guest driver encryption code uses Crypto API for SNP guest messaging to interact with AMD Security processor. For enabling SecureTSC, SEV-SNP guests need to send a TSC_INFO request guest message before the smpboot phase starts. Details from the TSC_INFO response will be used to program the VMSA before the secondary CPUs are brought up. The Crypto API is not available this early in the boot phase.

In preparation of moving the encryption code out of sev-guest driver to support SecureTSC and make reviewing the diff easier, start using AES GCM library implementation instead of Crypto API.

Drop __enc_payload() and dec_payload() helpers as both are pretty small and can be moved to the respective callers.

---
 drivers/virt/coco/sev-guest/Kconfig     |   4 +-
 drivers/virt/coco/sev-guest/sev-guest.c | 175 ++++++------------------
 drivers/virt/coco/sev-guest/sev-guest.h |   3 +
 3 files changed, 43 insertions(+), 139 deletions(-)
drivers/virt/coco/sev-guest/Kconfig +1 -3
index 1cffc72c41cb..0b772bd921d8 100644
@@ -2,9 +2,7 @@ config SEV_GUEST
 	tristate "AMD SEV Guest driver"
 	default m
 	depends on AMD_MEM_ENCRYPT
-	select CRYPTO
-	select CRYPTO_AEAD2
-	select CRYPTO_GCM
+	select CRYPTO_LIB_AESGCM
 	select TSM_REPORTS
 	help
 	  SEV-SNP firmware provides the guest a mechanism to communicate with
drivers/virt/coco/sev-guest/sev-guest.c +39 -136

295 lines not on this page · raw message

drivers/virt/coco/sev-guest/sev-guest.h +3 -0
index 21bda26fdb95..ceb798a404d6 100644
@@ -13,6 +13,9 @@
 #include <linux/types.h>
 
 #define MAX_AUTHTAG_LEN		32
+#define AUTHTAG_LEN		16
+#define AAD_LEN			48
+#define MSG_HDR_VER		1
 
 /* See SNP spec SNP_GUEST_REQUEST section for the structure */
 enum msg_type {
Signature · 1 line
2.34.1
maintainer Acked-by on v8 1/16
#2 lore raw

Re: [PATCH v8 01/16] virt: sev-guest: Use AES GCM crypto library

Quoted 18 lines · Nikunj A Dadhania

The sev-guest driver encryption code uses Crypto API for SNP guest messaging to interact with AMD Security processor. For enabling SecureTSC, SEV-SNP guests need to send a TSC_INFO request guest message before the smpboot phase starts. Details from the TSC_INFO response will be used to program the VMSA before the secondary CPUs are brought up. The Crypto API is not available this early in the boot phase.

In preparation of moving the encryption code out of sev-guest driver to support SecureTSC and make reviewing the diff easier, start using AES GCM library implementation instead of Crypto API.

Drop __enc_payload() and dec_payload() helpers as both are pretty small and can be moved to the respective callers.

Quoted 5 lines
---
 drivers/virt/coco/sev-guest/Kconfig     |   4 +-
 drivers/virt/coco/sev-guest/sev-guest.c | 175 ++++++------------------
 drivers/virt/coco/sev-guest/sev-guest.h |   3 +
 3 files changed, 43 insertions(+), 139 deletions(-)
Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#3 lore raw

PATCH v8 2/16 virt: sev-guest: Replace dev_dbg with pr_debug

In reply to Nikunj A Dadhania

In preparation of moving code to arch/x86/kernel/sev.c, replace dev_dbg with pr_debug.

---
 drivers/virt/coco/sev-guest/sev-guest.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/virt/coco/sev-guest/sev-guest.c +5 -4
index af35259bc584..01b565170729 100644
@@ -178,8 +178,9 @@ static int verify_and_dec_payload(struct snp_guest_dev *snp_dev, void *payload,
 	struct aesgcm_ctx *ctx = snp_dev->ctx;
 	u8 iv[GCM_AES_IV_SIZE] = {};
 
-	dev_dbg(snp_dev->dev, "response [seqno %lld type %d version %d sz %d]\n",
-		resp_hdr->msg_seqno, resp_hdr->msg_type, resp_hdr->msg_version, resp_hdr->msg_sz);
+	pr_debug("response [seqno %lld type %d version %d sz %d]\n",
+		 resp_hdr->msg_seqno, resp_hdr->msg_type, resp_hdr->msg_version,
+		 resp_hdr->msg_sz);
 
 	/* Copy response from shared memory to encrypted memory. */
 	memcpy(resp, snp_dev->response, sizeof(*resp));
@@ -232,8 +233,8 @@ static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, int version, u8
 	if (!hdr->msg_seqno)
 		return -ENOSR;
 
-	dev_dbg(snp_dev->dev, "request [seqno %lld type %d version %d sz %d]\n",
-		hdr->msg_seqno, hdr->msg_type, hdr->msg_version, hdr->msg_sz);
+	pr_debug("request [seqno %lld type %d version %d sz %d]\n",
+		 hdr->msg_seqno, hdr->msg_type, hdr->msg_version, hdr->msg_sz);
 
 	if (WARN_ON((sz + ctx->authsize) > sizeof(req->payload)))
 		return -EBADMSG;
Signature · 1 line
2.34.1
maintainer Reviewed-by on v8 2/16
#4 lore raw

Re: [PATCH v8 02/16] virt: sev-guest: Replace dev_dbg with pr_debug

Quoted 28 lines · Nikunj A Dadhania

In preparation of moving code to arch/x86/kernel/sev.c, replace dev_dbg with pr_debug.

---
 drivers/virt/coco/sev-guest/sev-guest.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/virt/coco/sev-guest/sev-guest.c +3 -2
index af35259bc584..01b565170729 100644
@@ -178,8 +178,9 @@ static int verify_and_dec_payload(struct snp_guest_dev *snp_dev, void *payload,
 	struct aesgcm_ctx *ctx = snp_dev->ctx;
 	u8 iv[GCM_AES_IV_SIZE] = {};
 
-	dev_dbg(snp_dev->dev, "response [seqno %lld type %d version %d sz %d]\n",
-		resp_hdr->msg_seqno, resp_hdr->msg_type, resp_hdr->msg_version, resp_hdr->msg_sz);
+	pr_debug("response [seqno %lld type %d version %d sz %d]\n",
+		 resp_hdr->msg_seqno, resp_hdr->msg_type, resp_hdr->msg_version,
+		 resp_hdr->msg_sz);
 
 	/* Copy response from shared memory to encrypted memory. */
 	memcpy(resp, snp_dev->response, sizeof(*resp));

drivers/virt/coco/sev-guest/sev-guest.c static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, int version, u8 · line 240

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +2 -2
@@ -232,8 +233,8 @@ static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, int version, u8
 	if (!hdr->msg_seqno)
 		return -ENOSR;
 
-	dev_dbg(snp_dev->dev, "request [seqno %lld type %d version %d sz %d]\n",
-		hdr->msg_seqno, hdr->msg_type, hdr->msg_version, hdr->msg_sz);
+	pr_debug("request [seqno %lld type %d version %d sz %d]\n",
+		 hdr->msg_seqno, hdr->msg_type, hdr->msg_version, hdr->msg_sz);
 
 	if (WARN_ON((sz + ctx->authsize) > sizeof(req->payload)))
 		return -EBADMSG;
Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#5 lore raw

PATCH v8 3/16 virt: sev-guest: Add SNP guest request structure

In reply to Nikunj A Dadhania

Add a snp_guest_req structure to simplify the function arguments. The structure will be used to call the SNP Guest message request API instead of passing a long list of parameters.

Update snp_issue_guest_request() prototype to include the new guest request structure and move the prototype to sev.h.

---
 arch/x86/include/asm/sev.h              |  75 ++++++++-
 arch/x86/kernel/sev.c                   |  15 +-
 drivers/virt/coco/sev-guest/sev-guest.c | 195 +++++++++++++-----------
 drivers/virt/coco/sev-guest/sev-guest.h |  66 --------
 4 files changed, 187 insertions(+), 164 deletions(-)
 delete mode 100644 drivers/virt/coco/sev-guest/sev-guest.h
arch/x86/include/asm/sev.h +71 -4
index bed95e1f4d52..0c0b11af9f89 100644
@@ -111,8 +111,6 @@ struct rmp_state {
 struct snp_req_data {
 	unsigned long req_gpa;
 	unsigned long resp_gpa;
-	unsigned long data_gpa;
-	unsigned int data_npages;
 };
 
 struct sev_guest_platform_data {
@@ -154,6 +152,73 @@ struct snp_secrets_page_layout {
 	u8 rsvd3[3840];
 } __packed;
 
+#define MAX_AUTHTAG_LEN		32
+#define AUTHTAG_LEN		16
+#define AAD_LEN			48
+#define MSG_HDR_VER		1
+
+/* See SNP spec SNP_GUEST_REQUEST section for the structure */
+enum msg_type {
+	SNP_MSG_TYPE_INVALID = 0,
+	SNP_MSG_CPUID_REQ,
+	SNP_MSG_CPUID_RSP,
+	SNP_MSG_KEY_REQ,
+	SNP_MSG_KEY_RSP,
+	SNP_MSG_REPORT_REQ,
+	SNP_MSG_REPORT_RSP,
+	SNP_MSG_EXPORT_REQ,
+	SNP_MSG_EXPORT_RSP,
+	SNP_MSG_IMPORT_REQ,
+	SNP_MSG_IMPORT_RSP,
+	SNP_MSG_ABSORB_REQ,
+	SNP_MSG_ABSORB_RSP,
+	SNP_MSG_VMRK_REQ,
+	SNP_MSG_VMRK_RSP,
+
+	SNP_MSG_TYPE_MAX
+};
+
+enum aead_algo {
+	SNP_AEAD_INVALID,
+	SNP_AEAD_AES_256_GCM,
+};
+
+struct snp_guest_msg_hdr {
+	u8 authtag[MAX_AUTHTAG_LEN];
+	u64 msg_seqno;
+	u8 rsvd1[8];
+	u8 algo;
+	u8 hdr_version;
+	u16 hdr_sz;
+	u8 msg_type;
+	u8 msg_version;
+	u16 msg_sz;
+	u32 rsvd2;
+	u8 msg_vmpck;
+	u8 rsvd3[35];
+} __packed;
+
+struct snp_guest_msg {
+	struct snp_guest_msg_hdr hdr;
+	u8 payload[4000];
+} __packed;
+
+struct snp_guest_req {
+	void *req_buf;
+	size_t req_sz;
+
+	void *resp_buf;
+	size_t resp_sz;
+
+	void *data;
+	size_t data_npages;
+
+	u64 exit_code;
+	unsigned int vmpck_id;
+	u8 msg_version;
+	u8 msg_type;
+};
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 extern void __sev_es_ist_enter(struct pt_regs *regs);
 extern void __sev_es_ist_exit(void);
@@ -223,7 +288,8 @@ void snp_set_memory_private(unsigned long vaddr, unsigned long npages);
 void snp_set_wakeup_secondary_cpu(void);
 bool snp_init(struct boot_params *bp);
 void __init __noreturn snp_abort(void);
-int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio);
+int snp_issue_guest_request(struct snp_guest_req *req, struct snp_req_data *input,
+			    struct snp_guest_request_ioctl *rio);
 void snp_accept_memory(phys_addr_t start, phys_addr_t end);
 u64 snp_get_unsupported_features(u64 status);
 u64 sev_get_status(void);
@@ -248,7 +314,8 @@ static inline void snp_set_memory_private(unsigned long vaddr, unsigned long npa
 static inline void snp_set_wakeup_secondary_cpu(void) { }
 static inline bool snp_init(struct boot_params *bp) { return false; }
 static inline void snp_abort(void) { }
-static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio)
+static inline int snp_issue_guest_request(struct snp_guest_req *req, struct snp_req_data *input,
+					  struct snp_guest_request_ioctl *rio)
 {
 	return -ENOTTY;
 }
arch/x86/kernel/sev.c +8 -7
index 1ef7ae806a01..479b68e00a54 100644
@@ -2173,7 +2173,8 @@ static int __init init_sev_config(char *str)
 }
 __setup("sev=", init_sev_config);
 
-int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio)
+int snp_issue_guest_request(struct snp_guest_req *req, struct snp_req_data *input,
+			    struct snp_guest_request_ioctl *rio)
 {
 	struct ghcb_state state;
 	struct es_em_ctxt ctxt;
@@ -2197,12 +2198,12 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct sn
 
 	vc_ghcb_invalidate(ghcb);
 
-	if (exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST) {
-		ghcb_set_rax(ghcb, input->data_gpa);
-		ghcb_set_rbx(ghcb, input->data_npages);
+	if (req->exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST) {
+		ghcb_set_rax(ghcb, __pa(req->data));
+		ghcb_set_rbx(ghcb, req->data_npages);
 	}
 
-	ret = sev_es_ghcb_hv_call(ghcb, &ctxt, exit_code, input->req_gpa, input->resp_gpa);
+	ret = sev_es_ghcb_hv_call(ghcb, &ctxt, req->exit_code, input->req_gpa, input->resp_gpa);
 	if (ret)
 		goto e_put;
 
@@ -2217,8 +2218,8 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct sn
 
 	case SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVALID_LEN):
 		/* Number of expected pages are returned in RBX */
-		if (exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST) {
-			input->data_npages = ghcb_get_rbx(ghcb);
+		if (req->exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST) {
+			req->data_npages = ghcb_get_rbx(ghcb);
 			ret = -ENOSPC;
 			break;
 		}
drivers/virt/coco/sev-guest/sev-guest.c +108 -87

407 lines not on this page · raw message

drivers/virt/coco/sev-guest/sev-guest.h deleted +0 -66

66 lines not on this page · raw message

Signature · 1 line
2.34.1
#6 lore raw

Re: [PATCH v8 03/16] virt: sev-guest: Add SNP guest request structure

Quoted 72 lines · Nikunj A Dadhania

Add a snp_guest_req structure to simplify the function arguments. The structure will be used to call the SNP Guest message request API instead of passing a long list of parameters.

Update snp_issue_guest_request() prototype to include the new guest request structure and move the prototype to sev.h.

---
  arch/x86/include/asm/sev.h              |  75 ++++++++-
  arch/x86/kernel/sev.c                   |  15 +-
  drivers/virt/coco/sev-guest/sev-guest.c | 195 +++++++++++++-----------
  drivers/virt/coco/sev-guest/sev-guest.h |  66 --------
  4 files changed, 187 insertions(+), 164 deletions(-)

delete mode 100644 drivers/virt/coco/sev-guest/sev-guest.h

arch/x86/include/asm/sev.h +39 -2
index bed95e1f4d52..0c0b11af9f89 100644
@@ -111,8 +111,6 @@ struct rmp_state {
  struct snp_req_data {
  	unsigned long req_gpa;
  	unsigned long resp_gpa;
-	unsigned long data_gpa;
-	unsigned int data_npages;
  };
 
  struct sev_guest_platform_data {
@@ -154,6 +152,73 @@ struct snp_secrets_page_layout {
  	u8 rsvd3[3840];
  } __packed;
 
+#define MAX_AUTHTAG_LEN		32
+#define AUTHTAG_LEN		16
+#define AAD_LEN			48
+#define MSG_HDR_VER		1
+
+/* See SNP spec SNP_GUEST_REQUEST section for the structure */
+enum msg_type {
+	SNP_MSG_TYPE_INVALID = 0,
+	SNP_MSG_CPUID_REQ,
+	SNP_MSG_CPUID_RSP,
+	SNP_MSG_KEY_REQ,
+	SNP_MSG_KEY_RSP,
+	SNP_MSG_REPORT_REQ,
+	SNP_MSG_REPORT_RSP,
+	SNP_MSG_EXPORT_REQ,
+	SNP_MSG_EXPORT_RSP,
+	SNP_MSG_IMPORT_REQ,
+	SNP_MSG_IMPORT_RSP,
+	SNP_MSG_ABSORB_REQ,
+	SNP_MSG_ABSORB_RSP,
+	SNP_MSG_VMRK_REQ,
+	SNP_MSG_VMRK_RSP,
+
+	SNP_MSG_TYPE_MAX
+};
+
+enum aead_algo {
+	SNP_AEAD_INVALID,
+	SNP_AEAD_AES_256_GCM,
+};
+
+struct snp_guest_msg_hdr {
+	u8 authtag[MAX_AUTHTAG_LEN];
+	u64 msg_seqno;
+	u8 rsvd1[8];
+	u8 algo;
+	u8 hdr_version;
+	u16 hdr_sz;
+	u8 msg_type;

arch/x86/include/asm/sev.h struct snp_secrets_page_layout { · line 203

Quoted 10 lines
arch/x86/include/asm/sev.h +10 -0
+	u8 msg_version;
+	u16 msg_sz;
+	u32 rsvd2;
+	u8 msg_vmpck;
+	u8 rsvd3[35];
+} __packed;
+
+struct snp_guest_msg {
+	struct snp_guest_msg_hdr hdr;
+	u8 payload[4000];

If the idea is to ensure that payload never goes beyond a page boundary (assuming page allocation/backing), it would be better to have:

	u8 payload[PAGE_SIZE - sizeof(struct snp_guest_msg_hdr)];

instead of hard-coding 4000 (I realize this is existing code). Although, since you probably want to ensure that you don't exceed the page allocation by testing against the size or page offset, you can just make this a variable length array:

	u8 payload[];

and ensure that you don't overrun.

Thanks,
Tom

Quoted 2 lines
arch/x86/include/asm/sev.h +2 -0
+} __packed;
+
#7 lore raw

Re: [PATCH v8 03/16] virt: sev-guest: Add SNP guest request structure

Quoted 84 lines · Tom Lendacky
Quoted 70 lines · Nikunj A Dadhania

Add a snp_guest_req structure to simplify the function arguments. The structure will be used to call the SNP Guest message request API instead of passing a long list of parameters.

Update snp_issue_guest_request() prototype to include the new guest request structure and move the prototype to sev.h.

---
  arch/x86/include/asm/sev.h              |  75 ++++++++- arch/x86/kernel/sev.c                   |  15 +- drivers/virt/coco/sev-guest/sev-guest.c | 195 +++++++++++++----------- drivers/virt/coco/sev-guest/sev-guest.h |  66 --------

  4 files changed, 187 insertions(+), 164 deletions(-)

  delete mode 100644 drivers/virt/coco/sev-guest/sev-guest.h

arch/x86/include/asm/sev.h +0 -2
index bed95e1f4d52..0c0b11af9f89 100644
@@ -111,8 +111,6 @@ struct rmp_state {
   struct snp_req_data {
       unsigned long req_gpa;
       unsigned long resp_gpa;
-    unsigned long data_gpa;
-    unsigned int data_npages;
   };
     struct sev_guest_platform_data {
@@ -154,6 +152,73 @@ struct snp_secrets_page_layout {
       u8 rsvd3[3840];
   } __packed;

  +#define MAX_AUTHTAG_LEN        32

arch/x86/include/asm/sev.h +38 -0
+#define AUTHTAG_LEN        16
+#define AAD_LEN            48
+#define MSG_HDR_VER        1
+
+/* See SNP spec SNP_GUEST_REQUEST section for the structure */
+enum msg_type {
+    SNP_MSG_TYPE_INVALID = 0,
+    SNP_MSG_CPUID_REQ,
+    SNP_MSG_CPUID_RSP,
+    SNP_MSG_KEY_REQ,
+    SNP_MSG_KEY_RSP,
+    SNP_MSG_REPORT_REQ,
+    SNP_MSG_REPORT_RSP,
+    SNP_MSG_EXPORT_REQ,
+    SNP_MSG_EXPORT_RSP,
+    SNP_MSG_IMPORT_REQ,
+    SNP_MSG_IMPORT_RSP,
+    SNP_MSG_ABSORB_REQ,
+    SNP_MSG_ABSORB_RSP,
+    SNP_MSG_VMRK_REQ,
+    SNP_MSG_VMRK_RSP,
+
+    SNP_MSG_TYPE_MAX
+};
+
+enum aead_algo {
+    SNP_AEAD_INVALID,
+    SNP_AEAD_AES_256_GCM,
+};
+
+struct snp_guest_msg_hdr {
+    u8 authtag[MAX_AUTHTAG_LEN];
+    u64 msg_seqno;
+    u8 rsvd1[8];
+    u8 algo;
+    u8 hdr_version;
+    u16 hdr_sz;
+    u8 msg_type;

arch/x86/include/asm/sev.h

Quoted 10 lines
arch/x86/include/asm/sev.h +10 -0
+    u8 msg_version;
+    u16 msg_sz;
+    u32 rsvd2;
+    u8 msg_vmpck;
+    u8 rsvd3[35];
+} __packed;
+
+struct snp_guest_msg {
+    struct snp_guest_msg_hdr hdr;
+    u8 payload[4000];

If the idea is to ensure that payload never goes beyond a page boundary (assuming page allocation/backing), it would be better to have:

Quoted 7 lines

    u8 payload[PAGE_SIZE - sizeof(struct snp_guest_msg_hdr)];

instead of hard-coding 4000 (I realize this is existing code). Although, since you probably want to ensure that you don't exceed the page allocation by testing against the size or page offset, you can just make this a variable length array:

    u8 payload[];

and ensure that you don't overrun.

Sure, below is the delta to make payload a variable length array. I will squash it with current patch.

arch/x86/include/asm/sev.h +4 -1
index 0c0b11af9f89..85cf160f6203 100644
@@ -200,9 +200,12 @@ struct snp_guest_msg_hdr {
 
 struct snp_guest_msg {
 	struct snp_guest_msg_hdr hdr;
-	u8 payload[4000];
+	u8 payload[];
 } __packed;
 
+#define SNP_GUEST_MSG_SIZE 4096
+#define SNP_GUEST_MSG_PAYLOAD_SIZE (SNP_GUEST_MSG_SIZE - sizeof(struct snp_guest_msg))
+
 struct snp_guest_req {
 	void *req_buf;
 	size_t req_sz;
drivers/virt/coco/sev-guest/sev-guest.c +30 -16

111 lines not on this page · raw message

maintainer
#8 lore raw

Re: [PATCH v8 03/16] virt: sev-guest: Add SNP guest request structure

In reply to Nikunj A Dadhania
Quoted 1 line · Nikunj A Dadhania

+enum aead_algo {

Looks unused.

Add new struct definitions, etc, together with their first user - not preemptively.

Please audit all your patches.

Quoted 12 lines
arch/x86/include/asm/sev.h +12 -0
+	SNP_AEAD_INVALID,
+	SNP_AEAD_AES_256_GCM,
+};
+
+struct snp_guest_msg_hdr {
+	u8 authtag[MAX_AUTHTAG_LEN];
+	u64 msg_seqno;
+	u8 rsvd1[8];
+	u8 algo;
+	u8 hdr_version;
+	u16 hdr_sz;
+	u8 msg_type;

arch/x86/include/asm/sev.h struct snp_secrets_page_layout { · line 203

Quoted 10 lines
arch/x86/include/asm/sev.h +10 -0
+	u8 msg_version;
+	u16 msg_sz;
+	u32 rsvd2;
+	u8 msg_vmpck;
+	u8 rsvd3[35];
+} __packed;
+
+struct snp_guest_msg {
+	struct snp_guest_msg_hdr hdr;
+	u8 payload[4000];

What Tom said.

Quoted 22 lines
arch/x86/include/asm/sev.h +18 -0
+} __packed;
+
+struct snp_guest_req {
+	void *req_buf;
+	size_t req_sz;
+
+	void *resp_buf;
+	size_t resp_sz;
+
+	void *data;
+	size_t data_npages;
+
+	u64 exit_code;
+	unsigned int vmpck_id;
+	u8 msg_version;
+	u8 msg_type;
+};
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 extern void __sev_es_ist_enter(struct pt_regs *regs);
 extern void __sev_es_ist_exit(void);

arch/x86/include/asm/sev.h void snp_set_memory_private(unsigned long vaddr, unsigned long npages); · line 292

Quoted 6 lines
arch/x86/include/asm/sev.h +2 -1
@@ -223,7 +288,8 @@ void snp_set_memory_private(unsigned long vaddr, unsigned long npages);
 void snp_set_wakeup_secondary_cpu(void);
 bool snp_init(struct boot_params *bp);
 void __init __noreturn snp_abort(void);
-int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio);
+int snp_issue_guest_request(struct snp_guest_req *req, struct snp_req_data *input,
+			    struct snp_guest_request_ioctl *rio);

Much nicer!

...

drivers/virt/coco/sev-guest/sev-guest.c static int __init sev_guest_probe(struct platform_device *pdev) · line 890

Quoted 2 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -0
@@ -868,7 +890,6 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	/* initial the input address for guest request */

That comment reads wrong - might fix it while here.

Quoted 7 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -1
 	snp_dev->input.req_gpa = __pa(snp_dev->request);
 	snp_dev->input.resp_gpa = __pa(snp_dev->response);
-	snp_dev->input.data_gpa = __pa(snp_dev->certs_data);
 
 
 	ret = tsm_register(&sev_tsm_ops, snp_dev, &tsm_report_extra_type);

if (ret)

Rest looks ok. I'd chose shorter local vars if I were you but yours are ok too.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#9 lore raw

Re: [PATCH v8 03/16] virt: sev-guest: Add SNP guest request structure

Quoted 4 lines · Borislav Petkov
Quoted 1 line · Nikunj A Dadhania

+enum aead_algo {

Looks unused.

This is being moved from drivers/virt/coco/sev-guest/sev-guest.h and is used in drivers/virt/coco/sev-guest/sev-guest.c::enc_payload()

hdr->algo = SNP_AEAD_AES_256_GCM;

Quoted 4 lines

Add new struct definitions, etc, together with their first user - not preemptively.

Please audit all your patches.

Sure.

Quoted 17 lines
Quoted 17 lines
arch/x86/include/asm/sev.h +16 -0
+	SNP_AEAD_INVALID,
+	SNP_AEAD_AES_256_GCM,
+};
+
+struct snp_guest_msg_hdr {
+	u8 authtag[MAX_AUTHTAG_LEN];
+	u64 msg_seqno;
+	u8 rsvd1[8];
+	u8 algo;
+	u8 hdr_version;
+	u16 hdr_sz;
+	u8 msg_type;
+	u8 msg_version;
+	u16 msg_sz;
+	u32 rsvd2;
+	u8 msg_vmpck;
Quoted 8 lines

arch/x86/include/asm/sev.h struct snp_secrets_page_layout { · line 203

Quoted 6 lines
arch/x86/include/asm/sev.h +6 -0
+	u8 rsvd3[35];
+} __packed;
+
+struct snp_guest_msg {
+	struct snp_guest_msg_hdr hdr;
+	u8 payload[4000];

What Tom said.

Responded on that thread.

Quoted 23 lines
Quoted 23 lines
arch/x86/include/asm/sev.h +18 -0
+} __packed;
+
+struct snp_guest_req {
+	void *req_buf;
+	size_t req_sz;
+
+	void *resp_buf;
+	size_t resp_sz;
+
+	void *data;
+	size_t data_npages;
+
+	u64 exit_code;
+	unsigned int vmpck_id;
+	u8 msg_version;
+	u8 msg_type;
+};
+
 #ifdef CONFIG_AMD_MEM_ENCRYPT
 extern void __sev_es_ist_enter(struct pt_regs *regs);
 extern void __sev_es_ist_exit(void);
@@ -223,7 +288,8 @@ void snp_set_memory_private(unsigned long vaddr, unsigned long npages);
Quoted 8 lines

arch/x86/include/asm/sev.h void snp_set_memory_private(unsigned long vaddr, unsigned long npages); · line 292

Quoted 6 lines
arch/x86/include/asm/sev.h +2 -1
 void snp_set_wakeup_secondary_cpu(void);
 bool snp_init(struct boot_params *bp);
 void __init __noreturn snp_abort(void);
-int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, struct snp_guest_request_ioctl *rio);
+int snp_issue_guest_request(struct snp_guest_req *req, struct snp_req_data *input,
+			    struct snp_guest_request_ioctl *rio);

Much nicer!

Thank you.

Quoted 7 lines

...

drivers/virt/coco/sev-guest/sev-guest.c static int __init sev_guest_probe(struct platform_device *pdev) · line 890

Quoted 2 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -0
@@ -868,7 +890,6 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	/* initial the input address for guest request */

That comment reads wrong - might fix it while here.

Will update.

Quoted 11 lines
Quoted 7 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -1
 	snp_dev->input.req_gpa = __pa(snp_dev->request);
 	snp_dev->input.resp_gpa = __pa(snp_dev->response);
-	snp_dev->input.data_gpa = __pa(snp_dev->certs_data);
 
 
 	ret = tsm_register(&sev_tsm_ops, snp_dev, &tsm_report_extra_type);

if (ret)

Rest looks ok. I'd chose shorter local vars if I were you but yours are ok too.

Regards
Nikunj

#10 lore raw

PATCH v8 4/16 virt: sev-guest: Add vmpck_id to snp_guest_dev struct

In reply to Nikunj A Dadhania

Drop vmpck and os_area_msg_seqno pointers so that secret page layout does not need to be exposed to the sev-guest driver after the rework. Instead, add helper APIs to access vmpck and os_area_msg_seqno when needed. Added define for maximum supported VMPCK.

Also, change function is_vmpck_empty() to snp_is_vmpck_empty() in preparation for moving to sev.c.

---
 arch/x86/include/asm/sev.h              |  1 +
 drivers/virt/coco/sev-guest/sev-guest.c | 95 ++++++++++++-------------
 2 files changed, 48 insertions(+), 48 deletions(-)
arch/x86/include/asm/sev.h +1 -0
index 0c0b11af9f89..e4f52a606487 100644
@@ -135,6 +135,7 @@ struct secrets_os_area {
 } __packed;
 
 #define VMPCK_KEY_LEN		32
+#define VMPCK_MAX_NUM		4
 
 /* See the SNP spec version 0.9 for secrets page format */
 struct snp_secrets_page_layout {
drivers/virt/coco/sev-guest/sev-guest.c +47 -48

196 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#11 lore raw

Re: [PATCH v8 04/16] virt: sev-guest: Add vmpck_id to snp_guest_dev struct

Quoted 4 lines · Nikunj A Dadhania

Drop vmpck and os_area_msg_seqno pointers so that secret page layout does not need to be exposed to the sev-guest driver after the rework. Instead, add helper APIs to access vmpck and os_area_msg_seqno when needed. Added define for maximum supported VMPCK.

Do not talk about *what* the patch is doing in the commit message - that should be obvious from the diff itself. Rather, concentrate on the *why* it needs to be done.

Imagine one fine day you're doing git archeology, you find the place in the code about which you want to find out why it was changed the way it is now.

You do git annotate <filename> ... find the line, see the commit id and you do:

git show <commit id>

You read the commit message and there's just gibberish and nothing's explaining *why* that change was done. And you start scratching your head, trying to figure out why...

I'm sure you're getting the idea.

Quoted 38 lines

Also, change function is_vmpck_empty() to snp_is_vmpck_empty() in preparation for moving to sev.c.

---
 arch/x86/include/asm/sev.h              |  1 +
 drivers/virt/coco/sev-guest/sev-guest.c | 95 ++++++++++++-------------
 2 files changed, 48 insertions(+), 48 deletions(-)
arch/x86/include/asm/sev.h +1 -0
index 0c0b11af9f89..e4f52a606487 100644
@@ -135,6 +135,7 @@ struct secrets_os_area {
 } __packed;
 
 #define VMPCK_KEY_LEN		32
+#define VMPCK_MAX_NUM		4
 
 /* See the SNP spec version 0.9 for secrets page format */
 struct snp_secrets_page_layout {
drivers/virt/coco/sev-guest/sev-guest.c +1 -2
index 596cec03f9eb..646eb215f3c7 100644
@@ -55,8 +55,7 @@ struct snp_guest_dev {
 		struct snp_derived_key_req derived_key;
 		struct snp_ext_report_req ext_report;
 	} req;
-	u32 *os_area_msg_seqno;
-	u8 *vmpck;
+	unsigned int vmpck_id;
 };
 
 static u32 vmpck_id;

drivers/virt/coco/sev-guest/sev-guest.c MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP. · line 68

Quoted 5 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -1
@@ -66,14 +65,22 @@ MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP.
 /* Mutex to serialize the shared buffer access and command handling. */
 static DEFINE_MUTEX(snp_cmd_mutex);
 
-static bool is_vmpck_empty(struct snp_guest_dev *snp_dev)
+static inline u8 *snp_get_vmpck(struct snp_guest_dev *snp_dev)

static functions don't need a prefix like "snp_".

drivers/virt/coco/sev-guest/sev-guest.c MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP. · line 73

Quoted 8 lines
drivers/virt/coco/sev-guest/sev-guest.c +3 -3
 {
-	char zero_key[VMPCK_KEY_LEN] = {0};
+	return snp_dev->layout->vmpck0 + snp_dev->vmpck_id * VMPCK_KEY_LEN;
+}
 
-	if (snp_dev->vmpck)
-		return !memcmp(snp_dev->vmpck, zero_key, VMPCK_KEY_LEN);
+static inline u32 *snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

Ditto.

Quoted 15 lines
drivers/virt/coco/sev-guest/sev-guest.c +9 -1
+{
+	return &snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id;
+}
 
-	return true;
+static bool snp_is_vmpck_empty(struct snp_guest_dev *snp_dev)
+{
+	char zero_key[VMPCK_KEY_LEN] = {0};
+	u8 *key = snp_get_vmpck(snp_dev);
+
+	return !memcmp(key, zero_key, VMPCK_KEY_LEN);
 }
 
 /*

drivers/virt/coco/sev-guest/sev-guest.c static bool is_vmpck_empty(struct snp_guest_dev *snp_dev) · line 105

Quoted 8 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -4
@@ -95,20 +102,22 @@ static bool is_vmpck_empty(struct snp_guest_dev *snp_dev)
  */
 static void snp_disable_vmpck(struct snp_guest_dev *snp_dev)
 {
-	dev_alert(snp_dev->dev, "Disabling vmpck_id %d to prevent IV reuse.\n",
-		  vmpck_id);
-	memzero_explicit(snp_dev->vmpck, VMPCK_KEY_LEN);
-	snp_dev->vmpck = NULL;
+	u8 *key = snp_get_vmpck(snp_dev);

Check whether is_vmpck_empty before you disable?

Quoted 6 lines
drivers/virt/coco/sev-guest/sev-guest.c +4 -0
+
+	dev_alert(snp_dev->dev, "Disabling vmpck_id %u to prevent IV reuse.\n",
+		  snp_dev->vmpck_id);
+	memzero_explicit(key, VMPCK_KEY_LEN);
 }
 

drivers/virt/coco/sev-guest/sev-guest.c static bool is_vmpck_empty(struct snp_guest_dev *snp_dev) · line 120

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +2 -1
 static inline u64 __snp_get_msg_seqno(struct snp_guest_dev *snp_dev)
 {
+	u32 *os_area_msg_seqno = snp_get_os_area_msg_seqno(snp_dev);
 	u64 count;
 
 	lockdep_assert_held(&snp_cmd_mutex);
 
 	/* Read the current message sequence counter from secrets pages */
-	count = *snp_dev->os_area_msg_seqno;
+	count = *os_area_msg_seqno;

Why does that snp_get_os_area_msg_seqno() returns a pointer when you deref it here again?

A function which returns a sequence number should return that number
- not a pointer to it.

Which then makes that u32 *os_area_msg_seqno redundant and you can use the function directly.

IOW:

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
        return snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id;
}

Simple.

Quoted 5 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -0
 	return count + 1;
 }
@@ -136,11 +145,13 @@ static u64 snp_get_msg_seqno(struct snp_guest_dev *snp_dev)
 

drivers/virt/coco/sev-guest/sev-guest.c static u64 snp_get_msg_seqno(struct snp_guest_dev *snp_dev) · line 154

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +3 -1
 static void snp_inc_msg_seqno(struct snp_guest_dev *snp_dev)
 {
+	u32 *os_area_msg_seqno = snp_get_os_area_msg_seqno(snp_dev);
+
 	/*
 	 * The counter is also incremented by the PSP, so increment it by 2
 	 * and save in secrets page.
 	 */
-	*snp_dev->os_area_msg_seqno += 2;
+	*os_area_msg_seqno += 2;

Yah, you have a getter but not a setter. You're setting it through the pointer. Do you see the imbalance in the APIs?

Quoted 5 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -0
 }
 
 static inline struct snp_guest_dev *to_snp_dev(struct file *file)
@@ -150,15 +161,22 @@ static inline struct snp_guest_dev *to_snp_dev(struct file *file)
 	return container_of(dev, struct snp_guest_dev, misc);

drivers/virt/coco/sev-guest/sev-guest.c static inline struct snp_guest_dev *to_snp_dev(struct file *file) · line 170

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +5 -1
 }
 
-static struct aesgcm_ctx *snp_init_crypto(u8 *key, size_t keylen)
+static struct aesgcm_ctx *snp_init_crypto(struct snp_guest_dev *snp_dev)
 {
 	struct aesgcm_ctx *ctx;
+	u8 *key;
+
+	if (snp_is_vmpck_empty(snp_dev)) {
+		pr_err("VM communication key VMPCK%u is null\n", vmpck_id);

"Empty/invalid VMPCK%u communication key"

or so.

In a pre-patch, fix all your user-visible strings to say "VMPCK"
- capitalized as it is an abbreviation.

Quoted 23 lines
drivers/virt/coco/sev-guest/sev-guest.c +5 -2
+		return NULL;
+	}
 
 	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL_ACCOUNT);
 	if (!ctx)
 		return NULL;
 
-	if (aesgcm_expandkey(ctx, key, keylen, AUTHTAG_LEN)) {
+	key = snp_get_vmpck(snp_dev);
+	if (aesgcm_expandkey(ctx, key, VMPCK_KEY_LEN, AUTHTAG_LEN)) {
 		pr_err("Crypto context initialization failed\n");
 		kfree(ctx);
 		return NULL;
@@ -590,7 +608,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long
 	mutex_lock(&snp_cmd_mutex);
 
 	/* Check if the VMPCK is not empty */
-	if (is_vmpck_empty(snp_dev)) {
+	if (snp_is_vmpck_empty(snp_dev)) {
 		dev_err_ratelimited(snp_dev->dev, "VMPCK is disabled\n");
 		mutex_unlock(&snp_cmd_mutex);
 		return -ENOTTY;

drivers/virt/coco/sev-guest/sev-guest.c static const struct file_operations snp_guest_fops = { · line 691

Quoted 9 lines
drivers/virt/coco/sev-guest/sev-guest.c +3 -2
@@ -667,32 +685,14 @@ static const struct file_operations snp_guest_fops = {
 	.unlocked_ioctl = snp_guest_ioctl,
 };
 
-static u8 *get_vmpck(int id, struct snp_secrets_page_layout *layout, u32 **seqno)
+static bool snp_assign_vmpck(struct snp_guest_dev *dev, unsigned int vmpck_id)
 {
-	u8 *key = NULL;
+	if (WARN_ON((vmpck_id + 1) > VMPCK_MAX_NUM))
+		return false;

So this will warn *and*, at the call site too. Let's tone that down.

Quoted 11 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -10
-	switch (id) {
-	case 0:
-		*seqno = &layout->os_area.msg_seqno_0;
-		key = layout->vmpck0;
-		break;
-	case 1:
-		*seqno = &layout->os_area.msg_seqno_1;
-		key = layout->vmpck1;
-		break;
-	case 2:

drivers/virt/coco/sev-guest/sev-guest.c static const struct file_operations snp_guest_fops = { · line 693

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -10
-		*seqno = &layout->os_area.msg_seqno_2;
-		key = layout->vmpck2;
-		break;
-	case 3:
-		*seqno = &layout->os_area.msg_seqno_3;
-		key = layout->vmpck3;
-		break;
-	default:
-		break;
-	}

Your commit message could explain why this is not needed, all of a sudden.

Quoted 24 lines
drivers/virt/coco/sev-guest/sev-guest.c +4 -5
+	dev->vmpck_id = vmpck_id;
 
-	return key;
+	return true;
 }
 
 struct snp_msg_report_resp_hdr {
@@ -728,7 +728,7 @@ static int sev_report_new(struct tsm_report *report, void *data)
 	guard(mutex)(&snp_cmd_mutex);
 
 	/* Check if the VMPCK is not empty */
-	if (is_vmpck_empty(snp_dev)) {
+	if (snp_is_vmpck_empty(snp_dev)) {
 		dev_err_ratelimited(snp_dev->dev, "VMPCK is disabled\n");
 		return -ENOTTY;
 	}
@@ -848,21 +848,20 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 		goto e_unmap;
 
 	ret = -EINVAL;
-	snp_dev->vmpck = get_vmpck(vmpck_id, layout, &snp_dev->os_area_msg_seqno);
-	if (!snp_dev->vmpck) {
-		dev_err(dev, "invalid vmpck id %d\n", vmpck_id);
+	snp_dev->layout = layout;

drivers/virt/coco/sev-guest/sev-guest.c static int __init sev_guest_probe(struct platform_device *pdev) · line 859

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +4 -2
+	if (!snp_assign_vmpck(snp_dev, vmpck_id)) {
+		dev_err(dev, "invalid vmpck id %u\n", vmpck_id);
 		goto e_unmap;
 	}
 
 	/* Verify that VMPCK is not zero. */
-	if (is_vmpck_empty(snp_dev)) {
-		dev_err(dev, "vmpck id %d is null\n", vmpck_id);
+	if (snp_is_vmpck_empty(snp_dev)) {
+		dev_err(dev, "vmpck id %u is null\n", vmpck_id);

s!null!Invalid/Empty!

Quoted 19 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -2
 		goto e_unmap;
 	}
 
 	platform_set_drvdata(pdev, snp_dev);
 	snp_dev->dev = dev;
-	snp_dev->layout = layout;
 
 	/* Allocate the shared page used for the request and response message. */
 	snp_dev->request = alloc_shared_pages(dev, sizeof(struct snp_guest_msg));
@@ -878,7 +877,7 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 		goto e_free_response;
 
 	ret = -EIO;
-	snp_dev->ctx = snp_init_crypto(snp_dev->vmpck, VMPCK_KEY_LEN);
+	snp_dev->ctx = snp_init_crypto(snp_dev);
 	if (!snp_dev->ctx)
 		goto e_free_cert_data;
 

drivers/virt/coco/sev-guest/sev-guest.c static int __init sev_guest_probe(struct platform_device *pdev) · line 905

Quoted 5 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -1
@@ -903,7 +902,7 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	if (ret)
 		goto e_free_ctx;
 
-	dev_info(dev, "Initialized SEV guest driver (using vmpck_id %d)\n", vmpck_id);
+	dev_info(dev, "Initialized SEV guest driver (using vmpck_id %u)\n", vmpck_id);

Yet another spelling: "vmpck_id". Unify all those in a pre-patch pls because it looks stupid.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#12 lore raw

Re: [PATCH v8 04/16] virt: sev-guest: Add vmpck_id to snp_guest_dev struct

Quoted 17 lines · Borislav Petkov
Quoted 4 lines · Nikunj A Dadhania

Drop vmpck and os_area_msg_seqno pointers so that secret page layout does not need to be exposed to the sev-guest driver after the rework. Instead, add helper APIs to access vmpck and os_area_msg_seqno when needed. Added define for maximum supported VMPCK.

Do not talk about *what* the patch is doing in the commit message - that should be obvious from the diff itself. Rather, concentrate on the *why* it needs to be done.

Imagine one fine day you're doing git archeology, you find the place in the code about which you want to find out why it was changed the way it is now.

You do git annotate <filename> ... find the line, see the commit id and you do:

Quoted 7 lines

git show <commit id>

You read the commit message and there's just gibberish and nothing's explaining *why* that change was done. And you start scratching your head, trying to figure out why...

I'm sure you're getting the idea.

Sure, will reword the commit message and send the patch.

Quoted 42 lines
Quoted 42 lines

Also, change function is_vmpck_empty() to snp_is_vmpck_empty() in preparation for moving to sev.c.

---
 arch/x86/include/asm/sev.h              |  1 +
 drivers/virt/coco/sev-guest/sev-guest.c | 95 ++++++++++++-------------
 2 files changed, 48 insertions(+), 48 deletions(-)
arch/x86/include/asm/sev.h +1 -0
index 0c0b11af9f89..e4f52a606487 100644
@@ -135,6 +135,7 @@ struct secrets_os_area {
 } __packed;
 
 #define VMPCK_KEY_LEN		32
+#define VMPCK_MAX_NUM		4
 
 /* See the SNP spec version 0.9 for secrets page format */
 struct snp_secrets_page_layout {
drivers/virt/coco/sev-guest/sev-guest.c +1 -2
index 596cec03f9eb..646eb215f3c7 100644
@@ -55,8 +55,7 @@ struct snp_guest_dev {
 		struct snp_derived_key_req derived_key;
 		struct snp_ext_report_req ext_report;
 	} req;
-	u32 *os_area_msg_seqno;
-	u8 *vmpck;
+	unsigned int vmpck_id;
 };
 
 static u32 vmpck_id;
@@ -66,14 +65,22 @@ MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP.
 /* Mutex to serialize the shared buffer access and command handling. */
 static DEFINE_MUTEX(snp_cmd_mutex);
 
Quoted 4 lines

drivers/virt/coco/sev-guest/sev-guest.c MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP. · line 68

Quoted 2 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -1
-static bool is_vmpck_empty(struct snp_guest_dev *snp_dev)
+static inline u8 *snp_get_vmpck(struct snp_guest_dev *snp_dev)

static functions don't need a prefix like "snp_".

Sure

Quoted 11 lines

drivers/virt/coco/sev-guest/sev-guest.c MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP. · line 73

Quoted 9 lines
drivers/virt/coco/sev-guest/sev-guest.c +3 -3
 {
-	char zero_key[VMPCK_KEY_LEN] = {0};
+	return snp_dev->layout->vmpck0 + snp_dev->vmpck_id * VMPCK_KEY_LEN;
+}
 
-	if (snp_dev->vmpck)
-		return !memcmp(snp_dev->vmpck, zero_key, VMPCK_KEY_LEN);
+static inline u32 *snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

Ditto.

Sure

Quoted 50 lines
Quoted 16 lines
drivers/virt/coco/sev-guest/sev-guest.c +9 -1
+{
+	return &snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id;
+}
 
-	return true;
+static bool snp_is_vmpck_empty(struct snp_guest_dev *snp_dev)
+{
+	char zero_key[VMPCK_KEY_LEN] = {0};
+	u8 *key = snp_get_vmpck(snp_dev);
+
+	return !memcmp(key, zero_key, VMPCK_KEY_LEN);
 }
 
 /*

drivers/virt/coco/sev-guest/sev-guest.c static bool is_vmpck_empty(struct snp_guest_dev *snp_dev) · line 105

Quoted 8 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -4
@@ -95,20 +102,22 @@ static bool is_vmpck_empty(struct snp_guest_dev *snp_dev)
  */
 static void snp_disable_vmpck(struct snp_guest_dev *snp_dev)
 {
-	dev_alert(snp_dev->dev, "Disabling vmpck_id %d to prevent IV reuse.\n",
-		  vmpck_id);
-	memzero_explicit(snp_dev->vmpck, VMPCK_KEY_LEN);
-	snp_dev->vmpck = NULL;
+	u8 *key = snp_get_vmpck(snp_dev);

Check whether is_vmpck_empty before you disable?

Quoted 6 lines
drivers/virt/coco/sev-guest/sev-guest.c +4 -0
+
+	dev_alert(snp_dev->dev, "Disabling vmpck_id %u to prevent IV reuse.\n",
+		  snp_dev->vmpck_id);
+	memzero_explicit(key, VMPCK_KEY_LEN);
 }
 

drivers/virt/coco/sev-guest/sev-guest.c static bool is_vmpck_empty(struct snp_guest_dev *snp_dev) · line 120

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +2 -1
 static inline u64 __snp_get_msg_seqno(struct snp_guest_dev *snp_dev)
 {
+	u32 *os_area_msg_seqno = snp_get_os_area_msg_seqno(snp_dev);
 	u64 count;
 
 	lockdep_assert_held(&snp_cmd_mutex);
 
 	/* Read the current message sequence counter from secrets pages */
-	count = *snp_dev->os_area_msg_seqno;
+	count = *os_area_msg_seqno;

Why does that snp_get_os_area_msg_seqno() returns a pointer when you deref it here again?

A function which returns a sequence number should return that number
- not a pointer to it.

Quoted 8 lines

Which then makes that u32 *os_area_msg_seqno redundant and you can use the function directly.

IOW:

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
        return snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id;

This patch removes setting of layour page in snp_dev structure.

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
        if (!platform_data)
                return NULL;

        return *(&platform_data->layout->os_area.msg_seqno_0 + vmpck_id);
}
Quoted 10 lines

}

Simple.

Quoted 5 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -0
 	return count + 1;
 }
@@ -136,11 +145,13 @@ static u64 snp_get_msg_seqno(struct snp_guest_dev *snp_dev)
 
Quoted 13 lines

drivers/virt/coco/sev-guest/sev-guest.c static u64 snp_get_msg_seqno(struct snp_guest_dev *snp_dev) · line 154

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +3 -1
 static void snp_inc_msg_seqno(struct snp_guest_dev *snp_dev)
 {
+	u32 *os_area_msg_seqno = snp_get_os_area_msg_seqno(snp_dev);
+
 	/*
 	 * The counter is also incremented by the PSP, so increment it by 2
 	 * and save in secrets page.
 	 */
-	*snp_dev->os_area_msg_seqno += 2;
+	*os_area_msg_seqno += 2;

Yah, you have a getter but not a setter. You're setting it through the pointer.

I had a getter for getting the os_area_msg_seqno pointer, probably not a good function name.

Quoted 1 line

Do you see the imbalance in the APIs?

The msg_seqno should only be incremented by 2 (always), that was the reason to avoid a setter.

Quoted 19 lines
Quoted 6 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -0
 }
 
 static inline struct snp_guest_dev *to_snp_dev(struct file *file)
@@ -150,15 +161,22 @@ static inline struct snp_guest_dev *to_snp_dev(struct file *file)
 	return container_of(dev, struct snp_guest_dev, misc);

drivers/virt/coco/sev-guest/sev-guest.c static inline struct snp_guest_dev *to_snp_dev(struct file *file) · line 170

Quoted 10 lines
drivers/virt/coco/sev-guest/sev-guest.c +5 -1
 }
 
-static struct aesgcm_ctx *snp_init_crypto(u8 *key, size_t keylen)
+static struct aesgcm_ctx *snp_init_crypto(struct snp_guest_dev *snp_dev)
 {
 	struct aesgcm_ctx *ctx;
+	u8 *key;
+
+	if (snp_is_vmpck_empty(snp_dev)) {
+		pr_err("VM communication key VMPCK%u is null\n", vmpck_id);

"Empty/invalid VMPCK%u communication key"

Quoted 4 lines

or so.

In a pre-patch, fix all your user-visible strings to say "VMPCK"
- capitalized as it is an abbreviation.

Sure, will do.

Quoted 27 lines
Quoted 27 lines
drivers/virt/coco/sev-guest/sev-guest.c +5 -2
+		return NULL;
+	}
 
 	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL_ACCOUNT);
 	if (!ctx)
 		return NULL;
 
-	if (aesgcm_expandkey(ctx, key, keylen, AUTHTAG_LEN)) {
+	key = snp_get_vmpck(snp_dev);
+	if (aesgcm_expandkey(ctx, key, VMPCK_KEY_LEN, AUTHTAG_LEN)) {
 		pr_err("Crypto context initialization failed\n");
 		kfree(ctx);
 		return NULL;
@@ -590,7 +608,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long
 	mutex_lock(&snp_cmd_mutex);
 
 	/* Check if the VMPCK is not empty */
-	if (is_vmpck_empty(snp_dev)) {
+	if (snp_is_vmpck_empty(snp_dev)) {
 		dev_err_ratelimited(snp_dev->dev, "VMPCK is disabled\n");
 		mutex_unlock(&snp_cmd_mutex);
 		return -ENOTTY;
@@ -667,32 +685,14 @@ static const struct file_operations snp_guest_fops = {
 	.unlocked_ioctl = snp_guest_ioctl,
 };
 
Quoted 8 lines

drivers/virt/coco/sev-guest/sev-guest.c static const struct file_operations snp_guest_fops = { · line 691

Quoted 6 lines
drivers/virt/coco/sev-guest/sev-guest.c +3 -2
-static u8 *get_vmpck(int id, struct snp_secrets_page_layout *layout, u32 **seqno)
+static bool snp_assign_vmpck(struct snp_guest_dev *dev, unsigned int vmpck_id)
 {
-	u8 *key = NULL;
+	if (WARN_ON((vmpck_id + 1) > VMPCK_MAX_NUM))
+		return false;

So this will warn *and*, at the call site too. Let's tone that down.

Sure.

Quoted 17 lines
Quoted 17 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -15
-	switch (id) {
-	case 0:
-		*seqno = &layout->os_area.msg_seqno_0;
-		key = layout->vmpck0;
-		break;
-	case 1:
-		*seqno = &layout->os_area.msg_seqno_1;
-		key = layout->vmpck1;
-		break;
-	case 2:
-		*seqno = &layout->os_area.msg_seqno_2;
-		key = layout->vmpck2;
-		break;
-	case 3:
-		*seqno = &layout->os_area.msg_seqno_3;
Quoted 8 lines

drivers/virt/coco/sev-guest/sev-guest.c static const struct file_operations snp_guest_fops = { · line 693

Quoted 5 lines
drivers/virt/coco/sev-guest/sev-guest.c +0 -5
-		key = layout->vmpck3;
-		break;
-	default:
-		break;
-	}

Your commit message could explain why this is not needed, all of a sudden.

This was replaced by two independent APIs returning pointers to VMPCK and seqno pointer.

static inline u8 *snp_get_vmpck(unsigned int vmpck_id)

{
        if (!platform_data)
                return NULL;

        return platform_data->layout->vmpck0 + vmpck_id * VMPCK_KEY_LEN;
}

static inline u32 *snp_get_os_area_msg_seqno(unsigned int vmpck_id)

{
        if (!platform_data)
                return NULL;

        return &platform_data->layout->os_area.msg_seqno_0 + vmpck_id;
}

I will add more details.

Quoted 30 lines
Quoted 30 lines
drivers/virt/coco/sev-guest/sev-guest.c +6 -5
+	dev->vmpck_id = vmpck_id;
 
-	return key;
+	return true;
 }
 
 struct snp_msg_report_resp_hdr {
@@ -728,7 +728,7 @@ static int sev_report_new(struct tsm_report *report, void *data)
 	guard(mutex)(&snp_cmd_mutex);
 
 	/* Check if the VMPCK is not empty */
-	if (is_vmpck_empty(snp_dev)) {
+	if (snp_is_vmpck_empty(snp_dev)) {
 		dev_err_ratelimited(snp_dev->dev, "VMPCK is disabled\n");
 		return -ENOTTY;
 	}
@@ -848,21 +848,20 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 		goto e_unmap;
 
 	ret = -EINVAL;
-	snp_dev->vmpck = get_vmpck(vmpck_id, layout, &snp_dev->os_area_msg_seqno);
-	if (!snp_dev->vmpck) {
-		dev_err(dev, "invalid vmpck id %d\n", vmpck_id);
+	snp_dev->layout = layout;
+	if (!snp_assign_vmpck(snp_dev, vmpck_id)) {
+		dev_err(dev, "invalid vmpck id %u\n", vmpck_id);
 		goto e_unmap;
 	}
 
Quoted 7 lines

drivers/virt/coco/sev-guest/sev-guest.c static int __init sev_guest_probe(struct platform_device *pdev) · line 859

Quoted 5 lines
drivers/virt/coco/sev-guest/sev-guest.c +2 -2
 	/* Verify that VMPCK is not zero. */
-	if (is_vmpck_empty(snp_dev)) {
-		dev_err(dev, "vmpck id %d is null\n", vmpck_id);
+	if (snp_is_vmpck_empty(snp_dev)) {
+		dev_err(dev, "vmpck id %u is null\n", vmpck_id);

s!null!Invalid/Empty!

Okay

Quoted 23 lines
Quoted 23 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -2
 		goto e_unmap;
 	}
 
 	platform_set_drvdata(pdev, snp_dev);
 	snp_dev->dev = dev;
-	snp_dev->layout = layout;
 
 	/* Allocate the shared page used for the request and response message. */
 	snp_dev->request = alloc_shared_pages(dev, sizeof(struct snp_guest_msg));
@@ -878,7 +877,7 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 		goto e_free_response;
 
 	ret = -EIO;
-	snp_dev->ctx = snp_init_crypto(snp_dev->vmpck, VMPCK_KEY_LEN);
+	snp_dev->ctx = snp_init_crypto(snp_dev);
 	if (!snp_dev->ctx)
 		goto e_free_cert_data;
 
@@ -903,7 +902,7 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	if (ret)
 		goto e_free_ctx;
 
Quoted 5 lines

drivers/virt/coco/sev-guest/sev-guest.c static int __init sev_guest_probe(struct platform_device *pdev) · line 905

Quoted 2 lines
drivers/virt/coco/sev-guest/sev-guest.c +1 -1
-	dev_info(dev, "Initialized SEV guest driver (using vmpck_id %d)\n", vmpck_id);
+	dev_info(dev, "Initialized SEV guest driver (using vmpck_id %u)\n", vmpck_id);

Yet another spelling: "vmpck_id". Unify all those in a pre-patch pls because it looks stupid.

Sure

Quoted 3 lines

Thx.

Thanks for the review.

Regards
Nikunj

maintainer
#13 lore raw

Re: [PATCH v8 04/16] virt: sev-guest: Add vmpck_id to snp_guest_dev struct

Quoted 9 lines · Nikunj A. Dadhania
Quoted 9 lines

Why does that snp_get_os_area_msg_seqno() returns a pointer when you deref it here again?

A function which returns a sequence number should return that number
- not a pointer to it.

Which then makes that u32 *os_area_msg_seqno redundant and you can use the function directly.

Quoted 7 lines
Quoted 5 lines

IOW:

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
        return snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id;

This patch removes setting of layour page in snp_dev structure.

So?

Quoted 7 lines

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
        if (!platform_data)
                return NULL;

        return *(&platform_data->layout->os_area.msg_seqno_0 + vmpck_id);
}

What?!

This snp_get_os_area_msg_seqno() is a new function added by this patch.

Quoted 2 lines

I had a getter for getting the os_area_msg_seqno pointer, probably not a good function name.

Probably you need to go back to the drawing board and think about how this thing should look like.

Quoted 3 lines
Quoted 1 line

Do you see the imbalance in the APIs?

The msg_seqno should only be incremented by 2 (always), that was the reason to avoid a setter.

And what's wrong with the setter doing the incrementation so that callers can't even get it wrong?

It sounds to me like you should redesign this sequence number handling in a *separate* patch.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#14 lore raw

Re: [PATCH v8 04/16] virt: sev-guest: Add vmpck_id to snp_guest_dev struct

Quoted 12 lines · Borislav Petkov
Quoted 11 lines · Nikunj A. Dadhania
Quoted 11 lines

Why does that snp_get_os_area_msg_seqno() returns a pointer when you deref it here again?

A function which returns a sequence number should return that number
- not a pointer to it.

Which then makes that u32 *os_area_msg_seqno redundant and you can use the function directly.

IOW:

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
        return snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id;

This patch removes setting of layour page in snp_dev structure.

So?

* Instead of using snp_dev->layout, we will need to access it using platform_data->layout structure.
* Below will give incorrect value of sequence number, it will get VMPCK_0's sequence number and will add vmpck_id to that. Will work by fluke for VMPCK=0, but will fail for all other keys.

return snp_dev->layout->os_area.msg_seqno_0 + snp_dev->vmpck_id;

struct secrets_os_area {
...

        u32 msg_seqno_0;
        u32 msg_seqno_1;
        u32 msg_seqno_2;
        u32 msg_seqno_3;

...
}

* I am using vmpck_id to index to correct msg_seqno_*

Changing this to

struct secrets_os_area {
...

        u32 msg_seqno[VMPCK_MAX_NUM];

...
}

Quoted 10 lines
Quoted 8 lines

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
        if (!platform_data)
                return NULL;

        return *(&platform_data->layout->os_area.msg_seqno_0 + vmpck_id);
}

What?!

I can change the secrets_os_area like below to simplify things:

struct secrets_os_area {
...

        u32 msg_seqno[VMPCK_MAX_NUM];

...
}

static inline u32 snp_get_os_area_msg_seqno(struct snp_guest_dev *snp_dev)

{
         if (!platform_data)
                 return NULL;

         return platform_data->layout->os_area.msg_seqno[snp_dev->vmpck_id];
}
Quoted 9 lines

This snp_get_os_area_msg_seqno() is a new function added by this patch.

Quoted 2 lines

I had a getter for getting the os_area_msg_seqno pointer, probably not a good function name.

Probably you need to go back to the drawing board and think about how this thing should look like.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Do you see the imbalance in the APIs?

The msg_seqno should only be incremented by 2 (always), that was the reason to avoid a setter.

And what's wrong with the setter doing the incrementation so that callers can't even get it wrong?

Are you suggesting that setter should always increment by 2?

static inline u32 snp_set_os_area_msg_seqno(struct snp_guest_dev *snp_dev) {
...

	os_area.msg_seqno[snp_dev->vmpck_id] += 2;

...

}

Quoted 3 lines

It sounds to me like you should redesign this sequence number handling in a *separate* patch.

Sure, let me rethink and will post it as separate patch.

Regards
Nikunj

#15 lore raw

PATCH v8 5/16 x86/sev: Cache the secrets page address

In reply to Nikunj A Dadhania

Save the secrets page address during snp_init() from the CC blob. Use secrets_pa instead of calling get_secrets_page() that remaps the CC blob for getting the secrets page every time.

---
 arch/x86/kernel/sev.c | 54 +++++++++++++++----------------------------
 1 file changed, 18 insertions(+), 36 deletions(-)
arch/x86/kernel/sev.c +18 -36
index 479b68e00a54..eda43c35a9f2 100644
@@ -71,6 +71,9 @@ static struct ghcb *boot_ghcb __section(".data");
 /* Bitmap of SEV features supported by the hypervisor */
 static u64 sev_hv_features __ro_after_init;
 
+/* Secrets page physical address from the CC blob */
+static u64 secrets_pa __ro_after_init;
+
 /* #VC handler runtime per-CPU data */
 struct sev_es_runtime_data {
 	struct ghcb ghcb_page;
@@ -597,45 +600,16 @@ void noinstr __sev_es_nmi_complete(void)
 	__sev_put_ghcb(&state);
 }
 
-static u64 __init get_secrets_page(void)
-{
-	u64 pa_data = boot_params.cc_blob_address;
-	struct cc_blob_sev_info info;
-	void *map;
-
-	/*
-	 * The CC blob contains the address of the secrets page, check if the
-	 * blob is present.
-	 */
-	if (!pa_data)
-		return 0;
-
-	map = early_memremap(pa_data, sizeof(info));
-	if (!map) {
-		pr_err("Unable to locate SNP secrets page: failed to map the Confidential Computing blob.\n");
-		return 0;
-	}
-	memcpy(&info, map, sizeof(info));
-	early_memunmap(map, sizeof(info));
-
-	/* smoke-test the secrets page passed */
-	if (!info.secrets_phys || info.secrets_len != PAGE_SIZE)
-		return 0;
-
-	return info.secrets_phys;
-}
-
 static u64 __init get_snp_jump_table_addr(void)
 {
 	struct snp_secrets_page_layout *layout;
 	void __iomem *mem;
-	u64 pa, addr;
+	u64 addr;
 
-	pa = get_secrets_page();
-	if (!pa)
+	if (!secrets_pa)
 		return 0;
 
-	mem = ioremap_encrypted(pa, PAGE_SIZE);
+	mem = ioremap_encrypted(secrets_pa, PAGE_SIZE);
 	if (!mem) {
 		pr_err("Unable to locate AP jump table address: failed to map the SNP secrets page.\n");
 		return 0;
@@ -2088,6 +2062,12 @@ static __init struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp)
 	return cc_info;
 }
 
+static void __init set_secrets_pa(const struct cc_blob_sev_info *cc_info)
+{
+	if (cc_info && cc_info->secrets_phys && cc_info->secrets_len == PAGE_SIZE)
+		secrets_pa = cc_info->secrets_phys;
+}
+
 bool __init snp_init(struct boot_params *bp)
 {
 	struct cc_blob_sev_info *cc_info;
@@ -2099,6 +2079,8 @@ bool __init snp_init(struct boot_params *bp)
 	if (!cc_info)
 		return false;
 
+	set_secrets_pa(cc_info);
+
 	setup_cpuid_table(cc_info);
 
 	/*
@@ -2246,16 +2228,16 @@ static struct platform_device sev_guest_device = {
 static int __init snp_init_platform_device(void)
 {
 	struct sev_guest_platform_data data;
-	u64 gpa;
 
 	if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP))
 		return -ENODEV;
 
-	gpa = get_secrets_page();
-	if (!gpa)
+	if (!secrets_pa) {
+		pr_err("SNP secrets page not found\n");
 		return -ENODEV;
+	}
 
-	data.secrets_gpa = gpa;
+	data.secrets_gpa = secrets_pa;
 	if (platform_device_add_data(&sev_guest_device, &data, sizeof(data)))
 		return -ENODEV;
 
Signature · 1 line
2.34.1
maintainer
#16 lore raw

Re: [PATCH v8 05/16] x86/sev: Cache the secrets page address

arch/x86/kernel/sev.c static struct ghcb *boot_ghcb __section(".data"); · line 75

Quoted 2 lines · Nikunj A Dadhania
arch/x86/kernel/sev.c +2 -0
+/* Secrets page physical address from the CC blob */
+static u64 secrets_pa __ro_after_init;

Since you're going to use this during runtime (are you?), why don't you put in here the result of:

	ioremap_encrypted(secrets_pa, PAGE_SIZE);

so that you can have it ready and not even have to ioremap each time?

And then you iounmap on driver teardown.

arch/x86/kernel/sev.c static __init struct cc_blob_sev_info *find_cc_blob(struct boot_params *bp) · line 2069

Quoted 5 lines
arch/x86/kernel/sev.c +5 -0
+static void __init set_secrets_pa(const struct cc_blob_sev_info *cc_info)
+{
+	if (cc_info && cc_info->secrets_phys && cc_info->secrets_len == PAGE_SIZE)
+		secrets_pa = cc_info->secrets_phys;
+}

Why is this a separate function if it is called only once and it is a trivial function at that?

Also, can the driver continue without secrets page?

If not, then you need to unwind.

Quoted 17 lines

bool __init snp_init(struct boot_params *bp)
{

arch/x86/kernel/sev.c +2 -1
 	struct cc_blob_sev_info *cc_info;
@@ -2099,6 +2079,8 @@ bool __init snp_init(struct boot_params *bp)
 	if (!cc_info)
 		return false;
 
+	set_secrets_pa(cc_info);
+
 	setup_cpuid_table(cc_info);
 
 	/*
@@ -2246,16 +2228,16 @@ static struct platform_device sev_guest_device = {
 static int __init snp_init_platform_device(void)
 {
 	struct sev_guest_platform_data data;
-	u64 gpa;

arch/x86/kernel/sev.c static struct platform_device sev_guest_device = { · line 2238

Quoted 10 lines
arch/x86/kernel/sev.c +3 -2
 
 	if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP))
 		return -ENODEV;
 
-	gpa = get_secrets_page();
-	if (!gpa)
+	if (!secrets_pa) {
+		pr_err("SNP secrets page not found\n");
 		return -ENODEV;
+	}

Yeah, no, you need to error out in snp_init() and not drag it around to snp_init_platform_device().

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#17 lore raw

Re: [PATCH v8 05/16] x86/sev: Cache the secrets page address

Quoted 5 lines · Borislav Petkov

Review

Quoted 2 lines · Nikunj A Dadhania
(quoted diff) +2 -0
+/* Secrets page physical address from the CC blob */
+static u64 secrets_pa __ro_after_init;

Since you're going to use this during runtime (are you?),

Yes, this is used during runtime, during initial boot will be used by Secure TSC and later by sev-guest driver.

Quoted 5 lines

why don't you put in here the result of:

	ioremap_encrypted(secrets_pa, PAGE_SIZE);

so that you can have it ready and not even have to ioremap each time?

Yes, that is a good idea. If I map in sev.c, what is the right place to iounmap ? Is it safe to keep it mapped until reboot/shutdown ?

Quoted 10 lines

And then you iounmap on driver teardown.

Review

Quoted 5 lines
(quoted diff) +5 -0
+static void __init set_secrets_pa(const struct cc_blob_sev_info *cc_info)
+{
+	if (cc_info && cc_info->secrets_phys && cc_info->secrets_len == PAGE_SIZE)
+		secrets_pa = cc_info->secrets_phys;
+}

Why is this a separate function if it is called only once and it is a trivial function at that?

Sure, I will change it.

Quoted 2 lines

Also, can the driver continue without secrets page?

No.

Quoted 2 lines

If not, then you need to unwind.

By unwind, do you mean unmapping in the driver?

Quoted 21 lines
Quoted 21 lines

bool __init snp_init(struct boot_params *bp)
{

(quoted diff) +2 -1
 	struct cc_blob_sev_info *cc_info;
@@ -2099,6 +2079,8 @@ bool __init snp_init(struct boot_params *bp)
 	if (!cc_info)
 		return false;
 
+	set_secrets_pa(cc_info);
+
 	setup_cpuid_table(cc_info);
 
 	/*
@@ -2246,16 +2228,16 @@ static struct platform_device sev_guest_device = {
 static int __init snp_init_platform_device(void)
 {
 	struct sev_guest_platform_data data;
-	u64 gpa;
 
 	if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP))
 		return -ENODEV;
 
Quoted 9 lines

Review static struct platform_device sev_guest_device = { · line 2238

Quoted 6 lines
(quoted diff) +3 -2
-	gpa = get_secrets_page();
-	if (!gpa)
+	if (!secrets_pa) {
+		pr_err("SNP secrets page not found\n");
 		return -ENODEV;
+	}

Yeah, no, you need to error out in snp_init() and not drag it around to snp_init_platform_device().

snp_init() is called from sme_enable(), and does not handle failure from snp_init()

How about the below diff?

arch/x86/include/asm/sev.h +1 -1
index e9925df21010..5e052f972138 100644
@@ -114,7 +114,7 @@ struct snp_req_data {
 };
 
 struct sev_guest_platform_data {
-	u64 secrets_gpa;
+	void *secrets_page;
 };
 
 /*
arch/x86/kernel/sev.c +16 -48

98 lines not on this page · raw message

drivers/virt/coco/sev-guest/sev-guest.c +6 -14
index dbc04229f7ac..4cef4e108130 100644
@@ -820,12 +820,10 @@ static void unregister_sev_tsm(void *data)
 
 static int __init sev_guest_probe(struct platform_device *pdev)
 {
-	struct snp_secrets_page_layout *layout;
 	struct sev_guest_platform_data *data;
 	struct device *dev = &pdev->dev;
 	struct snp_guest_dev *snp_dev;
 	struct miscdevice *misc;
-	void __iomem *mapping;
 	int ret;
 
 	if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP))
@@ -835,28 +833,24 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 		return -ENODEV;
 
 	data = (struct sev_guest_platform_data *)dev->platform_data;
-	mapping = ioremap_encrypted(data->secrets_gpa, PAGE_SIZE);
-	if (!mapping)
+	if (!data->secrets_page)
 		return -ENODEV;
 
-	layout = (__force void *)mapping;
-
-	ret = -ENOMEM;
 	snp_dev = devm_kzalloc(&pdev->dev, sizeof(struct snp_guest_dev), GFP_KERNEL);
 	if (!snp_dev)
-		goto e_unmap;
+		return -ENOMEM;
 
 	ret = -EINVAL;
-	snp_dev->layout = layout;
+	snp_dev->layout = (__force struct snp_secrets_page_layout *)data->secrets_page;
 	if (!snp_assign_vmpck(snp_dev, vmpck_id)) {
 		dev_err(dev, "invalid vmpck id %u\n", vmpck_id);
-		goto e_unmap;
+		return ret;
 	}
 
 	/* Verify that VMPCK is not zero. */
 	if (snp_is_vmpck_empty(snp_dev)) {
 		dev_err(dev, "vmpck id %u is null\n", vmpck_id);
-		goto e_unmap;
+		return ret;
 	}
 
 	platform_set_drvdata(pdev, snp_dev);
@@ -865,7 +859,7 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	/* Allocate secret request and response message for double buffering */
 	snp_dev->secret_request = kzalloc(SNP_GUEST_MSG_SIZE, GFP_KERNEL);
 	if (!snp_dev->secret_request)
-		goto e_unmap;
+		return ret;
 
 	snp_dev->secret_response = kzalloc(SNP_GUEST_MSG_SIZE, GFP_KERNEL);
 	if (!snp_dev->secret_response)
@@ -925,8 +919,6 @@ static int __init sev_guest_probe(struct platform_device *pdev)
 	kfree(snp_dev->secret_response);
 e_free_secret_req:
 	kfree(snp_dev->secret_request);
-e_unmap:
-	iounmap(mapping);
 	return ret;
 }
 

Regards
Nikunj

#18 lore raw

Re: [PATCH v8 05/16] x86/sev: Cache the secrets page address

Quoted 17 lines · Nikunj A. Dadhania
Quoted 5 lines · Borislav Petkov

Review

Quoted 2 lines · Nikunj A Dadhania
(quoted diff) +2 -0
+/* Secrets page physical address from the CC blob */
+static u64 secrets_pa __ro_after_init;

Since you're going to use this during runtime (are you?),

Yes, this is used during runtime, during initial boot will be used by Secure TSC and later by sev-guest driver.

Quoted 5 lines

why don't you put in here the result of:

	ioremap_encrypted(secrets_pa, PAGE_SIZE);

so that you can have it ready and not even have to ioremap each time?

arch/x86/kernel/sev.c bool __init snp_init(struct boot_params *bp) · line 2087

Quoted 5 lines
arch/x86/kernel/sev.c +2 -0
@@ -2118,6 +2083,14 @@ bool __init snp_init(struct boot_params *bp)
 	if (!cc_info)
 		return false;
 
+	if (cc_info->secrets_phys && cc_info->secrets_len == PAGE_SIZE) {
+		secrets_page = ioremap_encrypted(cc_info->secrets_phys, PAGE_SIZE);

ioremap_encrypted() does not work this early, snp guest boot fails, will debug further.

Regards,
Nikunj

#19 lore raw

PATCH v8 6/16 virt: sev-guest: Move SNP Guest command mutex

In reply to Nikunj A Dadhania

SNP command mutex is used to serialize the shared buffer access, command handling and message sequence number races. Move the SNP guest command mutex out of the sev guest driver and provide accessors to sev-guest driver. Remove multiple lockdep check in sev-guest driver, next patch adds a single lockdep check in snp_send_guest_request().

---
 arch/x86/include/asm/sev.h              |  4 ++++
 arch/x86/kernel/sev.c                   | 15 +++++++++++++++
 drivers/virt/coco/sev-guest/sev-guest.c | 23 +++++++----------------
 3 files changed, 26 insertions(+), 16 deletions(-)
arch/x86/include/asm/sev.h +4 -0
index e4f52a606487..8578b33d8fc4 100644
@@ -295,6 +295,8 @@ void snp_accept_memory(phys_addr_t start, phys_addr_t end);
 u64 snp_get_unsupported_features(u64 status);
 u64 sev_get_status(void);
 void kdump_sev_callback(void);
+void snp_guest_cmd_lock(void);
+void snp_guest_cmd_unlock(void);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -325,6 +327,8 @@ static inline void snp_accept_memory(phys_addr_t start, phys_addr_t end) { }
 static inline u64 snp_get_unsupported_features(u64 status) { return 0; }
 static inline u64 sev_get_status(void) { return 0; }
 static inline void kdump_sev_callback(void) { }
+static inline void snp_guest_cmd_lock(void) { }
+static inline void snp_guest_cmd_unlock(void) { }
 #endif
 
 #ifdef CONFIG_KVM_AMD_SEV
arch/x86/kernel/sev.c +15 -0
index eda43c35a9f2..bc4a705d989c 100644
@@ -940,6 +940,21 @@ static void snp_cleanup_vmsa(struct sev_es_save_area *vmsa)
 		free_page((unsigned long)vmsa);
 }
 
+/*  SNP Guest command mutex to serialize the shared buffer access and command handling. */
+static DEFINE_MUTEX(snp_guest_cmd_mutex);
+
+void snp_guest_cmd_lock(void)
+{
+	mutex_lock(&snp_guest_cmd_mutex);
+}
+EXPORT_SYMBOL_GPL(snp_guest_cmd_lock);
+
+void snp_guest_cmd_unlock(void)
+{
+	mutex_unlock(&snp_guest_cmd_mutex);
+}
+EXPORT_SYMBOL_GPL(snp_guest_cmd_unlock);
+
 static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip)
 {
 	struct sev_es_save_area *cur_vmsa, *vmsa;
drivers/virt/coco/sev-guest/sev-guest.c +7 -16
index 646eb215f3c7..ba9ffaee647c 100644
@@ -62,9 +62,6 @@ static u32 vmpck_id;
 module_param(vmpck_id, uint, 0444);
 MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP.");
 
-/* Mutex to serialize the shared buffer access and command handling. */
-static DEFINE_MUTEX(snp_cmd_mutex);
-
 static inline u8 *snp_get_vmpck(struct snp_guest_dev *snp_dev)
 {
 	return snp_dev->layout->vmpck0 + snp_dev->vmpck_id * VMPCK_KEY_LEN;
@@ -114,8 +111,6 @@ static inline u64 __snp_get_msg_seqno(struct snp_guest_dev *snp_dev)
 	u32 *os_area_msg_seqno = snp_get_os_area_msg_seqno(snp_dev);
 	u64 count;
 
-	lockdep_assert_held(&snp_cmd_mutex);
-
 	/* Read the current message sequence counter from secrets pages */
 	count = *os_area_msg_seqno;
 
@@ -408,8 +403,6 @@ static int get_report(struct snp_guest_dev *snp_dev, struct snp_guest_request_io
 	struct snp_report_resp *report_resp;
 	int rc, resp_len;
 
-	lockdep_assert_held(&snp_cmd_mutex);
-
 	if (!arg->req_data || !arg->resp_data)
 		return -EINVAL;
 
@@ -456,8 +449,6 @@ static int get_derived_key(struct snp_guest_dev *snp_dev, struct snp_guest_reque
 	/* Response data is 64 bytes and max authsize for GCM is 16 bytes. */
 	u8 buf[64 + 16];
 
-	lockdep_assert_held(&snp_cmd_mutex);
-
 	if (!arg->req_data || !arg->resp_data)
 		return -EINVAL;
 
@@ -508,8 +499,6 @@ static int get_ext_report(struct snp_guest_dev *snp_dev, struct snp_guest_reques
 	sockptr_t certs_address;
 	int ret, resp_len;
 
-	lockdep_assert_held(&snp_cmd_mutex);
-
 	if (sockptr_is_null(io->req_data) || sockptr_is_null(io->resp_data))
 		return -EINVAL;
 
@@ -605,12 +594,12 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long
 	if (!input.msg_version)
 		return -EINVAL;
 
-	mutex_lock(&snp_cmd_mutex);
+	snp_guest_cmd_lock();
 
 	/* Check if the VMPCK is not empty */
 	if (snp_is_vmpck_empty(snp_dev)) {
 		dev_err_ratelimited(snp_dev->dev, "VMPCK is disabled\n");
-		mutex_unlock(&snp_cmd_mutex);
+		snp_guest_cmd_unlock();
 		return -ENOTTY;
 	}
 
@@ -635,7 +624,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long
 		break;
 	}
 
-	mutex_unlock(&snp_cmd_mutex);
+	snp_guest_cmd_unlock();
 
 	if (input.exitinfo2 && copy_to_user(argp, &input, sizeof(input)))
 		return -EFAULT;
@@ -725,14 +714,14 @@ static int sev_report_new(struct tsm_report *report, void *data)
 	if (!buf)
 		return -ENOMEM;
 
-	guard(mutex)(&snp_cmd_mutex);
-
 	/* Check if the VMPCK is not empty */
 	if (snp_is_vmpck_empty(snp_dev)) {
 		dev_err_ratelimited(snp_dev->dev, "VMPCK is disabled\n");
 		return -ENOTTY;
 	}
 
+	snp_guest_cmd_lock();
+
 	cert_table = buf + report_size;
 	struct snp_ext_report_req ext_req = {
 		.data = { .vmpl = desc->privlevel },
@@ -753,6 +742,8 @@ static int sev_report_new(struct tsm_report *report, void *data)
 	};
 
 	ret = get_ext_report(snp_dev, &input, &io);
+	snp_guest_cmd_unlock();
+
 	if (ret)
 		return ret;
 
Signature · 1 line
2.34.1
maintainer
#20 lore raw

Re: [PATCH v8 06/16] virt: sev-guest: Move SNP Guest command mutex

Quoted 3 lines · Nikunj A Dadhania

SNP command mutex is used to serialize the shared buffer access, command handling and message sequence number races. Move the SNP guest command mutex out of the sev guest driver and provide accessors to sev-guest

And why in the hell are we doing this?

Always, *ALWAYS* make sure a patch's commit message answers *why* a change is done. This doesn't explain why so I'm reading "just because" and "just because" doesn't fly.

Quoted 2 lines

driver. Remove multiple lockdep check in sev-guest driver, next patch adds a single lockdep check in snp_send_guest_request().

The concept of "next patch" is meaningless once the patch is in git.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#21 lore raw

Re: [PATCH v8 06/16] virt: sev-guest: Move SNP Guest command mutex

Quoted 6 lines · Borislav Petkov
Quoted 3 lines · Nikunj A Dadhania

SNP command mutex is used to serialize the shared buffer access, command handling and message sequence number races. Move the SNP guest command mutex out of the sev guest driver and provide accessors to sev-guest

And why in the hell are we doing this?

SNP guest messaging will be moving as part of sev.c, and Secure TSC code will use this mutex.

Quoted 3 lines

Always, *ALWAYS* make sure a patch's commit message answers *why* a change is done. This doesn't explain why so I'm reading "just because" and "just because" doesn't fly.

Sure, will change.

Quoted 5 lines
Quoted 3 lines

driver. Remove multiple lockdep check in sev-guest driver, next patch adds a single lockdep check in snp_send_guest_request().

The concept of "next patch" is meaningless once the patch is in git.

Sure. As direct access to the mutex was not available now, I had removed lockdep check here and documented that lockdep gets added at later point.

Regards
Nikunj

maintainer
#22 lore raw

Re: [PATCH v8 06/16] virt: sev-guest: Move SNP Guest command mutex

Quoted 2 lines · Nikunj A. Dadhania

SNP guest messaging will be moving as part of sev.c, and Secure TSC code will use this mutex.

No, this is all backwards.

You have a *static* function in sev-guest - snp_guest_ioctl- which takes an exported lock - snp_guest_cmd_lock - in order to synchronize with other callers which are only in that same sev-guest driver.

Why do you even need the guest messaging in sev.c?

I guess this: "Many of the required functions are implemented in the sev-guest driver and therefore not available at early boot."

But then your API is misdesigned: the lock should be private to sev.c and none of the callers should pay attention to grabbing it - the callers simply call the functions and underneath the locking works automatically for them - they don't care. Just like any other shared resource, users see only the API they call and the actual synchronization is done behind the scenes.

Sounds like you need to go back to the drawing board and think how this thing should look like.

And when you have it, make sure to explain the commit messages *why* it is done this way.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#23 lore raw

Re: [PATCH v8 06/16] virt: sev-guest: Move SNP Guest command mutex

Quoted 6 lines · Borislav Petkov
Quoted 2 lines · Nikunj A. Dadhania

SNP guest messaging will be moving as part of sev.c, and Secure TSC code will use this mutex.

No, this is all backwards.

Quoted 8 lines

You have a *static* function in sev-guest - snp_guest_ioctl- which takes an exported lock - snp_guest_cmd_lock - in order to synchronize with other callers which are only in that same sev-guest driver.

Why do you even need the guest messaging in sev.c?

I guess this: "Many of the required functions are implemented in the sev-guest driver and therefore not available at early boot."

Yes.

Quoted 10 lines

But then your API is misdesigned: the lock should be private to sev.c and none of the callers should pay attention to grabbing it - the callers simply call the functions and underneath the locking works automatically for them - they don't care. Just like any other shared resource, users see only the API they call and the actual synchronization is done behind the scenes.

Sounds like you need to go back to the drawing board and think how this thing should look like.

Something like below ?

snp_guest_ioctl()
-> get_report()/get_derived_key()/get_ext_report()
-> snp_send_guest_request()

       snp_guest_cmd_lock();
       ...
       snp_guest_cmd_lock();

With this the cmd_lock will be private to sev.c and lock/unlock function doesn't need to be exported.

Regards
Nikunj

maintainer
#24 lore raw

Re: [PATCH v8 06/16] virt: sev-guest: Move SNP Guest command mutex

Quoted 11 lines · Nikunj A. Dadhania

Something like below ?

snp_guest_ioctl()
-> get_report()/get_derived_key()/get_ext_report()
-> snp_send_guest_request()

       snp_guest_cmd_lock();
       ...
       snp_guest_cmd_lock();

With this the cmd_lock will be private to sev.c and lock/unlock function doesn't need to be exported.

Yes, something like that.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#25 lore raw

PATCH v8 7/16 x86/sev: Move and reorganize sev guest request api

In reply to Nikunj A Dadhania

For enabling Secure TSC, SEV-SNP guests need to communicate with the AMD Security Processor early during boot. Many of the required functions are implemented in the sev-guest driver and therefore not available at early boot. Move the required functions and provide API to the sev guest driver for sending guest message and vmpck routines.

As there is no external caller for snp_issue_guest_request() anymore, make it static and drop the prototype from the header.

---
 arch/x86/Kconfig                        |   1 +
 arch/x86/include/asm/sev.h              | 111 +++++-
 arch/x86/kernel/sev.c                   | 459 ++++++++++++++++++++++-
 drivers/virt/coco/sev-guest/Kconfig     |   1 -
 drivers/virt/coco/sev-guest/sev-guest.c | 476 ++----------------------
 5 files changed, 561 insertions(+), 487 deletions(-)
arch/x86/Kconfig +1 -0
index 58d3593bc4f2..200fcf68b1e4 100644
@@ -1534,6 +1534,7 @@ config AMD_MEM_ENCRYPT
 	select ARCH_HAS_CC_PLATFORM
 	select X86_MEM_ENCRYPT
 	select UNACCEPTED_MEMORY
+	select CRYPTO_LIB_AESGCM
 	help
 	  Say yes to enable support for the encryption of system memory.
 	  This requires an AMD processor that supports Secure Memory
arch/x86/include/asm/sev.h +93 -18

169 lines not on this page · raw message

arch/x86/kernel/sev.c +445 -14

489 lines not on this page · raw message

drivers/virt/coco/sev-guest/Kconfig +0 -1
index 0b772bd921d8..a6405ab6c2c3 100644
@@ -2,7 +2,6 @@ config SEV_GUEST
 	tristate "AMD SEV Guest driver"
 	default m
 	depends on AMD_MEM_ENCRYPT
-	select CRYPTO_LIB_AESGCM
 	select TSM_REPORTS
 	help
 	  SEV-SNP firmware provides the guest a mechanism to communicate with
drivers/virt/coco/sev-guest/sev-guest.c +22 -454

566 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#26 lore raw

Re: [PATCH v8 07/16] x86/sev: Move and reorganize sev guest request api

Quoted 1 line · Nikunj A Dadhania

Subject: Re: [PATCH v8 07/16] x86/sev: Move and reorganize sev guest request api

s/api/API/g

Please check your whole patchset for proper naming/abbreviations spelling, etc.

Another one: s/sev/SEV/g and so on. You should know the drill by now.

Quoted 6 lines

For enabling Secure TSC, SEV-SNP guests need to communicate with the AMD Security Processor early during boot. Many of the required functions are implemented in the sev-guest driver and therefore not available at early boot. Move the required functions and provide API to the sev guest driver for sending guest message and vmpck routines.

Patches which move and reorganize must always be split: first patch(es):
you do *purely* *mechanical* move without any code changes. Then you do the code changes ontop so that a reviewer can have a chance of seeing what you're doing.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#27 lore raw

Re: [PATCH v8 07/16] x86/sev: Move and reorganize sev guest request api

Quoted 9 lines · Borislav Petkov
Quoted 1 line · Nikunj A Dadhania

Subject: Re: [PATCH v8 07/16] x86/sev: Move and reorganize sev guest request api

s/api/API/g

Please check your whole patchset for proper naming/abbreviations spelling, etc.

Another one: s/sev/SEV/g and so on. You should know the drill by now.

Sure.

Quoted 9 lines
Quoted 6 lines

For enabling Secure TSC, SEV-SNP guests need to communicate with the AMD Security Processor early during boot. Many of the required functions are implemented in the sev-guest driver and therefore not available at early boot. Move the required functions and provide API to the sev guest driver for sending guest message and vmpck routines.

Patches which move and reorganize must always be split: first patch(es):
you do *purely* *mechanical* move without any code changes.

Yes, I had tried that compilation/guest boot does not break at this stage. That was the reason for intermixing movement and code change.

Let me give a second stab at this and I will try just to make sure compilation does not break.

Quoted 2 lines

Then you do the code changes ontop so that a reviewer can have a chance of seeing what you're doing.

Sure

Regards,
Nikunj

maintainer
#28 lore raw

Re: [PATCH v8 07/16] x86/sev: Move and reorganize sev guest request api

Quoted 5 lines · Nikunj A. Dadhania

Yes, I had tried that compilation/guest boot does not break at this stage. That was the reason for intermixing movement and code change.

Let me give a second stab at this and I will try just to make sure compilation does not break.

Yes, you can also do preparatory changes which get removed later, if that helps.

It is perfectly fine to have a couple more patches preparing and doing the move which are trivial to review and verify vs one combo patch which makes you stare at it a long time and you're still not sure it doesn't change something...

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#29 lore raw

PATCH v8 8/16 x86/mm: Add generic guest initialization hook

In reply to Nikunj A Dadhania

Add generic enc_init guest hook for performing any type of initialization that is vendor specific. Generic enc_init hook can be used for early guest feature initialization before secondary processors are up.

---
 arch/x86/include/asm/x86_init.h | 2 ++
 arch/x86/kernel/x86_init.c      | 2 ++
 arch/x86/mm/mem_encrypt.c       | 2 ++
 3 files changed, 6 insertions(+)
arch/x86/include/asm/x86_init.h +2 -0
index c878616a18b8..8095553e14a7 100644
@@ -148,12 +148,14 @@ struct x86_init_acpi {
  * @enc_status_change_finish	Notify HV after the encryption status of a range is changed
  * @enc_tlb_flush_required	Returns true if a TLB flush is needed before changing page encryption status
  * @enc_cache_flush_required	Returns true if a cache flush is needed before changing page encryption status
+ * @enc_init			Prepare and initialize encryption features
  */
 struct x86_guest {
 	bool (*enc_status_change_prepare)(unsigned long vaddr, int npages, bool enc);
 	bool (*enc_status_change_finish)(unsigned long vaddr, int npages, bool enc);
 	bool (*enc_tlb_flush_required)(bool enc);
 	bool (*enc_cache_flush_required)(void);
+	void (*enc_init)(void);
 };
 
 /**
arch/x86/kernel/x86_init.c +2 -0
index a37ebd3b4773..a07985a96ca5 100644
@@ -136,6 +136,7 @@ static bool enc_status_change_finish_noop(unsigned long vaddr, int npages, bool
 static bool enc_tlb_flush_required_noop(bool enc) { return false; }
 static bool enc_cache_flush_required_noop(void) { return false; }
 static bool is_private_mmio_noop(u64 addr) {return false; }
+static void enc_init_noop(void) { }
 
 struct x86_platform_ops x86_platform __ro_after_init = {
 	.calibrate_cpu			= native_calibrate_cpu_early,
@@ -158,6 +159,7 @@ struct x86_platform_ops x86_platform __ro_after_init = {
 		.enc_status_change_finish  = enc_status_change_finish_noop,
 		.enc_tlb_flush_required	   = enc_tlb_flush_required_noop,
 		.enc_cache_flush_required  = enc_cache_flush_required_noop,
+		.enc_init		   = enc_init_noop,
 	},
 };
 
arch/x86/mm/mem_encrypt.c +2 -0
index d035bce3a2b0..68aa06852466 100644
@@ -89,6 +89,8 @@ void __init mem_encrypt_init(void)
 	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
 	swiotlb_update_mem_attributes();
 
+	x86_platform.guest.enc_init();
+
 	print_mem_encrypt_feature_info();
 }
 
Signature · 1 line
2.34.1
maintainer
#30 lore raw

Re: [PATCH v8 08/16] x86/mm: Add generic guest initialization hook

arch/x86/mm/mem_encrypt.c void __init mem_encrypt_init(void) · line 94

Quoted 11 lines · Nikunj A Dadhania
arch/x86/mm/mem_encrypt.c +2 -0
index d035bce3a2b0..68aa06852466 100644
@@ -89,6 +89,8 @@ void __init mem_encrypt_init(void)
 	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
 	swiotlb_update_mem_attributes();
 
+	x86_platform.guest.enc_init();
+
 	print_mem_encrypt_feature_info();

Why all this hoopla if all you need is to call it once in mem_encrypt.c?

IOW, you can simply call snp_secure_tsc_prepare() there, no?

Those function pointers are to be used in generic code in order to hide all the platform-specific hackery but mem_encrypt.c is not really generic code, I'd say...

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#31 lore raw

Re: [PATCH v8 08/16] x86/mm: Add generic guest initialization hook

Quoted 9 lines · Borislav Petkov
Quoted 8 lines · Nikunj A Dadhania
arch/x86/mm/mem_encrypt.c +0 -0
index d035bce3a2b0..68aa06852466 100644
@@ -89,6 +89,8 @@ void __init mem_encrypt_init(void)
 	/* Call into SWIOTLB to update the SWIOTLB DMA buffers */
 	swiotlb_update_mem_attributes();
 
Quoted 5 lines

arch/x86/mm/mem_encrypt.c void __init mem_encrypt_init(void) · line 94

Quoted 3 lines
arch/x86/mm/mem_encrypt.c +2 -0
+	x86_platform.guest.enc_init();
+
 	print_mem_encrypt_feature_info();

Why all this hoopla if all you need is to call it once in mem_encrypt.c?

This was added thinking in mind that any SNP/TDX init can happen in this hook.

Quoted 1 line

IOW, you can simply call snp_secure_tsc_prepare() there, no?

Yes, that is very simple, will drop this change.

Quoted 4 lines

Those function pointers are to be used in generic code in order to hide all the platform-specific hackery but mem_encrypt.c is not really generic code, I'd say...

Sure.

Regards
Nikunj

#32 lore raw

PATCH v8 9/16 x86/cpufeatures: Add synthetic Secure TSC bit

In reply to Nikunj A Dadhania

Add support for the synthetic CPUID flag which indicates that the SNP guest is running with secure tsc enabled (MSR_AMD64_SEV Bit 11 - SecureTsc_Enabled) . This flag is there so that this capability in the guests can be detected easily without reading MSRs every time accessors.

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index 0fa702673e73..24ac7fe97806 100644
@@ -236,6 +236,7 @@
 #define X86_FEATURE_PVUNLOCK		( 8*32+20) /* "" PV unlock function */
 #define X86_FEATURE_VCPUPREEMPT		( 8*32+21) /* "" PV vcpu_is_preempted function */
 #define X86_FEATURE_TDX_GUEST		( 8*32+22) /* Intel Trust Domain Extensions Guest */
+#define X86_FEATURE_SNP_SECURE_TSC	( 8*32+23) /* "" AMD SNP Secure TSC */
 
 /* Intel-defined CPU features, CPUID level 0x00000007:0 (EBX), word 9 */
 #define X86_FEATURE_FSGSBASE		( 9*32+ 0) /* RDFSBASE, WRFSBASE, RDGSBASE, WRGSBASE instructions*/
Signature · 1 line
2.34.1
maintainer
#33 lore raw

Re: [PATCH v8 09/16] x86/cpufeatures: Add synthetic Secure TSC bit

Quoted 2 lines · Nikunj A Dadhania

Add support for the synthetic CPUID flag which indicates that the SNP guest is running with secure tsc enabled (MSR_AMD64_SEV Bit 11 -

"TSC"

Quoted 2 lines

SecureTsc_Enabled) . This flag is there so that this capability in the guests can be detected easily without reading MSRs every time accessors.

Why?

What's wrong with cc_platform_has(CC_ATTR_GUEST_SECURE_TSC) or so?

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#34 lore raw

Re: [PATCH v8 09/16] x86/cpufeatures: Add synthetic Secure TSC bit

Quoted 5 lines · Borislav Petkov
Quoted 2 lines · Nikunj A Dadhania

Add support for the synthetic CPUID flag which indicates that the SNP guest is running with secure tsc enabled (MSR_AMD64_SEV Bit 11 -

"TSC"

Sure

Quoted 8 lines
Quoted 3 lines

SecureTsc_Enabled) . This flag is there so that this capability in the guests can be detected easily without reading MSRs every time accessors.

Why?

What's wrong with cc_platform_has(CC_ATTR_GUEST_SECURE_TSC) or so?

That was the initial implementation, and there were review comments[1] to use synthetic flag.

Regards
Nikunj

1. https://lore.kernel.org/lkml/[email protected]/

maintainer
#35 lore raw

Re: [PATCH v8 09/16] x86/cpufeatures: Add synthetic Secure TSC bit

Quoted 1 line · Nikunj A. Dadhania

1. https://lore.kernel.org/lkml/[email protected]/

I have no clue what "We have better instrumentation around features." means and am not convinced.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#36 lore raw

PATCH v8 10/16 x86/sev: Add Secure TSC support for SNP guests

In reply to Nikunj A Dadhania

Add support for Secure TSC in SNP enabled guests. Secure TSC allows guest to securely use RDTSC/RDTSCP instructions as the parameters being used cannot be changed by hypervisor once the guest is launched.

During the boot-up of the secondary cpus, SecureTSC enabled guests need to query TSC info from AMD Security Processor. This communication channel is encrypted between the AMD Security Processor and the guest, the hypervisor is just the conduit to deliver the guest messages to the AMD Security Processor. Each message is protected with an AEAD (AES-256 GCM). Use minimal AES GCM library to encrypt/decrypt SNP Guest messages to communicate with the PSP.

Use the guest enc_init hook to fetch SNP TSC info from the AMD Security Processor and initialize the snp_tsc_scale and snp_tsc_offset. During secondary CPU initialization set VMSA fields GUEST_TSC_SCALE (offset 2F0h) and GUEST_TSC_OFFSET(offset 2F8h) with snp_tsc_scale and snp_tsc_offset respectively.

---
 arch/x86/include/asm/sev-common.h |   1 +
 arch/x86/include/asm/sev.h        |  23 +++++++
 arch/x86/include/asm/svm.h        |   6 +-
 arch/x86/kernel/sev.c             | 107 ++++++++++++++++++++++++++++--
 arch/x86/mm/mem_encrypt_amd.c     |   6 ++
 5 files changed, 134 insertions(+), 9 deletions(-)
arch/x86/include/asm/sev-common.h +1 -0
index b463fcbd4b90..6adc8e27feeb 100644
@@ -159,6 +159,7 @@ struct snp_psc_desc {
 #define GHCB_TERM_NOT_VMPL0		3	/* SNP guest is not running at VMPL-0 */
 #define GHCB_TERM_CPUID			4	/* CPUID-validation failure */
 #define GHCB_TERM_CPUID_HV		5	/* CPUID failure during hypervisor fallback */
+#define GHCB_TERM_SECURE_TSC		6	/* Secure TSC initialization failed */
 
 #define GHCB_RESP_CODE(v)		((v) & GHCB_MSR_INFO_MASK)
 
arch/x86/include/asm/sev.h +23 -0
index d950a3ac5694..16bf5afa7731 100644
@@ -170,6 +170,8 @@ enum msg_type {
 	SNP_MSG_ABSORB_RSP,
 	SNP_MSG_VMRK_REQ,
 	SNP_MSG_VMRK_RSP,
+	SNP_MSG_TSC_INFO_REQ = 17,
+	SNP_MSG_TSC_INFO_RSP,
 
 	SNP_MSG_TYPE_MAX
 };
@@ -214,6 +216,23 @@ struct sev_guest_platform_data {
 	struct snp_req_data input;
 };
 
+#define SNP_TSC_INFO_REQ_SZ 128
+
+struct snp_tsc_info_req {
+	/* Must be zero filled */
+	u8 rsvd[SNP_TSC_INFO_REQ_SZ];
+} __packed;
+
+struct snp_tsc_info_resp {
+	/* Status of TSC_INFO message */
+	u32 status;
+	u32 rsvd1;
+	u64 tsc_scale;
+	u64 tsc_offset;
+	u32 tsc_factor;
+	u8 rsvd2[100];
+} __packed;
+
 struct snp_guest_dev {
 	struct device *dev;
 	struct miscdevice misc;
@@ -233,6 +252,7 @@ struct snp_guest_dev {
 		struct snp_report_req report;
 		struct snp_derived_key_req derived_key;
 		struct snp_ext_report_req ext_report;
+		struct snp_tsc_info_req tsc_info;
 	} req;
 	unsigned int vmpck_id;
 };
@@ -370,6 +390,8 @@ static inline void *alloc_shared_pages(size_t sz)
 
 	return page_address(page);
 }
+
+void __init snp_secure_tsc_prepare(void);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -404,6 +426,7 @@ static inline int snp_send_guest_request(struct snp_guest_dev *dev, struct snp_g
 					 struct snp_guest_request_ioctl *rio) { return 0; }
 static inline void free_shared_pages(void *buf, size_t sz) { }
 static inline void *alloc_shared_pages(size_t sz) { return NULL; }
+static inline void __init snp_secure_tsc_prepare(void) { }
 #endif
 
 #ifdef CONFIG_KVM_AMD_SEV
arch/x86/include/asm/svm.h +4 -2
index 87a7b917d30e..3a8294bbd109 100644
@@ -410,7 +410,9 @@ struct sev_es_save_area {
 	u8 reserved_0x298[80];
 	u32 pkru;
 	u32 tsc_aux;
-	u8 reserved_0x2f0[24];
+	u64 tsc_scale;
+	u64 tsc_offset;
+	u8 reserved_0x300[8];
 	u64 rcx;
 	u64 rdx;
 	u64 rbx;
@@ -542,7 +544,7 @@ static inline void __unused_size_checks(void)
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x1c0);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x248);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x298);
-	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x2f0);
+	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x300);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x320);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x380);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x3f0);
arch/x86/kernel/sev.c +100 -7

142 lines not on this page · raw message

arch/x86/mm/mem_encrypt_amd.c +6 -0
index 70b91de2e053..c81b57ca03b6 100644
@@ -214,6 +214,11 @@ void __init sme_map_bootdata(char *real_mode_data)
 	__sme_early_map_unmap_mem(__va(cmdline_paddr), COMMAND_LINE_SIZE, true);
 }
 
+static void __init amd_enc_init(void)
+{
+	snp_secure_tsc_prepare();
+}
+
 static unsigned long pg_level_to_pfn(int level, pte_t *kpte, pgprot_t *ret_prot)
 {
 	unsigned long pfn = 0;
@@ -467,6 +472,7 @@ void __init sme_early_init(void)
 	x86_platform.guest.enc_status_change_finish  = amd_enc_status_change_finish;
 	x86_platform.guest.enc_tlb_flush_required    = amd_enc_tlb_flush_required;
 	x86_platform.guest.enc_cache_flush_required  = amd_enc_cache_flush_required;
+	x86_platform.guest.enc_init                  = amd_enc_init;
 
 	/*
 	 * AMD-SEV-ES intercepts the RDMSR to read the X2APIC ID in the
Signature · 1 line
2.34.1
maintainer
#37 lore raw

Re: [PATCH v8 10/16] x86/sev: Add Secure TSC support for SNP guests

Quoted 5 lines · Nikunj A Dadhania

Add support for Secure TSC in SNP enabled guests. Secure TSC allows guest to securely use RDTSC/RDTSCP instructions as the parameters being used cannot be changed by hypervisor once the guest is launched.

During the boot-up of the secondary cpus, SecureTSC enabled guests

"CPUs"

Quoted 40 lines

need to query TSC info from AMD Security Processor. This communication channel is encrypted between the AMD Security Processor and the guest, the hypervisor is just the conduit to deliver the guest messages to the AMD Security Processor. Each message is protected with an AEAD (AES-256 GCM). Use minimal AES GCM library to encrypt/decrypt SNP Guest messages to communicate with the PSP.

Use the guest enc_init hook to fetch SNP TSC info from the AMD Security Processor and initialize the snp_tsc_scale and snp_tsc_offset. During secondary CPU initialization set VMSA fields GUEST_TSC_SCALE (offset 2F0h) and GUEST_TSC_OFFSET(offset 2F8h) with snp_tsc_scale and snp_tsc_offset respectively.

---
 arch/x86/include/asm/sev-common.h |   1 +
 arch/x86/include/asm/sev.h        |  23 +++++++
 arch/x86/include/asm/svm.h        |   6 +-
 arch/x86/kernel/sev.c             | 107 ++++++++++++++++++++++++++++--
 arch/x86/mm/mem_encrypt_amd.c     |   6 ++
 5 files changed, 134 insertions(+), 9 deletions(-)
arch/x86/include/asm/sev-common.h +1 -0
index b463fcbd4b90..6adc8e27feeb 100644
@@ -159,6 +159,7 @@ struct snp_psc_desc {
 #define GHCB_TERM_NOT_VMPL0		3	/* SNP guest is not running at VMPL-0 */
 #define GHCB_TERM_CPUID			4	/* CPUID-validation failure */
 #define GHCB_TERM_CPUID_HV		5	/* CPUID failure during hypervisor fallback */
+#define GHCB_TERM_SECURE_TSC		6	/* Secure TSC initialization failed */
 
 #define GHCB_RESP_CODE(v)		((v) & GHCB_MSR_INFO_MASK)
 
arch/x86/include/asm/sev.h +0 -0
index d950a3ac5694..16bf5afa7731 100644

arch/x86/include/asm/sev.h enum msg_type { · line 172

Quoted 3 lines
arch/x86/include/asm/sev.h +0 -0
@@ -170,6 +170,8 @@ enum msg_type {
 	SNP_MSG_ABSORB_RSP,
 	SNP_MSG_VMRK_REQ,
 	SNP_MSG_VMRK_RSP,

<-- Pls leave an empty newline here to denote that there's a hole in the define numbers. Alternatively, you can add the missing ones too.

arch/x86/include/asm/sev.h struct sev_guest_platform_data { · line 222

Quoted 13 lines
arch/x86/include/asm/sev.h +6 -0
+	SNP_MSG_TSC_INFO_REQ = 17,
+	SNP_MSG_TSC_INFO_RSP,
 
 	SNP_MSG_TYPE_MAX
 };
@@ -214,6 +216,23 @@ struct sev_guest_platform_data {
 	struct snp_req_data input;
 };
 
+#define SNP_TSC_INFO_REQ_SZ 128
+
+struct snp_tsc_info_req {
+	/* Must be zero filled */

Instead of adding a comment which people might very likely miss, add a check for that array to warn when it is not zeroed.

arch/x86/include/asm/sev.h struct sev_guest_platform_data { · line 227

Quoted 5 lines
arch/x86/include/asm/sev.h +5 -0
+	u8 rsvd[SNP_TSC_INFO_REQ_SZ];
+} __packed;
+
+struct snp_tsc_info_resp {
+	/* Status of TSC_INFO message */

The other struct members don't need a comment?

Quoted 76 lines
arch/x86/include/asm/sev.h +12 -0
+	u32 status;
+	u32 rsvd1;
+	u64 tsc_scale;
+	u64 tsc_offset;
+	u32 tsc_factor;
+	u8 rsvd2[100];
+} __packed;
+
 struct snp_guest_dev {
 	struct device *dev;
 	struct miscdevice misc;
@@ -233,6 +252,7 @@ struct snp_guest_dev {
 		struct snp_report_req report;
 		struct snp_derived_key_req derived_key;
 		struct snp_ext_report_req ext_report;
+		struct snp_tsc_info_req tsc_info;
 	} req;
 	unsigned int vmpck_id;
 };
@@ -370,6 +390,8 @@ static inline void *alloc_shared_pages(size_t sz)
 
 	return page_address(page);
 }
+
+void __init snp_secure_tsc_prepare(void);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -404,6 +426,7 @@ static inline int snp_send_guest_request(struct snp_guest_dev *dev, struct snp_g
 					 struct snp_guest_request_ioctl *rio) { return 0; }
 static inline void free_shared_pages(void *buf, size_t sz) { }
 static inline void *alloc_shared_pages(size_t sz) { return NULL; }
+static inline void __init snp_secure_tsc_prepare(void) { }
 #endif
 
 #ifdef CONFIG_KVM_AMD_SEV
arch/x86/include/asm/svm.h +4 -2
index 87a7b917d30e..3a8294bbd109 100644
@@ -410,7 +410,9 @@ struct sev_es_save_area {
 	u8 reserved_0x298[80];
 	u32 pkru;
 	u32 tsc_aux;
-	u8 reserved_0x2f0[24];
+	u64 tsc_scale;
+	u64 tsc_offset;
+	u8 reserved_0x300[8];
 	u64 rcx;
 	u64 rdx;
 	u64 rbx;
@@ -542,7 +544,7 @@ static inline void __unused_size_checks(void)
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x1c0);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x248);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x298);
-	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x2f0);
+	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x300);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x320);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x380);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x3f0);
arch/x86/kernel/sev.c +4 -0
index a9c1efd6d4e3..20a1e50b7638 100644
@@ -75,6 +75,10 @@ static u64 sev_hv_features __ro_after_init;
 /* Secrets page physical address from the CC blob */
 static u64 secrets_pa __ro_after_init;
 
+/* Secure TSC values read using TSC_INFO SNP Guest request */
+static u64 snp_tsc_scale __ro_after_init;
+static u64 snp_tsc_offset __ro_after_init;
+
 /* #VC handler runtime per-CPU data */
 struct sev_es_runtime_data {
 	struct ghcb ghcb_page;

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 967

Quoted 8 lines
arch/x86/kernel/sev.c +5 -0
@@ -956,6 +960,83 @@ void snp_guest_cmd_unlock(void)
 }
 EXPORT_SYMBOL_GPL(snp_guest_cmd_unlock);
 
+static struct snp_guest_dev tsc_snp_dev __initdata;
+
+static int __snp_send_guest_request(struct snp_guest_dev *snp_dev, struct snp_guest_req *req,
+				    struct snp_guest_request_ioctl *rio);
+

Pls design your code without the need for a forward declaration.

Quoted 7 lines
arch/x86/kernel/sev.c +7 -0
+static int __init snp_get_tsc_info(void)
+{
+	struct snp_tsc_info_req *tsc_req = &tsc_snp_dev.req.tsc_info;
+	static u8 buf[SNP_TSC_INFO_REQ_SZ + AUTHTAG_LEN];
+	struct snp_guest_request_ioctl rio;
+	struct snp_tsc_info_resp tsc_resp;
+	struct snp_guest_req req;

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 984

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+	int rc, resp_len;
+
+	/*
+	 * The intermediate response buffer is used while decrypting the
+	 * response payload. Make sure that it has enough space to cover the
+	 * authtag.
+	 */
+	resp_len = sizeof(tsc_resp) + AUTHTAG_LEN;
+	if (sizeof(buf) < resp_len)
+		return -EINVAL;

Huh, those both are static buffers. Such checks are done with BUILD_BUG_ON.

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 992

Quoted 7 lines
arch/x86/kernel/sev.c +7 -0
+	memset(tsc_req, 0, sizeof(*tsc_req));
+	memset(&req, 0, sizeof(req));
+	memset(&rio, 0, sizeof(rio));
+	memset(buf, 0, sizeof(buf));
+
+	if (!snp_assign_vmpck(&tsc_snp_dev, 0))
+		return -EINVAL;

Do that before the memsetting.

Quoted 6 lines
arch/x86/kernel/sev.c +6 -0
+
+	/* Initialize the PSP channel to send snp messages */
+	rc = snp_setup_psp_messaging(&tsc_snp_dev);
+	if (rc)
+		return rc;
+

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 1008

Quoted 10 lines
arch/x86/kernel/sev.c +10 -0
+	req.msg_version = MSG_HDR_VER;
+	req.msg_type = SNP_MSG_TSC_INFO_REQ;
+	req.vmpck_id = tsc_snp_dev.vmpck_id;
+	req.req_buf = tsc_req;
+	req.req_sz = sizeof(*tsc_req);
+	req.resp_buf = buf;
+	req.resp_sz = resp_len;
+	req.exit_code = SVM_VMGEXIT_GUEST_REQUEST;
+
+	rc = __snp_send_guest_request(&tsc_snp_dev, &req, &rio);

The changes to *snp_send_guest_request are unrelated to the secure TSC enablement. Pls do them in a pre-patch.

Ok, I'm going to stop here and give you a chance to work in all the review feedback and send a new revision.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#38 lore raw

Re: [PATCH v8 10/16] x86/sev: Add Secure TSC support for SNP guests

Quoted 8 lines · Borislav Petkov
Quoted 5 lines · Nikunj A Dadhania

Add support for Secure TSC in SNP enabled guests. Secure TSC allows guest to securely use RDTSC/RDTSCP instructions as the parameters being used cannot be changed by hypervisor once the guest is launched.

During the boot-up of the secondary cpus, SecureTSC enabled guests

"CPUs"

Sure

Quoted 35 lines
Quoted 35 lines

need to query TSC info from AMD Security Processor. This communication channel is encrypted between the AMD Security Processor and the guest, the hypervisor is just the conduit to deliver the guest messages to the AMD Security Processor. Each message is protected with an AEAD (AES-256 GCM). Use minimal AES GCM library to encrypt/decrypt SNP Guest messages to communicate with the PSP.

Use the guest enc_init hook to fetch SNP TSC info from the AMD Security Processor and initialize the snp_tsc_scale and snp_tsc_offset. During secondary CPU initialization set VMSA fields GUEST_TSC_SCALE (offset 2F0h) and GUEST_TSC_OFFSET(offset 2F8h) with snp_tsc_scale and snp_tsc_offset respectively.

---
 arch/x86/include/asm/sev-common.h |   1 +
 arch/x86/include/asm/sev.h        |  23 +++++++
 arch/x86/include/asm/svm.h        |   6 +-
 arch/x86/kernel/sev.c             | 107 ++++++++++++++++++++++++++++--
 arch/x86/mm/mem_encrypt_amd.c     |   6 ++
 5 files changed, 134 insertions(+), 9 deletions(-)
arch/x86/include/asm/sev-common.h +1 -0
index b463fcbd4b90..6adc8e27feeb 100644
@@ -159,6 +159,7 @@ struct snp_psc_desc {
 #define GHCB_TERM_NOT_VMPL0		3	/* SNP guest is not running at VMPL-0 */
 #define GHCB_TERM_CPUID			4	/* CPUID-validation failure */
 #define GHCB_TERM_CPUID_HV		5	/* CPUID failure during hypervisor fallback */
+#define GHCB_TERM_SECURE_TSC		6	/* Secure TSC initialization failed */
 
 #define GHCB_RESP_CODE(v)		((v) & GHCB_MSR_INFO_MASK)
 
Quoted 11 lines

arch/x86/include/asm/sev.h enum msg_type { · line 172

Quoted 8 lines
arch/x86/include/asm/sev.h +0 -0
index d950a3ac5694..16bf5afa7731 100644
@@ -170,6 +170,8 @@ enum msg_type {
 	SNP_MSG_ABSORB_RSP,
 	SNP_MSG_VMRK_REQ,
 	SNP_MSG_VMRK_RSP,

<-- Pls leave an empty newline here to denote that there's a hole in the define numbers. Alternatively, you can add the missing ones too.

I will add empty line.

Quoted 9 lines
Quoted 9 lines
arch/x86/include/asm/sev.h +2 -0
+	SNP_MSG_TSC_INFO_REQ = 17,
+	SNP_MSG_TSC_INFO_RSP,
 
 	SNP_MSG_TYPE_MAX
 };
@@ -214,6 +216,23 @@ struct sev_guest_platform_data {
 	struct snp_req_data input;
 };
 
Quoted 7 lines

arch/x86/include/asm/sev.h struct sev_guest_platform_data { · line 222

Quoted 4 lines
arch/x86/include/asm/sev.h +4 -0
+#define SNP_TSC_INFO_REQ_SZ 128
+
+struct snp_tsc_info_req {
+	/* Must be zero filled */

Instead of adding a comment which people might very likely miss, add a check for that array to warn when it is not zeroed.

Sure.

Quoted 8 lines

arch/x86/include/asm/sev.h struct sev_guest_platform_data { · line 227

Quoted 6 lines
arch/x86/include/asm/sev.h +5 -0
+	u8 rsvd[SNP_TSC_INFO_REQ_SZ];
+} __packed;
+
+struct snp_tsc_info_resp {
+	/* Status of TSC_INFO message */

The other struct members don't need a comment?

Sure.

Quoted 79 lines
Quoted 79 lines
arch/x86/include/asm/sev.h +12 -0
+	u32 status;
+	u32 rsvd1;
+	u64 tsc_scale;
+	u64 tsc_offset;
+	u32 tsc_factor;
+	u8 rsvd2[100];
+} __packed;
+
 struct snp_guest_dev {
 	struct device *dev;
 	struct miscdevice misc;
@@ -233,6 +252,7 @@ struct snp_guest_dev {
 		struct snp_report_req report;
 		struct snp_derived_key_req derived_key;
 		struct snp_ext_report_req ext_report;
+		struct snp_tsc_info_req tsc_info;
 	} req;
 	unsigned int vmpck_id;
 };
@@ -370,6 +390,8 @@ static inline void *alloc_shared_pages(size_t sz)
 
 	return page_address(page);
 }
+
+void __init snp_secure_tsc_prepare(void);
 #else
 static inline void sev_es_ist_enter(struct pt_regs *regs) { }
 static inline void sev_es_ist_exit(void) { }
@@ -404,6 +426,7 @@ static inline int snp_send_guest_request(struct snp_guest_dev *dev, struct snp_g
 					 struct snp_guest_request_ioctl *rio) { return 0; }
 static inline void free_shared_pages(void *buf, size_t sz) { }
 static inline void *alloc_shared_pages(size_t sz) { return NULL; }
+static inline void __init snp_secure_tsc_prepare(void) { }
 #endif
 
 #ifdef CONFIG_KVM_AMD_SEV
arch/x86/include/asm/svm.h +4 -2
index 87a7b917d30e..3a8294bbd109 100644
@@ -410,7 +410,9 @@ struct sev_es_save_area {
 	u8 reserved_0x298[80];
 	u32 pkru;
 	u32 tsc_aux;
-	u8 reserved_0x2f0[24];
+	u64 tsc_scale;
+	u64 tsc_offset;
+	u8 reserved_0x300[8];
 	u64 rcx;
 	u64 rdx;
 	u64 rbx;
@@ -542,7 +544,7 @@ static inline void __unused_size_checks(void)
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x1c0);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x248);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x298);
-	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x2f0);
+	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x300);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x320);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x380);
 	BUILD_BUG_RESERVED_OFFSET(sev_es_save_area, 0x3f0);
arch/x86/kernel/sev.c +4 -0
index a9c1efd6d4e3..20a1e50b7638 100644
@@ -75,6 +75,10 @@ static u64 sev_hv_features __ro_after_init;
 /* Secrets page physical address from the CC blob */
 static u64 secrets_pa __ro_after_init;
 
+/* Secure TSC values read using TSC_INFO SNP Guest request */
+static u64 snp_tsc_scale __ro_after_init;
+static u64 snp_tsc_offset __ro_after_init;
+
 /* #VC handler runtime per-CPU data */
 struct sev_es_runtime_data {
 	struct ghcb ghcb_page;
@@ -956,6 +960,83 @@ void snp_guest_cmd_unlock(void)
 }
 EXPORT_SYMBOL_GPL(snp_guest_cmd_unlock);
 
Quoted 7 lines

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 967

Quoted 5 lines
arch/x86/kernel/sev.c +5 -0
+static struct snp_guest_dev tsc_snp_dev __initdata;
+
+static int __snp_send_guest_request(struct snp_guest_dev *snp_dev, struct snp_guest_req *req,
+				    struct snp_guest_request_ioctl *rio);
+

Pls design your code without the need for a forward declaration.

Sure

Quoted 13 lines
Quoted 13 lines
arch/x86/kernel/sev.c +12 -0
+static int __init snp_get_tsc_info(void)
+{
+	struct snp_tsc_info_req *tsc_req = &tsc_snp_dev.req.tsc_info;
+	static u8 buf[SNP_TSC_INFO_REQ_SZ + AUTHTAG_LEN];
+	struct snp_guest_request_ioctl rio;
+	struct snp_tsc_info_resp tsc_resp;
+	struct snp_guest_req req;
+	int rc, resp_len;
+
+	/*
+	 * The intermediate response buffer is used while decrypting the
+	 * response payload. Make sure that it has enough space to cover the
Quoted 8 lines

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 984

Quoted 5 lines
arch/x86/kernel/sev.c +5 -0
+	 * authtag.
+	 */
+	resp_len = sizeof(tsc_resp) + AUTHTAG_LEN;
+	if (sizeof(buf) < resp_len)
+		return -EINVAL;

Huh, those both are static buffers. Such checks are done with BUILD_BUG_ON.

Ok

Quoted 11 lines

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 992

Quoted 8 lines
arch/x86/kernel/sev.c +7 -0
+	memset(tsc_req, 0, sizeof(*tsc_req));
+	memset(&req, 0, sizeof(req));
+	memset(&rio, 0, sizeof(rio));
+	memset(buf, 0, sizeof(buf));
+
+	if (!snp_assign_vmpck(&tsc_snp_dev, 0))
+		return -EINVAL;

Do that before the memsetting.

Sure

Quoted 11 lines
Quoted 11 lines
arch/x86/kernel/sev.c +11 -0
+
+	/* Initialize the PSP channel to send snp messages */
+	rc = snp_setup_psp_messaging(&tsc_snp_dev);
+	if (rc)
+		return rc;
+
+	req.msg_version = MSG_HDR_VER;
+	req.msg_type = SNP_MSG_TSC_INFO_REQ;
+	req.vmpck_id = tsc_snp_dev.vmpck_id;
+	req.req_buf = tsc_req;
+	req.req_sz = sizeof(*tsc_req);
Quoted 8 lines

arch/x86/kernel/sev.c void snp_guest_cmd_unlock(void) · line 1008

Quoted 5 lines
arch/x86/kernel/sev.c +5 -0
+	req.resp_buf = buf;
+	req.resp_sz = resp_len;
+	req.exit_code = SVM_VMGEXIT_GUEST_REQUEST;
+
+	rc = __snp_send_guest_request(&tsc_snp_dev, &req, &rio);

The changes to *snp_send_guest_request are unrelated to the secure TSC enablement. Pls do them in a pre-patch.

Sure

Quoted 3 lines

Ok, I'm going to stop here and give you a chance to work in all the review feedback and send a new revision.

Thank you for detailed review/feedback, will address them and send new revision.

Regards
Nikunj

#39 lore raw

PATCH v8 11/16 x86/sev: Change TSC MSR behavior for Secure TSC enabled guests

In reply to Nikunj A Dadhania

Secure TSC enabled guests should not write MSR_IA32_TSC(10H) register as the subsequent TSC value reads are undefined. MSR_IA32_TSC related accesses should not exit to the hypervisor for such guests.

Accesses to MSR_IA32_TSC needs special handling in the #VC handler for the guests with Secure TSC enabled. Writes to MSR_IA32_TSC should be ignored, and reads of MSR_IA32_TSC should return the result of the RDTSC instruction.

---
 arch/x86/kernel/sev.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
arch/x86/kernel/sev.c +24 -0
index 20a1e50b7638..64243c44a7d3 100644
@@ -1279,6 +1279,30 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	/* Is it a WRMSR? */
 	exit_info_1 = (ctxt->insn.opcode.bytes[1] == 0x30) ? 1 : 0;
 
+	/*
+	 * TSC related accesses should not exit to the hypervisor when a
+	 * guest is executing with SecureTSC enabled, so special handling
+	 * is required for accesses of MSR_IA32_TSC:
+	 *
+	 * Writes: Writing to MSR_IA32_TSC can cause subsequent reads
+	 *         of the TSC to return undefined values, so ignore all
+	 *         writes.
+	 * Reads:  Reads of MSR_IA32_TSC should return the current TSC
+	 *         value, use the value returned by RDTSC.
+	 */
+	if (regs->cx == MSR_IA32_TSC && cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC)) {
+		u64 tsc;
+
+		if (exit_info_1)
+			return ES_OK;
+
+		tsc = rdtsc();
+		regs->ax = UINT_MAX & tsc;
+		regs->dx = UINT_MAX & (tsc >> 32);
+
+		return ES_OK;
+	}
+
 	ghcb_set_rcx(ghcb, regs->cx);
 	if (exit_info_1) {
 		ghcb_set_rax(ghcb, regs->ax);
Signature · 1 line
2.34.1
#40 lore raw

PATCH v8 12/16 x86/sev: Prevent RDTSC/RDTSCP interception for Secure TSC enabled guests

In reply to Nikunj A Dadhania

The hypervisor should not be intercepting RDTSC/RDTSCP when Secure TSC is enabled. A #VC exception will be generated if the RDTSC/RDTSCP instructions are being intercepted. If this should occur and Secure TSC is enabled, terminate guest execution.

---
 arch/x86/kernel/sev-shared.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
arch/x86/kernel/sev-shared.c +10 -0
index ae79f9505298..26379c70878b 100644
@@ -1000,6 +1000,16 @@ static enum es_result vc_handle_rdtsc(struct ghcb *ghcb,
 	bool rdtscp = (exit_code == SVM_EXIT_RDTSCP);
 	enum es_result ret;
 
+	/*
+	 * RDTSC and RDTSCP should not be intercepted when Secure TSC is
+	 * enabled. Terminate the SNP guest when the interception is enabled.
+	 * This file is included from kernel/sev.c and boot/compressed/sev.c,
+	 * use sev_status here as cpu_feature_enabled() is not available when
+	 * compiling boot/compressed/sev.c.
+	 */
+	if (sev_status & MSR_AMD64_SNP_SECURE_TSC)
+		return ES_VMM_ERROR;
+
 	ret = sev_es_ghcb_hv_call(ghcb, ctxt, exit_code, 0, 0);
 	if (ret != ES_OK)
 		return ret;
Signature · 1 line
2.34.1
#41 lore raw

PATCH v8 13/16 x86/kvmclock: Skip kvmclock when Secure TSC is available

In reply to Nikunj A Dadhania

For AMD SNP guests having Secure TSC enabled, skip using the kvmclock. The guest kernel will fallback and use Secure TSC based clocksource.

---
 arch/x86/kernel/kvmclock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/kvmclock.c +1 -1
index 5bb395551c44..855ef65dbc4a 100644
@@ -289,7 +289,7 @@ void __init kvmclock_init(void)
 {
 	u8 flags;
 
-	if (!kvm_para_available() || !kvmclock)
+	if (!kvm_para_available() || !kvmclock || cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
 		return;
 
 	if (kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE2)) {
Signature · 1 line
2.34.1
#42 lore raw

PATCH v8 14/16 x86/sev: Mark Secure TSC as reliable

In reply to Nikunj A Dadhania

AMD SNP guests may have Secure TSC feature enabled. Use the Secure TSC as the only reliable clock source in SEV-SNP guests when enabled, bypassing unstable calibration.

---
 arch/x86/mm/mem_encrypt_amd.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/mm/mem_encrypt_amd.c +4 -0
index c81b57ca03b6..cc936999efc8 100644
@@ -498,6 +498,10 @@ void __init sme_early_init(void)
 	 */
 	if (sev_status & MSR_AMD64_SEV_ENABLED)
 		ia32_disable();
+
+	/* Mark the TSC as reliable when Secure TSC is enabled */
+	if (sev_status & MSR_AMD64_SNP_SECURE_TSC)
+		setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
 }
 
 void __init mem_encrypt_free_decrypted_mem(void)
Signature · 1 line
2.34.1
#43 lore raw

PATCH v8 15/16 x86/cpu/amd: Do not print FW_BUG for Secure TSC

In reply to Nikunj A Dadhania

When SecureTSC is enabled and TscInvariant (bit 8) in CPUID_8000_0007_edx is set, kernel complains with the below firmware bug:

[Firmware Bug]: TSC doesn't count with P0 frequency!

Secure TSC need not run at P0 frequency, the TSC frequency is set by the VMM as part of the SNP_LAUNCH_START command. Avoid the check when Secure TSC is enabled

---
 arch/x86/kernel/cpu/amd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/amd.c +2 -1
index ce89281640a9..2e7d67e1f795 100644
@@ -471,7 +471,8 @@ static void early_init_amd_mc(struct cpuinfo_x86 *c)
 
 static void bsp_init_amd(struct cpuinfo_x86 *c)
 {
-	if (cpu_has(c, X86_FEATURE_CONSTANT_TSC)) {
+	if (cpu_has(c, X86_FEATURE_CONSTANT_TSC) &&
+	    !cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC)) {
 
 		if (c->x86 > 0x10 ||
 		    (c->x86 == 0x10 && c->x86_model >= 0x2)) {
Signature · 1 line
2.34.1
#44 lore raw

PATCH v8 16/16 x86/sev: Enable Secure TSC for SNP guests

In reply to Nikunj A Dadhania

Now that all the required plumbing is done for enabling SNP Secure TSC feature, add Secure TSC to snp features present list.

Set the CPUID feature bit (X86_FEATURE_SNP_SECURE_TSC) when SNP guest is started with Secure TSC.

---
 arch/x86/boot/compressed/sev.c |  3 ++-
 arch/x86/mm/mem_encrypt.c      | 10 ++++++++--
 arch/x86/mm/mem_encrypt_amd.c  |  4 +++-
 3 files changed, 13 insertions(+), 4 deletions(-)
arch/x86/boot/compressed/sev.c +2 -1
index 073291832f44..d7e28084333a 100644
@@ -379,7 +379,8 @@ static void enforce_vmpl0(void)
  * by the guest kernel. As and when a new feature is implemented in the
  * guest kernel, a corresponding bit should be added to the mask.
  */
-#define SNP_FEATURES_PRESENT	MSR_AMD64_SNP_DEBUG_SWAP
+#define SNP_FEATURES_PRESENT	(MSR_AMD64_SNP_DEBUG_SWAP |	\
+				 MSR_AMD64_SNP_SECURE_TSC)
 
 u64 snp_get_unsupported_features(u64 status)
 {
arch/x86/mm/mem_encrypt.c +8 -2
index 68aa06852466..350ba605509d 100644
@@ -70,8 +70,14 @@ static void print_mem_encrypt_feature_info(void)
 			pr_cont(" SEV-ES");
 
 		/* Secure Nested Paging */
-		if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP))
-			pr_cont(" SEV-SNP");
+		if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) {
+			pr_cont(" SEV-SNP\n");
+			pr_cont("SNP Features active: ");
+
+			/* SNP Secure TSC */
+			if (cpu_feature_enabled(X86_FEATURE_SNP_SECURE_TSC))
+				pr_cont(" SECURE-TSC");
+		}
 
 		pr_cont("\n");
 		break;
arch/x86/mm/mem_encrypt_amd.c +3 -1
index cc936999efc8..7ee0a537a22e 100644
@@ -500,8 +500,10 @@ void __init sme_early_init(void)
 		ia32_disable();
 
 	/* Mark the TSC as reliable when Secure TSC is enabled */
-	if (sev_status & MSR_AMD64_SNP_SECURE_TSC)
+	if (sev_status & MSR_AMD64_SNP_SECURE_TSC) {
+		setup_force_cpu_cap(X86_FEATURE_SNP_SECURE_TSC);
 		setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+	}
 }
 
 void __init mem_encrypt_free_decrypted_mem(void)
Signature · 1 line
2.34.1