Allow guest to query AMD SEV(-ES) runtime attestation evidence

This patch series is provided to allow the guest application to query AMD SEV(-ES) runtime attestation evidence by communicating with the host service (Attestation Evidence Broker).

The following is the design document.

Compared with SEV-SNP and Intel TDX, the runtime attestation of SEV(-ES) does NOT support guest-provided data included in the attestation report [1,2]. In addition, SEV(-ES) also does NOT support the dynamic measurement. During runtime, it can only generate static attestation report with the constant launch digest reflecting the initial measurement.

Although SEV(-ES) has above limitations, its runtime attestation is still useful. When the SEV(-ES) guest is running, it can report the attestation report with fixed launch digest as a heartbeat for trusted healthy check.

SEV(-ES) runtime attestation includes two participants:
attester and verifier.

- The attester running in a SEV(-ES) guest is responsible for collecting attestation evidence.
- The verifier running in a trusted environment is responsible for verifying the attestation evidence provided by the attester. Note that the verifier can run on any platform even non-TEE.

Verifier uses the following SEV(-ES) certificate chains to verify the signature of the attestation report generated by the ATTESTATION command [2]:

1. certificate chain for device identity:
ARK -> ASK -> CEK -> PEK -> report
2. certificate chain for platform owner identity:
OCA -> PEK -> report

- foo -> bar indicates using the public key of foo to verify the signature of bar.
- ARK is the root of trust of AMD and OCA is the root of trust for platform owners. OCA has two ways:
1. Self-owned: The OCA Key Pair and self-signed OCA certificate are automatically generated by the SEV(-ES) firmware.
2. Externally-owned: External users use OCA Key Pair to generate self-signed OCA certificates in a trusted environment.

Verifier needs to verify the attestation report with the certificate chain. ARK and ASK can be obtained directly from the AMD KDS server. CEK, PEK, OCA, and attestation report are related to the specific SEV(-ES) platform, therefore SEV(-ES) Attestation Evidence collected by attester should include attestation report (with the constant launch digest), PEK, CEK, and OCA certificate.

| Contents of SEV(-ES) Attestation Evidence 	| SEV(-ES) firmware command	|
| :-: | :-: |
| attestation report				| ATTESTATION			|
| CEK						| GET_ID			|
| OCA,PEK					| PDH_CERT_EXPORT		|

According to the official feedback[3], SEV(-ES) firmware APIs don't support query attestation report in SEV(-ES) guest and there is no plan to support it in the future. Instead, this capability will be available in SEV-SNP.

In some scenarios, the guest application needs to query the attestation report to establish an attested channel with the remote peer. There are two approaches for a guest application to query an attestation evidence:

- Hypercall approach
- VSOCK approach

Considering time and cost, we only need to implement one of them.

- Hypercall approach

SEV(-ES) guest exits to VMM using hypercall and then interacts with SEV firmware to query the components composing an attestation evidence, including attestation report, PEK, CEK, OCA certificate. To build an attestation evidence, the steps include:

1. The guest application requests a shared memory page, initiates a hypercall, and switches from the guest mode to the host mode.
2. In the host mode, KVM sends the GET_ID, PDH_CERT_EXPORT, ATTESTATION command requests to SEV firmware.
3. The shared memory page is filled with the data returned by the SEV firmware.
4. The guest application can obtain attestation evidence by reading the data in the shared memory.

Although this method can meet our requirements, it requires a lot of modifications to the guest kernel and KVM.

- VSOCK approach

In the current implementation, QEMU provides the QMP interface "query-sev-attestation-report" to query the attestation report in the host.
However, QEMU is not the only VMM. In order to support various VMM in
different scenarios, it is necessary to design a general host service, such as attestation evidence Broker (AEB) to query attestation evidence from the host.

The workflow of AEB is as followed:

1. The user-level application in the guest sends a request (including guest firmware handle) to AEB through VSOCK.
2. AEB requests to query attestation report, PEK, CEK, OCA certificate by calling multiple SEV firmware APIs (refer to the table above for specific API commands) and assembles these information into the attestation evidence.
3. AEB returns the attestation evidence to the application in the guest.

To query the attestation report in host with AEB, we provides three patches to achieve the following two goals.

1. It is necessary to add a SEV_GET_REPORT interface in ccp driver so that AEB can execute ioctl on the /dev/sev device and then call the SEV_GET_REPORT interface to send the ATTESTATION command to the SEV firmware to query attestation report.

2. In order to obtain the guest handle required by the ATTESTATION command to the SEV firmware, a new hypercall needs to be added to the KVM. The application in the guest obtains the guest handle through this newly added hypercal, and then sends it to the AEB service through VSOCK. The AEB service uses the guest handle as the input parameter of the ATTESTATION command to interact with the SEV firmware.

Note that hypercall is not the only way to obtain the guest handle. Actually the qmp interface query-sev can query the guest handle as well. However, as mentioned previously, qemu is not the only VMM.

Below is the communication protocol between the guest application and AEB.

```protobuf
syntax = "proto3";
...
message RetrieveAttestationEvidenceSizeRequest{
    uint32 guest_handle = 1;
}
message RetrieveAttestationEvidenceRequest{
    uint32 guest_handle = 1;
    uint32 evidence_size = 2;
}
message RetrieveAttestationEvidenceSizeResponse{
    uint32 error_code = 1;
    uint32 evidence_size = 2;
}
message RetrieveAttestationEvidenceResponse{
    uint32 error_code = 1;
    uint32 evidence_size = 2;
    bytes evidence = 3;
}
service AEBService {
    rpc RetrieveAttestationEvidenceSize
         (RetrieveAttestationEvidenceSizeRequest)
         returns (RetrieveAttestationEvidenceSizeResponse);
    rpc RetrieveAttestationEvidence(RetrieveAttestationEvidenceRequest)
         returns (RetrieveAttestationEvidenceResponse);
}
```

[1] https://www.amd.com/system/files/TechDocs/
55766_SEV-KM_API_Specification.pdf
[2] https://www.amd.com/system/files/TechDocs/56860.pdf
[3] https://github.com/AMDESE/AMDSEV/issues/71#issuecomment-926118314

Shirong Hao (3):
KVM: X86: Introduce KVM_HC_VM_HANDLE hypercall
KVM/SVM: move the implementation of sev_get_attestation_report to ccp
driver
crypto: ccp: Implement SEV_GET_REPORT ioctl command

 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/svm/sev.c          | 49 +++-------------------
 arch/x86/kvm/svm/svm.c          | 11 +++++
 arch/x86/kvm/x86.c              |  7 +++-
 drivers/crypto/ccp/sev-dev.c    | 74 +++++++++++++++++++++++++++++++++
 include/linux/psp-sev.h         |  7 ++++
 include/uapi/linux/kvm_para.h   |  1 +
 include/uapi/linux/psp-sev.h    | 17 ++++++++
 8 files changed, 123 insertions(+), 44 deletions(-)

2.27.0

This hypercall is used by the SEV guest to get the firmware handle.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/svm/svm.c          | 11 +++++++++++
 arch/x86/kvm/x86.c              |  7 ++++++-
 include/uapi/linux/kvm_para.h   |  1 +
 4 files changed, 19 insertions(+), 1 deletion(-)

index 2164b9f4c7b0..fe745f4e6954 100644
@@ -1493,6 +1493,7 @@ struct kvm_x86_ops {
 	int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
 
 	void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector);
+	int (*vm_handle)(struct kvm *kvm);
 };
 
 struct kvm_x86_nested_ops {

index d0f68d11ec70..c0eb310cb4c3 100644
@@ -4576,6 +4576,16 @@ static int svm_vm_init(struct kvm *kvm)
 	return 0;
 }
 
+static int sev_vm_handle(struct kvm *kvm)
+{
+	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
+
+	if (!sev_guest(kvm))
+		return -ENOTTY;
+
+	return sev->handle;
+}
+
 static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.name = "kvm_amd",
 

@@ -4705,6 +4715,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = {
 	.complete_emulated_msr = svm_complete_emulated_msr,
 
 	.vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
+	.vm_handle = sev_vm_handle,
 };
 
 static struct kvm_x86_init_ops svm_init_ops __initdata = {

index 0cf1082455df..24acf0f2a539 100644
@@ -8906,7 +8906,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 		a3 &= 0xFFFFFFFF;
 	}
 
-	if (static_call(kvm_x86_get_cpl)(vcpu) != 0) {
+	if (static_call(kvm_x86_get_cpl)(vcpu) != 0 && nr != KVM_HC_VM_HANDLE) {
 		ret = -KVM_EPERM;
 		goto out;
 	}

@@ -8965,6 +8965,11 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
 		vcpu->arch.complete_userspace_io = complete_hypercall_exit;
 		return 0;
 	}
+	case KVM_HC_VM_HANDLE:
+		ret = -KVM_ENOSYS;
+		if (kvm_x86_ops.vm_handle)
+			ret = kvm_x86_ops.vm_handle(vcpu->kvm);
+		break;
 	default:
 		ret = -KVM_ENOSYS;
 		break;

index 960c7e93d1a9..b64469a12707 100644
@@ -30,6 +30,7 @@
 #define KVM_HC_SEND_IPI		10
 #define KVM_HC_SCHED_YIELD		11
 #define KVM_HC_MAP_GPA_RANGE		12
+#define KVM_HC_VM_HANDLE		13
 
 /*
  * hypercalls use architecture specific

2.27.0

This is completely insufficient to justify why KVM is providing host information to the guest, let alone why KVM is providing that information to guest _userspace_.

...

The sev_get_attestation_report is called by qemu-kvm to get sev attestation report. However, getting the sev attestation report is a general function, the need for host program to get the sev attestation report by interacting with /dev/sev device exists.

Considering this need, it is more reasonable to move the main implementation of sev_get_attestation_report into sev_do_get_report defined in the ccp driver. Any host program getting the guest firmware handle can directly interact with the sev device based on sev_do_get_report to get the sev attestation report.

In addition, after moving the general code of sev_get_attestation_report to the ccp, move the check for sev fd in __sev_issue_cmd to the sev_get_attestation_report function is necessary.

---
 arch/x86/kvm/svm/sev.c       | 49 ++++---------------------------
 drivers/crypto/ccp/sev-dev.c | 56 ++++++++++++++++++++++++++++++++++++
 include/linux/psp-sev.h      |  7 +++++
 3 files changed, 69 insertions(+), 43 deletions(-)

index 7656a2c5662a..016acc96c5dc 100644
@@ -1066,10 +1066,8 @@ static int sev_get_attestation_report(struct kvm *kvm, struct kvm_sev_cmd *argp)
 {
 	void __user *report = (void __user *)(uintptr_t)argp->data;
 	struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
-	struct sev_data_attestation_report data;
 	struct kvm_sev_attestation_report params;
-	void __user *p;
-	void *blob = NULL;
+	struct fd f;
 	int ret;
 
 	if (!sev_guest(kvm))

@@ -1078,48 +1076,13 @@ static int sev_get_attestation_report(struct kvm *kvm, struct kvm_sev_cmd *argp)
 	if (copy_from_user(&params, (void __user *)(uintptr_t)argp->data, sizeof(params)))
 		return -EFAULT;
 
-	memset(&data, 0, sizeof(data));
-
-	/* User wants to query the blob length */
-	if (!params.len)
-		goto cmd;
-
-	p = (void __user *)(uintptr_t)params.uaddr;
-	if (p) {
-		if (params.len > SEV_FW_BLOB_MAX_SIZE)
-			return -EINVAL;
-
-		blob = kmalloc(params.len, GFP_KERNEL_ACCOUNT);
-		if (!blob)
-			return -ENOMEM;
-
-		data.address = __psp_pa(blob);
-		data.len = params.len;
-		memcpy(data.mnonce, params.mnonce, sizeof(params.mnonce));
-	}
-cmd:
-	data.handle = sev->handle;
-	ret = sev_issue_cmd(kvm, SEV_CMD_ATTESTATION_REPORT, &data, &argp->error);
-	/*
-	 * If we query the session length, FW responded with expected data.
-	 */
-	if (!params.len)
-		goto done;
-
-	if (ret)
-		goto e_free_blob;
+	f = fdget(sev->fd);
+	if (!f.file)
+		return -EBADF;
 
-	if (blob) {
-		if (copy_to_user(p, blob, params.len))
-			ret = -EFAULT;
-	}
+	ret = sev_do_get_report(report, &params, f.file, sev->handle, &argp->error);
 
-done:
-	params.len = data.len;
-	if (copy_to_user(report, &params, sizeof(params)))
-		ret = -EFAULT;
-e_free_blob:
-	kfree(blob);
+	fdput(f);
 	return ret;
 }
 

index d48a7192e881..0cbf39a7d116 100644
@@ -541,6 +541,9 @@ int sev_platform_init(int *error);
  */
 int sev_platform_status(struct sev_user_data_status *status, int *error);
 
+int sev_do_get_report(void __user *report, struct kvm_sev_attestation_report *input,
+		      struct file *filep, u32 handle, u32 *error);
+
 /**
  * sev_issue_cmd_external_user - issue SEV command by other driver with a file
  * handle.

@@ -649,6 +652,10 @@ sev_issue_cmd_external_user(struct file *filep, unsigned int id, void *data, int
 
 static inline void *psp_copy_user_blob(u64 __user uaddr, u32 len) { return ERR_PTR(-EINVAL); }
 
+static inline int
+sev_do_get_report(void __user *report, struct kvm_sev_attestation_report *input,
+		  struct file *filep, u32 handle, u32 *error) { return -ENODEV; }
+
 #endif	/* CONFIG_CRYPTO_DEV_SP_PSP */
 
 #endif	/* __PSP_SEV_H__ */

2.27.0

The SEV_GET_REPORT command can be used by host service with guest firmware handle to query the attestation report.

---
 drivers/crypto/ccp/sev-dev.c | 20 +++++++++++++++++++-
 include/uapi/linux/psp-sev.h | 17 +++++++++++++++++
 2 files changed, 36 insertions(+), 1 deletion(-)

index 2f6b81742d28..2e479b88aa29 100644
@@ -414,7 +414,10 @@ int sev_do_get_report(void __user *report, struct kvm_sev_attestation_report *in
 	}
 cmd:
 	data.handle = handle;
-	ret = sev_issue_cmd_external_user(filep, SEV_CMD_ATTESTATION_REPORT, &data, error);
+	if (!filep)
+		ret = __sev_do_cmd_locked(SEV_CMD_ATTESTATION_REPORT, &data, error);
+	else
+		ret = sev_issue_cmd_external_user(filep, SEV_CMD_ATTESTATION_REPORT, &data, error);
 
 	/*
 	 * If we query the session length, FW responded with expected data.

@@ -440,6 +443,18 @@ int sev_do_get_report(void __user *report, struct kvm_sev_attestation_report *in
 }
 EXPORT_SYMBOL_GPL(sev_do_get_report);
 
+static int sev_ioctl_do_get_report(struct sev_issue_cmd *argp)
+{
+	void __user *report = (void __user *)(uintptr_t)argp->data;
+	struct sev_user_data_attestation_report input;
+
+	if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
+		return -EFAULT;
+
+	return sev_do_get_report(report, (struct kvm_sev_attestation_report *)&input,
+				 NULL, input.handle, &argp->error);
+}
+
 static int sev_ioctl_do_pek_pdh_gen(int cmd, struct sev_issue_cmd *argp, bool writable)
 {
 	struct sev_device *sev = psp_master->sev_data;

@@ -926,6 +941,9 @@ static long sev_ioctl(struct file *file, unsigned int ioctl, unsigned long arg)
 	case SEV_GET_ID2:
 		ret = sev_ioctl_do_get_id2(&input);
 		break;
+	case SEV_GET_REPORT:
+		ret = sev_ioctl_do_get_report(&input);
+		break;
 	default:
 		ret = -EINVAL;
 		goto out;

index 91b4c63d5cbf..c7d70fc0ac1e 100644
@@ -28,6 +28,7 @@ enum {
 	SEV_PEK_CERT_IMPORT,
 	SEV_GET_ID,	/* This command is deprecated, use SEV_GET_ID2 */
 	SEV_GET_ID2,
+	SEV_GET_REPORT,
 
 	SEV_MAX,
 };

@@ -147,6 +148,22 @@ struct sev_user_data_get_id2 {
 	__u32 length;				/* In/Out */
 } __packed;
 
+/**
+ * struct sev_user_data_attestation_report - ATTESTATION command parameters
+ *
+ * @mnonce: mnonce to compute HMAC
+ * @uaddr: physical address containing the attestation report
+ * @len: length of attestation report
+ * @handle: handle of the VM to process
+ */
+
+struct sev_user_data_attestation_report {
+	__u8 mnonce[16];			/* In */
+	__u64 uaddr;				/* In */
+	__u32 len;				/* In/Out */
+	__u32 handle;				/* In */
+};
+
 /**
  * struct sev_issue_cmd - SEV ioctl parameters
  *

2.27.0

Hi Shirong,

SEV (-ES) is not designed to support the runtime attestation. Still, your approach here somehow gives the impression to the guest application that it's getting the runtime attestation report from the hardware. I am not sure if it's a good idea.

In your above example, what stops KVM from providing a wrong handle on step #2. How does the guest owner (=customer) know that it is getting the report from their VM? Maybe one way to create an association is for the guest owner to inject a nonce during the launch flow, and the guest application uses this nonce to request the report once.

Alternatively, you can implement a virtual device that can be used by guest applications to request the attestation report from the VMM. In this approach, the VMM can emulate virtual device, and on read, it can call down to PSP to get the attestation report. Now it all starts sounding like a vTPM ;)

thanks