Skip to contentSkip to conversation

Questions about the usage of the vTPM implemented in Xen 4.3

PATCH In review Jordi Cucurull Juan 3 participants 12 messages Feb 5, 2014 – Feb 13, 2014

lore
Share & tools
lore
#0 lore raw

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

2.In the documentation it is recommended to avoid accessing the physical TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0 without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain? With solutions such as TBOOT and IntelTXT?

Best regards,
Jordi.

#1 lore raw

CCing the vTPM maintainer.

Quoted 6 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 6 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

I don't know much about vTPM but this seems odd to me. Which toolstack are you using? Can you provide details of your config and logs from both the startup and shutdown etc please.

I've no clue about #2 or #3 I'm afraid.

Quoted 19 lines

2.In the documentation it is recommended to avoid accessing the physical TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0 without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain? With solutions such as TBOOT and IntelTXT?

Best regards,
Jordi.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#2 lore raw

Hello Ian,

I am using the "xl" toolstack. I have included the configuration and screen logs of the vTPM-Mgr stub domain, vTPM stub domain and DomU.

As you can see in the logs, I have enabled the vTPM Mgr and vTPM stub domains once. Then I have enabled the DomU two consecutive times without disconnecting the stub domains (in all the cases issuing the command "xl create -c /var/xen/configuration.cfg).

When the DomU shuts down (after issuing a poweroff command with an ssh connection) the vTPM stub domain does not stop. Instead the following entries appear on its log:

Tpmback:Info Frontend 14/0 disconnected^M Failed to read /local/domain/14/device/vtpm/0/state.^M Tpmback:Info Frontend 14/0 disconnected^M

and later, when the DomU is started again:

Tpmback:Info Frontend 15/0 connected^M

In addition, one can see that the measurements performed by the "pv-grub" differ from the first to the second boot of the DomU (since the vTPM domain instance has been kept alive):

[root@localhost ~]# cat /sys/class/misc/tpm0/device/pcrs
...
PCR-04: 5A 4D CA AA C4 90 19 78 9A CB 7A C9 87 A6 08 A8 7C A2 7B DB
PCR-05: E5 6C FC F9 65 D2 D0 FC 7A 24 7F 42 66 28 D5 F9 D3 10 EF 72
...

[root@localhost ~]# cat /sys/class/misc/tpm0/device/pcrs
...
PCR-04: BB 67 AA F3 9E B6 4B 8F 7E 76 57 7A 16 14 FB 0C B2 57 DF 69
PCR-05: C0 A5 04 68 85 93 1B CD AE 61 F7 DA 49 ED 72 9E 2E D7 06 F0
...

Does anybody know if this is the expected behaviour? Can this be changed?

Thanks!
Jordi.

Quoted 15 lines · Ian Campbell

CCing the vTPM maintainer.

Quoted 6 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 6 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

Quoted 26 lines

I don't know much about vTPM but this seems odd to me. Which toolstack are you using? Can you provide details of your config and logs from both the startup and shutdown etc please.

I've no clue about #2 or #3 I'm afraid.

Quoted 19 lines

2.In the documentation it is recommended to avoid accessing the physical TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0 without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain? With solutions such as TBOOT and IntelTXT?

Best regards,
Jordi.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

# Configuration of pv-grub
kernel = "/usr/local/lib/xen/boot/pv-grub-x86_64.gz" extra= "(hd0,0)/grub/grub.conf"

# Configuration of guest
name = "virtual1"
memory = "512"
disk = [ 'tap:aio:/var/xen/virtual1/virtual1.img,xvda,w' ] vif = [ 'mac=00:16:3E:5C:48:A2,ip=10.0.0.1' ] vcpus=1
on_reboot = 'destroy'
on_crash = 'destroy'
vtpm=["backend=domu-vtpm1"]

kernel="/usr/local/lib/xen/boot/vtpm-stubdom.gz" memory=8 disk=["file:/home/jcucurull/Xen/virtual1/vtpm.img,hda,w"] name="domu-vtpm1" vtpm=["backend=vtpmmgr,uuid=b85cd52c-d39c-4364-9306-2bfa476be2e2"] extra="hwinitpcr=none"

kernel="/usr/local/lib/xen/boot/vtpmmgr-stubdom.gz" memory=16 disk=["file:/var/xen/vtpmmgr-stubdom.img,hda,w"] name="vtpmmgr"
iomem=["fed40,5"]

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#3 lore raw
In reply to Jordi Cucurull Juan
Quoted 6 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 6 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

The documentation is in error here; while this was originally how the vTPM domain behaved, this automatic shutdown was not reliable: it was not done if the peer domain did not use the vTPM, and it was incorrectly triggered by pv-grub's use of the vTPM to record guest kernel measurements (which was the immediate reason for its removal). The solution now is to either send a shutdown request or simply destroy the vTPM upon guest shutdown.

An alternative that may require less work on your part is to destroy the vTPM stub domain during a guest's construction, something like:

#!/bin/sh -e
xl destroy "$1-vtpm" || true
xl create $1-vtpm.cfg
xl create $1-domu.cfg

Allowing a vTPM to remain active across a guest restart will cause the PCR values extended by pv-grub to be incorrect, as you observed in your second email. In order for the vTPM's PCRs to be useful for quotes or releasing sealed secrets, you need to ensure that a new vTPM is started if and only if it is paired with a corresponding guest.

Quoted 5 lines

2.In the documentation it is recommended to avoid accessing the physical TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0 without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

While most of the time it is not a problem to have two entities talking to the physical TPM, it is possible for the trousers daemon in dom0 to interfere with key handles used by the TPM Manager. There are also certain operations of the TPM that may not handle concurrency, although I do not believe that trousers uses them - SHA1Start, the DAA commands, and certain audit logs come to mind.

The other reason why it is recommended to avoid pTPM access from dom0 is because the ability to send unseal/unbind requests to the physical TPM makes it possible for applications running in dom0 to decrypt the TPM Manager's data (and thereby access vTPM private keys).

At present, sharing the physical TPM between dom0 and the TPM Manager is the only way to get full integrity checks.

Quoted 3 lines

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain? With solutions such as TBOOT and IntelTXT?

While the TPM Manager in Xen 4.3/4.4 does not yet have this functionality, an update which I will be submitting for inclusion in Xen 4.5 has the ability to get physical TPM quotes using a virtual TPM. Combined with an early domain builder, the eventual goal is to have dom0 use a vTPM for its integrity/reporting/sealing operations, and use the physical TPM only to secure the secrets of vTPMs and for deep quotes to provide fresh proofs of the system's state.

Quoted 2 lines

Best regards,
Jordi.

Signature · 2 lines
Daniel De Graaf
National Security Agency
#4 lore raw
Quoted 7 lines · Daniel De Graaf
Quoted 6 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 8 lines
Quoted 6 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

The documentation is in error here;

Can you send a patch please.

Ian.

#5 lore raw

PATCH 1/1 docs/vtpm: fix auto-shutdown reference

Quoted 15 lines · Ian Campbell
Quoted 14 lines · Daniel De Graaf
Quoted 12 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

Quoted 6 lines
Quoted 1 line

The documentation is in error here;

Can you send a patch please.

Ian.

Patch below.

------------------------->8--------------------------------------

The automatic shutdown feature of the vTPM was removed because it interfered with pv-grub measurement support and was also not triggered if the guest did not use the vTPM. Virtual TPM domains will need to be shut down or destroyed on guest shutdown via a script or other user action.

This also fixes an incorrect reference to the vTPM being PV-only.

---
 docs/misc/vtpm.txt | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)
docs/misc/vtpm.txt +3 -9
index b8979a3..df1dfae 100644
@@ -234,7 +234,7 @@ the Linux tpmfront driver. Add the following line:
 
 vtpm=["backend=domu-vtpm"]
 
-Currently only paravirtualized guests are supported.
+Currently only Linux guests are supported (PV or HVM with PV drivers).
 
 Launching and shut down:
 ------------------------
@@ -280,14 +280,8 @@ You should also see the command being sent to the vtpm console as well
 as the vtpm saving its state. You should see the vtpm key being
 encrypted and stored on the vtpmmgr console.
 
-To shutdown the guest and its vtpm, you just have to shutdown the guest
-normally. As soon as the guest vm disconnects, the vtpm will shut itself
-down automatically.
-
-On guest:
-# shutdown -h now
-
-You may wish to write a script to start your vtpm and guest together.
+You may wish to write a script to start your vtpm and guest together and
+to destroy the vtpm when the guest shuts down.
 
 ------------------------------
 INTEGRATION WITH PV-GRUB
Signature · 1 line
1.8.5.3
Acked-by on v1 1/1
#6 lore raw

Re: [PATCH] docs/vtpm: fix auto-shutdown reference

Quoted 16 lines · Daniel De Graaf
Quoted 15 lines · Ian Campbell
Quoted 14 lines · Daniel De Graaf
Quoted 12 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

The documentation is in error here;

Can you send a patch please.

Ian.

Patch below.

Thanks.

Quoted 11 lines

------------------------->8--------------------------------------

The automatic shutdown feature of the vTPM was removed because it interfered with pv-grub measurement support and was also not triggered if the guest did not use the vTPM. Virtual TPM domains will need to be shut down or destroyed on guest shutdown via a script or other user action.

This also fixes an incorrect reference to the vTPM being PV-only.

I'm holding off committing while preparations are made for another rc, but once that is out the way I see no reason to hold off on this.

Quoted 34 lines
---
  docs/misc/vtpm.txt | 12 +++---------
  1 file changed, 3 insertions(+), 9 deletions(-)
docs/misc/vtpm.txt +3 -9
index b8979a3..df1dfae 100644
@@ -234,7 +234,7 @@ the Linux tpmfront driver. Add the following line:
 
  vtpm=["backend=domu-vtpm"]
 
-Currently only paravirtualized guests are supported.
+Currently only Linux guests are supported (PV or HVM with PV drivers).
 
  Launching and shut down:
  ------------------------
@@ -280,14 +280,8 @@ You should also see the command being sent to the vtpm console as well
  as the vtpm saving its state. You should see the vtpm key being
  encrypted and stored on the vtpmmgr console.
 
-To shutdown the guest and its vtpm, you just have to shutdown the guest
-normally. As soon as the guest vm disconnects, the vtpm will shut itself
-down automatically.
-
-On guest:
-# shutdown -h now
-
-You may wish to write a script to start your vtpm and guest together.
+You may wish to write a script to start your vtpm and guest together and
+to destroy the vtpm when the guest shuts down.
 
  ------------------------------
  INTEGRATION WITH PV-GRUB
#7 lore raw

Re: [PATCH] docs/vtpm: fix auto-shutdown reference

Quoted 8 lines · Ian Campbell
Quoted 2 lines · Daniel De Graaf

I'm holding off committing while preparations are made for another rc, but once that is out the way I see no reason to hold off on this.

I've applied this. It looks like you have some whitespace differences in your baseline (everything was indented one column, some other patch in your queue perhaps?) which I resolved as I went.

Ian.

#8 lore raw
In reply to Daniel De Graaf

Hello Daniel,

Thanks for your thorough answer. I have a few comments below.

Quoted 32 lines · Daniel De Graaf
Quoted 6 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 6 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

The documentation is in error here; while this was originally how the vTPM
domain behaved, this automatic shutdown was not reliable: it was not done if the peer domain did not use the vTPM, and it was incorrectly triggered by pv-grub's use of the vTPM to record guest kernel measurements (which was
the immediate reason for its removal). The solution now is to either send a
shutdown request or simply destroy the vTPM upon guest shutdown.

An alternative that may require less work on your part is to destroy the vTPM stub domain during a guest's construction, something like:

#!/bin/sh -e
xl destroy "$1-vtpm" || true
xl create $1-vtpm.cfg
xl create $1-domu.cfg

Quoted 5 lines

Allowing a vTPM to remain active across a guest restart will cause the PCR values extended by pv-grub to be incorrect, as you observed in your second email. In order for the vTPM's PCRs to be useful for quotes or releasing sealed secrets, you need to ensure that a new vTPM is started if and only if it is paired with a corresponding guest.

I see a potential threat due to this behaviour (please correct me if I am wrong).

Assume an administrator of Dom0 becomes malicious. Since the hypervisor does not enforce the shut down of the vTPM domain, the malicious administrator could try the following: 1) make a copy of the peer domain, 2) manipulate the copy of the peer domain and disable its measurements, 3) boot the original peer domain, 4) switch it off or pause it, 5) boot the manipulated copy of the peer domain.

Then, the shown PCR values of the manipulated copy of the peer domain are the ones measured by the original peer domain during the first boot. But the manipulated copy is the one actually running. Hence, this could not be detected nor by quoting the vTPM neither the pTPM.

May be, one possible solution could be to enforce an XSM FLASK policy to prevent any user in Dom0 from destroying, shutting down or pausing a domain. Then, measure the policy when Dom0 starts into a PCR of the phsyical TPM. Nevertheless, on one hand I do not know if this is feasible and, in the other hand, this prevents the system from destroying the vTPM domain when the peer domain shuts down.

Quoted 18 lines
Quoted 6 lines

2.In the documentation it is recommended to avoid accessing the physical TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0 without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

While most of the time it is not a problem to have two entities talking to
the physical TPM, it is possible for the trousers daemon in dom0 to interfere
with key handles used by the TPM Manager. There are also certain operations
of the TPM that may not handle concurrency, although I do not believe that
trousers uses them - SHA1Start, the DAA commands, and certain audit logs come to mind.

Quoted 8 lines

The other reason why it is recommended to avoid pTPM access from dom0 is because the ability to send unseal/unbind requests to the physical TPM makes
it possible for applications running in dom0 to decrypt the TPM Manager's data (and thereby access vTPM private keys).

At present, sharing the physical TPM between dom0 and the TPM Manager is the only way to get full integrity checks.

OK, I see. Hence leaving the TPM support enabled in Dom0 opens a security problem to the vTPM. But if we do not enable the support, the integrity of Dom0 cannot be proved using the TPM (e.g. by remote attestation).

Quoted 5 lines
Quoted 4 lines

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain? With solutions such as TBOOT and IntelTXT?

Quoted 9 lines

While the TPM Manager in Xen 4.3/4.4 does not yet have this functionality,
an update which I will be submitting for inclusion in Xen 4.5 has the ability to get physical TPM quotes using a virtual TPM. Combined with an early domain builder, the eventual goal is to have dom0 use a vTPM for its integrity/reporting/sealing operations, and use the physical TPM only to secure the secrets of vTPMs and for deep quotes to provide fresh proofs
of the system's state.

This sounds really good. I look forward to try it in Xen 4.5!!

Thank you for your answers!
Jordi.

Quoted 4 lines
Quoted 3 lines

Best regards,
Jordi.

#9 lore raw
Quoted 53 lines · Jordi Cucurull Juan

Hello Daniel,

Thanks for your thorough answer. I have a few comments below.

Quoted 32 lines · Daniel De Graaf
Quoted 6 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 6 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

The documentation is in error here; while this was originally how the vTPM
domain behaved, this automatic shutdown was not reliable: it was not done if the peer domain did not use the vTPM, and it was incorrectly triggered by pv-grub's use of the vTPM to record guest kernel measurements (which was
the immediate reason for its removal). The solution now is to either send a
shutdown request or simply destroy the vTPM upon guest shutdown.

An alternative that may require less work on your part is to destroy the vTPM stub domain during a guest's construction, something like:

#!/bin/sh -e
xl destroy "$1-vtpm" || true
xl create $1-vtpm.cfg
xl create $1-domu.cfg

Quoted 5 lines

Allowing a vTPM to remain active across a guest restart will cause the PCR values extended by pv-grub to be incorrect, as you observed in your second email. In order for the vTPM's PCRs to be useful for quotes or releasing sealed secrets, you need to ensure that a new vTPM is started if and only if it is paired with a corresponding guest.

I see a potential threat due to this behaviour (please correct me if I am wrong).

Assume an administrator of Dom0 becomes malicious. Since the hypervisor does not enforce the shut down of the vTPM domain, the malicious administrator could try the following: 1) make a copy of the peer domain, 2) manipulate the copy of the peer domain and disable its measurements, 3) boot the original peer domain, 4) switch it off or pause it, 5) boot the manipulated copy of the peer domain.

Quoted 5 lines

Then, the shown PCR values of the manipulated copy of the peer domain are the ones measured by the original peer domain during the first boot. But the manipulated copy is the one actually running. Hence, this could not be detected nor by quoting the vTPM neither the pTPM.

A malicious dom0 has a much simpler attack vector: start the domain with a custom version of pv-grub that extends arbitrary measurements instead of the real kernel's measurements. Then, a user kernel with disabled or similarly false measuring capabilities can be booted. Alternatively, if XSM polices do not restrict it, a debugger could be attached to the guest so that it can be manipulated online.

Quoted 6 lines

May be, one possible solution could be to enforce an XSM FLASK policy to prevent any user in Dom0 from destroying, shutting down or pausing a domain. Then, measure the policy when Dom0 starts into a PCR of the phsyical TPM. Nevertheless, on one hand I do not know if this is feasible and, in the other hand, this prevents the system from destroying the vTPM domain when the peer domain shuts down.

The solution to this problem is to disaggregate dom0 and relocate the domain building component to a stub domain that is completely measured in the pTPM (perhaps by TBOOT). The domain builder could use a static library of domains to build (hardware domain and TPM Manager built only once; vTPM and pv-grub domain pairs built on request). An XSM policy could then restrict vTPM communication so that only correctly built guests are allowed to talk to their paired vTPM. In this case, dom0 would have permission to shut down either VM, but could not start a replacement.

Quoted 23 lines
Quoted 23 lines
Quoted 5 lines

2.In the documentation it is recommended to avoid accessing the physical TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0 without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

While most of the time it is not a problem to have two entities talking to
the physical TPM, it is possible for the trousers daemon in dom0 to interfere
with key handles used by the TPM Manager. There are also certain operations
of the TPM that may not handle concurrency, although I do not believe that
trousers uses them - SHA1Start, the DAA commands, and certain audit logs come to mind.

The other reason why it is recommended to avoid pTPM access from dom0 is because the ability to send unseal/unbind requests to the physical TPM makes
it possible for applications running in dom0 to decrypt the TPM Manager's data (and thereby access vTPM private keys).

Quoted 7 lines
Quoted 2 lines

At present, sharing the physical TPM between dom0 and the TPM Manager is the only way to get full integrity checks.

OK, I see. Hence leaving the TPM support enabled in Dom0 opens a security problem to the vTPM. But if we do not enable the support, the integrity of Dom0 cannot be proved using the TPM (e.g. by remote attestation).

Right. Since dom0 currently must be trusted (as discussed above) this is currently the best way to handle the dom0 attestation problem.

Quoted 21 lines
Quoted 5 lines
Quoted 4 lines

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain? With solutions such as TBOOT and IntelTXT?

Quoted 9 lines

While the TPM Manager in Xen 4.3/4.4 does not yet have this functionality,
an update which I will be submitting for inclusion in Xen 4.5 has the ability to get physical TPM quotes using a virtual TPM. Combined with an early domain builder, the eventual goal is to have dom0 use a vTPM for its integrity/reporting/sealing operations, and use the physical TPM only to secure the secrets of vTPMs and for deep quotes to provide fresh proofs
of the system's state.

This sounds really good. I look forward to try it in Xen 4.5!!

Thank you for your answers!
Jordi.

Signature · 2 lines
Daniel De Graaf
National Security Agency
#10 lore raw

Hello Daniel,

Quoted 64 lines · Daniel De Graaf
Quoted 57 lines · Jordi Cucurull Juan

Hello Daniel,

Thanks for your thorough answer. I have a few comments below.

Quoted 36 lines · Daniel De Graaf
Quoted 7 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a
guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 7 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown
the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

The documentation is in error here; while this was originally how the vTPM
domain behaved, this automatic shutdown was not reliable: it was not done
if the peer domain did not use the vTPM, and it was incorrectly triggered
by pv-grub's use of the vTPM to record guest kernel measurements (which was
the immediate reason for its removal). The solution now is to either send a
shutdown request or simply destroy the vTPM upon guest shutdown.

An alternative that may require less work on your part is to destroy the vTPM stub domain during a guest's construction, something like:

#!/bin/sh -e
xl destroy "$1-vtpm" || true
xl create $1-vtpm.cfg
xl create $1-domu.cfg

Quoted 5 lines

Allowing a vTPM to remain active across a guest restart will cause the PCR values extended by pv-grub to be incorrect, as you observed in your second email. In order for the vTPM's PCRs to be useful for quotes or releasing sealed secrets, you need to ensure that a new vTPM is started if and only if it is paired with a corresponding guest.

I see a potential threat due to this behaviour (please correct me if I am wrong).

Assume an administrator of Dom0 becomes malicious. Since the hypervisor does not enforce the shut down of the vTPM domain, the malicious administrator could try the following: 1) make a copy of the peer domain, 2) manipulate the copy of the peer domain and disable its measurements, 3) boot the original peer domain, 4) switch it off or pause it, 5) boot the manipulated copy of the peer domain.

Quoted 5 lines

Then, the shown PCR values of the manipulated copy of the peer domain are the ones measured by the original peer domain during the first boot. But the manipulated copy is the one actually running. Hence, this could not be detected nor by quoting the vTPM neither the pTPM.

Quoted 6 lines

A malicious dom0 has a much simpler attack vector: start the domain with a custom version of pv-grub that extends arbitrary measurements instead of the real kernel's measurements. Then, a user kernel with disabled or similarly false measuring capabilities can be booted. Alternatively, if XSM polices do not restrict it, a debugger could be attached to the guest so that it can be manipulated online.

This is the reason why I wanted to measure Dom0, to detect a possible manipulation, e.g. of a custom version of pv-grub. Nevertheless, still the administrator could try to inject a manipulated copy of it into the system after booting it. Hence, I agree with the solutions you propose below.

Quoted 8 lines
Quoted 7 lines

May be, one possible solution could be to enforce an XSM FLASK policy to prevent any user in Dom0 from destroying, shutting down or pausing a domain. Then, measure the policy when Dom0 starts into a PCR of the phsyical TPM. Nevertheless, on one hand I do not know if this is feasible and, in the other hand, this prevents the system from destroying the vTPM domain when the peer domain shuts down.

Quoted 10 lines

The solution to this problem is to disaggregate dom0 and relocate the domain building component to a stub domain that is completely measured in the pTPM (perhaps by TBOOT). The domain builder could use a static library of domains to build (hardware domain and TPM Manager built only once; vTPM and pv-grub domain pairs built on request). An XSM policy could then restrict vTPM communication so that only correctly built guests are allowed to talk to their paired vTPM. In this case, dom0 would have permission to shut down either VM, but could not start a replacement.

I understand this cannot be done with the current implementation of Xen. Are there any plans to do this in the future?

Quoted 66 lines
Quoted 28 lines
Quoted 28 lines
Quoted 7 lines

2.In the documentation it is recommended to avoid accessing the physical
TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0
without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

While most of the time it is not a problem to have two entities talking to
the physical TPM, it is possible for the trousers daemon in dom0 to interfere
with key handles used by the TPM Manager. There are also certain operations
of the TPM that may not handle concurrency, although I do not believe that
trousers uses them - SHA1Start, the DAA commands, and certain audit logs
come to mind.

The other reason why it is recommended to avoid pTPM access from dom0 is
because the ability to send unseal/unbind requests to the physical TPM makes
it possible for applications running in dom0 to decrypt the TPM Manager's
data (and thereby access vTPM private keys).

Quoted 8 lines
Quoted 3 lines

At present, sharing the physical TPM between dom0 and the TPM Manager is
the only way to get full integrity checks.

OK, I see. Hence leaving the TPM support enabled in Dom0 opens a security problem to the vTPM. But if we do not enable the support, the integrity of Dom0 cannot be proved using the TPM (e.g. by remote attestation).

Right. Since dom0 currently must be trusted (as discussed above) this is currently the best way to handle the dom0 attestation problem.

Quoted 24 lines
Quoted 6 lines
Quoted 5 lines

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain?
With solutions such as TBOOT and IntelTXT?

Quoted 11 lines

While the TPM Manager in Xen 4.3/4.4 does not yet have this functionality,
an update which I will be submitting for inclusion in Xen 4.5 has the ability to get physical TPM quotes using a virtual TPM. Combined with an
early domain builder, the eventual goal is to have dom0 use a vTPM for its integrity/reporting/sealing operations, and use the physical TPM only
to secure the secrets of vTPMs and for deep quotes to provide fresh proofs
of the system's state.

This sounds really good. I look forward to try it in Xen 4.5!!

Thank you for your answers!
Jordi.

Signature · 8 lines
Jordi Cucurull Juan
Researcher
Scytl Secure Electronic Voting
Plaça Gal·la Placidia, 1-3, 1st floor · 08006 Barcelona
Phone:     + 34 934 230 324
Fax        + 34 933 251 028
[email protected]
http://www.scytl.com

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#11 lore raw
Quoted 99 lines · Jordi Cucurull Juan

Hello Daniel,

Quoted 64 lines · Daniel De Graaf
Quoted 57 lines · Jordi Cucurull Juan

Hello Daniel,

Thanks for your thorough answer. I have a few comments below.

Quoted 36 lines · Daniel De Graaf
Quoted 7 lines · Jordi Cucurull Juan

Dear all,

I have recently configured a Xen 4.3 server with the vTPM enabled and a
guest virtual machine that takes advantage of it. After playing a bit with it, I have a few questions:

Quoted 7 lines

1.According to the documentation, to shutdown the vTPM stubdom it is only needed to normally shutdown the guest VM. Theoretically, the vTPM stubdom automatically shuts down after this. Nevertheless, if I shutdown
the guest the vTPM stubdom continues active and, moreover, I can start the machine again and the values of the vTPM are the last ones there were in the previous instance of the guest. Is this normal?

The documentation is in error here; while this was originally how the vTPM
domain behaved, this automatic shutdown was not reliable: it was not done
if the peer domain did not use the vTPM, and it was incorrectly triggered
by pv-grub's use of the vTPM to record guest kernel measurements (which was
the immediate reason for its removal). The solution now is to either send a
shutdown request or simply destroy the vTPM upon guest shutdown.

An alternative that may require less work on your part is to destroy the vTPM stub domain during a guest's construction, something like:

#!/bin/sh -e
xl destroy "$1-vtpm" || true
xl create $1-vtpm.cfg
xl create $1-domu.cfg

Quoted 5 lines

Allowing a vTPM to remain active across a guest restart will cause the PCR values extended by pv-grub to be incorrect, as you observed in your second email. In order for the vTPM's PCRs to be useful for quotes or releasing sealed secrets, you need to ensure that a new vTPM is started if and only if it is paired with a corresponding guest.

I see a potential threat due to this behaviour (please correct me if I am wrong).

Assume an administrator of Dom0 becomes malicious. Since the hypervisor does not enforce the shut down of the vTPM domain, the malicious administrator could try the following: 1) make a copy of the peer domain, 2) manipulate the copy of the peer domain and disable its measurements, 3) boot the original peer domain, 4) switch it off or pause it, 5) boot the manipulated copy of the peer domain.

Quoted 5 lines

Then, the shown PCR values of the manipulated copy of the peer domain are the ones measured by the original peer domain during the first boot. But the manipulated copy is the one actually running. Hence, this could not be detected nor by quoting the vTPM neither the pTPM.

Quoted 6 lines

A malicious dom0 has a much simpler attack vector: start the domain with a custom version of pv-grub that extends arbitrary measurements instead of the real kernel's measurements. Then, a user kernel with disabled or similarly false measuring capabilities can be booted. Alternatively, if XSM polices do not restrict it, a debugger could be attached to the guest so that it can be manipulated online.

This is the reason why I wanted to measure Dom0, to detect a possible manipulation, e.g. of a custom version of pv-grub. Nevertheless, still the administrator could try to inject a manipulated copy of it into the system after booting it. Hence, I agree with the solutions you propose below.

Quoted 8 lines
Quoted 7 lines

May be, one possible solution could be to enforce an XSM FLASK policy to prevent any user in Dom0 from destroying, shutting down or pausing a domain. Then, measure the policy when Dom0 starts into a PCR of the phsyical TPM. Nevertheless, on one hand I do not know if this is feasible and, in the other hand, this prevents the system from destroying the vTPM domain when the peer domain shuts down.

Quoted 10 lines

The solution to this problem is to disaggregate dom0 and relocate the domain building component to a stub domain that is completely measured in the pTPM (perhaps by TBOOT). The domain builder could use a static library of domains to build (hardware domain and TPM Manager built only once; vTPM and pv-grub domain pairs built on request). An XSM policy could then restrict vTPM communication so that only correctly built guests are allowed to talk to their paired vTPM. In this case, dom0 would have permission to shut down either VM, but could not start a replacement.

Quoted 2 lines

I understand this cannot be done with the current implementation of Xen. Are there any plans to do this in the future?

Yes, although I am not sure how easy it will be to upstream the changes. The hypervisor changes to make the hardware domain distinct from dom0 are mostly present in 4.4 - it just lacks the hooks to postpone IOMMU setup for the hardware domain instead of dom0. I plan to post these for review once 4.4 is branched.

The domain builder needed to run this setup also exists but the version which handles build requests depends on an out-of-tree patch to the hypervisor implementing inter-domain message sending (IVC). Since V4V is more feature-complete than IVC, I think changing our domain builder to use V4V instead of IVC will be necessary for it to be incorporated into Xen. The bootstrapping version of the domain builder (which would be run with domain ID 0) does not depend on IVC, but only builds a single list of domains from its ramdisk.

Quoted 68 lines
Quoted 66 lines
Quoted 28 lines
Quoted 28 lines
Quoted 7 lines

2.In the documentation it is recommended to avoid accessing the physical
TPM from Dom0 at the same time than the vTPM Manager stubdom. Nevertheless, I currently have the IMA and the Trousers enabled in Dom0
without any apparent issue. Why is not recommended directly accessing the physical TPM of Dom0?

While most of the time it is not a problem to have two entities talking to
the physical TPM, it is possible for the trousers daemon in dom0 to interfere
with key handles used by the TPM Manager. There are also certain operations
of the TPM that may not handle concurrency, although I do not believe that
trousers uses them - SHA1Start, the DAA commands, and certain audit logs
come to mind.

The other reason why it is recommended to avoid pTPM access from dom0 is
because the ability to send unseal/unbind requests to the physical TPM makes
it possible for applications running in dom0 to decrypt the TPM Manager's
data (and thereby access vTPM private keys).

Quoted 8 lines
Quoted 3 lines

At present, sharing the physical TPM between dom0 and the TPM Manager is
the only way to get full integrity checks.

OK, I see. Hence leaving the TPM support enabled in Dom0 opens a security problem to the vTPM. But if we do not enable the support, the integrity of Dom0 cannot be proved using the TPM (e.g. by remote attestation).

Right. Since dom0 currently must be trusted (as discussed above) this is currently the best way to handle the dom0 attestation problem.

Quoted 24 lines
Quoted 6 lines
Quoted 5 lines

3.If it is not recommended to directly accessing the physical TPM in Dom0, which is the advisable way to check the integrity of this domain?
With solutions such as TBOOT and IntelTXT?

Quoted 11 lines

While the TPM Manager in Xen 4.3/4.4 does not yet have this functionality,
an update which I will be submitting for inclusion in Xen 4.5 has the ability to get physical TPM quotes using a virtual TPM. Combined with an
early domain builder, the eventual goal is to have dom0 use a vTPM for its integrity/reporting/sealing operations, and use the physical TPM only
to secure the secrets of vTPMs and for deep quotes to provide fresh proofs
of the system's state.

This sounds really good. I look forward to try it in Xen 4.5!!

Thank you for your answers!
Jordi.

Signature · 2 lines
Daniel De Graaf
National Security Agency