PCI / iommu / thunderbolt: IOMMU based DMA protection
Subsystems (5)
v2 cover PCI / iommu / thunderbolt: IOMMU based DMA protection
Recent systems shipping with Windows 10 version 1803 or newer may be utilizing IOMMU to prevent DMA attacks via Thunderbolt ports. This is different from the previous security level based scheme because the connected device cannot access system memory outside of the regions allocated for it by the driver.
When enabled the BIOS makes sure no device can do DMA outside of RMRR (Reserved Memory Region Record) regions. This means that during OS boot, before it enables IOMMU, none of the connected devices can bypass DMA protection for instance by overwriting the data structures used by the IOMMU. The BIOS communicates support for this to the OS by setting a new bit in ACPI DMAR table [1].
Because these systems utilize an IOMMU to block possible DMA attacks, typically (but not always) the Thunderbolt security level is set to "none" which means that all PCIe devices are immediately usable. This also means that Linux needs to follow Windows 10 and enable IOMMU automatically when running on such system otherwise connected devices can read/write system memory pretty much without any restrictions.
Since there is a way to identify PCIe root ports that are "external facing" we can put all internal devices to pass through (identity mapping) mode and only external devices need to go through full IOMMU mappings. We add a new flag "is_untrusted" that is supposed to cover various PCIe devices that may be used to conduct DMA attacks.
We also make sure PCIe ATS (Address Translation Service) is not enabled for devices flagged as untrusted because it could be used to bypass IOMMU completely as explained in the changelog of patch 3/4.
Finally we expose this information to userspace so tools such as bolt can do more accurate decision whether or not authorize the connected device.
I can take these through Thunderbolt tree assuming Bjorn and Rafael are fine with these.
[1] https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf
Previous version of the patch series can be found here:
https://www.spinics.net/lists/linux-pci/msg77751.html
Changes from v1:
* Reword Documentation/admin-guide/thunderbolt.rst to make the feature time frame/platform oriented as there will be systems shipping with Linux installed by default.
* Rename the flag is_external to is_untrusted so that we could use the same flag to cover all kinds of "untrusted" PCI devices, not just Thunderbolt connected devices. I still parse the _DSD in PCI/ACPI core because that's where we currently handle "HotPlugSupportInD3" as well. Also updated comments in patch [1/4].
* Added tags from Ashok, Joerg and Yehezkel. I'm assuming they still apply because I did not change the code with the exception of few comments and rename of the flag. Let me know if that's not the case anymore.
Lu Baolu (1):
iommu/vt-d: Force IOMMU on for platform opt in hint
Mika Westerberg (3):
PCI / ACPI: Identify untrusted PCI devices
iommu/vt-d: Do not enable ATS for untrusted devices
thunderbolt: Export IOMMU based DMA protection support to userspace
.../ABI/testing/sysfs-bus-thunderbolt | 9 +++ Documentation/admin-guide/thunderbolt.rst | 20 +++++++ drivers/acpi/property.c | 3 + drivers/iommu/dmar.c | 25 +++++++++ drivers/iommu/intel-iommu.c | 56 ++++++++++++++++++- drivers/pci/pci-acpi.c | 18 ++++++ drivers/pci/probe.c | 22 ++++++++ drivers/thunderbolt/domain.c | 17 ++++++ include/linux/dmar.h | 8 +++ include/linux/pci.h | 8 +++ 10 files changed, 183 insertions(+), 3 deletions(-)
Signature · 1 line
2.19.1
PATCH v2 1/4 PCI / ACPI: Identify untrusted PCI devices
Recent systems with Thunderbolt ports may support IOMMU natively. This means that the platform utilizes IOMMU to prevent DMA attacks over externally exposed PCIe root ports (typically Thunderbolt ports)
The system BIOS marks these PCIe root ports as being externally facing ports by implementing following ACPI _DSD [1] under the root port in question:
Name (_DSD, Package () {
ToUUID ("efcc06cc-73ac-4bc3-bff0-76143807c389"),
Package () {
Package () {"ExternalFacingPort", 1},
Package () {"UID", 0 }
}})
To make it possible for IOMMU code to identify these devices, we look up for this property and mark all children devices (including the root port itself) with a new flag (->is_untrusted). This flag is meant to be used with all PCIe devices (not just Thunderbolt) that cannot be trusted in the same level than integrated devices and may need to put behind full IOMMU protection.
--- drivers/acpi/property.c | 3 +++ drivers/pci/pci-acpi.c | 18 ++++++++++++++++++ drivers/pci/probe.c | 22 ++++++++++++++++++++++ include/linux/pci.h | 8 ++++++++ 4 files changed, 51 insertions(+)
drivers/acpi/property.c +3 -0
@@ -31,6 +31,9 @@ static const guid_t prp_guids[] = {/* Hotplug in D3 GUID: 6211e2c0-58a3-4af3-90e1-927a4e0c55a4 */ GUID_INIT (0x6211e2c0 ,0x58a3 ,0x4af3 ,0x90 ,0xe1 ,0x92 ,0x7a ,0x4e ,0x0c ,0x55 ,0xa4 ), +/* External facing port GUID: efcc06cc-73ac-4bc3-bff0-76143807c389 */ +GUID_INIT (0xefcc06cc ,0x73ac ,0x4bc3 , + 0xbf, 0xf0, 0x76, 0x14, 0x38, 0x07, 0xc3, 0x89), }; static const guid_t ads_guid =
drivers/pci/pci-acpi.c +18 -0
@@ -789,6 +789,23 @@ static void pci_acpi_optimize_delay(struct pci_dev *pdev,ACPI_FREE (obj); } +static void pci_acpi_set_untrusted (struct pci_dev *dev) +{ +u8 val; + +if (pci_pcie_type (dev) !=PCI_EXP_TYPE_ROOT_PORT ) +return ; +if (device_property_read_u8 (&dev->dev,"ExternalFacingPort" , &val)) +return ; + +/* +* These root ports expose PCIe (including DMA) outside of the +* system so make sure we treat them and everything behind as +* untrusted. +*/ + dev->is_untrusted = val ==1 ; +} +static void pci_acpi_setup (struct device *dev) {struct pci_dev *pci_dev =to_pci_dev (dev);
@@ -798,6 +815,7 @@ static void pci_acpi_setup(struct device *dev)return ;pci_acpi_optimize_delay (pci_dev, adev->handle); +pci_acpi_set_untrusted (pci_dev);pci_acpi_add_pm_notifier (adev, pci_dev);if (!adev->wakeup.flags.valid)
drivers/pci/probe.c +22 -0
@@ -1378,6 +1378,26 @@ static void set_pcie_thunderbolt(struct pci_dev *dev) } } +static void set_pcie_untrusted (struct pci_dev *dev) +{ +struct pci_dev *parent; + +/* +* Walk up the device hierarchy and check for any upstream bridge +* that has is_untrusted set to true. In that case treat this one +* untrusted as well. +*/ + parent =pci_upstream_bridge (dev); +while (parent) { +if (parent->is_untrusted) { + dev->is_untrusted = true; +break ; + } + + parent =pci_upstream_bridge (parent); + } +} +/** * pci_ext_cfg_is_aliased - Is ext config space just an alias of std config? * @dev: PCI device
@@ -1638,6 +1658,8 @@ int pci_setup_device(struct pci_dev *dev)/* Need to have dev->cfg_size ready */ set_pcie_thunderbolt (dev); +set_pcie_untrusted (dev); +/* "Unknown power state" */ dev->current_state =PCI_UNKNOWN ;
include/linux/pci.h +8 -0
@@ -396,6 +396,14 @@ struct pci_dev { unsigned int is_hotplug_bridge:1; unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ +/* +* Devices marked being untrusted are the ones that can potentially +* execute DMA attacks and similar. They are typically connected +* through external ports such as Thunderbolt but not limited to +* that. When an IOMMU is enabled they should be getting full +* mappings to make sure they cannot access arbitrary memory. +*/ + unsigned int is_untrusted:1; unsigned int __aer_firmware_first_valid:1; unsigned int __aer_firmware_first:1; unsigned int broken_intx_masking:1; /* INTx masking can't be used */
Signature · 1 line
2.19.1
Re: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
Hi Mika,
Quoted 1 line · Mika Westerberg
Recent systems with Thunderbolt ports may support IOMMU natively.
This sentence doesn't make sense to me. There's no logical connection between having an IOMMU and having a Thunderbolt port.
Quoted 3 lines
This means that the platform utilizes IOMMU to prevent DMA attacks over externally exposed PCIe root ports (typically Thunderbolt ports)
Nor this one. The platform only uses the IOMMU to prevent DMA attacks if the OS chooses to do that.
Quoted 3 lines
The system BIOS marks these PCIe root ports as being externally facing ports by implementing following ACPI
_DSD[1] under the root port in question:
There's no standard that requires this, so the best we can say is that a system BIOS *may* mark externally facing ports with this mechanism.
I think it would make sense to say something like:
A malicious PCI device may use DMA to attack the system. An external Thunderbolt port is a convenient point to attach such a device. The OS may use an IOMMU to defend against DMA attacks.
Some BIOSes mark externally facing Root Ports with the this ACPI _DSD:
...
If we find such a Root Port, mark it and all its children as "is_untrusted". The rest of the OS may use this to prevent use of the device unless we have an IOMMU to prevent malicious DMA [I don't know your actual policy yet].
Quoted 52 lines
Name (_DSD, Package () { ToUUID ("efcc06cc-73ac-4bc3-bff0-76143807c389"), Package () { Package () {"ExternalFacingPort", 1}, Package () {"UID", 0 } }})
To make it possible for IOMMU code to identify these devices, we look up for this property and mark all children devices (including the root port itself) with a new flag (->is_untrusted). This flag is meant to be used with all PCIe devices (not just Thunderbolt) that cannot be trusted in the same level than integrated devices and may need to put behind full IOMMU protection.
--- drivers/acpi/property.c | 3 +++ drivers/pci/pci-acpi.c | 18 ++++++++++++++++++ drivers/pci/probe.c | 22 ++++++++++++++++++++++ include/linux/pci.h | 8 ++++++++ 4 files changed, 51 insertions(+)drivers/acpi/property.c +3
-0@@ -31,6 +31,9 @@ static const guid_t prp_guids[] = {/* Hotplug in D3 GUID: 6211e2c0-58a3-4af3-90e1-927a4e0c55a4 */ GUID_INIT (0x6211e2c0 ,0x58a3 ,0x4af3 ,0x90 ,0xe1 ,0x92 ,0x7a ,0x4e ,0x0c ,0x55 ,0xa4 ), +/* External facing port GUID: efcc06cc-73ac-4bc3-bff0-76143807c389 */ +GUID_INIT (0xefcc06cc ,0x73ac ,0x4bc3 , + 0xbf, 0xf0, 0x76, 0x14, 0x38, 0x07, 0xc3, 0x89), }; static const guid_t ads_guid =drivers/pci/pci-acpi.c +5
-0@@ -789,6 +789,23 @@ static void pci_acpi_optimize_delay(struct pci_dev *pdev,ACPI_FREE (obj); } +static void pci_acpi_set_untrusted (struct pci_dev *dev) +{ +u8 val; + +if (pci_pcie_type (dev) !=PCI_EXP_TYPE_ROOT_PORT )
drivers/pci/pci-acpi.c static void pci_acpi_optimize_delay(struct pci_dev *pdev, · line 806
Quoted 10 lines
drivers/pci/pci-acpi.c +10
-0+return ; +if (device_property_read_u8 (&dev->dev,"ExternalFacingPort" , &val)) +return ; + +/* +* These root ports expose PCIe (including DMA) outside of the +* system so make sure we treat them and everything behind as +* untrusted. +*/ + dev->is_untrusted = val ==1 ;
Simpler to parse:
if (val)
dev->is_untrusted = 1;Quoted 29 lines
drivers/pci/pci-acpi.c +3
-0+} +static void pci_acpi_setup (struct device *dev) {struct pci_dev *pci_dev =to_pci_dev (dev);@@ -798,6 +815,7 @@ static void pci_acpi_setup(struct device *dev)return ;pci_acpi_optimize_delay (pci_dev, adev->handle); +pci_acpi_set_untrusted (pci_dev);pci_acpi_add_pm_notifier (adev, pci_dev);if (!adev->wakeup.flags.valid)drivers/pci/probe.c +8
-0@@ -1378,6 +1378,26 @@ static void set_pcie_thunderbolt(struct pci_dev *dev) } } +static void set_pcie_untrusted (struct pci_dev *dev) +{ +struct pci_dev *parent; + +/* +* Walk up the device hierarchy and check for any upstream bridge +* that has is_untrusted set to true. In that case treat this one +* untrusted as well.
drivers/pci/probe.c static void set_pcie_thunderbolt(struct pci_dev *dev) · line 1398
Quoted 10 lines
drivers/pci/probe.c +10
-0+*/ + parent =pci_upstream_bridge (dev); +while (parent) { +if (parent->is_untrusted) { + dev->is_untrusted = true; +break ; + } + + parent =pci_upstream_bridge (parent); + }
In what circumstance is this loop needed? I expected this would be sufficient:
bridge = pci_upstream_bridge(dev);
if (bridge && bridge->is_untrusted)
dev->is_untrusted = true;Quoted 20 lines
drivers/pci/probe.c +4
-0+} +/** * pci_ext_cfg_is_aliased - Is ext config space just an alias of std config? * @dev: PCI device @@ -1638,6 +1658,8 @@ int pci_setup_device(struct pci_dev *dev)/* Need to have dev->cfg_size ready */ set_pcie_thunderbolt (dev); +set_pcie_untrusted (dev); +/* "Unknown power state" */ dev->current_state =PCI_UNKNOWN ;include/linux/pci.h +0
-0@@ -396,6 +396,14 @@ struct pci_dev { unsigned int is_hotplug_bridge:1;
include/linux/pci.h struct pci_dev { · line 406
Quoted 10 lines
include/linux/pci.h +8
-0unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ +/* +* Devices marked being untrusted are the ones that can potentially +* execute DMA attacks and similar. They are typically connected +* through external ports such as Thunderbolt but not limited to +* that. When an IOMMU is enabled they should be getting full +* mappings to make sure they cannot access arbitrary memory. +*/ + unsigned int is_untrusted:1;
We have a mix of "is_X" and "X", but I think "untrusted" is sufficient since "untrusted" is a perfectly good predicate all by itself, i.e.,
if (dev->untrusted)
reads easily and makes good sense.
Quoted 6 lines
include/linux/pci.h +0
-0unsigned int __aer_firmware_first_valid:1; unsigned int __aer_firmware_first:1; unsigned int broken_intx_masking:1; /* INTx masking can't be used */Signature · 1 line
2.19.1
Re: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
Quoted 1 line · Bjorn Helgaas
Hi Mika,
Hi,
Quoted 6 lines · Mika Westerberg
Quoted 1 line
Recent systems with Thunderbolt ports may support IOMMU natively.
This sentence doesn't make sense to me. There's no logical connection between having an IOMMU and having a Thunderbolt port.
Quoted 6 lines
Quoted 3 lines
This means that the platform utilizes IOMMU to prevent DMA attacks over externally exposed PCIe root ports (typically Thunderbolt ports)
Nor this one. The platform only uses the IOMMU to prevent DMA attacks if the OS chooses to do that.
I guess I'm trying to say here that the recent changes add such support to the platform BIOS that allows the OS to enable IOMMU without being compromised by a malicious device that is already connected. The BIOS sets the new ACPI DMAR bit in that case.
Quoted 6 lines
Quoted 3 lines
The system BIOS marks these PCIe root ports as being externally facing ports by implementing following ACPI
_DSD[1] under the root port in question:There's no standard that requires this, so the best we can say is that a system BIOS *may* mark externally facing ports with this mechanism.
There is no standard but I'm quite sure this is something that will be required to be implemented properly by the OEM by Microsoft hardware compatibility suite.
Quoted 9 lines
I think it would make sense to say something like:
A malicious PCI device may use DMA to attack the system. An external Thunderbolt port is a convenient point to attach such a device. The OS may use an IOMMU to defend against DMA attacks.
Some BIOSes mark externally facing Root Ports with the this ACPI
_DSD:
Quoted 6 lines
...If we find such a Root Port, mark it and all its children as "
is_untrusted". The rest of the OS may use this to prevent use of the device unless we have an IOMMU to prevent malicious DMA [I don't know your actual policy yet].
OK, I will update the commit message accordingly.
Quoted 65 lines
Quoted 52 lines
Name (_DSD, Package () { ToUUID ("efcc06cc-73ac-4bc3-bff0-76143807c389"), Package () { Package () {"ExternalFacingPort", 1}, Package () {"UID", 0 } }})
To make it possible for IOMMU code to identify these devices, we look up for this property and mark all children devices (including the root port itself) with a new flag (->is_untrusted). This flag is meant to be used with all PCIe devices (not just Thunderbolt) that cannot be trusted in the same level than integrated devices and may need to put behind full IOMMU protection.
--- drivers/acpi/property.c | 3 +++ drivers/pci/pci-acpi.c | 18 ++++++++++++++++++ drivers/pci/probe.c | 22 ++++++++++++++++++++++ include/linux/pci.h | 8 ++++++++ 4 files changed, 51 insertions(+)drivers/acpi/property.c +3
-0@@ -31,6 +31,9 @@ static const guid_t prp_guids[] = {/* Hotplug in D3 GUID: 6211e2c0-58a3-4af3-90e1-927a4e0c55a4 */ GUID_INIT (0x6211e2c0 ,0x58a3 ,0x4af3 ,0x90 ,0xe1 ,0x92 ,0x7a ,0x4e ,0x0c ,0x55 ,0xa4 ), +/* External facing port GUID: efcc06cc-73ac-4bc3-bff0-76143807c389 */ +GUID_INIT (0xefcc06cc ,0x73ac ,0x4bc3 , + 0xbf, 0xf0, 0x76, 0x14, 0x38, 0x07, 0xc3, 0x89), }; static const guid_t ads_guid =drivers/pci/pci-acpi.c +5
-0@@ -789,6 +789,23 @@ static void pci_acpi_optimize_delay(struct pci_dev *pdev,ACPI_FREE (obj); } +static void pci_acpi_set_untrusted (struct pci_dev *dev) +{ +u8 val; + +if (pci_pcie_type (dev) !=PCI_EXP_TYPE_ROOT_PORT )drivers/pci/pci-acpi.c static void pci_acpi_optimize_delay(struct pci_dev *pdev, · line 806
Quoted 10 lines
drivers/pci/pci-acpi.c +10
-0+return ; +if (device_property_read_u8 (&dev->dev,"ExternalFacingPort" , &val)) +return ; + +/* +* These root ports expose PCIe (including DMA) outside of the +* system so make sure we treat them and everything behind as +* untrusted. +*/ + dev->is_untrusted = val ==1 ;Simpler to parse:
Quoted 2 lines
if (val)
dev->is_untrusted = 1;OK.
Quoted 43 lines
Quoted 29 lines
drivers/pci/pci-acpi.c +3
-0+} +static void pci_acpi_setup (struct device *dev) {struct pci_dev *pci_dev =to_pci_dev (dev);@@ -798,6 +815,7 @@ static void pci_acpi_setup(struct device *dev)return ;pci_acpi_optimize_delay (pci_dev, adev->handle); +pci_acpi_set_untrusted (pci_dev);pci_acpi_add_pm_notifier (adev, pci_dev);if (!adev->wakeup.flags.valid)drivers/pci/probe.c +8
-0@@ -1378,6 +1378,26 @@ static void set_pcie_thunderbolt(struct pci_dev *dev) } } +static void set_pcie_untrusted (struct pci_dev *dev) +{ +struct pci_dev *parent; + +/* +* Walk up the device hierarchy and check for any upstream bridge +* that has is_untrusted set to true. In that case treat this one +* untrusted as well. drivers/pci/probe.c static void set_pcie_thunderbolt(struct pci_dev *dev) · line 1398
Quoted 10 lines
drivers/pci/probe.c +10
-0+*/ + parent =pci_upstream_bridge (dev); +while (parent) { +if (parent->is_untrusted) { + dev->is_untrusted = true; +break ; + } + + parent =pci_upstream_bridge (parent); + }In what circumstance is this loop needed? I expected this would be sufficient:
Quoted 3 lines
bridge = pci_upstream_bridge(dev);
if (bridge && bridge->is_untrusted)
dev->is_untrusted = true;I agree the loop is not necessary. I'll change it in the next version.
Quoted 34 lines
Quoted 20 lines
drivers/pci/probe.c +4
-0+} +/** * pci_ext_cfg_is_aliased - Is ext config space just an alias of std config? * @dev: PCI device @@ -1638,6 +1658,8 @@ int pci_setup_device(struct pci_dev *dev)/* Need to have dev->cfg_size ready */ set_pcie_thunderbolt (dev); +set_pcie_untrusted (dev); +/* "Unknown power state" */ dev->current_state =PCI_UNKNOWN ;include/linux/pci.h +0
-0@@ -396,6 +396,14 @@ struct pci_dev { unsigned int is_hotplug_bridge:1;include/linux/pci.h struct pci_dev { · line 406
Quoted 10 lines
include/linux/pci.h +8
-0unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ +/* +* Devices marked being untrusted are the ones that can potentially +* execute DMA attacks and similar. They are typically connected +* through external ports such as Thunderbolt but not limited to +* that. When an IOMMU is enabled they should be getting full +* mappings to make sure they cannot access arbitrary memory. +*/ + unsigned int is_untrusted:1;We have a mix of "
is_X" and "X", but I think "untrusted" is sufficient since "untrusted" is a perfectly good predicate all by itself, i.e.,
Quoted 3 lines
if (
dev->untrusted)reads easily and makes good sense.
Works for me :)
Rafael J. Wysocki
maintainerRe: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
Quoted 28 lines · Mika Westerberg
Quoted 1 line · Bjorn Helgaas
Hi Mika,
Hi,
Quoted 6 lines · Mika Westerberg
Quoted 1 line
Recent systems with Thunderbolt ports may support IOMMU natively.
This sentence doesn't make sense to me. There's no logical connection between having an IOMMU and having a Thunderbolt port.
Quoted 6 lines
Quoted 3 lines
This means that the platform utilizes IOMMU to prevent DMA attacks over externally exposed PCIe root ports (typically Thunderbolt ports)
Nor this one. The platform only uses the IOMMU to prevent DMA attacks if the OS chooses to do that.
I guess I'm trying to say here that the recent changes add such support to the platform BIOS that allows the OS to enable IOMMU without being compromised by a malicious device that is already connected. The BIOS sets the new ACPI DMAR bit in that case.
Quoted 4 lines
Quoted 3 lines
The system BIOS marks these PCIe root ports as being externally facing ports by implementing following ACPI
_DSD[1] under the root port in question:
Quoted 6 lines
Quoted 2 lines
There's no standard that requires this, so the best we can say is that a system BIOS *may* mark externally facing ports with this mechanism.
There is no standard but I'm quite sure this is something that will be required to be implemented properly by the OEM by Microsoft hardware compatibility suite.
I think it would be fair to say that future versions of Windows will expect the firmware to identify the "externally facing" root PCIe ports as per the above which practically means that it is as good as a formal standard in the Windows world.
Re: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
In reply to Mika WesterbergQuoted 17 lines · Mika Westerberg
Quoted 1 line · Bjorn Helgaas
Hi Mika,
Hi,
Quoted 12 lines · Mika Westerberg
Quoted 1 line
Recent systems with Thunderbolt ports may support IOMMU natively.
This sentence doesn't make sense to me. There's no logical connection between having an IOMMU and having a Thunderbolt port.
Quoted 3 lines
This means that the platform utilizes IOMMU to prevent DMA attacks over externally exposed PCIe root ports (typically Thunderbolt ports)
Nor this one. The platform only uses the IOMMU to prevent DMA attacks if the OS chooses to do that.
I think by "platform" you're referring to the system firmware; I was only thinking of the hardware, so the IOMMU wouldn't be used unless someone (the OS) enabled it. But your cover letter talks about the BIOS enabling some IOMMU functionality.
Quoted 4 lines
I guess I'm trying to say here that the recent changes add such support to the platform BIOS that allows the OS to enable IOMMU without being compromised by a malicious device that is already connected. The BIOS sets the new ACPI DMAR bit in that case.
Ah, there's useful info to this effect in your [0/4] cover letter. That info and the URL should be in the changelog of one of the patches so it doesn't get lost.
Quoted 10 lines
Quoted 6 lines
Quoted 3 lines
The system BIOS marks these PCIe root ports as being externally facing ports by implementing following ACPI
_DSD[1] under the root port in question:There's no standard that requires this, so the best we can say is that a system BIOS *may* mark externally facing ports with this mechanism.
There is no standard but I'm quite sure this is something that will be required to be implemented properly by the OEM by Microsoft hardware compatibility suite.
Sure. Your statement suggests that all external ports will be marked with the _DSD. I'm just pointing out that the OS can't assume that because there are probably systems in the field that predate the _DSD.
Bjorn
Rafael J. Wysocki
maintainerRe: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
In reply to Mika WesterbergQuoted 39 lines · Mika Westerberg
Recent systems with Thunderbolt ports may support IOMMU natively. This means that the platform utilizes IOMMU to prevent DMA attacks over externally exposed PCIe root ports (typically Thunderbolt ports)
The system BIOS marks these PCIe root ports as being externally facing ports by implementing following ACPI
_DSD[1] under the root port in question:Name (_DSD, Package () { ToUUID ("efcc06cc-73ac-4bc3-bff0-76143807c389"), Package () { Package () {"ExternalFacingPort", 1}, Package () {"UID", 0 } } })To make it possible for IOMMU code to identify these devices, we look up for this property and mark all children devices (including the root port itself) with a new flag (->is_untrusted). This flag is meant to be used with all PCIe devices (not just Thunderbolt) that cannot be trusted in the same level than integrated devices and may need to put behind full IOMMU protection.
--- drivers/acpi/property.c | 3 +++ drivers/pci/pci-acpi.c | 18 ++++++++++++++++++ drivers/pci/probe.c | 22 ++++++++++++++++++++++ include/linux/pci.h | 8 ++++++++ 4 files changed, 51 insertions(+)drivers/acpi/property.c +0
-0
drivers/acpi/property.c static const guid_t prp_guids[] = { · line 37
Quoted 7 lines
drivers/acpi/property.c +3
-0@@ -31,6 +31,9 @@ static const guid_t prp_guids[] = {/* Hotplug in D3 GUID: 6211e2c0-58a3-4af3-90e1-927a4e0c55a4 */ GUID_INIT (0x6211e2c0 ,0x58a3 ,0x4af3 ,0x90 ,0xe1 ,0x92 ,0x7a ,0x4e ,0x0c ,0x55 ,0xa4 ), +/* External facing port GUID: efcc06cc-73ac-4bc3-bff0-76143807c389 */ +GUID_INIT (0xefcc06cc ,0x73ac ,0x4bc3 , + 0xbf, 0xf0, 0x76, 0x14, 0x38, 0x07, 0xc3, 0x89), };
One observation here. Note that I really have no strong opinion on that, so this is not an objection, but IMO it is fair to make things clear for everybody.
So this causes the External facing port GUID (which already is the case with the Hotplug in D3 GUID for that matter) to be practically equivalent to the ACPI _DSD device properties GUID. This means that Linux will consider a combination of any of these GUIDs with any properties defined for any of them as valid, which need not be the case in Windows.
For example, since the ExternalFacingPort property is defined along with the External facing port GUID, Windows will likely ignore it if it is used in a combination with the Hotplug in D3 GUID or the ACPI _DSD device properties GUID. If the firmware combines the Hotplug in D3 GUID or the ACPI _DSD device properties GUID with that property, Windows will not regard it as valid, most likely, but Linux will use it just fine. In the face of ASL bugs, which are inevitable (at least just because ASL is code written by humans), that may become a real problem, as systems successfully tested with Windows may not work with Linux as expected and vice versa because of it.
Quoted 1 line
From the ecosystem purity perspective this should be avoided, but then
I do realize that this allows code to be re-used and avoids quite a bit of complexity. The likelihood of an ASL bug that will expose this issue is arguably small, so maybe it is not a practical concern after all.
RE: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
Quoted 86 lines
-----Original Message-----
From: Rafael J. Wysocki <[email protected]>
Sent: Tuesday, November 27, 2018 11:15 AM
To: Mika Westerberg
Cc: open list:AMD IOMMU (AMD-VI); Joerg Roedel; David Woodhouse; Lu Baolu; Raj, Ashok; Bjorn Helgaas; Rafael J. Wysocki; Pan, Jacob jun; Andreas Noever; Michael Jamet; Yehezkel Bernat; Lukas Wunner; [email protected]; Limonciello, Mario; Anthony Wong; Lorenzo Pieralisi; Christoph Hellwig; Alex Williamson; ACPI Devel Maling List; Linux PCI; Linux Kernel Mailing List
Subject: Re: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices[EXTERNAL EMAIL]
Quoted 17 lines · Mika Westerberg
Recent systems with Thunderbolt ports may support IOMMU natively. This means that the platform utilizes IOMMU to prevent DMA attacks over externally exposed PCIe root ports (typically Thunderbolt ports)
The system BIOS marks these PCIe root ports as being externally facing ports by implementing following ACPI
_DSD[1] under the root port in question:Name (_DSD, Package () { ToUUID ("efcc06cc-73ac-4bc3-bff0-76143807c389"), Package () { Package () {"ExternalFacingPort", 1}, Package () {"UID", 0 } } })Quoted 8 lines
To make it possible for IOMMU code to identify these devices, we look up for this property and mark all children devices (including the root port itself) with a new flag (->is_untrusted). This flag is meant to be used with all PCIe devices (not just Thunderbolt) that cannot be trusted in the same level than integrated devices and may need to put behind full IOMMU protection.
[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/pci/dsd-for-
pcie-root-ports#identifying-externally-exposed-pcie-root-ports
Quoted 14 lines
Signed-off-by Mika Westerberg <[email protected]>--- drivers/acpi/property.c | 3 +++ drivers/pci/pci-acpi.c | 18 ++++++++++++++++++ drivers/pci/probe.c | 22 ++++++++++++++++++++++ include/linux/pci.h | 8 ++++++++ 4 files changed, 51 insertions(+)drivers/acpi/property.c +0
-0drivers/acpi/property.c static const guid_t prp_guids[] = { · line 37
Quoted 7 lines
drivers/acpi/property.c +3
-0@@ -31,6 +31,9 @@ static const guid_t prp_guids[] = {/* Hotplug in D3 GUID: 6211e2c0-58a3-4af3-90e1-927a4e0c55a4 */ GUID_INIT (0x6211e2c0 ,0x58a3 ,0x4af3 ,0x90 ,0xe1 ,0x92 ,0x7a ,0x4e ,0x0c ,0x55 ,0xa4 ), +/* External facing port GUID: efcc06cc-73ac-4bc3-bff0-76143807c389 */ +GUID_INIT (0xefcc06cc ,0x73ac ,0x4bc3 , + 0xbf, 0xf0, 0x76, 0x14, 0x38, 0x07, 0xc3, 0x89), };One observation here. Note that I really have no strong opinion on that, so this is not an objection, but IMO it is fair to make things clear for everybody.
So this causes the External facing port GUID (which already is the case with the Hotplug in D3 GUID for that matter) to be practically equivalent to the ACPI
_DSDdevice properties GUID. This means that Linux will consider a combination of any of these GUIDs with any properties defined for any of them as valid, which need not be the case in Windows.For example, since the ExternalFacingPort property is defined along with the External facing port GUID, Windows will likely ignore it if it is used in a combination with the Hotplug in D3 GUID or the ACPI
_DSDdevice properties GUID. If the firmware combines the Hotplug in D3 GUID or the ACPI_DSDdevice properties GUID with that property, Windows will not regard it as valid, most likely, but Linux will use it just fine. In the face of ASL bugs, which are inevitable (at least just because ASL is code written by humans), that may become a real problem, as systems successfully tested with Windows may not work with Linux as expected and vice versa because of it.
Quoted 5 lines
From the ecosystem purity perspective this should be avoided, but then I do realize that this allows code to be re-used and avoids quite a bit of complexity. The likelihood of an ASL bug that will expose this issue is arguably small, so maybe it is not a practical concern after all.
This is the perfect type of situation that should be raised as a highly marked bug in FWTS. FWTS is already in the UEFI self certification suite and is being used by IBVs, OEMs and ODMs to find and fix ASL issues.
Re: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
In reply to Rafael J. WysockiQuoted 24 lines · Rafael J. Wysocki
drivers/acpi/property.c static const guid_t prp_guids[] = { · line 37
Quoted 12 lines
drivers/acpi/property.c +3
-0@@ -31,6 +31,9 @@ static const guid_t prp_guids[] = {/* Hotplug in D3 GUID: 6211e2c0-58a3-4af3-90e1-927a4e0c55a4 */ GUID_INIT (0x6211e2c0 ,0x58a3 ,0x4af3 ,0x90 ,0xe1 ,0x92 ,0x7a ,0x4e ,0x0c ,0x55 ,0xa4 ), +/* External facing port GUID: efcc06cc-73ac-4bc3-bff0-76143807c389 */ +GUID_INIT (0xefcc06cc ,0x73ac ,0x4bc3 , + 0xbf, 0xf0, 0x76, 0x14, 0x38, 0x07, 0xc3, 0x89), };One observation here. Note that I really have no strong opinion on that, so this is not an objection, but IMO it is fair to make things clear for everybody.
So this causes the External facing port GUID (which already is the case with the Hotplug in D3 GUID for that matter) to be practically equivalent to the ACPI
_DSDdevice properties GUID. This means that Linux will consider a combination of any of these GUIDs with any properties defined for any of them as valid, which need not be the case in Windows.
Quoted 10 lines
For example, since the ExternalFacingPort property is defined along with the External facing port GUID, Windows will likely ignore it if it is used in a combination with the Hotplug in D3 GUID or the ACPI
_DSDdevice properties GUID. If the firmware combines the Hotplug in D3 GUID or the ACPI_DSDdevice properties GUID with that property, Windows will not regard it as valid, most likely, but Linux will use it just fine. In the face of ASL bugs, which are inevitable (at least just because ASL is code written by humans), that may become a real problem, as systems successfully tested with Windows may not work with Linux as expected and vice versa because of it.
That's a fair point.
Quoted 5 lines
Quoted 1 line
From the ecosystem purity perspective this should be avoided, but then
I do realize that this allows code to be re-used and avoids quite a bit of complexity. The likelihood of an ASL bug that will expose this issue is arguably small, so maybe it is not a practical concern after all.
One option assuming we want to prevent the potential discrepancy you described above is to add ACPI specific property accessors that take GUID as parameter. Then it would only look properties inside that particular _DSD entry. I'm not sure if it is worth the added complexity, though.
Rafael J. Wysocki
maintainerRe: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
Quoted 45 lines · Mika Westerberg
Quoted 24 lines · Rafael J. Wysocki
drivers/acpi/property.c static const guid_t prp_guids[] = { · line 37
Quoted 12 lines
drivers/acpi/property.c +3
-0@@ -31,6 +31,9 @@ static const guid_t prp_guids[] = {/* Hotplug in D3 GUID: 6211e2c0-58a3-4af3-90e1-927a4e0c55a4 */ GUID_INIT (0x6211e2c0 ,0x58a3 ,0x4af3 ,0x90 ,0xe1 ,0x92 ,0x7a ,0x4e ,0x0c ,0x55 ,0xa4 ), +/* External facing port GUID: efcc06cc-73ac-4bc3-bff0-76143807c389 */ +GUID_INIT (0xefcc06cc ,0x73ac ,0x4bc3 , + 0xbf, 0xf0, 0x76, 0x14, 0x38, 0x07, 0xc3, 0x89), };One observation here. Note that I really have no strong opinion on that, so this is not an objection, but IMO it is fair to make things clear for everybody.
So this causes the External facing port GUID (which already is the case with the Hotplug in D3 GUID for that matter) to be practically equivalent to the ACPI
_DSDdevice properties GUID. This means that Linux will consider a combination of any of these GUIDs with any properties defined for any of them as valid, which need not be the case in Windows.Quoted 10 lines
For example, since the ExternalFacingPort property is defined along with the External facing port GUID, Windows will likely ignore it if it is used in a combination with the Hotplug in D3 GUID or the ACPI
_DSDdevice properties GUID. If the firmware combines the Hotplug in D3 GUID or the ACPI_DSDdevice properties GUID with that property, Windows will not regard it as valid, most likely, but Linux will use it just fine. In the face of ASL bugs, which are inevitable (at least just because ASL is code written by humans), that may become a real problem, as systems successfully tested with Windows may not work with Linux as expected and vice versa because of it.That's a fair point.
Quoted 5 lines
Quoted 1 line
From the ecosystem purity perspective this should be avoided, but then
I do realize that this allows code to be re-used and avoids quite a bit of complexity. The likelihood of an ASL bug that will expose this issue is arguably small, so maybe it is not a practical concern after all.
Quoted 5 lines
One option assuming we want to prevent the potential discrepancy you described above is to add ACPI specific property accessors that take GUID as parameter. Then it would only look properties inside that particular
_DSDentry. I'm not sure if it is worth the added complexity, though.
I'm not sure if this is worth the extra complexity either, which is why I have no strong opinion here. :-)
Maybe you can add a comment, next to the prp_guids[] definition, to explain that the GUIDs are made equivalent to each other in order to avoid extra complexity in the properties handling code, with the caveat that the kernel will accept certain combinations of GUIDs and properties that are not defined strictly speaking without warning, but those combinations of GUIDs and properties are not expected to be used by firmware and they should be caught by firmware validation tools and reported as errors anyway.
Re: [PATCH v2 1/4] PCI / ACPI: Identify untrusted PCI devices
Quoted 11 lines · Rafael J. Wysocki
I'm not sure if this is worth the extra complexity either, which is why I have no strong opinion here. :-)
Maybe you can add a comment, next to the prp_guids[] definition, to explain that the GUIDs are made equivalent to each other in order to avoid extra complexity in the properties handling code, with the caveat that the kernel will accept certain combinations of GUIDs and properties that are not defined strictly speaking without warning, but those combinations of GUIDs and properties are not expected to be used by firmware and they should be caught by firmware validation tools and reported as errors anyway.
Sure, I'll add the comment in the next version of the series.
PATCH v2 2/4 iommu/vt-d: Force IOMMU on for platform opt in hint
In reply to Mika WesterbergFrom: Lu Baolu <[email protected]>
Intel VT-d spec added a new DMA_CTRL_PLATFORM_OPT_IN_FLAG flag in DMAR ACPI table for BIOS to report compliance about platform initiated DMA restricted to RMRR ranges when transferring control to the OS. The OS treats this as a hint that the IOMMU should be enabled to prevent DMA attacks from possible malicious devices.
A use of this flag is Kernel DMA protection for Thunderbolt[1] which in practice means that IOMMU should be enabled for PCIe devices connected to the Thunderbolt ports. With IOMMU enabled for these devices, all DMA operations are limited in the range reserved for it, thus the DMA attacks are prevented. All these devices are enumerated in the PCI/PCIe module and marked with an is_untrusted flag.
This forces IOMMU to be enabled if DMA_CTRL_PLATFORM_OPT_IN_FLAG is set in DMAR ACPI table and there are PCIe devices marked as is_untrusted in the system. This can be turned off by adding "intel_iommu=off" in the kernel command line, if any problems are found.
--- drivers/iommu/dmar.c | 25 +++++++++++++++++ drivers/iommu/intel-iommu.c | 53 +++++++++++++++++++++++++++++++++++-- include/linux/dmar.h | 8 ++++++ 3 files changed, 84 insertions(+), 2 deletions(-)
drivers/iommu/dmar.c +25 -0
@@ -2042,3 +2042,28 @@ int dmar_device_remove(acpi_handle handle) {return dmar_device_hotplug (handle, false); } + +/* +* dmar_platform_optin - Is %DMA_CTRL_PLATFORM_OPT_IN_FLAG set in DMAR table +* +* Returns true if the platform has %DMA_CTRL_PLATFORM_OPT_IN_FLAG set in +* the ACPI DMAR table. This means that the platform boot firmware has made +* sure no device can issue DMA outside of RMRR regions. +*/ +bool dmar_platform_optin (void ) +{ +struct acpi_table_dmar *dmar; +acpi_status status; +bool ret; + + status =acpi_get_table (ACPI_SIG_DMAR ,0 , + (struct acpi_table_header **)&dmar); +if (ACPI_FAILURE (status)) +return false; + + ret = !!(dmar->flags &DMAR_PLATFORM_OPT_IN ); +acpi_put_table ((struct acpi_table_header *)dmar); + +return ret; +} +EXPORT_SYMBOL_GPL (dmar_platform_optin);
drivers/iommu/intel-iommu.c +51 -2
83 lines not on this page · raw message
include/linux/dmar.h +8 -0
@@ -39,6 +39,7 @@ struct acpi_dmar_header;/* DMAR Flags */ #define DMAR_INTR_REMAP 0x1#define DMAR_X2APIC_OPT_OUT 0x2 +#define DMAR_PLATFORM_OPT_IN 0x4struct intel_iommu ;
@@ -170,6 +171,8 @@ static inline int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool insert) {return 0 ; }#endif /* CONFIG_IRQ_REMAP */ +extern bool dmar_platform_optin (void ); +#else /* CONFIG_DMAR_TABLE */ static inline int dmar_device_add (void *handle)
@@ -182,6 +185,11 @@ static inline int dmar_device_remove(void *handle)return 0 ; } +static inline bool dmar_platform_optin (void ) +{ +return false; +} +#endif /* CONFIG_DMAR_TABLE */ struct irte {
Signature · 1 line
2.19.1
PATCH v2 3/4 iommu/vt-d: Do not enable ATS for untrusted devices
In reply to Mika WesterbergCurrently Linux automatically enables ATS (Address Translation Service) for any device that supports it (and IOMMU is turned on). ATS is used to accelerate DMA access as the device can cache translations locally so there is no need to do full translation on IOMMU side. However, as pointed out in [1] ATS can be used to bypass IOMMU based security completely by simply sending PCIe read/write transaction with AT (Address Translation) field set to "translated".
To mitigate this modify the Intel IOMMU code so that it does not enable ATS for any device that is marked as being untrusted. In case this turns out to cause performance issues we may selectively allow ATS based on user decision but currently use big hammer and disable it completely to be on the safe side.
[1] https://www.repository.cam.ac.uk/handle/1810/274352
--- drivers/iommu/intel-iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +2 -1
@@ -1473,7 +1473,8 @@ static void iommu_enable_dev_iotlb(struct device_domain_info *info)if (info->pri_supported && !pci_reset_pri (pdev) && !pci_enable_pri (pdev,32 )) info->pri_enabled =1 ;#endif -if (info->ats_supported && !pci_enable_ats (pdev,VTD_PAGE_SHIFT )) { +if (!pdev->is_untrusted && info->ats_supported && + !pci_enable_ats (pdev,VTD_PAGE_SHIFT )) { info->ats_enabled =1 ;domain_update_iotlb (info->domain); info->ats_qdep =pci_ats_queue_depth (pdev);
Signature · 1 line
2.19.1
PATCH v2 4/4 thunderbolt: Export IOMMU based DMA protection support to userspace
In reply to Mika WesterbergRecent systems with Thunderbolt ports may support IOMMU natively. In practice this means that Thunderbolt connected devices are placed behind an IOMMU during the whole time it is connected (including during boot) making Thunderbolt security levels redundant. This is called Kernel DMA protection [1] by Microsoft.
Some of these systems still have Thunderbolt security level set to "user" in order to support OS downgrade (the older version of the OS might not support IOMMU based DMA protection so connecting a device still relies on user approval).
Export this information to userspace by introducing a new sysfs attribute (iommu_dma_protection). Based on it userspace tools can make more accurate decision whether or not authorize the connected device.
In addition update Thunderbolt documentation regarding IOMMU based DMA protection.
--- .../ABI/testing/sysfs-bus-thunderbolt | 9 +++++++++ Documentation/admin-guide/thunderbolt.rst | 20 +++++++++++++++++++ drivers/thunderbolt/domain.c | 17 ++++++++++++++++ 3 files changed, 46 insertions(+)
Documentation/ABI/testing/sysfs-bus-thunderbolt +9 -0
@@ -21,6 +21,15 @@ Description: Holds a comma separated list of device unique_ids that If a device is authorized automatically during boot its boot attribute is set to 1. +What: /sys/bus/thunderbolt/devices/.../domainX/iommu_dma_protection +Date: Mar 2019 +KernelVersion: 4.21 +Contact: [email protected] +Description: This attribute tells whether the system uses IOMMU + for DMA protection. Value of 1 means IOMMU is used 0 means + it is not (DMA protection is solely based on Thunderbolt + security levels). + What: /sys/bus/thunderbolt/devices/.../domainX/security Date: Sep 2017 KernelVersion: 4.13
Documentation/admin-guide/thunderbolt.rst +20 -0
@@ -133,6 +133,26 @@ If the user still wants to connect the device they can either approve the device without a key or write a new key and write 1 to the ``authorized`` file to get the new key stored on the device NVM. +DMA protection utilizing IOMMU +------------------------------ +Recent systems from 2018 and forward with Thunderbolt ports may natively +support IOMMU. This means that Thunderbolt security is handled by an IOMMU +so connected devices cannot access memory regions outside of what is +allocated for them by drivers. When Linux is running on such system it +automatically enables IOMMU if not enabled by the user already. These +systems can be identified by reading ``1`` from +``/sys/bus/thunderbolt/devices/domainX/iommu_dma_protection`` attribute. + +The driver does not do anything special in this case but because DMA +protection is handled by the IOMMU, security levels (if set) are +redundant. For this reason some systems ship with security level set to +``none``. Other systems have security level set to ``user`` in order to +support downgrade to older OS, so users who want to automatically +authorize devices when IOMMU DMA protection is enabled can use the +following ``udev`` rule:: + + ACTION=="add", SUBSYSTEM=="thunderbolt", ATTRS{iommu_dma_protection}=="1", ATTR{authorized}=="0", ATTR{authorized}="1" + Upgrading NVM on Thunderbolt device or host ------------------------------------------- Since most of the functionality is handled in firmware running on a
drivers/thunderbolt/domain.c +17 -0
@@ -7,7 +7,9 @@*/ #include <linux/device.h> +#include <linux/dmar.h> #include <linux/idr.h> +#include <linux/iommu.h> #include <linux/module.h> #include <linux/pm_runtime.h> #include <linux/slab.h>
@@ -236,6 +238,20 @@ static ssize_t boot_acl_store(struct device *dev, struct device_attribute *attr, }static DEVICE_ATTR_RW (boot_acl); +static ssize_t iommu_dma_protection_show (struct device *dev, +struct device_attribute *attr, +char *buf) +{ +/* +* Kernel DMA protection is a feature where Thunderbolt security is +* handled natively using IOMMU. It is enabled when IOMMU is +* enabled and ACPI DMAR table has DMAR_PLATFORM_OPT_IN set. +*/ +return sprintf (buf,"%d\n" , +iommu_present (&pci_bus_type) &&dmar_platform_optin ()); +} +static DEVICE_ATTR_RO (iommu_dma_protection); +static ssize_t security_show (struct device *dev,struct device_attribute *attr,char *buf) {
@@ -251,6 +267,7 @@ static DEVICE_ATTR_RO(security);static struct attribute *domain_attrs[] = { &dev_attr_boot_acl.attr, + &dev_attr_iommu_dma_protection.attr, &dev_attr_security.attr,NULL , };
Signature · 1 line
2.19.1