Device Evidence and Trust for PCI Security Protocol (TDISP)
Subsystems (11)
cover Device Evidence and Trust for PCI Security Protocol (TDISP)
Major changes since: "[PATCH v2 00/19] PCI/TSM: TEE I/O infrastructure" [1]
- Implement a new netlink schema for dumping large SPDM blobs (Jakub, Lukas)
- Introduce a device core implementation for evidence gathering given SPDM is applicable to devices outside of PCI (Lukas)
- Replace the "CC acceptance" mechanism with "Device Trust" (Greg, Jason)
- Fix TDISP report parsing to use bytes instead of pfns for the start value of an MMIO report (PCI spec clarification)
- Support the CCA expectation that MMIO reports are BAR aligned (Aneesh)
[1]: http://lore.kernel.org/[email protected]
[ Note, this series is back to a v1 posting given the new theme of
"Evidence and Trust". It defers some of the lower level details, like touching arch-level ioremap implementations to consult encrypted_iomem, until the higher order details are settled ].
---
Summary:
A threat model wants to arrange for devices to not be operated until they present evidence. Add device evidence gathering, device trust levels, and hook those up for the PCI/TSM (PCI device security protocol) implementation.
---
Confidential Computing and other use cases specify a threat model where devices are potentially adversarial. To date the device core trusts devices and their drivers, and only specific buses have local "adversary" device mitigations. The task is define a general trust concept to up-level existing trust mitigations, and allow for incremental tightening of mitigations across bus types and IOMMU drivers.
Goals:
* Create a mechanism for booting with a limited set devices + drivers and permit selectively enabling more devices + drivers at runtime.
* Up-level bus specific trust mitigations like PCI untrusted to a core concept.
* Let buses coordinate entry into the TCB via the existing ->dma_configure() callback
The proposal considers existing trust mitigation in various buses. The PCI device "untrusted" flags is an example of limiting but not fully blocking operation of a device. The USB interface authorized flag is an example of not trusting a device to operate at all. The implementation starts with these trust level DEVICE_TRUST_NONE, no operation of the device, and DEVICE_TRUST_ADVERSARY, limited operation of the device. For example, the PCI "untrusted" flag turns off device translation capabilities (ATS) and strictly enforces IOMMU mappings, but otherwise allows operation of the device.
The proposal does not currently incorporate other bus specific security mechanisms that are independent of individual device operational trust. For example, USB device (not interface) authorization and Thunderbolt switch authorization are transport gates to other devices. PCIe Link Encryption is also a transport security mechanism that informs a trust decision, but it is still possible to distrust a device with Link Encryption and conversely distrust a device protected by Link Encryption. Those security mechanisms are orthogonal to trust policy for driver bind, address space access, and acceleration feature enabling.
The result is device trust is an input to the driver core and IOMMU layers. A bus can opt to sync its local bind policy, address space access, and acceleration features to the trust level, or treat it as an overriding second-stage policy from the bus's first stage mechanism.
The trust levels established at enumeration and enforced at driver attach are DEVICE_TRUST_{NONE,ADVERSARY,AUTO,TCB}:
* NONE: no usage of the device unless the trust is explicitly overridden by user policy specified via a driver flag, module flag, or uapi (TBD).
* ADVERSARY: needs acknowledgement from the bus and IOMMU / DMA layers that the device is limited to strict IOMMU translation behavior. Drivers can use this as a signal to limit functionality. This designation implies follow-on IOMMU and bus enabling work for features like arranging for the device to attach to a blocked IOMMU domain when detached from a driver.
* AUTO: typical / historical Linux driver model.
* TCB: a trust level that only exists in Confidential Computing environments. When acked by the IOMMU / DMA layer it enables the device to issue direct-DMA to private/encrypted addresses or otherwise attach to a secure vIOMMU within the TCB.
Overview of changes:
Patches 1-6: Update netlink to be able to support large blob transfers that the SPDM standard requires. SPDM can be associated with any device type, so the implementation lands in the device core with PCI/TSM as the first consumer.
Patches 7-9: Introduce DEVICE_TRUST_NONE and DEVICE_TRUST_ADVERSARY. Support blocking driver bind by default, and allow either build policy or module policy to override the the trust level.
Patches 10-12,14-15: Plumb the PCI/TSM support for the PCI device security (TDISP) protocol to transition the device through UNLOCKED->LOCKED->RUN.
Patch 13: Support access to private memory either by the bus either coordinating with a TSM driver, or relying on the bus provider to have already arranged access (paravisor case).
This passes basic checkout with the sample TSM driver and a sample netlink client for the device evidence facility. Those patches are only on the git tree for now [2].
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/log/?h=devsec-phase2
Dan Williams (15):
netlink: specs: Introduce multi-message blobs for SPDM
tools: ynl: Teach pyynl to handle blobs
tools: ynl: Teach ynl_gen_c to validate and dump 'blob' attributes device core: Introduce "device evidence" over netlink device core: Add "device evidence" 'validate' command PCI/TSM: Add device evidence support
modules: Document the global async_probe parameter
device core: Initial device trust infrastructure
PCI, device core: Move "untrusted" concept to DEVICE_TRUST_ADVERSARY PCI/TSM: Add device interface security LOCKED support PCI/TSM: Add device interface security RUN support PCI/TSM: Add device interface security DMA enable/disable PCI, device core: Add private memory access for DEVICE_TRUST_TCB PCI/TSM: Create MMIO descriptors via TDISP Report PCI/TSM: Add relative MMIO offset support?
drivers/base/Kconfig | 91 +++
drivers/pci/Kconfig | 2 +
drivers/base/Makefile | 2 +
drivers/pci/Makefile | 2 +-
drivers/pci/tsm/Makefile | 8 +
Documentation/ABI/stable/sysfs-module | 17 +
Documentation/ABI/testing/sysfs-bus-pci | 47 +-
Documentation/ABI/testing/sysfs-class-tsm | 19 +
Documentation/driver-api/pci/tsm.rst | 47 ++
Documentation/netlink/genetlink-legacy.yaml | 6 +
Documentation/netlink/genetlink.yaml | 7 +
Documentation/netlink/netlink-raw.yaml | 7 +
.../netlink/specs/device-evidence.yaml | 203 ++++++
drivers/base/base.h | 15 +
drivers/base/device-evidence-netlink.h | 25 +
include/linux/device.h | 3 +
include/linux/device/evidence.h | 88 +++
include/linux/device/trust.h | 56 ++
include/linux/ioport.h | 2 +
include/linux/module.h | 6 +-
include/linux/pci-tsm.h | 104 +++-
include/linux/pci.h | 13 +-
include/uapi/linux/device-evidence.h | 111 ++++
tools/net/ynl/lib/ynl-priv.h | 4 +
tools/net/ynl/lib/ynl.h | 14 +
drivers/base/core.c | 7 +
drivers/base/dd.c | 5 +
drivers/base/device-evidence-netlink.c | 58 ++
drivers/base/evidence.c | 495 +++++++++++++++
drivers/base/trust.c | 103 ++++
drivers/iommu/amd/iommu.c | 2 +-
drivers/iommu/dma-iommu.c | 13 +-
drivers/iommu/intel/iommu.c | 2 +-
drivers/iommu/iommu.c | 2 +-
drivers/pci/ats.c | 2 +-
drivers/pci/pci-driver.c | 25 +-
drivers/pci/pci.c | 2 +-
drivers/pci/probe.c | 10 +-
drivers/pci/quirks.c | 4 +-
drivers/pci/{tsm.c => tsm/core.c} | 579 +++++++++++++++++-
drivers/pci/tsm/evidence.c | 110 ++++
drivers/virt/coco/tsm-core.c | 43 +-
kernel/module/main.c | 13 +
kernel/resource.c | 8 +
tools/net/ynl/lib/ynl.c | 33 +
MAINTAINERS | 6 +-
tools/net/ynl/pyynl/lib/nlspec.py | 12 +
tools/net/ynl/pyynl/lib/ynl.py | 62 +-
tools/net/ynl/pyynl/ynl_gen_c.py | 80 ++-
49 files changed, 2509 insertions(+), 66 deletions(-)
create mode 100644 drivers/pci/tsm/Makefile
create mode 100644 Documentation/netlink/specs/device-evidence.yaml
create mode 100644 drivers/base/device-evidence-netlink.h
create mode 100644 include/linux/device/evidence.h
create mode 100644 include/linux/device/trust.h
create mode 100644 include/uapi/linux/device-evidence.h
create mode 100644 drivers/base/device-evidence-netlink.c
create mode 100644 drivers/base/evidence.c
create mode 100644 drivers/base/trust.c
rename drivers/pci/{tsm.c => tsm/core.c} (61%)
create mode 100644 drivers/pci/tsm/evidence.cbase-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
Signature · 1 line
2.54.0
PATCH 1/15 netlink: specs: Introduce multi-message blobs for SPDM
The SPDM, Security Protocol and Data Model, underpins PCI device security and other use cases. It defines objects that allow for verification of device identity and configuration. These objects can be large in size 16MB. Netlink is otherwise suitable to define the operations, with optional parameters, and notifications for working with these objects. For example, operations like "regenerate evidence with nonce", "mark evidence validated", and "broadcast evidence / security state change events".
A netlink 'blob' is introduced as a way to teach YNL that one instance of a attribute may span multiple messages. It enables netlink to convey all the data needed for verification and manipulation of SPDM transported evidence.
The schema change to allows YNL to infer that an attribute may span multiple messages and interrogate its length to preallocate an appropriately sized receive buffer.
The design direction to extend the netlink schema for a "multi-message object receive" case was the result of this discussion [1].
--- Documentation/netlink/genetlink-legacy.yaml | 6 ++++++ Documentation/netlink/genetlink.yaml | 7 +++++++ Documentation/netlink/netlink-raw.yaml | 7 +++++++ 3 files changed, 20 insertions(+)
Documentation/netlink/genetlink-legacy.yaml +6 -0
@@ -239,6 +239,12 @@ properties: enum: [ little-endian, big-endian ] multi-attr: type: boolean + bloblen: + description: | + Marks this attribute as a variable-length blob that may be split across + multiple messages. The value names a separate scalar attribute that carries + the total blob length, and which is sent ahead of the blob data so a consumer + can preallocate the full payload. nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string
Documentation/netlink/genetlink.yaml +7 -0
@@ -153,6 +153,13 @@ properties: enum: [ little-endian, big-endian ] multi-attr: type: boolean + bloblen: + description: | + Marks this attribute as a variable-length blob that may be split across + multiple messages. The value names a separate scalar attribute that carries + the total blob length, and which is sent ahead of the blob data so a consumer + can preallocate the full payload. + type: string nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string
Documentation/netlink/netlink-raw.yaml +7 -0
@@ -263,6 +263,13 @@ properties: enum: [ little-endian, big-endian ] multi-attr: type: boolean + bloblen: + description: | + Marks this attribute as a variable-length blob that may be split across + multiple messages. The value names a separate scalar attribute that carries + the total blob length, and which is sent ahead of the blob data so a consumer + can preallocate the full payload. + type: string nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 01/15] netlink: specs: Introduce multi-message blobs for SPDM
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The `bloblen` property in `genetlink-legacy.yaml` is missing its `type: string` specification…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The bloblen property in genetlink-legacy.yaml is missing its type: string specification, rendering the schema validation incomplete. --
commit cab827e2cbb54e443717f126a29803e1306e23f6
Author: Dan Williams <[email protected]>
netlink: specs: Introduce multi-message blobs for SPDM
This commit introduces a netlink blob property to teach YNL that an attribute may span multiple messages. It modifies the netlink schemas to allow YNL to infer this structure and interrogate the total blob length to preallocate buffers.
Quoted 4 lines
Documentation/netlink/genetlink-legacy.yaml +0
-0
[ ... ]
Documentation/netlink/genetlink-legacy.yaml properties: · line 247
Quoted 10 lines
Documentation/netlink/genetlink-legacy.yaml +6
-0@@ -239,6 +239,12 @@ properties: enum: [ little-endian, big-endian ] multi-attr: type: boolean + bloblen: + description: | + Marks this attribute as a variable-length blob that may be split across + multiple messages. The value names a separate scalar attribute that carries + the total blob length, and which is sent ahead of the blob data so a consumer + can preallocate the full payload.
[Severity: Low]
Is this missing a type definition?
Unlike the additions in genetlink.yaml and netlink-raw.yaml, this property does not specify that the type is a string. Without a specified type, the schema validation defaults to allowing any data type for bloblen in legacy families, which could bypass intended type validation.
Quoted 3 lines
Documentation/netlink/genetlink-legacy.yaml +0
-0nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Donald Hunter
maintainerRe: [PATCH 01/15] netlink: specs: Introduce multi-message blobs for SPDM
In reply to Dan WilliamsQuoted 8 lines · Dan Williams
The SPDM, Security Protocol and Data Model, underpins PCI device security and other use cases. It defines objects that allow for verification of device identity and configuration. These objects can be large in size 16MB. Netlink is otherwise suitable to define the operations, with optional parameters, and notifications for working with these objects. For example, operations like "regenerate evidence with nonce", "mark evidence validated", and "broadcast evidence / security state change events".
Quoted 5 lines
A netlink 'blob' is introduced as a way to teach YNL that one instance of a attribute may span multiple messages. It enables netlink to convey all the data needed for verification and manipulation of SPDM transported evidence.
The schema change to allows YNL to infer that an attribute may span
change to allows -> change allows
Quoted 12 lines
multiple messages and interrogate its length to preallocate an appropriately sized receive buffer.
The design direction to extend the netlink schema for a "multi-message object receive" case was the result of this discussion [1].
Cc Alistair Francis <[email protected]>Cc Lukas Wunner <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Donald Hunter <[email protected]>Signed-off-by Dan Williams <[email protected]>
Quoted 4 lines
--- Documentation/netlink/genetlink-legacy.yaml | 6 ++++++ Documentation/netlink/genetlink.yaml | 7 +++++++ Documentation/netlink/netlink-raw.yaml | 7 +++++++
I suggest you drop the changes to genetlink-legacy and netlink-raw because I don't think we want to support the blob functionality for legacy families.
Can you also update Documentation/userspace-api/netlink/specs.rst to describe the multi-attr / bloblen API behaviour.
Re: [PATCH 01/15] netlink: specs: Introduce multi-message blobs for SPDM
Quoted 10 lines · Donald Hunter
Quoted 8 lines · Dan Williams
The SPDM, Security Protocol and Data Model, underpins PCI device security and other use cases. It defines objects that allow for verification of device identity and configuration. These objects can be large in size 16MB. Netlink is otherwise suitable to define the operations, with optional parameters, and notifications for working with these objects. For example, operations like "regenerate evidence with nonce", "mark evidence validated", and "broadcast evidence / security state change events".
Quoted 7 lines
Quoted 5 lines
A netlink 'blob' is introduced as a way to teach YNL that one instance of a attribute may span multiple messages. It enables netlink to convey all the data needed for verification and manipulation of SPDM transported evidence.
The schema change to allows YNL to infer that an attribute may span
change to allows -> change allows
ack.
Quoted 17 lines
Quoted 12 lines
multiple messages and interrogate its length to preallocate an appropriately sized receive buffer.
The design direction to extend the netlink schema for a "multi-message object receive" case was the result of this discussion [1].
Cc Alistair Francis <[email protected]>Cc Lukas Wunner <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Donald Hunter <[email protected]>Signed-off-by Dan Williams <[email protected]>Quoted 4 lines
--- Documentation/netlink/genetlink-legacy.yaml | 6 ++++++ Documentation/netlink/genetlink.yaml | 7 +++++++ Documentation/netlink/netlink-raw.yaml | 7 +++++++
Quoted 3 lines
I suggest you drop the changes to genetlink-legacy and netlink-raw because I don't think we want to support the blob functionality for legacy families.
Done.
Quoted 2 lines
Can you also update
Documentation/userspace-api/netlink/specs.rstto describe the multi-attr / bloblen API behaviour.
Added:
@@ -239,6 +239,16 @@ Boolean property signifying that the attribute may be present multiple times.
Allowing an attribute to repeat is the recommended way of implementing arrays
(no extra nesting).
+bloblen (multi-message binary blob)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+String property specified on a ``type: binary`` ``multi-attr: true`` +attribute to indicate that this single byte array may span multiple +messages. The string names an attribute that indicates the final size of +the blob when all messages are received. This differs from other ``type:
+binary`` ``multi-attr: true`` instances without bloblen where each +message is an attribute boundary.
+
byte-order
~~~~~~~~~~
Donald Hunter
maintainerRe: [PATCH 01/15] netlink: specs: Introduce multi-message blobs for SPDM
In reply to Dan WilliamsQuoted 8 lines · Dan Williams
Documentation/netlink/genetlink.yaml +0
-0@@ -153,6 +153,13 @@ properties: enum: [ little-endian, big-endian ] multi-attr: type: boolean
Documentation/netlink/genetlink.yaml properties: · line 165
Quoted 10 lines
Documentation/netlink/genetlink.yaml +7
-0+ bloblen: + description: | + Marks this attribute as a variable-length blob that may be split across + multiple messages. The value names a separate scalar attribute that carries + the total blob length, and which is sent ahead of the blob data so a consumer + can preallocate the full payload. + type: string nested-attributes: description: Name of the space (sub-space) used inside the attribute. type: string
Can you add a schema constraint so that bloblen allowed only when multi-attr == true and type == binary
Jakub Kicinski
maintainerRe: [PATCH 01/15] netlink: specs: Introduce multi-message blobs for SPDM
In reply to Dan WilliamsQuoted 12 lines · On Sun, 5 Jul 2026 15:08:05 -0700 Dan Williams wrote:
The SPDM, Security Protocol and Data Model, underpins PCI device security and other use cases. It defines objects that allow for verification of device identity and configuration. These objects can be large in size 16MB. Netlink is otherwise suitable to define the operations, with optional parameters, and notifications for working with these objects. For example, operations like "regenerate evidence with nonce", "mark evidence validated", and "broadcast evidence / security state change events".
A netlink 'blob' is introduced as a way to teach YNL that one instance of a attribute may span multiple messages. It enables netlink to convey all the data needed for verification and manipulation of SPDM transported evidence.
Quoted 6 lines
The schema change to allows YNL to infer that an attribute may span multiple messages and interrogate its length to preallocate an appropriately sized receive buffer.
The design direction to extend the netlink schema for a "multi-message object receive" case was the result of this discussion [1].
.. continuing the tradition of creative abuse of Netlink.
I don't like this, sorry. You need to come up with something outside of netlink or more netlink-y.
Re: [PATCH 01/15] netlink: specs: Introduce multi-message blobs for SPDM
Quoted 2 lines
I don't like this, sorry. You need to come up with something outside of netlink or more netlink-y.
Hi Jakub, picking this up and continuing on with the Dan's work.
In an earlier discussion [1], two options were mentioned.
Quoted 1 line
Alright, so to make this more Netlink-y you can either:
> - delete the F_MULTI and replicate other attrs in each message and > add an offset attr; this will make each message in the dump more > standalone.
> - keep the F_MULTI but object_val has to be a multi-attr, and then > we have to teach YNL to correctly append the attrs.
So I'll update to the first option of standalone messages with an offset attribute. i.e. every dump message with a new offset for the chunk it carries and with reassembly left to the consumer. I suppose that aligns with your preference?
Jakub Kicinski
maintainerRe: [PATCH 01/15] netlink: specs: Introduce multi-message blobs for SPDM
Quoted 14 lines · On Sun, 2 Aug 2026 17:58:06 +0000 Ankit Agrawal wrote:
Quoted 2 lines
I don't like this, sorry. You need to come up with something outside of netlink or more netlink-y.
Hi Jakub, picking this up and continuing on with the Dan's work.
In an earlier discussion [1], two options were mentioned.
Quoted 1 line
Alright, so to make this more Netlink-y you can either:
> - delete the
F_MULTIand replicate other attrs in each message and > add an offset attr; this will make each message in the dump more > standalone.
> - keep theF_MULTIbutobject_valhas to be a multi-attr, and then > we have to teach YNL to correctly append the attrs.
Quoted 5 lines
So I'll update to the first option of standalone messages with an offset attribute. i.e. every dump message with a new offset for the chunk it carries and with reassembly left to the consumer. I suppose that aligns with your preference?
Yes, cleanly building the blob in netlink is going to be tough. First option would be uncontroversial from Netlink PoV, AFAICT. But also please spend a bit of time on trying to come up with a better way of moving the blobs.
PATCH 2/15 tools: ynl: Teach pyynl to handle blobs
In reply to Dan WilliamsThe generic "device evidence" facility wants to convey large objects (certificate chains, measurements) and support various security commands relative to that evidence (like validate) [1].
Update pyynl to understand the "bloblen" property indicates:
* the associated attribute is a variable length byte array
* it is multi-attr where each message with the same attribute represents chunks of the byte array
* the byte array's size is conveyed in a scalar attribute named by the "bloblen" property.
* the attribute identified by "bloblen" always arrives in a message before the first chunk of the associated blob
A "multi-attr" without "bloblen" retains the legacy behavior where repeated reception of the same attribute across messages results in a new instance of the object. The new behavior causes repeated reception of an attribute to be treated as filling consecutive bytes of an array.
--- tools/net/ynl/pyynl/lib/nlspec.py | 12 ++++++ tools/net/ynl/pyynl/lib/ynl.py | 62 ++++++++++++++++++++++++++++++- 2 files changed, 72 insertions(+), 2 deletions(-)
tools/net/ynl/pyynl/lib/nlspec.py +12 -0
@@ -172,6 +172,7 @@ class SpecAttr(SpecElement): selector string, name of attribute used to select sub-message type + bloblen string, name of attr which reports the blob length is_auto_scalar bool, attr is a variable-size scalar """ def __init__(self, family, attr_set, yaml, value):
@@ -181,6 +182,7 @@ class SpecAttr(SpecElement): self.value = value self.attr_set = attr_set self.is_multi = yaml.get('multi-attr', False) + self.bloblen = yaml.get('bloblen') self.struct_name = yaml.get('struct') self.sub_type = yaml.get('sub-type') self.byte_order = yaml.get('byte-order')
@@ -191,6 +193,10 @@ class SpecAttr(SpecElement): self.is_auto_scalar = self.type in ("sint", "uint") + if self.bloblen and not self.is_multi: + raise SpecException( + f"Attribute '{attr_set.name}.{self.name}' has bloblen without multi-attr") + class SpecAttrSet(SpecElement): """ Netlink Attribute Set class.
@@ -355,6 +361,8 @@ class SpecOperation(SpecElement): attr_set attribute set name fixed_header string, optional name of fixed header struct + dump_blob_attrs set, reply blob attribute names which may span messages + yaml raw spec as loaded from the spec file """ def __init__(self, family, yaml, req_value, rsp_value):
@@ -369,6 +377,7 @@ class SpecOperation(SpecElement): self.is_async = 'notify' in yaml or 'event' in yaml self.is_resv = not self.is_async and not self.is_call self.fixed_header = self.yaml.get('fixed-header', family.fixed_header) + self.dump_blob_attrs = frozenset() # Added by resolve: self.attr_set = None
@@ -388,6 +397,9 @@ class SpecOperation(SpecElement): raise SpecException(f"Can't resolve attribute set for op '{self.name}'") if attr_set_name: self.attr_set = self.family.attr_sets[attr_set_name] + reply_attrs = self.yaml.get('dump', {}).get('reply', {}).get('attributes', ()) + self.dump_blob_attrs = frozenset( + name for name in reply_attrs if self.attr_set[name].bloblen) class SpecMcastGroup(SpecElement):
tools/net/ynl/pyynl/lib/ynl.py +60 -2
104 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 02/15] tools: ynl: Teach pyynl to handle blobs
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] _blob_extend incorrectly rejects continuation messages and treats them as new objects if they…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] _blob_extend incorrectly rejects continuation messages and treats them as new objects if they contain repeated scalar attributes (e.g., object identifiers).
- [Medium] PyYNL crashes with a false length exception or type error if a blob's first chunk arrives in a continuation message instead of the initial message.
- [Low] Blob attribute coalescing is restricted exclusively to dump operations, causing 'do' operations to inconsistently return lists of chunks. --
commit 6f54aff032c9d8543801051eb2d44c4da3645009
Author: Dan Williams <[email protected]>
tools: ynl: Teach pyynl to handle blobs
This commit updates pyynl to handle blobs by supporting the "bloblen" property, where multiple messages with the same attribute represent chunks of a byte array. It coalesces these chunks into a single byte array instead of treating them as separate objects.
Quoted 4 lines
tools/net/ynl/pyynl/lib/nlspec.py +0
-0
[ ... ]
tools/net/ynl/pyynl/lib/nlspec.py class SpecOperation(SpecElement): · line 402
Quoted 7 lines
tools/net/ynl/pyynl/lib/nlspec.py +3
-0@@ -388,6 +397,9 @@ class SpecOperation(SpecElement): raise SpecException(f"Can't resolve attribute set for op '{self.name}'") if attr_set_name: self.attr_set = self.family.attr_sets[attr_set_name] + reply_attrs = self.yaml.get('dump', {}).get('reply', {}).get('attributes', ()) + self.dump_blob_attrs = frozenset( + name for name in reply_attrs if self.attr_set[name].bloblen)
[Severity: Low]
Is blob attribute coalescing meant to be restricted only to dump operations? Since we only check the dump reply block here, a 'do' operation returning a multi-attr blob will bypass the coalescing logic entirely and return a list of bytes chunks instead of a single contiguous bytes object.
Quoted 4 lines
tools/net/ynl/pyynl/lib/ynl.py +0
-0
[ ... ]
Quoted 13 lines
tools/net/ynl/pyynl/lib/ynl.py +9
-0@@ -998,6 +998,47 @@ class YnlFamily(SpecFamily): else: rsp[name] = [decoded] + def _blob_fill(self, buf, off, chunks, name): + for chunk in chunks: + end = off + len(chunk) + if end > len(buf): + raise YnlException(f"Blob '{name}' data exceeds reported length") + buf[off:end] = chunk + off = end + return off +
tools/net/ynl/pyynl/lib/ynl.py class YnlFamily(SpecFamily): · line 1019
Quoted 10 lines
tools/net/ynl/pyynl/lib/ynl.py +10
-0+ def _blob_init(self, op, rsp, blob_attrs): + """ + Start a new blob object. Preallocate each blob payload using the + length carried ahead of the data and copy in the first chunk(s). + Return the per-attribute write offsets for any following messages. + """ + offsets = {} + for name in blob_attrs: + if name not in rsp: + continue
[Severity: Medium]
Could this lead to a false length exception or type error if a blob's first chunk arrives in a continuation message instead of the initial message?
If the initial message contains the bloblen attribute but no data chunks, _blob_init() skips initializing the buf for that attribute here.
When the first chunk arrives in a subsequent message, _blob_extend() will fall into its else branch and assign the raw decoded value (a list of byte chunks) directly to rsp[name].
If a third message arrives with more chunks, _blob_extend() passes this list as buf to _blob_fill(). The call to len(buf) will evaluate to the number of chunks rather than the byte length, incorrectly triggering the "exceeds reported length" exception.
tools/net/ynl/pyynl/lib/ynl.py class YnlFamily(SpecFamily): · line 1032
Quoted 13 lines
tools/net/ynl/pyynl/lib/ynl.py +13
-0+ length = rsp.get(op.attr_set[name].bloblen, 0) + buf = bytearray(length) + offsets[name] = self._blob_fill(buf, 0, rsp[name], name) + rsp[name] = buf + return offsets + + def _blob_extend(self, rsp, update, blob_attrs, offsets): + """ + Continue filling the previous object's blob(s) when the current + message carries only blob continuation attributes. + """ + if any(name in rsp and name not in blob_attrs for name in update): + return False
[Severity: Medium]
Does this logic incorrectly reject continuation messages and treat them as new objects if they contain repeated scalar attributes?
When the kernel sends a continuation message in a dump, it typically includes the object's identifying keys (like dev-id) alongside the blob chunks. The above check will evaluate to True for these keys because they are already in rsp (from the first message) and are not blob attributes.
This causes _blob_extend() to return False, treating the continuation message as a new object and splitting the blob. Would it be better to verify if the overlapping key values actually differ?
Quoted 8 lines
tools/net/ynl/pyynl/lib/ynl.py +8
-0+ + for name, value in update.items(): + if name in blob_attrs and name in rsp: + offsets[name] = self._blob_fill(rsp[name], offsets.get(name, 0), + value, name) + else: + rsp[name] = value + return True
[ ... ]
tools/net/ynl/pyynl/lib/ynl.py class YnlFamily(SpecFamily): · line 1402
Quoted 8 lines
tools/net/ynl/pyynl/lib/ynl.py +3
-1@@ -1356,7 +1397,10 @@ class YnlFamily(SpecFamily): for (method, vals, flags) in ops: op = self.ops[method] msg = self._encode_message(op, vals, flags, req_seq) - reqs_by_seq[req_seq] = (op, vals, msg, flags) + blob_attrs = None + if Netlink.NLM_F_DUMP in flags: + blob_attrs = op.dump_blob_attrs or None
[Severity: Low]
Is there a reason blob attribute coalescing is explicitly disabled if the request is not a dump operation? This seems to restrict do operations from returning coalesced blobs, making them fall back to returning lists of chunks.
Quoted 3 lines
tools/net/ynl/pyynl/lib/ynl.py +1
-0+ reqs_by_seq[req_seq] = (op, vals, msg, flags, blob_attrs) payload += msg req_seq += 1
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Donald Hunter
maintainerRe: [PATCH 02/15] tools: ynl: Teach pyynl to handle blobs
In reply to Dan WilliamsQuoted 26 lines · Dan Williams
The generic "device evidence" facility wants to convey large objects (certificate chains, measurements) and support various security commands relative to that evidence (like validate) [1].
Update pyynl to understand the "bloblen" property indicates:
* the associated attribute is a variable length byte array
* it is multi-attr where each message with the same attribute represents chunks of the byte array
* the byte array's size is conveyed in a scalar attribute named by the "bloblen" property.
* the attribute identified by "bloblen" always arrives in a message before the first chunk of the associated blobA "multi-attr" without "bloblen" retains the legacy behavior where repeated reception of the same attribute across messages results in a new instance of the object. The new behavior causes repeated reception of an attribute to be treated as filling consecutive bytes of an array.
Assisted-by Codex:GPT-5[codex: drafted initial python changes]Cc Alistair Francis <[email protected]>Cc Lukas Wunner <[email protected]>Cc Donald Hunter <[email protected]>Cc Jakub Kicinski <[email protected]>Signed-off-by Dan Williams <[email protected]>
Quoted 4 lines
--- tools/net/ynl/pyynl/lib/nlspec.py | 12 ++++++ tools/net/ynl/pyynl/lib/ynl.py | 62 ++++++++++++++++++++++++++++++- 2 files changed, 72 insertions(+), 2 deletions(-)
I am mid-review of this but I think it could be simplified. See initial thoughts below. Can you show me how to configure a test env so I can try it out?
Quoted 42 lines
tools/net/ynl/pyynl/lib/nlspec.py +8
-0@@ -172,6 +172,7 @@ class SpecAttr(SpecElement): selector string, name of attribute used to select sub-message type + bloblen string, name of attr which reports the blob length is_auto_scalar bool, attr is a variable-size scalar """ def __init__(self, family, attr_set, yaml, value):@@ -181,6 +182,7 @@ class SpecAttr(SpecElement): self.value = value self.attr_set = attr_set self.is_multi = yaml.get('multi-attr', False) + self.bloblen = yaml.get('bloblen') self.struct_name = yaml.get('struct') self.sub_type = yaml.get('sub-type') self.byte_order = yaml.get('byte-order')@@ -191,6 +193,10 @@ class SpecAttr(SpecElement): self.is_auto_scalar = self.type in ("sint", "uint") + if self.bloblen and not self.is_multi: + raise SpecException( + f"Attribute '{attr_set.name}.{self.name}' has bloblen without multi-attr") + class SpecAttrSet(SpecElement): """ Netlink Attribute Set class.@@ -355,6 +361,8 @@ class SpecOperation(SpecElement): attr_set attribute set name fixed_header string, optional name of fixed header struct + dump_blob_attrs set, reply blob attribute names which may span messages + yaml raw spec as loaded from the spec file """ def __init__(self, family, yaml, req_value, rsp_value):
tools/net/ynl/pyynl/lib/nlspec.py class SpecOperation(SpecElement): · line 380
Quoted 4 lines
tools/net/ynl/pyynl/lib/nlspec.py +1
-0@@ -369,6 +377,7 @@ class SpecOperation(SpecElement): self.is_async = 'notify' in yaml or 'event' in yaml self.is_resv = not self.is_async and not self.is_call self.fixed_header = self.yaml.get('fixed-header', family.fixed_header) + self.dump_blob_attrs = frozenset()
Initialising to None would avoid some other logic.
Quoted 66 lines
tools/net/ynl/pyynl/lib/nlspec.py +3
-0# Added by resolve: self.attr_set = None@@ -388,6 +397,9 @@ class SpecOperation(SpecElement): raise SpecException(f"Can't resolve attribute set for op '{self.name}'") if attr_set_name: self.attr_set = self.family.attr_sets[attr_set_name] + reply_attrs = self.yaml.get('dump', {}).get('reply', {}).get('attributes', ()) + self.dump_blob_attrs = frozenset( + name for name in reply_attrs if self.attr_set[name].bloblen) class SpecMcastGroup(SpecElement):tools/net/ynl/pyynl/lib/ynl.py +41
-0@@ -998,6 +998,47 @@ class YnlFamily(SpecFamily): else: rsp[name] = [decoded] + def _blob_fill(self, buf, off, chunks, name): + for chunk in chunks: + end = off + len(chunk) + if end > len(buf): + raise YnlException(f"Blob '{name}' data exceeds reported length") + buf[off:end] = chunk + off = end + return off + + def _blob_init(self, op, rsp, blob_attrs): + """ + Start a new blob object. Preallocate each blob payload using the + length carried ahead of the data and copy in the first chunk(s). + Return the per-attribute write offsets for any following messages. + """ + offsets = {} + for name in blob_attrs: + if name not in rsp: + continue + length = rsp.get(op.attr_set[name].bloblen, 0) + buf = bytearray(length) + offsets[name] = self._blob_fill(buf, 0, rsp[name], name) + rsp[name] = buf + return offsets + + def _blob_extend(self, rsp, update, blob_attrs, offsets): + """ + Continue filling the previous object's blob(s) when the current + message carries only blob continuation attributes. + """ + if any(name in rsp and name not in blob_attrs for name in update): + return False + + for name, value in update.items(): + if name in blob_attrs and name in rsp: + offsets[name] = self._blob_fill(rsp[name], offsets.get(name, 0), + value, name) + else: + rsp[name] = value + return True + def _resolve_selector(self, attr_spec, search_attrs): sub_msg = attr_spec.sub_message if sub_msg not in self.sub_msgs:
tools/net/ynl/pyynl/lib/ynl.py class YnlFamily(SpecFamily): · line 1403
Quoted 8 lines
tools/net/ynl/pyynl/lib/ynl.py +4
-1@@ -1356,7 +1397,10 @@ class YnlFamily(SpecFamily): for (method, vals, flags) in ops: op = self.ops[method] msg = self._encode_message(op, vals, flags, req_seq) - reqs_by_seq[req_seq] = (op, vals, msg, flags) + blob_attrs = None + if Netlink.NLM_F_DUMP in flags: + blob_attrs = op.dump_blob_attrs or None + reqs_by_seq[req_seq] = (op, vals, msg, flags, blob_attrs)
I think this hunk can be dropped and instead use op.dump_blob_attrs directly
Quoted 40 lines
tools/net/ynl/pyynl/lib/ynl.py +8
-1payload += msg req_seq += 1@@ -1365,6 +1409,7 @@ class YnlFamily(SpecFamily): done = False rsp = [] op_rsp = [] + blob_off = {} while not done: reply, ancdata = self._recvmsg() nsid = self._decode_nsid(ancdata)@@ -1372,13 +1417,14 @@ class YnlFamily(SpecFamily): self._recv_dbg_print(reply, nms) for nl_msg in nms: if nl_msg.nl_seq in reqs_by_seq: - (op, vals, req_msg, req_flags) = reqs_by_seq[nl_msg.nl_seq] + (op, vals, req_msg, req_flags, blob_attrs) = reqs_by_seq[nl_msg.nl_seq] if nl_msg.extack: nl_msg.annotate_extack(op.attr_set) self._decode_extack(req_msg, op, nl_msg.extack, vals) else: op = None req_flags = [] + blob_attrs = None if nl_msg.error: raise NlError(nl_msg)@@ -1388,6 +1434,11 @@ class YnlFamily(SpecFamily): print(nl_msg) if Netlink.NLM_F_DUMP in req_flags: + if blob_attrs: + for obj in op_rsp: + for name in blob_attrs: + if isinstance(obj.get(name), bytearray): + obj[name] = bytes(obj[name]) rsp.append(op_rsp) elif not op_rsp: rsp.append(None)
tools/net/ynl/pyynl/lib/ynl.py class YnlFamily(SpecFamily): · line 1474
Quoted 10 lines
tools/net/ynl/pyynl/lib/ynl.py +7
-0@@ -1414,6 +1465,13 @@ class YnlFamily(SpecFamily): rsp_msg = self._decode(decoded.raw_attrs, op.attr_set.name) if op.fixed_header: rsp_msg.update(self._decode_struct(decoded.raw, op.fixed_header)) + + if blob_attrs: + if op_rsp and self._blob_extend(op_rsp[-1], rsp_msg, + blob_attrs, blob_off): + continue + blob_off = self._blob_init(op, rsp_msg, blob_attrs) +
This approach seems unnecessary for pyynl. Just let multi-attr do its thing and then concatenate them at the end.
Quoted 3 lines
tools/net/ynl/pyynl/lib/ynl.py +0
-0op_rsp.append(rsp_msg) return rsp
PATCH 3/15 tools: ynl: Teach ynl_gen_c to validate and dump 'blob' attributes
In reply to Dan WilliamsNow that blobs are recognized by pyynl and coalesced over successive netlink messages, do the same for the generated C. Recall that blobs support large objects for protocols like SPDM.
With blobs the generator can assume that the length of the blob will be conveyed in the response messages before the blob data message(s). Update the C generation to:
* Mark when a given operation may have blobs (objects that need more than one message to fill) with a new ynl_more_cb_t in the dump state.
* Use @more() determine when to continue filling a partially received blob, and check if the end of the message stream is reached without receiving all of its data.
* Pre-allocate the blob and parse successive messages into that shared object.
This proposal is the follow up to a discussion [1] where an alternative solution of adding an @offset attribute to make each chunk standalone was discarded. That approach commits userspace to inefficient attribute reassembly where it needs to retrieve a series of structures with offset properties and then reassemble them into the full blob.
--- tools/net/ynl/lib/ynl-priv.h | 4 ++ tools/net/ynl/lib/ynl.h | 14 ++++++ tools/net/ynl/lib/ynl.c | 33 +++++++++++++ tools/net/ynl/pyynl/ynl_gen_c.py | 80 +++++++++++++++++++++++++++++--- 4 files changed, 124 insertions(+), 7 deletions(-)
tools/net/ynl/lib/ynl-priv.h +4 -0
@@ -41,6 +41,9 @@ enum ynl_parse_result {typedef int (*ynl_parse_cb_t )(const struct nlmsghdr *nlh,struct ynl_parse_arg *yarg); +typedef bool (*ynl_more_cb_t )(void *data); + +struct ynl_blob *ynl_blob_alloc (unsigned int len);struct ynl_policy_attr {enum ynl_policy_type type:8 ;
@@ -123,6 +126,7 @@ struct ynl_dump_state {struct ynl_dump_list_type *last;size_t alloc_sz;ynl_parse_cb_t cb; +ynl_more_cb_t more;__u32 rsp_cmd; };
tools/net/ynl/lib/ynl.h +14 -0
@@ -100,6 +100,20 @@ struct ynl_string {char str[]; }; +/** +* struct ynl_blob - variable-length blob attribute +* @len: length of the blob data +* @data: value of the blob attribute +* +* Parsed binary value. This struct is used for variable-length blob values +* that may be built from one or more attributes spanning multiple netlink +* messages. +*/ +struct ynl_blob { +unsigned int len; +unsigned char data[]; +}; +struct ynl_sock *ynl_sock_create (const struct ynl_family *yf,struct ynl_error *e);void ynl_sock_destroy (struct ynl_sock *ys);
tools/net/ynl/lib/ynl.c +33 -0
@@ -1003,6 +1003,27 @@ int ynl_exec(struct ynl_sock *ys, struct nlmsghdr *req_nlh,return err; } +struct ynl_blob *ynl_blob_alloc (unsigned int len) +{ +size_t size =sizeof (struct ynl_blob ) + len; +struct ynl_blob *blob; + +if (size < len) +return NULL ; + blob =malloc (size); +if (blob) + blob->len = len; +return blob; +} + +/* continue filling a blob attribute? */ +static bool ynl_check_more (struct ynl_dump_state *ds) +{ +struct ynl_dump_list_type *prev = ds->last; + +return prev && ds->more && ds->more (&prev->data); +} +static int ynl_dump_trampoline (const struct nlmsghdr *nlh,struct ynl_parse_arg *data) {
@@ -1015,6 +1036,13 @@ ynl_dump_trampoline(const struct nlmsghdr *nlh, struct ynl_parse_arg *data)if (ret)return ret <0 ?YNL_PARSE_CB_ERROR :YNL_PARSE_CB_OK ; +if (ynl_check_more (ds)) { + yarg = ds->yarg; + yarg.data = &ds->last->data; + +return ds->cb (nlh, &yarg); + } + obj =calloc (1 , ds->alloc_sz);if (!obj)return YNL_PARSE_CB_ERROR ;
@@ -1059,6 +1087,11 @@ int ynl_exec_dump(struct ynl_sock *ys, struct nlmsghdr *req_nlh, goto err_close_list; }while (err >0 ); +if (ynl_check_more (yds)) { +yerr (ys,YNL_ERROR_ATTR_MISSING ,"Dump ended with an incomplete blob" ); + goto err_close_list; + } + yds->first =ynl_dump_end (yds);return 0 ;
tools/net/ynl/pyynl/ynl_gen_c.py +73 -7
139 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 03/15] tools: ynl: Teach ynl_gen_c to validate and dump 'blob' attributes
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] The generated `_more()` function fails to coalesce blob chunks if the length attribute arrives in…
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] The generated _more() function fails to coalesce blob chunks if the length attribute arrives in a message prior to the blob data, breaking multi-message coalescing.
- [High] The generated C setter function for blobs lacks a NULL check for malloc and is vulnerable to integer overflow on 32-bit systems, causing memory corruption or crashes.
- [High] Request serialization (attr_put) does not support chunking large blobs, leading to silent data corruption due to 16-bit nla_len truncation if the blob exceeds 64KB.
- [Medium] ynl_exec fails to check for incomplete blobs, leading to silent data truncation for do commands.
--
commit 5e458f41fb05443af96a2ab1a2945032cdac2de5
Author: Dan Williams <[email protected]>
tools: ynl: Teach ynl_gen_c to validate and dump 'blob' attributes
Now that blobs are recognized by pyynl and coalesced over successive netlink messages, do the same for the generated C. Recall that blobs support large objects for protocols like SPDM.
Quoted 4 lines
tools/net/ynl/lib/ynl.c +0
-0
[ ... ]
tools/net/ynl/lib/ynl.c int ynl_exec_dump(struct ynl_sock *ys, struct nlmsghdr *req_nlh, · line 1093
Quoted 8 lines
tools/net/ynl/lib/ynl.c +4
-0@@ -1059,6 +1087,11 @@ int ynl_exec_dump(struct ynl_sock *ys, struct nlmsghdr *req_nlh, goto err_close_list; }while (err >0 ); +if (ynl_check_more (yds)) { +yerr (ys,YNL_ERROR_ATTR_MISSING ,"Dump ended with an incomplete blob" ); + goto err_close_list; + }
[Severity: Medium]
While ynl_exec_dump() verifies that the blob is complete here, does ynl_exec() for single command responses completely omit this validation?
If the kernel returns a response where the provided blob data is shorter than the advertised length, will the parser silently succeed and leave userspace with a truncated blob?
Should struct ynl_req_state also include a more callback?
Quoted 4 lines
tools/net/ynl/pyynl/ynl_gen_c.py +0
-0
[ ... ]
tools/net/ynl/pyynl/ynl_gen_c.py class TypeMultiAttr(Type): · line 824
Quoted 8 lines
tools/net/ynl/pyynl/ynl_gen_c.py +4
-0@@ -791,6 +818,10 @@ class TypeMultiAttr(Type): put_type = self.type ri.cw.p(f"for (i = 0; i < {var}->_count.{self.c_name}; i++)") ri.cw.p(f"ynl_attr_put_{put_type}(nlh, {self.enum_name}, {var}->{self.c_name}[i]);") + elif self.attr['type'] == 'binary' and self.is_blob(): + ri.cw.p(f"if ({var}->{self.c_name})") + ri.cw.p(f"ynl_attr_put(nlh, {self.enum_name}, " + + f"{var}->{self.c_name}->data, {var}->{self.c_name}->len);")
[Severity: High]
Does ynl_attr_put() support chunking blobs larger than 64KB?
If a user application attempts to send a blob larger than 65531 bytes, it appears nla_len will overflow its 16-bit capacity in ynl_attr_put():
attr->nla_len = NLA_HDRLEN + size;
memcpy(ynl_attr_data(attr), value, size);
nlh->nlmsg_len += NLMSG_ALIGN(attr->nla_len);Will nlmsg_len then advance incorrectly while memcpy() copies the full data, causing subsequent attributes to overwrite the middle of the blob?
[ ... ]
tools/net/ynl/pyynl/ynl_gen_c.py class TypeMultiAttr(Type): · line 842
Quoted 8 lines
tools/net/ynl/pyynl/ynl_gen_c.py +4
-0@@ -805,6 +836,10 @@ class TypeMultiAttr(Type): raise Exception(f"Put of MultiAttr sub-type {self.attr['type']} not supported yet") def _setter_lines(self, ri, member, presence): + if self.is_blob(): + return [f"{presence} = {self.c_name}->len;", + f"{member} = malloc(sizeof(*{member}) + {presence});", + f"memcpy({member}, {self.c_name}, sizeof(*{member}) + {presence});"]
[Severity: High]
Does this generated C code lack a NULL check for the malloc() return value?
If malloc() fails, the returned NULL pointer will be passed directly to memcpy(), potentially causing a crash.
Additionally, on 32-bit systems, could sizeof(*member) + presence result in an integer overflow if the length is large? If the addition wraps around, it might allocate a small buffer while leaving the length field corrupted.
[ ... ]
Quoted 15 lines
tools/net/ynl/pyynl/ynl_gen_c.py +11
-0@@ -2388,6 +2424,33 @@ def parse_rsp_msg(ri, deref=False): ri.cw.nl() +def _blob_attrs(ri): + return [attr for _, attr in ri.struct['reply'].member_list() if attr.bloblen] + + +def print_rsp_more(ri): + blob_attrs = _blob_attrs(ri) + if not blob_attrs: + return + + local_vars = [f'{type_name(ri, "reply", deref=True)} *rsp = data;'] +
tools/net/ynl/pyynl/ynl_gen_c.py def parse_rsp_msg(ri, deref=False): · line 2447
Quoted 10 lines
tools/net/ynl/pyynl/ynl_gen_c.py +10
-0+ ri.cw.write_func_prot('static bool', + f'{op_prefix(ri, "reply", deref=True)}_more', + ['void *data'], suffix='') + ri.cw.block_start() + ri.cw.write_func_lvar(local_vars) + + for attr in blob_attrs: + ri.cw.p(f"if (rsp->{attr.c_name} && " + + f"rsp->_len.{attr.c_name} < rsp->{attr.c_name}->len)") + ri.cw.p('return true;')
[Severity: High]
If the blob length attribute arrives in a netlink message prior to the blob data message, will the blob pointer still be NULL?
If the blob is NULL, this function will return false. Would that cause ynl_dump_trampoline() to fall through and allocate a new object instead of coalescing the data, dropping the previously parsed length?
When the next message with the data chunk arrives, wouldn't it allocate a 0-length blob and fail with a "blob data exceeds length" error?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
PATCH 4/15 device core: Introduce "device evidence" over netlink
In reply to Dan WilliamsMultiple device security mechanisms are built on top of the SPDM protocol from the DMTF. The protocol arranges for devices to verify their identity and establish secure sessions between requesters and responders. In turn, device security protocols like PCIe IDE use SPDM sessions to program link encryption keys, and protocols like PCIe TDISP use exclusive secure sessions for managing the device's security state.
All of these uses share a need to retrieve the device's installed certificates, its measurements, and other protocol specific objects like interface reports. Unify all of those object retrieval and regeneration cases into a device-scoped ABI that buses can leverage.
When PCI CMA was presented at Linux Plumbers the consensus reaction was that it was not suitable to be driven via sysfs. Given the varied use cases of the same fundamental objects the consensus converged on netlink. With the new 'blob' extensions to netlink, it can support up to the 16MB signed transcripts that SPDM generates.
A bus opts in to evidence gathering via device_evidence_register() and passing device_evidence_ops. The ops resolve a device handle to its corresponding 'struct device_evidence' context.
See inline documentation in Documentation/netlink/specs/device-evidence.yaml for the object types and options. This includes objects like TDISP interface reports and options like digests instead of full blobs.
--- drivers/base/Kconfig | 4 + drivers/base/Makefile | 1 + .../netlink/specs/device-evidence.yaml | 186 +++++++ drivers/base/device-evidence-netlink.h | 23 + include/linux/device/evidence.h | 85 ++++ include/uapi/linux/device-evidence.h | 110 +++++ drivers/base/device-evidence-netlink.c | 44 ++ drivers/base/evidence.c | 453 ++++++++++++++++++ MAINTAINERS | 4 + 9 files changed, 910 insertions(+) create mode 100644 Documentation/netlink/specs/device-evidence.yaml create mode 100644 drivers/base/device-evidence-netlink.h create mode 100644 include/linux/device/evidence.h create mode 100644 include/uapi/linux/device-evidence.h create mode 100644 drivers/base/device-evidence-netlink.c create mode 100644 drivers/base/evidence.c
drivers/base/Kconfig +4 -0
@@ -190,6 +190,10 @@ config HMEM_REPORTING Enable reporting for heterogeneous memory access attributes under their non-uniform memory nodes. +config DEVICE_EVIDENCE + bool + depends on NET + source "drivers/base/test/Kconfig" config SYS_HYPERVISOR
drivers/base/Makefile +1 -0
@@ -26,6 +26,7 @@ obj-$(CONFIG_DEV_COREDUMP) += devcoredump.o obj-$(CONFIG_GENERIC_MSI_IRQ) += platform-msi.o obj-$(CONFIG_GENERIC_ARCH_TOPOLOGY) += arch_topology.o obj-$(CONFIG_GENERIC_ARCH_NUMA) += arch_numa.o +obj-$(CONFIG_DEVICE_EVIDENCE) += evidence.o device-evidence-netlink.o obj-$(CONFIG_ACPI) += physical_location.o obj-y += test/
Documentation/netlink/specs/device-evidence.yaml new file +186 -0
186 lines not on this page · raw message
drivers/base/device-evidence-netlink.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */ +/* Do not edit directly, auto-generated from: */ +/* Documentation/netlink/specs/device-evidence.yaml */ +/* YNL-GEN kernel header */ +/* To regenerate run: tools/net/ynl/ynl-regen.sh */ + +#ifndef _LINUX_DEVICE_EVIDENCE_GEN_H +#define _LINUX_DEVICE_EVIDENCE_GEN_H + +#include <net/netlink.h> +#include <net/genetlink.h> + +#include <uapi/linux/device-evidence.h> + +int device_evidence_nl_read_pre (struct netlink_callback *cb); +int device_evidence_nl_read_post (struct netlink_callback *cb); + +int device_evidence_nl_read_dumpit (struct sk_buff *skb, +struct netlink_callback *cb); + +extern struct genl_family device_evidence_nl_family; + +#endif /* _LINUX_DEVICE_EVIDENCE_GEN_H */
include/linux/device/evidence.h new file +85 -0
85 lines not on this page · raw message
include/uapi/linux/device-evidence.h new file +110 -0
110 lines not on this page · raw message
drivers/base/device-evidence-netlink.c new file +44 -0
44 lines not on this page · raw message
drivers/base/evidence.c new file +453 -0
453 lines not on this page · raw message
MAINTAINERS +4 -0
@@ -27470,8 +27470,12 @@ S: Maintained F: Documentation/ABI/testing/configfs-tsm-report F: Documentation/driver-api/coco/ F: Documentation/driver-api/pci/tsm.rst +F: Documentation/netlink/specs/device-evidence.yaml +F: drivers/base/*evidence* F: drivers/pci/tsm.c F: drivers/virt/coco/guest/ +F: include/uapi/linux/device-evidence.h +F: include/linux/device/evidence.h F: include/linux/*tsm*.h F: samples/tsm-mr/
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 04/15] device core: Introduce "device evidence" over netlink
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Premature state advancement causes missing netlink attributes when a dump message is split…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Premature state advancement causes missing netlink attributes when a dump message is split across multiple skbs. --
commit 011b819de43e88cfea717a04a2344bfd9d08116a
Author: Dan Williams <[email protected]>
device core: Introduce "device evidence" over netlink
This commit introduces a device-scoped ABI over netlink to retrieve device security evidence objects, such as SPDM certificates and measurements. A bus opts in via device_evidence_register() to resolve a device handle to its evidence context and provide these objects to userspace.
Quoted 5 lines
drivers/base/evidence.c new file +0
-0
[ ... ]
Quoted 4 lines
drivers/base/evidence.c new file +4
-0+static int __device_evidence_read (struct sk_buff *skb, +struct netlink_callback *cb, +struct device_evidence *evidence) +{
[ ... ]
Quoted 8 lines
drivers/base/evidence.c new file +8
-0+if (ctx->offset ==DEVICE_EVIDENCE_OBJECT_START ) { +if (nla_put_u32 (skb,DEVICE_EVIDENCE_A_OBJECT_TYPE , +current_type (ctx)) || +nla_put_u32 (skb,DEVICE_EVIDENCE_A_OBJECT_GENERATION , + ctx->generation) || +nla_put_u32 (skb,DEVICE_EVIDENCE_A_OBJECT_LENGTH , + object_len)) + goto out_cancel;
drivers/base/evidence.c line 341
Quoted 10 lines
drivers/base/evidence.c new file +10
-0+ + ctx->offset =0 ; + } + + available =skb_tailroom (skb); + overhead =nla_total_size (0 ) +NLA_ALIGNTO ; +if (available <= overhead) { + rc = -EMSGSIZE ; + goto out_cancel; + }
[Severity: Medium]
If available <= overhead evaluates to true and the code takes the goto out_cancel path, does ctx->offset incorrectly remain 0 while the netlink message is canceled?
On the next invocation with a fresh skb, ctx->offset will not equal DEVICE_EVIDENCE_OBJECT_START, which means the mandatory type, generation, and length attributes will be skipped, leading to a malformed netlink reply.
Could ctx->offset be updated after the tailroom check or reverted in the out_cancel error path?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Donald Hunter
maintainerRe: [PATCH 04/15] device core: Introduce "device evidence" over netlink
In reply to Dan WilliamsQuoted 18 lines · Dan Williams
Multiple device security mechanisms are built on top of the SPDM protocol from the DMTF. The protocol arranges for devices to verify their identity and establish secure sessions between requesters and responders. In turn, device security protocols like PCIe IDE use SPDM sessions to program link encryption keys, and protocols like PCIe TDISP use exclusive secure sessions for managing the device's security state.
All of these uses share a need to retrieve the device's installed certificates, its measurements, and other protocol specific objects like interface reports. Unify all of those object retrieval and regeneration cases into a device-scoped ABI that buses can leverage.
When PCI CMA was presented at Linux Plumbers the consensus reaction was that it was not suitable to be driven via sysfs. Given the varied use cases of the same fundamental objects the consensus converged on netlink. With the new 'blob' extensions to netlink, it can support up to the 16MB signed transcripts that SPDM generates.
Quoted 8 lines
A bus opts in to evidence gathering via
device_evidence_register()and passingdevice_evidence_ops. The ops resolve a device handle to its corresponding 'structdevice_evidence' context.See inline documentation in
Documentation/netlink/specs/device-evidence.yamlfor the object types and options. This includes objects like TDISP interface reports and options like digests instead of full blobs.
Can you include sample pyynl output; it helps immensely during review.
Quoted 10 lines
Cc Alistair Francis <[email protected]>Cc Lukas Wunner <[email protected]>Cc Donald Hunter <[email protected]>Cc Jakub Kicinski <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Xu Yilun <[email protected]>Cc "Aneesh Kumar K.V (Arm)" <[email protected]>Cc Alexey Kardashevskiy <[email protected]>Signed-off-by Dan Williams <[email protected]>
You probably need to cc the netdev list so that the series gets picked up by patchwork.
Quoted 51 lines
--- drivers/base/Kconfig | 4 + drivers/base/Makefile | 1 + .../netlink/specs/device-evidence.yaml | 186 +++++++ drivers/base/device-evidence-netlink.h | 23 + include/linux/device/evidence.h | 85 ++++ include/uapi/linux/device-evidence.h | 110 +++++ drivers/base/device-evidence-netlink.c | 44 ++ drivers/base/evidence.c | 453 ++++++++++++++++++ MAINTAINERS | 4 + 9 files changed, 910 insertions(+) create mode 100644 Documentation/netlink/specs/device-evidence.yaml create mode 100644 drivers/base/device-evidence-netlink.h create mode 100644 include/linux/device/evidence.h create mode 100644 include/uapi/linux/device-evidence.h create mode 100644 drivers/base/device-evidence-netlink.c create mode 100644 drivers/base/evidence.cdrivers/base/Kconfig +4
-0@@ -190,6 +190,10 @@ config HMEM_REPORTING Enable reporting for heterogeneous memory access attributes under their non-uniform memory nodes. +config DEVICE_EVIDENCE + bool + depends on NET + source "drivers/base/test/Kconfig" config SYS_HYPERVISORdrivers/base/Makefile +1
-0@@ -26,6 +26,7 @@ obj-$(CONFIG_DEV_COREDUMP) += devcoredump.o obj-$(CONFIG_GENERIC_MSI_IRQ) += platform-msi.o obj-$(CONFIG_GENERIC_ARCH_TOPOLOGY) += arch_topology.o obj-$(CONFIG_GENERIC_ARCH_NUMA) += arch_numa.o +obj-$(CONFIG_DEVICE_EVIDENCE) += evidence.o device-evidence-netlink.o obj-$(CONFIG_ACPI) += physical_location.o obj-y += test/Documentation/netlink/specs/device-evidence.yaml new file +0
-0
Documentation/netlink/specs/device-evidence.yaml line 4
Quoted 4 lines
Documentation/netlink/specs/device-evidence.yaml new file +4
-0@@ -0,0 +1,186 @@ +# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) +# +--- +name: device-evidence
The naming seems to be too narrow - if the scope expands at all then the name would need to change. A subsystem name like spdm, tsm or tsm-device seems better?
Quoted 99 lines
Documentation/netlink/specs/device-evidence.yaml new file +99
-0+protocol: genetlink +uapi-header: linux/device-evidence.h +doc: Device evidence retrieval over generic netlink + +definitions: + - + type: const + name: max-object-size + value: 0x01000000 + - + type: const + name: max-nonce-size + value: 32 + - + name: type + type: enum + doc: Device security evidence objects + render-max: true + entries: + - + name: cert0 + doc: SPDM certificate chain from device slot0 + - + name: cert1 + doc: SPDM certificate chain from device slot1 + - + name: cert2 + doc: SPDM certificate chain from device slot2 + - + name: cert3 + doc: SPDM certificate chain from device slot3 + - + name: cert4 + doc: SPDM certificate chain from device slot4 + - + name: cert5 + doc: SPDM certificate chain from device slot5 + - + name: cert6 + doc: SPDM certificate chain from device slot6 + - + name: cert7 + doc: SPDM certificate chain from device slot7 + - + name: vca + doc: | + SPDM version, capabilities, and algorithms transcript + negotiated at session establishment. An implementation may not + provide this separately and instead include it in the + measurements transcript. + - + name: measurements + doc: SPDM GET_MEASUREMENTS response + - + name: report + doc: | + A bus that implements a device interface security protocol + like TDISP may convey an interface report that details + interface settings and capabilities. + - + name: type-flag + type: flags + doc: Device security evidence request flags + render-max: true + # NOTE! these values must match the type enum name and order + entries: + - + name: cert0 + - + name: cert1 + - + name: cert2 + - + name: cert3 + - + name: cert4 + - + name: cert5 + - + name: cert6 + - + name: cert7 + - + name: vca + - + name: measurements + - + name: report + - + name: flag + type: flags + render-max: true + doc: Flags to control evidence retrieval + entries: + - + name: digest + doc: | + Request a secure hash of objects like vca and measurements. + The expectation is that this digest is produced by a responder
Documentation/netlink/specs/device-evidence.yaml line 113
Quoted 10 lines
Documentation/netlink/specs/device-evidence.yaml new file +10
-0+ within the TCB like a platform TSM. It validates a blob that + may have traversed a transport without integrity protections. + +attribute-sets: + - + name: object + attributes: + - + name: type + type: u32
Is this an instance of the type enum? If so then:
enum: type
Documentation/netlink/specs/device-evidence.yaml line 117
Quoted 4 lines
Documentation/netlink/specs/device-evidence.yaml new file +4
-0+ doc: evidence type-id + - + name: type-mask + type: u32
Likewise:
enum: type-flag
Documentation/netlink/specs/device-evidence.yaml line 121
Quoted 4 lines
Documentation/netlink/specs/device-evidence.yaml new file +4
-0+ doc: evidence types as a flag mask + - + name: flags + type: u32
And:
enum: flags
Quoted 21 lines
Documentation/netlink/specs/device-evidence.yaml new file +21
-0+ doc: evidence modifier flags like 'request object digest' + - + name: subsys + type: string + doc: device subsystem name (e.g. bus or class name) + - + name: dev-name + type: string + doc: device name + - + name: nonce + type: binary + checks: + max-len: max-nonce-size + - + name: generation + type: u32 + doc: detect local threads racing evidence refresh + - + name: count + type: u64
Documentation/netlink/specs/device-evidence.yaml line 152
Quoted 10 lines
Documentation/netlink/specs/device-evidence.yaml new file +10
-0+ doc: TSM, if present, observed count of evidence refresh events + - + name: length + type: u32 + checks: + max: max-object-size + doc: | + Final size of 'val' blob after all messages received. + - + name: val
Any reason for the abbreviated name?
Quoted 24 lines
Documentation/netlink/specs/device-evidence.yaml new file +24
-0+ type: binary + multi-attr: true + bloblen: length + doc: | + Ordered evidence object value chunk. Large evidence objects + may be split over several dump reply messages up to 'length'. + 'length' is transmitted before the first chunk. + +operations: + list: + - + name: read + doc: | + Read device evidence objects of a given type mask. The dump reply + reports the total value length before val attributes fill the evidence + object, possibly across multiple messages. + attribute-set: object + flags: [admin-perm] + dump: + pre: device-evidence-nl-read-pre + post: device-evidence-nl-read-post + request: + attributes: + - type-mask
Documentation/netlink/specs/device-evidence.yaml line 186
Quoted 10 lines
Documentation/netlink/specs/device-evidence.yaml new file +10
-0+ - flags + - subsys + - dev-name + - nonce + reply: + attributes: + - type + - generation + - length + - val
Is the intention that the dump filter must narrow to a single device? Is the user expected to enumerate the devices in subsystem then read device evidence for each in turn?
PATCH 5/15 device core: Add "device evidence" 'validate' command
In reply to Dan WilliamsWhen DMA is to be enabled for a PCI TDISP device the CC VM passes a digest of the device evidence used to make the trust decision. Require the CC VM to ack and not invalidate the evidence before the next DMA enable event.
Each evidence gathering conveys a current "generation" number that rolls each time the evidence is requested to be refreshed by the CC VM. The validate command is a debug aid setups that should not be racing evidence gathering, device RUN state transitions and DMA enable events.
Note that this generation number does count host side events that invalidate the evidence. That TOCTOU race belongs to the TSM to resolve and tell the CC VM that the digests it used to enable DMA are stale.
--- .../netlink/specs/device-evidence.yaml | 17 ++++++++ drivers/base/device-evidence-netlink.h | 2 + include/linux/device/evidence.h | 5 ++- include/uapi/linux/device-evidence.h | 1 + drivers/base/device-evidence-netlink.c | 14 +++++++ drivers/base/evidence.c | 42 +++++++++++++++++++ 6 files changed, 80 insertions(+), 1 deletion(-)
Documentation/netlink/specs/device-evidence.yaml +17 -0
@@ -184,3 +184,20 @@ operations: - generation - length - val + - + name: validate + doc: | + Assert that userspace accepts the evidence at 'generation' (e.g. as a + precondition for enabling DMA). If 'generation' matches the device's + current evidence generation the acceptance is recorded; it is + implicitly revoked when the evidence is regenerated. + attribute-set: object + flags: [admin-perm] + do: + request: + attributes: + - subsys + - dev-name + - generation + reply: + attributes: []
drivers/base/device-evidence-netlink.h +2 -0
@@ -17,6 +17,8 @@ int device_evidence_nl_read_post(struct netlink_callback *cb);int device_evidence_nl_read_dumpit (struct sk_buff *skb,struct netlink_callback *cb); +int device_evidence_nl_validate_doit (struct sk_buff *skb, +struct genl_info *info);extern struct genl_family device_evidence_nl_family;
include/linux/device/evidence.h +4 -1
@@ -29,15 +29,18 @@ struct device_evidence_object {* struct device_evidence - Retrieved device evidence * @slot: certificate slot used by a link TSM for connect * @generation: refresh_evidence() invocation detection +* @validated_generation: generation last accepted by userspace via "validate" * @digest_algo: payload size of DEVICE_EVIDENCE_FLAG_DIGEST requests * @lock: synchronize dumps vs refresh_evidence() * @obj: array of evidence objects a TSM might populate * -* An increment of @generation causes in flight dumps to fail with -EAGAIN. +* An increment of @generation causes in flight dumps to fail with -EAGAIN and +* implicitly revokes a prior userspace validation. */ struct device_evidence {int slot;u32 generation; +u32 validated_generation;enum hash_algo digest_algo;struct rw_semaphore lock;struct device_evidence_object obj[DEVICE_EVIDENCE_TYPE_MAX +1 ];
include/uapi/linux/device-evidence.h +1 -0
@@ -102,6 +102,7 @@ enum {enum {DEVICE_EVIDENCE_CMD_READ =1 , +DEVICE_EVIDENCE_CMD_VALIDATE , __DEVICE_EVIDENCE_CMD_MAX,DEVICE_EVIDENCE_CMD_MAX = (__DEVICE_EVIDENCE_CMD_MAX -1 )
drivers/base/device-evidence-netlink.c +14 -0
@@ -20,6 +20,13 @@ static const struct nla_policy device_evidence_read_nl_policy[DEVICE_EVIDENCE_A_ [DEVICE_EVIDENCE_A_OBJECT_NONCE] = NLA_POLICY_MAX_LEN(DEVICE_EVIDENCE_MAX_NONCE_SIZE), }; +/* DEVICE_EVIDENCE_CMD_VALIDATE - do */ +static const struct nla_policy device_evidence_validate_nl_policy[DEVICE_EVIDENCE_A_OBJECT_GENERATION +1 ] = { + [DEVICE_EVIDENCE_A_OBJECT_SUBSYS ] = { .type =NLA_NUL_STRING , }, + [DEVICE_EVIDENCE_A_OBJECT_DEV_NAME ] = { .type =NLA_NUL_STRING , }, + [DEVICE_EVIDENCE_A_OBJECT_GENERATION ] = { .type =NLA_U32 , }, +}; +/* Ops table for device_evidence */ static const struct genl_split_ops device_evidence_nl_ops[] = { {
@@ -31,6 +38,13 @@ static const struct genl_split_ops device_evidence_nl_ops[] = { .maxattr = DEVICE_EVIDENCE_A_OBJECT_NONCE, .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DUMP, }, + { + .cmd = DEVICE_EVIDENCE_CMD_VALIDATE, + .doit = device_evidence_nl_validate_doit, + .policy = device_evidence_validate_nl_policy, + .maxattr = DEVICE_EVIDENCE_A_OBJECT_GENERATION, + .flags = GENL_ADMIN_PERM | GENL_CMD_CAP_DO, + }, };struct genl_family device_evidence_nl_family__ro_after_init = {
drivers/base/evidence.c +42 -0
48 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 05/15] device core: Add "device evidence" 'validate' command
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Holding the global `device_evidence_lock` across the provider's `evidence_read_begin()` callback…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Holding the global device_evidence_lock across the provider's evidence_read_begin() callback blocks system-wide operations and causes global lock contention. --
commit e225a84deccb758c0117766e62b3f41123a9e5e3
Author: Dan Williams <[email protected]>
device core: Add "device evidence" 'validate' command
When DMA is to be enabled for a PCI TDISP device, the CC VM passes a digest of the device evidence used to make the trust decision. This commit adds a validate command to require the CC VM to ack and not invalidate the evidence before the next DMA enable event.
Quoted 3 lines
drivers/base/evidence.c +0
-0
[ ... ]
Quoted 20 lines
drivers/base/evidence.c +16
-0@@ -446,6 +446,48 @@ int device_evidence_nl_read_dumpit(struct sk_buff *skb,return device_evidence_read (skb, cb); } +int device_evidence_nl_validate_doit (struct sk_buff *skb, +struct genl_info *info) +{ +const struct device_evidence_ops *ops; +struct device_evidence *evidence; +struct device_evidence_ctx _ctx; +u32 generation; +int rc; + +if (GENL_REQ_ATTR_CHECK (info,DEVICE_EVIDENCE_A_OBJECT_GENERATION )) { +NL_SET_ERR_MSG (info->extack,"missing evidence generation" ); +return -EINVAL ; + } + generation =nla_get_u32 (info->attrs[DEVICE_EVIDENCE_A_OBJECT_GENERATION ]); + +ACQUIRE (mutex_intr, lock)(&device_evidence_lock);
drivers/base/evidence.c int device_evidence_nl_read_dumpit(struct sk_buff *skb, · line 474
Quoted 10 lines
drivers/base/evidence.c +10
-0+if ((rc =ACQUIRE_ERR (mutex_intr, &lock))) +return rc; + + struct device_evidence_ctx *ctx __free(put_ctx) = +device_evidence_ctx_setup (info, &_ctx); +if (IS_ERR (ctx)) +return PTR_ERR (ctx); + + ops = ctx->subsys->ops; + evidence = ops->evidence_read_begin (ctx->dev);
[Severity: Medium]
Will holding device_evidence_lock across the evidence_read_begin() callback cause global lock contention and system-wide stalls?
If a slow hardware refresh operation starts on one device and acquires its per-device evidence lock, and a concurrent validate command on that same device blocks in evidence_read_begin() while holding the global device_evidence_lock:
Device A (refresh):
acquires evidence->lock for write
(slow hardware communication)
Device A (validate):
device_evidence_nl_validate_doit()
acquires device_evidence_lock
evidence_read_begin()
blocks on evidence->lockIt seems this would cause any concurrent read or validate commands on unrelated devices (like Device B) to block on the global device_evidence_lock until Device A's hardware operation completes.
Could the global lock scope be reduced here to prevent this stall?
Quoted 4 lines
drivers/base/evidence.c +4
-0+if (IS_ERR (evidence)) { +NL_SET_ERR_MSG (info->extack,"failed to acquire evidence context" ); +return PTR_ERR (evidence); + }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
PATCH 6/15 PCI/TSM: Add device evidence support
In reply to Dan WilliamsRegister the PCI Trusted Execution Environment Security Manager (TSM) framework with the device-evidence netlink ABI. The security operations that PCI core coordinates with a device security manager (DSM) through a platform TEE security manager (TSM), builds upon the SPDM protocol.
A TSM owns an SPDM session and publishes the corresponding evidence through TSM firwmware ABIs. A low level TSM driver is responsible for creating a 'struct device_evidence' context and carrying out refresh_evidence() requests for regenerating measurement transcripts with a nonce.
---
drivers/pci/Kconfig | 1 +
drivers/pci/Makefile | 2 +-
drivers/pci/tsm/Makefile | 8 +++
include/linux/pci-tsm.h | 14 ++++
drivers/pci/{tsm.c => tsm/core.c} | 9 +--
drivers/pci/tsm/evidence.c | 110 ++++++++++++++++++++++++++++++
MAINTAINERS | 2 +-
7 files changed, 137 insertions(+), 9 deletions(-)
create mode 100644 drivers/pci/tsm/Makefile
rename drivers/pci/{tsm.c => tsm/core.c} (99%)
create mode 100644 drivers/pci/tsm/evidence.cdrivers/pci/Kconfig +1 -0
@@ -126,6 +126,7 @@ config PCI_IDE config PCI_TSM bool "PCI TSM: Device security protocol support" + select DEVICE_EVIDENCE if NET select PCI_IDE select PCI_DOE select TSM
drivers/pci/Makefile +1 -1
@@ -35,7 +35,7 @@ obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o obj-$(CONFIG_VGA_ARB) += vgaarb.o obj-$(CONFIG_PCI_DOE) += doe.o obj-$(CONFIG_PCI_IDE) += ide.o -obj-$(CONFIG_PCI_TSM) += tsm.o +obj-$(CONFIG_PCI_TSM) += tsm/ obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o obj-$(CONFIG_PCI_NPEM) += npem.o obj-$(CONFIG_PCIE_TPH) += tph.o
drivers/pci/tsm/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for the PCI/TSM infrastructure + +obj-$(CONFIG_PCI_TSM) += tsm.o + +tsm-y := core.o +tsm-$(CONFIG_DEVICE_EVIDENCE) += evidence.o
include/linux/pci-tsm.h +14 -0
@@ -3,7 +3,10 @@#define __PCI_TSM_H#include <linux/mutex.h> #include <linux/pci.h> +#include <linux/rwsem.h> #include <linux/sockptr.h> +#include <linux/tsm.h> +#include <linux/device/evidence.h> struct pci_tsm ;struct tsm_dev ;
@@ -18,6 +21,7 @@ enum pci_tsm_req_scope;* @devsec_ops: Lock, unlock, and interrogate the security state of the * function via the platform TSM (typically virtual function * operations). +* @refresh_evidence: Common operation to regenerate attestation objects * * This operations are mutually exclusive either a tsm_dev instance * manages physical link properties or it manages function security
@@ -75,6 +79,9 @@ struct pci_tsm_ops { struct pci_dev *pdev);void (*unlock)(struct pci_tsm *tsm); ); + +int (*refresh_evidence)(struct pci_tsm *tsm,const void *nonce, +size_t nonce_len); };/**
@@ -96,6 +103,8 @@ struct pci_tdi {* @tsm_dev: PCI TEE Security Manager device for Link Confidentiality or Device * Function Security operations * @tdi: TDI context established by the @bind link operation +* @evidence: cached evidence from SPDM session establishment (connect), or +* TDISP bind (lock) * * This structure is wrapped by low level TSM driver data and returned by * probe()/lock(), it is freed by the corresponding remove()/unlock().
@@ -112,6 +121,7 @@ struct pci_tsm {struct pci_dev *dsm_dev;struct tsm_dev *tsm_dev;struct pci_tdi *tdi; +struct device_evidence *evidence; };/**
@@ -216,6 +226,10 @@ void pci_tsm_tdi_constructor(struct pci_dev *pdev, struct pci_tdi *tdi,ssize_t pci_tsm_guest_req (struct pci_dev *pdev,enum pci_tsm_req_scope scope,sockptr_t req_in,size_t in_len,sockptr_t req_out,size_t out_len,u64 *tsm_code); +static inline const struct pci_tsm_ops *to_pci_tsm_ops (struct pci_tsm *tsm) +{ +return tsm->tsm_dev->pci_ops; +}#else static inline int pci_tsm_register (struct tsm_dev *tsm_dev) {
drivers/pci/tsm/core.c renamed from drivers/pci/tsm.c +2 -7
@@ -15,13 +15,13 @@#include <linux/sysfs.h> #include <linux/tsm.h> #include <linux/xarray.h> -#include "pci.h" +#include "../pci.h" /* * Provide a read/write lock against the init / exit of pdev tsm * capabilities and arrival/departure of a TSM instance */ -static DECLARE_RWSEM (pci_tsm_rwsem); +DECLARE_RWSEM (pci_tsm_rwsem);/* * Count of TSMs registered that support physical link operations vs device
@@ -30,11 +30,6 @@ static DECLARE_RWSEM(pci_tsm_rwsem);static int pci_tsm_link_count;static int pci_tsm_devsec_count; -static const struct pci_tsm_ops *to_pci_tsm_ops (struct pci_tsm *tsm) -{ -return tsm->tsm_dev->pci_ops; -} -static inline bool is_dsm (struct pci_dev *pdev) {return pdev->tsm && pdev->tsm->dsm_dev == pdev;
drivers/pci/tsm/evidence.c new file +110 -0
110 lines not on this page · raw message
MAINTAINERS +1 -1
@@ -27472,7 +27472,7 @@ F: Documentation/driver-api/coco/ F: Documentation/driver-api/pci/tsm.rst F: Documentation/netlink/specs/device-evidence.yaml F: drivers/base/*evidence* -F: drivers/pci/tsm.c +F: drivers/pci/tsm/ F: drivers/virt/coco/guest/ F: include/uapi/linux/device-evidence.h F: include/linux/device/evidence.h
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 06/15] PCI/TSM: Add device evidence support
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Missing NULL check for `pdev->tsm->evidence` in `pci_tsm_refresh_evidence()` leads to a NULL…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Missing NULL check for pdev->tsm->evidence in pci_tsm_refresh_evidence() leads to a NULL pointer dereference.
- [Medium] pci_tsm_evidence_read_begin() incorrectly masks wait interrupt errors, returning -ENXIO instead of the original error code. --
commit 59da43773186eeb0cfa223261094a555b176c018
Author: Dan Williams <[email protected]>
PCI/TSM: Add device evidence support
This commit registers the PCI TSM framework with the device-evidence netlink ABI. It introduces a struct device_evidence context for TSMs to own SPDM sessions and handle evidence refresh requests.
Quoted 5 lines
drivers/pci/tsm/evidence.c new file +0
-0
[ ... ]
Quoted 2 lines
drivers/pci/tsm/evidence.c new file +2
-0+static struct device_evidence *pci_tsm_evidence_read_begin (struct device *dev) +{
[ ... ]
drivers/pci/tsm/evidence.c line 60
Quoted 10 lines
drivers/pci/tsm/evidence.c new file +10
-0+/* Hold the evidence stable against conflicting refresh updates */ + evidence = pdev->tsm->evidence; + rc =down_read_interruptible (&evidence->lock); +if (rc) + goto err; + +return evidence; +err: +up_read (&pci_tsm_rwsem); +return ERR_PTR (-ENXIO );
[Severity: Medium]
Does this error path intentionally mask the error code from down_read_interruptible()?
If the wait is interrupted, rc is populated with -EINTR, but the err label unconditionally returns -ENXIO. Could this lead to userspace receiving a misleading 'No such device or address' error instead of a retryable interrupt signal?
[ ... ]
Quoted 3 lines
drivers/pci/tsm/evidence.c new file +3
-0+static int pci_tsm_refresh_evidence (struct device *dev,const void *nonce, +size_t nonce_len) +{
[ ... ]
drivers/pci/tsm/evidence.c line 91
Quoted 3 lines
drivers/pci/tsm/evidence.c new file +3
-0+/* Sync against pci_tsm_evidence_read_begin */ + evidence = pdev->tsm->evidence; +ACQUIRE (rwsem_write_kill, elock)(&evidence->lock);
[Severity: High]
Is it possible for pdev->tsm->evidence to be NULL here?
Looking at the evidence_available() helper earlier in this file, it appears structurally valid for a TSM to exist without an associated evidence context.
If evidence is NULL, would passing &evidence->lock into ACQUIRE() cause a NULL pointer dereference?
Quoted 5 lines
drivers/pci/tsm/evidence.c new file +5
-0+if ((rc =ACQUIRE_ERR (rwsem_write_kill, &elock))) +return rc; + +return ops->refresh_evidence (pdev->tsm, nonce, nonce_len); +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Alexey Kardashevskiy
maintainerRe: [PATCH 06/15] PCI/TSM: Add device evidence support
In reply to Dan WilliamsQuoted 8 lines · Dan Williams
Register the PCI Trusted Execution Environment Security Manager (TSM) framework with the device-evidence netlink ABI. The security operations that PCI core coordinates with a device security manager (DSM) through a platform TEE security manager (TSM), builds upon the SPDM protocol.
A TSM owns an SPDM session and publishes the corresponding evidence through TSM firwmware ABIs. A low level TSM driver is responsible for creating a 'struct
device_evidence' context and carrying outrefresh_evidence()
a nit: so it is really refresh_measurements() as it won't refresh the report? may be call it so?
Quoted 66 lines
requests for regenerating measurement transcripts with a nonce.
Cc Bjorn Helgaas <[email protected]>Cc Alistair Francis <[email protected]>Cc Lukas Wunner <[email protected]>Cc Xu Yilun <[email protected]>Cc "Aneesh Kumar K.V (Arm)" <[email protected]>Cc Alexey Kardashevskiy <[email protected]>Signed-off-by Dan Williams <[email protected]>--- drivers/pci/Kconfig | 1 + drivers/pci/Makefile | 2 +- drivers/pci/tsm/Makefile | 8 +++ include/linux/pci-tsm.h | 14 ++++ drivers/pci/{tsm.c => tsm/core.c} | 9 +-- drivers/pci/tsm/evidence.c | 110 ++++++++++++++++++++++++++++++ MAINTAINERS | 2 +- 7 files changed, 137 insertions(+), 9 deletions(-)create mode 100644
drivers/pci/tsm/Makefile
renamedrivers/pci/{tsm.c=>tsm/core.c} (99%)
create mode 100644drivers/pci/tsm/evidence.cdrivers/pci/Kconfig +1
-0@@ -126,6 +126,7 @@ config PCI_IDE config PCI_TSM bool "PCI TSM: Device security protocol support" + select DEVICE_EVIDENCE if NET select PCI_IDE select PCI_DOE select TSMdrivers/pci/Makefile +1
-1@@ -35,7 +35,7 @@ obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o obj-$(CONFIG_VGA_ARB) += vgaarb.o obj-$(CONFIG_PCI_DOE) += doe.o obj-$(CONFIG_PCI_IDE) += ide.o -obj-$(CONFIG_PCI_TSM) += tsm.o +obj-$(CONFIG_PCI_TSM) += tsm/ obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o obj-$(CONFIG_PCI_NPEM) += npem.o obj-$(CONFIG_PCIE_TPH) += tph.odrivers/pci/tsm/Makefile new file +8
-0@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for the PCI/TSM infrastructure + +obj-$(CONFIG_PCI_TSM) += tsm.o + +tsm-y := core.o +tsm-$(CONFIG_DEVICE_EVIDENCE) += evidence.oinclude/linux/pci-tsm.h +0
-0
include/linux/pci-tsm.h line 6
Quoted 4 lines
include/linux/pci-tsm.h +1
-0@@ -3,7 +3,10 @@#define __PCI_TSM_H#include <linux/mutex.h> #include <linux/pci.h> +#include <linux/rwsem.h>
then this file is missing extern struct rw_semaphore pci_tsm_rwsem;
drivers/pci/tsm/core.c
Quoted 56 lines
include/linux/pci-tsm.h +13
-0#include <linux/sockptr.h> +#include <linux/tsm.h> +#include <linux/device/evidence.h> struct pci_tsm ;struct tsm_dev ;@@ -18,6 +21,7 @@ enum pci_tsm_req_scope;* @devsec_ops: Lock, unlock, and interrogate the security state of the * function via the platform TSM (typically virtual function * operations). +* @refresh_evidence: Common operation to regenerate attestation objects * * This operations are mutually exclusive either a tsm_dev instance * manages physical link properties or it manages function security @@ -75,6 +79,9 @@ struct pci_tsm_ops { struct pci_dev *pdev);void (*unlock)(struct pci_tsm *tsm); ); + +int (*refresh_evidence)(struct pci_tsm *tsm,const void *nonce, +size_t nonce_len); };/** @@ -96,6 +103,8 @@ struct pci_tdi {* @tsm_dev: PCI TEE Security Manager device for Link Confidentiality or Device * Function Security operations * @tdi: TDI context established by the @bind link operation +* @evidence: cached evidence from SPDM session establishment (connect), or +* TDISP bind (lock) * * This structure is wrapped by low level TSM driver data and returned by * probe()/lock(), it is freed by the corresponding remove()/unlock(). @@ -112,6 +121,7 @@ struct pci_tsm {struct pci_dev *dsm_dev;struct tsm_dev *tsm_dev;struct pci_tdi *tdi; +struct device_evidence *evidence; };/** @@ -216,6 +226,10 @@ void pci_tsm_tdi_constructor(struct pci_dev *pdev, struct pci_tdi *tdi,ssize_t pci_tsm_guest_req (struct pci_dev *pdev,enum pci_tsm_req_scope scope,sockptr_t req_in,size_t in_len,sockptr_t req_out,size_t out_len,u64 *tsm_code); +static inline const struct pci_tsm_ops *to_pci_tsm_ops (struct pci_tsm *tsm) +{ +return tsm->tsm_dev->pci_ops; +}#else static inline int pci_tsm_register (struct tsm_dev *tsm_dev) {drivers/pci/tsm/core.c renamed from drivers/pci/tsm.c +0
-0
This move + related makefile changes + exposing pci_tsm_rwsem are better be in a separate patch. Thanks,
Quoted 160 lines
index
5fdcd7f2e820..220842df42bc100644drivers/pci/tsm/core.c renamed from drivers/pci/tsm.c +2
-7@@ -15,13 +15,13 @@#include <linux/sysfs.h> #include <linux/tsm.h> #include <linux/xarray.h> -#include "pci.h" +#include "../pci.h" /* * Provide a read/write lock against the init / exit of pdev tsm * capabilities and arrival/departure of a TSM instance */ -static DECLARE_RWSEM (pci_tsm_rwsem); +DECLARE_RWSEM (pci_tsm_rwsem);/* * Count of TSMs registered that support physical link operations vs device @@ -30,11 +30,6 @@ static DECLARE_RWSEM(pci_tsm_rwsem);static int pci_tsm_link_count;static int pci_tsm_devsec_count; -static const struct pci_tsm_ops *to_pci_tsm_ops (struct pci_tsm *tsm) -{ -return tsm->tsm_dev->pci_ops; -} -static inline bool is_dsm (struct pci_dev *pdev) {return pdev->tsm && pdev->tsm->dsm_dev == pdev;drivers/pci/tsm/evidence.c new file +110
-0@@ -0,0 +1,110 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (C) 2026 NVIDIA Corporation & Affiliates */ + +#include <linux/device/evidence.h> +#include <linux/init.h> +#include <linux/mutex.h> +#include <linux/pci.h> +#include <linux/pci-tsm.h> +#include <linux/tsm.h> + +extern struct rw_semaphore pci_tsm_rwsem; + +static bool evidence_available (struct pci_dev *pdev) +{ +return pdev->tsm && pdev->tsm->evidence; +} + +static struct device *pci_tsm_evidence_find_device (const char *name) +{ +ACQUIRE (rwsem_read_intr, lock)(&pci_tsm_rwsem); +if (ACQUIRE_ERR (rwsem_read_intr, &lock)) +return NULL ; + + struct device *dev __free(put_device) = +bus_find_device_by_name (&pci_bus_type,NULL , name); + +/* +* Bail evidence gathering early if we know at this point that +* the device has no valid evidence provider, but still need to +* revalidate the same in pci_tsm_evidence_read_begin(). +*/ +if (!dev || !evidence_available (to_pci_dev (dev))) +return NULL ; + +return no_free_ptr (dev); +} + +static struct device_evidence *pci_tsm_evidence_read_begin (struct device *dev) +{ +struct pci_dev *pdev =to_pci_dev (dev); +struct device_evidence *evidence; +int rc; + + rc =down_read_interruptible (&pci_tsm_rwsem); +if (rc) +return ERR_PTR (rc); + +if (!evidence_available (pdev)) + goto err; + +/* Hold the evidence stable against conflicting refresh updates */ + evidence = pdev->tsm->evidence; + rc =down_read_interruptible (&evidence->lock); +if (rc) + goto err; + +return evidence; +err: +up_read (&pci_tsm_rwsem); +return ERR_PTR (-ENXIO ); +} + +static void pci_tsm_evidence_read_end (struct device_evidence *evidence) +{ +up_read (&evidence->lock); +up_read (&pci_tsm_rwsem); +} + +static int pci_tsm_refresh_evidence (struct device *dev,const void *nonce, +size_t nonce_len) +{ +struct pci_dev *pdev =to_pci_dev (dev); +struct device_evidence *evidence; +const struct pci_tsm_ops *ops; +int rc; + +/* Sync against disconnect */ +ACQUIRE (rwsem_read_intr, lock)(&pci_tsm_rwsem); +if ((rc =ACQUIRE_ERR (rwsem_read_intr, &lock))) +return rc; + +if (!pdev->tsm) +return -ENXIO ; + + ops =to_pci_tsm_ops (pdev->tsm); +if (!ops->refresh_evidence) +return -EOPNOTSUPP ; + +/* Sync against pci_tsm_evidence_read_begin */ + evidence = pdev->tsm->evidence; +ACQUIRE (rwsem_write_kill, elock)(&evidence->lock); +if ((rc =ACQUIRE_ERR (rwsem_write_kill, &elock))) +return rc; + +return ops->refresh_evidence (pdev->tsm, nonce, nonce_len); +} + +static const struct device_evidence_ops pci_tsm_evidence_ops = { + .subsys_name ="pci" , + .find_device = pci_tsm_evidence_find_device, + .evidence_read_begin = pci_tsm_evidence_read_begin, + .evidence_read_end = pci_tsm_evidence_read_end, + .refresh_evidence = pci_tsm_refresh_evidence, +}; + +static int __init pci_tsm_evidence_init (void ) +{ +return device_evidence_register (&pci_tsm_evidence_ops); +} +subsys_initcall (pci_tsm_evidence_init);MAINTAINERS +1
-1@@ -27472,7 +27472,7 @@ F: Documentation/driver-api/coco/ F: Documentation/driver-api/pci/tsm.rst F: Documentation/netlink/specs/device-evidence.yaml F: drivers/base/*evidence* -F: drivers/pci/tsm.c +F: drivers/pci/tsm/ F: drivers/virt/coco/guest/ F: include/uapi/linux/device-evidence.h F: include/linux/device/evidence.h
Signature · 1 line
Alexey
Re: [PATCH 06/15] PCI/TSM: Add device evidence support
Quoted 6 lines · Alexey Kardashevskiy
Quoted 5 lines · Dan Williams
Register the PCI Trusted Execution Environment Security Manager (TSM) framework with the device-evidence netlink ABI. The security operations that PCI core coordinates with a device security manager (DSM) through a platform TEE security manager (TSM), builds upon the SPDM protocol.
Quoted 6 lines
Quoted 3 lines
A TSM owns an SPDM session and publishes the corresponding evidence through TSM firwmware ABIs. A low level TSM driver is responsible for creating a 'struct
device_evidence' context and carrying outrefresh_evidence()a nit: so it is really
refresh_measurements()as it won't refresh the report? may be call it so?
Sure.
Quoted 1 line
Quoted 1 line
requests for regenerating measurement transcripts with a nonce.
[..]
Quoted 11 lines
include/linux/pci-tsm.h line 6
Quoted 9 lines
include/linux/pci-tsm.h +1
-0@@ -3,7 +3,10 @@#define __PCI_TSM_H#include <linux/mutex.h> #include <linux/pci.h> +#include <linux/rwsem.h> then this file is missing extern struct
rw_semaphorepci_tsm_rwsem;
This include was a mistake. pci_tsm_rwsem is private to drivers/pci/tsm/, so I will remove this include. This is the only shared data structure between drivers/pci/tsm/core.c and drivers/pci/tsm/evidence.c. It could go go in a drivers/pci/tsm/private.h, but given just the one declaration have evidence.c do it directly.
Quoted 24 lines
Quoted 24 lines
include/linux/pci-tsm.h +6
-0#include <linux/sockptr.h> +#include <linux/tsm.h> +#include <linux/device/evidence.h> struct pci_tsm ;struct tsm_dev ;@@ -18,6 +21,7 @@ enum pci_tsm_req_scope;* @devsec_ops: Lock, unlock, and interrogate the security state of the * function via the platform TSM (typically virtual function * operations). +* @refresh_evidence: Common operation to regenerate attestation objects * * This operations are mutually exclusive either a tsm_dev instance * manages physical link properties or it manages function security @@ -75,6 +79,9 @@ struct pci_tsm_ops { struct pci_dev *pdev);void (*unlock)(struct pci_tsm *tsm); ); + +int (*refresh_evidence)(struct pci_tsm *tsm,const void *nonce, +size_t nonce_len); };/**
[..]
Quoted 9 lines
Quoted 9 lines
include/linux/pci-tsm.h +4
-0@@ -216,6 +226,10 @@ void pci_tsm_tdi_constructor(struct pci_dev *pdev, struct pci_tdi *tdi,ssize_t pci_tsm_guest_req (struct pci_dev *pdev,enum pci_tsm_req_scope scope,sockptr_t req_in,size_t in_len,sockptr_t req_out,size_t out_len,u64 *tsm_code); +static inline const struct pci_tsm_ops *to_pci_tsm_ops (struct pci_tsm *tsm) +{ +return tsm->tsm_dev->pci_ops; +}#else
Quoted 8 lines
drivers/pci/tsm/core.c
Quoted 6 lines
include/linux/pci-tsm.h +0
-0static inline int pci_tsm_register (struct tsm_dev *tsm_dev) {drivers/pci/tsm/core.c renamed from drivers/pci/tsm.c +0
-0This move + related makefile changes + exposing
pci_tsm_rwsemare better be in a separate patch. Thanks,
I'll do a prep patch.
PATCH 7/15 modules: Document the global async_probe parameter
In reply to Dan WilliamsIn preparation for adding another /sys/module/module/parameters entry, document the existing async_probe parameter.
--- Documentation/ABI/stable/sysfs-module | 10 ++++++++++ 1 file changed, 10 insertions(+)
Documentation/ABI/stable/sysfs-module +10 -0
@@ -45,3 +45,13 @@ Date: Jun 2005 Description: If the module source has MODULE_VERSION, this file will contain the version of the source code. + +What: /sys/module/module/parameters/async_probe +Description: + (RW) Emits "1" if drivers from loadable modules attempt async + probing by default. Emits "0" if drivers from loadable modules + attempt synchronous probing by default. This value is overridden + (in priority order) by: the module's built-in "PROBE_FORCE_*" + requests, the "driver_async_probe=..." kernel command line, the + "async_probe" module option, then this default. Write a valid + boolean value to toggle this policy.
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 07/15] modules: Document the global async_probe parameter
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines
In preparation for adding another
/sys/module/module/parametersentry, document the existingasync_probeparameter.Cc Saravana Kannan <[email protected]>Cc Luis Chamberlain <[email protected]>Signed-off-by Dan Williams <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Re: [PATCH 07/15] modules: Document the global async_probe parameter
In reply to Dan WilliamsQuoted 15 lines · Dan Williams
In preparation for adding another
/sys/module/module/parametersentry, document the existingasync_probeparameter.Cc Saravana Kannan <[email protected]>Cc Luis Chamberlain <[email protected]>Signed-off-by Dan Williams <[email protected]>--- Documentation/ABI/stable/sysfs-module | 10 ++++++++++ 1 file changed, 10 insertions(+)Documentation/ABI/stable/sysfs-module +0
-0
Documentation/ABI/stable/sysfs-module Date: Jun 2005 · line 52
Quoted 8 lines
Documentation/ABI/stable/sysfs-module +5
-0@@ -45,3 +45,13 @@ Date: Jun 2005 Description: If the module source has MODULE_VERSION, this file will contain the version of the source code. + +What: /sys/module/module/parameters/async_probe +Description: + (RW) Emits "1" if drivers from loadable modules attempt async + probing by default. Emits "0" if drivers from loadable modules
nit: I think there is some tautology happening here, i.e a module is a driver, no ? I.e perhaps just remove the driver word so this reads "emits 1 if modules atempts async probing".
Quoted 5 lines
Documentation/ABI/stable/sysfs-module +5
-0+ attempt synchronous probing by default. This value is overridden + (in priority order) by: the module's built-in "PROBE_FORCE_*" + requests, the "driver_async_probe=..." kernel command line, the + "async_probe" module option, then this default. Write a valid + boolean value to toggle this policy.
PATCH 8/15 device core: Initial device trust infrastructure
In reply to Dan WilliamsIntroduce a per-device trust concept [1]. To start, it plumbs "auto" and "none" levels to enable a scheme of requiring explicit opt-in to operate a device. This supports a Confidential Computing use case of operating a limited / vetted device and driver set to bootstrap the environment. It leans on module policy to effect changes to device's trust level.
The motivations for using module policy to effect trust are:
1/ It exists, no new ABI needed to inject "device firewall" policy into the
kernel [2].
2/ Trust is a function of how the device is operated. A driver can be
trusted to carefully operate an untrusted device.
It may prove too coarse for some situations, like a driver that talks to a mix of trusted and untrusted devices, or a module that contains multiple drivers, but it is a place to start.
Specifically this "module trust" scheme allows for only attaching drivers from built-in modules by default, or modules that explicitly pass a "trust=auto" option by building with CONFIG_DEVICE_TRUST_NONE=y. Modules can also be configured to skip device attach by default by setting /sys/module/module/parameters/require_trust.
The new dev->p->trust level is the operational trust level of a device. It is only in effect after device_add(). It is private to the device core because the "tcb" level is not suitable to be changed while a driver is attached to the device. The result force_dma_unencrypted() must be stable while the device is attached to its driver. That core coordination responsibility is not suitable to export to non-core code.
Note that the overly trivial helper builtin_driver_trust() grows support for another trust level in a follow-on change.
Also note that the IOMMU claims in the Kconfig commentary, where it says "possible" and "where available" imply follow-on work to assign DEVICE_TRUST_NONE devices to an IOMMU blocked domain.
--- drivers/base/Kconfig | 57 ++++++++++++++++++++ drivers/base/Makefile | 1 + Documentation/ABI/stable/sysfs-module | 7 +++ drivers/base/base.h | 15 ++++++ include/linux/device/trust.h | 42 +++++++++++++++ include/linux/module.h | 6 ++- drivers/base/core.c | 7 +++ drivers/base/dd.c | 5 ++ drivers/base/trust.c | 77 +++++++++++++++++++++++++++ kernel/module/main.c | 13 +++++ 10 files changed, 229 insertions(+), 1 deletion(-) create mode 100644 include/linux/device/trust.h create mode 100644 drivers/base/trust.c
drivers/base/Kconfig +57 -0
@@ -262,4 +262,61 @@ config FW_DEVLINK_SYNC_STATE_TIMEOUT command line option on every system/board your kernel is expected to work on. +config DEVICE_TRUST + bool "Device core support for specifying trust levels for devices" + help + Enable support for generic device trust levels when building a kernel + that needs to operate in the presence of potentially adversarial + devices. This is selected by buses that want to operate devices with + reduced privileges, like externally connected devices, and it is a + pre-requisite for operating devices with access to private memory in a + Confidential Computing VM. See + Documentation/ABI/testing/sysfs-kernel-iommu_groups for the type of + IOMMU enforcement in effect. + +choice + prompt "Default device trust" + default DEVICE_TRUST_AUTO + depends on DEVICE_TRUST + help + Specify the device trust level at initial attachment. Any choice other + than "Auto" assumes auditing the built-in driver set for trusted + drivers, and an enlightened userspace modprobe policy for other + devices + drivers. + +config DEVICE_TRUST_NONE + bool "None" + help + Devices are disallowed from attaching to a driver, and where + possible, the device is blocked by an IOMMU from accessing + assets. + +config DEVICE_TRUST_AUTO + bool "Auto" + help + Typical historical driver model, devices eagerly attempt to attach to + a driver and deploy all available mechanisms to allow performant + direct memory access. + +endchoice + +choice + prompt "Trust devices with built-in drivers" + default BUILTIN_DEVICE_TRUST_AUTO + depends on DEVICE_TRUST + help + Built-in drivers always bind to devices they match. Only + select a setting other than "Auto" when building a kernel image + targeted for use in Confidential Computing or other known + adversarial environments. + +config BUILTIN_DEVICE_TRUST_AUTO + bool "Auto" + help + Typical historical driver model, devices eagerly attempt to attach to + a driver and deploy all available mechanisms to allow performant + direct memory access This trust level does not grant TCB privileges. + +endchoice + endmenu
drivers/base/Makefile +1 -0
@@ -8,6 +8,7 @@ obj-y := component.o core.o bus.o dd.o syscore.o \ topology.o container.o property.o cacheinfo.o \ swnode.o faux.o obj-$(CONFIG_AUXILIARY_BUS) += auxiliary.o +obj-$(CONFIG_DEVICE_TRUST) += trust.o obj-$(CONFIG_DEVTMPFS) += devtmpfs.o obj-y += power/ obj-$(CONFIG_ISA_BUS_API) += isa.o
Documentation/ABI/stable/sysfs-module +7 -0
@@ -55,3 +55,10 @@ Description: requests, the "driver_async_probe=..." kernel command line, the "async_probe" module option, then this default. Write a valid boolean value to toggle this policy. + +What: /sys/module/module/parameters/require_trust +Description: + (RW) Emits "1" if modules are required to specify a trust= + option on load to permit driver(s) in the module to attach to + matched devices. By default drivers always attach on match + unless the kernel was compiled with DEVICE_TRUST_NONE.
drivers/base/base.h +15 -0
@@ -103,6 +103,7 @@ struct driver_private {* dev_err_probe() for later retrieval via debugfs * @device: pointer back to the struct device that this structure is * associated with. +* @trust: device operational trust level * @dead: This device is currently either in the process of or has been * removed from the system. Any asynchronous events scheduled for this * device should exit without taking any action.
@@ -119,6 +120,7 @@ struct device_private {const struct device_driver *async_driver;char *deferred_probe_reason;struct device *device; +enum device_trust trust; u8 dead:1; };#define to_device_private_parent (obj) \
@@ -149,6 +151,19 @@ void auxiliary_bus_init(void);#else static inline void auxiliary_bus_init (void ) { }#endif +#ifdef CONFIG_DEVICE_TRUST +bool device_trust_bind (const struct device_driver *drv,struct device *dev); +void device_initialize_trust (struct device *dev); +#else +static inline void device_initialize_trust (struct device *dev) +{ +} +static inline bool device_trust_bind (const struct device_driver *drv, +struct device *dev) +{ +return true; +} +#endif struct kobject *virtual_device_parent (void );
include/linux/device/trust.h new file +42 -0
42 lines not on this page · raw message
include/linux/module.h +5 -1
@@ -29,6 +29,7 @@#include <linux/srcu.h> #include <linux/static_call_types.h> #include <linux/dynamic_debug.h> +#include <linux/device/trust.h> #include <linux/percpu.h> #include <asm/module.h>
@@ -443,6 +444,10 @@ struct module {#endif bool async_probe_requested; +#ifdef CONFIG_DEVICE_TRUST +/* Trust level override for devices matching this module's drivers. */ +enum device_trust trust; +#endif /* Exception table */ unsigned int num_exentries;
@@ -865,7 +870,6 @@ static inline bool module_requested_async_probing(struct module *module)return false; } -/* Dereference module function descriptor */ static inline void *dereference_module_function_descriptor (struct module *mod,void *ptr)
drivers/base/core.c +7 -0
@@ -3754,6 +3754,13 @@ int device_add(struct device *dev)fw_devlink_link_device (dev); } +/* +* The bus and all device add notifiers have had a chance to +* initialize trust, set a fallback default in case the trust +* level is not yet established. +*/ +device_initialize_trust (dev); +/* * The moment the device was linked into the bus's "klist_devices" in * bus_add_device() then it's possible that probe could have been
drivers/base/dd.c +5 -0
@@ -857,6 +857,11 @@ static int __driver_probe_device(const struct device_driver *drv, struct devicedev_dbg (dev,"bus: '%s': %s: matched device with driver %s\n" , drv->bus->name, __func__, drv->name); +if (!device_trust_bind (drv, dev)) { +dev_dbg (dev,"not trusted to bind\n" ); +return -ENODEV ; + } +pm_runtime_get_suppliers (dev);if (dev->parent)pm_runtime_get_sync (dev->parent);
drivers/base/trust.c new file +77 -0
77 lines not on this page · raw message
kernel/module/main.c +13 -0
@@ -36,6 +36,7 @@#include <linux/notifier.h> #include <linux/sched.h> #include <linux/device.h> +#include <linux/device/trust.h> #include <linux/string.h> #include <linux/mutex.h> #include <linux/rculist.h>
@@ -3068,6 +3069,12 @@ void flush_module_init_free_work(void)static bool async_probe;module_param (async_probe, bool,0644 ); +#ifdef CONFIG_DEVICE_TRUST +/* Default value for module trust. When true, trust= override required */ +static bool require_trust; +module_param (require_trust, bool,0644 ); +#endif +/* * This is where the real work happens. *
@@ -3375,6 +3382,11 @@ static int unknown_module_param_cb(char *param, char *val, const char *modname,return 0 ; } +if (strcmp (param,"trust" ) ==0 ) { +module_driver_trust (mod, val); +return 0 ; + } +/* Check for magic 'dyndbg' arg */ ret =ddebug_dyndbg_module_param_cb (param, val, modname);if (ret !=0 )
@@ -3544,6 +3556,7 @@ static int load_module(struct load_info *info, const char __user *uargs, goto bug_cleanup; mod->async_probe_requested = async_probe; +module_driver_trust_init (mod, require_trust);/* Module is ready to execute: parsing args may do that. */ after_dashes = parse_args(mod->name, mod->args, mod->kp, mod->num_kp,
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 08/15] device core: Initial device trust infrastructure
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] A device's operational trust level (`dev->p->trust`) is persistently mutated before probe and not…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] A device's operational trust level (dev->p->trust) is persistently mutated before probe and not reverted on failure or unbind, leaking elevated privileges.
- [High] Providing an invalid trust= module parameter silently bypasses the restrictive module trust policy.
- [Low] The kernel fails to compile when CONFIG_DEVICE_TRUST is disabled due to an undeclared require_trust variable.
--
commit b17dbaab0dbd4dc3c3e572b9b6cec407be3b263a
Author: Dan Williams <[email protected]>
device core: Initial device trust infrastructure
Introduce a per-device trust concept. To start, it plumbs "auto" and "none" levels to enable a scheme of requiring explicit opt-in to operate a device. This supports a Confidential Computing use case of operating a limited / vetted device and driver set to bootstrap the environment.
drivers/base/dd.c static int __driver_probe_device(const struct device_driver *drv, struct device · line 864
Quoted 13 lines
drivers/base/dd.c +5
-0@@ -857,6 +857,11 @@ static int __driver_probe_device(const struct device_driver *drv, struct devicedev_dbg (dev,"bus: '%s': %s: matched device with driver %s\n" , drv->bus->name, __func__, drv->name); +if (!device_trust_bind (drv, dev)) { +dev_dbg (dev,"not trusted to bind\n" ); +return -ENODEV ; + } +
[Severity: High]
If really_probe() fails later in this function, or if the driver is unbound via sysfs, does the device retain its newly elevated operational trust level?
When device_trust_bind() is called, it destructively overwrites dev->p->trust:
drivers/base/trust.c:device_trust_bind() {
...
if (drv_trust != DEVICE_TRUST_UNSET)
dev->p->trust = drv_trust;
...
}Should there be a mechanism to rollback the device's trust level to its baseline state on probe failure or driver unbind?
Quoted 8 lines
drivers/base/dd.c +0
-0pm_runtime_get_suppliers (dev);if (dev->parent)pm_runtime_get_sync (dev->parent);kernel/module/main.c +0
-0
[ ... ]
kernel/module/main.c static int unknown_module_param_cb(char *param, char *val, const char *modname, · line 3389
Quoted 9 lines
kernel/module/main.c +5
-0@@ -3375,6 +3382,11 @@ static int unknown_module_param_cb(char *param, char *val, const char *modname,return 0 ; } +if (strcmp (param,"trust" ) ==0 ) { +module_driver_trust (mod, val); +return 0 ; + } +
[Severity: High]
Can a malicious or unvetted driver bypass the explicit opt-in policy (require_trust=1) by passing an invalid trust parameter?
If an unrecognized value like "trust=garbage" is passed, device_trust_parse() returns DEVICE_TRUST_UNSET (0). module_driver_trust() will then overwrite mod->trust with DEVICE_TRUST_UNSET.
Because this function unconditionally returns 0 (success) without checking for invalid input, the module load succeeds. During probe, device_trust_bind() ignores DEVICE_TRUST_UNSET and falls back to the device's existing trust level:
drivers/base/trust.c:device_trust_bind() {
enum device_trust drv_trust = driver_trust(drv->owner);
if (drv_trust != DEVICE_TRUST_UNSET)
dev->p->trust = drv_trust;
return dev->p->trust > DEVICE_TRUST_NONE;
}This seems to erase the restrictive DEVICE_TRUST_NONE default policy and might allow an unvetted driver to successfully attach. Should we return an error or reject the load on invalid trust= values?
Quoted 2 lines
kernel/module/main.c +0
-0/* Check for magic 'dyndbg' arg */ ret =ddebug_dyndbg_module_param_cb (param, val, modname);
[ ... ]
kernel/module/main.c static int load_module(struct load_info *info, const char __user *uargs, · line 3560
Quoted 6 lines
kernel/module/main.c +1
-0@@ -3544,6 +3556,7 @@ static int load_module(struct load_info *info, const char __user *uargs, goto bug_cleanup; mod->async_probe_requested = async_probe; +module_driver_trust_init (mod, require_trust);
[Severity: Low]
Does this cause a build failure when CONFIG_DEVICE_TRUST is disabled?
The require_trust variable is declared inside an #ifdef CONFIG_DEVICE_TRUST block earlier in this file, but it is passed unconditionally here.
Since module_driver_trust_init() is defined as a static inline function in include/linux/device/trust.h when the config is disabled, the compiler will still evaluate its arguments, which would result in an undeclared variable error.
Quoted 2 lines
kernel/module/main.c +0
-0/* Module is ready to execute: parsing args may do that. */ after_dashes = parse_args(mod->name, mod->args, mod->kp, mod->num_kp,
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Re: [PATCH 08/15] device core: Initial device trust infrastructure
In reply to Dan WilliamsQuoted 8 lines · Dan Williams
Introduce a per-device trust concept [1]. To start, it plumbs "auto" and "none" levels to enable a scheme of requiring explicit opt-in to operate a device. This supports a Confidential Computing use case of operating a limited / vetted device and driver set to bootstrap the environment. It leans on module policy to effect changes to device's trust level.
The motivations for using module policy to effect trust are:
Quoted 8 lines
1/ It exists, no new ABI needed to inject "device firewall" policy into the
kernel [2].
2/ Trust is a function of how the device is operated. A driver can be
trusted to carefully operate an untrusted device.It may prove too coarse for some situations, like a driver that talks to a mix of trusted and untrusted devices, or a module that contains multiple drivers, but it is a place to start.
I'm a little confused what this is is for?
Is this how we bootstrap enough drivers to get into the initrd? Ie is it only relavent for key boot time things like CLOCK, IRQ and IOMMU drivers?
Once we get to the initrd eveything should be driven on a per-device basis without anything to do with modules. We already have cases for mlx5 where it will have to drive trusted and untrusted devices in the same system.
Quoted 5 lines
Specifically this "module trust" scheme allows for only attaching drivers from built-in modules by default, or modules that explicitly pass a "trust=auto" option by building with
CONFIG_DEVICE_TRUST_NONE=y. Modules can also be configured to skip device attach by default by setting/sys/module/module/parameters/require_trust.
So I'd imagine most systems would make all modules have require_trust=1 and some userspace compoment in the initrd will evaluate and assign trust to devices to trigger binding and module loading?
Quoted 6 lines
The new
dev->p->trustlevel is the operational trust level of a device. It is only in effect afterdevice_add(). It is private to the device core because the "tcb" level is not suitable to be changed while a driver is attached to the device. The resultforce_dma_unencrypted()must be stable while the device is attached to its driver. That core coordination responsibility is not suitable to export to non-core code.
I don't think force_dma_unencrypted() should be driven from the trust level.. It is a property of the interconnect, the trust level sits on top of that.
Quoted 18 lines
drivers/base/Kconfig +18
-0+config DEVICE_TRUST + bool "Device core support for specifying trust levels for devices" + help + Enable support for generic device trust levels when building a kernel + that needs to operate in the presence of potentially adversarial + devices. This is selected by buses that want to operate devices with + reduced privileges, like externally connected devices, and it is a + pre-requisite for operating devices with access to private memory in a + Confidential Computing VM. See + Documentation/ABI/testing/sysfs-kernel-iommu_groups for the type of + IOMMU enforcement in effect. + +choice + prompt "Default device trust" + default DEVICE_TRUST_AUTO + depends on DEVICE_TRUST + help + Specify the device trust level at initial attachment. Any choice other
drivers/base/Kconfig config FW_DEVLINK_SYNC_STATE_TIMEOUT · line 292
Quoted 10 lines
drivers/base/Kconfig +10
-0+ than "Auto" assumes auditing the built-in driver set for trusted + drivers, and an enlightened userspace modprobe policy for other + devices + drivers. + +config DEVICE_TRUST_NONE + bool "None" + help + Devices are disallowed from attaching to a driver, and where + possible, the device is blocked by an IOMMU from accessing + assets.
It is a confusing what devices this applies to? I guess it is all devices that are not built in?
I think it should also be in a kernel commandline, we should be able to support a single kernel image that works for CC VM and non CC VM, I think. You'd want the CC VM to have a kernel command line to turn on some of this stuff.
Quoted 6 lines
drivers/base/Kconfig +6
-0+choice + prompt "Trust devices with built-in drivers" + default BUILTIN_DEVICE_TRUST_AUTO + depends on DEVICE_TRUST + help + Built-in drivers always bind to devices they match. Only
drivers/base/Kconfig config FW_DEVLINK_SYNC_STATE_TIMEOUT · line 318
Quoted 10 lines
drivers/base/Kconfig +10
-0+ select a setting other than "Auto" when building a kernel image + targeted for use in Confidential Computing or other known + adversarial environments. + +config BUILTIN_DEVICE_TRUST_AUTO + bool "Auto" + help + Typical historical driver model, devices eagerly attempt to attach to + a driver and deploy all available mechanisms to allow performant + direct memory access This trust level does not grant TCB privileges.
So what's the other choice?
Is this "AUTO" or is it just all builtin devices?
I think it does grant TCB privileges in some cases, ie the VIOMMU driver that binds is in the TCB and does have T=1 equivilent DMA access.
What we want for a CC VM is a curated set of pre-initrd devices can auto probe:
- Drivers internally have some kind of attestation, eg on ARM the VIOMMU driver is going to have to ask the RMM to prove the VIOMMU is legitimate
- Drivers that don't do any DMA at all, and have approved drivers
- Drivers that are known to be hardened for untrusted DMA
Maybe these options should be named a bit differently?
config DEVICE_TRUST <-- If the device trust framework is compiled in at all.
DEVICE_TRUST_BUILTIN_DEFAULT = ALL, AUTO
Decide how pre-initrd devices are handled. ALL is current Linux, every built in driver binds and runs, AUTO follows some module policy?
DEVICE_TRUST_MODULE_DEFUALT = AUTO, NONE
Decide how post-initrd devices are handled. AUTO is current Linux, follow the ACPI/etc. NONE means the initrd must run a policy agent to assign trust.
Quoted 4 lines
include/linux/device/trust.h new file +4
-0+/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2026 NVIDIA Corporation & Affiliates */ +#ifndef __DEVICE_TRUST_H__ +#define __DEVICE_TRUST_H__
include/linux/device/trust.h line 14
Quoted 10 lines
include/linux/device/trust.h new file +10
-0+ +/** +* enum device_trust - Level of restrictions and privileges for a +* device. Trust is initially assigned by the bus, and the bus is +* responsible for coordinating transitions between trust levels with +* DMA/IOMMU and its own device security mechanisms. +* +* @DEVICE_TRUST_UNSET: Unregistered device object with no current bus +* @DEVICE_TRUST_NONE: Blocked when idle, cannot bind +* @DEVICE_TRUST_AUTO: All typical privileges granted
I expect there would be two enums, the policy - eg from the sysfs/etc which is possibly this, and then the actual in-effect trust level on the actual device which should only be DISABLED/FULL/ADVERSARY ?
So maybe call this enum device_trust_policy ?
Jason
PATCH 9/15 PCI, device core: Move "untrusted" concept to DEVICE_TRUST_ADVERSARY
In reply to Dan WilliamsAn "adversary" device is one that the kernel is allowed to operate, but with extra safety / paranoia added. It aims to keep the device bounded in both address space and time (no batched invalidation) for DMA accesses. The concept of adversarial devices already exists in the PCI core. The categorization is applied to externally attached PCI devices by default (like Thunderbolt attached devices). Unlike DEVICE_TRUST_NONE that says "do not allow driver bind", DEVICE_TRUST_ADVERSARY says "allow driver bind as long as the driver and/or IOMMU are taking precautions." Uplevel this trust mechanism from a PCI device "untrusted" boolean flag to the core device trust level.
The rationale for dev->bus_trust separate from dev->p->trust is to give buses the ability to establish a trust level before device_add(). Then, after device_add(), the dependency on 'struct device_private' mandates that the device-core own and coordinate changes to the device's trust level.
There are implications of the bus expressing less than full trust in a device and a module later expressing higher levels of trust. For PCI, the "adversary" / pci_untrusted() determination is made prior to initial iommu_probe_device(). The default IOMMU domain determination is latched from bus's specified trust value. Later changes to the trust level, for example by "modprobe $module trust=auto" skip that IOMMU probing. The result is that moving a device to DEVICE_TRUST_AUTO may lift translation blocking (PCI_ACS_TB), but not change the default IOMMU domain from IOMMU_DOMAIN_DMA.
The default domain can be validated / changed via iommu_group sysfs. The longer term goal is to dynamically adjust iommu domain at the next $bus_dma_configure() event, but that needs more surgery and consideration for multi-device iommu groups.
--- drivers/base/Kconfig | 13 +++++++++++++ drivers/pci/Kconfig | 1 + include/linux/device.h | 3 +++ include/linux/device/trust.h | 20 +++++++++++++++----- include/linux/pci.h | 13 +++++-------- drivers/base/trust.c | 20 +++++++++++++++++++- drivers/iommu/amd/iommu.c | 2 +- drivers/iommu/dma-iommu.c | 13 ++++--------- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/iommu.c | 2 +- drivers/pci/ats.c | 2 +- drivers/pci/pci.c | 2 +- drivers/pci/probe.c | 10 +++++++--- drivers/pci/quirks.c | 4 ++-- 14 files changed, 74 insertions(+), 33 deletions(-)
drivers/base/Kconfig +13 -0
@@ -291,6 +291,14 @@ config DEVICE_TRUST_NONE possible, the device is blocked by an IOMMU from accessing assets. +config DEVICE_TRUST_ADVERSARY + bool "Adversary" + help + Device is allowed to bind. Bus, IOMMU, and driver layers may + react to this trust level by disabling access validation + bypass mechanisms like PCI ATS. When device is unbound from a + driver the device is blocked by an IOMMU where possible. + config DEVICE_TRUST_AUTO bool "Auto" help
@@ -317,6 +325,11 @@ config BUILTIN_DEVICE_TRUST_AUTO a driver and deploy all available mechanisms to allow performant direct memory access This trust level does not grant TCB privileges. +config BUILTIN_DEVICE_TRUST_ADVERSARY + bool "Adversary" + help + Deploy mitigations in the IOMMU layer and driver to limit access. + endchoice endmenu
drivers/pci/Kconfig +1 -0
@@ -22,6 +22,7 @@ menuconfig PCI bool "PCI support" depends on HAVE_PCI depends on MMU + select DEVICE_TRUST help This option enables support for the PCI local bus, including support for PCI-X and the foundations for PCI Express support.
include/linux/device.h +3 -0
@@ -688,6 +688,8 @@ enum struct_device_flags {* @removable: Whether the device can be removed from the system. This * should be set by the subsystem / bus driver that discovered * the device. +* @bus_trust: Device's initial / prior to device_add() trust level. +* * @flags: DEV_FLAG_XXX flags. Use atomic bitfield operations to modify. * * At the lowest level, every device in a Linux system is represented by an
@@ -791,6 +793,7 @@ struct device {struct device_physical_location *physical_location;enum device_removable removable; +enum device_trust bus_trust;DECLARE_BITMAP (flags,DEV_FLAG_COUNT ); };
include/linux/device/trust.h +15 -5
@@ -11,30 +11,40 @@* * @DEVICE_TRUST_UNSET: Unregistered device object with no current bus * @DEVICE_TRUST_NONE: Blocked when idle, cannot bind +* @DEVICE_TRUST_ADVERSARY: Blocked when idle, constrained when active. * @DEVICE_TRUST_AUTO: All typical privileges granted +* +* Devices flagged as adversarial are the ones that can potentially +* execute DMA attacks and similar. They are typically connected through +* external ports such as Thunderbolt but not limited to that. When an +* IOMMU is enabled they should be getting full mappings to make sure +* they cannot access arbitrary memory. */ enum device_trust {DEVICE_TRUST_UNSET ,DEVICE_TRUST_NONE , +DEVICE_TRUST_ADVERSARY ,DEVICE_TRUST_AUTO , }; -#define DEVICE_DEFAULT_TRUST \ - (IS_ENABLED(CONFIG_DEVICE_TRUST_NONE) ? DEVICE_TRUST_NONE : \ - DEVICE_TRUST_AUTO) +#define DEVICE_DEFAULT_TRUST \ + (IS_ENABLED(CONFIG_DEVICE_TRUST_NONE) ? DEVICE_TRUST_NONE : \ + IS_ENABLED(CONFIG_DEVICE_TRUST_ADVERSARY) ? DEVICE_TRUST_ADVERSARY : \ + DEVICE_TRUST_AUTO)struct device ;struct device_driver ;#ifdef CONFIG_DEVICE_TRUST +bool device_untrusted (struct device *dev);void module_driver_trust (struct module *mod,const char *val); -void module_driver_trust_init (struct module *mod,bool distrust); +void module_driver_trust_init (struct module *mod,bool require_trust);#else static inline void module_driver_trust (struct module *mod,const char *val) {pr_warn ("module: %s: trust= support disabled\n" , mod->name); } -static inline void module_driver_trust_init (struct module *mod,bool distrust) +static inline void module_driver_trust_init (struct module *mod,bool require_trust) { }#endif
include/linux/pci.h +5 -8
@@ -485,14 +485,6 @@ struct pci_dev { unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ unsigned int is_cxl:1; /* Compute Express Link (CXL) */ -/* -* Devices marked being untrusted are the ones that can potentially -* execute DMA attacks and similar. They are typically connected -* through external ports such as Thunderbolt but not limited to -* that. When an IOMMU is enabled they should be getting full -* mappings to make sure they cannot access arbitrary memory. -*/ - unsigned int untrusted:1;/* * Info from the platform, e.g., ACPI or device tree, may mark a * device as "external-facing". An external-facing device is
@@ -812,6 +804,11 @@ static inline bool pcie_is_cxl(struct pci_dev *pci_dev)return pci_dev->is_cxl; } +static inline bool pci_untrusted (struct pci_dev *pdev) +{ +return device_untrusted (&pdev->dev); +} +#define for_each_pci_bridge (dev, bus) \ list_for_each_entry(dev, &bus->devices, bus_list) \ if (!pci_is_bridge(dev)) {} else
drivers/base/trust.c +19 -1
@@ -6,15 +6,28 @@#include <linux/module.h> #include "base.h" +/* If the bus did not initialize trust, set a default */ void device_initialize_trust (struct device *dev) { + dev->p->trust = dev->bus_trust;if (dev->p->trust ==DEVICE_TRUST_UNSET ) dev->p->trust =DEVICE_DEFAULT_TRUST ; } +/* +* ->bus_trust is evaluated / manipulated prior to device_add() and +* synced with dev->p->trust post device_add() under device_lock(). +*/ +bool device_untrusted (struct device *dev) +{ +return dev->bus_trust && dev->bus_trust <=DEVICE_TRUST_ADVERSARY ; +} +/* Driver trust policy requires modules, builtin drivers always attach */ static enum device_trust builtin_driver_trust (void ) { +if (IS_ENABLED (CONFIG_BUILTIN_DEVICE_TRUST_ADVERSARY )) +return DEVICE_TRUST_ADVERSARY ;return DEVICE_TRUST_AUTO ; }
@@ -30,18 +43,23 @@ static enum device_trust driver_trust(struct module *mod)* policy on trusting devices it attaches, update the device's trust * level from that policy. Trust privileges beyond driver bind are * realized in a bus's ->dma_configure(). +* +* Reflect the operational trust level back to the public indicator. */ bool device_trust_bind (const struct device_driver *drv,struct device *dev) {enum device_trust drv_trust =driver_trust (drv->owner); -if (drv_trust !=DEVICE_TRUST_UNSET ) +if (drv_trust !=DEVICE_TRUST_UNSET ) { dev->p->trust = drv_trust; + dev->bus_trust = drv_trust; + }return dev->p->trust >DEVICE_TRUST_NONE ; }static const char *const device_trust_names[] = { [DEVICE_TRUST_NONE ] ="none" , + [DEVICE_TRUST_ADVERSARY ] ="adversary" , [DEVICE_TRUST_AUTO ] ="auto" , };
drivers/iommu/amd/iommu.c +1 -1
@@ -3121,7 +3121,7 @@ static int amd_iommu_def_domain_type(struct device *dev)return 0 ;/* Always use DMA domain for untrusted device */ -if (dev_is_pci (dev) &&to_pci_dev (dev)->untrusted) +if (device_untrusted (dev))return IOMMU_DOMAIN_DMA ;/*
drivers/iommu/dma-iommu.c +4 -9
@@ -588,16 +588,11 @@ static int iova_reserve_iommu_regions(struct device *dev,return ret; } -static bool dev_is_untrusted (struct device *dev) -{ -return dev_is_pci (dev) &&to_pci_dev (dev)->untrusted; -} -static bool dev_use_swiotlb (struct device *dev,size_t size,enum dma_data_direction dir) {return IS_ENABLED (CONFIG_SWIOTLB ) && - (dev_is_untrusted (dev) || + (device_untrusted (dev) ||dma_kmalloc_needs_bounce (dev, size, dir)); }
@@ -610,7 +605,7 @@ static bool dev_use_sg_swiotlb(struct device *dev, struct scatterlist *sg,if (!IS_ENABLED (CONFIG_SWIOTLB ))return false; -if (dev_is_untrusted (dev)) +if (device_untrusted (dev))return true;/*
@@ -1188,7 +1183,7 @@ static phys_addr_t iommu_dma_map_swiotlb(struct device *dev, phys_addr_t phys,* swiotlb_tbl_map_single() has initialized the bounce buffer proper to * the contents of the original memory buffer. */ -if (phys != (phys_addr_t )DMA_MAPPING_ERROR &&dev_is_untrusted (dev)) { +if (phys != (phys_addr_t )DMA_MAPPING_ERROR &&device_untrusted (dev)) {size_t start, virt = (size_t )phys_to_virt (phys);/* Pre-padding */
@@ -1761,7 +1756,7 @@ size_t iommu_dma_opt_mapping_size(void)size_t iommu_dma_max_mapping_size (struct device *dev) { -if (dev_is_untrusted (dev)) +if (device_untrusted (dev))return swiotlb_max_mapping_size (dev);return SIZE_MAX ;
drivers/iommu/intel/iommu.c +1 -1
@@ -3501,7 +3501,7 @@ static bool intel_iommu_is_attach_deferred(struct device *dev)*/ static bool risky_device (struct pci_dev *pdev) { -if (pdev->untrusted) { +if (pci_untrusted (pdev)) {pci_info (pdev,"Skipping IOMMU quirk for dev [%04X:%04X] on untrusted PCI link\n" , pdev->vendor, pdev->device);
drivers/iommu/iommu.c +1 -1
@@ -1901,7 +1901,7 @@ static int iommu_get_default_domain_type(struct iommu_group *group, driver_type =iommu_get_def_domain_type (group, gdev->dev, driver_type); -if (dev_is_pci (gdev->dev) &&to_pci_dev (gdev->dev)->untrusted) { +if (device_untrusted (gdev->dev)) {/* * No ARM32 using systems will set untrusted, it cannot * work.
drivers/pci/ats.c +1 -1
@@ -43,7 +43,7 @@ bool pci_ats_supported(struct pci_dev *dev)if (!dev->ats_cap)return false; -return (dev->untrusted ==0 ); +return !pci_untrusted (dev); }EXPORT_SYMBOL_GPL (pci_ats_supported);
drivers/pci/pci.c +1 -1
@@ -994,7 +994,7 @@ static void pci_std_enable_acs(struct pci_dev *dev, struct pci_acs *caps) caps->ctrl |= (dev->acs_capabilities &PCI_ACS_UF );/* Enable Translation Blocking for external devices and noats */ -if (pci_ats_disabled () || dev->external_facing || dev->untrusted) +if (pci_ats_disabled () || dev->external_facing ||pci_untrusted (dev)) caps->ctrl |= (dev->acs_capabilities &PCI_ACS_TB ); }
drivers/pci/probe.c +7 -3
@@ -6,6 +6,7 @@#include <linux/array_size.h> #include <linux/kernel.h> #include <linux/delay.h> +#include <linux/device/trust.h> #include <linux/init.h> #include <linux/pci.h> #include <linux/msi.h>
@@ -1737,20 +1738,23 @@ static void set_pcie_untrusted(struct pci_dev *dev) {struct pci_dev *parent =pci_upstream_bridge (dev); + dev->dev.bus_trust =DEVICE_DEFAULT_TRUST ;if (!parent)return ;/* * If the upstream bridge is untrusted we treat this device as * untrusted as well. */ -if (parent->untrusted) { - dev->untrusted = true; +if (pci_untrusted (parent)) { + dev->dev.bus_trust = +min (dev->dev.bus_trust, parent->dev.bus_trust);return ; }if (arch_pci_dev_is_removable (dev)) {pci_dbg (dev,"marking as untrusted\n" ); - dev->untrusted = true; + dev->dev.bus_trust = +min (dev->dev.bus_trust,DEVICE_TRUST_ADVERSARY ); } }
drivers/pci/quirks.c +2 -2
@@ -5370,7 +5370,7 @@ static void pci_quirk_enable_intel_rp_mpc_acs(struct pci_dev *dev)* PCI_ACS_SV | PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_UF * * TODO: This quirk also needs to do equivalent of PCI_ACS_TB, -* if dev->external_facing || dev->untrusted +* if dev->external_facing || pci_untrusted(dev) */ static int pci_quirk_enable_intel_pch_acs (struct pci_dev *dev) {
@@ -5411,7 +5411,7 @@ static int pci_quirk_enable_intel_spt_pch_acs(struct pci_dev *dev) ctrl |= (cap &PCI_ACS_CR ); ctrl |= (cap &PCI_ACS_UF ); -if (pci_ats_disabled () || dev->external_facing || dev->untrusted) +if (pci_ats_disabled () || dev->external_facing ||pci_untrusted (dev)) ctrl |= (cap &PCI_ACS_TB );pci_write_config_dword (dev, pos +INTEL_SPT_ACS_CTRL , ctrl);
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 09/15] PCI, device core: Move "untrusted" concept to DEVICE_TRUST_ADVERSARY
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider: - [High] Global device trust default is bypassed for non-PCI devices during IOMMU initialization because…
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Global device trust default is bypassed for non-PCI devices during IOMMU initialization because device_initialize_trust() fails to initialize bus_trust.
- [High] Driver binding overwrites dev->bus_trust, disabling dynamic IOMMU DMA protections and breaking PCI trust inheritance for downstream devices.
- [Low] Missing #else stub for device_untrusted() causes compilation errors when CONFIG_DEVICE_TRUST is disabled.
- [Low] enum device_trust is used as a value type in struct device, but it is neither defined nor forward-declared in <linux/device.h>. --
commit e078ccf16f66fc647e7473fedd70781ac6e8fd5d
Author: Dan Williams <[email protected]>
PCI, device core: Move "untrusted" concept to DEVICE_TRUST_ADVERSARY
This commit elevates the PCI "untrusted" concept to a core device trust level called DEVICE_TRUST_ADVERSARY. It adds a new bus_trust field to struct device and uses it to manage trust states before device_add(), syncing it to the private device trust structure afterwards. It updates IOMMU and PCI subsystems to use the generic device_untrusted() interface.
Quoted 5 lines
drivers/base/trust.c +0
-0
drivers/base/trust.c line 15
Quoted 10 lines
drivers/base/trust.c +2
-0@@ -6,15 +6,28 @@#include <linux/module.h> #include "base.h" +/* If the bus did not initialize trust, set a default */ void device_initialize_trust (struct device *dev) { + dev->p->trust = dev->bus_trust;if (dev->p->trust ==DEVICE_TRUST_UNSET ) dev->p->trust =DEVICE_DEFAULT_TRUST ; }
[Severity: High]
Does this bypass the global trust default for non-PCI devices?
The fallback logic updates dev->p->trust to DEVICE_DEFAULT_TRUST but leaves dev->bus_trust as DEVICE_TRUST_UNSET (0). Since device_untrusted() strictly evaluates dev->bus_trust, does this leave the trust uninitialized?
Furthermore, IOMMU probes devices via BUS_NOTIFY_ADD_DEVICE in device_add() before device_initialize_trust() is called. Is bus_trust always 0 during IOMMU probing for these non-PCI devices, disabling protections like SWIOTLB bounce buffering?
[ ... ]
Quoted 8 lines
drivers/base/trust.c +2
-0@@ -30,18 +43,23 @@ static enum device_trust driver_trust(struct module *mod)* policy on trusting devices it attaches, update the device's trust * level from that policy. Trust privileges beyond driver bind are * realized in a bus's ->dma_configure(). +* +* Reflect the operational trust level back to the public indicator. */ bool device_trust_bind (const struct device_driver *drv,struct device *dev)
drivers/base/trust.c static enum device_trust driver_trust(struct module *mod) · line 58
Quoted 10 lines
drivers/base/trust.c +3
-1{enum device_trust drv_trust =driver_trust (drv->owner); -if (drv_trust !=DEVICE_TRUST_UNSET ) +if (drv_trust !=DEVICE_TRUST_UNSET ) { dev->p->trust = drv_trust; + dev->bus_trust = drv_trust; + }return dev->p->trust >DEVICE_TRUST_NONE ; }
[Severity: High]
Does overwriting dev->bus_trust with the driver's software trust level disable dynamic IOMMU DMA protections?
When device_untrusted() evaluates dev->bus_trust after binding, it will transition to false. Will this silently disable protections like SWIOTLB bounce buffering in dev_use_swiotlb() for active untrusted devices?
Also, if an untrusted parent bridge (e.g., a Thunderbolt port) binds to the pcieport driver, its bus_trust becomes AUTO. Will hot-plugged downstream devices fail to inherit the untrusted state because pci_untrusted(parent) evaluates to false in set_pcie_untrusted()?
[ ... ]
include/linux/device.h struct device { · line 799
Quoted 12 lines
include/linux/device.h +1
-0@@ -791,6 +793,7 @@ struct device {struct device_physical_location *physical_location;enum device_removable removable; +enum device_trust bus_trust;DECLARE_BITMAP (flags,DEV_FLAG_COUNT ); };
[Severity: Low]
Is enum device_trust an incomplete type here?
Since <linux/device/trust.h> is not included in <linux/device.h>, does using it as a value-type field cause a compilation error in files that don't include <linux/device/trust.h> manually?
[ ... ]
Quoted 4 lines
include/linux/device/trust.h +0
-0
[ ... ]
Quoted 5 lines
#ifdef
CONFIG_DEVICE_TRUSTinclude/linux/device/trust.h +2
-1+bool device_untrusted (struct device *dev);void module_driver_trust (struct module *mod,const char *val); -void module_driver_trust_init (struct module *mod,bool distrust); +void module_driver_trust_init (struct module *mod,bool require_trust);
include/linux/device/trust.h line 50
Quoted 10 lines
include/linux/device/trust.h +1
-1#else static inline void module_driver_trust (struct module *mod,const char *val) {pr_warn ("module: %s: trust= support disabled\n" , mod->name); } -static inline void module_driver_trust_init (struct module *mod,bool distrust) +static inline void module_driver_trust_init (struct module *mod,bool require_trust) { }#endif
[Severity: Low]
Does this missing #else stub for device_untrusted() cause build failures?
Since CONFIG_DEVICE_TRUST is selected by PCI, configuring a kernel without PCI but with IOMMU DMA support leaves device_untrusted() implicitly declared in core IOMMU code like dev_use_swiotlb(). Should there be a fallback inline function here?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Re: [PATCH 09/15] PCI, device core: Move "untrusted" concept to DEVICE_TRUST_ADVERSARY
In reply to Dan WilliamsQuoted 20 lines · Dan Williams
drivers/base/Kconfig +8
-0@@ -291,6 +291,14 @@ config DEVICE_TRUST_NONE possible, the device is blocked by an IOMMU from accessing assets. +config DEVICE_TRUST_ADVERSARY + bool "Adversary" + help + Device is allowed to bind. Bus, IOMMU, and driver layers may + react to this trust level by disabling access validation + bypass mechanisms like PCI ATS. When device is unbound from a + driver the device is blocked by an IOMMU where possible. + config DEVICE_TRUST_AUTO bool "Auto" help
drivers/base/Kconfig config BUILTIN_DEVICE_TRUST_AUTO · line 331
Quoted 7 lines
drivers/base/Kconfig +4
-0@@ -317,6 +325,11 @@ config BUILTIN_DEVICE_TRUST_AUTO a driver and deploy all available mechanisms to allow performant direct memory access This trust level does not grant TCB privileges. +config BUILTIN_DEVICE_TRUST_ADVERSARY + bool "Adversary" + help + Deploy mitigations in the IOMMU layer and driver to limit access.
I'm not sure it makes sense to have these options as compile time defaults. Arguably the non-built in path should only be auto or none and everything else should be handled by the initrd at runtime. If necessary a sysfs for the initrd to set a kernel default might make sense, IDK.
Ideally built in devices should not be any kind that is hooked up to the iommu.
Quoted 11 lines
drivers/iommu/dma-iommu.c +0
-3@@ -588,16 +588,11 @@ static int iova_reserve_iommu_regions(struct device *dev,return ret; } -static bool dev_is_untrusted (struct device *dev) -{ -return dev_is_pci (dev) &&to_pci_dev (dev)->untrusted;
drivers/iommu/dma-iommu.c static int iova_reserve_iommu_regions(struct device *dev, · line 597
Quoted 10 lines
drivers/iommu/dma-iommu.c +1
-3-} -static bool dev_use_swiotlb (struct device *dev,size_t size,enum dma_data_direction dir) {return IS_ENABLED (CONFIG_SWIOTLB ) && - (dev_is_untrusted (dev) || + (device_untrusted (dev) ||dma_kmalloc_needs_bounce (dev, size, dir)); }
I think it would be nice to use the new 'adversarial' word here and in other places instead of untrusted.
Jason
Re: [PATCH 09/15] PCI, device core: Move "untrusted" concept to DEVICE_TRUST_ADVERSARY
In reply to Dan WilliamsQuoted 43 lines · Dan Williams
An "adversary" device is one that the kernel is allowed to operate, but with extra safety / paranoia added. It aims to keep the device bounded in both address space and time (no batched invalidation) for DMA accesses. The concept of adversarial devices already exists in the PCI core. The categorization is applied to externally attached PCI devices by default (like Thunderbolt attached devices). Unlike
DEVICE_TRUST_NONEthat says "do not allow driver bind",DEVICE_TRUST_ADVERSARYsays "allow driver bind as long as the driver and/or IOMMU are taking precautions." Uplevel this trust mechanism from a PCI device "untrusted" boolean flag to the core device trust level.The rationale for
dev->bus_trustseparate fromdev->p->trustis to give buses the ability to establish a trust level beforedevice_add(). Then, afterdevice_add(), the dependency on 'structdevice_private' mandates that the device-core own and coordinate changes to the device's trust level.There are implications of the bus expressing less than full trust in a device and a module later expressing higher levels of trust. For PCI, the "adversary" /
pci_untrusted()determination is made prior to initialiommu_probe_device(). The default IOMMU domain determination is latched from bus's specified trust value. Later changes to the trust level, for example by "modprobe $module trust=auto" skip that IOMMU probing. The result is that moving a device toDEVICE_TRUST_AUTOmay lift translation blocking (PCI_ACS_TB), but not change the default IOMMU domain fromIOMMU_DOMAIN_DMA.The default domain can be validated / changed via
iommu_groupsysfs. The longer term goal is to dynamically adjust iommu domain at the next $bus_dma_configure() event, but that needs more surgery and consideration for multi-device iommu groups.Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Danilo Krummrich <[email protected]>Cc Will Deacon <[email protected]>Cc Robin Murphy <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Alexey Kardashevskiy <[email protected]>Cc Xu Yilun <[email protected]>Cc "Aneesh Kumar K.V" <[email protected]>Cc Dexuan Cui <[email protected]>Signed-off-by Dan Williams <[email protected]>
Quoted 11 lines
--- drivers/base/Kconfig | 13 +++++++++++++ drivers/pci/Kconfig | 1 + include/linux/device.h | 3 +++ include/linux/device/trust.h | 20 +++++++++++++++----- include/linux/pci.h | 13 +++++-------- drivers/base/trust.c | 20 +++++++++++++++++++- drivers/iommu/amd/iommu.c | 2 +- drivers/iommu/dma-iommu.c | 13 ++++--------- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/iommu.c | 2 +-
For the straightforward drivers/iommu changes:
I don't have time to have a strong opinion about the internal implementation details, but I agree a driver-core-level abstraction of hierarchical bus/device trust seems like the right step to take in general.
Cheers,
Robin.
Quoted 384 lines
drivers/pci/ats.c | 2 +- drivers/pci/pci.c | 2 +- drivers/pci/probe.c | 10 +++++++--- drivers/pci/quirks.c | 4 ++-- 14 files changed, 74 insertions(+), 33 deletions(-)drivers/base/Kconfig +13
-0@@ -291,6 +291,14 @@ config DEVICE_TRUST_NONE possible, the device is blocked by an IOMMU from accessing assets. +config DEVICE_TRUST_ADVERSARY + bool "Adversary" + help + Device is allowed to bind. Bus, IOMMU, and driver layers may + react to this trust level by disabling access validation + bypass mechanisms like PCI ATS. When device is unbound from a + driver the device is blocked by an IOMMU where possible. + config DEVICE_TRUST_AUTO bool "Auto" help@@ -317,6 +325,11 @@ config BUILTIN_DEVICE_TRUST_AUTO a driver and deploy all available mechanisms to allow performant direct memory access This trust level does not grant TCB privileges. +config BUILTIN_DEVICE_TRUST_ADVERSARY + bool "Adversary" + help + Deploy mitigations in the IOMMU layer and driver to limit access. + endchoice endmenudrivers/pci/Kconfig +1
-0@@ -22,6 +22,7 @@ menuconfig PCI bool "PCI support" depends on HAVE_PCI depends on MMU + select DEVICE_TRUST help This option enables support for the PCI local bus, including support for PCI-X and the foundations for PCI Express support.include/linux/device.h +3
-0@@ -688,6 +688,8 @@ enum struct_device_flags {* @removable: Whether the device can be removed from the system. This * should be set by the subsystem / bus driver that discovered * the device. +* @bus_trust: Device's initial / prior to device_add() trust level. +* * @flags: DEV_FLAG_XXX flags. Use atomic bitfield operations to modify. * * At the lowest level, every device in a Linux system is represented by an @@ -791,6 +793,7 @@ struct device {struct device_physical_location *physical_location;enum device_removable removable; +enum device_trust bus_trust;DECLARE_BITMAP (flags,DEV_FLAG_COUNT ); };include/linux/device/trust.h +15
-5@@ -11,30 +11,40 @@* * @DEVICE_TRUST_UNSET: Unregistered device object with no current bus * @DEVICE_TRUST_NONE: Blocked when idle, cannot bind +* @DEVICE_TRUST_ADVERSARY: Blocked when idle, constrained when active. * @DEVICE_TRUST_AUTO: All typical privileges granted +* +* Devices flagged as adversarial are the ones that can potentially +* execute DMA attacks and similar. They are typically connected through +* external ports such as Thunderbolt but not limited to that. When an +* IOMMU is enabled they should be getting full mappings to make sure +* they cannot access arbitrary memory. */ enum device_trust {DEVICE_TRUST_UNSET ,DEVICE_TRUST_NONE , +DEVICE_TRUST_ADVERSARY ,DEVICE_TRUST_AUTO , }; -#define DEVICE_DEFAULT_TRUST \ - (IS_ENABLED(CONFIG_DEVICE_TRUST_NONE) ? DEVICE_TRUST_NONE : \ - DEVICE_TRUST_AUTO) +#define DEVICE_DEFAULT_TRUST \ + (IS_ENABLED(CONFIG_DEVICE_TRUST_NONE) ? DEVICE_TRUST_NONE : \ + IS_ENABLED(CONFIG_DEVICE_TRUST_ADVERSARY) ? DEVICE_TRUST_ADVERSARY : \ + DEVICE_TRUST_AUTO)struct device ;struct device_driver ;#ifdef CONFIG_DEVICE_TRUST +bool device_untrusted (struct device *dev);void module_driver_trust (struct module *mod,const char *val); -void module_driver_trust_init (struct module *mod,bool distrust); +void module_driver_trust_init (struct module *mod,bool require_trust);#else static inline void module_driver_trust (struct module *mod,const char *val) {pr_warn ("module: %s: trust= support disabled\n" , mod->name); } -static inline void module_driver_trust_init (struct module *mod,bool distrust) +static inline void module_driver_trust_init (struct module *mod,bool require_trust) { }#endif include/linux/pci.h +5
-8@@ -485,14 +485,6 @@ struct pci_dev { unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ unsigned int is_cxl:1; /* Compute Express Link (CXL) */ -/* -* Devices marked being untrusted are the ones that can potentially -* execute DMA attacks and similar. They are typically connected -* through external ports such as Thunderbolt but not limited to -* that. When an IOMMU is enabled they should be getting full -* mappings to make sure they cannot access arbitrary memory. -*/ - unsigned int untrusted:1;/* * Info from the platform, e.g., ACPI or device tree, may mark a * device as "external-facing". An external-facing device is @@ -812,6 +804,11 @@ static inline bool pcie_is_cxl(struct pci_dev *pci_dev)return pci_dev->is_cxl; } +static inline bool pci_untrusted (struct pci_dev *pdev) +{ +return device_untrusted (&pdev->dev); +} +#define for_each_pci_bridge (dev, bus) \ list_for_each_entry(dev, &bus->devices, bus_list) \ if (!pci_is_bridge(dev)) {} elsedrivers/base/trust.c +19
-1@@ -6,15 +6,28 @@#include <linux/module.h> #include "base.h" +/* If the bus did not initialize trust, set a default */ void device_initialize_trust (struct device *dev) { + dev->p->trust = dev->bus_trust;if (dev->p->trust ==DEVICE_TRUST_UNSET ) dev->p->trust =DEVICE_DEFAULT_TRUST ; } +/* +* ->bus_trust is evaluated / manipulated prior to device_add() and +* synced with dev->p->trust post device_add() under device_lock(). +*/ +bool device_untrusted (struct device *dev) +{ +return dev->bus_trust && dev->bus_trust <=DEVICE_TRUST_ADVERSARY ; +} +/* Driver trust policy requires modules, builtin drivers always attach */ static enum device_trust builtin_driver_trust (void ) { +if (IS_ENABLED (CONFIG_BUILTIN_DEVICE_TRUST_ADVERSARY )) +return DEVICE_TRUST_ADVERSARY ;return DEVICE_TRUST_AUTO ; }@@ -30,18 +43,23 @@ static enum device_trust driver_trust(struct module *mod)* policy on trusting devices it attaches, update the device's trust * level from that policy. Trust privileges beyond driver bind are * realized in a bus's ->dma_configure(). +* +* Reflect the operational trust level back to the public indicator. */ bool device_trust_bind (const struct device_driver *drv,struct device *dev) {enum device_trust drv_trust =driver_trust (drv->owner); -if (drv_trust !=DEVICE_TRUST_UNSET ) +if (drv_trust !=DEVICE_TRUST_UNSET ) { dev->p->trust = drv_trust; + dev->bus_trust = drv_trust; + }return dev->p->trust >DEVICE_TRUST_NONE ; }static const char *const device_trust_names[] = { [DEVICE_TRUST_NONE ] ="none" , + [DEVICE_TRUST_ADVERSARY ] ="adversary" , [DEVICE_TRUST_AUTO ] ="auto" , };drivers/iommu/amd/iommu.c +1
-1@@ -3121,7 +3121,7 @@ static int amd_iommu_def_domain_type(struct device *dev)return 0 ;/* Always use DMA domain for untrusted device */ -if (dev_is_pci (dev) &&to_pci_dev (dev)->untrusted) +if (device_untrusted (dev))return IOMMU_DOMAIN_DMA ;/* drivers/iommu/dma-iommu.c +4
-9@@ -588,16 +588,11 @@ static int iova_reserve_iommu_regions(struct device *dev,return ret; } -static bool dev_is_untrusted (struct device *dev) -{ -return dev_is_pci (dev) &&to_pci_dev (dev)->untrusted; -} -static bool dev_use_swiotlb (struct device *dev,size_t size,enum dma_data_direction dir) {return IS_ENABLED (CONFIG_SWIOTLB ) && - (dev_is_untrusted (dev) || + (device_untrusted (dev) ||dma_kmalloc_needs_bounce (dev, size, dir)); }@@ -610,7 +605,7 @@ static bool dev_use_sg_swiotlb(struct device *dev, struct scatterlist *sg,if (!IS_ENABLED (CONFIG_SWIOTLB ))return false; -if (dev_is_untrusted (dev)) +if (device_untrusted (dev))return true;/* @@ -1188,7 +1183,7 @@ static phys_addr_t iommu_dma_map_swiotlb(struct device *dev, phys_addr_t phys,* swiotlb_tbl_map_single() has initialized the bounce buffer proper to * the contents of the original memory buffer. */ -if (phys != (phys_addr_t )DMA_MAPPING_ERROR &&dev_is_untrusted (dev)) { +if (phys != (phys_addr_t )DMA_MAPPING_ERROR &&device_untrusted (dev)) {size_t start, virt = (size_t )phys_to_virt (phys);/* Pre-padding */ @@ -1761,7 +1756,7 @@ size_t iommu_dma_opt_mapping_size(void)size_t iommu_dma_max_mapping_size (struct device *dev) { -if (dev_is_untrusted (dev)) +if (device_untrusted (dev))return swiotlb_max_mapping_size (dev);return SIZE_MAX ;drivers/iommu/intel/iommu.c +1
-1@@ -3501,7 +3501,7 @@ static bool intel_iommu_is_attach_deferred(struct device *dev)*/ static bool risky_device (struct pci_dev *pdev) { -if (pdev->untrusted) { +if (pci_untrusted (pdev)) {pci_info (pdev,"Skipping IOMMU quirk for dev [%04X:%04X] on untrusted PCI link\n" , pdev->vendor, pdev->device);drivers/iommu/iommu.c +1
-1@@ -1901,7 +1901,7 @@ static int iommu_get_default_domain_type(struct iommu_group *group, driver_type =iommu_get_def_domain_type (group, gdev->dev, driver_type); -if (dev_is_pci (gdev->dev) &&to_pci_dev (gdev->dev)->untrusted) { +if (device_untrusted (gdev->dev)) {/* * No ARM32 using systems will set untrusted, it cannot * work. drivers/pci/ats.c +1
-1@@ -43,7 +43,7 @@ bool pci_ats_supported(struct pci_dev *dev)if (!dev->ats_cap)return false; -return (dev->untrusted ==0 ); +return !pci_untrusted (dev); }EXPORT_SYMBOL_GPL (pci_ats_supported);drivers/pci/pci.c +1
-1@@ -994,7 +994,7 @@ static void pci_std_enable_acs(struct pci_dev *dev, struct pci_acs *caps) caps->ctrl |= (dev->acs_capabilities &PCI_ACS_UF );/* Enable Translation Blocking for external devices and noats */ -if (pci_ats_disabled () || dev->external_facing || dev->untrusted) +if (pci_ats_disabled () || dev->external_facing ||pci_untrusted (dev)) caps->ctrl |= (dev->acs_capabilities &PCI_ACS_TB ); }drivers/pci/probe.c +7
-3@@ -6,6 +6,7 @@#include <linux/array_size.h> #include <linux/kernel.h> #include <linux/delay.h> +#include <linux/device/trust.h> #include <linux/init.h> #include <linux/pci.h> #include <linux/msi.h> @@ -1737,20 +1738,23 @@ static void set_pcie_untrusted(struct pci_dev *dev) {struct pci_dev *parent =pci_upstream_bridge (dev); + dev->dev.bus_trust =DEVICE_DEFAULT_TRUST ;if (!parent)return ;/* * If the upstream bridge is untrusted we treat this device as * untrusted as well. */ -if (parent->untrusted) { - dev->untrusted = true; +if (pci_untrusted (parent)) { + dev->dev.bus_trust = +min (dev->dev.bus_trust, parent->dev.bus_trust);return ; }if (arch_pci_dev_is_removable (dev)) {pci_dbg (dev,"marking as untrusted\n" ); - dev->untrusted = true; + dev->dev.bus_trust = +min (dev->dev.bus_trust,DEVICE_TRUST_ADVERSARY ); } }drivers/pci/quirks.c +2
-2@@ -5370,7 +5370,7 @@ static void pci_quirk_enable_intel_rp_mpc_acs(struct pci_dev *dev)* PCI_ACS_SV | PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_UF * * TODO: This quirk also needs to do equivalent of PCI_ACS_TB, -* if dev->external_facing || dev->untrusted +* if dev->external_facing || pci_untrusted(dev) */ static int pci_quirk_enable_intel_pch_acs (struct pci_dev *dev) {@@ -5411,7 +5411,7 @@ static int pci_quirk_enable_intel_spt_pch_acs(struct pci_dev *dev) ctrl |= (cap &PCI_ACS_CR ); ctrl |= (cap &PCI_ACS_UF ); -if (pci_ats_disabled () || dev->external_facing || dev->untrusted) +if (pci_ats_disabled () || dev->external_facing ||pci_untrusted (dev)) ctrl |= (cap &PCI_ACS_TB );pci_write_config_dword (dev, pos +INTEL_SPT_ACS_CTRL , ctrl);
PATCH 10/15 PCI/TSM: Add device interface security LOCKED support
In reply to Dan WilliamsPCIe Trusted Execution Environment Device Interface Security Protocol (TDISP) has two distinct sets of operations. The first, already enabled in driver/pci/tsm.c, enables the VMM to authenticate the physical function (PCIe Component Measurement and Authentication (CMA)), establish a secure message passing session (DMTF SPDM), and establish physical link security (PCIe Integrity and Data Encryption (IDE)). The second set of operations lets a VM manage the security state of assigned devices (TEE Device Interfaces (TDIs)). Implement the LOCK/UNLOCK operations in pci_tsm_ops, to be followed with an operation to transition the interface to RUN.
- lock(): Transition the device to the TDISP LOCKED state. In this state measurements can validated to ensure that the VM trusts the device's security manager (DSM).
- unlock(): Transition the device to the TDISP UNLOCKED state.
The device shall not be attached to a driver during these transitions.
--- Documentation/ABI/testing/sysfs-bus-pci | 34 +++- Documentation/ABI/testing/sysfs-class-tsm | 19 ++ Documentation/driver-api/pci/tsm.rst | 47 +++++ include/linux/pci-tsm.h | 21 ++- drivers/pci/tsm/core.c | 213 +++++++++++++++++++++- drivers/virt/coco/tsm-core.c | 43 ++++- 6 files changed, 366 insertions(+), 11 deletions(-)
Documentation/ABI/testing/sysfs-bus-pci +32 -2
@@ -647,13 +647,16 @@ Description: Encryption) establishment. Reads from this attribute return the name of the connected TSM or the empty string if not connected. A TSM device signals its readiness to accept PCI - connection via a KOBJ_CHANGE event. + connection via a KOBJ_CHANGE event. This is a "link" TSM + attribute, see Documentation/ABI/testing/sysfs-class-tsm. What: /sys/bus/pci/devices/.../tsm/disconnect Contact: [email protected] Description: (WO) Write the name of the TSM device that was specified - to 'connect' to teardown the connection. + to 'connect' to teardown the connection. This is a + "link" TSM attribute, see + Documentation/ABI/testing/sysfs-class-tsm. What: /sys/bus/pci/devices/.../tsm/dsm Contact: [email protected]
@@ -702,3 +705,30 @@ Description: When present and the tsm/ attribute directory is present, the authenticated attribute is an alias for the device 'connect' state. See the 'tsm/connect' attribute for more details. + This is a "link" TSM attribute, see + Documentation/ABI/testing/sysfs-class-tsm. + +What: /sys/bus/pci/devices/.../tsm/lock +Contact: [email protected] +Description: + (RW) Write the name of a TSM (TEE Security Manager) device + instance to this file to request that the platform transition + the PCIe device configuration to the TDISP LOCKED state. This + puts the device in a state where security sensitive + configuration setting can not be changed without transitioning + the device the PCIe TDISP ERROR state. Reads from this attribute + return the name of the TSM device instance that has arranged for + the device to be locked, or the empty string if not locked. A + TSM class device signals its readiness for lock requests via a + KOBJ_CHANGE event. Writes fail with EBUSY if this device is + bound to a driver. This is a "devsec" TSM attribute, see + Documentation/ABI/testing/sysfs-class-tsm. See + Documentation/driver-api/pci/tsm.rst for more details. + +What: /sys/bus/pci/devices/.../tsm/unlock +Contact: [email protected] +Description: + (WO) Write the name of the TSM device that was specified to + 'lock' to teardown the connection. Writes fail with EBUSY if + this device is bound to a driver. This is a "devsec" TSM + attribute, see Documentation/ABI/testing/sysfs-class-tsm.
Documentation/ABI/testing/sysfs-class-tsm +19 -0
@@ -7,3 +7,22 @@ Description: signals when the PCI layer is able to support establishment of link encryption and other device-security features coordinated through a platform tsm. + +What: /sys/class/tsm/tsmN/pci_mode +Contact: [email protected] +Description: + (RO) A TSM with PCIe TDISP capability can be in one of two + modes. + + "link": typically for a hypervisor (VMM) to authenticate, + establish a secure session, and setup link + encryption. + + "devsec": typically for a confidential guest (TVM) to + transition assigned devices through the TDISP + state machine UNLOCKED->LOCKED->RUN. + + See the "tsm/" entries in + Documentation/ABI/testing/sysfs-bus-pci for the available PCI + device attributes when a TSM with the given "pci_mode" is + registered.
Documentation/driver-api/pci/tsm.rst +47 -0
53 lines not on this page · raw message
include/linux/pci-tsm.h +20 -1
@@ -72,7 +72,8 @@ struct pci_tsm_ops {* @unlock: destroy TSM context and return device to UNLOCKED state * * Context: @lock and @unlock run under pci_tsm_rwsem held for write to -* sync with TSM unregistration and each other +* sync with TSM unregistration and each other. All operations run under +* the device lock for mutual exclusion with driver attach and detach. */ struct_group_tagged (pci_tsm_devsec_ops, devsec_ops, struct pci_tsm *(*lock)(struct tsm_dev *tsm_dev,
@@ -115,6 +116,13 @@ struct pci_tdi {* sub-function (SR-IOV virtual function, or non-function0 * multifunction-device), or a downstream endpoint (PCIe upstream switch-port as * DSM). +* +* For devsec operations it serves to indicate that the function / TDI has been +* locked to a given TSM. +* +* The common expectation is that there is only ever one TSM, but this is not +* enforced. The implementation only enforces that a device can be "connected" +* to a TSM instance or "locked" to a different TSM. */ struct pci_tsm {struct pci_dev *pdev;
@@ -136,6 +144,14 @@ struct pci_tsm_pf0 {struct pci_doe_mb *doe_mb; }; +/** +* struct pci_tsm_devsec - context for tracking private/accepted PCI resources +* @base_tsm: generic core "tsm" context +*/ +struct pci_tsm_devsec { +struct pci_tsm base_tsm; +}; +/* physical function0 and capable of 'connect' */ static inline bool is_pci_tsm_pf0 (struct pci_dev *pdev) {
@@ -216,6 +232,8 @@ int pci_tsm_link_constructor(struct pci_dev *pdev, struct pci_tsm *tsm, struct tsm_dev *tsm_dev);int pci_tsm_pf0_constructor (struct pci_dev *pdev,struct pci_tsm_pf0 *tsm,struct tsm_dev *tsm_dev); +int pci_tsm_devsec_constructor (struct pci_dev *pdev,struct pci_tsm_devsec *tsm, +struct tsm_dev *tsm_dev);void pci_tsm_pf0_destructor (struct pci_tsm_pf0 *tsm);int pci_tsm_doe_transfer (struct pci_dev *pdev,u8 type,const void *req,size_t req_sz,void *resp,size_t resp_sz);
@@ -230,6 +248,7 @@ static inline const struct pci_tsm_ops *to_pci_tsm_ops(struct pci_tsm *tsm) {return tsm->tsm_dev->pci_ops; } +struct pci_tsm_devsec *to_pci_tsm_devsec (struct pci_tsm *tsm);#else static inline int pci_tsm_register (struct tsm_dev *tsm_dev) {
drivers/pci/tsm/core.c +206 -7
284 lines not on this page · raw message
drivers/virt/coco/tsm-core.c +42 -1
58 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 10/15] PCI/TSM: Add device interface security LOCKED support
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
New issues:
- [High] Sysfs operations unlock_store and disconnect_store do not verify the active TSM context type (pdev->tsm), allowing userspace to trigger kernel panics by performing operations on the wrong context type.
- [High] pci_tsm_unlock() returns early without clearing the TSM context if a driver is attached, causing a dangling pointer and Use-After-Free during subsystem teardown.
Pre-existing issues:
- [High] A TOCTOU race condition in tsm_unregister() allows lock_store() to attach a PCI device to a TSM that is actively being destroyed. --
commit 82cae08595d79f2459048e59ff4784f6bfb600ba
Author: Dan Williams <[email protected]>
PCI/TSM: Add device interface security LOCKED support
This patch implements the LOCK/UNLOCK operations in pci_tsm_ops to allow a VM to manage the security state of assigned devices. It provides the lock and unlock sysfs attributes to transition the device between TDISP LOCKED and UNLOCKED states.
Quoted 4 lines
drivers/pci/tsm/core.c +0
-0
[ ... ]
Quoted 10 lines
drivers/pci/tsm/core.c +6
-0@@ -531,6 +552,125 @@ static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, }static DEVICE_ATTR_RO (dsm); +/** +* pci_tsm_unlock() - Transition TDI from LOCKED/RUN to UNLOCKED +* @pdev: TDI device to unlock +* +* Returns void, requires all callers to have satisfied dependencies like making +* sure the device is locked and detached from its driver.
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 570
Quoted 10 lines
drivers/pci/tsm/core.c +10
-0+*/ +static void pci_tsm_unlock (struct pci_dev *pdev) +{ +lockdep_assert_held_write (&pci_tsm_rwsem); +device_lock_assert (&pdev->dev); + +if (dev_WARN_ONCE (&pdev->dev, pdev->dev.driver, +"unlock attempted on driver attached device\n" )) +return ; +
[Severity: High]
If pci_tsm_unlock() aborts early here because a driver is attached, pdev->tsm is not cleared.
During subsystem teardown, __pci_tsm_destroy() calls pci_tsm_unlock(). If it aborts here, does pdev->tsm remain pointing to the context while the TSM provider unregisters and frees its memory, leaving a dangling pointer?
Quoted 6 lines
drivers/pci/tsm/core.c +6
-0+device_cc_reject (&pdev->dev); +to_pci_tsm_ops (pdev->tsm)->unlock (pdev->tsm); + pdev->tsm =NULL ; +} + +static int pci_tsm_lock (struct pci_dev *pdev,struct tsm_dev *tsm_dev)
[ ... ]
Quoted 12 lines
drivers/pci/tsm/core.c +12
-0+static ssize_t lock_store (struct device *dev,struct device_attribute *attr, +const char *buf,size_t len) +{ +struct pci_dev *pdev =to_pci_dev (dev); +int rc, id; + + rc =sscanf (buf,"tsm%d\n" , &id); +if (rc !=1 ) +return -EINVAL ; + +ACQUIRE (rwsem_write_kill, lock)(&pci_tsm_rwsem); +if ((rc =ACQUIRE_ERR (rwsem_write_kill, &lock)))
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 618
Quoted 10 lines
drivers/pci/tsm/core.c +10
-0+return rc; + +if (pdev->tsm) +return -EBUSY ; + + struct tsm_dev *tsm_dev __free(put_tsm_dev) = find_tsm_dev(id); +if (!is_devsec_tsm (tsm_dev)) +return -ENXIO ; + + rc =pci_tsm_lock (pdev, tsm_dev);
[Severity: High]
This is a pre-existing issue, but can lock_store() attach a PCI device to a TSM that is actively being destroyed?
In tsm_unregister(), pci_tsm_unregister() sweeps the devices and drops pci_tsm_rwsem. Before the device is actually unregistered and freed by device_unregister(), could lock_store() acquire the rwsem, look up the dying tsm_dev via find_tsm_dev(), and attach it to a PCI device?
This seems like it could lead to a use-after-free when the TSM memory is freed shortly after.
Quoted 5 lines
drivers/pci/tsm/core.c +5
-0+if (rc) +return rc; +return len; +} +
[ ... ]
Quoted 4 lines
drivers/pci/tsm/core.c +4
-0+static ssize_t unlock_store (struct device *dev,struct device_attribute *attr, +const char *buf,size_t len) +{ +struct pci_dev *pdev =to_pci_dev (dev);
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 656
Quoted 10 lines
drivers/pci/tsm/core.c +10
-0+struct tsm_dev *tsm_dev; +int rc; + +ACQUIRE (rwsem_write_kill, lock)(&pci_tsm_rwsem); +if ((rc =ACQUIRE_ERR (rwsem_write_kill, &lock))) +return rc; + +if (!pdev->tsm) +return -EINVAL ; +
[Severity: High]
Does unlock_store() need to verify that pdev->tsm is a devsec context before proceeding?
A PCI device can hold only one TSM context at a time, but if it has both capabilities, both sets of sysfs attributes are visible. If a user writes to unlock when the device has a link context, pci_tsm_unlock() executes and dereferences ops->unlock(). Since link TSMs do not implement unlock, this pointer is NULL, potentially causing a kernel panic.
Similarly, does the existing disconnect_store() verify if the context is a link context? If disconnect is written to when the device has a devsec context, __pci_tsm_disconnect() attempts to cast via to_pci_tsm_pf0(), receiving NULL, and then dereferences it via guard(mutex)(&tsm_pf0->lock).
Quoted 16 lines
drivers/pci/tsm/core.c +16
-0+ tsm_dev = pdev->tsm->tsm_dev; +if (!sysfs_streq (buf,dev_name (&tsm_dev->dev))) +return -EINVAL ; + +ACQUIRE (device_intr, dev_lock)(&pdev->dev); +if ((rc =ACQUIRE_ERR (device_intr, &dev_lock))) +return rc; + +if (pdev->dev.driver) +return -EBUSY ; + +pci_tsm_unlock (pdev); + +return len; +} +static DEVICE_ATTR_WO (unlock);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
PATCH 11/15 PCI/TSM: Add device interface security RUN support
In reply to Dan WilliamsThe final device-side operation of the PCIe Trusted Execution Environment (TEE) Device Interface Security Protocol (TDISP) state machine is to ask the TSM to flip the device from LOCKED to RUN.
The story so far in the security lifecycle of the device is that the VMM setup an SPDM session and link encryption with the device's physical function0. The VMM then assigned either that physical function or other virtual function of that device to a VM. The VM asked the TSM to transition the device from TDISP UNLOCKED->LOCKED. With the device LOCKED the VM is enabled to validate signed fresh evidence of the device's configuration and expected MMIO mappings.
The VM now accepts the device to transition it from LOCKED to RUN.
Implement a sysfs trigger to flip the device to private operation and plumb that to a 'struct pci_tsm_ops::run()' operation.
Note that the RUN state only allows the device to be accessed via private MMIO. DMA enable is a separate step triggered at pci_dma_configure() time.
--- Documentation/ABI/testing/sysfs-bus-pci | 13 ++++ include/linux/pci-tsm.h | 12 +++- drivers/pci/tsm/core.c | 89 ++++++++++++++++++++++++- 3 files changed, 110 insertions(+), 4 deletions(-)
Documentation/ABI/testing/sysfs-bus-pci +13 -0
@@ -732,3 +732,16 @@ Description: 'lock' to teardown the connection. Writes fail with EBUSY if this device is bound to a driver. This is a "devsec" TSM attribute, see Documentation/ABI/testing/sysfs-class-tsm. + +What: /sys/bus/pci/devices/.../tsm/accept +Contact: [email protected] +Description: + (RW) Write "1" (or any boolean "true" string) to this file to + request that TSM transition the device from the TDISP LOCKED + state to the RUN state and arrange the for the secure IOMMU to + accept requests with T=1 in the PCIe packet header (TLP) + targeting private memory. Per TDISP the only exits from the RUN + state are via an explicit unlock request or an event that + transitions the device to the ERROR state. Writes fail with + EBUSY if this device is bound to a driver. This is a "devsec" + TSM attribute, see Documentation/ABI/testing/sysfs-class-tsm.
include/linux/pci-tsm.h +10 -2
@@ -70,15 +70,18 @@ struct pci_tsm_ops {* pci_tsm') for follow-on security state transitions from the * LOCKED state * @unlock: destroy TSM context and return device to UNLOCKED state +* @run: Move a device to the RUN state / prepare to enable DMA * * Context: @lock and @unlock run under pci_tsm_rwsem held for write to -* sync with TSM unregistration and each other. All operations run under -* the device lock for mutual exclusion with driver attach and detach. +* sync with TSM unregistration and each other. @run runs under +* pci_tsm_rwsem held for read. All operations run under the device lock +* for mutual exclusion with driver attach and detach. */ struct_group_tagged (pci_tsm_devsec_ops, devsec_ops, struct pci_tsm *(*lock)(struct tsm_dev *tsm_dev, struct pci_dev *pdev);void (*unlock)(struct pci_tsm *tsm); +int (*run)(struct pci_dev *pdev); ); int (*refresh_evidence)(struct pci_tsm *tsm, const void *nonce,
@@ -97,6 +100,9 @@ struct pci_tdi {u32 tdi_id; }; +/* Private operation acknowledged, future ioremap will use private alias */ +#define PCI_TSM_F_ACCEPT (1UL << 0) +/** * struct pci_tsm - Core TSM context for a given PCIe endpoint * @pdev: Back ref to device function, distinguishes type of pci_tsm context
@@ -104,6 +110,7 @@ struct pci_tdi {* @tsm_dev: PCI TEE Security Manager device for Link Confidentiality or Device * Function Security operations * @tdi: TDI context established by the @bind link operation +* @flags: Convey TDISP availabilty and / or state * @evidence: cached evidence from SPDM session establishment (connect), or * TDISP bind (lock) *
@@ -130,6 +137,7 @@ struct pci_tsm {struct tsm_dev *tsm_dev;struct pci_tdi *tdi;struct device_evidence *evidence; +unsigned long flags; };/**
drivers/pci/tsm/core.c +87 -2
115 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 11/15] PCI/TSM: Add device interface security RUN support
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
New issues:
- [High] pci_tsm_accept() unconditionally invokes the run callback without verifying the TSM type or callback presence, leading to a NULL pointer dereference.
- [High] The PCI_TSM_F_ACCEPT flag is not explicitly cleared, allowing a reused context to bypass the TDISP RUN state transition.
Pre-existing issues:
- [High] Missing validation of TSM type in unlock_store and disconnect_store can cause NULL pointer dereferences.
- [High] Lock order inversion between pci_tsm_rwsem and device_lock(&pdev->dev) causes an ABBA deadlock. --
commit dff8c6ed3353a6c7f7bb4d5a991bf30cccd06be9
Author: Dan Williams <[email protected]>
PCI/TSM: Add device interface security RUN support
This commit introduces a sysfs trigger to transition a device from the TDISP LOCKED state to the RUN state. It connects the accept trigger to a new run operation in the pci_tsm_ops structure. The RUN state prepares the device for private operation by enabling private MMIO access.
Quoted 4 lines
drivers/pci/tsm/core.c +0
-0
[ ... ]
Quoted 15 lines
drivers/pci/tsm/core.c +11
-0@@ -552,6 +552,89 @@ static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, }static DEVICE_ATTR_RO (dsm); +/** +* pci_tsm_accept() - accept a device for private MMIO operation +* @pdev: PCI device to accept +* +* "Accept" transitions the device interface to the RUN state, it is +* only suitable to make that transition from a known idle (no active +* DMA / MMIO mappings) state. The "driver detached" state is a coarse +* way to assert that requirement. +*/ +static int pci_tsm_accept (struct pci_dev *pdev) +{
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 575
Quoted 10 lines
drivers/pci/tsm/core.c +10
-0+int rc; + +ACQUIRE (rwsem_read_intr, lock)(&pci_tsm_rwsem); +if ((rc =ACQUIRE_ERR (rwsem_read_intr, &lock))) +return rc; + +if (!pdev->tsm) +return -EINVAL ; + +ACQUIRE (device_intr, dev_lock)(&pdev->dev);
[Severity: High]
This is a pre-existing issue, but does acquiring pci_tsm_rwsem and then device_lock(&pdev->dev) cause an ABBA deadlock?
If a device driver's remove() callback is invoked holding device_lock, and it calls pci_tsm_unbind(), which then attempts to acquire pci_tsm_rwsem, this could invert the lock ordering.
Concurrently, sysfs operations like unlock_store or lock_store acquire pci_tsm_rwsem before device_lock, which could trigger a deadlock.
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 582
Quoted 7 lines
drivers/pci/tsm/core.c +7
-0+if ((rc =ACQUIRE_ERR (device_intr, &dev_lock))) +return rc; + +if (pdev->dev.driver) +return -EBUSY ; + +if (test_bit (PCI_TSM_F_ACCEPT , &pdev->tsm->flags))
[Severity: High]
Can a reused context bypass the TDISP RUN state transition because PCI_TSM_F_ACCEPT is never explicitly cleared?
When a device is unlocked, the provider's ops->unlock() retains the struct pci_tsm object in memory. When the device is locked again, the core constructor pci_tsm_devsec_constructor() initializes the base fields but does not clear flags.
Does this allow a subsequent call to pci_tsm_accept() to observe the leftover PCI_TSM_F_ACCEPT bit and return success without calling the hardware run operation?
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 585
Quoted 3 lines
drivers/pci/tsm/core.c +3
-0+return 0 ; + + rc =to_pci_tsm_ops (pdev->tsm)->run (pdev);
[Severity: High]
Does this code unconditionally invoke the run callback without verifying the TSM type or callback presence, leading to a NULL pointer dereference?
If a user space process writes to the accept sysfs attribute of a device that is bound to a link TSM (which does not populate devsec_ops) or a devsec TSM that omitted the run callback, it seems this would result in a kernel crash.
Quoted 7 lines
drivers/pci/tsm/core.c +7
-0+if (rc) +return rc; + +set_bit (PCI_TSM_F_ACCEPT , &pdev->tsm->flags); + +return 0 ; +}
[ ... ]
drivers/pci/tsm/core.c static void pci_tsm_unlock(struct pci_dev *pdev) · line 654
Quoted 6 lines
drivers/pci/tsm/core.c +0
-1@@ -568,9 +651,9 @@ static void pci_tsm_unlock(struct pci_dev *pdev) "unlock attempted on driver attached device\n")) return; -device_cc_reject (&pdev->dev);to_pci_tsm_ops (pdev->tsm)->unlock (pdev->tsm);
[Severity: High]
This isn't a bug introduced by this patch, but could the lack of TSM type validation in unlock_store and disconnect_store cause NULL pointer dereferences?
If a user writes the name of a link TSM to the unlock attribute, or a devsec TSM to the disconnect attribute, the sysfs handler matches the name.
Since it fails to verify the specific type of operations supported by the TSM before calling ops->unlock or ops->disconnect, could this lead to a crash?
drivers/pci/tsm/core.c static void pci_tsm_unlock(struct pci_dev *pdev) · line 656
Quoted 2 lines
drivers/pci/tsm/core.c +1
-0pdev->tsm =NULL ; +/* implicit clear of PCI_TSM_F_ACCEPT */
[Severity: High]
Is this assumption correct that un-assigning the pointer clears the memory of the provider-managed flags field?
As asked above, if the struct pci_tsm object remains in memory and flags are not cleared during initialization, it looks like the PCI_TSM_F_ACCEPT flag could persist across locking cycles.
Quoted 1 line
drivers/pci/tsm/core.c +0
-0}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
PATCH 12/15 PCI/TSM: Add device interface security DMA enable/disable
In reply to Dan WilliamsAfter the device interface has been switched to the RUN state, the TSM is still responsible for enabling the device to access private memory. Placing the DSM into RUN state causes the device to accept T=1 (within TCB) traffic to its MMIO address space. The enable DMA step has the TSM configure the IOMMU to accept T=1 traffic to the CC VM's private memory in addition to all the memory previously accessible with T=0 DMA.
--- include/linux/pci-tsm.h | 13 +++++++++++++ drivers/pci/tsm/core.c | 42 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+)
include/linux/pci-tsm.h +13 -0
@@ -82,6 +82,8 @@ struct pci_tsm_ops { struct pci_dev *pdev);void (*unlock)(struct pci_tsm *tsm);int (*run)(struct pci_dev *pdev); +int (*enable_dma)(struct pci_dev *pdev); +void (*disable_dma)(struct pci_dev *pdev); ); int (*refresh_evidence)(struct pci_tsm *tsm, const void *nonce,
@@ -102,6 +104,8 @@ struct pci_tdi {/* Private operation acknowledged, future ioremap will use private alias */ #define PCI_TSM_F_ACCEPT (1UL << 0) +/* DMA access to private memory enabled */ +#define PCI_TSM_F_DMA (1UL << 1)/** * struct pci_tsm - Core TSM context for a given PCIe endpoint
@@ -247,6 +251,8 @@ int pci_tsm_doe_transfer(struct pci_dev *pdev, u8 type, const void *req, size_t req_sz, void *resp, size_t resp_sz);int pci_tsm_bind (struct pci_dev *pdev,struct kvm *kvm,u32 tdi_id);void pci_tsm_unbind (struct pci_dev *pdev); +int pci_tsm_enable_dma (struct pci_dev *pdev); +void pci_tsm_disable_dma (struct pci_dev *pdev);void pci_tsm_tdi_constructor (struct pci_dev *pdev,struct pci_tdi *tdi,struct kvm *kvm,u32 tdi_id); ssize_t pci_tsm_guest_req(struct pci_dev *pdev, enum pci_tsm_req_scope scope,
@@ -269,6 +275,13 @@ static inline int pci_tsm_bind(struct pci_dev *pdev, struct kvm *kvm, u64 tdi_id {return -ENXIO ; } +static inline int pci_tsm_enable_dma (struct pci_dev *pdev) +{ +return 0 ; +} +static inline void pci_tsm_disable_dma (struct pci_dev *pdev) +{ +}static inline void pci_tsm_unbind (struct pci_dev *pdev) { }
drivers/pci/tsm/core.c +42 -0
48 lines not on this page · raw message
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 12/15] PCI/TSM: Add device interface security DMA enable/disable
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
New issues:
- [High] Missing NULL pointer check for optional ops->disable_dma in pci_tsm_disable_dma().
- [Medium] TSM flags are defined as bit masks instead of bit numbers, misusing bitops APIs and creating a latent out-of-bounds write hazard.
- [Low] Invalid member access syntax in __must_hold() lock annotations causes build failures for static analysis tools.
Pre-existing issues:
- [High] Missing NULL pointer checks for optional TSM operations (run, unlock) in the TSM core. --
commit b2823259422c2c0267dccac4b1d9d8a5703f85ac
Author: Dan Williams <[email protected]>
PCI/TSM: Add device interface security DMA enable/disable
This patch adds helper functions to enable and disable device access to private memory for the PCI TSM interface. It also introduces a new flag to track when DMA access is enabled for a given device.
drivers/pci/tsm/core.c static ssize_t unlock_store(struct device *dev, struct device_attribute *attr, · line 758
Quoted 9 lines
drivers/pci/tsm/core.c +2
-0@@ -754,6 +754,48 @@ static ssize_t unlock_store(struct device *dev, struct device_attribute *attr, }static DEVICE_ATTR_WO (unlock); +/* pci_dma_configure() helper to finalize access to private memory. */ +int pci_tsm_enable_dma(struct pci_dev *pdev) __must_hold(&pdev->dev->mutex)
[Severity: Low]
Does this __must_hold() annotation cause build failures with static analysis tools?
The dev field in struct pci_dev is an embedded struct device, not a pointer, so it looks like this should use the dot operator instead of the arrow operator for accessing the mutex.
Quoted 4 lines
drivers/pci/tsm/core.c +4
-0+{ +struct device *dev = &pdev->dev; +const struct pci_tsm_ops *ops; +int rc;
[ ... ]
drivers/pci/tsm/core.c static ssize_t unlock_store(struct device *dev, struct device_attribute *attr, · line 777
Quoted 7 lines
drivers/pci/tsm/core.c +7
-0+ ops =to_pci_tsm_ops (pdev->tsm); +if (!ops->enable_dma) +return 0 ; + + rc = ops->enable_dma (pdev); +if (rc ==0 ) +set_bit (PCI_TSM_F_DMA , &pdev->tsm->flags);
[Severity: Medium]
Can this corrupt memory?
The set_bit() function expects a bit number as its argument, but the PCI_TSM_F_DMA flag is defined as a bit mask. Passing the evaluated mask value instead of the bit index causes it to set the wrong bit in the flags field.
If a future flag shifts by 64, wouldn't passing 64 to set_bit() write past the end of the unsigned long flags field and silently corrupt adjacent memory?
drivers/pci/tsm/core.c static ssize_t unlock_store(struct device *dev, struct device_attribute *attr, · line 785
Quoted 8 lines
drivers/pci/tsm/core.c +8
-0+return rc; +} + +/* +* pci_dma_cleanup() helper to block private memory access as device is +* going idle +*/ +void pci_tsm_disable_dma(struct pci_dev *pdev) __must_hold(&pdev->dev->mutex)
[Severity: Low]
Is this __must_hold() annotation affected by the same embedded struct access issue mentioned in pci_tsm_enable_dma()?
drivers/pci/tsm/core.c static ssize_t unlock_store(struct device *dev, struct device_attribute *attr, · line 796
Quoted 11 lines
drivers/pci/tsm/core.c +11
-0+{ +const struct pci_tsm_ops *ops; + +if (!pdev->tsm) +return ; + +if (!test_and_clear_bit (PCI_TSM_F_DMA , &pdev->tsm->flags)) +return ; + + ops =to_pci_tsm_ops (pdev->tsm); + ops->disable_dma (pdev);
[Severity: High]
Will this crash if a backend driver implements enable_dma but leaves disable_dma as NULL?
The disable_dma pointer is unconditionally dereferenced here, whereas pci_tsm_enable_dma() checks for the presence of the enable_dma callback before calling it.
Quoted 1 line
+}
[ ... ]
Quoted 3 lines
include/linux/pci-tsm.h +0
-0
[ ... ]
include/linux/pci-tsm.h struct pci_tdi { · line 108
Quoted 6 lines
include/linux/pci-tsm.h +2
-0@@ -102,6 +104,8 @@ struct pci_tdi {/* Private operation acknowledged, future ioremap will use private alias */ #define PCI_TSM_F_ACCEPT (1UL << 0) +/* DMA access to private memory enabled */ +#define PCI_TSM_F_DMA (1UL << 1)
[Severity: Medium]
Is this flag intended to be used with atomic bitops?
Because it is defined as a bit mask instead of a bit number, using it with functions like set_bit() or test_and_clear_bit() will operate on bit 2 instead of bit 1.
[ ... ]
While reviewing this patch, I also noticed a potential issue in the existing TSM core code:
drivers/pci/tsm/core.c:pci_tsm_accept() {
...
rc = to_pci_tsm_ops(pdev->tsm)->run(pdev);
if (rc)
return rc;
...
}[Severity: High]
This is a pre-existing issue, but could this cause a kernel panic if a backend driver omits the optional run or unlock callbacks?
The code in pci_tsm_accept() and pci_tsm_unlock() seems to unconditionally dereference these pointers without checking if they are NULL, similar to the disable_dma issue mentioned above.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
PATCH 13/15 PCI, device core: Add private memory access for DEVICE_TRUST_TCB
In reply to Dan WilliamsA device that wants to access private memory needs to have its trust elevated to DEVICE_TRUST_TCB. That trust is established either at compile time (unlikely), the bus knows the device is within the TCB to start (some paravisor setups), or the device is dynamically added to the TCB in coordination with a TSM driver (primary TDISP use case) and the trust is elevated by driver match.
When a PCI device is associated with a TSM for security services the low level TSM driver in the CC VM has the opportunity validate DMA access. That validation happens at ->dma_configure() time when the device attaches to a driver. The TSM driver is responsible for proving to the platform TSM that the VM is enabling DMA with respect to the most recently generated evidence. If that fails, driver attach fails.
When a PCI device is not associated with a TSM provider for security services, but the device is trusted there are 3 options.
1/ Arch requires all DMA enable events to be acked by TSM driver
2/ Arch does not require, but admin policy is responsible for knowing which
devices need TSM coordination to become active within the TCB.
3/ Device is approved by a paravisor to operate within the TCB, no TSM
coordination required.
In cases 2 and 3 if the device needed TSM driver coordination, but the TSM driver or association to the device is missing, it triggers hardware errors. Those errors are a configuration error that the kernel does not actively prevent.
Architectures still need to fixup force_dma_unencrypted() to call device_tcb_trusted() to tell the DMA layer that TCB access is granted.
--- drivers/base/Kconfig | 19 ++++++++++++++++++- include/linux/device/trust.h | 4 ++++ drivers/base/trust.c | 8 ++++++++ drivers/pci/pci-driver.c | 25 ++++++++++++++++--------- 4 files changed, 46 insertions(+), 10 deletions(-)
drivers/base/Kconfig +18 -1
@@ -304,7 +304,17 @@ config DEVICE_TRUST_AUTO help Typical historical driver model, devices eagerly attempt to attach to a driver and deploy all available mechanisms to allow performant - direct memory access. + direct memory access. This trust level does not include TCB privileges. + +config DEVICE_TRUST_TCB + bool "TCB" + depends on EXPERT + help + Devices are assumed to be within the Trusted Compute Boundary (TCB) + with access to private TCB resources by default. This requires buses + to have trust awareness and opt devices out of private operation, + otherwise TCB integrity may fail the device's access or escalate to + terminating the execution context (kill the CC VM). endchoice
@@ -330,6 +340,13 @@ config BUILTIN_DEVICE_TRUST_ADVERSARY help Deploy mitigations in the IOMMU layer and driver to limit access. +config BUILTIN_DEVICE_TRUST_TCB + bool "TCB" + depends on EXPERT + help + Devices bound by built-in drivers are assumed to be within the + Trusted Compute Boundary with access to private/encrypted memory. + endchoice endmenu
include/linux/device/trust.h +4 -0
@@ -13,6 +13,7 @@* @DEVICE_TRUST_NONE: Blocked when idle, cannot bind * @DEVICE_TRUST_ADVERSARY: Blocked when idle, constrained when active. * @DEVICE_TRUST_AUTO: All typical privileges granted +* @DEVICE_TRUST_TCB: AUTO privileges + private/encrypted memory access * * Devices flagged as adversarial are the ones that can potentially * execute DMA attacks and similar. They are typically connected through
@@ -25,11 +26,13 @@ enum device_trust {DEVICE_TRUST_NONE ,DEVICE_TRUST_ADVERSARY ,DEVICE_TRUST_AUTO , + DEVICE_TRUST_TCB, };#define DEVICE_DEFAULT_TRUST \ (IS_ENABLED(CONFIG_DEVICE_TRUST_NONE) ? DEVICE_TRUST_NONE : \ IS_ENABLED(CONFIG_DEVICE_TRUST_ADVERSARY) ? DEVICE_TRUST_ADVERSARY : \ + IS_ENABLED(CONFIG_DEVICE_TRUST_TCB) ? DEVICE_TRUST_TCB : \ DEVICE_TRUST_AUTO)struct device ;
@@ -37,6 +40,7 @@ struct device_driver;#ifdef CONFIG_DEVICE_TRUST bool device_untrusted (struct device *dev); +bool device_tcb_trusted (struct device *dev);void module_driver_trust (struct module *mod,const char *val);void module_driver_trust_init (struct module *mod,bool require_trust);#else
drivers/base/trust.c +8 -0
@@ -23,11 +23,18 @@ bool device_untrusted(struct device *dev)return dev->bus_trust && dev->bus_trust <=DEVICE_TRUST_ADVERSARY ; } +bool device_tcb_trusted (struct device *dev) +{ +return dev->p->trust >=DEVICE_TRUST_TCB ; +} +/* Driver trust policy requires modules, builtin drivers always attach */ static enum device_trust builtin_driver_trust (void ) {if (IS_ENABLED (CONFIG_BUILTIN_DEVICE_TRUST_ADVERSARY ))return DEVICE_TRUST_ADVERSARY ; +else if (IS_ENABLED (CONFIG_BUILTIN_DEVICE_TRUST_TCB )) +return DEVICE_TRUST_TCB ;return DEVICE_TRUST_AUTO ; }
@@ -61,6 +68,7 @@ static const char * const device_trust_names[] = { [DEVICE_TRUST_NONE] = "none", [DEVICE_TRUST_ADVERSARY] = "adversary", [DEVICE_TRUST_AUTO] = "auto", + [DEVICE_TRUST_TCB] = "tcb", };static enum device_trust device_trust_parse (const char *name)
drivers/pci/pci-driver.c +16 -9
@@ -19,6 +19,7 @@#include <linux/kexec.h> #include <linux/of_device.h> #include <linux/acpi.h> +#include <linux/pci-tsm.h> #include <linux/dma-map-ops.h> #include <linux/iommu.h> #include "pci.h"
@@ -1668,10 +1669,11 @@ static int pci_bus_num_vf(struct device *dev)static int pci_dma_configure (struct device *dev) {const struct device_driver *drv =READ_ONCE (dev->driver); -struct device *bridge; +struct pci_dev *pdev =to_pci_dev (dev);int ret =0 ; - bridge =pci_get_host_bridge_device (to_pci_dev (dev)); + struct device *bridge __free(put_device) = +pci_get_host_bridge_device (pdev);if (IS_ENABLED (CONFIG_OF ) && bridge->parent && bridge->parent->of_node) {
@@ -1688,16 +1690,21 @@ static int pci_dma_configure(struct device *dev)* the standard ACS capability but still support ACS via those * quirks. */ -pci_enable_acs (to_pci_dev (dev)); - -pci_put_host_bridge_device (bridge); +pci_enable_acs (pdev);/* @drv may not be valid when we're called from the IOMMU layer */ -if (!ret && drv && !to_pci_driver (drv)->driver_managed_dma) { +if (ret || !drv) +return ret; + +if (device_tcb_trusted (dev)) + ret =pci_tsm_enable_dma (pdev); + +if (!ret && !to_pci_driver (drv)->driver_managed_dma) ret =iommu_device_use_default_domain (dev); -if (ret) -arch_teardown_dma_ops (dev); - } + +/* undo {of,acpi}_dma_configure() */ +if (ret) +arch_teardown_dma_ops (dev);return ret; }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 13/15] PCI, device core: Add private memory access for DEVICE_TRUST_TCB
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Missing pci_tsm_disable_dma() in pci_dma_configure() error path and pci_dma_cleanup() to revoke…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Missing pci_tsm_disable_dma() in pci_dma_configure() error path and pci_dma_cleanup() to revoke TCB DMA access.
- [Low] Invalid syntax in __must_hold attribute for pci_tsm_enable_dma and pci_tsm_disable_dma.
--
commit 02fe1d53a8bdd83c53510c3c6636b82d7cd34b02
Author: Dan Williams <[email protected]>
PCI, device core: Add private memory access for DEVICE_TRUST_TCB
This commit introduces the DEVICE_TRUST_TCB trust level to allow devices within the Trusted Compute Boundary to access private memory. It updates the PCI DMA configuration to validate DMA access with a TSM driver when a device is TCB-trusted.
Quoted 4 lines
drivers/pci/pci-driver.c +0
-0
[ ... ]
Quoted 16 lines
drivers/pci/pci-driver.c +6
-4@@ -1688,16 +1690,21 @@ static int pci_dma_configure(struct device *dev)* the standard ACS capability but still support ACS via those * quirks. */ -pci_enable_acs (to_pci_dev (dev)); - -pci_put_host_bridge_device (bridge); +pci_enable_acs (pdev);/* @drv may not be valid when we're called from the IOMMU layer */ -if (!ret && drv && !to_pci_driver (drv)->driver_managed_dma) { +if (ret || !drv) +return ret; + +if (device_tcb_trusted (dev)) + ret =pci_tsm_enable_dma (pdev);
drivers/pci/pci-driver.c static int pci_dma_configure(struct device *dev) · line 1707
Quoted 10 lines
drivers/pci/pci-driver.c +6
-3+ +if (!ret && !to_pci_driver (drv)->driver_managed_dma) ret =iommu_device_use_default_domain (dev); -if (ret) -arch_teardown_dma_ops (dev); - } + +/* undo {of,acpi}_dma_configure() */ +if (ret) +arch_teardown_dma_ops (dev);
[Severity: High]
If iommu_device_use_default_domain() fails, does this error path need to call pci_tsm_disable_dma() to revoke the TCB DMA access granted above?
Also, when the device driver unbinds, should pci_dma_cleanup() be updated to call pci_tsm_disable_dma() as well?
Without these cleanup calls, it appears the TSM could incorrectly maintain DMA authorization for a device even after a probe failure or driver unbind.
[Severity: Low]
On a related note regarding the TSM core API modified in this series, are the __must_hold() attributes in drivers/pci/tsm/core.c syntactically correct?
drivers/pci/tsm/core.c:pci_tsm_enable_dma() {
...
int pci_tsm_enable_dma(struct pci_dev *pdev)
__must_hold(&pdev->dev->mutex)Since dev is a nested struct within struct pci_dev, not a pointer, shouldn't this be &pdev->dev.mutex to avoid compiler errors? The same appears to apply to pci_tsm_disable_dma().
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
Aneesh Kumar K.V
maintainerRe: [PATCH 13/15] PCI, device core: Add private memory access for DEVICE_TRUST_TCB
In reply to Dan WilliamsQuoted 7 lines · Dan Williams
A device that wants to access private memory needs to have its trust elevated to
DEVICE_TRUST_TCB. That trust is established either at compile time (unlikely), the bus knows the device is within the TCB to start (some paravisor setups), or the device is dynamically added to the TCB in coordination with a TSM driver (primary TDISP use case) and the trust is elevated by driver match.
Do we have the last case, where the device is dynamically added to the TCB in coordination with a TSM, implemented in this series? Do we expect the CCA driver to set that up?
Quoted 19 lines
When a PCI device is associated with a TSM for security services the low level TSM driver in the CC VM has the opportunity validate DMA access. That validation happens at ->dma_configure() time when the device attaches to a driver. The TSM driver is responsible for proving to the platform TSM that the VM is enabling DMA with respect to the most recently generated evidence. If that fails, driver attach fails.
When a PCI device is not associated with a TSM provider for security services, but the device is trusted there are 3 options.
1/ Arch requires all DMA enable events to be acked by TSM driver
2/ Arch does not require, but admin policy is responsible for knowing which
devices need TSM coordination to become active within the TCB.3/ Device is approved by a paravisor to operate within the TCB, no TSM
coordination required.
Quoted 8 lines
In cases 2 and 3 if the device needed TSM driver coordination, but the TSM driver or association to the device is missing, it triggers hardware errors. Those errors are a configuration error that the kernel does not actively prevent.
Architectures still need to fixup
force_dma_unencrypted()to calldevice_tcb_trusted()to tell the DMA layer that TCB access is granted.
-aneesh
Re: [PATCH 13/15] PCI, device core: Add private memory access for DEVICE_TRUST_TCB
Quoted 11 lines · Aneesh Kumar K.V
Quoted 7 lines · Dan Williams
A device that wants to access private memory needs to have its trust elevated to
DEVICE_TRUST_TCB. That trust is established either at compile time (unlikely), the bus knows the device is within the TCB to start (some paravisor setups), or the device is dynamically added to the TCB in coordination with a TSM driver (primary TDISP use case) and the trust is elevated by driver match.
Quoted 3 lines
Do we have the last case, where the device is dynamically added to the TCB in coordination with a TSM, implemented in this series? Do we expect the CCA driver to set that up?
In this case "dynamically added" means pre-existing device converted from shared to private operation. I.e. the model the work-in-progress CCA patches already cover if that was your question?
Aneesh Kumar K.V
maintainerRe: [PATCH 13/15] PCI, device core: Add private memory access for DEVICE_TRUST_TCB
Quoted 12 lines · Dan Williams (nvidia)
Quoted 11 lines · Aneesh Kumar K.V
Quoted 7 lines · Dan Williams
A device that wants to access private memory needs to have its trust elevated to
DEVICE_TRUST_TCB. That trust is established either at compile time (unlikely), the bus knows the device is within the TCB to start (some paravisor setups), or the device is dynamically added to the TCB in coordination with a TSM driver (primary TDISP use case) and the trust is elevated by driver match.
Quoted 7 lines
Quoted 3 lines
Do we have the last case, where the device is dynamically added to the TCB in coordination with a TSM, implemented in this series? Do we expect the CCA driver to set that up?
In this case "dynamically added" means pre-existing device converted from shared to private operation. I.e. the model the work-in-progress CCA patches already cover if that was your question?
So we accept the device and then load the driver with trust=tcb ?. If not, where do we set the device->bus_trust = DEVICE_TRUST_TCB?
-aneesh
Alexey Kardashevskiy
maintainerRe: [PATCH 13/15] PCI, device core: Add private memory access for DEVICE_TRUST_TCB
In reply to Dan WilliamsQuoted 127 lines · Dan Williams
A device that wants to access private memory needs to have its trust elevated to
DEVICE_TRUST_TCB. That trust is established either at compile time (unlikely), the bus knows the device is within the TCB to start (some paravisor setups), or the device is dynamically added to the TCB in coordination with a TSM driver (primary TDISP use case) and the trust is elevated by driver match.When a PCI device is associated with a TSM for security services the low level TSM driver in the CC VM has the opportunity validate DMA access. That validation happens at ->dma_configure() time when the device attaches to a driver. The TSM driver is responsible for proving to the platform TSM that the VM is enabling DMA with respect to the most recently generated evidence. If that fails, driver attach fails.
When a PCI device is not associated with a TSM provider for security services, but the device is trusted there are 3 options.
1/ Arch requires all DMA enable events to be acked by TSM driver
2/ Arch does not require, but admin policy is responsible for knowing which
devices need TSM coordination to become active within the TCB.3/ Device is approved by a paravisor to operate within the TCB, no TSM
coordination required.In cases 2 and 3 if the device needed TSM driver coordination, but the TSM driver or association to the device is missing, it triggers hardware errors. Those errors are a configuration error that the kernel does not actively prevent.
Architectures still need to fixup
force_dma_unencrypted()to calldevice_tcb_trusted()to tell the DMA layer that TCB access is granted.Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Danilo Krummrich <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Dexuan Cui <[email protected]>Signed-off-by Dan Williams <[email protected]>--- drivers/base/Kconfig | 19 ++++++++++++++++++- include/linux/device/trust.h | 4 ++++ drivers/base/trust.c | 8 ++++++++ drivers/pci/pci-driver.c | 25 ++++++++++++++++--------- 4 files changed, 46 insertions(+), 10 deletions(-)drivers/base/Kconfig +18
-1@@ -304,7 +304,17 @@ config DEVICE_TRUST_AUTO help Typical historical driver model, devices eagerly attempt to attach to a driver and deploy all available mechanisms to allow performant - direct memory access. + direct memory access. This trust level does not include TCB privileges. + +config DEVICE_TRUST_TCB + bool "TCB" + depends on EXPERT + help + Devices are assumed to be within the Trusted Compute Boundary (TCB) + with access to private TCB resources by default. This requires buses + to have trust awareness and opt devices out of private operation, + otherwise TCB integrity may fail the device's access or escalate to + terminating the execution context (kill the CC VM). endchoice@@ -330,6 +340,13 @@ config BUILTIN_DEVICE_TRUST_ADVERSARY help Deploy mitigations in the IOMMU layer and driver to limit access. +config BUILTIN_DEVICE_TRUST_TCB + bool "TCB" + depends on EXPERT + help + Devices bound by built-in drivers are assumed to be within the + Trusted Compute Boundary with access to private/encrypted memory. + endchoice endmenuinclude/linux/device/trust.h +4
-0@@ -13,6 +13,7 @@* @DEVICE_TRUST_NONE: Blocked when idle, cannot bind * @DEVICE_TRUST_ADVERSARY: Blocked when idle, constrained when active. * @DEVICE_TRUST_AUTO: All typical privileges granted +* @DEVICE_TRUST_TCB: AUTO privileges + private/encrypted memory access * * Devices flagged as adversarial are the ones that can potentially * execute DMA attacks and similar. They are typically connected through @@ -25,11 +26,13 @@ enum device_trust {DEVICE_TRUST_NONE ,DEVICE_TRUST_ADVERSARY ,DEVICE_TRUST_AUTO , + DEVICE_TRUST_TCB, };#define DEVICE_DEFAULT_TRUST \ (IS_ENABLED(CONFIG_DEVICE_TRUST_NONE) ? DEVICE_TRUST_NONE : \ IS_ENABLED(CONFIG_DEVICE_TRUST_ADVERSARY) ? DEVICE_TRUST_ADVERSARY : \ + IS_ENABLED(CONFIG_DEVICE_TRUST_TCB) ? DEVICE_TRUST_TCB : \ DEVICE_TRUST_AUTO)struct device ;@@ -37,6 +40,7 @@ struct device_driver;#ifdef CONFIG_DEVICE_TRUST bool device_untrusted (struct device *dev); +bool device_tcb_trusted (struct device *dev);void module_driver_trust (struct module *mod,const char *val);void module_driver_trust_init (struct module *mod,bool require_trust);#else drivers/base/trust.c +2
-0@@ -23,11 +23,18 @@ bool device_untrusted(struct device *dev)return dev->bus_trust && dev->bus_trust <=DEVICE_TRUST_ADVERSARY ; } +bool device_tcb_trusted (struct device *dev) +{
drivers/base/trust.c bool device_untrusted(struct device *dev) · line 37
Quoted 10 lines
drivers/base/trust.c +5
-0+return dev->p->trust >=DEVICE_TRUST_TCB ; +} +/* Driver trust policy requires modules, builtin drivers always attach */ static enum device_trust builtin_driver_trust (void ) {if (IS_ENABLED (CONFIG_BUILTIN_DEVICE_TRUST_ADVERSARY ))return DEVICE_TRUST_ADVERSARY ; +else if (IS_ENABLED (CONFIG_BUILTIN_DEVICE_TRUST_TCB )) +return DEVICE_TRUST_TCB ;
No plan to override this via the kernel cmdline?
Quoted 43 lines
drivers/base/trust.c +1
-0return DEVICE_TRUST_AUTO ; }@@ -61,6 +68,7 @@ static const char * const device_trust_names[] = { [DEVICE_TRUST_NONE] = "none", [DEVICE_TRUST_ADVERSARY] = "adversary", [DEVICE_TRUST_AUTO] = "auto", + [DEVICE_TRUST_TCB] = "tcb", };static enum device_trust device_trust_parse (const char *name)drivers/pci/pci-driver.c +4
-4@@ -19,6 +19,7 @@#include <linux/kexec.h> #include <linux/of_device.h> #include <linux/acpi.h> +#include <linux/pci-tsm.h> #include <linux/dma-map-ops.h> #include <linux/iommu.h> #include "pci.h" @@ -1668,10 +1669,11 @@ static int pci_bus_num_vf(struct device *dev)static int pci_dma_configure (struct device *dev) {const struct device_driver *drv =READ_ONCE (dev->driver); -struct device *bridge; +struct pci_dev *pdev =to_pci_dev (dev);int ret =0 ; - bridge =pci_get_host_bridge_device (to_pci_dev (dev)); + struct device *bridge __free(put_device) = +pci_get_host_bridge_device (pdev);if (IS_ENABLED (CONFIG_OF ) && bridge->parent && bridge->parent->of_node) {@@ -1688,16 +1690,21 @@ static int pci_dma_configure(struct device *dev)* the standard ACS capability but still support ACS via those * quirks. */ -pci_enable_acs (to_pci_dev (dev)); -
drivers/pci/pci-driver.c static int pci_dma_configure(struct device *dev) · line 1700
Quoted 10 lines
drivers/pci/pci-driver.c +6
-2-pci_put_host_bridge_device (bridge); +pci_enable_acs (pdev);/* @drv may not be valid when we're called from the IOMMU layer */ -if (!ret && drv && !to_pci_driver (drv)->driver_managed_dma) { +if (ret || !drv) +return ret; + +if (device_tcb_trusted (dev)) + ret =pci_tsm_enable_dma (pdev);
pci_tsm_disable_dma() is missing in pci_dma_cleanup() then. Otherwise works fine. Thanks!
Quoted 13 lines
drivers/pci/pci-driver.c +6
-3+ +if (!ret && !to_pci_driver (drv)->driver_managed_dma) ret =iommu_device_use_default_domain (dev); -if (ret) -arch_teardown_dma_ops (dev); - } + +/* undo {of,acpi}_dma_configure() */ +if (ret) +arch_teardown_dma_ops (dev);return ret; }
Signature · 1 line
Alexey
Alexey Kardashevskiy
maintainerRe: [PATCH 13/15] PCI, device core: Add private memory access for DEVICE_TRUST_TCB
Quoted 125 lines · Alexey Kardashevskiy
Quoted 124 lines · Dan Williams
A device that wants to access private memory needs to have its trust elevated to
DEVICE_TRUST_TCB. That trust is established either at compile time (unlikely), the bus knows the device is within the TCB to start (some paravisor setups), or the device is dynamically added to the TCB in coordination with a TSM driver (primary TDISP use case) and the trust is elevated by driver match.When a PCI device is associated with a TSM for security services the low level TSM driver in the CC VM has the opportunity validate DMA access. That validation happens at ->dma_configure() time when the device attaches to a driver. The TSM driver is responsible for proving to the platform TSM that the VM is enabling DMA with respect to the most recently generated evidence. If that fails, driver attach fails.
When a PCI device is not associated with a TSM provider for security services, but the device is trusted there are 3 options.
1/ Arch requires all DMA enable events to be acked by TSM driver
2/ Arch does not require, but admin policy is responsible for knowing which devices need TSM coordination to become active within the TCB.
3/ Device is approved by a paravisor to operate within the TCB, no TSM coordination required.
In cases 2 and 3 if the device needed TSM driver coordination, but the TSM driver or association to the device is missing, it triggers hardware errors. Those errors are a configuration error that the kernel does not actively prevent.
Architectures still need to fixup
force_dma_unencrypted()to calldevice_tcb_trusted()to tell the DMA layer that TCB access is granted.Cc Greg Kroah-Hartman <[email protected]>Cc "Rafael J. Wysocki" <[email protected]>Cc Danilo Krummrich <[email protected]>Cc Bjorn Helgaas <[email protected]>Cc Dexuan Cui <[email protected]>Signed-off-by Dan Williams <[email protected]>---
drivers/base/Kconfig | 19 ++++++++++++++++++-include/linux/device/trust.h| 4 ++++ drivers/base/trust.c | 8 ++++++++ drivers/pci/pci-driver.c | 25 ++++++++++++++++---------4 files changed, 46 insertions(+), 10 deletions(-)drivers/base/Kconfig +0
-0@@ -304,7 +304,17 @@ config DEVICE_TRUST_AUTOhelp
Typical historical driver model, devices eagerly attempt to attach to a driver and deploy all available mechanisms to allow performant - direct memory access.
+ direct memory access. This trust level does not include TCB privileges.drivers/base/Kconfig +10
-0+ +config DEVICE_TRUST_TCB + bool "TCB" + depends on EXPERT + help + Devices are assumed to be within the Trusted Compute Boundary (TCB) + with access to private TCB resources by default. This requires buses + to have trust awareness and opt devices out of private operation, + otherwise TCB integrity may fail the device's access or escalate to + terminating the execution context (kill the CC VM).endchoice
drivers/base/Kconfig +0
-0@@ -330,6 +340,13 @@ config BUILTIN_DEVICE_TRUST_ADVERSARYhelp
Deploy mitigations in the IOMMU layer and driver to limit access.drivers/base/Kconfig +7
-0+config BUILTIN_DEVICE_TRUST_TCB + bool "TCB" + depends on EXPERT + help + Devices bound by built-in drivers are assumed to be within the + Trusted Compute Boundary with access to private/encrypted memory. +endchoice
endmenuinclude/linux/device/trust.h +4
-0@@ -13,6 +13,7 @@ * @DEVICE_TRUST_NONE: Blocked when idle, cannot bind * @DEVICE_TRUST_ADVERSARY: Blocked when idle, constrained when active. * @DEVICE_TRUST_AUTO: All typical privileges granted + * @DEVICE_TRUST_TCB: AUTO privileges + private/encrypted memory access * * Devices flagged as adversarial are the ones that can potentially * execute DMA attacks and similar. They are typically connected through@@ -25,11 +26,13 @@ enum device_trust { DEVICE_TRUST_NONE, DEVICE_TRUST_ADVERSARY, DEVICE_TRUST_AUTO, + DEVICE_TRUST_TCB, }; #define DEVICE_DEFAULT_TRUST \ (IS_ENABLED(CONFIG_DEVICE_TRUST_NONE) ? DEVICE_TRUST_NONE : \ IS_ENABLED(CONFIG_DEVICE_TRUST_ADVERSARY) ? DEVICE_TRUST_ADVERSARY : \ + IS_ENABLED(CONFIG_DEVICE_TRUST_TCB) ? DEVICE_TRUST_TCB : \ DEVICE_TRUST_AUTO) struct device;@@ -37,6 +40,7 @@ struct device_driver; #ifdef CONFIG_DEVICE_TRUST bool device_untrusted(struct device *dev); +bool device_tcb_trusted (struct device *dev); void module_driver_trust(struct module *mod, const char *val); void module_driver_trust_init(struct module *mod, bool require_trust); #elsedrivers/base/trust.c +5
-0@@ -23,11 +23,18 @@ bool device_untrusted(struct device *dev) return dev->bus_trust && dev->bus_trust <= DEVICE_TRUST_ADVERSARY; } +bool device_tcb_trusted (struct device *dev) +{ + return dev->p->trust >= DEVICE_TRUST_TCB; +} + /* Driver trust policy requires modules, builtin drivers always attach */ static enum device_trust builtin_driver_trust(void)
Quoted 8 lines
drivers/base/trust.c bool device_untrusted(struct device *dev) · line 37
Quoted 5 lines
drivers/base/trust.c +2
-0{ if (IS_ENABLED(CONFIG_BUILTIN_DEVICE_TRUST_ADVERSARY)) return DEVICE_TRUST_ADVERSARY; + else if (IS_ENABLED(CONFIG_BUILTIN_DEVICE_TRUST_TCB)) + return DEVICE_TRUST_TCB;No plan to override this via the kernel cmdline?
Just tried - my test guest kernel has everything built-in. Since there is no module parameter - I did CONFIG_BUILTIN_DEVICE_TRUST_TCB=y. But now every driver gets this level, including virtio, which cannot ever be "TCB" (as anything emulated by QEMU). Thanks,
Signature · 1 line
Alexey
PATCH 14/15 PCI/TSM: Create MMIO descriptors via TDISP Report
In reply to Dan WilliamsAfter pci_tsm_bind() and pci_tsm_lock() the low level TSM driver is expected to populate PCI_TSM_EVIDENCE_TYPE_REPORT in its evidence store. This report is defined by the TDISP GET_DEVICE_INTERFACE_REPORT response payload.
Add a helper to create encrypted MMIO descriptors from that report data. With those descriptors the TSM driver can use pci_tsm_mmio_setup() to inform ioremap() how to map the device per the device's expectations. The VM is expected to validate the interface with the relying party before accepting the device for operation.
The helper also provides the obfuscated starting address for each encrypted MMIO range as the VM is never disclosed on the hpa that correlates to the gpa of the device's mmio. The obfuscated address is BAR aligned.
Based on an original patch by Aneesh [1]
--- include/linux/ioport.h | 2 + include/linux/pci-tsm.h | 35 +++++++ drivers/pci/tsm/core.c | 224 ++++++++++++++++++++++++++++++++++++++++ kernel/resource.c | 8 ++ 4 files changed, 269 insertions(+)
include/linux/ioport.h +2 -0
@@ -144,6 +144,7 @@ enum {IORES_DESC_RESERVED =7 ,IORES_DESC_SOFT_RESERVED =8 ,IORES_DESC_CXL =9 , + IORES_DESC_ENCRYPTED = 10, };/*
@@ -240,6 +241,7 @@ struct resource_constraint {extern struct resource ioport_resource;extern struct resource iomem_resource;extern struct resource soft_reserve_resource; +extern struct resource encrypted_iomem_resource;extern struct resource *request_resource_conflict (struct resource *root,struct resource *new);extern int request_resource (struct resource *root,struct resource *new);
include/linux/pci-tsm.h +35 -0
@@ -156,12 +156,42 @@ struct pci_tsm_pf0 {struct pci_doe_mb *doe_mb; }; +/** +* struct pci_tsm_mmio_entry - an encrypted MMIO range +* @res: MMIO address range (typically Guest Physical Address, GPA) +* @tsm_offset: Host Physical Address, HPA obfuscation offset added by the TSM. +* Translates report addresses to GPA. +*/ +struct pci_tsm_mmio_entry { +struct resource res; +u64 tsm_offset; +}; + +struct pci_tsm_mmio { +int nr; +struct pci_tsm_mmio_entry mmio[]; +}; + +static inline struct pci_tsm_mmio_entry * +pci_tsm_mmio_entry (struct pci_tsm_mmio *mmio,int idx) +{ +return &mmio->mmio[idx]; +} + +static inline struct resource *pci_tsm_mmio_resource (struct pci_tsm_mmio *mmio, +int idx) +{ +return &mmio->mmio[idx].res; +} +/** * struct pci_tsm_devsec - context for tracking private/accepted PCI resources * @base_tsm: generic core "tsm" context +* @mmio: encrypted MMIO resources for this assigned device */ struct pci_tsm_devsec {struct pci_tsm base_tsm; +struct pci_tsm_mmio *mmio; };/* physical function0 and capable of 'connect' */
@@ -263,6 +293,11 @@ static inline const struct pci_tsm_ops *to_pci_tsm_ops(struct pci_tsm *tsm) return tsm->tsm_dev->pci_ops; }struct pci_tsm_devsec *to_pci_tsm_devsec (struct pci_tsm *tsm); +int pci_tsm_mmio_setup (struct pci_dev *pdev,struct pci_tsm_mmio *mmio); +void pci_tsm_mmio_teardown (struct pci_tsm_mmio *mmio); + +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev); +int pci_tsm_mmio_free (struct pci_dev *pdev,struct pci_tsm_mmio *mmio);#else static inline int pci_tsm_register (struct tsm_dev *tsm_dev) {
drivers/pci/tsm/core.c +224 -0
236 lines not on this page · raw message
kernel/resource.c +8 -0
@@ -56,6 +56,14 @@ struct resource soft_reserve_resource = { .flags = IORESOURCE_MEM, }; +struct resource encrypted_iomem_resource = { + .name ="Encrypted MMIO" , + .start =0 , + .end =-1 , + .desc =IORES_DESC_ENCRYPTED , + .flags =IORESOURCE_MEM , +}; +static DEFINE_RWLOCK (resource_lock);/*
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 14/15] PCI/TSM: Create MMIO descriptors via TDISP Report
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] 32-bit unsigned multiplication overflow in MMIO region size calculation silently truncates the…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] 32-bit unsigned multiplication overflow in MMIO region size calculation silently truncates the size for regions >= 4GB.
- [Medium] Bitwise NOT of a 32-bit unsigned long mask zeroes out the upper 32 bits of a 64-bit physical address on 32-bit architectures. --
commit cfcbb8abea88df9f3a56b20ab3e4fac1bdbf37ea
Author: Dan Williams <[email protected]>
PCI/TSM: Create MMIO descriptors via TDISP Report
This commit introduces a helper function to create encrypted MMIO descriptors from a TDISP Report. The descriptors allow the low-level TSM driver to inform ioremap() how to map the device's MMIO correctly based on the device's expectations. It also computes the obfuscated base addresses for each encrypted MMIO range.
Quoted 3 lines
drivers/pci/tsm/core.c +0
-0
[ ... ]
Quoted 2 lines
drivers/pci/tsm/core.c +2
-0+struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev) +{
[ ... ]
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 691
Quoted 8 lines
drivers/pci/tsm/core.c +8
-0+for (i =0 ; i < mmio_range_count; i++) { +u64 range_off; +struct range range; +const struct pci_tsm_devif_mmio *mmio_data = &report->mmio[i]; +struct pci_tsm_mmio_entry *entry = +pci_tsm_mmio_entry (mmio, mmio->nr); +u64 tsm_offset =__le64_to_cpu (mmio_data->phys); +u64 size =__le32_to_cpu (mmio_data->nr_pfns) *SZ_4K ;
[Severity: High]
Does this calculation suffer from a 32-bit multiplication overflow?
Since __le32_to_cpu() returns a 32-bit unsigned integer and SZ_4K is also 32-bit, if nr_pfns is large enough (representing 4GB or more), the multiplication will overflow before it is zero-extended and assigned to the 64-bit size variable.
Could this lead to the size being truncated (e.g., yielding 0 for exactly 4GB) and exposing intended secure MMIO space to the VM as unencrypted memory?
Quoted 3 lines
drivers/pci/tsm/core.c +3
-0+u32 attr =__le32_to_cpu (mmio_data->attributes); +int bar =FIELD_GET (PCI_TSM_DEVIF_REPORT_MMIO_ATTR_RANGE_ID , + attr);
[ ... ]
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 718
Quoted 11 lines
drivers/pci/tsm/core.c +11
-0+if (last_bar < bar) { +unsigned long mask =pci_resource_len (pdev, bar) -1 ; + +/* Transition to a new bar */ + last_bar = bar; + +/* +* Determine the obfuscated base of the BAR. BAR +* offsets are never obfuscated. +*/ + reporting_bar_base = tsm_offset & ~mask;
[Severity: Medium]
Will this bitwise operation truncate the upper 32 bits of the physical address on 32-bit architectures?
On 32-bit platforms with 64-bit physical addresses (like PAE), unsigned long is 32 bits. The mask truncates the BAR size, and ~mask flips those 32 bits. When it is promoted to u64 for the bitwise AND with tsm_offset, the upper 32 bits of the zero-extended ~mask will be 0.
Could this completely erase the upper 32 bits of reporting_bar_base when compiled for 32-bit platforms?
Quoted 1 line
drivers/pci/tsm/core.c +1
-0+ }else if (tsm_offset < last_reporting_end) {
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
Alexey Kardashevskiy
maintainerRe: [PATCH 14/15] PCI/TSM: Create MMIO descriptors via TDISP Report
In reply to Dan WilliamsQuoted 249 lines · Dan Williams
After
pci_tsm_bind()andpci_tsm_lock()the low level TSM driver is expected to populatePCI_TSM_EVIDENCE_TYPE_REPORTin its evidence store. This report is defined by the TDISPGET_DEVICE_INTERFACE_REPORTresponse payload.Add a helper to create encrypted MMIO descriptors from that report data. With those descriptors the TSM driver can use
pci_tsm_mmio_setup()to informioremap()how to map the device per the device's expectations. The VM is expected to validate the interface with the relying party before accepting the device for operation.The helper also provides the obfuscated starting address for each encrypted MMIO range as the VM is never disclosed on the hpa that correlates to the gpa of the device's mmio. The obfuscated address is BAR aligned.
Based on an original patch by Aneesh [1]
Cc Arnd Bergmann <[email protected]>Co-developed-by Xu Yilun <[email protected]>Signed-off-by Xu Yilun <[email protected]>Signed-off-by Dan Williams <[email protected]>--- include/linux/ioport.h | 2 + include/linux/pci-tsm.h | 35 +++++++ drivers/pci/tsm/core.c | 224 ++++++++++++++++++++++++++++++++++++++++ kernel/resource.c | 8 ++ 4 files changed, 269 insertions(+)include/linux/ioport.h +2
-0@@ -144,6 +144,7 @@ enum {IORES_DESC_RESERVED =7 ,IORES_DESC_SOFT_RESERVED =8 ,IORES_DESC_CXL =9 , + IORES_DESC_ENCRYPTED = 10, };/* @@ -240,6 +241,7 @@ struct resource_constraint {extern struct resource ioport_resource;extern struct resource iomem_resource;extern struct resource soft_reserve_resource; +extern struct resource encrypted_iomem_resource;extern struct resource *request_resource_conflict (struct resource *root,struct resource *new);extern int request_resource (struct resource *root,struct resource *new);include/linux/pci-tsm.h +35
-0@@ -156,12 +156,42 @@ struct pci_tsm_pf0 {struct pci_doe_mb *doe_mb; }; +/** +* struct pci_tsm_mmio_entry - an encrypted MMIO range +* @res: MMIO address range (typically Guest Physical Address, GPA) +* @tsm_offset: Host Physical Address, HPA obfuscation offset added by the TSM. +* Translates report addresses to GPA. +*/ +struct pci_tsm_mmio_entry { +struct resource res; +u64 tsm_offset; +}; + +struct pci_tsm_mmio { +int nr; +struct pci_tsm_mmio_entry mmio[]; +}; + +static inline struct pci_tsm_mmio_entry * +pci_tsm_mmio_entry (struct pci_tsm_mmio *mmio,int idx) +{ +return &mmio->mmio[idx]; +} + +static inline struct resource *pci_tsm_mmio_resource (struct pci_tsm_mmio *mmio, +int idx) +{ +return &mmio->mmio[idx].res; +} +/** * struct pci_tsm_devsec - context for tracking private/accepted PCI resources * @base_tsm: generic core "tsm" context +* @mmio: encrypted MMIO resources for this assigned device */ struct pci_tsm_devsec {struct pci_tsm base_tsm; +struct pci_tsm_mmio *mmio; };/* physical function0 and capable of 'connect' */ @@ -263,6 +293,11 @@ static inline const struct pci_tsm_ops *to_pci_tsm_ops(struct pci_tsm *tsm) return tsm->tsm_dev->pci_ops; }struct pci_tsm_devsec *to_pci_tsm_devsec (struct pci_tsm *tsm); +int pci_tsm_mmio_setup (struct pci_dev *pdev,struct pci_tsm_mmio *mmio); +void pci_tsm_mmio_teardown (struct pci_tsm_mmio *mmio); + +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev); +int pci_tsm_mmio_free (struct pci_dev *pdev,struct pci_tsm_mmio *mmio);#else static inline int pci_tsm_register (struct tsm_dev *tsm_dev) {drivers/pci/tsm/core.c +126
-0@@ -15,6 +15,7 @@#include <linux/pci-tsm.h> #include <linux/sysfs.h> #include <linux/tsm.h> +#include <linux/unaligned.h> #include <linux/xarray.h> #include "../pci.h" @@ -552,6 +553,229 @@ static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, }static DEVICE_ATTR_RO (dsm); +static void mmio_teardown (struct pci_tsm_mmio *mmio,int nr) +{ +while (nr--) +remove_resource (pci_tsm_mmio_resource (mmio, nr)); +} + +/** +* pci_tsm_mmio_setup() - mark device MMIO as encrypted in iomem +* @pdev: device owner of MMIO resources +* @mmio: container of an array of resources to mark encrypted +*/ +int pci_tsm_mmio_setup (struct pci_dev *pdev,struct pci_tsm_mmio *mmio) +{ +int i; + +device_lock_assert (&pdev->dev); +if (pdev->dev.driver) +return -EBUSY ; + +for (i =0 ; i < mmio->nr; i++) { +struct resource *res =pci_tsm_mmio_resource (mmio, i); +int j; + +if (resource_size (res) ==0 || !res->end) +break ; + +/* Only require the caller to set the range, init remainder */ + *res =DEFINE_RES_NAMED_DESC (res->start,resource_size (res), +pci_name (pdev),IORESOURCE_MEM , +IORES_DESC_ENCRYPTED ); + +for (j =0 ; j <PCI_NUM_RESOURCES ; j++) +if (resource_contains (pci_resource_n (pdev, j), res)) +break ; + +/* Request is outside of device MMIO */ +if (j >=PCI_NUM_RESOURCES ) +break ; + +if (insert_resource (&encrypted_iomem_resource, res) !=0 ) +break ; + } + +if (i >= mmio->nr) +return 0 ; + +mmio_teardown (mmio, i); + +return -EINVAL ; +} +EXPORT_SYMBOL_GPL (pci_tsm_mmio_setup); + +void pci_tsm_mmio_teardown (struct pci_tsm_mmio *mmio) +{ +mmio_teardown (mmio, mmio->nr); +} +EXPORT_SYMBOL_GPL (pci_tsm_mmio_teardown); + +/* +* PCIe ECN TEE Device Interface Security Protocol (TDISP) +* +* Device Interface Report data object layout as defined by PCIe r7.0 section +* 11.3.11 +*/ +#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_MSIX_TABLE BIT(0) +#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_MSIX_PBA BIT(1) +#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_IS_NON_TEE BIT(2) +#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_IS_UPDATABLE BIT(3) +#define PCI_TSM_DEVIF_REPORT_MMIO_ATTR_RANGE_ID GENMASK(31, 16) + +/* An interface report 'pfn' is 4K in size */ +struct pci_tsm_devif_mmio { +__le64 phys; +__le32 nr_pfns; +__le32 attributes; +}; + +struct pci_tsm_devif_report { +__le16 interface_info; +__le16 reserved; +__le16 msi_x_message_control; +__le16 lnr_control; +__le32 tph_control; +__le32 mmio_range_count; +struct pci_tsm_devif_mmio mmio[]; +}; + +/** +* pci_tsm_mmio_alloc() - allocate encrypted MMIO range descriptor +* @pdev: device owner of MMIO ranges +* +* Return: the encrypted MMIO range descriptor on success, NULL on failure +* +* Assumes that this is called within the live lifetime of a PCI device's +* association with a low level TSM. +*/ +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev) +{ +struct device_evidence *evidence = pdev->tsm->evidence; +u64 reporting_bar_base, last_reporting_end; +struct device_evidence_object *report_obj; +const struct pci_tsm_devif_report *report; +u32 mmio_range_count; +int last_bar =-1 ; +int i; + +if (!evidence) +return NULL ; + + report_obj = &evidence->obj[DEVICE_EVIDENCE_TYPE_REPORT ]; + +guard (rwsem_read)(&evidence->lock); +if (report_obj->len <sizeof (struct pci_tsm_devif_report )) +return NULL ; + + report = report_obj->data; + mmio_range_count =__le32_to_cpu (report->mmio_range_count); + +/* check that the report object is self-consistent on mmio entries */ +if (report_obj->len <struct_size (report, mmio, mmio_range_count)) +return NULL ; + +/* create pci_tsm_mmio descriptors from the report data */ + struct pci_tsm_mmio *mmio __free(kfree) = +kzalloc_flex (*mmio, mmio, mmio_range_count);
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 690
Quoted 10 lines
drivers/pci/tsm/core.c +10
-0+if (!mmio) +return NULL ; + +for (i =0 ; i < mmio_range_count; i++) { +u64 range_off; +struct range range; +const struct pci_tsm_devif_mmio *mmio_data = &report->mmio[i]; +struct pci_tsm_mmio_entry *entry = +pci_tsm_mmio_entry (mmio, mmio->nr); +u64 tsm_offset =__le64_to_cpu (mmio_data->phys);
* SZ_4K is missing.
Quoted 31 lines
drivers/pci/tsm/core.c +31
-0+u64 size =__le32_to_cpu (mmio_data->nr_pfns) *SZ_4K ; +u32 attr =__le32_to_cpu (mmio_data->attributes); +int bar =FIELD_GET (PCI_TSM_DEVIF_REPORT_MMIO_ATTR_RANGE_ID , + attr); + +if (bar >=PCI_STD_NUM_BARS || + !(pci_resource_flags (pdev, bar) &IORESOURCE_MEM ) || + (pci_resource_flags (pdev, bar) &IORESOURCE_UNSET )) { +pci_dbg (pdev,"Invalid reporting bar ID %d\n" , bar); +return NULL ; + } + +if (last_bar > bar) { +pci_dbg (pdev,"Reporting bar ID not in ascending order\n" ); +return NULL ; + } + +if (last_bar < bar) { +unsigned long mask =pci_resource_len (pdev, bar) -1 ; + +/* Transition to a new bar */ + last_bar = bar; + +/* +* Determine the obfuscated base of the BAR. BAR +* offsets are never obfuscated. +*/ + reporting_bar_base = tsm_offset & ~mask; + }else if (tsm_offset < last_reporting_end) { +pci_dbg (pdev,"Reporting ranges within BAR not in ascending order\n" ); +return NULL ;
drivers/pci/tsm/core.c static ssize_t dsm_show(struct device *dev, struct device_attribute *attr, · line 731
Quoted 10 lines
drivers/pci/tsm/core.c +10
-0+ } + +/* Per spec the tsm_offset never results in overflow / underflow */ + last_reporting_end = tsm_offset + size; +if (last_reporting_end < tsm_offset) { +pci_dbg (pdev,"Reporting range overflow\n" ); +return NULL ; + } + + range_off = tsm_offset - reporting_bar_base;
If we all do the mmio_reporting_offset in the same way (use the bar size low bits), then this can be simplified and should be as it does not account for the non-locked (==not reported) MSIX anyway. Thanks,
Quoted 69 lines
drivers/pci/tsm/core.c +47
-0+if (pci_resource_len (pdev, bar) < range_off + size) { +pci_dbg (pdev,"Reporting range larger than BAR size\n" ); +return NULL ; + } + + range.start =pci_resource_start (pdev, bar) + range_off; + range.end = range.start + size -1 ; + +/* Only record the TEE ranges for later consideration by ioremap() */ +if (FIELD_GET (PCI_TSM_DEVIF_REPORT_MMIO_ATTR_IS_NON_TEE , + attr)) { +pci_dbg (pdev,"Skipping non-TEE range, BAR%d %pra\n" , + bar, &range); +continue ; + } + + entry->res.start = range.start; + entry->res.end = range.end; + entry->tsm_offset = tsm_offset; + mmio->nr++; + } + +return_ptr (mmio); +} +EXPORT_SYMBOL_GPL (pci_tsm_mmio_alloc); + +/** +* pci_tsm_mmio_free() - free a pci_tsm_mmio instance +* @pdev: device owner of MMIO ranges +* @mmio: instance to free +* +* Returns 0 if @mmio was idle on entry, -EBUSY otherwise +*/ +int pci_tsm_mmio_free (struct pci_dev *pdev,struct pci_tsm_mmio *mmio) +{ +for (int i =0 ; i < mmio->nr; i++) { +struct resource *res =pci_tsm_mmio_resource (mmio, i); + +if (dev_WARN_ONCE (&pdev->dev,resource_assigned (res), +"MMIO resource still assigned %pr\n" , res)) +return -EBUSY ; + } +kfree (mmio); +return 0 ; +} +EXPORT_SYMBOL_GPL (pci_tsm_mmio_free); +/** * pci_tsm_accept() - accept a device for private MMIO operation * @pdev: PCI device to accept kernel/resource.c +8
-0@@ -56,6 +56,14 @@ struct resource soft_reserve_resource = { .flags = IORESOURCE_MEM, }; +struct resource encrypted_iomem_resource = { + .name ="Encrypted MMIO" , + .start =0 , + .end =-1 , + .desc =IORES_DESC_ENCRYPTED , + .flags =IORESOURCE_MEM , +}; +static DEFINE_RWLOCK (resource_lock);/*
Signature · 1 line
Alexey
PATCH 15/15 PCI/TSM: Add relative MMIO offset support?
In reply to Dan WilliamsThe RMM specification, DEN0137-2.0-bet2 section A9.6.2 "Realm validation of device memory mappings" documents the expectation that the MMIO_REPORTING_OFFSET chosen for TDISP Interface Reports is always BAR aligned.
Ideally this change is not needed and all implementations share the same expectation.
If this semantic is already shipping in production and/or the PCI-SIG clarifies that an implementation can hold this assumption then Linux will need to ask the TSM drivers for this hint.
--- include/linux/pci-tsm.h | 15 ++++++++++++++- drivers/pci/tsm/core.c | 14 ++++++++++---- 2 files changed, 24 insertions(+), 5 deletions(-)
include/linux/pci-tsm.h +14 -1
@@ -296,7 +296,20 @@ struct pci_tsm_devsec *to_pci_tsm_devsec(struct pci_tsm *tsm);int pci_tsm_mmio_setup (struct pci_dev *pdev,struct pci_tsm_mmio *mmio);void pci_tsm_mmio_teardown (struct pci_tsm_mmio *mmio); -struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev); +/** +* enum tdisp_offset_scheme - MMIO_REPORTING_OFFSET assumptions +* @TDISP_OFFSET_BAR_ALIGN: mask by bar size to recover offset +* @TDISP_OFFSET_RELATIVE: first mmio report per bar is bar-offset-0 +* +* A TSM driver may know that the default TDISP_OFFSET_BAR_ALIGN +* assumption is being violated. +*/ +enum tdisp_offset_scheme { +TDISP_OFFSET_BAR_ALIGN , +TDISP_OFFSET_RELATIVE , +}; +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev, +enum tdisp_offset_scheme scheme);int pci_tsm_mmio_free (struct pci_dev *pdev,struct pci_tsm_mmio *mmio);#else static inline int pci_tsm_register (struct tsm_dev *tsm_dev)
drivers/pci/tsm/core.c +10 -4
@@ -643,13 +643,15 @@ struct pci_tsm_devif_report {/** * pci_tsm_mmio_alloc() - allocate encrypted MMIO range descriptor * @pdev: device owner of MMIO ranges +* @scheme: allow the low level TSM driver to hint the offset calc scheme * * Return: the encrypted MMIO range descriptor on success, NULL on failure * * Assumes that this is called within the live lifetime of a PCI device's * association with a low level TSM. */ -struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev) +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev, +enum tdisp_offset_scheme scheme) {struct device_evidence *evidence = pdev->tsm->evidence;u64 reporting_bar_base, last_reporting_end;
@@ -712,10 +714,14 @@ struct pci_tsm_mmio *pci_tsm_mmio_alloc(struct pci_dev *pdev) last_bar = bar;/* -* Determine the obfuscated base of the BAR. BAR -* offsets are never obfuscated. +* Either the first range per bar always maps +* the start of the BAR, or the reporting_offset +* is BAR size aligned. */ - reporting_bar_base = tsm_offset & ~mask; +if (scheme ==TDISP_OFFSET_RELATIVE ) + reporting_bar_base = tsm_offset; +else + reporting_bar_base = tsm_offset & ~mask; }else if (tsm_offset < last_reporting_end) {pci_dbg (pdev,"Reporting ranges within BAR not in ascending order\n" );return NULL ;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH 15/15] PCI/TSM: Add relative MMIO offset support?
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] The BAR mask calculation in pci_tsm_mmio_alloc() uses a 32-bit unsigned long for mask on 32-bit architectures, corrupting 64-bit tsm_offset calculations.
- [High] A 32-bit integer overflow when calculating size in pci_tsm_mmio_alloc() silently truncates MMIO ranges of 4GB or larger.
--
commit 39c72d1b357cb53c2cb3c0f7292c8909846cff30
Author: Dan Williams <[email protected]>
PCI/TSM: Add relative MMIO offset support?
The RMM specification documents the expectation that the MMIO_REPORTING_OFFSET chosen for TDISP Interface Reports is always BAR aligned. This patch adds support to ask the TSM drivers for the offset calculation scheme if it deviates from this assumption.
drivers/pci/tsm/core.c struct pci_tsm_devif_report { · line 648
Quoted 11 lines
drivers/pci/tsm/core.c +1
-0@@ -643,13 +643,15 @@ struct pci_tsm_devif_report {/** * pci_tsm_mmio_alloc() - allocate encrypted MMIO range descriptor * @pdev: device owner of MMIO ranges +* @scheme: allow the low level TSM driver to hint the offset calc scheme * * Return: the encrypted MMIO range descriptor on success, NULL on failure
[Severity: High]
This is a pre-existing issue, but I noticed a potential 32-bit multiplication overflow in this function.
When calculating the size in pci_tsm_mmio_alloc():
u64 size = __le32_to_cpu(mmio_data->nr_pfns) * SZ_4K;
Since both operands are 32-bit, will this overflow if nr_pfns is 1,048,576 (representing 4GB) or greater? The overflow would occur before being promoted and assigned to the 64-bit size variable, causing MMIO ranges of 4GB or larger to be silently truncated.
Quoted 8 lines
drivers/pci/tsm/core.c +2
-1* * Assumes that this is called within the live lifetime of a PCI device's * association with a low level TSM. */ -struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev) +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev, +enum tdisp_offset_scheme scheme) {
[ ... ]
Quoted 5 lines
drivers/pci/tsm/core.c +0
-1@@ -712,10 +714,14 @@ struct pci_tsm_mmio *pci_tsm_mmio_alloc(struct pci_dev *pdev) last_bar = bar;/* -* Determine the obfuscated base of the BAR. BAR
drivers/pci/tsm/core.c struct pci_tsm_mmio *pci_tsm_mmio_alloc(struct pci_dev *pdev) · line 724
Quoted 10 lines
drivers/pci/tsm/core.c +7
-2-* offsets are never obfuscated. +* Either the first range per bar always maps +* the start of the BAR, or the reporting_offset +* is BAR size aligned. */ - reporting_bar_base = tsm_offset & ~mask; +if (scheme ==TDISP_OFFSET_RELATIVE ) + reporting_bar_base = tsm_offset; +else + reporting_bar_base = tsm_offset & ~mask;
[Severity: High]
This is a pre-existing issue, but could this bitwise AND corrupt 64-bit calculations on 32-bit architectures?
Looking at the definition of mask earlier in pci_tsm_mmio_alloc():
unsigned long mask = pci_resource_len(pdev, bar) - 1;
Since pci_resource_len() returns a 64-bit resource_size_t on PAE systems, it gets truncated to a 32-bit unsigned long mask on 32-bit architectures. When ~mask is evaluated, it results in a 32-bit value.
Would applying bitwise AND with the 64-bit u64 tsm_offset here cause zero-extension of ~mask, explicitly clearing all bits above 4GB in tsm_offset?
Quoted 1 line
drivers/pci/tsm/core.c +0
-0}else if (tsm_offset < last_reporting_end) {
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
Alexey Kardashevskiy
maintainerRe: [PATCH 15/15] PCI/TSM: Add relative MMIO offset support?
In reply to Dan WilliamsQuoted 11 lines · Dan Williams
The RMM specification, DEN0137-2.0-bet2 section A9.6.2 "Realm validation of device memory mappings" documents the expectation that the
MMIO_REPORTING_OFFSETchosen for TDISP Interface Reports is always BAR aligned.Ideally this change is not needed and all implementations share the same expectation.
If this semantic is already shipping in production and/or the PCI-SIG clarifies that an implementation can hold this assumption then Linux will need to ask the TSM drivers for this hint.
We are changing it on SEV-TIO such that the host os calculates the offset to allow TDISP_OFFSET_BAR_ALIGN and passes it to the PSP during TDI_BIND (==interface start), others did not need it in the first place so I guess we can drop this one. Thanks,
Quoted 74 lines
Cc Alexey Kardashevskiy <[email protected]>Cc Xu Yilun <[email protected]>Cc "Aneesh Kumar K.V" <[email protected]>Signed-off-by Dan Williams <[email protected]>--- include/linux/pci-tsm.h | 15 ++++++++++++++- drivers/pci/tsm/core.c | 14 ++++++++++---- 2 files changed, 24 insertions(+), 5 deletions(-)include/linux/pci-tsm.h +14
-1@@ -296,7 +296,20 @@ struct pci_tsm_devsec *to_pci_tsm_devsec(struct pci_tsm *tsm);int pci_tsm_mmio_setup (struct pci_dev *pdev,struct pci_tsm_mmio *mmio);void pci_tsm_mmio_teardown (struct pci_tsm_mmio *mmio); -struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev); +/** +* enum tdisp_offset_scheme - MMIO_REPORTING_OFFSET assumptions +* @TDISP_OFFSET_BAR_ALIGN: mask by bar size to recover offset +* @TDISP_OFFSET_RELATIVE: first mmio report per bar is bar-offset-0 +* +* A TSM driver may know that the default TDISP_OFFSET_BAR_ALIGN +* assumption is being violated. +*/ +enum tdisp_offset_scheme { +TDISP_OFFSET_BAR_ALIGN , +TDISP_OFFSET_RELATIVE , +}; +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev, +enum tdisp_offset_scheme scheme);int pci_tsm_mmio_free (struct pci_dev *pdev,struct pci_tsm_mmio *mmio);#else static inline int pci_tsm_register (struct tsm_dev *tsm_dev)drivers/pci/tsm/core.c +10
-4@@ -643,13 +643,15 @@ struct pci_tsm_devif_report {/** * pci_tsm_mmio_alloc() - allocate encrypted MMIO range descriptor * @pdev: device owner of MMIO ranges +* @scheme: allow the low level TSM driver to hint the offset calc scheme * * Return: the encrypted MMIO range descriptor on success, NULL on failure * * Assumes that this is called within the live lifetime of a PCI device's * association with a low level TSM. */ -struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev) +struct pci_tsm_mmio *pci_tsm_mmio_alloc (struct pci_dev *pdev, +enum tdisp_offset_scheme scheme) {struct device_evidence *evidence = pdev->tsm->evidence;u64 reporting_bar_base, last_reporting_end;@@ -712,10 +714,14 @@ struct pci_tsm_mmio *pci_tsm_mmio_alloc(struct pci_dev *pdev) last_bar = bar;/* -* Determine the obfuscated base of the BAR. BAR -* offsets are never obfuscated. +* Either the first range per bar always maps +* the start of the BAR, or the reporting_offset +* is BAR size aligned. */ - reporting_bar_base = tsm_offset & ~mask; +if (scheme ==TDISP_OFFSET_RELATIVE ) + reporting_bar_base = tsm_offset; +else + reporting_bar_base = tsm_offset & ~mask; }else if (tsm_offset < last_reporting_end) {pci_dbg (pdev,"Reporting ranges within BAR not in ascending order\n" );return NULL ;
Signature · 1 line
Alexey
Re: [PATCH 15/15] PCI/TSM: Add relative MMIO offset support?
Quoted 9 lines · Alexey Kardashevskiy
Quoted 8 lines · Dan Williams
The RMM specification, DEN0137-2.0-bet2 section A9.6.2 "Realm validation of device memory mappings" documents the expectation that the
MMIO_REPORTING_OFFSETchosen for TDISP Interface Reports is always BAR aligned.Ideally this change is not needed and all implementations share the same expectation.
Quoted 8 lines
Quoted 3 lines
If this semantic is already shipping in production and/or the PCI-SIG clarifies that an implementation can hold this assumption then Linux will need to ask the TSM drivers for this hint.
We are changing it on SEV-TIO such that the host os calculates the offset to allow
TDISP_OFFSET_BAR_ALIGNand passes it to the PSP duringTDI_BIND(==interface start), others did not need it in the first place so I guess we can drop this one. Thanks,
Ok, yes, then this one can be dropped.
Quoted 10 lines · Dan Williams
* NONE: no usage of the device unless the trust is explicitly overridden by user policy specified via a driver flag, module flag, or uapi (TBD).
* ADVERSARY: needs acknowledgement from the bus and IOMMU / DMA layers that the device is limited to strict IOMMU translation behavior. Drivers can use this as a signal to limit functionality. This designation implies follow-on IOMMU and bus enabling work for features like arranging for the device to attach to a blocked IOMMU domain when detached from a driver.
Quoted 6 lines
* AUTO: typical / historical Linux driver model.
* TCB: a trust level that only exists in Confidential Computing environments. When acked by the IOMMU / DMA layer it enables the device to issue direct-DMA to private/encrypted addresses or otherwise attach to a secure vIOMMU within the TCB.
I'm not sure I entirely like this one, certainly it needs to be possible to have both T=1 and ADVERSARY together.
Arguably the T=0/1 decision is much more like link encryption, it effects the transport of the DMA into the CPU. That it also impacts how the VIOMMU works is the only thing that make it sort of trust-like in this model.
I'd also argue this list is missing "FULL" trust, which is the historical Linux behavior for a normal device. AUTO should be selecting between FULL/ADVERSARY based on things like the ACPI/etc as it does today.
In a CC VM if we have a T=0 device we probably want to operate it with ADVERSARY (there is no T=0 VIOMMU so this is equivilent to FULL)
For a T=1 device we need to have the choice of FULL or ADVERSARY. When a VIOMMU is present ADVERSARY will further restrict the T=1 traffic as a defense in depth.
So, it is really necessary to have "TCB" here?
If the trust level is reduced to just be a command to the kernel how it should operate the device then it would be up to userspace to confirm things like T=1 before setting the trust. The kernel would have to set the trust to NONE when security sensitive changes are detected.
If we still want a kernel-side policy gate, eg kernel will not progress unless T=1 is met, then I think that would be better as an independent pair'd policy field, ie
trust=FULL when policy=T=1, LINK_ENCRYPTION=1, etc
As bundling the two into "TCB" is probably going to turn problematic as we have more and more combinations of policy conditions.
Jason
Quoted 13 lines · Jason Gunthorpe
Quoted 12 lines · Dan Williams
* NONE: no usage of the device unless the trust is explicitly overridden by user policy specified via a driver flag, module flag, or uapi (TBD).
* ADVERSARY: needs acknowledgement from the bus and IOMMU / DMA layers that the device is limited to strict IOMMU translation behavior. Drivers can use this as a signal to limit functionality. This designation implies follow-on IOMMU and bus enabling work for features like arranging for the device to attach to a blocked IOMMU domain when detached from a driver.
* AUTO: typical / historical Linux driver model.
Quoted 7 lines
Quoted 4 lines
* TCB: a trust level that only exists in Confidential Computing environments. When acked by the IOMMU / DMA layer it enables the device to issue direct-DMA to private/encrypted addresses or otherwise attach to a secure vIOMMU within the TCB.
I'm not sure I entirely like this one, certainly it needs to be possible to have both T=1 and ADVERSARY together.
T=1 and ADVERSARY are independent for link encryption and private MMIO. In other words the device is placed into the TDISP RUN state independent of its trust level. Downstream accesses to the device must have T=1, and its upstream accesses will have T=1, but with force_dma_unencrypted() == true.
The only combination missing is T=1 with force_dma_unencrypted() == false && VIOMMU enforcing IOMMU_DOMAIN_DMA.
Quoted 9 lines
Arguably the T=0/1 decision is much more like link encryption, it effects the transport of the DMA into the CPU. That it also impacts how the VIOMMU works is the only thing that make it sort of trust-like in this model.
I'd also argue this list is missing "FULL" trust, which is the historical Linux behavior for a normal device. AUTO should be selecting between FULL/ADVERSARY based on things like the ACPI/etc as it does today.
Two notes,
1/ that is effectively how the UNSET level behaves. If the bus has not set ADVERSARY before device_add() then the default behavior is the AUTO level. Where AUTO means all of the automatic privileges a device can be offered without needing any other coordination.
2/ The ambiguity and conflict occurs at ->dma_configure() time when the bus and IOMMU layer want to reject the device's access to some privilege by failing. When FULL is defined as !ADVERSARY then it is difficult to describe the semantics when FULL trust honors rejections to private DMA and when it falls back to shared operation.
The above more points to a need to have an explicit trust level for adversarial private memory access. The address spaces are distinct assets with different levels of trust.
UNSET: bus picks initial level, or leaves it to the device_core(). NONE:
ADVERSARY: Device can be in T=0, or T=1 mode (UNLOCKED, or RUN).
AUTO: Could rename this to be FULL or ALL or DEFAULT, I still keep
coming back to the "AUTO" name because the privileges are not uniform based on the IOMMU / DMA topology and device capability. Again, the TDISP state is independent. The TSM driver does not get called to gatekeep and verify access in this mode.TCB_ADVERSARY: or PRIVATE_ADVERSARY. Device can access private platform
resources iff an enforcing IOMMU is present.
TCB: or PRIVATE_FULL, automatically enable all access privileges
including private memory access.
Direct-dma to private memory without a Secure VIOMMU present requires "TCB".
Quoted 6 lines
In a CC VM if we have a T=0 device we probably want to operate it with ADVERSARY (there is no T=0 VIOMMU so this is equivilent to FULL)
For a T=1 device we need to have the choice of FULL or ADVERSARY. When a VIOMMU is present ADVERSARY will further restrict the T=1 traffic as a defense in depth.
Yes, a mode that can setup IOMMU_DOMAIN_DMA for private memory makes sense. It needs a new trust level to distinguish if the device is trusted to access one or both address spaces.
Quoted 1 line
So, it is really necessary to have "TCB" here?
I think it is important to distinguish that the CC case adds another class of memory to the trust decision.
Quoted 3 lines
If the trust level is reduced to just be a command to the kernel how it should operate the device then it would be up to userspace to confirm things like T=1 before setting the trust.
This discussion gets strained for me when T=1 is used to mean both "device is in TDISP RUN (with link encryption and private MMIO)" and "device is in TDISP RUN + force_dma_unencrypted() == false".
As implemented, userspace can confirm the former before setting the trust. The latter only gets confirmed / denied at ->dma_configure() time, because that is the point at which driver policy has been applied to set the operational trust level.
Otherwise, full bi-directional T=1 before setting the trust would require an IOMMU to be blocking the device until that final confirmation point. Given that is not always available the proposal is to defer acknowledging the trust level with the TSM until ->dma_configure().
As far as userspace is concerned it has enabled T=1 at the device and validated device evidence. Any failures to operate the device from that point are either the TSM finding fault with the evidence the CC VM is relying upon, or, in the new TCB_ADVERSARY case, the IOMMU layer says, "no IOMMU present to enforce IOMMU_DOMAIN_DMA".
Quoted 10 lines
The kernel would have to set the trust to NONE when security sensitive changes are detected.
If we still want a kernel-side policy gate, eg kernel will not progress unless T=1 is met, then I think that would be better as an independent pair'd policy field, ie
trust=FULL when policy=T=1, LINK_ENCRYPTION=1, etcAs bundling the two into "TCB" is probably going to turn problematic as we have more and more combinations of policy conditions.
Agree that "TCB" alone is insufficient per above. However, a new "trust when/if" ABI does not appear to scale.
Userspace policy simply needs to contend with the mechanisms that "Device Core, BUS, IOMMU or TSM driver may fail to enable a device at a given trust level".
NONE: Device core rejects device operation
ADVERSARY: reject device operation if an IOMMU to set IOMMU_DOMAIN_DMA
not available (not in current patches)
AUTO: no rejections, but no private memory access eitherTCB_ADVERSARY: reject device operation if IOMMU_DOMAIN_DMA not
available, or TSM rejects the evidence used to enable private memory access.
TCB: reject device operation if TSM rejects chosen evidence
Overrides and bypasses for those rejection decisions belong with their respective drivers. For example, if a paravisor is handling TDISP then there is no TSM driver to reject TCB operation. Similarly ->dma_configure() when no IOMMU is present is a nop today. New enabling could turn to enforcing, or userspace policy otherwise understands the risks.
Quoted 19 lines · Dan Williams (nvidia)
Quoted 18 lines · Jason Gunthorpe
Quoted 16 lines · Dan Williams
* NONE: no usage of the device unless the trust is explicitly overridden by user policy specified via a driver flag, module flag, or uapi (TBD).
* ADVERSARY: needs acknowledgement from the bus and IOMMU / DMA layers that the device is limited to strict IOMMU translation behavior. Drivers can use this as a signal to limit functionality. This designation implies follow-on IOMMU and bus enabling work for features like arranging for the device to attach to a blocked IOMMU domain when detached from a driver.
* AUTO: typical / historical Linux driver model.
* TCB: a trust level that only exists in Confidential Computing environments. When acked by the IOMMU / DMA layer it enables the device to issue direct-DMA to private/encrypted addresses or otherwise attach to a secure vIOMMU within the TCB.
Quoted 6 lines
Quoted 2 lines
I'm not sure I entirely like this one, certainly it needs to be possible to have both T=1 and ADVERSARY together.
T=1 and ADVERSARY are independent for link encryption and private MMIO. In other words the device is placed into the TDISP RUN state independent of its trust level.
That's the right thing
Quoted 3 lines
Downstream accesses to the device must have T=1, and
its upstream accesses will have T=1, but withforce_dma_unencrypted()== true.
That should never happen. Once in RUN force_dma_unencrypyted() == false, it has nothing to do with the trust level.
Even if you set ADVERSARY it should still be bouncing partial page DMAs into private memory. The point of running something like this is to remove the shared memory attack surface - ie the hypervisor SW. The attack surface is reduced to the device itself by remaining in shared memory.
Quoted 9 lines
Quoted 4 lines
I'd also argue this list is missing "FULL" trust, which is the historical Linux behavior for a normal device. AUTO should be selecting between FULL/ADVERSARY based on things like the ACPI/etc as it does today.
1/ that is effectively how the UNSET level behaves. If the bus has not set ADVERSARY before
device_add()then the default behavior is the AUTO level. Where AUTO means all of the automatic privileges a device can be offered without needing any other coordination.
I think my other remark about two enums is some of the issue, the policy can have things like UNSET or AUTO, but once the driver starts to probe an in-effect mode should be computed and be concrete. Having a driver run with a trust mode of AUTO or UNSET is just confusing.
Quoted 5 lines
2/ The ambiguity and conflict occurs at ->dma_configure() time when the bus and IOMMU layer want to reject the device's access to some privilege by failing. When FULL is defined as !ADVERSARY then it is difficult to describe the semantics when FULL trust honors rejections to private DMA and when it falls back to shared operation.
Given that the trust level shouldn't impact force_dma_unencrypted(), the only thing left is to setup the IOMMU differently, and maybe operate a driver in a hardened mode or something like that.
I don't see what the TCB is supposed to be changing here.
That leaves it just as a policy gate to check that T=1, I'm not sure if that is worthwhile enough for dedicated UAPI?
Quoted 4 lines
The above more points to a need to have an explicit trust level for adversarial private memory access. The address spaces are distinct assets with different levels of trust.
Quoted 12 lines
UNSET: bus picks initial level, or leaves it to the
device_core(). NONE:
ADVERSARY: Device can be in T=0, or T=1 mode (UNLOCKED, or RUN).
AUTO: Could rename this to be FULL or ALL or DEFAULT, I still keep
coming back to the "AUTO" name because the privileges are not uniform based on the IOMMU / DMA topology and device capability. Again, the TDISP state is independent. The TSM driver does not get called to gatekeep and verify access in this mode.TCB_ADVERSARY: orPRIVATE_ADVERSARY. Device can access private platform
resources iff an enforcing IOMMU is present.
TCB: orPRIVATE_FULL, automatically enable all access privileges
including private memory access.
Yeah, we can keep adding more modes to make a big cross product, but I do wonder if this is going to get too big..
IDK, maybe it should be a bitmap instead of a level?
bit 0 = Force Disable
bit 1 = Device is adversarial:
- Enable strong IOMMU protections
- Enable driver protections
bit 2 = Require T=1
bit 3 = Require IOMMU
bit 4 = Require DMA/MMIO security (eg Link IDE)
Where value 0 means the current level of full trust.
It is a little easier to explain what each thing is doing and easier to add new things
Then from a sysfs perpsective the policy would have special string values like 'use bus default'
Quoted 7 lines
Quoted 3 lines
If the trust level is reduced to just be a command to the kernel how it should operate the device then it would be up to userspace to confirm things like T=1 before setting the trust.
This discussion gets strained for me when T=1 is used to mean both "device is in TDISP RUN (with link encryption and private MMIO)" and "device is in TDISP RUN +
force_dma_unencrypted()== false".
It means both things though, we really must not run with force_dma_unencrypted() = true when T=1, that's pointless and harmful.
Quoted 4 lines
Otherwise, full bi-directional T=1 before setting the trust would require an IOMMU to be blocking the device until that final confirmation point. Given that is not always available the proposal is to defer acknowledging the trust level with the TSM until ->dma_configure().
?? If you have no iommu the instance you set T=1 and do the platform step to activate DMA the device has 100% acess to all memory. force_dma_unencrypted() does nothing to constrain device access, it is all about accommodating an addressing limitation.
ARM at least has a dedicated call to enable DMA. It would be nice to place that call right before the driver probes so DMA remains off until we commit to using the device. Maybe other platforms have the same - but I'm not sure it is *essential* as the point of setting RUN can reasonably be the acceptance point.
Quoted 8 lines
NONE: Device core rejects device operation
ADVERSARY: reject device operation if an IOMMU to setIOMMU_DOMAIN_DMA
not available (not in current patches)
AUTO: no rejections, but no private memory access eitherTCB_ADVERSARY: reject device operation ifIOMMU_DOMAIN_DMAnot
available, or TSM rejects the evidence used to enable private memory access.
TCB: reject device operation if TSM rejects chosen evidence
What does "TSM rejectes chosen evidence" mean? Kernel isn't supposed to be evaluating evidence?
Jason
Quoted 27 lines · Jason Gunthorpe
Quoted 19 lines · Dan Williams (nvidia)
Quoted 18 lines · Jason Gunthorpe
Quoted 16 lines · Dan Williams
* NONE: no usage of the device unless the trust is explicitly overridden by user policy specified via a driver flag, module flag, or uapi (TBD).
* ADVERSARY: needs acknowledgement from the bus and IOMMU / DMA layers that the device is limited to strict IOMMU translation behavior. Drivers can use this as a signal to limit functionality. This designation implies follow-on IOMMU and bus enabling work for features like arranging for the device to attach to a blocked IOMMU domain when detached from a driver.
* AUTO: typical / historical Linux driver model.
* TCB: a trust level that only exists in Confidential Computing environments. When acked by the IOMMU / DMA layer it enables the device to issue direct-DMA to private/encrypted addresses or otherwise attach to a secure vIOMMU within the TCB.
Quoted 6 lines
Quoted 2 lines
I'm not sure I entirely like this one, certainly it needs to be possible to have both T=1 and ADVERSARY together.
T=1 and ADVERSARY are independent for link encryption and private MMIO. In other words the device is placed into the TDISP RUN state independent of its trust level.
Quoted 8 lines
That's the right thing
Quoted 3 lines
Downstream accesses to the device must have T=1, and
its upstream accesses will have T=1, but withforce_dma_unencrypted()== true.That should never happen. Once in RUN
force_dma_unencrypyted()== false, it has nothing to do with the trust level.
2 concerns:
1/ We previously discussed a use case to operate a device's private MMIO while not allowing access to private memory (software encrypted NVME with private MMIO [1]). Many of the following comments are based on preserving this assumption so you can save some reading if we agree that use case can be abandoned.
[1]: https://lore.kernel.org/all/[email protected]/
If the private-MMIO-no-private-DMA case exists then ->dma_configure() needs a "trusted to access private-DMA" signal not a "required to access private-DMA" signal to determine how to set force_dma_unencrypyted() while the device is in the RUN state. I am ok to reconsider that use case, but that was part of what lead to the proposal for force_dma_unencrypyted() to use the trust level.
2/ The confirmation of the trust level and the enabling of DMA are separated in time from setting the trust level and entering the RUN state.
All the archs separate the RUN step from the ENABLE DMA step, and the implementation separates those steps in time.
echo tsm0 > $pdev/tsm/lock
cat $nonce | device-evidence dump $pdev device-evidence validate $pdev $generation
echo full > $pdev/trust
echo 1 > $pdev/tsm/accept
<--- T=1 and force_dma_unencrypted() remains true in this window echo $pdev > $driver/bind
The configuration window where $pdev/trust and $pdev/accept can be dynamically changing should not be changing the force_dma_unencrypyted() result if only because that value will mismatch the hardware state until the next ->dma_configure() event.
Another factor that lead to the assumption that "operational trust level configures force_dma_unencrypted()" is taken from a comment from Greg [2].
There are also buses and paravisors that may know that private-DMA is enabled for a device by construction. In that case it is also a "trusted to access" signal, and not a "required to access" signal.
My answer, given the concerns of drivers dangerously flipping the force_dma_unencrypted() result at runtime was to place it in 'struct device_private' and make private memory access a result of the trust level. Otherwise it is confusing when 2 devices are at FULL trust, but one has private memory access and the other does not. One is FULL the other is FULL+.
[2]: http://lore.kernel.org/2026031230-mastiff-create-7593@gregkh
Quoted 5 lines
Even if you set ADVERSARY it should still be bouncing partial page DMAs into private memory. The point of running something like this is to remove the shared memory attack surface - ie the hypervisor SW. The attack surface is reduced to the device itself by remaining in shared memory.
Yes.
Quoted 11 lines
Quoted 10 lines
Quoted 5 lines
I'd also argue this list is missing "FULL" trust, which is the historical Linux behavior for a normal device. AUTO should be selecting between FULL/ADVERSARY based on things like the ACPI/etc as it does today.
1/ that is effectively how the UNSET level behaves. If the bus has not set ADVERSARY before
device_add()then the default behavior is the AUTO level. Where AUTO means all of the automatic privileges a device can be offered without needing any other coordination.
Quoted 4 lines
I think my other remark about two enums is some of the issue, the policy can have things like UNSET or AUTO, but once the driver starts to probe an in-effect mode should be computed and be concrete. Having a driver run with a trust mode of AUTO or UNSET is just confusing.
UNSET never escapes to something a driver would see, but yes, appears AUTO is too confusing a term compared to FULL.
Quoted 6 lines
Quoted 5 lines
2/ The ambiguity and conflict occurs at ->dma_configure() time when the bus and IOMMU layer want to reject the device's access to some privilege by failing. When FULL is defined as !ADVERSARY then it is difficult to describe the semantics when FULL trust honors rejections to private DMA and when it falls back to shared operation.
Quoted 8 lines
Given that the trust level shouldn't impact
force_dma_unencrypted(), the only thing left is to setup the IOMMU differently, and maybe operate a driver in a hardened mode or something like that.I don't see what the TCB is supposed to be changing here.
That leaves it just as a policy gate to check that T=1, I'm not sure if that is worthwhile enough for dedicated UAPI?
If we ditch the private-MMIO-no-private-DMA case then yes I can see T=1 being a uniform requirement.
Quoted 17 lines
Quoted 4 lines
The above more points to a need to have an explicit trust level for adversarial private memory access. The address spaces are distinct assets with different levels of trust.
Quoted 12 lines
UNSET: bus picks initial level, or leaves it to the
device_core(). NONE:
ADVERSARY: Device can be in T=0, or T=1 mode (UNLOCKED, or RUN).
AUTO: Could rename this to be FULL or ALL or DEFAULT, I still keep
coming back to the "AUTO" name because the privileges are not uniform based on the IOMMU / DMA topology and device capability. Again, the TDISP state is independent. The TSM driver does not get called to gatekeep and verify access in this mode.TCB_ADVERSARY: orPRIVATE_ADVERSARY. Device can access private platform
resources iff an enforcing IOMMU is present.
TCB: orPRIVATE_FULL, automatically enable all access privileges
including private memory access.
Quoted 2 lines
Yeah, we can keep adding more modes to make a big cross product, but I do wonder if this is going to get too big..
The bus specific requirements like private MMIO + link encryption are pushed out to the TSM driver. Userspace policy that wants to require that the TSM driver is consulted on enable private DMA events needs to arrange for it to be loaded otherwise ->dma_configure() can just proceed with what the bus pre-configured.
Quoted 1 line
IDK, maybe it should be a bitmap instead of a level?
...not sure
Quoted 8 lines
bit 0 = Force Disable
bit 1 = Device is adversarial:
- Enable strong IOMMU protections
- Enable driver protections
bit 2 = Require T=1
bit 3 = Require IOMMU
bit 4 = Require DMA/MMIO security (eg Link IDE)
Quoted 4 lines
Where value 0 means the current level of full trust.
It is a little easier to explain what each thing is doing and easier to add new things
The question is whether these requirements belong on a central device trust mechanism bitmap or should be pushed out to other ABI. For example, if the presence of the secure IOMMU is enumerated after the device is in the LOCKED state then userspace policy can know what it is getting into without needing to tell the kernel. If T=1 is required then either a TSM driver is present to validate it can enable DMA, or the bus/paravisor arranges for that T=1 capability to be pre-enabled.
Perhaps the levels are indeed:
DISABLED
ADVERSARY
FULL
...with a T=1 flag on the side that leaf drivers can not set on their own.
Quoted 7 lines
Then from a sysfs perpsective the policy would have special string values like 'use bus default'
Quoted 4 lines
Quoted 3 lines
If the trust level is reduced to just be a command to the kernel how it should operate the device then it would be up to userspace to confirm things like T=1 before setting the trust.
Quoted 6 lines
Quoted 3 lines
This discussion gets strained for me when T=1 is used to mean both "device is in TDISP RUN (with link encryption and private MMIO)" and "device is in TDISP RUN +
force_dma_unencrypted()== false".It means both things though, we really must not run with
force_dma_unencrypted()= true when T=1, that's pointless and harmful.
Like I said above, I am ok to ditch the consideration of the private-MMIO-no-private-DMA case.
Quoted 9 lines
Quoted 4 lines
Otherwise, full bi-directional T=1 before setting the trust would require an IOMMU to be blocking the device until that final confirmation point. Given that is not always available the proposal is to defer acknowledging the trust level with the TSM until ->dma_configure().
?? If you have no iommu the instance you set T=1 and do the platform step to activate DMA the device has 100% acess to all memory.
force_dma_unencrypted()does nothing to constrain device access, it is all about accommodating an addressing limitation.
Right, this is my point above about why "echo 1 > $pdev/accept" does not immediately set force_dma_unencrypted to false nor enable private DMA.
Quoted 5 lines
ARM at least has a dedicated call to enable DMA. It would be nice to place that call right before the driver probes so DMA remains off until we commit to using the device. Maybe other platforms have the same - but I'm not sure it is *essential* as the point of setting RUN can reasonably be the acceptance point.
Not essential, but it is useful to have the "operate device" intent, IOMMU default domain configuration, trust level, and private DMA enable all in the same bus operation (->dma_configure). Which is why "accept" was demoted to just set RUN state and leave the rest to be finalized later.
Quoted 11 lines
Quoted 8 lines
NONE: Device core rejects device operation
ADVERSARY: reject device operation if an IOMMU to setIOMMU_DOMAIN_DMA
not available (not in current patches)
AUTO: no rejections, but no private memory access eitherTCB_ADVERSARY: reject device operation ifIOMMU_DOMAIN_DMAnot
available, or TSM rejects the evidence used to enable private memory access.
TCB: reject device operation if TSM rejects chosen evidenceWhat does "TSM rejectes chosen evidence" mean? Kernel isn't supposed to be evaluating evidence?
The host / platform TSM is protecting the VM against evidence that has changed since the last time the VM asked. I.e. the TSM is tracking if the untrusted VMM relocked the device without the VM having re-read the evidence.
Quoted 5 lines · Dan Williams (nvidia)
1/ We previously discussed a use case to operate a device's private MMIO while not allowing access to private memory (software encrypted NVME with private MMIO [1]). Many of the following comments are based on preserving this assumption so you can save some reading if we agree that use case can be abandoned.
force_dma_unencrypted() does not *prevent* device access to private memory and provides no security properties on its own. It's only purpose is to inform the DMA API what the HW restrictions are for doing DMA.
The use case I had in mind relied on the vIOMMU to police access to the private memory. I think if you do T=1 and ADVERSARY you must have a vIOMMU or it is an illogical configuration.
So, IMHO there is no use case for a T=1 device to only use shared memory, I would abandon that combination.
Quoted 6 lines
2/ The confirmation of the trust level and the enabling of DMA are separated in time from setting the trust level and entering the RUN state.
All the archs separate the RUN step from the ENABLE DMA step, and the implementation separates those steps in time.
Yes, until the device is fully accepted the DMA has to be kept off. If you have to measure in the RUN state then we must have a second step after measurement completes, however we are measuring in LOCKED right?
Quoted 3 lines
echo tsm0 > $pdev/tsm/lock
cat $nonce | device-evidence dump $pdev
device-evidence validate $pdev $generation
So the device is in LOCKED here, DMA remains off both because of LOCKED and becuase ENABLE DMA has not been done.
Quoted 1 line
echo 1 > $pdev/tsm/accept
And now it is RUN. So I don't see the issue with enabling DMA at the same time as gonig to RUN? (though defering it to driver probe would be a very nice touch as well)
With this API you can't set the devices to RUN and *then* collect the evidence, is that restriction OK?
Quoted 2 lines
echo full > $pdev/trust
echo $pdev > $driver/bind
Then we can set the trust and bind the driver, knowing that the T=1 security property is in place.
Quoted 4 lines
The configuration window where $pdev/trust and $pdev/accept can be dynamically changing should not be changing the
force_dma_unencrypyted()result if only because that value will mismatch the hardware state until the next ->dma_configure() event.
T=x cannot be changed while a driver is bound, we need to block that. Changing it will mess up all DMA and the DMA API won't work anymore.
Thus force_dma_unencrypted() can only change when a driver is unbound.
If a driver is unbound nothing ever reads force_dma_unencrypted() so it is safe to change it, thus we can just synchronize T=1 and force_dma_unencrypted() whenever T changes.
The error case has to wait for the driver to unbind if userspace wants to go from ERROR->UNLOCKED. IMHO for RAS we are going to need a flow to logically go from ERROR->RUN, probably with the 'same device acceptance' I suggested earlier. In this case the DMAs will be halted, the devices gets back to T=1 and RUN, then the DMAs are turned on again. The driver remains in T=1 and force_dma_unencrypted() is always false.
Quoted 3 lines
There are also buses and paravisors that may know that private-DMA is enabled for a device by construction. In that case it is also a "trusted to access" signal, and not a "required to access" signal.
In this case they wire force_dma_unencrypted()=false.
Quoted 3 lines
My answer, given the concerns of drivers dangerously flipping the
force_dma_unencrypted()result at runtime was to place it in 'structdevice_private'
Yes, this seems good idea in general
Quoted 3 lines
Otherwise it is confusing when 2 devices are at FULL trust, but one has private memory access and the other does not. One is FULL the other is FULL+.
It is confusing when phrased like this, but this is also why I was suggesting below that a pure trust level may not be sufficient as we have quite a cross product of scenarios.
If the user wants to operate a device without T=1 and without ADVERSARY then that's goofy, but technically the kernel doesn't really need to care? It can do it.
IMHO a big question is how much do we want the kernel to nanny userspace by having options to double check its work?
Either we trust the user to have checked all the policy conditions before writing /sys/trust, or we encode some policy conditions and tell the kernel so it can double-check prior to probe.
At least if we omit the double check it can be fairly easy to add in later if it really was needed for some time of use reason.
Quoted 2 lines
The question is whether these requirements belong on a central device trust mechanism bitmap or should be pushed out to other ABI.
Yes, for sure, I don't have a good sense what is better.
Quoted 4 lines
For
example, if the presence of the secure IOMMU is enumerated after the device is in the LOCKED state then userspace policy can know what it is getting into without needing to tell the kernel.
Yes, I think for almost all of this we could push it out to userspace and have knobs all over. Like IOMMU already has a knob to switch to DMA mode, we could add an IOMMU specific trust level to set ADVERSARY also if we go in this direction.
But, I also do really like the idea of a central knob that was fairly general and simple DISABLE/ADVERSARY/FULL which all sorts of things can hang off. Including IOMMU and the drivers themselves that may change operation in ADVERSARY mode.
That means they need simple general definitions for what they are, and I really like this ADVERSARY naming vs trust as it makes it very clear what the device disposition intention is.
Quoted 5 lines
Not essential, but it is useful to have the "operate device" intent, IOMMU default domain configuration, trust level, and private DMA enable all in the same bus operation (->dma_configure). Which is why "accept" was demoted to just set RUN state and leave the rest to be finalized later.
I agree it is nice, and I think this can work fine, the bus code when it does the dma configure step can reach out to the TSM and have it enable DMA.
We also want the IOMMU to change to a default-disabled state, so we also want its hook in dma configure to switch to non-default too.
It makes alot of sense to me that *all* DMA enable knobs will be disabled until right before probe starts and then disabled again once remove completes. TSM, IOMMU, PCI, etc.
Jason
Quoted 8 lines · Jason Gunthorpe
Quoted 5 lines · Dan Williams (nvidia)
1/ We previously discussed a use case to operate a device's private MMIO while not allowing access to private memory (software encrypted NVME with private MMIO [1]). Many of the following comments are based on preserving this assumption so you can save some reading if we agree that use case can be abandoned.
Quoted 4 lines
force_dma_unencrypted()does not *prevent* device access to private memory and provides no security properties on its own. It's only purpose is to inform the DMA API what the HW restrictions are for doing DMA.
Right, to be clear, this mode's security properties come from never asking the TSM to enable private DMA while the device is in RUN.
force_dma_unencrypted() has no security properties.
Quoted 3 lines
The use case I had in mind relied on the vIOMMU to police access to the private memory. I think if you do T=1 and ADVERSARY you must have a vIOMMU or it is an illogical configuration.
Ok, the software encryption aspect of the suggestion was what threw me. So we are saying, "no need, outside of defense in depth or data at rest concerns, to include software encryption on top of an NVME device being operated adversarially with T=1."
Quoted 7 lines
So, IMHO there is no use case for a T=1 device to only use shared memory, I would abandon that combination.
Quoted 4 lines
2/ The confirmation of the trust level and the enabling of DMA are separated in time from setting the trust level and entering the RUN state.
Quoted 6 lines
Quoted 2 lines
All the archs separate the RUN step from the ENABLE DMA step, and the implementation separates those steps in time.
Yes, until the device is fully accepted the DMA has to be kept off. If you have to measure in the RUN state then we must have a second step after measurement completes, however we are measuring in LOCKED right?
Yes, measure while in LOCKED, but nothing strictly prevents taking measurements after RUN.
Quoted 4 lines
Quoted 3 lines
echo tsm0 > $pdev/tsm/lock
cat $nonce | device-evidence dump $pdev
device-evidence validate $pdev $generation
Quoted 8 lines
So the device is in LOCKED here, DMA remains off both because of LOCKED and becuase ENABLE DMA has not been done.
Quoted 1 line
echo 1 > $pdev/tsm/accept
And now it is RUN. So I don't see the issue with enabling DMA at the same time as gonig to RUN? (though defering it to driver probe would be a very nice touch as well)
It requires "accept" to consider the trust level. E.g. what does it mean to do something like change requirements after accept?
echo full $pdev/trust
echo 1 > $pdev/tsm/accept
echo adversary > $pdev/trust
In that scenario this now adversary device may have been allowed to operate without an enforcing vIOMMU, and needs to unlock the device to correct that.
Delaying enable DMA means the implementation can say, "all trust and security privileges are reconciled at device driver bind, and revoked at unbind".
Quoted 2 lines
With this API you can't set the devices to RUN and *then* collect the evidence, is that restriction OK?
The Enable DMA step requires evidence in the end. So it is more implementation ergonomics driving whether "accept" includes Enable DMA or defers to driver bind.
Quoted 5 lines
Quoted 2 lines
echo full > $pdev/trust
echo $pdev > $driver/bindThen we can set the trust and bind the driver, knowing that the T=1 security property is in place.
Or, "accept" success tells you the full T=1 property is required for "bind" to succeed.
Quoted 16 lines
Quoted 4 lines
The configuration window where $pdev/trust and $pdev/accept can be dynamically changing should not be changing the
force_dma_unencrypyted()result if only because that value will mismatch the hardware state until the next ->dma_configure() event.T=x cannot be changed while a driver is bound, we need to block that. Changing it will mess up all DMA and the DMA API won't work anymore.
Thus
force_dma_unencrypted()can only change when a driver is unbound.If a driver is unbound nothing ever reads
force_dma_unencrypted()so it is safe to change it, thus we can just synchronize T=1 andforce_dma_unencrypted()whenever T changes.
Quoted 7 lines
The error case has to wait for the driver to unbind if userspace wants to go from
ERROR->UNLOCKED. IMHO for RAS we are going to need a flow to logically go fromERROR->RUN, probably with the 'same device acceptance' I suggested earlier. In this case the DMAs will be halted, the devices gets back to T=1 and RUN, then the DMAs are turned on again. The driver remains in T=1 andforce_dma_unencrypted()is always false.
Ideally this recovery does not need userspace round trips, but for that the kernel needs SPDM transcript parsing. The measurement digest that CCA provides is over the full previous transcript. A fresh nonce to validate same device leads to the kernel need to validate the signature with a cached key and compare the extracted measurement payload from the previous transcript.
Quoted 5 lines
Quoted 3 lines
There are also buses and paravisors that may know that private-DMA is enabled for a device by construction. In that case it is also a "trusted to access" signal, and not a "required to access" signal.
In this case they wire force_dma_unencrypted()=false.
Yeah, I just need a scheme where modular bus providers do not end up compromising the private / unexported method of changing the flag that force_dma_unencrypted() consumes.
Quoted 21 lines
Quoted 3 lines
My answer, given the concerns of drivers dangerously flipping the
force_dma_unencrypted()result at runtime was to place it in 'structdevice_private'Yes, this seems good idea in general
Quoted 3 lines
Otherwise it is confusing when 2 devices are at FULL trust, but one has private memory access and the other does not. One is FULL the other is FULL+.
It is confusing when phrased like this, but this is also why I was suggesting below that a pure trust level may not be sufficient as we have quite a cross product of scenarios.
If the user wants to operate a device without T=1 and without ADVERSARY then that's goofy, but technically the kernel doesn't really need to care? It can do it.
IMHO a big question is how much do we want the kernel to nanny userspace by having options to double check its work?
Quoted 6 lines
Either we trust the user to have checked all the policy conditions before writing
/sys/trust, or we encode some policy conditions and tell the kernel so it can double-check prior to probe.At least if we omit the double check it can be fairly easy to add in later if it really was needed for some time of use reason.
As long as IOMMU presence can be enumerated prior to acceptance, T=1 always includes private DMA, and the T=1 status is tracked independent of the trust level then yes, the cross product can be avoided.
Quoted 15 lines
Quoted 2 lines
The question is whether these requirements belong on a central device trust mechanism bitmap or should be pushed out to other ABI.
Yes, for sure, I don't have a good sense what is better.
Quoted 4 lines
For
example, if the presence of the secure IOMMU is enumerated after the device is in the LOCKED state then userspace policy can know what it is getting into without needing to tell the kernel.Yes, I think for almost all of this we could push it out to userspace and have knobs all over. Like IOMMU already has a knob to switch to DMA mode, we could add an IOMMU specific trust level to set ADVERSARY also if we go in this direction.
Quoted 4 lines
But, I also do really like the idea of a central knob that was fairly general and simple DISABLE/ADVERSARY/FULL which all sorts of things can hang off. Including IOMMU and the drivers themselves that may change operation in ADVERSARY mode.
Yes, even if some of the ancillary requirements move to other ABI the central question of DISABLE/ADVERSARY/FULL stays common.
Quoted 3 lines
That means they need simple general definitions for what they are, and I really like this ADVERSARY naming vs trust as it makes it very clear what the device disposition intention is.
I will do the "device_adversarial()" conversion throughout.
'struct device' grows some "request" policy for trust and T=1 that can be changed by modules etc. 'struct device_private' grows the operational trust level stable under device_lock() and a new flag to reflect T=1.
Quoted 7 lines · Dan Williams (nvidia)
Quoted 4 lines
force_dma_unencrypted()does not *prevent* device access to private memory and provides no security properties on its own. It's only purpose is to inform the DMA API what the HW restrictions are for doing DMA.Right, to be clear, this mode's security properties come from never asking the TSM to enable private DMA while the device is in RUN.
Ok, that's a twist I hadn't thought about. I don't see a reason to support a driver probed with RUN but T=1 DMA disabled by the TSM.
Still, if we do this, I think having the TSM deal with it is probably better than making a cross product of the trust level, something like:
echo only-shared-dma > $pdev/tsm/accept
So maybe accept should be tweaked:
echo full > $pdev/tsm/accept
? It doesn't matter much
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
echo 1 > $pdev/tsm/accept
And now it is RUN. So I don't see the issue with enabling DMA at the same time as gonig to RUN? (though defering it to driver probe would be a very nice touch as well)
It requires "accept" to consider the trust level. E.g. what does it mean to do something like change requirements after accept?
Quoted 7 lines
echo full $pdev/trust
echo 1 > $pdev/tsm/accept
echo adversary > $pdev/trustIn that scenario this now adversary device may have been allowed to operate without an enforcing vIOMMU, and needs to unlock the device to correct that.
If accept enables DMA then it is up to userspace to ensure all policy objectives are met before accepting, not after.
I think the statement kernel makes to userspace is the moment it writes trust or accept the kernel is free to take action on it. ie don't write a trust or accept until you mean it.
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
There are also buses and paravisors that may know that private-DMA is enabled for a device by construction. In that case it is also a "trusted to access" signal, and not a "required to access" signal.
In this case they wire force_dma_unencrypted()=false.
Yeah, I just need a scheme where modular bus providers do not end up compromising the private / unexported method of changing the flag that
force_dma_unencrypted()consumes.
Yeah, the bus and TSM really have to control this property, it is like the other DMA attributes (eg dma seg boundary, etc)
Quoted 6 lines
Quoted 2 lines
At least if we omit the double check it can be fairly easy to add in later if it really was needed for some time of use reason.
As long as IOMMU presence can be enumerated prior to acceptance, T=1 always includes private DMA, and the T=1 status is tracked independent of the trust level then yes, the cross product can be avoided.
Ok. iommu has to be setup the moment the struct device is created, it can't add on later. I *think* the iommu related sysfs ordering is done properly before any uevents, but I haven't validated that.
Quoted 3 lines
'struct device' grows some "request" policy for trust and T=1 that can be changed by modules etc. 'struct
device_private' grows the operational trust level stable underdevice_lock()and a new flag to reflect T=1.
I guess
- The active trust level should be RO visible to the driver, iommu, etc It should be stable under a bound driver
- The "dma require unencrypted" property needs to RO visible to the DMA API and stable under a bound driver. This would input where force_dma_unecrpyted() is in the flow [the name should align with all the other per-device DMA API specific properties like seg limit, boundary, mask, etc]
- The requested trust policy should be internal to the driver core and be converted to the active trust level right before probe
- We should have ways to enable/disable all DMA before/after probe, including both TSM and IOMMU approaches
Broadly all busses get some way to convert the requested trust policy to the active trust level (eg by mixing in ACPI, etc, etc)
All busses get some way to set the "dma require unencrypted" property
IOMMU and drivers should be sensitive to the trust level
TSM is sensitive to accept, not the trust level
Userspace must ensure any security policy is met before accepting or binding.
??
Jason
Alexey Kardashevskiy
maintainerQuoted 11 lines · Jason Gunthorpe
Quoted 7 lines · Dan Williams (nvidia)
Quoted 4 lines
force_dma_unencrypted()does not *prevent* device access to private memory and provides no security properties on its own. It's only purpose is to inform the DMA API what the HW restrictions are for doing DMA.Right, to be clear, this mode's security properties come from never asking the TSM to enable private DMA while the device is in RUN.
Ok, that's a twist I hadn't thought about. I don't see a reason to support a driver probed with RUN but T=1 DMA disabled by the TSM.
afaik you cannot have RUN and T=0 DMA at the same time. You can though have C=0 (i.e. IOMMU <-> memory in the plain).
Quoted 10 lines
Still, if we do this, I think having the TSM deal with it is probably better than making a cross product of the trust level, something like:
echo only-shared-dma > $pdev/tsm/accept
So maybe accept should be tweaked:
echo full > $pdev/tsm/accept
? It doesn't matter much
I configure my hw to allow "accept" (== T=1 for DMA and MMIO) but still only allow unencrypted guest memory for DMA (set vTOM to 0 to say "all unencrypted) if the driver was loaded with "trust" other than "full" (and this series does not call the enable_dma() hook if not "full", Dan is changing it though) so the module parameter works...
Quoted 62 lines
Quoted 9 lines
Quoted 5 lines
Quoted 1 line
echo 1 > $pdev/tsm/accept
And now it is RUN. So I don't see the issue with enabling DMA at the same time as gonig to RUN? (though defering it to driver probe would be a very nice touch as well)
It requires "accept" to consider the trust level. E.g. what does it mean to do something like change requirements after accept?
Quoted 7 lines
echo full $pdev/trust
echo 1 > $pdev/tsm/accept
echo adversary > $pdev/trustIn that scenario this now adversary device may have been allowed to operate without an enforcing vIOMMU, and needs to unlock the device to correct that.
If accept enables DMA then it is up to userspace to ensure all policy objectives are met before accepting, not after.
I think the statement kernel makes to userspace is the moment it writes trust or accept the kernel is free to take action on it. ie don't write a trust or accept until you mean it.
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
There are also buses and paravisors that may know that private-DMA is enabled for a device by construction. In that case it is also a "trusted to access" signal, and not a "required to access" signal.
In this case they wire force_dma_unencrypted()=false.
Yeah, I just need a scheme where modular bus providers do not end up compromising the private / unexported method of changing the flag that
force_dma_unencrypted()consumes.Yeah, the bus and TSM really have to control this property, it is like the other DMA attributes (eg dma seg boundary, etc)
Quoted 6 lines
Quoted 2 lines
At least if we omit the double check it can be fairly easy to add in later if it really was needed for some time of use reason.
As long as IOMMU presence can be enumerated prior to acceptance, T=1 always includes private DMA, and the T=1 status is tracked independent of the trust level then yes, the cross product can be avoided.
Ok. iommu has to be setup the moment the struct device is created, it can't add on later. I *think* the iommu related sysfs ordering is done properly before any uevents, but I haven't validated that.
Quoted 3 lines
'struct device' grows some "request" policy for trust and T=1 that can be changed by modules etc. 'struct
device_private' grows the operational trust level stable underdevice_lock()and a new flag to reflect T=1.I guess
- The active trust level should be RO visible to the driver, iommu, etc It should be stable under a bound driver- The "dma require unencrypted" property needs to RO visible to the DMA API and stable under a bound driver. This would input where
force_dma_unecrpyted()is in the flow [the name should align with all the other per-device DMA API specific properties like seg limit, boundary, mask, etc]
Quoted 4 lines
- The requested trust policy should be internal to the driver core and be converted to the active trust level right before probe
- We should have ways to enable/disable all DMA before/after probe,
"echo 1 > unlock" should do that (but also stops encrypted MMIO) or we want a finer knob?
Quoted 9 lines
including both TSM and IOMMU approaches
Broadly all busses get some way to convert the requested trust policy to the active trust level (eg by mixing in ACPI, etc, etc)All busses get some way to set the "dma require unencrypted" property
IOMMU and drivers should be sensitive to the trust level
TSM is sensitive to accept, not the trust level
This makes the module's "trust" parameter useless, right?
Quoted 6 lines
Userspace must ensure any security policy is met before accepting or binding.
??
Jason
Signature · 1 line
Alexey
Quoted 7 lines · Alexey Kardashevskiy
Quoted 6 lines · Jason Gunthorpe
Quoted 5 lines · Dan Williams (nvidia)
Quoted 4 lines
force_dma_unencrypted()does not *prevent* device access to private memory and provides no security properties on its own. It's only purpose is to inform the DMA API what the HW restrictions are for doing DMA.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Right, to be clear, this mode's security properties come from never asking the TSM to enable private DMA while the device is in RUN.
Ok, that's a twist I hadn't thought about. I don't see a reason to support a driver probed with RUN but T=1 DMA disabled by the TSM.
afaik you cannot have RUN and T=0 DMA at the same time.
Right, that's a great point. The devices we are making simply won't do T=0 DMA once they are in RUN, I expect that to be the norm.
So if you disable T=1 DMA at the TSM then the device doesn't work anymore because there was no standard way to tell the device it shouldn't do T=1.
Quoted 6 lines
I configure my hw to allow "accept" (== T=1 for DMA and MMIO) but still only allow unencrypted guest memory for DMA (set vTOM to 0 to say "all unencrypted) if the driver was loaded with "trust" other than "full" (and this series does not call the
enable_dma()hook if not "full", Dan is changing it though) so the module parameter works...
Yeah, this would be an interesting configuration from the TSM - support T=1 but change the T=1 translation so that only shared memory is mapped. vTOM on AMD and other tricks on other arches.
But AFIAK this isn't generally supported so I'd just leave it out for now.
Quoted 10 lines
Quoted 10 lines
I guess
- The active trust level should be RO visible to the driver, iommu, etc It should be stable under a bound driver- The "dma require unencrypted" property needs to RO visible to the DMA API and stable under a bound driver. This would input where
force_dma_unecrpyted()is in the flow [the name should align with all the other per-device DMA API specific properties like seg limit, boundary, mask, etc]
Quoted 6 lines
Quoted 4 lines
- The requested trust policy should be internal to the driver core and be converted to the active trust level right before probe
- We should have ways to enable/disable all DMA before/after probe,
"echo 1 > unlock" should do that (but also stops encrypted MMIO) or we want a finer knob?
You shouldn't be able to unlock while a driver is bind. I'm aruging we should also be able to keep the device in run and block all DMA through the TSM.
Quoted 3 lines
Quoted 1 line
TSM is sensitive to accept, not the trust level
This makes the module's "trust" parameter useless, right?
Yes, ideally it should act based on callbacks I think
Jason
Alexey Kardashevskiy
maintainerQuoted 1 line · Dan Williams
Major changes since: "[PATCH v2 00/19] PCI/TSM: TEE I/O infrastructure" [1]
soo with the recent sudden sad news about Dan, I do not know where to start really so I just start...
Here is his unfinished work which I'd really like to see finished to progress with the phase2 of the plan. How do we proceed from now on?
The direction appears to be set, imho the immediate TODO list is:
- address "not-netlink-y enough" concerns
- teach pci_iomap() to map mixed private+shared BARs (ones which contain MSIX for the starter)
- always call pci_tsm_ops::enable_dma/disable_dma
- ditch the module trust parameter?
- add a knob to enable/disable DMA and keep TDI in RUN
Any volunteers? Yilun, Aneesh, Jason?
Which maintainer's tree is this going to go to?
I am happy to amend the patches for the time being, and do the testing on my HW, and posting.
And we are going to continue running the bi-weekly TSM+IOMMUFD community call (Thursday evening in the USA) so we can talk this through then, the next one on August 6th:
Thanks,
Quoted 192 lines
- Implement a new netlink schema for dumping large SPDM blobs (Jakub, Lukas)
- Introduce a device core implementation for evidence gathering given SPDM is applicable to devices outside of PCI (Lukas)
- Replace the "CC acceptance" mechanism with "Device Trust" (Greg, Jason)
- Fix TDISP report parsing to use bytes instead of pfns for the start value of an MMIO report (PCI spec clarification)
- Support the CCA expectation that MMIO reports are BAR aligned (Aneesh)[1]: http://lore.kernel.org/[email protected]
[ Note, this series is back to a v1 posting given the new theme of
"Evidence and Trust". It defers some of the lower level details, like touching arch-level ioremap implementations to consultencrypted_iomem, until the higher order details are settled ].---
Summary:
A threat model wants to arrange for devices to not be operated until they present evidence. Add device evidence gathering, device trust levels, and hook those up for the PCI/TSM (PCI device security protocol) implementation.
---Confidential Computing and other use cases specify a threat model where devices are potentially adversarial. To date the device core trusts devices and their drivers, and only specific buses have local "adversary" device mitigations. The task is define a general trust concept to up-level existing trust mitigations, and allow for incremental tightening of mitigations across bus types and IOMMU drivers.
Goals:
* Create a mechanism for booting with a limited set devices + drivers and permit selectively enabling more devices + drivers at runtime.
* Up-level bus specific trust mitigations like PCI untrusted to a core concept.
* Let buses coordinate entry into the TCB via the existing ->dma_configure() callbackThe proposal considers existing trust mitigation in various buses. The PCI device "untrusted" flags is an example of limiting but not fully blocking operation of a device. The USB interface authorized flag is an example of not trusting a device to operate at all. The implementation starts with these trust level
DEVICE_TRUST_NONE, no operation of the device, andDEVICE_TRUST_ADVERSARY, limited operation of the device. For example, the PCI "untrusted" flag turns off device translation capabilities (ATS) and strictly enforces IOMMU mappings, but otherwise allows operation of the device.The proposal does not currently incorporate other bus specific security mechanisms that are independent of individual device operational trust. For example, USB device (not interface) authorization and Thunderbolt switch authorization are transport gates to other devices. PCIe Link Encryption is also a transport security mechanism that informs a trust decision, but it is still possible to distrust a device with Link Encryption and conversely distrust a device protected by Link Encryption. Those security mechanisms are orthogonal to trust policy for driver bind, address space access, and acceleration feature enabling.
The result is device trust is an input to the driver core and IOMMU layers. A bus can opt to sync its local bind policy, address space access, and acceleration features to the trust level, or treat it as an overriding second-stage policy from the bus's first stage mechanism.
The trust levels established at enumeration and enforced at driver attach are DEVICE_TRUST_{NONE,ADVERSARY,AUTO,TCB}:
* NONE: no usage of the device unless the trust is explicitly overridden by user policy specified via a driver flag, module flag, or uapi (TBD).
* ADVERSARY: needs acknowledgement from the bus and IOMMU / DMA layers that the device is limited to strict IOMMU translation behavior. Drivers can use this as a signal to limit functionality. This designation implies follow-on IOMMU and bus enabling work for features like arranging for the device to attach to a blocked IOMMU domain when detached from a driver.
* AUTO: typical / historical Linux driver model.
* TCB: a trust level that only exists in Confidential Computing environments. When acked by the IOMMU / DMA layer it enables the device to issue direct-DMA to private/encrypted addresses or otherwise attach to a secure vIOMMU within the TCB.
Overview of changes:
Patches 1-6: Update netlink to be able to support large blob transfers that the SPDM standard requires. SPDM can be associated with any device type, so the implementation lands in the device core with PCI/TSM as the first consumer.Patches 7-9: Introduce
DEVICE_TRUST_NONEandDEVICE_TRUST_ADVERSARY. Support blocking driver bind by default, and allow either build policy or module policy to override the the trust level.Patches 10-12,14-15: Plumb the PCI/TSM support for the PCI device security (TDISP) protocol to transition the device through
UNLOCKED->LOCKED->RUN.Patch 13: Support access to private memory either by the bus either coordinating with a TSM driver, or relying on the bus provider to have already arranged access (paravisor case).
This passes basic checkout with the sample TSM driver and a sample netlink client for the device evidence facility. Those patches are only on the git tree for now [2].
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/log/?h=devsec-phase2
Dan Williams (15):
netlink: specs: Introduce multi-message blobs for SPDM
tools: ynl: Teach pyynl to handle blobs
tools: ynl: Teachynl_gen_cto validate and dump 'blob' attributes device core: Introduce "device evidence" over netlink device core: Add "device evidence" 'validate' command PCI/TSM: Add device evidence support
modules: Document the globalasync_probeparameter
device core: Initial device trust infrastructure
PCI, device core: Move "untrusted" concept toDEVICE_TRUST_ADVERSARYPCI/TSM: Add device interface security LOCKED support PCI/TSM: Add device interface security RUN support PCI/TSM: Add device interface security DMA enable/disable PCI, device core: Add private memory access forDEVICE_TRUST_TCBPCI/TSM: Create MMIO descriptors via TDISP Report PCI/TSM: Add relative MMIO offset support?drivers/base/Kconfig | 91 +++ drivers/pci/Kconfig | 2 + drivers/base/Makefile | 2 + drivers/pci/Makefile | 2 +- drivers/pci/tsm/Makefile | 8 + Documentation/ABI/stable/sysfs-module | 17 + Documentation/ABI/testing/sysfs-bus-pci | 47 +- Documentation/ABI/testing/sysfs-class-tsm | 19 + Documentation/driver-api/pci/tsm.rst | 47 ++ Documentation/netlink/genetlink-legacy.yaml | 6 + Documentation/netlink/genetlink.yaml | 7 + Documentation/netlink/netlink-raw.yaml | 7 + .../netlink/specs/device-evidence.yaml | 203 ++++++ drivers/base/base.h | 15 + drivers/base/device-evidence-netlink.h | 25 + include/linux/device.h | 3 + include/linux/device/evidence.h | 88 +++ include/linux/device/trust.h | 56 ++ include/linux/ioport.h | 2 + include/linux/module.h | 6 +- include/linux/pci-tsm.h | 104 +++- include/linux/pci.h | 13 +- include/uapi/linux/device-evidence.h | 111 ++++ tools/net/ynl/lib/ynl-priv.h | 4 + tools/net/ynl/lib/ynl.h | 14 + drivers/base/core.c | 7 + drivers/base/dd.c | 5 + drivers/base/device-evidence-netlink.c | 58 ++ drivers/base/evidence.c | 495 +++++++++++++++ drivers/base/trust.c | 103 ++++ drivers/iommu/amd/iommu.c | 2 +- drivers/iommu/dma-iommu.c | 13 +- drivers/iommu/intel/iommu.c | 2 +- drivers/iommu/iommu.c | 2 +- drivers/pci/ats.c | 2 +- drivers/pci/pci-driver.c | 25 +- drivers/pci/pci.c | 2 +- drivers/pci/probe.c | 10 +- drivers/pci/quirks.c | 4 +- drivers/pci/{tsm.c => tsm/core.c} | 579 +++++++++++++++++- drivers/pci/tsm/evidence.c | 110 ++++ drivers/virt/coco/tsm-core.c | 43 +- kernel/module/main.c | 13 + kernel/resource.c | 8 + tools/net/ynl/lib/ynl.c | 33 + MAINTAINERS | 6 +- tools/net/ynl/pyynl/lib/nlspec.py | 12 + tools/net/ynl/pyynl/lib/ynl.py | 62 +- tools/net/ynl/pyynl/ynl_gen_c.py | 80 ++- 49 files changed, 2509 insertions(+), 66 deletions(-)create mode 100644
drivers/pci/tsm/Makefile
create mode 100644Documentation/netlink/specs/device-evidence.yamlcreate mode 100644drivers/base/device-evidence-netlink.hcreate mode 100644include/linux/device/evidence.hcreate mode 100644include/linux/device/trust.h
create mode 100644include/uapi/linux/device-evidence.hcreate mode 100644drivers/base/device-evidence-netlink.ccreate mode 100644drivers/base/evidence.c
create mode 100644drivers/base/trust.c
renamedrivers/pci/{tsm.c=>tsm/core.c} (61%)
create mode 100644drivers/pci/tsm/evidence.cbase-commit:
dc59e4fea9d83f03bad6bddf3fa2e52491777482
Signature · 1 line
Alexey
Quoted 7 lines · Alexey Kardashevskiy
Quoted 1 line · Dan Williams
Major changes since: "[PATCH v2 00/19] PCI/TSM: TEE I/O infrastructure" [1]
soo with the recent sudden sad news about Dan, I do not know where to start really so I just start...
Here is his unfinished work which I'd really like to see finished to progress with the phase2 of the plan. How do we proceed from now on?
Yes, we do, but at least I need some more time before considering what to do, sorry.
Quoted 6 lines
The direction appears to be set, imho the immediate TODO list is:
- address "not-netlink-y enough" concerns
- teachpci_iomap()to map mixed private+shared BARs (ones which contain MSIX for the starter)
- always call pci_tsm_ops::enable_dma/disable_dma
- ditch the module trust parameter?
- add a knob to enable/disable DMA and keep TDI in RUN
I somehow suspect we should probably reduce the scope somewhat into smaller chunks at a time and try to follow this as a skeleton, but I haven't studied it carefully.
Jason
Quoted 19 lines · Jason Gunthorpe
Quoted 7 lines · Alexey Kardashevskiy
Quoted 1 line · Dan Williams
Major changes since: "[PATCH v2 00/19] PCI/TSM: TEE I/O infrastructure" [1]
soo with the recent sudden sad news about Dan, I do not know where to start really so I just start...
Here is his unfinished work which I'd really like to see finished to progress with the phase2 of the plan. How do we proceed from now on?
Yes, we do, but at least I need some more time before considering what to do, sorry.
Quoted 6 lines
The direction appears to be set, imho the immediate TODO list is:
- address "not-netlink-y enough" concerns
- teachpci_iomap()to map mixed private+shared BARs (ones which contain MSIX for the starter)
- always call pci_tsm_ops::enable_dma/disable_dma
- ditch the module trust parameter?
- add a knob to enable/disable DMA and keep TDI in RUN
Quoted 3 lines
I somehow suspect we should probably reduce the scope somewhat into smaller chunks at a time and try to follow this as a skeleton, but I haven't studied it carefully.
Hi Jason & Ankit:
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
Quoted 8 lines
Hi Jason & Ankit:
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
Hi Yilun & Alexey
Thanks for the suggestion for the Aug 6 bi-weekly call. Unfortunately I'm on vacation for the rest of the week and the meeting timing is inconvenient for Jason (PST+4).
We can perhaps consider syncup sometime after Aug 16th. Does that sound fine?
Quoted 7 lines · Ankit Agrawal
Quoted 7 lines
Hi Jason & Ankit:
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
Quoted 8 lines
Quoted 1 line
Hi Yilun & Alexey
Thanks for the suggestion for the Aug 6 bi-weekly call. Unfortunately I'm on vacation for the rest of the week and the meeting timing is inconvenient for Jason (PST+4).
We can perhaps consider syncup sometime after Aug 16th. Does that sound fine?
It's good. Let's sync up later.
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Jason
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Hi Ankit & Jason,
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438c
Thanks,
Yilun
Alexey Kardashevskiy
maintainerQuoted 11 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Quoted 6 lines
Hi Ankit & Jason,
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438c
So I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)
Scenario B:
Seattle (PDT, UTC-7) 18:00 – 19:00 Day 1
Atlantic Time (ADT, UTC-3) 22:00 – 23:00 Day 1
Berlin (CEST, UTC+2) 03:00 – 04:00 Day 2
India (IST, UTC+5:30) 06:30 – 07:30 Day 2
China (CST, UTC+8) 09:00 – 10:00 Day 2
Sydney (AEST, UTC+10) 11:00 – 12:00 Day 2
Scenario C (which is the current schedule):
Seattle (PDT, UTC-7) 20:00 – 21:00 Thursday, Sep 3
Atlantic Time Zone (ADT, UTC-3) 00:00 – 01:00 Friday, Sep 4
Berlin (CEST, UTC+2) 05:00 – 06:00 Day 2
India (IST, UTC+5:30) 08:30 – 09:30 Friday, Sep 4
China (CST, UTC+8) 11:00 – 12:00 Friday, Sep 4
Sydney (AEST, UTC+10) 13:00 – 14:00 Friday, Sep 4
but if we want to change the time - who do we contact? Or set up a new call (again, how)?
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
Alternatively, wait until the LPC in a month but I suspect not everybody is going to be there.
Signature · 1 line
Alexey
Quoted 48 lines · Alexey Kardashevskiy
Quoted 11 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Quoted 6 lines
Hi Ankit & Jason,
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)Scenario B:
Seattle (PDT, UTC-7) 18:00 – 19:00 Day 1
Atlantic Time (ADT, UTC-3) 22:00 – 23:00 Day 1
Berlin (CEST, UTC+2) 03:00 – 04:00 Day 2
India (IST, UTC+5:30) 06:30 – 07:30 Day 2
China (CST, UTC+8) 09:00 – 10:00 Day 2
Sydney (AEST, UTC+10) 11:00 – 12:00 Day 2Scenario C (which is the current schedule):
Seattle (PDT, UTC-7) 20:00 – 21:00 Thursday, Sep 3
Atlantic Time Zone (ADT, UTC-3) 00:00 – 01:00 Friday, Sep 4
Berlin (CEST, UTC+2) 05:00 – 06:00 Day 2
India (IST, UTC+5:30) 08:30 – 09:30 Friday, Sep 4
China (CST, UTC+8) 11:00 – 12:00 Friday, Sep 4
Sydney (AEST, UTC+10) 13:00 – 14:00 Friday, Sep 4
Quoted 6 lines
but if we want to change the time - who do we contact? Or set up a new call (again, how)?
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
Berlin (CEST, UTC+2) 03:00 – 04:00 Day 2
No wonder that you don't have Europe :)
Thanks
Quoted 48 lines · Alexey Kardashevskiy
Quoted 11 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Quoted 6 lines
Hi Ankit & Jason,
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)Scenario B:
Seattle (PDT, UTC-7) 18:00 – 19:00 Day 1
Atlantic Time (ADT, UTC-3) 22:00 – 23:00 Day 1
Berlin (CEST, UTC+2) 03:00 – 04:00 Day 2
India (IST, UTC+5:30) 06:30 – 07:30 Day 2
China (CST, UTC+8) 09:00 – 10:00 Day 2
Sydney (AEST, UTC+10) 11:00 – 12:00 Day 2Scenario C (which is the current schedule):
Seattle (PDT, UTC-7) 20:00 – 21:00 Thursday, Sep 3
Atlantic Time Zone (ADT, UTC-3) 00:00 – 01:00 Friday, Sep 4
Berlin (CEST, UTC+2) 05:00 – 06:00 Day 2
India (IST, UTC+5:30) 08:30 – 09:30 Friday, Sep 4
China (CST, UTC+8) 11:00 – 12:00 Friday, Sep 4
Sydney (AEST, UTC+10) 13:00 – 14:00 Friday, Sep 4
Quoted 6 lines
but if we want to change the time - who do we contact? Or set up a new call (again, how)?
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
I'm from china. I'd say every listed option is perfect to me. Thanks.
Quoted 1 line
Quoted 14 lines · Alexey Kardashevskiy
Quoted 13 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Hi Ankit & Jason,
Quoted 7 lines
Quoted 4 lines
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
I've asked several people in Europe to help on parts of this series, and combined with China/Aus someone is going to end up left out.
I know people are anxious about their projects. In the short term I will volunteer to send patches (ie bug fixes!) to Linus in the CC area Dan was responsible for. Please email me if something needs some attention and we can figure it out.
To fully move forward we need to establish a new TSM maintainer team. In my past experience a consensus has emerged off list, I suggest we try that here as well. Several people should be recruited. I will also volunteer to help train and launch a new team.
I can say that several people from NVIDIA are working on the ideas in this patch series. Ankit is setting up a team to focus on upstream patches for all of CC inside NVIDIA. I've asked Jiri Pirko to work on the PCI evidence uAPI based on his deep netlink experience, and Leon Romanovsky to work on the binding problem/adversarial/trust level driver core idea. They will share their approaches for solving these problems as seperate proposals.
Quoted 7 lines
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)
This is the only time Europe could reasonably participate in, obviously it is not OK for China and Aus.
Quoted 2 lines
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
Calls are undoutably useful!
Quoted 2 lines
Alternatively, wait until the LPC in a month but I suspect not everybody is going to be there.
LPC is not that far away, I think we should have a goal of having a consensus for or at LPC
Regards,
Jason
Alexey Kardashevskiy
maintainerQuoted 31 lines · Jason Gunthorpe
Quoted 14 lines · Alexey Kardashevskiy
Quoted 13 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Hi Ankit & Jason,
Quoted 7 lines
Quoted 4 lines
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
I've asked several people in Europe to help on parts of this series, and combined with China/Aus someone is going to end up left out.
I know people are anxious about their projects. In the short term I will volunteer to send patches (ie bug fixes!) to Linus in the CC area Dan was responsible for. Please email me if something needs some attention and we can figure it out.
Quoted 4 lines
To fully move forward we need to establish a new TSM maintainer team. In my past experience a consensus has emerged off list, I suggest we try that here as well. Several people should be recruited. I will also volunteer to help train and launch a new team.
I would learn a thing or two.
Quoted 7 lines
I can say that several people from NVIDIA are working on the ideas in this patch series. Ankit is setting up a team to focus on upstream patches for all of CC inside NVIDIA. I've asked Jiri Pirko to work on the PCI evidence uAPI based on his deep netlink experience, and Leon Romanovsky to work on the binding problem/adversarial/trust level driver core idea. They will share their approaches for solving these problems as seperate proposals.
Sounds good!
Quoted 10 lines
Quoted 7 lines
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)This is the only time Europe could reasonably participate in, obviously it is not OK for China and Aus.
What range would be reasonable? I prefer midnight to 6am, for example. At least for now, Yilun (China) and myself (Australia) are ok with these times.
Let's use a tool: https://crab.fit/iommufd-tsm-call-593432
Please vote.
And later on, Australia goes into DST, the nothern hemisphere - off DST (but not the same weekend, of course), and we'll start this all over again, may be.
We'll still have the tomorrow call at the old time. Thanks,
Quoted 13 lines
Quoted 2 lines
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
Calls are undoutably useful!
Quoted 2 lines
Alternatively, wait until the LPC in a month but I suspect not everybody is going to be there.
LPC is not that far away, I think we should have a goal of having a consensus for or at LPC
Regards,
Jason
Signature · 1 line
Alexey
Aneesh Kumar K.V
maintainerQuoted 36 lines · Jason Gunthorpe
Quoted 14 lines · Alexey Kardashevskiy
Quoted 13 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Hi Ankit & Jason,
Quoted 7 lines
Quoted 4 lines
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
I've asked several people in Europe to help on parts of this series, and combined with China/Aus someone is going to end up left out.
I know people are anxious about their projects. In the short term I will volunteer to send patches (ie bug fixes!) to Linus in the CC area Dan was responsible for. Please email me if something needs some attention and we can figure it out.
To fully move forward we need to establish a new TSM maintainer team. In my past experience a consensus has emerged off list, I suggest we try that here as well. Several people should be recruited. I will also volunteer to help train and launch a new team.
Quoted 8 lines
I can say that several people from NVIDIA are working on the ideas in this patch series. Ankit is setting up a team to focus on upstream patches for all of CC inside NVIDIA. I've asked Jiri Pirko to work on the PCI evidence uAPI based on his deep netlink experience, and Leon Romanovsky to work on the binding problem/adversarial/trust level driver core idea. They will share their approaches for solving these problems as seperate proposals.
I can also help with the device trust and iommufd interfaces, as well as the CCA DA pieces. We need to find someone to take on dma-buf, as that is one area we have not yet integrated with the full TDISP flow.
Quoted 9 lines
Quoted 8 lines
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)
Quoted 3 lines
This is the only time Europe could reasonably participate in, obviously it is not OK for China and Aus.
I'm based in India, and all the proposed times work for me. Who is based in Europe?
Jason -> Canada ?
Yilun -> China
Alexey -> Australia
Ankit Agarwal -> US ?
Aneesh -> India
Quoted 6 lines
Quoted 3 lines
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
Calls are undoutably useful!
Quoted 8 lines
Quoted 2 lines
Alternatively, wait until the LPC in a month but I suspect not everybody is going to be there.
LPC is not that far away, I think we should have a goal of having a consensus for or at LPC
Regards,
Jason
-aneesh
Quoted 48 lines · Aneesh Kumar K.V
Quoted 36 lines · Jason Gunthorpe
Quoted 14 lines · Alexey Kardashevskiy
Quoted 13 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Hi Ankit & Jason,
Quoted 7 lines
Quoted 4 lines
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
I've asked several people in Europe to help on parts of this series, and combined with China/Aus someone is going to end up left out.
I know people are anxious about their projects. In the short term I will volunteer to send patches (ie bug fixes!) to Linus in the CC area Dan was responsible for. Please email me if something needs some attention and we can figure it out.
To fully move forward we need to establish a new TSM maintainer team. In my past experience a consensus has emerged off list, I suggest we try that here as well. Several people should be recruited. I will also volunteer to help train and launch a new team.
Quoted 8 lines
I can say that several people from NVIDIA are working on the ideas in this patch series. Ankit is setting up a team to focus on upstream patches for all of CC inside NVIDIA. I've asked Jiri Pirko to work on the PCI evidence uAPI based on his deep netlink experience, and Leon Romanovsky to work on the binding problem/adversarial/trust level driver core idea. They will share their approaches for solving these problems as seperate proposals.
Quoted 3 lines
I can also help with the device trust and iommufd interfaces, as well as the CCA DA pieces. We need to find someone to take on dma-buf, as that is one area we have not yet integrated with the full TDISP flow.
Given my recent involvement in dma-buf, I can try to fill that gap.
Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)
Quoted 6 lines
Quoted 3 lines
This is the only time Europe could reasonably participate in, obviously it is not OK for China and Aus.
I'm based in India, and all the proposed times work for me. Who is based in Europe?
Jiri and I are in the European time zone.
Quoted 2 lines
Jason -> Canada ?
Yes
Quoted 4 lines
Yilun -> China
Alexey -> Australia
Ankit Agarwal -> US ?
Aneesh -> India
Thanks
Quoted 3 lines
Quoted 3 lines
Yilun -> China
Alexey -> Australia
Ankit Agarwal -> US ?
I'm in India and the suggested schedule options works for me.
Thanks
Quoted 7 lines
I can say that several people from NVIDIA are working on the ideas in this patch series. Ankit is setting up a team to focus on upstream patches for all of CC inside NVIDIA. I've asked Jiri Pirko to work on the PCI evidence uAPI based on his deep netlink experience, and Leon Romanovsky to work on the binding problem/adversarial/trust level driver core idea. They will share their approaches for solving these problems as seperate proposals.
I can volunteer for updating and refreshing the TDISP state machine part of this patch series based on the Jiri's uAPI.
Lukas Wunner
reviewer[+cc Jonathan, start of thread is here:
https://lore.kernel.org/all/[email protected]/
]
Quoted 7 lines · Jason Gunthorpe
In the short term I
will volunteer to send patches (ie bug fixes!) to Linus in the CC area Dan was responsible for. Please email me if something needs some attention and we can figure it out.To fully move forward we need to establish a new TSM maintainer team.
I created the "devsec" group on git.kernel.org a few years back to host trees related to device security. The first repository of that group was spdm.git for the in-kernel SPDM implementation and native PCI device authentication, with Jonathan and I as co-maintainers:
https://git.kernel.org/pub/scm/linux/kernel/git/devsec/spdm.git/
Dan was added as third devsec maintainer to host tsm.git:
https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/
Jonathan and I retain access to both repositories. I cannot speak for Jonathan but I would be available to apply patches to tsm.git and forward them to Linus. It would also be possible of course to add additional maintainers to the group.
Quoted 2 lines
I've asked Jiri Pirko to work on
the PCI evidence uAPI based on his deep netlink experience
netlink isn't well suited to transport large blobs because the nlattr len is u16. (The len of the enclosing nlmsg is u32, which is sufficient.)
Previous approaches, including the one proposed by Dan in this series, work around the problem by splitting the blob into a sequence of nlattrs. I think we should instead extend the netlink protocol with 32-bit "jumbo" attributes.
I suggest we reserve bit 13 of nla_type as NLA_F_JUMBO and use the the first 4 bytes after the struct nlattr header as length (if the jumbo flag is set).
A second problem is that the size of a socket buffer's linear data is limited. Also, copying the blob into the nlmsg is a bit wasteful and we'd want zero copy instead. The solution I've come up with is to attach the pages backing the blob as fragments to the skb. It's very simple, overcomes the skb size limitation and allows for zero-copy:
https://github.com/l1k/linux/commit/6e73bb999128
That commit is from January and the time I've been able to devote to this has since been limited as my employer prioritized various AER feature gaps and fixes.
I worked on this for native PCI device authentication, which faces the same netlink blob issue as TSM-mediated authentication. Both should use the same uABI for evidence exposure. Additionally, native device authentication may be used by non-PCI buses such as ATA or SCSI. The uABI should work for those use cases as well.
Quoted 2 lines
This is the only time Europe could reasonably participate in, obviously it is not OK for China and Aus.
FWIW, I'm likewise based in Europe, as is Jonathan.
Quoted 2 lines
Quoted 2 lines
I find these calls pretty productive syncups. So
far we were not having Europe in these calls.
There is a CCC call at a US/Europe friendly time but the meeting is usually canceled. There was a single occasion in June when it wasn't canceled but very few people showed up and in particular no moderator showed up. Dan said that someone at Google is responsible for the meeting, I don't know who that is:
https://youtu.be/_cNgyfYlDGo?t=501
Obviously, one cannot separate discussing Dan's work from Dan. It is still difficult to talk about either for people who were close to him, and he was close to many. Anyone who has lost somebody dear to them knows that grieving and eventually accepting loss is a process that can take years. We're only at the beginning.
I've suggested a celebration of life for Dan at LPC, to complement the service organized by his family last Sunday. I am happy that the organizers reacted positively:
https://lwn.net/Articles/1084564/
See you there!
Lukas
Quoted 2 lines · Lukas Wunner
I've suggested a celebration of life for Dan at LPC, to complement the service organized by his family last Sunday.
We are absolutely going to do something at Plumbers. Anyone interested should keep an eye on the Plumbers schedule. Or, send me an email and I'll keep you in the loop.
Quoted 5 lines · Lukas Wunner
Dan was added as third devsec maintainer to host tsm.git:
https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/
Jonathan and I retain access to both repositories.
Fortunately they are on kernel.org so Konstantin can always recover anything, we can get to that when there is a community consensus on the new team.
In the short term Linus doesn't care so much about where the tree comes from. Personally I'd rather leave that as is until the new team is ready to put work into it. It looks like there a number of things that may need preservation..
Quoted 4 lines
Quoted 2 lines
This is the only time Europe could reasonably participate in, obviously it is not OK for China and Aus.
FWIW, I'm likewise based in Europe, as is Jonathan.
I was wondering if we should try to have two calls on opposite sides.
Quoted 1 line
Obviously, one cannot separate discussing Dan's work from Dan.
Yes, please lets all have patience and understanding
Personally I feel seeing the overall work finished and products using it in the market will be a "in memorandum" dedication for Dan.
Jason
Quoted 16 lines · On Thu, 3 Sep 2026 14:53:32 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 5 lines · Lukas Wunner
Dan was added as third devsec maintainer to host tsm.git:
https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/
Jonathan and I retain access to both repositories.
Fortunately they are on kernel.org so Konstantin can always recover anything, we can get to that when there is a community consensus on the new team.
In the short term Linus doesn't care so much about where the tree comes from. Personally I'd rather leave that as is until the new team is ready to put work into it. It looks like there a number of things that may need preservation..
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
This is the only time Europe could reasonably participate in, obviously it is not OK for China and Aus.
FWIW, I'm likewise based in Europe, as is Jonathan.
I was wondering if we should try to have two calls on opposite sides.
I've shifted, now West Coast US. However, given many things going on at the moment, my direct involvement in this may be limited for now.
I'll be pulling in some other Qualcomm folk though to help out where we can. Most like mixture of US and India based folk.
Quoted 7 lines
Quoted 2 lines
Obviously, one cannot separate discussing Dan's work from Dan.
Yes, please lets all have patience and understanding
Personally I feel seeing the overall work finished and products using it in the market will be a "in memorandum" dedication for Dan.
Fully agree with this sentiment as one of the ways to remember Dan.
Jonathan
Quoted 2 lines
Jason
Quoted 5 lines · Thu, Sep 03, 2026 at 03:36:54PM +0200, [email protected] wrote:
[+cc Jonathan, start of thread is here:
https://lore.kernel.org/all/[email protected]/
]On Wed, Sep 02, 2026 at 12:01:25PM -0300, Jason Gunthorpe wrote:
[..]
Quoted 30 lines
Quoted 3 lines
I've asked Jiri Pirko to work on
the PCI evidence uAPI based on his deep netlink experiencenetlink isn't well suited to transport large blobs because the nlattr len is u16. (The len of the enclosing nlmsg is u32, which is sufficient.)
Previous approaches, including the one proposed by Dan in this series, work around the problem by splitting the blob into a sequence of nlattrs. I think we should instead extend the netlink protocol with 32-bit "jumbo" attributes.
I suggest we reserve bit 13 of
nla_typeasNLA_F_JUMBOand use the the first 4 bytes after the struct nlattr header as length (if the jumbo flag is set).A second problem is that the size of a socket buffer's linear data is limited. Also, copying the blob into the nlmsg is a bit wasteful and we'd want zero copy instead. The solution I've come up with is to attach the pages backing the blob as fragments to the skb. It's very simple, overcomes the skb size limitation and allows for zero-copy:
https://github.com/l1k/linux/commit/6e73bb999128
That commit is from January and the time I've been able to devote to this has since been limited as my employer prioritized various AER feature gaps and fixes.
Quoted 5 lines
I worked on this for native PCI device authentication, which faces the same netlink blob issue as TSM-mediated authentication. Both should use the same uABI for evidence exposure. Additionally, native device authentication may be used by non-PCI buses such as ATA or SCSI. The uABI should work for those use cases as well.
Not sure if netlink as actually the best fit for this purpose, for large blob transfers ioctl-based iface is probably much more convenient. I'm working on a uapi framework that make the best of netlink and takes it over to a fd-based ioctl. I call it CTLV, here's a link to an early pre-RFC draft:
https://github.com/jpirko/linux_mlxsw/commits/wip_ctlv_pre_rfc_draft1/
What are the benefits over generic netlink:
- No networking in the dependency chain. No netlink sockets, no CAP_NET_ADMIN, and no network namespace semantics to reason about for a device that has none.
- One character device per registered instance, not one global family. Access control is the file: udev rules, ACLs, an fd passed into a container.
- Events per open file description, not a multicast group. Each descriptor has its own subscriptions and queue, and an overflow is reported in-band: the next read returns an event-overflow record naming the first and last sequence lost and how many.
- Large payloads are referenced, not copied. A blob attribute carries a user VA, a memfd and offset, or a dma-buf fd. Nothing big travels through the message.
- One YAML specification, everything generated from it: kernel metadata, UAPI headers, userspace bindings and the reference documentation. Introspection is answered out of the same metadata the validator enforces, so a device cannot advertise an operation it will refuse. A checked-in ABI snapshot makes any wire change a reviewable diff.
- Family inheritance. A family inherits another's operations and fills declared extension points. The effective schema is resolved per device and the chain is exposed to user. A shared core with per-driver extensions is declared once, and both attributes and operations can be extended.
- Introspection is per-device and live. The framework answers three queries on every device: the family chain, one entry per published operation, and any operation's full attribute tree with its bounds and limits.
The answer is what this device accepts right now, including what a vendor extension added to an inherited operation and whether an op is currently disabled, and op-changed events carry the generation ops-dump reports, so a dump can be ordered against a change. GETFAMILY and GETPOLICY describe a family statically - there is no device in that model to ask.
- Fragmented queries are built in, with a consistency check. A continuation carries the generation it started from, and if the answer changed underneath it is refused with ESTALE, reporting the expected and the current generation instead of reassembling a torn reply.
- Attributes are 8-byte aligned. CTLV_ALIGNTO is 8, so a 64-bit payload is naturally aligned and there is no per-family padding to remember - netlink's 4-byte alignment is why nla_put_64bit() needs an explicit NOP pad attribute.
- Cheaper round trip. A one-attribute query measured 2.6x cheaper than a genetlink one in the same guest - a debug kernel.
- Async operations using io_uring are planned as a follow-up extension.
The framework owns validation, schema resolution, blob acquisition, reply serialization and the event queues. All getters and putters are generated helpers. A family implements semantics only, and the aim is to make that hard to get wrong.
[..]
Quoted 47 lines · Fri, Sep 04, 2026 at 11:52:12AM +0200, [email protected] wrote:
Quoted 5 lines · Thu, Sep 03, 2026 at 03:36:54PM +0200, [email protected] wrote:
[+cc Jonathan, start of thread is here:
https://lore.kernel.org/all/[email protected]/
]On Wed, Sep 02, 2026 at 12:01:25PM -0300, Jason Gunthorpe wrote:
[..]
Quoted 30 lines
Quoted 3 lines
I've asked Jiri Pirko to work on
the PCI evidence uAPI based on his deep netlink experiencenetlink isn't well suited to transport large blobs because the nlattr len is u16. (The len of the enclosing nlmsg is u32, which is sufficient.)
Previous approaches, including the one proposed by Dan in this series, work around the problem by splitting the blob into a sequence of nlattrs. I think we should instead extend the netlink protocol with 32-bit "jumbo" attributes.
I suggest we reserve bit 13 of
nla_typeasNLA_F_JUMBOand use the the first 4 bytes after the struct nlattr header as length (if the jumbo flag is set).A second problem is that the size of a socket buffer's linear data is limited. Also, copying the blob into the nlmsg is a bit wasteful and we'd want zero copy instead. The solution I've come up with is to attach the pages backing the blob as fragments to the skb. It's very simple, overcomes the skb size limitation and allows for zero-copy:
https://github.com/l1k/linux/commit/6e73bb999128
That commit is from January and the time I've been able to devote to this has since been limited as my employer prioritized various AER feature gaps and fixes.
Quoted 5 lines
I worked on this for native PCI device authentication, which faces the same netlink blob issue as TSM-mediated authentication. Both should use the same uABI for evidence exposure. Additionally, native device authentication may be used by non-PCI buses such as ATA or SCSI. The uABI should work for those use cases as well.
Quoted 8 lines
Not sure if netlink as actually the best fit for this purpose, for large blob transfers ioctl-based iface is probably much more convenient. I'm working on a uapi framework that make the best of netlink and takes it over to a fd-based ioctl. I call it CTLV, here's a link to an early pre-RFC draft:
https://github.com/jpirko/linux_mlxsw/commits/wip_ctlv_pre_rfc_draft1/
[..]
Following up on this, I have a very early draft of an attestation framework here:
https://github.com/jpirko/linux_mlxsw/commits/wip_attestation_pre_rfc_draft1/
It introduces a provider-neutral, fd-based interface for evidence retrieval, userspace verdicts tied to exact device/evidence generations, measurement registers and their journal, events, and device-security state transitions. Large evidence is written directly to referenced buffers instead of being split across Netlink messages.
For this series, the intent is to replace the device-evidence Generic Netlink UAPI and the draft PCI/TSM evidence-accept UAPI. It does not replace PCI/TSM connect/disconnect or lock/unlock, nor the underlying device-trust, SPDM/IDE/TDISP, MMIO, or DMA machinery.
The branch currently contains the core, a simulation provider with tests, and a TDX provider demonstrating the provider boundary. The PCI/TSM provider is not implemented yet.
Is there a way for this to be posted as its own patch set so we can provide feedback inline? Also do we have any other potential uses for your proposed ctlv mechanism or is attestation the only user at the moment?
Quoted 74 lines · Jiri Pirko
Quoted 43 lines · Fri, Sep 04, 2026 at 11:52:12AM +0200, [email protected] wrote:
Quoted 5 lines · Thu, Sep 03, 2026 at 03:36:54PM +0200, [email protected] wrote:
[+cc Jonathan, start of thread is here:
https://lore.kernel.org/all/[email protected]/
]On Wed, Sep 02, 2026 at 12:01:25PM -0300, Jason Gunthorpe wrote:
[..]
Quoted 28 lines
Quoted 2 lines
I've asked Jiri Pirko to work on
the PCI evidence uAPI based on his deep netlink experiencenetlink isn't well suited to transport large blobs because the nlattr len is u16. (The len of the enclosing nlmsg is u32, which is sufficient.)
Previous approaches, including the one proposed by Dan in this series, work around the problem by splitting the blob into a sequence of nlattrs. I think we should instead extend the netlink protocol with 32-bit "jumbo" attributes.
I suggest we reserve bit 13 of
nla_typeasNLA_F_JUMBOand use the the first 4 bytes after the struct nlattr header as length (if the jumbo flag is set).A second problem is that the size of a socket buffer's linear data is limited. Also, copying the blob into the nlmsg is a bit wasteful and we'd want zero copy instead. The solution I've come up with is to attach the pages backing the blob as fragments to the skb. It's very simple, overcomes the skb size limitation and allows for zero-copy:
https://github.com/l1k/linux/commit/6e73bb999128
That commit is from January and the time I've been able to devote to this has since been limited as my employer prioritized various AER feature gaps and fixes.
Quoted 5 lines
I worked on this for native PCI device authentication, which faces the same netlink blob issue as TSM-mediated authentication. Both should use the same uABI for evidence exposure. Additionally, native device authentication may be used by non-PCI buses such as ATA or SCSI. The uABI should work for those use cases as well.
Quoted 8 lines
Not sure if netlink as actually the best fit for this purpose, for large blob transfers ioctl-based iface is probably much more convenient. I'm working on a uapi framework that make the best of netlink and takes it over to a fd-based ioctl. I call it CTLV, here's a link to an early pre-RFC draft:
https://github.com/jpirko/linux_mlxsw/commits/wip_ctlv_pre_rfc_draft1/
[..]
Following up on this, I have a very early draft of an attestation framework here:
https://github.com/jpirko/linux_mlxsw/commits/wip_attestation_pre_rfc_draft1/
It introduces a provider-neutral, fd-based interface for evidence retrieval, userspace verdicts tied to exact device/evidence generations, measurement registers and their journal, events, and device-security state transitions. Large evidence is written directly to referenced buffers instead of being split across Netlink messages.
For this series, the intent is to replace the device-evidence Generic Netlink UAPI and the draft PCI/TSM evidence-accept UAPI. It does not replace PCI/TSM connect/disconnect or lock/unlock, nor the underlying device-trust, SPDM/IDE/TDISP, MMIO, or DMA machinery.
The branch currently contains the core, a simulation provider with tests, and a TDX provider demonstrating the provider boundary. The PCI/TSM provider is not implemented yet.
Quoted 3 lines · Wed, Sep 30, 2026 at 04:14:00PM +0200, [email protected] wrote:
Is there a way for this to be posted as its own patch set so we can provide feedback inline? Also do we have any other potential uses for your proposed ctlv mechanism or is attestation the only user at the moment?
Will do that, still need to do some work on it before though. So far I focus on attestation with ctlv, as that is my immediate target. But the mechanism is quite generic, easy to be used by anyone.
Quoted 76 lines · Jiri Pirko
Quoted 74 lines
Quoted 43 lines · Fri, Sep 04, 2026 at 11:52:12AM +0200, [email protected] wrote:
Quoted 5 lines · Thu, Sep 03, 2026 at 03:36:54PM +0200, [email protected] wrote:
[+cc Jonathan, start of thread is here:
https://lore.kernel.org/all/[email protected]/
]On Wed, Sep 02, 2026 at 12:01:25PM -0300, Jason Gunthorpe wrote:
[..]
Quoted 28 lines
Quoted 2 lines
I've asked Jiri Pirko to work on
the PCI evidence uAPI based on his deep netlink experiencenetlink isn't well suited to transport large blobs because the nlattr len is u16. (The len of the enclosing nlmsg is u32, which is sufficient.)
Previous approaches, including the one proposed by Dan in this series, work around the problem by splitting the blob into a sequence of nlattrs. I think we should instead extend the netlink protocol with 32-bit "jumbo" attributes.
I suggest we reserve bit 13 of
nla_typeasNLA_F_JUMBOand use the the first 4 bytes after the struct nlattr header as length (if the jumbo flag is set).A second problem is that the size of a socket buffer's linear data is limited. Also, copying the blob into the nlmsg is a bit wasteful and we'd want zero copy instead. The solution I've come up with is to attach the pages backing the blob as fragments to the skb. It's very simple, overcomes the skb size limitation and allows for zero-copy:
https://github.com/l1k/linux/commit/6e73bb999128
That commit is from January and the time I've been able to devote to this has since been limited as my employer prioritized various AER feature gaps and fixes.
Quoted 5 lines
I worked on this for native PCI device authentication, which faces the same netlink blob issue as TSM-mediated authentication. Both should use the same uABI for evidence exposure. Additionally, native device authentication may be used by non-PCI buses such as ATA or SCSI. The uABI should work for those use cases as well.
Quoted 8 lines
Not sure if netlink as actually the best fit for this purpose, for large blob transfers ioctl-based iface is probably much more convenient. I'm working on a uapi framework that make the best of netlink and takes it over to a fd-based ioctl. I call it CTLV, here's a link to an early pre-RFC draft:
https://github.com/jpirko/linux_mlxsw/commits/wip_ctlv_pre_rfc_draft1/
[..]
Following up on this, I have a very early draft of an attestation framework here:
https://github.com/jpirko/linux_mlxsw/commits/wip_attestation_pre_rfc_draft1/
It introduces a provider-neutral, fd-based interface for evidence retrieval, userspace verdicts tied to exact device/evidence generations, measurement registers and their journal, events, and device-security state transitions. Large evidence is written directly to referenced buffers instead of being split across Netlink messages.
For this series, the intent is to replace the device-evidence Generic Netlink UAPI and the draft PCI/TSM evidence-accept UAPI. It does not replace PCI/TSM connect/disconnect or lock/unlock, nor the underlying device-trust, SPDM/IDE/TDISP, MMIO, or DMA machinery.
The branch currently contains the core, a simulation provider with tests, and a TDX provider demonstrating the provider boundary. The PCI/TSM provider is not implemented yet.
Was there a decision made for a new meeting time? If not are we still going to be meeting this week at Dan's original meeting time?
Dave
Quoted 58 lines · Alexey Kardashevskiy
Quoted 11 lines · Xu Yilun
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this. Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?
I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Quoted 6 lines
Hi Ankit & Jason,
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1
Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1
Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)Scenario B:
Seattle (PDT, UTC-7) 18:00 – 19:00 Day 1 Atlantic Time (ADT, UTC-3) 22:00 – 23:00 Day 1 Berlin (CEST, UTC+2) 03:00 – 04:00 Day 2 India (IST, UTC+5:30) 06:30 – 07:30 Day 2 China (CST, UTC+8) 09:00 – 10:00 Day 2 Sydney (AEST, UTC+10) 11:00 – 12:00 Day 2Scenario C (which is the current schedule):
Seattle (PDT, UTC-7) 20:00 – 21:00 Thursday, Sep 3 Atlantic Time Zone (ADT, UTC-3) 00:00 – 01:00 Friday, Sep 4 Berlin (CEST, UTC+2) 05:00 – 06:00 Day 2 India (IST, UTC+5:30) 08:30 – 09:30 Friday, Sep 4 China (CST, UTC+8) 11:00 – 12:00 Friday, Sep 4 Sydney (AEST, UTC+10) 13:00 – 14:00 Friday, Sep 4but if we want to change the time - who do we contact? Or set up a new call (again, how)?
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
Alternatively, wait until the LPC in a month but I suspect not everybody is going to be there.
Alexey Kardashevskiy
maintainerQuoted 4 lines · David
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Was there a decision made for a new meeting time? If not are we still going to be meeting this week at Dan's original meeting time?
No new decision is made yet.
I dislike the idea of 2 separate calls, unlikely that both times are outside of my 00:00-06:00 so I am going to miss one of them constantly.
Here we voted: https://crab.fit/iommufd-tsm-call-593432
7/7 - no overlap. 6/7 - there are 2 options, 1hr each but I'll need Lu Baolu or Xu Yilun to cooperate :)
So this friday call is likely to happen at the same old time. Thanks,
Quoted 67 lines
Dave
Quoted 64 lines · Alexey Kardashevskiy
Quoted 12 lines · Xu Yilun
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Xu Yilun
Glad to see you all here to make progress! Intel can also help on this.
Can we have a meeting to discuss on these topics? As Alexey mentioned, previously we had a bi-weekly call, perhaps use that for our first meeting? Or what time works best for everyone?I prefer we defer such community discussions till September, that seems appropriate given some dates I've seen.
Quoted 6 lines
Hi Ankit & Jason,
Do we have a plan to schedule a meeting to discuss next steps?
If so, we can figure out a specific time on the Discord channel.
https://discord.gg/KKqC438cSo I've been trying to find a time slot. Say, keep it on Thursdays (USA) (kinda short notice), bi-weekly, we can do:
Scenario A:
Seattle (PDT, UTC-7) 07:00 – 08:00 Day 1 Atlantic Time (ADT, UTC-3) 11:00 – 12:00 Day 1 Berlin (CEST, UTC+2) 16:00 – 17:00 Day 1
India (IST, UTC+5:30) 19:30 – 20:30 Day 1
China (CST, UTC+8) 22:00 – 23:00 Day 1
Sydney (AEST, UTC+10) 00:00 – 01:00 Day 2 (I can do it once in 2 weeks)Scenario B:
Seattle (PDT, UTC-7) 18:00 – 19:00 Day 1 Atlantic Time (ADT, UTC-3) 22:00 – 23:00 Day 1 Berlin (CEST, UTC+2) 03:00 – 04:00 Day 2 India (IST, UTC+5:30) 06:30 – 07:30 Day 2 China (CST, UTC+8) 09:00 – 10:00 Day 2 Sydney (AEST, UTC+10) 11:00 – 12:00 Day 2Scenario C (which is the current schedule):
Seattle (PDT, UTC-7) 20:00 – 21:00 Thursday, Sep 3 Atlantic Time Zone (ADT, UTC-3) 00:00 – 01:00 Friday, Sep 4 Berlin (CEST, UTC+2) 05:00 – 06:00 Day 2 India (IST, UTC+5:30) 08:30 – 09:30 Friday, Sep 4 China (CST, UTC+8) 11:00 – 12:00 Friday, Sep 4 Sydney (AEST, UTC+10) 13:00 – 14:00 Friday, Sep 4but if we want to change the time - who do we contact? Or set up a new call (again, how)?
Please, comment. I find these calls pretty productive syncups. So far we were not having Europe in these calls.
Alternatively, wait until the LPC in a month but I suspect not everybody is going to be there.
Signature · 1 line
Alexey
Alexey Kardashevskiy
maintainerQuoted 5 lines · Alexey Kardashevskiy
Quoted 2 lines · David
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 8 lines
Quoted 2 lines
Was there a decision made for a new meeting time? If not are we still going to be meeting this week at Dan's original meeting time?
No new decision is made yet.
I dislike the idea of 2 separate calls, unlikely that both times are outside of my 00:00-06:00 so I am going to miss one of them constantly.
Here we voted: https://crab.fit/iommufd-tsm-call-593432
For those who lost the link - here is a friendly reminder to vote. Thanks,
Quoted 4 lines
7/7 - no overlap. 6/7 - there are 2 options, 1hr each but I'll need Lu Baolu or Xu Yilun to cooperate :)
So this friday call is likely to happen at the same old time. Thanks,
Signature · 1 line
Alexey
Quoted 1 line
Any volunteers? Yilun, Aneesh, Jason?
Hi Alexey, thanks for bringing this up.
I was working internally with Dan for the CCA support and will continue on his work.
Thanks
Ankit Agrawal
Quoted 6 lines
The direction appears to be set, imho the immediate TODO list is:
- address "not-netlink-y enough" concerns
- teachpci_iomap()to map mixed private+shared BARs (ones which contain MSIX for the starter)
- always call pci_tsm_ops::enable_dma/disable_dma
- ditch the module trust parameter?
- add a knob to enable/disable DMA and keep TDI in RUN
Agreed, couple of items I'll add.
- Donald's suggestion for subsystem name like spdm, tsm or tsm-device (uapi-breaking)
- Fix bugs such as missing pci_tsm_disable_dma() in pci_dma_cleanup()