dma-mapping: Track shared DMA state through direct, pool and swiotlb paths

This series tracks confidential-computing shared DMA state through the dma-direct, dma-pool, and swiotlb paths so that encrypted and decrypted DMA buffers are handled consistently.

Today, the direct DMA path mostly relies on force_dma_unencrypted() for shared/decrypted buffer handling. This series consolidates the force_dma_unencrypted() checks in the top-level functions and ensures that the remaining DMA interfaces use DMA attributes to make the correct decisions.

The series separates mapping and allocation state:
- DMA_ATTR_CC_SHARED describes the DMA address attribute requested for a mapping. It tells the DMA mapping path that the DMA address must target shared/decrypted memory.
- __DMA_ATTR_ALLOC_CC_SHARED is an internal DMA-mapping attribute used only by allocation paths after the DMA core decides that the backing pages must be allocated as shared/decrypted memory.

The series:
- moves swiotlb-backed allocations out of __dma_direct_alloc_pages(),
- uses __DMA_ATTR_ALLOC_CC_SHARED through the dma-direct alloc/free paths
- teaches the atomic DMA pools to track encrypted versus decrypted state
- tracks swiotlb pool encryption state and enforces strict pool selection
- centralizes encrypted/decrypted pgprot handling in dma_pgprot() using DMA attributes
- passes DMA attributes down to dma_capable() so capability checks can validate whether the selected DMA address encoding matches DMA_ATTR_CC_SHARED
- makes dma_direct_map_phys() choose the DMA address encoding from DMA_ATTR_CC_SHARED and fall back to swiotlb when a shared DMA request cannot use the direct mapping, which lets arm64 and x86 CCA guests stop relying on SWIOTLB_FORCE for DMA mappings
- use the selected swiotlb pool state to derive the returned DMA address
- reports CC_ATTR_GUEST_MEM_ENCRYPT for arm64 Realms, powerpc secure guests, and s390 protected virtualization guests.

Dependency:
This series depends on the pKVM changes posted at:
https://lore.kernel.org/all/[email protected]

Please merge this series only after the pKVM changes above are merged. Otherwise pKVM will be broken.

Changes since v7:
https://lore.kernel.org/all/[email protected]
* Rebased onto dma-mapping-for-next
* Prepared the series on top of the prerequisite pKVM changes, resolving conflicts so it can be applied directly to the pKVM topic branch once ready.
https://git.gitlab.arm.com/linux-arm/linux-cca/-/commits/scratch/pkvm/testing
* Added comments documenting possible follow-up improvements for CC_SHARED atomic pools and physical-address-based pool freeing.
* Retained virtual-address-based pool freeing when CONFIG_DMA_DIRECT_REMAP is disabled.
* Applied pgprot_decrypted() only when expanding a CC_SHARED atomic pool.

Changes since v6:
https://lore.kernel.org/all/[email protected]
* Rebase onto the latest kernel.
* Add __DMA_ATTR_ALLOC_CC_SHARED for allocation paths. DMA_ATTR_CC_SHARED is now used to describe the requested DMA mapping address attribute, while __DMA_ATTR_ALLOC_CC_SHARED is used internally when allocating shared/decrypted backing pages.
* Report CC_ATTR_GUEST_MEM_ENCRYPT for arm64 Realms, powerpc secure guests, and s390 protected virtualization guests.
* Add CC_ATTR_HOST_MEM_ENCRYPT and swiotlb=force fixes.

Changes since v5:
https://lore.kernel.org/all/[email protected]
* Add Tested-by
* Drop the pKVM patch, which has now been posted separately:
https://lore.kernel.org/all/[email protected]
* Remove the DO_NOT_MERGE tag from the s390 change.
* Add a patch to drop the SWIOTLB_FORCE flag.
* Rebase onto the latest kernel.

Changes since v4:
https://lore.kernel.org/all/[email protected]
* Add new patches based on Sashiko review:
swiotlb: Preserve allocation virtual address for dynamic pools
dma: free atomic pool pages by physical address
dma: swiotlb: handle set_memory_decrypted() failures
dma: swiotlb: free dynamic pools from process context
iommu/dma: Check atomic pool allocation result directly
* Include pKVM and s390 changes as dependent patches. These are not yet ready to merge and are waiting for subsystem testing feedback.
* Drop the AMD GART patch because it requires wider testing.
* Update swiotlb_tbl_map_single() to take attrs by reference.
* Switch swiotlb_free() to use rcu_work.
* Avoid calling swiotlb_find_pool() multiple times in the free path.
* Make DMA_ATTR_MMIO imply DMA_ATTR_CC_SHARED for devices requiring unencrypted DMA.

Changes from v3:
https://lore.kernel.org/all/[email protected]
* Handle DMA_ATTR_MMIO correctly in dma_direct_map_phys()
* Address most of sashiko review
* Rebase to latest kernel
* drop SWIOTLB_FORCE for s390 and powerpc secure guest.

Changes from v2:
https://lore.kernel.org/all/[email protected]
* pass attrs to dma_capable() and update direct, swiotlb, Xen swiotlb, and x86 GART paths so the capability checks see the DMA address attr value DMA_ATTR_CC_SHARED.
* rework dma_direct_map_phys() so DMA_ATTR_CC_SHARED selects phys_to_dma_unencrypted() while the default path uses phys_to_dma_encrypted(), with swiotlb fallback when the requested shared/private state cannot be satisfied by a direct DMA address.
* stop relying on SWIOTLB_FORCE for arm64 and x86 CC guest DMA mappings; swiotlb is still enabled there, but shared mappings is now selected through the generic dma_direct_map_phys()/dma_capable() decision instead of a global force-bounce flag.

Changes from v1:
https://lore.kernel.org/all/[email protected]
* rebased to latest kernel (change from DMA_ATTR_CC_DECRYPTED -> DMA_ATTR_CC_SHARED)
* update the alloc path so DMA_ATTR_CC_SHARED is not a caller-visible attribute.

Aneesh Kumar K.V (Arm) (23):
dma-direct: return struct page from dma_direct_alloc_from_pool()
dma-pool: fix page leak in atomic_pool_expand() cleanup iommu/dma: Check atomic pool allocation result directly
dma: free atomic pool pages by physical address
swiotlb: Preserve allocation virtual address for dynamic pools
s390: Expose protected virtualization through cc_platform_has()
dma-direct: swiotlb: handle swiotlb alloc/free outside
__dma_direct_alloc_pages
coco: arm64: s390: powerpc: Mark secure guests with
CC_ATTR_GUEST_MEM_ENCRYPT
dma-mapping: Add internal shared allocation attribute
dma-direct: use __DMA_ATTR_ALLOC_CC_SHARED in alloc/free paths
dma-pool: track decrypted atomic pools and select them via attrs
dma: swiotlb: pass mapping attributes by reference
dma: swiotlb: track pool encryption state and honor DMA_ATTR_CC_SHARED
dma-mapping: make dma_pgprot() honor __DMA_ATTR_ALLOC_CC_SHARED
dma-direct: pass attrs to dma_capable() for DMA_ATTR_CC_SHARED checks
dma-direct: Move dma_direct_map_phys() to dma/direct.c
dma-direct: make dma_direct_map_phys() honor DMA_ATTR_CC_SHARED
dma-direct: set decrypted flag for remapped DMA allocations
dma-direct: select DMA address encoding from
__DMA_ATTR_ALLOC_CC_SHARED
dma-direct: rename ret to cpu_addr in alloc helpers
dma: swiotlb: free dynamic pools from process context
dma: swiotlb: handle set_memory_decrypted() failures
swiotlb: remove unused SWIOTLB_FORCE flag

 Documentation/core-api/dma-attributes.rst    |  29 ++
 arch/arm64/mm/init.c                         |   5 +-
 arch/powerpc/platforms/pseries/cc_platform.c |   1 +
 arch/powerpc/platforms/pseries/svm.c         |   2 +-
 arch/s390/Kconfig                            |   1 +
 arch/s390/mm/init.c                          |  17 +-
 arch/x86/kernel/amd_gart_64.c                |  30 +-
 arch/x86/kernel/pci-dma.c                    |   4 +-
 drivers/iommu/dma-iommu.c                    |  20 +-
 drivers/xen/swiotlb-xen.c                    |   8 +-
 include/linux/dma-direct.h                   |  20 +-
 include/linux/dma-map-ops.h                  |   3 +-
 include/linux/dma-mapping.h                  |   8 +
 include/linux/swiotlb.h                      |  25 +-
 include/trace/events/dma.h                   |   3 +-
 kernel/dma/direct.c                          | 326 ++++++++++++++-----
 kernel/dma/direct.h                          |  56 +---
 kernel/dma/mapping.c                         |  25 +-
 kernel/dma/pool.c                            | 237 ++++++++++----
 kernel/dma/swiotlb.c                         | 292 +++++++++++++----
 20 files changed, 810 insertions(+), 302 deletions(-)

2.43.0