Skip to contentSkip to conversation

vfio/nvgrace-gpu: Convey kvm that the device is wc safe

PATCH In review [email protected] 3 participants 6 messages Feb 28, 2024 – Feb 29, 2024

Subsystems (4)
lore
Share & tools
lore

[email protected]

author maintainer
#0 lore raw

PATCH 1/1 vfio/nvgrace-gpu: Convey kvm that the device is wc safe

From: Ankit Agrawal <[email protected]>

The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].

Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.

KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by default. The fake device BARs thus displays DEVICE_nGnRE behavior in the VM.

The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].

S1           |  S2           | Result
NORMAL_WB    |  NORMAL_NC    | NORMAL_NC
NORMAL_WT    |  NORMAL_NC    | NORMAL_NC
NORMAL_NC    |  NORMAL_NC    | NORMAL_NC
NORMAL_WB    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_WT    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_NC    |  DEVICE_nGnRE | DEVICE_nGnRE

Recently a change was added that modifies this default behavior and make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag VM_ALLOW_ANY_UNCACHED is set. Setting S2 as MT_S2_FWB_NORMAL_NC provides the desired behavior (uncached, unaligned access) for resmem.

Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].

To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set the VM_ALLOW_ANY_UNCACHED flag in the VMA.

Applied over next-20240227.
base-commit: 22ba90670a51

Cc Alex Williamson <[email protected]>
Cc Kevin Tian <[email protected]>
Cc Jason Gunthorpe <[email protected]>
Cc Vikram Sethi <[email protected]>
Cc Zhi Wang <[email protected]>
---
 drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
drivers/vfio/pci/nvgrace-gpu/main.c +18 -0
index 25814006352d..5539c9057212 100644
@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
+	/*
+	 * The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64,
+	 * allowing KVM stage 2 device mapping attributes to use Normal-NC
+	 * rather than DEVICE_nGnRE, which allows guest mappings
+	 * supporting write-combining attributes (WC). This also
+	 * unlocks memory-like operations such as unaligned accesses.
+	 * This setting suits the fake BARs as they are expected to
+	 * demonstrate such properties within the guest.
+	 *
+	 * ARM does not architecturally guarantee this is safe, and indeed
+	 * some MMIO regions like the GICv2 VCPU interface can trigger
+	 * uncontained faults if Normal-NC is used. The nvgrace-gpu
+	 * however is safe in that the platform guarantees that no
+	 * action taken on the MMIO mapping can trigger an uncontained
+	 * failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags.
+	 */
+	vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
 	return 0;
 }
 
Signature · 1 line
2.34.1
#1 lore raw
Quoted 83 lines · On Wed, 28 Feb 2024 19:48:01 +0000 <[email protected]> wrote:

From: Ankit Agrawal <[email protected]>

The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].

Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.

KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by default. The fake device BARs thus displays DEVICE_nGnRE behavior in the VM.

The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].

S1           |  S2           | Result
NORMAL_WB    |  NORMAL_NC    | NORMAL_NC
NORMAL_WT    |  NORMAL_NC    | NORMAL_NC
NORMAL_NC    |  NORMAL_NC    | NORMAL_NC
NORMAL_WB    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_WT    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_NC    |  DEVICE_nGnRE | DEVICE_nGnRE

Recently a change was added that modifies this default behavior and make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag VM_ALLOW_ANY_UNCACHED is set. Setting S2 as MT_S2_FWB_NORMAL_NC provides the desired behavior (uncached, unaligned access) for resmem.

Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].

To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set the VM_ALLOW_ANY_UNCACHED flag in the VMA.

Applied over next-20240227.
base-commit: 22ba90670a51

Cc Alex Williamson <[email protected]>
Cc Kevin Tian <[email protected]>
Cc Jason Gunthorpe <[email protected]>
Cc Vikram Sethi <[email protected]>
Cc Zhi Wang <[email protected]>
---
 drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
drivers/vfio/pci/nvgrace-gpu/main.c +11 -0
index 25814006352d..5539c9057212 100644
@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
+	/*
+	 * The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64,
+	 * allowing KVM stage 2 device mapping attributes to use Normal-NC
+	 * rather than DEVICE_nGnRE, which allows guest mappings
+	 * supporting write-combining attributes (WC). This also
+	 * unlocks memory-like operations such as unaligned accesses.
+	 * This setting suits the fake BARs as they are expected to
+	 * demonstrate such properties within the guest.
+	 *
+	 * ARM does not architecturally guarantee this is safe, and indeed
+	 * some MMIO regions like the GICv2 VCPU interface can trigger

drivers/vfio/pci/nvgrace-gpu/main.c static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, · line 204

Quoted 10 lines
drivers/vfio/pci/nvgrace-gpu/main.c +7 -0
+	 * uncontained faults if Normal-NC is used. The nvgrace-gpu
+	 * however is safe in that the platform guarantees that no
+	 * action taken on the MMIO mapping can trigger an uncontained
+	 * failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags.
+	 */
+	vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
 	return 0;
 }
 

The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,

Alex

#2 lore raw
Quoted 94 lines · Alex Williamson
Quoted 91 lines · On Wed, 28 Feb 2024 19:48:01 +0000 <[email protected]> wrote:

From: Ankit Agrawal <[email protected]>

The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].

Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.

KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by default. The fake device BARs thus displays DEVICE_nGnRE behavior in the VM.

The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].

S1           |  S2           | Result
NORMAL_WB    |  NORMAL_NC    | NORMAL_NC
NORMAL_WT    |  NORMAL_NC    | NORMAL_NC
NORMAL_NC    |  NORMAL_NC    | NORMAL_NC
NORMAL_WB    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_WT    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_NC    |  DEVICE_nGnRE | DEVICE_nGnRE

Recently a change was added that modifies this default behavior and make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag VM_ALLOW_ANY_UNCACHED is set. Setting S2 as MT_S2_FWB_NORMAL_NC provides the desired behavior (uncached, unaligned access) for resmem.

Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].

To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set the VM_ALLOW_ANY_UNCACHED flag in the VMA.

Applied over next-20240227.
base-commit: 22ba90670a51

Cc Alex Williamson <[email protected]>
Cc Kevin Tian <[email protected]>
Cc Jason Gunthorpe <[email protected]>
Cc Vikram Sethi <[email protected]>
Cc Zhi Wang <[email protected]>
---
 drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
drivers/vfio/pci/nvgrace-gpu/main.c +18 -0
index 25814006352d..5539c9057212 100644
@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
+	/*
+	 * The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64,
+	 * allowing KVM stage 2 device mapping attributes to use Normal-NC
+	 * rather than DEVICE_nGnRE, which allows guest mappings
+	 * supporting write-combining attributes (WC). This also
+	 * unlocks memory-like operations such as unaligned accesses.
+	 * This setting suits the fake BARs as they are expected to
+	 * demonstrate such properties within the guest.
+	 *
+	 * ARM does not architecturally guarantee this is safe, and indeed
+	 * some MMIO regions like the GICv2 VCPU interface can trigger
+	 * uncontained faults if Normal-NC is used. The nvgrace-gpu
+	 * however is safe in that the platform guarantees that no
+	 * action taken on the MMIO mapping can trigger an uncontained
+	 * failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags.
+	 */
+	vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
 	return 0;
Quoted 8 lines

drivers/vfio/pci/nvgrace-gpu/main.c static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, · line 204

Quoted 2 lines
drivers/vfio/pci/nvgrace-gpu/main.c +0 -0
 }
 

The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,

Yeah, I sugged to fold that hunk into this:

        if (index == RESMEM_REGION_INDEX)
                vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);

So it makes more sense. VM_ALLOW_ANY_UNCACHED shouldn't be used on the cachable mapping. The comment should be more specific to this driver and not so generic:

/*
* nvgrace has no issue with uncontained failures on NORMAL_NC
* access. Tell KVM to open up guest usage of NORMAL_NC for this mapping. */

Jason

author maintainer
#3 lore raw
Quoted 34 lines
Quoted 23 lines
Quoted 14 lines


+   /*
+    * The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64, +    * allowing KVM stage 2 device mapping attributes to use Normal-NC +    * rather than DEVICE_nGnRE, which allows guest mappings +    * supporting write-combining attributes (WC). This also +    * unlocks memory-like operations such as unaligned accesses. +    * This setting suits the fake BARs as they are expected to +    * demonstrate such properties within the guest.
+    *
+    * ARM does not architecturally guarantee this is safe, and indeed +    * some MMIO regions like the GICv2 VCPU interface can trigger +    * uncontained faults if Normal-NC is used. The nvgrace-gpu +    * however is safe in that the platform guarantees that no

Quoted 8 lines

+    * action taken on the MMIO mapping can trigger an uncontained +    * failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags. +    */
+   vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
      return 0;
  }

Quoted 5 lines

The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access?  Thanks,

Yeah, I sugged to fold that hunk into this:

        if (index == RESMEM_REGION_INDEX)
                vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);

Quoted 8 lines

So it makes more sense. VM_ALLOW_ANY_UNCACHED shouldn't be used on the cachable mapping. The comment should be more specific to this driver and not so generic:

/*
* nvgrace has no issue with uncontained failures on NORMAL_NC
 * access. Tell KVM to open up guest usage of NORMAL_NC for this mapping. */

Sure, I will restrict this to resmem. Also will update the comment accordingly.

Quoted 2 lines

Jason

#4 lore raw
In reply to Alex Williamson
Quoted 94 lines · On Thu, 29 Feb 2024 08:56:39 -0700 Alex Williamson…
Quoted 91 lines · On Wed, 28 Feb 2024 19:48:01 +0000 <[email protected]> wrote:

From: Ankit Agrawal <[email protected]>

The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].

Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.

KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by default. The fake device BARs thus displays DEVICE_nGnRE behavior in the VM.

The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].

S1           |  S2           | Result
NORMAL_WB    |  NORMAL_NC    | NORMAL_NC
NORMAL_WT    |  NORMAL_NC    | NORMAL_NC
NORMAL_NC    |  NORMAL_NC    | NORMAL_NC
NORMAL_WB    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_WT    |  DEVICE_nGnRE | DEVICE_nGnRE
NORMAL_NC    |  DEVICE_nGnRE | DEVICE_nGnRE

Recently a change was added that modifies this default behavior and make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag VM_ALLOW_ANY_UNCACHED is set. Setting S2 as MT_S2_FWB_NORMAL_NC provides the desired behavior (uncached, unaligned access) for resmem.

Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].

To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set the VM_ALLOW_ANY_UNCACHED flag in the VMA.

Applied over next-20240227.
base-commit: 22ba90670a51

Cc Alex Williamson <[email protected]>
Cc Kevin Tian <[email protected]>
Cc Jason Gunthorpe <[email protected]>
Cc Vikram Sethi <[email protected]>
Cc Zhi Wang <[email protected]>
---
 drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
drivers/vfio/pci/nvgrace-gpu/main.c +18 -0
index 25814006352d..5539c9057212 100644
@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
+	/*
+	 * The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64,
+	 * allowing KVM stage 2 device mapping attributes to use Normal-NC
+	 * rather than DEVICE_nGnRE, which allows guest mappings
+	 * supporting write-combining attributes (WC). This also
+	 * unlocks memory-like operations such as unaligned accesses.
+	 * This setting suits the fake BARs as they are expected to
+	 * demonstrate such properties within the guest.
+	 *
+	 * ARM does not architecturally guarantee this is safe, and indeed
+	 * some MMIO regions like the GICv2 VCPU interface can trigger
+	 * uncontained faults if Normal-NC is used. The nvgrace-gpu
+	 * however is safe in that the platform guarantees that no
+	 * action taken on the MMIO mapping can trigger an uncontained
+	 * failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags.
+	 */
+	vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
 	return 0;
Quoted 8 lines

drivers/vfio/pci/nvgrace-gpu/main.c static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, · line 204

Quoted 2 lines
drivers/vfio/pci/nvgrace-gpu/main.c +0 -0
 }
 

The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,

Also, this is setting the vma flag *after* the call to remap_pfn_range(), which seems quite sketchy. Thanks,

Alex

author maintainer
#5 lore raw
Quoted 22 lines
Quoted 22 lines
Quoted 21 lines

+   /*
+    * The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64, +    * allowing KVM stage 2 device mapping attributes to use Normal-NC +    * rather than DEVICE_nGnRE, which allows guest mappings +    * supporting write-combining attributes (WC). This also +    * unlocks memory-like operations such as unaligned accesses. +    * This setting suits the fake BARs as they are expected to +    * demonstrate such properties within the guest.
+    *
+    * ARM does not architecturally guarantee this is safe, and indeed +    * some MMIO regions like the GICv2 VCPU interface can trigger +    * uncontained faults if Normal-NC is used. The nvgrace-gpu +    * however is safe in that the platform guarantees that no +    * action taken on the MMIO mapping can trigger an uncontained +    * failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags. +    */
+   vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
     return 0;
  }

Quoted 8 lines
Quoted 5 lines

The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access?  Thanks,

Also, this is setting the vma flag *after* the call to remap_pfn_range(), which seems quite sketchy.  Thanks,

I will move it to the block that Jason pointed.

Quoted 1 line

Alex