vfio/nvgrace-gpu: Convey kvm that the device is wc safe
[email protected]
maintainerPATCH 1/1 vfio/nvgrace-gpu: Convey kvm that the device is wc safe
From: Ankit Agrawal <[email protected]>
The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].
Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.
KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by default. The fake device BARs thus displays DEVICE_nGnRE behavior in the VM.
The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].
S1 | S2 | Result NORMAL_WB | NORMAL_NC | NORMAL_NC NORMAL_WT | NORMAL_NC | NORMAL_NC NORMAL_NC | NORMAL_NC | NORMAL_NC NORMAL_WB | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_WT | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_NC | DEVICE_nGnRE | DEVICE_nGnRE
Recently a change was added that modifies this default behavior and make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag VM_ALLOW_ANY_UNCACHED is set. Setting S2 as MT_S2_FWB_NORMAL_NC provides the desired behavior (uncached, unaligned access) for resmem.
Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].
To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set the VM_ALLOW_ANY_UNCACHED flag in the VMA.
Applied over next-20240227.
base-commit: 22ba90670a51
--- drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
drivers/vfio/pci/nvgrace-gpu/main.c +18 -0
@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, vma->vm_pgoff = start_pfn; +/* +* The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64, +* allowing KVM stage 2 device mapping attributes to use Normal-NC +* rather than DEVICE_nGnRE, which allows guest mappings +* supporting write-combining attributes (WC). This also +* unlocks memory-like operations such as unaligned accesses. +* This setting suits the fake BARs as they are expected to +* demonstrate such properties within the guest. +* +* ARM does not architecturally guarantee this is safe, and indeed +* some MMIO regions like the GICv2 VCPU interface can trigger +* uncontained faults if Normal-NC is used. The nvgrace-gpu +* however is safe in that the platform guarantees that no +* action taken on the MMIO mapping can trigger an uncontained +* failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags. +*/ +vm_flags_set (vma,VM_ALLOW_ANY_UNCACHED ); +return 0 ; }
Signature · 1 line
2.34.1
Quoted 83 lines · On Wed, 28 Feb 2024 19:48:01 +0000 <[email protected]> wrote:
From: Ankit Agrawal <[email protected]>
The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].
Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of
NORMAL_NCand host mapping (S2) ofMT_S2_FWB_NORMAL_NC.KVM currently maps the MMIO region in S2 as
MT_S2_FWB_DEVICE_nGnREby default. The fake device BARs thus displaysDEVICE_nGnREbehavior in the VM.The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].
S1 | S2 | Result NORMAL_WB | NORMAL_NC | NORMAL_NC NORMAL_WT | NORMAL_NC | NORMAL_NC NORMAL_NC | NORMAL_NC | NORMAL_NC NORMAL_WB | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_WT | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_NC | DEVICE_nGnRE | DEVICE_nGnRERecently a change was added that modifies this default behavior and make KVM map MMIO as
MT_S2_FWB_NORMAL_NCwhen a VMA flagVM_ALLOW_ANY_UNCACHEDis set. Setting S2 asMT_S2_FWB_NORMAL_NCprovides the desired behavior (uncached, unaligned access) for resmem.Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].
To use
VM_ALLOW_ANY_UNCACHEDflag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set theVM_ALLOW_ANY_UNCACHEDflag in the VMA.Applied over next-20240227.
base-commit:22ba90670a51Link https://developer.arm.com/documentation/ddi0487/latest/ section D8.5.5 [3]Cc Alex Williamson <[email protected]>Cc Kevin Tian <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Vikram Sethi <[email protected]>Cc Zhi Wang <[email protected]>Signed-off-by Ankit Agrawal <[email protected]>--- drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)drivers/vfio/pci/nvgrace-gpu/main.c +11
-0@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, vma->vm_pgoff = start_pfn; +/* +* The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64, +* allowing KVM stage 2 device mapping attributes to use Normal-NC +* rather than DEVICE_nGnRE, which allows guest mappings +* supporting write-combining attributes (WC). This also +* unlocks memory-like operations such as unaligned accesses. +* This setting suits the fake BARs as they are expected to +* demonstrate such properties within the guest. +* +* ARM does not architecturally guarantee this is safe, and indeed +* some MMIO regions like the GICv2 VCPU interface can trigger
drivers/vfio/pci/nvgrace-gpu/main.c static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, · line 204
Quoted 10 lines
drivers/vfio/pci/nvgrace-gpu/main.c +7
-0+* uncontained faults if Normal-NC is used. The nvgrace-gpu +* however is safe in that the platform guarantees that no +* action taken on the MMIO mapping can trigger an uncontained +* failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags. +*/ +vm_flags_set (vma,VM_ALLOW_ANY_UNCACHED ); +return 0 ; }
The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,
Alex
Jason Gunthorpe
reviewerQuoted 94 lines · Alex Williamson
Quoted 91 lines · On Wed, 28 Feb 2024 19:48:01 +0000 <[email protected]> wrote:
From: Ankit Agrawal <[email protected]>
The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].
Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of
NORMAL_NCand host mapping (S2) ofMT_S2_FWB_NORMAL_NC.KVM currently maps the MMIO region in S2 as
MT_S2_FWB_DEVICE_nGnREby default. The fake device BARs thus displaysDEVICE_nGnREbehavior in the VM.The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].
S1 | S2 | Result NORMAL_WB | NORMAL_NC | NORMAL_NC NORMAL_WT | NORMAL_NC | NORMAL_NC NORMAL_NC | NORMAL_NC | NORMAL_NC NORMAL_WB | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_WT | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_NC | DEVICE_nGnRE | DEVICE_nGnRERecently a change was added that modifies this default behavior and make KVM map MMIO as
MT_S2_FWB_NORMAL_NCwhen a VMA flagVM_ALLOW_ANY_UNCACHEDis set. Setting S2 asMT_S2_FWB_NORMAL_NCprovides the desired behavior (uncached, unaligned access) for resmem.Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].
To use
VM_ALLOW_ANY_UNCACHEDflag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set theVM_ALLOW_ANY_UNCACHEDflag in the VMA.Applied over next-20240227.
base-commit:22ba90670a51Link https://developer.arm.com/documentation/ddi0487/latest/ section D8.5.5 [3]Cc Alex Williamson <[email protected]>Cc Kevin Tian <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Vikram Sethi <[email protected]>Cc Zhi Wang <[email protected]>Signed-off-by Ankit Agrawal <[email protected]>--- drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)drivers/vfio/pci/nvgrace-gpu/main.c +18
-0@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, vma->vm_pgoff = start_pfn; +/* +* The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64, +* allowing KVM stage 2 device mapping attributes to use Normal-NC +* rather than DEVICE_nGnRE, which allows guest mappings +* supporting write-combining attributes (WC). This also +* unlocks memory-like operations such as unaligned accesses. +* This setting suits the fake BARs as they are expected to +* demonstrate such properties within the guest. +* +* ARM does not architecturally guarantee this is safe, and indeed +* some MMIO regions like the GICv2 VCPU interface can trigger +* uncontained faults if Normal-NC is used. The nvgrace-gpu +* however is safe in that the platform guarantees that no +* action taken on the MMIO mapping can trigger an uncontained +* failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags. +*/ +vm_flags_set (vma,VM_ALLOW_ANY_UNCACHED ); +return 0 ;
Quoted 8 lines
drivers/vfio/pci/nvgrace-gpu/main.c static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, · line 204
Quoted 2 lines
drivers/vfio/pci/nvgrace-gpu/main.c +0
-0}The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,
Yeah, I sugged to fold that hunk into this:
if (index == RESMEM_REGION_INDEX)
vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);So it makes more sense. VM_ALLOW_ANY_UNCACHED shouldn't be used on the cachable mapping. The comment should be more specific to this driver and not so generic:
/*
* nvgrace has no issue with uncontained failures on NORMAL_NC
* access. Tell KVM to open up guest usage of NORMAL_NC for this mapping. */
Jason
Ankit Agrawal
maintainerQuoted 34 lines
Quoted 23 lines
Quoted 14 lines
+ /*
+ * TheVM_ALLOW_ANY_UNCACHEDVMA flag is implemented for ARM64, + * allowing KVM stage 2 device mapping attributes to use Normal-NC + * rather thanDEVICE_nGnRE, which allows guest mappings + * supporting write-combining attributes (WC). This also + * unlocks memory-like operations such as unaligned accesses. + * This setting suits the fake BARs as they are expected to + * demonstrate such properties within the guest.
+ *
+ * ARM does not architecturally guarantee this is safe, and indeed + * some MMIO regions like the GICv2 VCPU interface can trigger + * uncontained faults if Normal-NC is used. The nvgrace-gpu + * however is safe in that the platform guarantees that noQuoted 8 lines
+ * action taken on the MMIO mapping can trigger an uncontained + * failure. Hence
VM_ALLOW_ANY_UNCACHEDis set in the VMA flags. + */
+vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
return 0;
}Quoted 5 lines
The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,
Yeah, I sugged to fold that hunk into this:
if (index ==
RESMEM_REGION_INDEX)
vma->vm_page_prot=pgprot_writecombine(vma->vm_page_prot);
Quoted 8 lines
So it makes more sense.
VM_ALLOW_ANY_UNCACHEDshouldn't be used on the cachable mapping. The comment should be more specific to this driver and not so generic:/*
* nvgrace has no issue with uncontained failures onNORMAL_NC
* access. Tell KVM to open up guest usage ofNORMAL_NCfor this mapping. */
Sure, I will restrict this to resmem. Also will update the comment accordingly.
Quoted 2 lines
Jason
Quoted 94 lines · On Thu, 29 Feb 2024 08:56:39 -0700 Alex Williamson…
Quoted 91 lines · On Wed, 28 Feb 2024 19:48:01 +0000 <[email protected]> wrote:
From: Ankit Agrawal <[email protected]>
The NVIDIA Grace Hopper GPUs have device memory that is supposed to be used as a regular RAM. It is accessible through CPU-GPU chip-to-chip cache coherent interconnect and is present in the system physical address space. The device memory is split into two regions - termed as usemem and resmem - in the system physical address space, with each region mapped and exposed to the VM as a separate fake device BAR [1].
Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2], there is a requirement - as a workaround - for the resmem BAR to display uncached memory characteristics. Based on [3], on system with FWB enabled such as Grace Hopper, the requisite properties (uncached, unaligned access) can be achieved through a VM mapping (S1) of
NORMAL_NCand host mapping (S2) ofMT_S2_FWB_NORMAL_NC.KVM currently maps the MMIO region in S2 as
MT_S2_FWB_DEVICE_nGnREby default. The fake device BARs thus displaysDEVICE_nGnREbehavior in the VM.The following table summarizes the behavior for the various S1 and S2 mapping combinations for systems with FWB enabled [3].
S1 | S2 | Result NORMAL_WB | NORMAL_NC | NORMAL_NC NORMAL_WT | NORMAL_NC | NORMAL_NC NORMAL_NC | NORMAL_NC | NORMAL_NC NORMAL_WB | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_WT | DEVICE_nGnRE | DEVICE_nGnRE NORMAL_NC | DEVICE_nGnRE | DEVICE_nGnRERecently a change was added that modifies this default behavior and make KVM map MMIO as
MT_S2_FWB_NORMAL_NCwhen a VMA flagVM_ALLOW_ANY_UNCACHEDis set. Setting S2 asMT_S2_FWB_NORMAL_NCprovides the desired behavior (uncached, unaligned access) for resmem.Such setting is extended to the usemem as a middle-of-the-road setting to take it closer to the desired final system memory characteristics (cached, unaligned). This will eventually be fixed with the ongoing proposal [4].
To use
VM_ALLOW_ANY_UNCACHEDflag, the platform must guarantee that no action taken on the MMIO mapping can trigger an uncontained failure. The Grace Hopper satisfies this requirement. So set theVM_ALLOW_ANY_UNCACHEDflag in the VMA.Applied over next-20240227.
base-commit:22ba90670a51Link https://developer.arm.com/documentation/ddi0487/latest/ section D8.5.5 [3]Cc Alex Williamson <[email protected]>Cc Kevin Tian <[email protected]>Cc Jason Gunthorpe <[email protected]>Cc Vikram Sethi <[email protected]>Cc Zhi Wang <[email protected]>Signed-off-by Ankit Agrawal <[email protected]>--- drivers/vfio/pci/nvgrace-gpu/main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)drivers/vfio/pci/nvgrace-gpu/main.c +18
-0@@ -181,6 +181,24 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, vma->vm_pgoff = start_pfn; +/* +* The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM64, +* allowing KVM stage 2 device mapping attributes to use Normal-NC +* rather than DEVICE_nGnRE, which allows guest mappings +* supporting write-combining attributes (WC). This also +* unlocks memory-like operations such as unaligned accesses. +* This setting suits the fake BARs as they are expected to +* demonstrate such properties within the guest. +* +* ARM does not architecturally guarantee this is safe, and indeed +* some MMIO regions like the GICv2 VCPU interface can trigger +* uncontained faults if Normal-NC is used. The nvgrace-gpu +* however is safe in that the platform guarantees that no +* action taken on the MMIO mapping can trigger an uncontained +* failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA flags. +*/ +vm_flags_set (vma,VM_ALLOW_ANY_UNCACHED ); +return 0 ;
Quoted 8 lines
drivers/vfio/pci/nvgrace-gpu/main.c static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, · line 204
Quoted 2 lines
drivers/vfio/pci/nvgrace-gpu/main.c +0
-0}The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,
Also, this is setting the vma flag *after* the call to remap_pfn_range(), which seems quite sketchy. Thanks,
Alex
Ankit Agrawal
maintainerQuoted 22 lines
Quoted 22 lines
Quoted 21 lines
+ /*
+ * TheVM_ALLOW_ANY_UNCACHEDVMA flag is implemented for ARM64, + * allowing KVM stage 2 device mapping attributes to use Normal-NC + * rather thanDEVICE_nGnRE, which allows guest mappings + * supporting write-combining attributes (WC). This also + * unlocks memory-like operations such as unaligned accesses. + * This setting suits the fake BARs as they are expected to + * demonstrate such properties within the guest.
+ *
+ * ARM does not architecturally guarantee this is safe, and indeed + * some MMIO regions like the GICv2 VCPU interface can trigger + * uncontained faults if Normal-NC is used. The nvgrace-gpu + * however is safe in that the platform guarantees that no + * action taken on the MMIO mapping can trigger an uncontained + * failure. HenceVM_ALLOW_ANY_UNCACHEDis set in the VMA flags. + */
+vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
return 0;
}
Quoted 8 lines
Quoted 5 lines
The commit log sort of covers it, but this comment doesn't seem to cover why we're setting an uncached attribute to the usemem region which we're specifically mapping as coherent... did we end up giving this flag a really poor name if it's being used here to allow unaligned access? Thanks,
Also, this is setting the vma flag *after* the call to
remap_pfn_range(), which seems quite sketchy. Thanks,
I will move it to the block that Jason pointed.
Quoted 1 line
Alex