Skip to contentSkip to conversation

vIOMMU&IOMMU gIOVA to HPA mapping

DISCUSSION [email protected] 2 participants 2 messages Nov 14, 2022 virtio core

lore
Share & tools
lore
#0 lore raw

Hi,
Here is my application scenario:
1. The NIC (Network Information Center) passes through to the VM(Virtual Machine);
2. The VM uses the user mode driver DPDK;

Question:
1. vIOMMU maintains the mapping gIOVA->gPA, When do you use this gPA ?
2. Physical IOMMU maintains the GIOVA->HPA mapping ? If so, by what means (gIOVA -> HPA) mapping ?
3. What does QEMU do in NIC pass-through address translation ?

---------------------------------------------------------------------------------------------------------------------

           _____________________________________________________________________________                             
          |             ___________________________________________________________     |                            
          |            |    _____________________________________________________  |    |                            
          |            |    |                                                    | |    |                            
          |            |    |             __________________________________     | |    |                            
          |            |    |            |    ______                        |    | |    |        ______              
          |            |    |   User     |   | Mem  |-----------------------|----|-|----|-----> |      |             
          |            |    |            |   |______|        DPDK           |    | |    | (gVA) |______|             
          |            |    |            |    (gVA)                         |    | |    |        |    |              
          |            |    |            |______|___________________________|    | |    |        |    |              
          |            |    |--GuestOS----------|--------------------------------| |    |        |    |              
          |            |    |             ______\/__________________________     | |    |        |    |              
          |            | VM |            |                VFIO              |    | |    |        |    |              
          |   User     |    |   Kernel   |___________     __________________|    | |    |        |    |              
          |            |    |            | vfio-pci  |   |                  |    | |    |        |    |              
          |            |    |            |  (gIOVA)  |   | vfio_iommu_type2 |    | |    |        |    |              
Software  |            |    |            |______|____|___|__________________|    | |    |        |    |              
          |            |    |___________________|________________________________| |    |        |    |              
          |            |     ____________________________________________________  |    |        |    |              
          |            |    |             ______|____     __________________     | |    |        \/   \/             
          |            |    |            |      \/   |   |                  |    | |    |        ______              
          |            |    |            |    NIC    |   |      vIOMMU      |    | |    |       |      |             
          |            |    |   QEMU     |  Instance |   |                --|----|-|----|-----> |______|             
          |            |    |            |  (gIOVA)  |   |   (GIOVA-->GPA)  |    | |    | (gPA)  |    |              
          |            |    |            |_____|_____|   |__________________|    | |    |        |    |              
          |            |    |__________________|_________________________________| |    |        |    |              
          |            |_______________________|___________________________________|    |        |    |              
          |---HostOS---------------------------|----------------------------------------|        |    |              
          |             ___________________________________________________________     |        |    |              
          |            |                       |                                   |    |        |    |              
          |            |                       |    VFIO                           |    |        |    |              
          |   Kernel   |_______________________|_____     _________________________|    |        |    |              
          |            |                       \/    |   |                         |    |        |    |              
          |            |        vfio-pci             |   |    vfio_iommu_type1     |    |        |    |              
          |            |                       |     |   |                         |    |        |    |              
          |            |_______________________|_____|___|_________________________|    |        |    |              
          |____________________________________|________________________________________|        |    |              
-----------------------------------------------|-------------------------------------------------|----|--------------
           ____________________________________\/____     _________________________           ___\/___\/___________  
          |            |                             |   |                         |         |  |      |           | 
Hardware  |    NIC     |          DMA     (gIOVA)  --|---|->        IOMMU        --|---------|->|______|           | 
          |            |_____________________________|   |      (gIOVA-->HPA)      |   (HPA) |   Physical Memory   | 
          |__________________________________________|   |_________________________|         |_____________________| 

---------------------------------------------------------------------------------------------------------------------

[email protected]

#1 lore raw
Quoted 7 lines · On Mon, 14 Nov 2022 20:31:49 +0800 "[email protected]" <[email protected]> wrote:

Hi,
Here is my application scenario:
1. The NIC (Network Information Center) passes through to the VM(Virtual Machine);
2. The VM uses the user mode driver DPDK;

Question:
1. vIOMMU maintains the mapping gIOVA->gPA, When do you use this gPA ?

QEMU in the host derives the hVA from the gPA. The vIOMMU driver in QEMU is triggering the gIOVA to hVA mapping through vfio in the host.

Quoted 1 line

2. Physical IOMMU maintains the GIOVA->HPA mapping ? If so, by what means (gIOVA -> HPA) mapping ?

As above, the vIOMMU in the guest provides gIOVA -> gPA, in QEMU we do the gPA -> hVA, then vfio in the host kernel performs hVA -> hPA via page pinning.

Quoted 1 line

3. What does QEMU do in NIC pass-through address translation ?

The guest visible vIOMMU triggers MemoryListener notifications in QEMU for the device address space, which insert and removes mappings to the vfio layer below it. Thanks,

Alex