Skip to contentSkip to conversation

doc: announce change in vfio dma mapping

PATCH In review Xuan Ding 4 participants 15 messages Aug 31, 2021 – Sep 8, 2021

lore
Share & tools
lore

Xuan Ding

author
#0 lore raw

PATCH 1/1 doc: announce change in vfio dma mapping

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current DPDK VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
 doc/guides/rel_notes/deprecation.rst | 3 +++
 1 file changed, 3 insertions(+)
doc/guides/rel_notes/deprecation.rst +3 -0
index 76a4abfd6b..1234420caf 100644
@@ -287,3 +287,6 @@ Deprecation Notices
   reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and other
   information from the crypto/security operation. This field will be used to
   communicate events such as soft expiry with IPsec in lookaside mode.
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.
Signature · 1 line
2.17.1
Acked-by on v1 1/1
#1 lore raw
Quoted 24 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current DPDK VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
  doc/guides/rel_notes/deprecation.rst | 3 +++
  1 file changed, 3 insertions(+)
doc/guides/rel_notes/deprecation.rst +0 -0
index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 292

Quoted 6 lines
doc/guides/rel_notes/deprecation.rst +3 -0
@@ -287,3 +287,6 @@ Deprecation Notices
    reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and other
    information from the crypto/security operation. This field will be used to
    communicate events such as soft expiry with IPsec in lookaside mode.
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.
Signature · 2 lines
Thanks,
Anatoly
#2 lore raw
In reply to Xuan Ding
Quoted 24 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current DPDK VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
 doc/guides/rel_notes/deprecation.rst | 3 +++
 1 file changed, 3 insertions(+)
doc/guides/rel_notes/deprecation.rst +0 -0
index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 292

Quoted 6 lines
doc/guides/rel_notes/deprecation.rst +3 -0
@@ -287,3 +287,6 @@ Deprecation Notices
   reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and other
   information from the crypto/security operation. This field will be used to
   communicate events such as soft expiry with IPsec in lookaside mode.
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

author
#3 lore raw

Hi Ferruh,

Quoted 37 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 5 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 6 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

Review

Quoted 8 lines

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
 doc/guides/rel_notes/deprecation.rst | 3 +++
 1 file changed, 3 insertions(+)
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 287

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices
   reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and

other

Quoted 9 lines

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 292

Quoted 6 lines
doc/guides/rel_notes/deprecation.rst +3 -0
   information from the crypto/security operation. This field will be used to
   communicate events such as soft expiry with IPsec in lookaside mode.
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new function name, so you mean, any changes to the API can only be done in the LTS version? If so, we can only add a new API and retire the old one in 22.11.

Thanks for you classification.

#4 lore raw
Quoted 49 lines · Xuan

Hi Ferruh,

Quoted 37 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 5 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 6 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

Review

Quoted 8 lines

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
 doc/guides/rel_notes/deprecation.rst | 3 +++
 1 file changed, 3 insertions(+)
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 287

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices
   reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and

other

Quoted 9 lines

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 292

Quoted 6 lines
doc/guides/rel_notes/deprecation.rst +3 -0
   information from the crypto/security operation. This field will be used to
   communicate events such as soft expiry with IPsec in lookaside mode.
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Quoted 3 lines

Our original plan is add a new parameter in order not to use a new function name, so you mean, any changes to the API can only be done in the LTS version? If so, we can only add a new API and retire the old one in 22.11.

We can add a new API anytime. Adding new parameter to an existing API can be done on the ABI break release.

You can add the new API in this release, and start using it.
And mark the old API as deprecated in this release. This lets existing binaries to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice before 22.11 for it.

Is above plan works for you?

#5 lore raw
Quoted 50 lines · Ferruh Yigit
Quoted 49 lines · Xuan

Hi Ferruh,

Quoted 46 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 5 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 6 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

Review

Quoted 8 lines

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
  doc/guides/rel_notes/deprecation.rst | 3 +++
  1 file changed, 3 insertions(+)
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 287

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices
    reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and

other

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 292

Quoted 6 lines
doc/guides/rel_notes/deprecation.rst +3 -0
    information from the crypto/security operation. This field will be used to
    communicate events such as soft expiry with IPsec in lookaside mode.
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Quoted 6 lines
Quoted 3 lines

Our original plan is add a new parameter in order not to use a new function name, so you mean, any changes to the API can only be done in the LTS version? If so, we can only add a new API and retire the old one in 22.11.

We can add a new API anytime. Adding new parameter to an existing API can be done on the ABI break release.

So, 22.11 then?

Quoted 9 lines

You can add the new API in this release, and start using it.
And mark the old API as deprecated in this release. This lets existing binaries to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice before 22.11 for it.

Is above plan works for you?

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS to break it.

Signature · 2 lines
Thanks,
Anatoly
#6 lore raw
Quoted 56 lines · Anatoly
Quoted 55 lines · Ferruh Yigit
Quoted 53 lines · Xuan

Hi Ferruh,

Quoted 37 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 5 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 6 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

Review

Quoted 8 lines

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
  doc/guides/rel_notes/deprecation.rst | 3 +++

  1 file changed, 3 insertions(+)
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices

    reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and

other

Quoted 9 lines

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 292

Quoted 6 lines

    information from the crypto/security operation. This field will be used to communicate events such as soft expiry with IPsec in lookaside mode.

doc/guides/rel_notes/deprecation.rst +3 -0
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new function name, so you mean, any changes to the API can only be done in the LTS version? If so, we can only add a new API and retire the old one in 22.11.

Quoted 5 lines
Quoted 2 lines

We can add a new API anytime. Adding new parameter to an existing API can be done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 10 lines
Quoted 9 lines

You can add the new API in this release, and start using it.
And mark the old API as deprecated in this release. This lets existing binaries to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice before 22.11 for it.

Is above plan works for you?

Quoted 9 lines

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS to break it.

Got it.

As far as I understand, main motivation in techboard decision was to prevent the ABI break as much as possible (main reason of decision wasn't deprecation notice being late). But if the correct thing to do is to rename the API (and break the ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename is better option, perhaps we can visit the issue again in techboard.

Can you please describe why renaming API is better option, comparing to adding new API with new parameter?

Thanks,
ferruh

#7 lore raw
Quoted 95 lines · Ferruh Yigit
Quoted 56 lines · Anatoly
Quoted 55 lines · Ferruh Yigit
Quoted 53 lines · Xuan

Hi Ferruh,

Quoted 37 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 5 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 6 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

Review

Quoted 8 lines

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
   doc/guides/rel_notes/deprecation.rst | 3 +++
   1 file changed, 3 insertions(+)
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 287

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices
     reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and

other

Quoted 9 lines

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 292

Quoted 6 lines
doc/guides/rel_notes/deprecation.rst +3 -0
     information from the crypto/security operation. This field will be used to
     communicate events such as soft expiry with IPsec in lookaside mode.
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new function name, so you mean, any changes to the API can only be done in the LTS version? If so, we can only add a new API and retire the old one in 22.11.

Quoted 5 lines
Quoted 2 lines

We can add a new API anytime. Adding new parameter to an existing API can be done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 10 lines
Quoted 9 lines

You can add the new API in this release, and start using it.
And mark the old API as deprecated in this release. This lets existing binaries to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice before 22.11 for it.

Is above plan works for you?

Quoted 9 lines

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS to break it.

Got it.

As far as I understand, main motivation in techboard decision was to prevent the ABI break as much as possible (main reason of decision wasn't deprecation notice being late). But if the correct thing to do is to rename the API (and break the ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename is better option, perhaps we can visit the issue again in techboard.

Quoted 2 lines

Can you please describe why renaming API is better option, comparing to adding new API with new parameter?

I take it you meant "why renaming API *isn't* a better option".

The problem we're solving is that the API in question does not know about page sizes and thus can't map segments page-by-page. I mean I /guess/ we could have two API's (one paged, one not paged), but then we get into all kinds of hairy things about the API leaking the details of underlying platform.

Bottom line: i like current API function name. It's concise, it's descriptive. It's only missing a parameter, which i would like to add. A rename has been suggested (deprecate old API, add new API with a different name, and with added parameter), but honestly, I don't see why we have to do that because this is predicated upon the assumption that we *can't* break ABI at all, under any circumstances.

Can you please explain to me what is wrong with keeping a versioned symbol? Like, keep the old function around to keep ABI compatibility, but break the API compatibility for those who target 22.02 or later? That's what symbol versioning is *for*, is it not?

Signature · 2 lines
Thanks,
Anatoly
#8 lore raw
Quoted 114 lines · Anatoly
Quoted 96 lines · Ferruh Yigit
Quoted 57 lines · Anatoly
Quoted 56 lines · Ferruh Yigit
Quoted 54 lines · Xuan

Hi Ferruh,

Quoted 37 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 5 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 6 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

Review

Quoted 8 lines

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
   doc/guides/rel_notes/deprecation.rst | 3 +++

   1 file changed, 3 insertions(+)
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices

     reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and

other

Quoted 10 lines

doc/guides/rel_notes/deprecation.rst

Quoted 7 lines

     information from the crypto/security operation. This field will be used to
     communicate events such as soft expiry with IPsec in lookaside mode.

doc/guides/rel_notes/deprecation.rst +3 -0
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new function name, so you mean, any changes to the API can only be done in the LTS version? If so, we can only add a new API and retire the old one in 22.11.

Quoted 5 lines
Quoted 2 lines

We can add a new API anytime. Adding new parameter to an existing API can be done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 10 lines
Quoted 9 lines

You can add the new API in this release, and start using it.
And mark the old API as deprecated in this release. This lets existing binaries to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice before 22.11 for it.

Is above plan works for you?

Quoted 9 lines

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS to break it.

Got it.

As far as I understand, main motivation in techboard decision was to prevent the ABI break as much as possible (main reason of decision wasn't deprecation notice being late). But if the correct thing to do is to rename the API (and break the ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename is better option, perhaps we can visit the issue again in techboard.

Quoted 2 lines

Can you please describe why renaming API is better option, comparing to adding new API with new parameter?

I take it you meant "why renaming API *isn't* a better option".

The problem we're solving is that the API in question does not know about page sizes and thus can't map segments page-by-page. I mean I /guess/ we could have two API's (one paged, one not paged), but then we get into all kinds of hairy things about the API leaking the details of underlying platform.

Bottom line: i like current API function name. It's concise, it's descriptive. It's only missing a parameter, which i would like to add. A rename has been suggested (deprecate old API, add new API with a different name, and with added parameter), but honestly, I don't see why we have to do that because this is predicated upon the assumption that we *can't* break ABI at all, under any circumstances.

Quoted 5 lines

Can you please explain to me what is wrong with keeping a versioned symbol? Like, keep the old function around to keep ABI compatibility, but break the API compatibility for those who target 22.02 or later? That's what symbol versioning is *for*, is it not?

Nothing wrong with symbol versioning, indeed that is preferred way if it works for you, I didn't get that symbol versioning is planned.

@Ray,
Since symbol versioning is planned, ABI won't break, but API will change, can this change be done in this release without deprecation notice? Later we can have a deprecation notice to remove old symbol on 22.11.

Thanks,
ferruh

#9 lore raw
Quoted 121 lines · Ferruh Yigit
Quoted 114 lines · Anatoly
Quoted 96 lines · Ferruh Yigit
Quoted 57 lines · Anatoly
Quoted 56 lines · Ferruh Yigit
Quoted 54 lines · Xuan

Hi Ferruh,

Quoted 37 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov, Anatoly <[email protected]>
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 5 lines · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for the purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to be adjacent in memory. Since VFIO implementation on IA platforms also does not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 6 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

Review

Quoted 8 lines

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
   doc/guides/rel_notes/deprecation.rst | 3 +++

   1 file changed, 3 insertions(+)
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices

     reserved bytes to 2 (from 3), and use 1 byte to indicate warnings and

other

Quoted 10 lines

doc/guides/rel_notes/deprecation.rst

Quoted 7 lines

     information from the crypto/security operation. This field will be used to
     communicate events such as soft expiry with IPsec in lookaside mode.

doc/guides/rel_notes/deprecation.rst +3 -0
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended to
+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new function name, so you mean, any changes to the API can only be done in the LTS version? If so, we can only add a new API and retire the old one in 22.11.

Quoted 5 lines
Quoted 2 lines

We can add a new API anytime. Adding new parameter to an existing API can be done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 10 lines
Quoted 9 lines

You can add the new API in this release, and start using it.
And mark the old API as deprecated in this release. This lets existing binaries to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice before 22.11 for it.

Is above plan works for you?

Quoted 9 lines

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS to break it.

Got it.

As far as I understand, main motivation in techboard decision was to prevent the ABI break as much as possible (main reason of decision wasn't deprecation notice being late). But if the correct thing to do is to rename the API (and break the ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename is better option, perhaps we can visit the issue again in techboard.

Quoted 2 lines

Can you please describe why renaming API is better option, comparing to adding new API with new parameter?

I take it you meant "why renaming API *isn't* a better option".

The problem we're solving is that the API in question does not know about page sizes and thus can't map segments page-by-page. I mean I /guess/ we could have two API's (one paged, one not paged), but then we get into all kinds of hairy things about the API leaking the details of underlying platform.

Bottom line: i like current API function name. It's concise, it's descriptive. It's only missing a parameter, which i would like to add. A rename has been suggested (deprecate old API, add new API with a different name, and with added parameter), but honestly, I don't see why we have to do that because this is predicated upon the assumption that we *can't* break ABI at all, under any circumstances.

Quoted 5 lines

Can you please explain to me what is wrong with keeping a versioned symbol? Like, keep the old function around to keep ABI compatibility, but break the API compatibility for those who target 22.02 or later? That's what symbol versioning is *for*, is it not?

Quoted 6 lines

Nothing wrong with symbol versioning, indeed that is preferred way if it works for you, I didn't get that symbol versioning is planned.

@Ray,
Since symbol versioning is planned, ABI won't break, but API will change, can this change be done in this release without deprecation notice?

Yes - I would think so.
Since we are going to the effort of using symbol versioning nothing is being depreciated as such (yet).

Quoted 5 lines

Later we can have a deprecation notice to remove old symbol on 22.11.

Thanks,
ferruh

author
#10 lore raw

Hi,

Quoted 162 lines

-----Original Message-----
From: Kinsella, Ray <[email protected]>
Sent: Friday, September 3, 2021 12:13 AM
To: Yigit, Ferruh <[email protected]>; Burakov, Anatoly <[email protected]>; Ding, Xuan <[email protected]>; [email protected]
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>; Thomas Monjalon <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 11 lines · Ferruh Yigit
Quoted 10 lines · Anatoly
Quoted 9 lines · Ferruh Yigit
Quoted 8 lines · Anatoly
Quoted 7 lines · Ferruh Yigit
Quoted 6 lines · Xuan

Hi Ferruh,

Quoted 4 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov,

Anatoly

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<[email protected]>
Cc: [email protected]; Xia, Chenbo

<[email protected]>; Hu,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Jiayu <[email protected]>; Richardson, Bruce

<[email protected]>

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 1 line · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to

be

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

adjacent in memory. Since VFIO implementation on IA platforms also

does

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 1 line

not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 13 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
   doc/guides/rel_notes/deprecation.rst | 3 +++

   1 file changed, 3 insertions(+)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices

     reserved bytes to 2 (from 3), and use 1 byte to indicate warnings

and

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

other

Quoted 3 lines

     information from the crypto/security operation. This field will be used to
     communicate events such as soft expiry with IPsec in lookaside

mode.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

doc/guides/rel_notes/deprecation.rst

Quoted 2 lines
doc/guides/rel_notes/deprecation.rst +2 -0
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended

to

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new

function

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

name, so you mean, any changes to the API can only be done in the LTS

version?

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

If so, we can only add a new API and retire the old one in 22.11.

We can add a new API anytime. Adding new parameter to an existing API

can be

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Quoted 1 line

done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 3 lines
Quoted 3 lines

You can add the new API in this release, and start using it. And mark the old API as deprecated in this release. This lets existing

binaries

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice

before

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

22.11 for it.

Is above plan works for you?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS

to

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

break it.

Got it.

As far as I understand, main motivation in techboard decision was to

prevent the

Quoted 1 line
Quoted 1 line
Quoted 1 line

ABI break as much as possible (main reason of decision wasn't deprecation

notice

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

being late). But if the correct thing to do is to rename the API (and break the ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename

is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

better option, perhaps we can visit the issue again in techboard.

Can you please describe why renaming API is better option, comparing to

adding

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

new API with new parameter?

I take it you meant "why renaming API *isn't* a better option".

The problem we're solving is that the API in question does not know about

page

Quoted 1 line
Quoted 1 line

sizes and thus can't map segments page-by-page. I mean I /guess/ we could

have

Quoted 6 lines
Quoted 6 lines

two API's (one paged, one not paged), but then we get into all kinds of hairy things about the API leaking the details of underlying platform.

Bottom line: i like current API function name. It's concise, it's descriptive. It's only missing a parameter, which i would like to add. A rename has been suggested (deprecate old API, add new API with a different name, and with

added

Quoted 6 lines
Quoted 6 lines

parameter), but honestly, I don't see why we have to do that because this is predicated upon the assumption that we *can't* break ABI at all, under any circumstances.

Can you please explain to me what is wrong with keeping a versioned symbol? Like, keep the old function around to keep ABI compatibility, but break the

API

Quoted 1 line
Quoted 1 line

compatibility for those who target 22.02 or later? That's what symbol

versioning

Quoted 6 lines
Quoted 2 lines

is *for*, is it not?

Nothing wrong with symbol versioning, indeed that is preferred way if it works for you, I didn't get that symbol versioning is planned.

Quoted 9 lines
Quoted 3 lines

@Ray,
Since symbol versioning is planned, ABI won't break, but API will change, can this change be done in this release without deprecation notice?

Yes - I would think so.
Since we are going to the effort of using symbol versioning nothing is being depreciated as such (yet).

Quoted 1 line

Later we can have a deprecation notice to remove old symbol on 22.11.

Thanks for your explanation.
@Yigit, Ferruh Does it mean that we can do API change in 21.11? If so, we will follow the process and target API change in this release. :)

Regards,
Xuan

Quoted 4 lines
Quoted 4 lines

Thanks,
ferruh

#11 lore raw
Quoted 172 lines · Xuan

Hi,

Quoted 170 lines

-----Original Message-----
From: Kinsella, Ray <[email protected]>
Sent: Friday, September 3, 2021 12:13 AM
To: Yigit, Ferruh <[email protected]>; Burakov, Anatoly <[email protected]>; Ding, Xuan <[email protected]>; [email protected]
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>; Thomas Monjalon <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 11 lines · Ferruh Yigit
Quoted 10 lines · Anatoly
Quoted 9 lines · Ferruh Yigit
Quoted 8 lines · Anatoly
Quoted 7 lines · Ferruh Yigit
Quoted 6 lines · Xuan

Hi Ferruh,

Quoted 4 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov,

Anatoly

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<[email protected]>
Cc: [email protected]; Xia, Chenbo

<[email protected]>; Hu,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Jiayu <[email protected]>; Richardson, Bruce

<[email protected]>

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 1 line · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to

be

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

adjacent in memory. Since VFIO implementation on IA platforms also

does

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 1 line

not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 13 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
   doc/guides/rel_notes/deprecation.rst | 3 +++

   1 file changed, 3 insertions(+)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices

     reserved bytes to 2 (from 3), and use 1 byte to indicate warnings

and

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

other

Quoted 3 lines

     information from the crypto/security operation. This field will be used to
     communicate events such as soft expiry with IPsec in lookaside

mode.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

doc/guides/rel_notes/deprecation.rst

Quoted 2 lines
doc/guides/rel_notes/deprecation.rst +2 -0
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended

to

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new

function

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

name, so you mean, any changes to the API can only be done in the LTS

version?

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

If so, we can only add a new API and retire the old one in 22.11.

We can add a new API anytime. Adding new parameter to an existing API

can be

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Quoted 1 line

done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 3 lines
Quoted 3 lines

You can add the new API in this release, and start using it. And mark the old API as deprecated in this release. This lets existing

binaries

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice

before

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

22.11 for it.

Is above plan works for you?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS

to

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

break it.

Got it.

As far as I understand, main motivation in techboard decision was to

prevent the

Quoted 1 line
Quoted 1 line
Quoted 1 line

ABI break as much as possible (main reason of decision wasn't deprecation

notice

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

being late). But if the correct thing to do is to rename the API (and break the ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename

is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

better option, perhaps we can visit the issue again in techboard.

Can you please describe why renaming API is better option, comparing to

adding

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

new API with new parameter?

I take it you meant "why renaming API *isn't* a better option".

The problem we're solving is that the API in question does not know about

page

Quoted 1 line
Quoted 1 line

sizes and thus can't map segments page-by-page. I mean I /guess/ we could

have

Quoted 6 lines
Quoted 6 lines

two API's (one paged, one not paged), but then we get into all kinds of hairy things about the API leaking the details of underlying platform.

Bottom line: i like current API function name. It's concise, it's descriptive. It's only missing a parameter, which i would like to add. A rename has been suggested (deprecate old API, add new API with a different name, and with

added

Quoted 6 lines
Quoted 6 lines

parameter), but honestly, I don't see why we have to do that because this is predicated upon the assumption that we *can't* break ABI at all, under any circumstances.

Can you please explain to me what is wrong with keeping a versioned symbol? Like, keep the old function around to keep ABI compatibility, but break the

API

Quoted 1 line
Quoted 1 line

compatibility for those who target 22.02 or later? That's what symbol

versioning

Quoted 9 lines
Quoted 2 lines

is *for*, is it not?

Nothing wrong with symbol versioning, indeed that is preferred way if it works for you, I didn't get that symbol versioning is planned.

@Ray,
Since symbol versioning is planned, ABI won't break, but API will change, can this change be done in this release without deprecation notice?

Yes - I would think so.
Since we are going to the effort of using symbol versioning nothing is being depreciated as such (yet).

Quoted 6 lines
Quoted 1 line
Quoted 1 line

Later we can have a deprecation notice to remove old symbol on 22.11.

Thanks for your explanation.
@Yigit, Ferruh Does it mean that we can do API change in 21.11? If so, we will follow the process and target API change in this release. :)

With symbol versioning, yes you can make the change in this release.

You can send another deprecation notice to remove the old symbol for 22.11, that can be sent anytime until 22.08 released.

#12 lore raw
Quoted 180 lines · Ferruh Yigit
Quoted 172 lines · Xuan

Hi,

Quoted 170 lines

-----Original Message-----
From: Kinsella, Ray <[email protected]>
Sent: Friday, September 3, 2021 12:13 AM
To: Yigit, Ferruh <[email protected]>; Burakov, Anatoly <[email protected]>; Ding, Xuan <[email protected]>; [email protected]
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>; Thomas Monjalon <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 11 lines · Ferruh Yigit
Quoted 10 lines · Anatoly
Quoted 9 lines · Ferruh Yigit
Quoted 8 lines · Anatoly
Quoted 7 lines · Ferruh Yigit
Quoted 6 lines · Xuan

Hi Ferruh,

Quoted 4 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov,

Anatoly

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<[email protected]>
Cc: [email protected]; Xia, Chenbo

<[email protected]>; Hu,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Jiayu <[email protected]>; Richardson, Bruce

<[email protected]>

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 1 line · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to

be

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

adjacent in memory. Since VFIO implementation on IA platforms also

does

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 1 line

not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 13 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
    doc/guides/rel_notes/deprecation.rst | 3 +++
    1 file changed, 3 insertions(+)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 287

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices
      reserved bytes to 2 (from 3), and use 1 byte to indicate warnings

and

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

other

Quoted 3 lines
doc/guides/rel_notes/deprecation.rst +0 -0
      information from the crypto/security operation. This field will be

used to
     communicate events such as soft expiry with IPsec in lookaside

mode.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

doc/guides/rel_notes/deprecation.rst

Quoted 2 lines
doc/guides/rel_notes/deprecation.rst +2 -0
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended

to

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new

function

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

name, so you mean, any changes to the API can only be done in the LTS

version?

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

If so, we can only add a new API and retire the old one in 22.11.

We can add a new API anytime. Adding new parameter to an existing API

can be

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Quoted 1 line

done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 3 lines
Quoted 3 lines

You can add the new API in this release, and start using it. And mark the old API as deprecated in this release. This lets existing

binaries

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice

before

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

22.11 for it.

Is above plan works for you?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be considered unrelated issues.

I still think it's a good idea to update the API that way, so I would like to do that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS

to

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

break it.

Got it.

As far as I understand, main motivation in techboard decision was to

prevent the

Quoted 1 line
Quoted 1 line
Quoted 1 line

ABI break as much as possible (main reason of decision wasn't deprecation

notice

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

being late). But if the correct thing to do is to rename the API (and break the ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename

is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

better option, perhaps we can visit the issue again in techboard.

Can you please describe why renaming API is better option, comparing to

adding

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

new API with new parameter?

I take it you meant "why renaming API *isn't* a better option".

The problem we're solving is that the API in question does not know about

page

Quoted 1 line
Quoted 1 line

sizes and thus can't map segments page-by-page. I mean I /guess/ we could

have

Quoted 6 lines
Quoted 6 lines

two API's (one paged, one not paged), but then we get into all kinds of hairy things about the API leaking the details of underlying platform.

Bottom line: i like current API function name. It's concise, it's descriptive. It's only missing a parameter, which i would like to add. A rename has been suggested (deprecate old API, add new API with a different name, and with

added

Quoted 6 lines
Quoted 6 lines

parameter), but honestly, I don't see why we have to do that because this is predicated upon the assumption that we *can't* break ABI at all, under any circumstances.

Can you please explain to me what is wrong with keeping a versioned symbol? Like, keep the old function around to keep ABI compatibility, but break the

API

Quoted 1 line
Quoted 1 line

compatibility for those who target 22.02 or later? That's what symbol

versioning

Quoted 9 lines
Quoted 2 lines

is *for*, is it not?

Nothing wrong with symbol versioning, indeed that is preferred way if it works for you, I didn't get that symbol versioning is planned.

@Ray,
Since symbol versioning is planned, ABI won't break, but API will change, can this change be done in this release without deprecation notice?

Yes - I would think so.
Since we are going to the effort of using symbol versioning nothing is being depreciated as such (yet).

Quoted 6 lines
Quoted 1 line
Quoted 1 line

Later we can have a deprecation notice to remove old symbol on 22.11.

Thanks for your explanation.
@Yigit, Ferruh Does it mean that we can do API change in 21.11? If so, we will follow the process and target API change in this release. :)

Quoted 5 lines

With symbol versioning, yes you can make the change in this release.

You can send another deprecation notice to remove the old symbol for 22.11, that can be sent anytime until 22.08 released.

Hi Ferruh and others,

We have decided to switch gears somewhat :)

The original intent was to ensure that two adjacent segments can be freely mapped and unmapped. The easiest solution was page-by-page mapping, so that's where the API change idea came from.

However, due to tech board decision to not grant the exception at the time, we have figured out a way to avoid the API change. The API change is still a good idea because mapping things page-by-page is a valid use case that is currently not covered by the API, so the next plan was to do the API change in a later release as a separate issue, not related to original Xuan's intent.

However, we've been discussing implementation details with Xuan, and we arrived at a realization that what Xuan wants to implement not only does not *require* page size, it *cannot* be implemented with page size API because there's no way to know page size for that memory at the time of the API call. So, turns out we need both paged and page-less versions :)

This means that there are no deprecation notices now, because we will be adding a new API after all, but we will *not* deprecate or remove the old API - that one will still stay valid.

Apologies for constantly shifting ground, but in the end i think we arrived at the best possible solution for this problem!

Signature · 2 lines
Thanks,
Anatoly
#13 lore raw
Quoted 213 lines · Anatoly
Quoted 186 lines · Ferruh Yigit
Quoted 178 lines · Xuan

Hi,

Quoted 176 lines

-----Original Message-----
From: Kinsella, Ray <[email protected]>
Sent: Friday, September 3, 2021 12:13 AM
To: Yigit, Ferruh <[email protected]>; Burakov, Anatoly <[email protected]>; Ding, Xuan <[email protected]>; [email protected]
Cc: [email protected]; Xia, Chenbo <[email protected]>; Hu, Jiayu <[email protected]>; Richardson, Bruce <[email protected]>; Thomas Monjalon <[email protected]>
Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 11 lines · Ferruh Yigit
Quoted 10 lines · Anatoly
Quoted 9 lines · Ferruh Yigit
Quoted 8 lines · Anatoly
Quoted 7 lines · Ferruh Yigit
Quoted 6 lines · Xuan

Hi Ferruh,

Quoted 4 lines

-----Original Message-----
From: Yigit, Ferruh <[email protected]>
Sent: Wednesday, September 1, 2021 12:01 AM
To: Ding, Xuan <[email protected]>; [email protected]; Burakov,

Anatoly

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<[email protected]>
Cc: [email protected]; Xia, Chenbo

<[email protected]>; Hu,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Jiayu <[email protected]>; Richardson, Bruce

<[email protected]>

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Subject: Re: [PATCH] doc: announce change in vfio dma mapping

Quoted 1 line · Xuan Ding

Currently, the VFIO subsystem will compact adjacent DMA regions for

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

purposes of saving space in the internal list of mappings. This has a side effect of compacting two separate mappings that just happen to

be

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

adjacent in memory. Since VFIO implementation on IA platforms also

does

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 1 line

not allow partial unmapping of memory mapped for DMA, the current

DPDK

Quoted 13 lines

VFIO implementation will prevent unmapping of accidentally adjacent maps even though it could have been unmapped [1].

The proper fix for this issue is to change the VFIO DMA mapping API to also include page size, and always map memory page-by-page.

[1] https://mails.dpdk.org/archives/dev/2021-July/213493.html

---
    doc/guides/rel_notes/deprecation.rst | 3 +++

    1 file changed, 3 insertions(+)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +0 -0

b/doc/guides/rel_notes/deprecation.rst

Quoted 5 lines

index 76a4abfd6b..1234420caf 100644

doc/guides/rel_notes/deprecation.rst +0 -0
@@ -287,3 +287,6 @@ Deprecation Notices

      reserved bytes to 2 (from 3), and use 1 byte to indicate warnings

and

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

other

Quoted 4 lines

      information from the crypto/security operation. This field will be
used to
      communicate events such as soft expiry with IPsec in lookaside

mode.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

doc/guides/rel_notes/deprecation.rst

Quoted 2 lines
doc/guides/rel_notes/deprecation.rst +2 -0
+
+* vfio: the functions `rte_vfio_container_dma_map` will be amended

to

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

+  include page size. This change is targeted for DPDK 22.02.

Is this means adding a new parameter to API?
If so this is an ABI/API break and we can't do this change in the 22.02.

Our original plan is add a new parameter in order not to use a new

function

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

name, so you mean, any changes to the API can only be done in the LTS

version?

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

If so, we can only add a new API and retire the old one in 22.11.

We can add a new API anytime. Adding new parameter to an existing API

can be

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
Quoted 1 line

done on the ABI break release.

So, 22.11 then?

Yes.

Quoted 3 lines
Quoted 3 lines

You can add the new API in this release, and start using it. And mark the old API as deprecated in this release. This lets existing

binaries

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

to keep using it, but app needs to switch to new API for compilation. Old API can be removed on 22.11, and you will need a deprecation notice

before

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines

22.11 for it.

Is above plan works for you?

We have slightly rethought our approach, and the functionality that Xuan requires does not rely on this API. They can, for all intents and purposes, be
considered unrelated issues.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I still think it's a good idea to update the API that way, so I would like to do
that, and if we have to wait until 22.11 to fix it, I'm OK with that. Since there no longer is any urgency here, it's acceptable to wait for the next LTS

to

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

break it.

Got it.

As far as I understand, main motivation in techboard decision was to

prevent the

Quoted 1 line
Quoted 1 line
Quoted 1 line

ABI break as much as possible (main reason of decision wasn't deprecation

notice

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

being late). But if the correct thing to do is to rename the API (and break the
ABI), I don't see the benefit to wait one more year, it is just delaying the impact and adding overhead to us.
I am for being pragmatic and doing the change in this release if API rename

is

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

better option, perhaps we can visit the issue again in techboard.

Can you please describe why renaming API is better option, comparing to

adding

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

new API with new parameter?

I take it you meant "why renaming API *isn't* a better option".

The problem we're solving is that the API in question does not know about

page

Quoted 1 line
Quoted 1 line

sizes and thus can't map segments page-by-page. I mean I /guess/ we could

have

Quoted 7 lines
Quoted 7 lines

two API's (one paged, one not paged), but then we get into all kinds of hairy things about the API leaking the details of underlying platform.

Bottom line: i like current API function name. It's concise, it's descriptive.
It's only missing a parameter, which i would like to add. A rename has been suggested (deprecate old API, add new API with a different name, and with

added

Quoted 6 lines
Quoted 6 lines

parameter), but honestly, I don't see why we have to do that because this is predicated upon the assumption that we *can't* break ABI at all, under any circumstances.

Can you please explain to me what is wrong with keeping a versioned symbol? Like, keep the old function around to keep ABI compatibility, but break the

API

Quoted 1 line
Quoted 1 line

compatibility for those who target 22.02 or later? That's what symbol

versioning

Quoted 9 lines
Quoted 2 lines

is *for*, is it not?

Nothing wrong with symbol versioning, indeed that is preferred way if it works for you, I didn't get that symbol versioning is planned.

@Ray,
Since symbol versioning is planned, ABI won't break, but API will change, can this change be done in this release without deprecation notice?

Yes - I would think so.
Since we are going to the effort of using symbol versioning nothing is being depreciated as such (yet).

Quoted 6 lines
Quoted 1 line
Quoted 1 line

Later we can have a deprecation notice to remove old symbol on 22.11.

Thanks for your explanation.
@Yigit, Ferruh Does it mean that we can do API change in 21.11? If so, we will follow the process and target API change in this release. :)

Quoted 5 lines

With symbol versioning, yes you can make the change in this release.

You can send another deprecation notice to remove the old symbol for 22.11, that can be sent anytime until 22.08 released.

Hi Ferruh and others,

We have decided to switch gears somewhat :)

The original intent was to ensure that two adjacent segments can be freely mapped and unmapped. The easiest solution was page-by-page mapping, so that's where the API change idea came from.

However, due to tech board decision to not grant the exception at the time, we have figured out a way to avoid the API change. The API change is still a good idea because mapping things page-by-page is a valid use case that is currently not covered by the API, so the next plan was to do the API change in a later release as a separate issue, not related to original Xuan's intent.

However, we've been discussing implementation details with Xuan, and we arrived at a realization that what Xuan wants to implement not only does not *require* page size, it *cannot* be implemented with page size API because there's no way to know page size for that memory at the time of the API call. So, turns out we need both paged and page-less versions :)

Quoted 7 lines

This means that there are no deprecation notices now, because we will be adding a new API after all, but we will *not* deprecate or remove the old API - that one will still stay valid.

Apologies for constantly shifting ground, but in the end i think we arrived at the best possible solution for this problem!

So there won't be symbol versioning but only new API, which means no deprecation notice is required, please update this patch's status accordingly.

Thanks for keep working on the issue to find a better solution.

#14 lore raw
Quoted 6 lines

So there won't be symbol versioning but only new API, which means no deprecation notice is required, please update this patch's status accordingly.

Thanks for keep working on the issue to find a better solution.

+1, good work