Skip to contentSkip to conversation

doc: clean ABI/API policy guide

RFC Superseded Ferruh Yigit 5 participants 29 messages Dec 19, 2018 – Mar 27, 2019

lore
Share & tools
lore
#0 lore raw

RFC 1/2 doc: clean ABI/API policy guide

The original document written from the point of ABI versioning but later additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Aim to clarify confusion between deprecation versioned ABI and overall ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 132 +++++++++++++------------
 1 file changed, 71 insertions(+), 61 deletions(-)
doc/guides/contributing/versioning.rst +71 -61
index 01b36247e..19af56cd2 100644
@@ -1,33 +1,31 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright 2018 The DPDK contributors
 
-Managing ABI updates
-====================
+DPDK ABI/API policy
+===================
 
 Description
 -----------
 
 This document details some methods for handling ABI management in the DPDK.
-Note this document is not exhaustive, in that C library versioning is flexible
-allowing multiple methods to achieve various goals, but it will provide the user
-with some introductory methods
 
 General Guidelines
 ------------------
 
 #. Whenever possible, ABI should be preserved
-#. Libraries or APIs marked in ``experimental`` state may change without constraint.
+#. ABI/API may be changed with a deprecation process
+#. The modification of symbols can generally be managed with versioning
+#. Libraries or APIs marked in ``experimental`` state may change without constraint
 #. New APIs will be marked as ``experimental`` for at least one release to allow
    any issues found by users of the new API to be fixed quickly
 #. The addition of symbols is generally not problematic
-#. The modification of symbols can generally be managed with versioning
 #. The removal of symbols generally is an ABI break and requires bumping of the
    LIBABIVER macro
 #. Updates to the minimum hardware requirements, which drop support for hardware which
    was previously supported, should be treated as an ABI change.
 
 What is an ABI
---------------
+~~~~~~~~~~~~~~
 
 An ABI (Application Binary Interface) is the set of runtime interfaces exposed
 by a library. It is similar to an API (Application Programming Interface) but
@@ -39,9 +37,13 @@ Therefore, in the case of dynamic linking, it is critical that an ABI is
 preserved, or (when modified), done in such a way that the application is unable
 to behave improperly or in an unexpected fashion.
 
-The DPDK ABI policy
+
+ABI/API Deprecation
 -------------------
 
+The DPDK ABI policy
+~~~~~~~~~~~~~~~~~~~
+
 ABI versions are set at the time of major release labeling, and the ABI may
 change multiple times, without warning, between the last release label and the
 HEAD label of the git tree.
@@ -99,8 +101,36 @@ readability purposes should be avoided.
    follow the relevant deprecation policy procedures as above: 3 acks and
    announcement at least one release in advance.
 
+Examples of Deprecation Notices
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following are some examples of ABI deprecation notices which would be
+added to the Release Notes:
+
+* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
+  to be replaced with the inline function ``rte_foo()``.
+
+* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
+  in version 2.0. Backwards compatibility will be maintained for this function
+  until the release of version 2.1
+
+* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
+  performance reasons. Existing binary applications will have backwards
+  compatibility in release 2.0, while newly built binaries will need to
+  reference the new structure variant ``struct rte_foo2``. Compatibility will
+  be removed in release 2.2, and all applications will require updating and
+  rebuilding to the new structure at that time, which will be renamed to the
+  original ``struct rte_foo``.
+
+* Significant ABI changes are planned for the ``librte_dostuff`` library. The
+  upcoming release 2.0 will not contain these changes, but release 2.1 will,
+  and no backwards compatibility is planned due to the extensive nature of
+  these changes. Binaries using this library built prior to version 2.1 will
+  require updating and recompilation.
+
+
 Experimental APIs
-~~~~~~~~~~~~~~~~~
+-----------------
 
 APIs marked as ``experimental`` are not considered part of the ABI and may
 change without warning at any time.  Since changes to APIs are most likely
@@ -130,35 +160,38 @@ is not required. Though, an API should remain in experimental state for at least
 one release. Thereafter, normal process of posting patch for review to mailing
 list can be followed.
 
-Examples of Deprecation Notices
--------------------------------
 
-The following are some examples of ABI deprecation notices which would be
-added to the Release Notes:
+Library versioning
+------------------
 
-* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
-  to be replaced with the inline function ``rte_foo()``.
+Downstreams might want to provide different DPDK releases at the same time to
+support multiple consumers of DPDK linked against older and newer sonames.
 
-* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
-  in version 2.0. Backwards compatibility will be maintained for this function
-  until the release of version 2.1
+Also due to the interdependencies that DPDK libraries can have applications
+might end up with an executable space in which multiple versions of a library
+are mapped by ld.so.
 
-* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
-  performance reasons. Existing binary applications will have backwards
-  compatibility in release 2.0, while newly built binaries will need to
-  reference the new structure variant ``struct rte_foo2``. Compatibility will
-  be removed in release 2.2, and all applications will require updating and
-  rebuilding to the new structure at that time, which will be renamed to the
-  original ``struct rte_foo``.
+Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
+depending on LibA.
 
-* Significant ABI changes are planned for the ``librte_dostuff`` library. The
-  upcoming release 2.0 will not contain these changes, but release 2.1 will,
-  and no backwards compatibility is planned due to the extensive nature of
-  these changes. Binaries using this library built prior to version 2.1 will
-  require updating and recompilation.
+.. note::
+
+    Application
+    \-> LibA.old
+    \-> LibB.new -> LibA.new
+
+That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
+If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
+library - versions defined in the libraries ``LIBABIVER``.
+An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
+``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
+
+
+ABI versioning
+--------------
 
 Versioning Macros
------------------
+~~~~~~~~~~~~~~~~~
 
 When a symbol is exported from a library to provide an API, it also provides a
 calling convention (ABI) that is embodied in its name, return type and
@@ -186,36 +219,11 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
-Setting a Major ABI version
----------------------------
-
-Downstreams might want to provide different DPDK releases at the same time to
-support multiple consumers of DPDK linked against older and newer sonames.
-
-Also due to the interdependencies that DPDK libraries can have applications
-might end up with an executable space in which multiple versions of a library
-are mapped by ld.so.
-
-Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
-depending on LibA.
-
-.. note::
-
-    Application
-    \-> LibA.old
-    \-> LibB.new -> LibA.new
-
-That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
-If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
-library - versions defined in the libraries ``LIBABIVER``.
-An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
-``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
-
 Examples of ABI Macro use
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Updating a public API
-~~~~~~~~~~~~~~~~~~~~~
+_____________________
 
 Assume we have a function as follows
 
@@ -425,7 +433,7 @@ and a new DPDK_2.1 version, used by future built applications.
 
 
 Deprecating part of a public API
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+________________________________
 
 Lets assume that you've done the above update, and after a few releases have
 passed you decide you would like to retire the old version of the function.
@@ -483,7 +491,7 @@ possibly incompatible library version:
    +LIBABIVER := 2
 
 Deprecating an entire ABI version
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+_________________________________
 
 While removing a symbol from and ABI may be useful, it is often more practical
 to remove an entire version node at once.  If a version node completely
@@ -532,6 +540,7 @@ Lastly, any VERSION_SYMBOL macros that point to the old version node should be
 removed, taking care to keep, where need old code in place to support newer
 versions of the symbol.
 
+
 Running the ABI Validator
 -------------------------
 
@@ -571,3 +580,4 @@ compile both tags) it will create compatibility reports in the
 follows::
 
   grep -lr Incompatible compat_reports/
+
Signature · 1 line
2.17.2
#1 lore raw

RFC 2/2 doc: add deprecation marker usage

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker until next LTS.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +9 -0
index 19af56cd2..e0edd2e20 100644
@@ -128,6 +128,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when new API becomes
+active old one marked with ``__rte_deprecated`` until next LTS. In next LTS API
+removed completely.
+
+Reminder that new API should follow deprecation process to become active.
+
 
 Experimental APIs
 -----------------
Signature · 1 line
2.17.2
Acked-by on v1 2/2
#2 lore raw

Re: [RFC 2/2] doc: add deprecation marker usage

Quoted 32 lines · Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker until next LTS.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>
---
 doc/guides/contributing/versioning.rst | 9 +++++++++

 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +0 -0
index 19af56cd2..e0edd2e20 100644
@@ -128,6 +128,15 @@ added to the Release Notes:

   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.

doc/guides/contributing/versioning.rst +4 -0
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when

new API becomes
+active old one marked with ``__rte_deprecated`` until next LTS. In next LTS API

doc/guides/contributing/versioning.rst

Quoted 2 lines
doc/guides/contributing/versioning.rst +2 -0
+removed completely.
+

Perhaps a bit of rephrasing, ie:

"If a new API proposed functionally replaces an existing one, when the new API becomes active then the old one is marked with ``__rte_deprecated`` until the next LTS. In the next LTS, the API is removed completely."

Other than that:

Signature · 2 lines
Kind regards,
Luca Boccassi
#3 lore raw

Re: [RFC 2/2] doc: add deprecation marker usage

Quoted 38 lines · Luca Boccassi
Quoted 32 lines · Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker until next LTS.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>
---
 doc/guides/contributing/versioning.rst | 9 +++++++++

 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +0 -0
index 19af56cd2..e0edd2e20 100644
@@ -128,6 +128,15 @@ added to the Release Notes:

   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.

doc/guides/contributing/versioning.rst +4 -0
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when

new API becomes
+active old one marked with ``__rte_deprecated`` until next LTS. In next LTS API

doc/guides/contributing/versioning.rst

Quoted 2 lines
doc/guides/contributing/versioning.rst +2 -0
+removed completely.
+

Perhaps a bit of rephrasing, ie:

Quoted 4 lines

"If a new API proposed functionally replaces an existing one, when the new API becomes active then the old one is marked with ``__rte_deprecated`` until the next LTS. In the next LTS, the API is removed completely."

Thanks for suggestion, I will send a new version with this update.

Quoted 5 lines

Other than that:

Acked-by on v1 1/2
#4 lore raw
In reply to Ferruh Yigit
Quoted 14 lines · Ferruh Yigit

The original document written from the point of ABI versioning but later
additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Aim to clarify confusion between deprecation versioned ABI and overall
ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

Quoted 6 lines

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

Signature · 2 lines
Kind regards,
Luca Boccassi
#5 lore raw
In reply to Ferruh Yigit
Quoted 10 lines · Ferruh Yigit

+    Application
+    \-> LibA.old
+    \-> LibB.new -> LibA.new

(quoted diff) +2 -0
+
+That is a conflict which can be avoided by setting

``CONFIG_RTE_MAJOR_ABI``.
+If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per

(quoted diff) +2 -0
+library - versions defined in the libraries ``LIBABIVER``.
+An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make
Quoted 2 lines

all libraries
+``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.

This is not related to your reorganisation, but to solve this problem properly should we start considering ABI bumps as "sticky", so that in the example above _both_ libA and libB would get an ABI bump?

Signature · 2 lines
Kind regards,
Luca Boccassi
#6 lore raw

PATCH v2 1/2 doc: clean ABI/API policy guide

In reply to Ferruh Yigit

The original document written from the point of ABI versioning but later additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Aim to clarify confusion between deprecation versioned ABI and overall ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 132 +++++++++++++------------
 1 file changed, 71 insertions(+), 61 deletions(-)
doc/guides/contributing/versioning.rst +71 -61
index 01b36247e..19af56cd2 100644
@@ -1,33 +1,31 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright 2018 The DPDK contributors
 
-Managing ABI updates
-====================
+DPDK ABI/API policy
+===================
 
 Description
 -----------
 
 This document details some methods for handling ABI management in the DPDK.
-Note this document is not exhaustive, in that C library versioning is flexible
-allowing multiple methods to achieve various goals, but it will provide the user
-with some introductory methods
 
 General Guidelines
 ------------------
 
 #. Whenever possible, ABI should be preserved
-#. Libraries or APIs marked in ``experimental`` state may change without constraint.
+#. ABI/API may be changed with a deprecation process
+#. The modification of symbols can generally be managed with versioning
+#. Libraries or APIs marked in ``experimental`` state may change without constraint
 #. New APIs will be marked as ``experimental`` for at least one release to allow
    any issues found by users of the new API to be fixed quickly
 #. The addition of symbols is generally not problematic
-#. The modification of symbols can generally be managed with versioning
 #. The removal of symbols generally is an ABI break and requires bumping of the
    LIBABIVER macro
 #. Updates to the minimum hardware requirements, which drop support for hardware which
    was previously supported, should be treated as an ABI change.
 
 What is an ABI
---------------
+~~~~~~~~~~~~~~
 
 An ABI (Application Binary Interface) is the set of runtime interfaces exposed
 by a library. It is similar to an API (Application Programming Interface) but
@@ -39,9 +37,13 @@ Therefore, in the case of dynamic linking, it is critical that an ABI is
 preserved, or (when modified), done in such a way that the application is unable
 to behave improperly or in an unexpected fashion.
 
-The DPDK ABI policy
+
+ABI/API Deprecation
 -------------------
 
+The DPDK ABI policy
+~~~~~~~~~~~~~~~~~~~
+
 ABI versions are set at the time of major release labeling, and the ABI may
 change multiple times, without warning, between the last release label and the
 HEAD label of the git tree.
@@ -99,8 +101,36 @@ readability purposes should be avoided.
    follow the relevant deprecation policy procedures as above: 3 acks and
    announcement at least one release in advance.
 
+Examples of Deprecation Notices
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following are some examples of ABI deprecation notices which would be
+added to the Release Notes:
+
+* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
+  to be replaced with the inline function ``rte_foo()``.
+
+* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
+  in version 2.0. Backwards compatibility will be maintained for this function
+  until the release of version 2.1
+
+* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
+  performance reasons. Existing binary applications will have backwards
+  compatibility in release 2.0, while newly built binaries will need to
+  reference the new structure variant ``struct rte_foo2``. Compatibility will
+  be removed in release 2.2, and all applications will require updating and
+  rebuilding to the new structure at that time, which will be renamed to the
+  original ``struct rte_foo``.
+
+* Significant ABI changes are planned for the ``librte_dostuff`` library. The
+  upcoming release 2.0 will not contain these changes, but release 2.1 will,
+  and no backwards compatibility is planned due to the extensive nature of
+  these changes. Binaries using this library built prior to version 2.1 will
+  require updating and recompilation.
+
+
 Experimental APIs
-~~~~~~~~~~~~~~~~~
+-----------------
 
 APIs marked as ``experimental`` are not considered part of the ABI and may
 change without warning at any time.  Since changes to APIs are most likely
@@ -130,35 +160,38 @@ is not required. Though, an API should remain in experimental state for at least
 one release. Thereafter, normal process of posting patch for review to mailing
 list can be followed.
 
-Examples of Deprecation Notices
--------------------------------
 
-The following are some examples of ABI deprecation notices which would be
-added to the Release Notes:
+Library versioning
+------------------
 
-* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
-  to be replaced with the inline function ``rte_foo()``.
+Downstreams might want to provide different DPDK releases at the same time to
+support multiple consumers of DPDK linked against older and newer sonames.
 
-* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
-  in version 2.0. Backwards compatibility will be maintained for this function
-  until the release of version 2.1
+Also due to the interdependencies that DPDK libraries can have applications
+might end up with an executable space in which multiple versions of a library
+are mapped by ld.so.
 
-* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
-  performance reasons. Existing binary applications will have backwards
-  compatibility in release 2.0, while newly built binaries will need to
-  reference the new structure variant ``struct rte_foo2``. Compatibility will
-  be removed in release 2.2, and all applications will require updating and
-  rebuilding to the new structure at that time, which will be renamed to the
-  original ``struct rte_foo``.
+Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
+depending on LibA.
 
-* Significant ABI changes are planned for the ``librte_dostuff`` library. The
-  upcoming release 2.0 will not contain these changes, but release 2.1 will,
-  and no backwards compatibility is planned due to the extensive nature of
-  these changes. Binaries using this library built prior to version 2.1 will
-  require updating and recompilation.
+.. note::
+
+    Application
+    \-> LibA.old
+    \-> LibB.new -> LibA.new
+
+That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
+If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
+library - versions defined in the libraries ``LIBABIVER``.
+An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
+``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
+
+
+ABI versioning
+--------------
 
 Versioning Macros
------------------
+~~~~~~~~~~~~~~~~~
 
 When a symbol is exported from a library to provide an API, it also provides a
 calling convention (ABI) that is embodied in its name, return type and
@@ -186,36 +219,11 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
-Setting a Major ABI version
----------------------------
-
-Downstreams might want to provide different DPDK releases at the same time to
-support multiple consumers of DPDK linked against older and newer sonames.
-
-Also due to the interdependencies that DPDK libraries can have applications
-might end up with an executable space in which multiple versions of a library
-are mapped by ld.so.
-
-Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
-depending on LibA.
-
-.. note::
-
-    Application
-    \-> LibA.old
-    \-> LibB.new -> LibA.new
-
-That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
-If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
-library - versions defined in the libraries ``LIBABIVER``.
-An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
-``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
-
 Examples of ABI Macro use
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Updating a public API
-~~~~~~~~~~~~~~~~~~~~~
+_____________________
 
 Assume we have a function as follows
 
@@ -425,7 +433,7 @@ and a new DPDK_2.1 version, used by future built applications.
 
 
 Deprecating part of a public API
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+________________________________
 
 Lets assume that you've done the above update, and after a few releases have
 passed you decide you would like to retire the old version of the function.
@@ -483,7 +491,7 @@ possibly incompatible library version:
    +LIBABIVER := 2
 
 Deprecating an entire ABI version
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+_________________________________
 
 While removing a symbol from and ABI may be useful, it is often more practical
 to remove an entire version node at once.  If a version node completely
@@ -532,6 +540,7 @@ Lastly, any VERSION_SYMBOL macros that point to the old version node should be
 removed, taking care to keep, where need old code in place to support newer
 versions of the symbol.
 
+
 Running the ABI Validator
 -------------------------
 
@@ -571,3 +580,4 @@ compile both tags) it will create compatibility reports in the
 follows::
 
   grep -lr Incompatible compat_reports/
+
Signature · 1 line
2.17.2
#7 lore raw

PATCH v2 2/2 doc: add deprecation marker usage

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker until next LTS.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

---
 doc/guides/contributing/versioning.rst | 10 ++++++++++
 1 file changed, 10 insertions(+)
doc/guides/contributing/versioning.rst +10 -0
index 19af56cd2..360238985 100644
@@ -128,6 +128,16 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the
+new API becomes active then the old one is marked with
+``__rte_deprecated`` until the next LTS. In the next LTS, the API is
+removed completely.
+
+Reminder that new API should follow deprecation process to become active.
+
 
 Experimental APIs
 -----------------
Signature · 1 line
2.17.2
#8 lore raw

PATCH v3 1/3 doc: clean ABI/API policy guide

In reply to Ferruh Yigit

The original document written from the point of ABI versioning but later additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Aim to clarify confusion between deprecation versioned ABI and overall ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 132 +++++++++++++------------
 1 file changed, 71 insertions(+), 61 deletions(-)
doc/guides/contributing/versioning.rst +71 -61
index 01b36247e..19af56cd2 100644
@@ -1,33 +1,31 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright 2018 The DPDK contributors
 
-Managing ABI updates
-====================
+DPDK ABI/API policy
+===================
 
 Description
 -----------
 
 This document details some methods for handling ABI management in the DPDK.
-Note this document is not exhaustive, in that C library versioning is flexible
-allowing multiple methods to achieve various goals, but it will provide the user
-with some introductory methods
 
 General Guidelines
 ------------------
 
 #. Whenever possible, ABI should be preserved
-#. Libraries or APIs marked in ``experimental`` state may change without constraint.
+#. ABI/API may be changed with a deprecation process
+#. The modification of symbols can generally be managed with versioning
+#. Libraries or APIs marked in ``experimental`` state may change without constraint
 #. New APIs will be marked as ``experimental`` for at least one release to allow
    any issues found by users of the new API to be fixed quickly
 #. The addition of symbols is generally not problematic
-#. The modification of symbols can generally be managed with versioning
 #. The removal of symbols generally is an ABI break and requires bumping of the
    LIBABIVER macro
 #. Updates to the minimum hardware requirements, which drop support for hardware which
    was previously supported, should be treated as an ABI change.
 
 What is an ABI
---------------
+~~~~~~~~~~~~~~
 
 An ABI (Application Binary Interface) is the set of runtime interfaces exposed
 by a library. It is similar to an API (Application Programming Interface) but
@@ -39,9 +37,13 @@ Therefore, in the case of dynamic linking, it is critical that an ABI is
 preserved, or (when modified), done in such a way that the application is unable
 to behave improperly or in an unexpected fashion.
 
-The DPDK ABI policy
+
+ABI/API Deprecation
 -------------------
 
+The DPDK ABI policy
+~~~~~~~~~~~~~~~~~~~
+
 ABI versions are set at the time of major release labeling, and the ABI may
 change multiple times, without warning, between the last release label and the
 HEAD label of the git tree.
@@ -99,8 +101,36 @@ readability purposes should be avoided.
    follow the relevant deprecation policy procedures as above: 3 acks and
    announcement at least one release in advance.
 
+Examples of Deprecation Notices
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following are some examples of ABI deprecation notices which would be
+added to the Release Notes:
+
+* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
+  to be replaced with the inline function ``rte_foo()``.
+
+* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
+  in version 2.0. Backwards compatibility will be maintained for this function
+  until the release of version 2.1
+
+* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
+  performance reasons. Existing binary applications will have backwards
+  compatibility in release 2.0, while newly built binaries will need to
+  reference the new structure variant ``struct rte_foo2``. Compatibility will
+  be removed in release 2.2, and all applications will require updating and
+  rebuilding to the new structure at that time, which will be renamed to the
+  original ``struct rte_foo``.
+
+* Significant ABI changes are planned for the ``librte_dostuff`` library. The
+  upcoming release 2.0 will not contain these changes, but release 2.1 will,
+  and no backwards compatibility is planned due to the extensive nature of
+  these changes. Binaries using this library built prior to version 2.1 will
+  require updating and recompilation.
+
+
 Experimental APIs
-~~~~~~~~~~~~~~~~~
+-----------------
 
 APIs marked as ``experimental`` are not considered part of the ABI and may
 change without warning at any time.  Since changes to APIs are most likely
@@ -130,35 +160,38 @@ is not required. Though, an API should remain in experimental state for at least
 one release. Thereafter, normal process of posting patch for review to mailing
 list can be followed.
 
-Examples of Deprecation Notices
--------------------------------
 
-The following are some examples of ABI deprecation notices which would be
-added to the Release Notes:
+Library versioning
+------------------
 
-* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
-  to be replaced with the inline function ``rte_foo()``.
+Downstreams might want to provide different DPDK releases at the same time to
+support multiple consumers of DPDK linked against older and newer sonames.
 
-* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
-  in version 2.0. Backwards compatibility will be maintained for this function
-  until the release of version 2.1
+Also due to the interdependencies that DPDK libraries can have applications
+might end up with an executable space in which multiple versions of a library
+are mapped by ld.so.
 
-* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
-  performance reasons. Existing binary applications will have backwards
-  compatibility in release 2.0, while newly built binaries will need to
-  reference the new structure variant ``struct rte_foo2``. Compatibility will
-  be removed in release 2.2, and all applications will require updating and
-  rebuilding to the new structure at that time, which will be renamed to the
-  original ``struct rte_foo``.
+Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
+depending on LibA.
 
-* Significant ABI changes are planned for the ``librte_dostuff`` library. The
-  upcoming release 2.0 will not contain these changes, but release 2.1 will,
-  and no backwards compatibility is planned due to the extensive nature of
-  these changes. Binaries using this library built prior to version 2.1 will
-  require updating and recompilation.
+.. note::
+
+    Application
+    \-> LibA.old
+    \-> LibB.new -> LibA.new
+
+That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
+If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
+library - versions defined in the libraries ``LIBABIVER``.
+An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
+``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
+
+
+ABI versioning
+--------------
 
 Versioning Macros
------------------
+~~~~~~~~~~~~~~~~~
 
 When a symbol is exported from a library to provide an API, it also provides a
 calling convention (ABI) that is embodied in its name, return type and
@@ -186,36 +219,11 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
-Setting a Major ABI version
----------------------------
-
-Downstreams might want to provide different DPDK releases at the same time to
-support multiple consumers of DPDK linked against older and newer sonames.
-
-Also due to the interdependencies that DPDK libraries can have applications
-might end up with an executable space in which multiple versions of a library
-are mapped by ld.so.
-
-Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
-depending on LibA.
-
-.. note::
-
-    Application
-    \-> LibA.old
-    \-> LibB.new -> LibA.new
-
-That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
-If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
-library - versions defined in the libraries ``LIBABIVER``.
-An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
-``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
-
 Examples of ABI Macro use
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Updating a public API
-~~~~~~~~~~~~~~~~~~~~~
+_____________________
 
 Assume we have a function as follows
 
@@ -425,7 +433,7 @@ and a new DPDK_2.1 version, used by future built applications.
 
 
 Deprecating part of a public API
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+________________________________
 
 Lets assume that you've done the above update, and after a few releases have
 passed you decide you would like to retire the old version of the function.
@@ -483,7 +491,7 @@ possibly incompatible library version:
    +LIBABIVER := 2
 
 Deprecating an entire ABI version
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+_________________________________
 
 While removing a symbol from and ABI may be useful, it is often more practical
 to remove an entire version node at once.  If a version node completely
@@ -532,6 +540,7 @@ Lastly, any VERSION_SYMBOL macros that point to the old version node should be
 removed, taking care to keep, where need old code in place to support newer
 versions of the symbol.
 
+
 Running the ABI Validator
 -------------------------
 
@@ -571,3 +580,4 @@ compile both tags) it will create compatibility reports in the
 follows::
 
   grep -lr Incompatible compat_reports/
+
Signature · 1 line
2.17.2
#9 lore raw

PATCH v3 2/3 doc: make RTE_NEXT_ABI optional

Initial process requires oncoming changes described in deprecation notice should be implemented in a RTE_NEXT_ABI gated way.

This has been discussed in technical board, and since this can cause a multiple #ifdef blocks in multiple locations of the code, can be confusing specially for the modifications that requires data structure changes. Anyway this was not happening in practice.

Making RTE_NEXT_ABI usage more optional based on techboard decision:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

The intention with using RTE_NEXT_ABI was to provide more information to the user about planned changes, and force developer to think more in coding level. Since RTE_NEXT_ABI become optional, now the preferred way to do this is, if possible, sending changes, described in deprecation notice, as a separate patch and reference it in deprecation notice.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
doc/guides/contributing/versioning.rst +6 -9
index 19af56cd2..bfc27fbe0 100644
@@ -73,19 +73,16 @@ being provided. The requirements for doing so are:
      interest" be sought for each deprecation, for example: from NIC vendors,
      CPU vendors, end-users, etc.
 
-#. The changes (including an alternative map file) must be gated with
-   the ``RTE_NEXT_ABI`` option, and provided with a deprecation notice at the
-   same time.
-   It will become the default ABI in the next release.
+#. The changes (including an alternative map file) can be included with
+   deprecation notice, in wrapped way by the ``RTE_NEXT_ABI`` option,
+   to provide more details about oncoming changes.
+   ``RTE_NEXT_ABI`` wrapper will be removed when it become the default ABI.
+   More preferred way to provide this information is sending the feature
+   as a separate patch and reference it in deprecation notice.
 
 #. A full deprecation cycle, as explained above, must be made to offer
    downstream consumers sufficient warning of the change.
 
-#. At the beginning of the next release cycle, every ``RTE_NEXT_ABI``
-   conditions will be removed, the ``LIBABIVER`` variable in the makefile(s)
-   where the ABI is changed will be incremented, and the map files will
-   be updated.
-
 Note that the above process for ABI deprecation should not be undertaken
 lightly. ABI stability is extremely important for downstream consumers of the
 DPDK, especially when distributed in shared object form. Every effort should
Signature · 1 line
2.17.2
#10 lore raw

PATCH v3 3/3 doc: add deprecation marker usage

In reply to Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +9 -0
index bfc27fbe0..977d06c60 100644
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the
+new API becomes active then the old one is marked with ``__rte_deprecated``.
+Deprecated APIs removed completely just after the next LTS.
+
+Reminder that new API should follow deprecation process to become active.
+
 
 Experimental APIs
 -----------------
Signature · 1 line
2.17.2
#11 lore raw

Re: [PATCH v3 3/3] doc: add deprecation marker usage

Quoted 30 lines · Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +0 -0
index bfc27fbe0..977d06c60 100644

doc/guides/contributing/versioning.rst added to the Release Notes: · line 132

Quoted 8 lines
doc/guides/contributing/versioning.rst +5 -0
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the
+new API becomes active then the old one is marked with ``__rte_deprecated``.

I don't think it's clear what 'active' means here. Can it be re-phrased as something like "..when the new API has it's experimental tag removed, then the old one..".

It might also be worth mentioning the reasoning behind this, perhaps something like: This is so an application continues to be provided with at least one stable (non-deprecated/non-experimental) API for this functionality.

Quoted 8 lines
doc/guides/contributing/versioning.rst +4 -0
+Deprecated APIs removed completely just after the next LTS.
+
+Reminder that new API should follow deprecation process to become active.
+
 
 Experimental APIs
 -----------------
#12 lore raw

Re: [PATCH v3 3/3] doc: add deprecation marker usage

Quoted 34 lines · Kevin Traynor
Quoted 33 lines · Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +0 -0
index bfc27fbe0..977d06c60 100644
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
Quoted 9 lines

doc/guides/contributing/versioning.rst added to the Release Notes: · line 132

Quoted 5 lines
doc/guides/contributing/versioning.rst +5 -0
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the
+new API becomes active then the old one is marked with ``__rte_deprecated``.

I don't think it's clear what 'active' means here. Can it be re-phrased as something like "..when the new API has it's experimental tag removed, then the old one..".

This was what in my mind by 'active' but didn't want to create confusion with details, and really it doesn't matter the "experimental" detail, by any means if the new API is not 'active' we shouldn't mark the old one as 'deprecated'.

But agree can be defined better than 'active'. Do you have any suggestion here, 'GA', 'public', 'official', 'supported'?

Quoted 15 lines

It might also be worth mentioning the reasoning behind this, perhaps something like: This is so an application continues to be provided with at least one stable (non-deprecated/non-experimental) API for this functionality.

Quoted 8 lines
doc/guides/contributing/versioning.rst +4 -0
+Deprecated APIs removed completely just after the next LTS.
+
+Reminder that new API should follow deprecation process to become active.
+
 
 Experimental APIs
 -----------------
#13 lore raw

Re: [PATCH v3 3/3] doc: add deprecation marker usage

Quoted 45 lines · Ferruh Yigit
Quoted 34 lines · Kevin Traynor
Quoted 33 lines · Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +0 -0
index bfc27fbe0..977d06c60 100644
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
Quoted 9 lines

doc/guides/contributing/versioning.rst added to the Release Notes: · line 132

Quoted 5 lines
doc/guides/contributing/versioning.rst +5 -0
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the
+new API becomes active then the old one is marked with ``__rte_deprecated``.

I don't think it's clear what 'active' means here. Can it be re-phrased as something like "..when the new API has it's experimental tag removed, then the old one..".

Quoted 7 lines

This was what in my mind by 'active' but didn't want to create confusion with details, and really it doesn't matter the "experimental" detail, by any means if the new API is not 'active' we shouldn't mark the old one as 'deprecated'.

But agree can be defined better than 'active'. Do you have any suggestion here, 'GA', 'public', 'official', 'supported'?

How about 'non-experimental' ? I think it would make it clear in meaning for general reading and also avoid a mis-interpretation of what the actual detail is.

Quoted 16 lines
Quoted 15 lines

It might also be worth mentioning the reasoning behind this, perhaps something like: This is so an application continues to be provided with at least one stable (non-deprecated/non-experimental) API for this functionality.

Quoted 8 lines
doc/guides/contributing/versioning.rst +4 -0
+Deprecated APIs removed completely just after the next LTS.
+
+Reminder that new API should follow deprecation process to become active.
+
 
 Experimental APIs
 -----------------
#14 lore raw

Re: [PATCH v3 3/3] doc: add deprecation marker usage

Quoted 50 lines · Kevin Traynor
Quoted 49 lines · Ferruh Yigit
Quoted 34 lines · Kevin Traynor
Quoted 33 lines · Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +0 -0
index bfc27fbe0..977d06c60 100644
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
Quoted 9 lines

doc/guides/contributing/versioning.rst added to the Release Notes: · line 132

Quoted 5 lines
doc/guides/contributing/versioning.rst +5 -0
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the
+new API becomes active then the old one is marked with ``__rte_deprecated``.

I don't think it's clear what 'active' means here. Can it be re-phrased as something like "..when the new API has it's experimental tag removed, then the old one..".

This was what in my mind by 'active' but didn't want to create confusion with details, and really it doesn't matter the "experimental" detail, by any means if the new API is not 'active' we shouldn't mark the old one as 'deprecated'.

Quoted 7 lines
Quoted 3 lines

But agree can be defined better than 'active'. Do you have any suggestion here, 'GA', 'public', 'official', 'supported'?

How about 'non-experimental' ? I think it would make it clear in meaning for general reading and also avoid a mis-interpretation of what the actual detail is.

OK, I will send a new versions with s/active/non-experimental/

Quoted 18 lines
Quoted 17 lines
Quoted 16 lines

It might also be worth mentioning the reasoning behind this, perhaps something like: This is so an application continues to be provided with at least one stable (non-deprecated/non-experimental) API for this functionality.

Quoted 8 lines
doc/guides/contributing/versioning.rst +4 -0
+Deprecated APIs removed completely just after the next LTS.
+
+Reminder that new API should follow deprecation process to become active.
+
 
 Experimental APIs
 -----------------
Acked-by on v3 1/3
#15 lore raw
In reply to Ferruh Yigit
Quoted 254 lines · Ferruh Yigit

The original document written from the point of ABI versioning but later additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Aim to clarify confusion between deprecation versioned ABI and overall ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 132 +++++++++++++------------
 1 file changed, 71 insertions(+), 61 deletions(-)
doc/guides/contributing/versioning.rst +71 -61
index 01b36247e..19af56cd2 100644
@@ -1,33 +1,31 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright 2018 The DPDK contributors
 
-Managing ABI updates
-====================
+DPDK ABI/API policy
+===================
 
 Description
 -----------
 
 This document details some methods for handling ABI management in the DPDK.
-Note this document is not exhaustive, in that C library versioning is flexible
-allowing multiple methods to achieve various goals, but it will provide the user
-with some introductory methods
 
 General Guidelines
 ------------------
 
 #. Whenever possible, ABI should be preserved
-#. Libraries or APIs marked in ``experimental`` state may change without constraint.
+#. ABI/API may be changed with a deprecation process
+#. The modification of symbols can generally be managed with versioning
+#. Libraries or APIs marked in ``experimental`` state may change without constraint
 #. New APIs will be marked as ``experimental`` for at least one release to allow
    any issues found by users of the new API to be fixed quickly
 #. The addition of symbols is generally not problematic
-#. The modification of symbols can generally be managed with versioning
 #. The removal of symbols generally is an ABI break and requires bumping of the
    LIBABIVER macro
 #. Updates to the minimum hardware requirements, which drop support for hardware which
    was previously supported, should be treated as an ABI change.
 
 What is an ABI
---------------
+~~~~~~~~~~~~~~
 
 An ABI (Application Binary Interface) is the set of runtime interfaces exposed
 by a library. It is similar to an API (Application Programming Interface) but
@@ -39,9 +37,13 @@ Therefore, in the case of dynamic linking, it is critical that an ABI is
 preserved, or (when modified), done in such a way that the application is unable
 to behave improperly or in an unexpected fashion.
 
-The DPDK ABI policy
+
+ABI/API Deprecation
 -------------------
 
+The DPDK ABI policy
+~~~~~~~~~~~~~~~~~~~
+
 ABI versions are set at the time of major release labeling, and the ABI may
 change multiple times, without warning, between the last release label and the
 HEAD label of the git tree.
@@ -99,8 +101,36 @@ readability purposes should be avoided.
    follow the relevant deprecation policy procedures as above: 3 acks and
    announcement at least one release in advance.
 
+Examples of Deprecation Notices
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following are some examples of ABI deprecation notices which would be
+added to the Release Notes:
+
+* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
+  to be replaced with the inline function ``rte_foo()``.
+
+* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
+  in version 2.0. Backwards compatibility will be maintained for this function
+  until the release of version 2.1
+
+* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
+  performance reasons. Existing binary applications will have backwards
+  compatibility in release 2.0, while newly built binaries will need to
+  reference the new structure variant ``struct rte_foo2``. Compatibility will
+  be removed in release 2.2, and all applications will require updating and
+  rebuilding to the new structure at that time, which will be renamed to the
+  original ``struct rte_foo``.
+
+* Significant ABI changes are planned for the ``librte_dostuff`` library. The
+  upcoming release 2.0 will not contain these changes, but release 2.1 will,
+  and no backwards compatibility is planned due to the extensive nature of
+  these changes. Binaries using this library built prior to version 2.1 will
+  require updating and recompilation.
+
+
 Experimental APIs
-~~~~~~~~~~~~~~~~~
+-----------------
 
 APIs marked as ``experimental`` are not considered part of the ABI and may
 change without warning at any time.  Since changes to APIs are most likely
@@ -130,35 +160,38 @@ is not required. Though, an API should remain in experimental state for at least
 one release. Thereafter, normal process of posting patch for review to mailing
 list can be followed.
 
-Examples of Deprecation Notices
--------------------------------
 
-The following are some examples of ABI deprecation notices which would be
-added to the Release Notes:
+Library versioning
+------------------
 
-* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
-  to be replaced with the inline function ``rte_foo()``.
+Downstreams might want to provide different DPDK releases at the same time to
+support multiple consumers of DPDK linked against older and newer sonames.
 
-* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
-  in version 2.0. Backwards compatibility will be maintained for this function
-  until the release of version 2.1
+Also due to the interdependencies that DPDK libraries can have applications
+might end up with an executable space in which multiple versions of a library
+are mapped by ld.so.
 
-* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
-  performance reasons. Existing binary applications will have backwards
-  compatibility in release 2.0, while newly built binaries will need to
-  reference the new structure variant ``struct rte_foo2``. Compatibility will
-  be removed in release 2.2, and all applications will require updating and
-  rebuilding to the new structure at that time, which will be renamed to the
-  original ``struct rte_foo``.
+Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
+depending on LibA.
 
-* Significant ABI changes are planned for the ``librte_dostuff`` library. The
-  upcoming release 2.0 will not contain these changes, but release 2.1 will,
-  and no backwards compatibility is planned due to the extensive nature of
-  these changes. Binaries using this library built prior to version 2.1 will
-  require updating and recompilation.
+.. note::
+
+    Application
+    \-> LibA.old
+    \-> LibB.new -> LibA.new
+
+That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
+If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
+library - versions defined in the libraries ``LIBABIVER``.
+An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
+``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
+
+
+ABI versioning
+--------------
 
 Versioning Macros
------------------
+~~~~~~~~~~~~~~~~~
 
 When a symbol is exported from a library to provide an API, it also provides a
 calling convention (ABI) that is embodied in its name, return type and
@@ -186,36 +219,11 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
-Setting a Major ABI version
----------------------------
-
-Downstreams might want to provide different DPDK releases at the same time to
-support multiple consumers of DPDK linked against older and newer sonames.
-
-Also due to the interdependencies that DPDK libraries can have applications
-might end up with an executable space in which multiple versions of a library
-are mapped by ld.so.
-
-Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
-depending on LibA.
-
-.. note::
-
-    Application
-    \-> LibA.old
-    \-> LibB.new -> LibA.new
-
-That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
-If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
-library - versions defined in the libraries ``LIBABIVER``.
-An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
-``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
-
 Examples of ABI Macro use
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Updating a public API
-~~~~~~~~~~~~~~~~~~~~~
+_____________________
 
 Assume we have a function as follows
 
@@ -425,7 +433,7 @@ and a new DPDK_2.1 version, used by future built applications.
 
 
 Deprecating part of a public API
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+________________________________
 
 Lets assume that you've done the above update, and after a few releases have
 passed you decide you would like to retire the old version of the function.
@@ -483,7 +491,7 @@ possibly incompatible library version:
    +LIBABIVER := 2
 
 Deprecating an entire ABI version
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+_________________________________
 
 While removing a symbol from and ABI may be useful, it is often more practical
 to remove an entire version node at once.  If a version node completely
@@ -532,6 +540,7 @@ Lastly, any VERSION_SYMBOL macros that point to the old version node should be
 removed, taking care to keep, where need old code in place to support newer
 versions of the symbol.
 
+
 Running the ABI Validator
 -------------------------
 
@@ -571,3 +580,4 @@ compile both tags) it will create compatibility reports in the
 follows::
 
   grep -lr Incompatible compat_reports/
+
Signature · 1 line
2.17.2

Series

#16 lore raw

PATCH v4 1/3 doc: clean ABI/API policy guide

In reply to Ferruh Yigit

The original document written from the point of ABI versioning but later additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Aim to clarify confusion between deprecation versioned ABI and overall ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 132 +++++++++++++------------
 1 file changed, 71 insertions(+), 61 deletions(-)
doc/guides/contributing/versioning.rst +71 -61
index 01b36247e..19af56cd2 100644
@@ -1,33 +1,31 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright 2018 The DPDK contributors
 
-Managing ABI updates
-====================
+DPDK ABI/API policy
+===================
 
 Description
 -----------
 
 This document details some methods for handling ABI management in the DPDK.
-Note this document is not exhaustive, in that C library versioning is flexible
-allowing multiple methods to achieve various goals, but it will provide the user
-with some introductory methods
 
 General Guidelines
 ------------------
 
 #. Whenever possible, ABI should be preserved
-#. Libraries or APIs marked in ``experimental`` state may change without constraint.
+#. ABI/API may be changed with a deprecation process
+#. The modification of symbols can generally be managed with versioning
+#. Libraries or APIs marked in ``experimental`` state may change without constraint
 #. New APIs will be marked as ``experimental`` for at least one release to allow
    any issues found by users of the new API to be fixed quickly
 #. The addition of symbols is generally not problematic
-#. The modification of symbols can generally be managed with versioning
 #. The removal of symbols generally is an ABI break and requires bumping of the
    LIBABIVER macro
 #. Updates to the minimum hardware requirements, which drop support for hardware which
    was previously supported, should be treated as an ABI change.
 
 What is an ABI
---------------
+~~~~~~~~~~~~~~
 
 An ABI (Application Binary Interface) is the set of runtime interfaces exposed
 by a library. It is similar to an API (Application Programming Interface) but
@@ -39,9 +37,13 @@ Therefore, in the case of dynamic linking, it is critical that an ABI is
 preserved, or (when modified), done in such a way that the application is unable
 to behave improperly or in an unexpected fashion.
 
-The DPDK ABI policy
+
+ABI/API Deprecation
 -------------------
 
+The DPDK ABI policy
+~~~~~~~~~~~~~~~~~~~
+
 ABI versions are set at the time of major release labeling, and the ABI may
 change multiple times, without warning, between the last release label and the
 HEAD label of the git tree.
@@ -99,8 +101,36 @@ readability purposes should be avoided.
    follow the relevant deprecation policy procedures as above: 3 acks and
    announcement at least one release in advance.
 
+Examples of Deprecation Notices
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following are some examples of ABI deprecation notices which would be
+added to the Release Notes:
+
+* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
+  to be replaced with the inline function ``rte_foo()``.
+
+* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
+  in version 2.0. Backwards compatibility will be maintained for this function
+  until the release of version 2.1
+
+* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
+  performance reasons. Existing binary applications will have backwards
+  compatibility in release 2.0, while newly built binaries will need to
+  reference the new structure variant ``struct rte_foo2``. Compatibility will
+  be removed in release 2.2, and all applications will require updating and
+  rebuilding to the new structure at that time, which will be renamed to the
+  original ``struct rte_foo``.
+
+* Significant ABI changes are planned for the ``librte_dostuff`` library. The
+  upcoming release 2.0 will not contain these changes, but release 2.1 will,
+  and no backwards compatibility is planned due to the extensive nature of
+  these changes. Binaries using this library built prior to version 2.1 will
+  require updating and recompilation.
+
+
 Experimental APIs
-~~~~~~~~~~~~~~~~~
+-----------------
 
 APIs marked as ``experimental`` are not considered part of the ABI and may
 change without warning at any time.  Since changes to APIs are most likely
@@ -130,35 +160,38 @@ is not required. Though, an API should remain in experimental state for at least
 one release. Thereafter, normal process of posting patch for review to mailing
 list can be followed.
 
-Examples of Deprecation Notices
--------------------------------
 
-The following are some examples of ABI deprecation notices which would be
-added to the Release Notes:
+Library versioning
+------------------
 
-* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
-  to be replaced with the inline function ``rte_foo()``.
+Downstreams might want to provide different DPDK releases at the same time to
+support multiple consumers of DPDK linked against older and newer sonames.
 
-* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
-  in version 2.0. Backwards compatibility will be maintained for this function
-  until the release of version 2.1
+Also due to the interdependencies that DPDK libraries can have applications
+might end up with an executable space in which multiple versions of a library
+are mapped by ld.so.
 
-* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
-  performance reasons. Existing binary applications will have backwards
-  compatibility in release 2.0, while newly built binaries will need to
-  reference the new structure variant ``struct rte_foo2``. Compatibility will
-  be removed in release 2.2, and all applications will require updating and
-  rebuilding to the new structure at that time, which will be renamed to the
-  original ``struct rte_foo``.
+Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
+depending on LibA.
 
-* Significant ABI changes are planned for the ``librte_dostuff`` library. The
-  upcoming release 2.0 will not contain these changes, but release 2.1 will,
-  and no backwards compatibility is planned due to the extensive nature of
-  these changes. Binaries using this library built prior to version 2.1 will
-  require updating and recompilation.
+.. note::
+
+    Application
+    \-> LibA.old
+    \-> LibB.new -> LibA.new
+
+That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
+If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
+library - versions defined in the libraries ``LIBABIVER``.
+An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
+``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
+
+
+ABI versioning
+--------------
 
 Versioning Macros
------------------
+~~~~~~~~~~~~~~~~~
 
 When a symbol is exported from a library to provide an API, it also provides a
 calling convention (ABI) that is embodied in its name, return type and
@@ -186,36 +219,11 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
-Setting a Major ABI version
----------------------------
-
-Downstreams might want to provide different DPDK releases at the same time to
-support multiple consumers of DPDK linked against older and newer sonames.
-
-Also due to the interdependencies that DPDK libraries can have applications
-might end up with an executable space in which multiple versions of a library
-are mapped by ld.so.
-
-Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
-depending on LibA.
-
-.. note::
-
-    Application
-    \-> LibA.old
-    \-> LibB.new -> LibA.new
-
-That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
-If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
-library - versions defined in the libraries ``LIBABIVER``.
-An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
-``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
-
 Examples of ABI Macro use
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Updating a public API
-~~~~~~~~~~~~~~~~~~~~~
+_____________________
 
 Assume we have a function as follows
 
@@ -425,7 +433,7 @@ and a new DPDK_2.1 version, used by future built applications.
 
 
 Deprecating part of a public API
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+________________________________
 
 Lets assume that you've done the above update, and after a few releases have
 passed you decide you would like to retire the old version of the function.
@@ -483,7 +491,7 @@ possibly incompatible library version:
    +LIBABIVER := 2
 
 Deprecating an entire ABI version
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+_________________________________
 
 While removing a symbol from and ABI may be useful, it is often more practical
 to remove an entire version node at once.  If a version node completely
@@ -532,6 +540,7 @@ Lastly, any VERSION_SYMBOL macros that point to the old version node should be
 removed, taking care to keep, where need old code in place to support newer
 versions of the symbol.
 
+
 Running the ABI Validator
 -------------------------
 
@@ -571,3 +580,4 @@ compile both tags) it will create compatibility reports in the
 follows::
 
   grep -lr Incompatible compat_reports/
+
Signature · 1 line
2.17.2
#17 lore raw

PATCH v4 2/3 doc: make RTE_NEXT_ABI optional

Initial process requires oncoming changes described in deprecation notice should be implemented in a RTE_NEXT_ABI gated way.

This has been discussed in technical board, and since this can cause a multiple #ifdef blocks in multiple locations of the code, can be confusing specially for the modifications that requires data structure changes. Anyway this was not happening in practice.

Making RTE_NEXT_ABI usage more optional based on techboard decision:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

The intention with using RTE_NEXT_ABI was to provide more information to the user about planned changes, and force developer to think more in coding level. Since RTE_NEXT_ABI become optional, now the preferred way to do this is, if possible, sending changes, described in deprecation notice, as a separate patch and reference it in deprecation notice.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
doc/guides/contributing/versioning.rst +6 -9
index 19af56cd2..bfc27fbe0 100644
@@ -73,19 +73,16 @@ being provided. The requirements for doing so are:
      interest" be sought for each deprecation, for example: from NIC vendors,
      CPU vendors, end-users, etc.
 
-#. The changes (including an alternative map file) must be gated with
-   the ``RTE_NEXT_ABI`` option, and provided with a deprecation notice at the
-   same time.
-   It will become the default ABI in the next release.
+#. The changes (including an alternative map file) can be included with
+   deprecation notice, in wrapped way by the ``RTE_NEXT_ABI`` option,
+   to provide more details about oncoming changes.
+   ``RTE_NEXT_ABI`` wrapper will be removed when it become the default ABI.
+   More preferred way to provide this information is sending the feature
+   as a separate patch and reference it in deprecation notice.
 
 #. A full deprecation cycle, as explained above, must be made to offer
    downstream consumers sufficient warning of the change.
 
-#. At the beginning of the next release cycle, every ``RTE_NEXT_ABI``
-   conditions will be removed, the ``LIBABIVER`` variable in the makefile(s)
-   where the ABI is changed will be incremented, and the map files will
-   be updated.
-
 Note that the above process for ABI deprecation should not be undertaken
 lightly. ABI stability is extremely important for downstream consumers of the
 DPDK, especially when distributed in shared object form. Every effort should
Signature · 1 line
2.17.2
Acked-by on v4 2/3
#18 lore raw

Re: [PATCH v4 2/3] doc: make RTE_NEXT_ABI optional

24/01/2019 19:10, Ferruh Yigit:

Quoted 11 lines

Initial process requires oncoming changes described in deprecation notice should be implemented in a RTE_NEXT_ABI gated way.

This has been discussed in technical board, and since this can cause a multiple #ifdef blocks in multiple locations of the code, can be confusing specially for the modifications that requires data structure changes. Anyway this was not happening in practice.

Making RTE_NEXT_ABI usage more optional based on techboard decision:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

Quoted 8 lines

The intention with using RTE_NEXT_ABI was to provide more information to the user about planned changes, and force developer to think more in coding level. Since RTE_NEXT_ABI become optional, now the preferred way to do this is, if possible, sending changes, described in deprecation notice, as a separate patch and reference it in deprecation notice.

#19 lore raw

PATCH v4 3/3 doc: add deprecation marker usage

In reply to Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

v4:
change working around 'active':
s/active/non-experimental/
s/become active/be able to replace old API

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +9 -0
index bfc27fbe0..491fae438 100644
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the new API
+becomes non-experimental then the old one is marked with ``__rte_deprecated``.
+Deprecated APIs removed completely just after the next LTS.
+
+Reminder that new API should follow deprecation process to be able to replace old API.
+
 
 Experimental APIs
 -----------------
Signature · 1 line
2.17.2
#20 lore raw

Re: [PATCH v4 3/3] doc: add deprecation marker usage

24/01/2019 19:10, Ferruh Yigit:

Quoted 8 lines

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

[...]

doc/guides/contributing/versioning.rst added to the Release Notes: · line 133

Quoted 6 lines
doc/guides/contributing/versioning.rst +6 -0
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the new API
+becomes non-experimental then the old one is marked with ``__rte_deprecated``.
+Deprecated APIs removed completely just after the next LTS.

APIs removed -> APIs are removed

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+Reminder that new API should follow deprecation process to be able to replace old API.

This last sentence is not clear to me.
I think you mean the old API should follow deprecation process to be removed (replaced).

#21 lore raw

Re: [PATCH v4 3/3] doc: add deprecation marker usage

Quoted 5 lines · Thomas Monjalon

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+Reminder that new API should follow deprecation process to be able to replace old API.

This last sentence is not clear to me.
I think you mean the old API should follow deprecation process to be removed (replaced).

Yes, we are trying to say same thing.

Deprecation process required for replacing API.

#22 lore raw

Re: [PATCH v4 3/3] doc: add deprecation marker usage

01/02/2019 18:06, Ferruh Yigit:

Quoted 10 lines · Thomas Monjalon
Quoted 5 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+Reminder that new API should follow deprecation process to be able to replace old API.

This last sentence is not clear to me.
I think you mean the old API should follow deprecation process to be removed (replaced).

Yes, we are trying to say same thing.

Deprecation process required for replacing API.

I think it is important to emphasize that the process applies to removal.

Acked-by on v4 1/3
#23 lore raw
In reply to Ferruh Yigit
Quoted 8 lines · Ferruh Yigit

The original document written from the point of ABI versioning but later additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Quoted 6 lines

Aim to clarify confusion between deprecation versioned ABI and overall ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

For series

#24 lore raw

PATCH v5 1/3 doc: clean ABI/API policy guide

In reply to Ferruh Yigit

The original document written from the point of ABI versioning but later additions make document confusing, convert document into a ABI/API policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning

Aim to clarify confusion between deprecation versioned ABI and overall ABI/API deprecation, also ABI versioning and Library versioning by organizing the sections.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 132 +++++++++++++------------
 1 file changed, 71 insertions(+), 61 deletions(-)
doc/guides/contributing/versioning.rst +71 -61
index 18b031998..0bd7e6c44 100644
@@ -1,33 +1,31 @@
 ..  SPDX-License-Identifier: BSD-3-Clause
     Copyright 2018 The DPDK contributors
 
-Managing ABI updates
-====================
+DPDK ABI/API policy
+===================
 
 Description
 -----------
 
 This document details some methods for handling ABI management in the DPDK.
-Note this document is not exhaustive, in that C library versioning is flexible
-allowing multiple methods to achieve various goals, but it will provide the user
-with some introductory methods
 
 General Guidelines
 ------------------
 
 #. Whenever possible, ABI should be preserved
-#. Libraries or APIs marked in ``experimental`` state may change without constraint.
+#. ABI/API may be changed with a deprecation process
+#. The modification of symbols can generally be managed with versioning
+#. Libraries or APIs marked in ``experimental`` state may change without constraint
 #. New APIs will be marked as ``experimental`` for at least one release to allow
    any issues found by users of the new API to be fixed quickly
 #. The addition of symbols is generally not problematic
-#. The modification of symbols can generally be managed with versioning
 #. The removal of symbols generally is an ABI break and requires bumping of the
    LIBABIVER macro
 #. Updates to the minimum hardware requirements, which drop support for hardware which
    was previously supported, should be treated as an ABI change.
 
 What is an ABI
---------------
+~~~~~~~~~~~~~~
 
 An ABI (Application Binary Interface) is the set of runtime interfaces exposed
 by a library. It is similar to an API (Application Programming Interface) but
@@ -39,9 +37,13 @@ Therefore, in the case of dynamic linking, it is critical that an ABI is
 preserved, or (when modified), done in such a way that the application is unable
 to behave improperly or in an unexpected fashion.
 
-The DPDK ABI policy
+
+ABI/API Deprecation
 -------------------
 
+The DPDK ABI policy
+~~~~~~~~~~~~~~~~~~~
+
 ABI versions are set at the time of major release labeling, and the ABI may
 change multiple times, without warning, between the last release label and the
 HEAD label of the git tree.
@@ -99,8 +101,36 @@ readability purposes should be avoided.
    follow the relevant deprecation policy procedures as above: 3 acks and
    announcement at least one release in advance.
 
+Examples of Deprecation Notices
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following are some examples of ABI deprecation notices which would be
+added to the Release Notes:
+
+* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
+  to be replaced with the inline function ``rte_foo()``.
+
+* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
+  in version 2.0. Backwards compatibility will be maintained for this function
+  until the release of version 2.1
+
+* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
+  performance reasons. Existing binary applications will have backwards
+  compatibility in release 2.0, while newly built binaries will need to
+  reference the new structure variant ``struct rte_foo2``. Compatibility will
+  be removed in release 2.2, and all applications will require updating and
+  rebuilding to the new structure at that time, which will be renamed to the
+  original ``struct rte_foo``.
+
+* Significant ABI changes are planned for the ``librte_dostuff`` library. The
+  upcoming release 2.0 will not contain these changes, but release 2.1 will,
+  and no backwards compatibility is planned due to the extensive nature of
+  these changes. Binaries using this library built prior to version 2.1 will
+  require updating and recompilation.
+
+
 Experimental APIs
-~~~~~~~~~~~~~~~~~
+-----------------
 
 APIs marked as ``experimental`` are not considered part of the ABI and may
 change without warning at any time.  Since changes to APIs are most likely
@@ -130,35 +160,38 @@ is not required. Though, an API should remain in experimental state for at least
 one release. Thereafter, normal process of posting patch for review to mailing
 list can be followed.
 
-Examples of Deprecation Notices
--------------------------------
 
-The following are some examples of ABI deprecation notices which would be
-added to the Release Notes:
+Library versioning
+------------------
 
-* The Macro ``#RTE_FOO`` is deprecated and will be removed with version 2.0,
-  to be replaced with the inline function ``rte_foo()``.
+Downstreams might want to provide different DPDK releases at the same time to
+support multiple consumers of DPDK linked against older and newer sonames.
 
-* The function ``rte_mbuf_grok()`` has been updated to include a new parameter
-  in version 2.0. Backwards compatibility will be maintained for this function
-  until the release of version 2.1
+Also due to the interdependencies that DPDK libraries can have applications
+might end up with an executable space in which multiple versions of a library
+are mapped by ld.so.
 
-* The members of ``struct rte_foo`` have been reorganized in release 2.0 for
-  performance reasons. Existing binary applications will have backwards
-  compatibility in release 2.0, while newly built binaries will need to
-  reference the new structure variant ``struct rte_foo2``. Compatibility will
-  be removed in release 2.2, and all applications will require updating and
-  rebuilding to the new structure at that time, which will be renamed to the
-  original ``struct rte_foo``.
+Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
+depending on LibA.
 
-* Significant ABI changes are planned for the ``librte_dostuff`` library. The
-  upcoming release 2.0 will not contain these changes, but release 2.1 will,
-  and no backwards compatibility is planned due to the extensive nature of
-  these changes. Binaries using this library built prior to version 2.1 will
-  require updating and recompilation.
+.. note::
+
+    Application
+    \-> LibA.old
+    \-> LibB.new -> LibA.new
+
+That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
+If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
+library - versions defined in the libraries ``LIBABIVER``.
+An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
+``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
+
+
+ABI versioning
+--------------
 
 Versioning Macros
------------------
+~~~~~~~~~~~~~~~~~
 
 When a symbol is exported from a library to provide an API, it also provides a
 calling convention (ABI) that is embodied in its name, return type and
@@ -186,36 +219,11 @@ The macros exported are:
   fully qualified function ``p``, so that if a symbol becomes versioned, it
   can still be mapped back to the public symbol name.
 
-Setting a Major ABI version
----------------------------
-
-Downstreams might want to provide different DPDK releases at the same time to
-support multiple consumers of DPDK linked against older and newer sonames.
-
-Also due to the interdependencies that DPDK libraries can have applications
-might end up with an executable space in which multiple versions of a library
-are mapped by ld.so.
-
-Think of LibA that got an ABI bump and LibB that did not get an ABI bump but is
-depending on LibA.
-
-.. note::
-
-    Application
-    \-> LibA.old
-    \-> LibB.new -> LibA.new
-
-That is a conflict which can be avoided by setting ``CONFIG_RTE_MAJOR_ABI``.
-If set, the value of ``CONFIG_RTE_MAJOR_ABI`` overwrites all - otherwise per
-library - versions defined in the libraries ``LIBABIVER``.
-An example might be ``CONFIG_RTE_MAJOR_ABI=16.11`` which will make all libraries
-``librte<?>.so.16.11`` instead of ``librte<?>.so.<LIBABIVER>``.
-
 Examples of ABI Macro use
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Updating a public API
-~~~~~~~~~~~~~~~~~~~~~
+_____________________
 
 Assume we have a function as follows
 
@@ -425,7 +433,7 @@ and a new DPDK_2.1 version, used by future built applications.
 
 
 Deprecating part of a public API
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+________________________________
 
 Lets assume that you've done the above update, and after a few releases have
 passed you decide you would like to retire the old version of the function.
@@ -483,7 +491,7 @@ possibly incompatible library version:
    +LIBABIVER := 2
 
 Deprecating an entire ABI version
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+_________________________________
 
 While removing a symbol from and ABI may be useful, it is often more practical
 to remove an entire version node at once.  If a version node completely
@@ -532,6 +540,7 @@ Lastly, any VERSION_SYMBOL macros that point to the old version node should be
 removed, taking care to keep, where need old code in place to support newer
 versions of the symbol.
 
+
 Running the ABI Validator
 -------------------------
 
@@ -571,3 +580,4 @@ compile both tags) it will create compatibility reports in the
 follows::
 
   grep -lr Incompatible compat_reports/
+
Signature · 1 line
2.20.1
#25 lore raw

PATCH v5 2/3 doc: make RTE_NEXT_ABI optional

Initial process requires oncoming changes described in deprecation notice should be implemented in a RTE_NEXT_ABI gated way.

This has been discussed in technical board, and since this can cause a multiple #ifdef blocks in multiple locations of the code, can be confusing specially for the modifications that requires data structure changes. Anyway this was not happening in practice.

Making RTE_NEXT_ABI usage more optional based on techboard decision:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

The intention with using RTE_NEXT_ABI was to provide more information to the user about planned changes, and force developer to think more in coding level. Since RTE_NEXT_ABI become optional, now the preferred way to do this is, if possible, sending changes, described in deprecation notice, as a separate patch and reference it in deprecation notice.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

---
 doc/guides/contributing/versioning.rst | 15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)
doc/guides/contributing/versioning.rst +6 -9
index 0bd7e6c44..2fcb8bafd 100644
@@ -73,19 +73,16 @@ being provided. The requirements for doing so are:
      interest" be sought for each deprecation, for example: from NIC vendors,
      CPU vendors, end-users, etc.
 
-#. The changes (including an alternative map file) must be gated with
-   the ``RTE_NEXT_ABI`` option, and provided with a deprecation notice at the
-   same time.
-   It will become the default ABI in the next release.
+#. The changes (including an alternative map file) can be included with
+   deprecation notice, in wrapped way by the ``RTE_NEXT_ABI`` option,
+   to provide more details about oncoming changes.
+   ``RTE_NEXT_ABI`` wrapper will be removed when it become the default ABI.
+   More preferred way to provide this information is sending the feature
+   as a separate patch and reference it in deprecation notice.
 
 #. A full deprecation cycle, as explained above, must be made to offer
    downstream consumers sufficient warning of the change.
 
-#. At the beginning of the next release cycle, every ``RTE_NEXT_ABI``
-   conditions will be removed, the ``LIBABIVER`` variable in the makefile(s)
-   where the ABI is changed will be incremented, and the map files will
-   be updated.
-
 Note that the above process for ABI deprecation should not be undertaken
 lightly. ABI stability is extremely important for downstream consumers of the
 DPDK, especially when distributed in shared object form. Every effort should
Signature · 1 line
2.20.1
#26 lore raw

PATCH v5 3/3 doc: add deprecation marker usage

In reply to Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

v4:
change working around 'active':
s/active/non-experimental/
s/become active/be able to replace old API

v5:
Grammar fix
Rephrase API removal reminder

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +9 -0
index 2fcb8bafd..30f8735ba 100644
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If a new API proposed functionally replaces an existing one, when the new API
+becomes non-experimental then the old one is marked with ``__rte_deprecated``.
+Deprecated APIs are removed completely just after the next LTS.
+
+Reminder that old API should follow deprecation process to be removed.
+
 
 Experimental APIs
 -----------------
Signature · 1 line
2.20.1
Acked-by on v5 3/3
#27 lore raw

Re: [PATCH v5 3/3] doc: add deprecation marker usage

Quoted 46 lines · Ferruh Yigit

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

---
Cc: Luca Boccassi <[email protected]>
Cc: Kevin Traynor <[email protected]>
Cc: Yongseok Koh <[email protected]>
Cc: Neil Horman <[email protected]>

v2:
* Rephrased as commented

v3:
* changed when to remove the deprecated API. It is now just after an LTS release, the motivation is to keep changes small in LTS. Based on techboard discussion:
http://mails.dpdk.org/archives/dev/2019-January/123519.html

v4:
change working around 'active':
s/active/non-experimental/
s/become active/be able to replace old API

v5:
Grammar fix
Rephrase API removal reminder

---
 doc/guides/contributing/versioning.rst | 9 +++++++++
 1 file changed, 9 insertions(+)
doc/guides/contributing/versioning.rst +2 -0
index 2fcb8bafd..30f8735ba 100644
@@ -125,6 +125,15 @@ added to the Release Notes:
   these changes. Binaries using this library built prior to version 2.1 will
   require updating and recompilation.
 
+New API replacing previous one
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

doc/guides/contributing/versioning.rst added to the Release Notes: · line 139

Quoted 10 lines
doc/guides/contributing/versioning.rst +7 -0
+
+If a new API proposed functionally replaces an existing one, when the new API
+becomes non-experimental then the old one is marked with ``__rte_deprecated``.
+Deprecated APIs are removed completely just after the next LTS.
+
+Reminder that old API should follow deprecation process to be removed.
+
 
 Experimental APIs
 -----------------

for series, again :-)

#28 lore raw

Re: [PATCH v5 3/3] doc: add deprecation marker usage

01/03/2019 18:40, Kevin Traynor:

Quoted 6 lines · Ferruh Yigit
Quoted 5 lines

Define '__rte_deprecated' usage process.

Suggests keeping old API with '__rte_deprecated' marker including next LTS, they will be removed just after the LTS release.

Quoted 7 lines
Quoted 3 lines

for series, again :-)

Applied, thanks