introduce fail-safe PMD

This PMD intercepts and manages Ethernet device removal events issued by slave PMDs and re-initializes them transparently when brought back so that existing applications do not need to be modified to benefit from true hot-plugging support.

The stacked PMD approach shares many similarities with the bonding PMD but with a different purpose. While bonding provides the ability to group several links into a single logical device for enhanced throughput and supports fail-over at link level, this one manages the sudden disappearance of the underlying device; it guarantees applications face a valid device in working order at all times.

Each fail-safe instance is configured to run atop one or several devices, with one defined as the preferred device. Hot-plug events are handled on all of them, and Tx is always directed to the preferred device if present or to the next available failover device (Rx is always performed on all devices for simplicity).

Moreover, the configured slaves (preferred or failover) do not need to be present at initialization time and may appear later.

Slaves configuration is continuously synchronized with that of the virtual device, which exposes their common set of capabilities to the application. Failure to apply the current configuration state to a slave for any reason simply reschedules its initialization.

This patchset depends on:

[dpdk-dev] [PATCH v4 0/8] fix hotplug API
http://dpdk.org/ml/archives/dev/2017-July/071340.html

v1 --> v2:

- Wrote documentation
- Fixed commit logs, signed-off-by
- Added LSC event support
- A few minor fixes

v2 --> v3:

- Numerous bug fixes.
- Complete sub-EAL rework to follow new bus API.
- burst protection on sub removal.
- more flexible sub definition.
- flow isolated mode support.

v3 --> v4:

- Split back commits
net/failsafe: add fast burst functions net/failsafe: support device removal
That were squashed by error during a rebase
- Fix segfault on port plugin
- Fix isolate mode support for MLX4 ports plugin

v4 --> v5:

- Follow new plug / unplug API.

v5 --> v6:

- Follow new hotplug API.
- Improve usability of hotplug API.
- Fix rte_dev hotplug API implementation.
- Introduce rte_eal_devargs_rmv API as EXPERIMENTAL.
- Use it to clean up resources on hotplug_remove.
- Fix hotplug implementation and support un pci bus. The scan was not idempotent, nor clean. Neither were the device fields.
- Implement plug operation for vdev bus.
This is needed for hotplug support and to make the EAL independent from vdev-specific API.
- Remove useless parameters from plug / unplug API.

This patchset is fairly big and complex. The hotplug API has been rushed and has never been tested outside of the special case of vdev bus.

These evolutions are proposed alongside this PMD as only this PMD allows to test this API at the moment, and without those evolutions this PMD cannot be used.

v6 --> v7

- Split the series in three, with
vlan filter configuration store standing alone Hotplug API fixes separated
Fail-safe PMD on its own.

These changes should ease integration.

v7 --> v8

- Fix usage examples in documentation.
- Use rte_eth_dev_allocated when relevant.

v8 --> v9

- Use calloc instead of rte_realloc for command line parsing.
- Include errno only when necessary.

v9 --> v10

- Fix rte_flow description double-free issue. Thanks Raslan for finding the issue :)
- Use RTE_ASSERT instead of assert.
Integrate the debug log of the fail-safe to RTE_LOG_DEBUG macro as well, allowing dynamic log levels.
- Remove devices post-state-sync during slave upkeep, to avoid a useless early probe attempt.

v10 --> v11

- Added Glossary in documentation.
- Fixed issues oulined in [1].
- Removed Rx offload advertizement in feature matrix.

[1]: http://dpdk.org/ml/archives/dev/2017-July/071406.html

Gaetan Rivet (11):
ethdev: add deferred intermediate device state
ethdev: count devices consistently
net/failsafe: add fail-safe PMD
net/failsafe: add plug-in support
net/failsafe: add flexible device definition
net/failsafe: support flow API
net/failsafe: support Rx offload capabilities
net/failsafe: add fast burst functions
net/failsafe: support device removal
net/failsafe: support link status change event
net/failsafe: support flow API isolation mode

 MAINTAINERS                                       |   5 +
 config/common_base                                |   5 +
 doc/guides/nics/fail_safe.rst                     | 221 ++++++
 doc/guides/nics/features/failsafe.ini             |  26 +
 doc/guides/nics/index.rst                         |   1 +
 drivers/net/Makefile                              |   2 +
 drivers/net/failsafe/Makefile                     |  60 ++
 drivers/net/failsafe/failsafe.c                   | 299 ++++++++
 drivers/net/failsafe/failsafe_args.c              | 468 ++++++++++++
 drivers/net/failsafe/failsafe_eal.c               | 118 +++
 drivers/net/failsafe/failsafe_ether.c             | 437 +++++++++++
 drivers/net/failsafe/failsafe_flow.c              | 244 ++++++
 drivers/net/failsafe/failsafe_ops.c               | 867 ++++++++++++++++++++++
 drivers/net/failsafe/failsafe_private.h           | 359 +++++++++
 drivers/net/failsafe/failsafe_rxtx.c              | 203 +++++
 drivers/net/failsafe/rte_pmd_failsafe_version.map |   4 +
 lib/librte_ether/rte_ethdev.c                     |  19 +-
 lib/librte_ether/rte_ethdev.h                     |  14 +-
 mk/rte.app.mk                                     |   1 +
 19 files changed, 3338 insertions(+), 15 deletions(-)
 create mode 100644 doc/guides/nics/fail_safe.rst
 create mode 100644 doc/guides/nics/features/failsafe.ini
 create mode 100644 drivers/net/failsafe/Makefile
 create mode 100644 drivers/net/failsafe/failsafe.c
 create mode 100644 drivers/net/failsafe/failsafe_args.c
 create mode 100644 drivers/net/failsafe/failsafe_eal.c
 create mode 100644 drivers/net/failsafe/failsafe_ether.c
 create mode 100644 drivers/net/failsafe/failsafe_flow.c
 create mode 100644 drivers/net/failsafe/failsafe_ops.c
 create mode 100644 drivers/net/failsafe/failsafe_private.h
 create mode 100644 drivers/net/failsafe/failsafe_rxtx.c
 create mode 100644 drivers/net/failsafe/rte_pmd_failsafe_version.map

2.1.4